Skip to main content

Research outputs monitoring and evaluation (ROME)

REF 2014 defined research generally as a process of investigation leading to new insights, effectively shared. Within the School of Health Sciences it is assumed that such research will generally be evidenced by publications in peer-review journals. The general guidance below addresses the bulk of the research that the ROME panel expects to receive.

We do appreciate that working practices vary considerably across academic fields. If you have produced research outputs that you regard as equivalent to those described below but do not fulfil the specific criteria then please submit them to the panel with an appropriate covering note.

  
****
(4*)

Quality that is world-leading in terms of originality, significance and rigour.

***   (3*)Quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence.
**     (2*)Quality that is recognised internationally in terms of originality, significance and rigour.
*       (1*)Quality that is recognised nationally in terms of originality, significance and rigour.
UQuality that falls below the standard of nationally recognised work or does not meet the published definition of research for the purposes of this assessment.

It is our belief that any paper that has been accepted in a scientific journal after full-peer review has, by definition, fulfilled at least the status of national recognition and we therefore assume that any paper being considered by the panel will be graded at least 1*.

Papers are graded on the basis of three categories. These are individually rated using the same system outlined in the table above and an overall grade is then awarded based upon these.

  • Originality: The extent to which the output introduces a new way of thinking about a subject, or is distinctive or transformative compared with previous work in an academic field.
  • Significance: The extent to which the work has exerted, or is likely to exert, an influence on an academic field or practical applications.
  • Rigour: The extent to which the purpose of the work is clearly articulated, an appropriate methodology for the research area has been adopted, and compelling evidence presented to show that the purpose has been achieved.

What we do review

REF 2014 defined research generally as a process of investigation leading to new insights, effectively shared. Within the School of Health Sciences it is assumed that such research will generally be evidenced by publications in peer-review journals.

The focus of the ROME panel is on monitoring and evaluating all articles

  • published in a scientific journal 
  • after a full-peer review process
  • with at least one of the listed authors a current or previous member of the School of Health Sciences acknowledged on the paper as affiliated to the University of Salford.

All papers fulfilling these criteria should be submitted to the panel by the most senior author who is a member of the School of Health Sciences following the guidance on the How does the process work? tab below.
We do not make a distinction between short communications, technical notes etc. and “full” papers as far as inclusion or exclusion is concerned but, in our fields, it is the “full” papers that are likely to be graded more highly.

Reviews
  • Systematic reviews are considered research and should be reviewed.
  • Narrative reviews will generally be regarded as summarising knowledge rather than leading to new insights and thus as not meeting the REF definition of research. They should be lodged on USIR but will not be reviewed by the Panel. 

There may be a small number of narrative reviews in which the synthesis of evidence or opinion is sufficiently strong to be regarded as leading to new insights. An example might be a review synthesising the opinions of a number of senior figures in a field, perhaps nominated by a professional or academic body, and generating a strong consensus view or guidelines or proposal for future directions. Such papers may be reviewed.

What we don’t review


In most subject areas the requirement for full-peer review will exclude:

  • Abstracts or brief papers submitted to accompany a conference presentation even if these are published in a scientific journal.
  • Editorials, commentaries or guest editorials
  • Letters to the editor or responses to such letters
  • Tributes or obituaries
  • Book reviews

There is no need to submit such outputs to the panel (but they must still be lodged on USIR).

What we monitor
  • Three categories of article were not submitted to the last REF but may become more important for the next one.
  • Pedagogical research
  • Trial protocols
  • Case studies

At present we are monitoring such articles but not grading them. Please unload these to USIR using the normal process (assuming they have been accepted for publication in a peer-reviewed journal).

1. Once you have had a paper accepted for publication the most senior author places a copy in the University of Salford Institutional Repository (USIR) within three months from the date of acceptance. If it is published as Open Access then a .pdf copy as provided to you by the publishers should be submitted. If itis not then a .pdf copy of the final version as you submitted it to the publisher (complete with figures and tables and their legends) should be submitted (this is called the Author Accepted Manuscript – AAD – and you own the copyright to this).

2. When the journal’s typeset .pdf version has become available the you should e-mail a copy to HSRC-Research@salford.ac.uk. The body of the e-mail should include:

  1. a link to the USIR record (the process will not proceed until it has confirmed that the paper has been lodged on USIR)
  2. the author’s own opinion of the paper’s REF grading (for originality, significance, and rigour and overall) See ROME policy for details
  3. the abstract (copied and pasted).

If any of these are missing then you will be informed of this and the process will not proceed until all the information has been received.

3. Four weeks before each panel an e-mail will be sent out to reminding academic staff of this requirement to notify us of any outputs that have been published. Only papers received at least two weeks before the panel date will be reviewed.

4. After the panel meeting you will be sent an e-mail acknowledging that the panel has reviewed the paper and informing of you of the grading that the panel has assigned. (Note that these may be reviewed before the next REF in the light of any additional citation information that is available by then).

5. You will not get any written feedback on the panel’s discussion but if you would like more information on this you should consult your Research Programme Lead who is a panel member.

The panel is comprised primarily of the Professoriate and the Research Programme Leads. Other staff with specific areas of expertise may also be seconded to it. The current members are:

  • Dr Kris Hollands, Chair
  • Professor Penny Cook
  • Professor Alison Hammond
  • Professor Peter Hogg
  • Professor Christopher Nester
  • Professor Malcolm Granat
  • Professor Los Kenney
  • Professor Rich Jones
  • Dr Sarah Norgate
  • Dr Steve Preece
  • Dr Anita Williams
  • Dr Tracy Collins
  • Dr Steve Atkins

Members are encouraged to send along alternates to meetings as a way of educating a wider range of staff members (particularly early and mid-career researchers) in how to judge research quality. It can be useful of such alternates consult with the Panel member they are replacing before the meeting to be mentored in this.

We also encourage a small number of observers at each meeting as part of our educational remit. Observers should not come along to Panel meetings at which their own research is discussed.

The panel meets four times each year (approx quarterly).

The panel is made up of research theme leads and thereby allows an appropriate level of both depth of experience and breadth of disciplines to reflect the range of research that falls under health sciences.  Papers are assigned to a lead reviewer who is in the closest field of study to the paper topic but is not the author or co-author (i.e. lead reviewer is an expert in the field). Other reviewers may or may not be directly in the field of study and this replicates REF Unit of Assessment panel make-ups; providing an accurate idea of how the paper is likely to be received within the wider Health Sciences Unit of Assessment. Given the full breadth of research that the members of the School engage in, the system may not work well for all research outputs. Occasionally, therefore the panel may send an output to another school panel, or externally for review. Please contact the panel Chair Kristen Hollands if you feel that your research outputs fall into this category for any reason.

Each article is allocated to at least three panel members (reviewers) before the meeting. One of these is allocated as the primary reviewer. They read the paper and submit their grades (originality, significance, rigour and overall) before the meeting.

  • If there is broad agreement between the author and the readers overall gradings then the paper is not discussed further and that grade is accepted.
  • If the readers agree with each other but not with the author then the primary reader will introduce the paper briefly and explain why. Other readers are then free to add their comments if useful. There readers grade will be accepted. If the readers do not agree then the primary reader will introduce the paper briefly and explain his or her opinion. The other readers will then explain their own opinions. There will then be sufficient discussion for the panel to reach a consensus and that grade will be accepted.                                                                                                                                              

If all readers are in agreement (whether this is in accordance with the author or not) then there may be very little discussion. This should not be interpreted as a lack of rigour as the consensus is confirming the rigour of how the readers have performed their original reviews.

It is assumed that an author’s Research Programme Lead is listening to the discussion and is able to discuss the grading with the author at a later date if this is deemed useful.

  1. Alshawabka, AZR, Liu, A, Tyson, SF and Jones, RK 2014, 'The use of a lateral wedge insole to reduce knee loading when ascending and descending stairs in medial knee osteoarthritis patients' , Clinical Biomechanics, 5 (11). (In Press)
  2. Armand, S, Sangeux, M and Baker, R 2014, 'Optimal markers' placement on the thorax for clinical gait analysis' , Gait and Posture,39 (1) , pp. 147-53.
  3. Collins, T 2014, 'Managing widowhood in later life: The challenges encountered' , International Journal of Therapy and Rehabilitation, 21 (2) , pp. 69-76.
  4. Cook, PA 2014, 'Comparison of patients diagnosed with gonorrhoea through community screening with those self-presenting to the genito-urinary medicine clinic' , BMJ Open, 4 (3) , pp. 1-7.
  5. Dickinson, J, Ghali, K, Cherrett, T, Speed, C, Davies, N and Norgate, SH 2012, 'Tourism and the smartphone app: Capabilities, emerging practice, and scope in the travel domain' , Current Issues in Tourism , pp. 1-18. Item not available from this repository.
  6. Greuter, S and Roberts, DJ 2014, 'Controlling viewpoint from markerless head tracking in an immersive ball game using a commodity depth-based camera' , Journal of Simulation , pp. 1-10. (Submitted)
  7. Hastings-Ison, T, Blackburn, CG, Opie, N, Graham, KE, Rawicki, B, Wolfendale, R, Simpson, P and Baker, RJ 2014,'Reproducibility of an instrumented measure for passive ankle dorsiflexion in conscious and anaesthetized children with cerebral palsy', Developmental Medicine and Child Neurology, 56 (4) , pp. 376-385.
  8. Higgins, RN, Hogg, P and Robinson, L 2013, 'Towards a research informed teaching experience within a diagnostic radiography curriculum: The level 4 (year 1) student holistic experience' , Radiography, 19 (1) , pp. 62-67.
  9. Higgins, RN, Robinson, L and Hogg, P 2014, 'Integrating research-informed teaching within an undergraduate diagnostic radiography curriculum: Results from a level 4 (year 1) student cohort' , Radiography, 20 (2) , pp. 100-106.
  10. Hollands, K, Pelton, TA, Wimperis, A, Whitham, D, Jowett, S, Wing, AM and van Vliet, PM 2013, 'Visual cue training to improve walking and turning after stroke: a study protocol for a multi-centre, single blind randomised pilot trial' , Trials, 14 (276).
  11. Hollands, K, Agnihotri, D and Tyson, SF 2014, 'Effects of dual task on turning ability in stroke survivors and older adults' , Gait and Posture, July (17) .
  12. Jones, RK, Chapman, GJ, Forythe, L, Parkes, MJ and Felson, DF 2014, 'The relationship between reductions in knee loading and immediate pain response whilst wearing lateral wedged insoles in knee Osteoarthritis' , Journal of Orthopaedic Research. (In Press)
  13. Lanca, L, Franco, L, Ahmed, A and Hogg, P 2014, '10 kVp rule an anthropomorphic pelvis phantom imaging study using a CR system: impact on image quality and effective dose using AEC and manual mode' , Radiography, 20 (4) , pp. 333-338.
  14. Ma, W, Norton, SR and Hogg, P 2014, 'Effects of kilovoltage, milliampere seconds, and focal spot size on image quality' , Radiologic Technology, 85 (5) , pp. 1-7.
  15. McGinley, J, Wolfendale, R, Morris, M, Pandy, M and Baker, RJ 2014, 'Variability of walking in able-bodied adults across different time intervals' , Journal of Physical Medicine and Rehabilitation Sciences, 17, pp. 6-10.
  16. McLeod, F, Erdogan, G, Cherrett, T, Bektas, T, Davies, N, Shingleton, D, Speed, C, Dickinson, J and Norgate, SH 2014, 'Improving collection efficiency through remote monitoring of charity assets' , Waste Management, 34 (2) , pp. 273-280.
  17. Mriaty, H, England, A and Hogg, P 2014, 'Developing and validating a psychometric scale for image quality assessment' , Radiography . (In Press)
  18. Murphy, FJ, Nightingale, JM, Hogg, P, Robinson, L, Seddon, D and Mackay, S 2014, 'Compression force behaviours: An exploration of the beliefs and values influencing the application of breast compression during screening mammography' , Radiography . (In Press)
  19. Norgate, SH, Davies, N, Speed, C, Cherrett, T and Dickinson, J 2014, 'The missing dimension: The relevance of people's conception of time' , Behavioral and Brain Sciences, 37 (01), pp. 93-94.
  20. Phethean, J, Pataky, TC, Nester, CJ and Findlow, AH 2014, 'A cross-sectional study of age-related changes in plantar pressure distribution between 4 and 7 years: A comparison of regional and pixel-level analyses' , Gait & Posture, 39 (1) , pp. 154-160. Item not available from this repository.
  21. Robinson, L, Harris, A and Burton, R 2015, 'Saving face: Managing rapport in a problem based learning group' , Active Learning in Higher Education. (In Press)
  22. Robinson, L 2014, 'Age difference and face-saving in an inter-generational problem based learning group' , Journal of Further and Higher Education.
  23. Robinson, L, Hogg, P and Higgins, RN 2014, 'An observational study of cross-cultural communication in short-term, diverse professional learning groups' , Radiography. (Submitted)
  24. Sindall, PA, Lenton, JP, Whytock, K, Tolfrey, K, Oyster, ML, Cooper, RA and Goosey-Tolfrey, VL 2013, 'Criterion validity and accuracy of global positioning satellite and data logging devices for wheelchair tennis court movement' , Journal of Spinal Cord Medicine, 36 (4), pp. 383-393.
  25. Sindall, PA, Lenton, JP, Tolfrey, K, Cooper, RA, Oyster, M and Goosey-Tolfrey, VL 2013, 'Wheelchair tennis match-play demands: Effect of player rank and result' , International Journal of Sports Physiology and Performance, 8 (1) , pp. 28-37.
  26. Sindall, PA, Lenton, JP, Malone, L, Douglas, S, Cooper, RA, Hiremath, S, Tolfrey, K and Goosey-Tolfrey, V 2014, 'Effect of low-compression balls on wheelchair tennis match-play' , International Journal of Sports Medicine, 2014 (35), pp. 424-431.
  27. Sobuh, MMD, Kenney, LPJ, Galpin, AJ, Sibylle, TB, McLaughlin, J, Kulkarni, J and Kyberd, P 2014, 'Visuomotor behaviours when using a myoelectric prosthesis' , Journal of NeuroEngineering and Rehabilitation. (In Press)
  28. Specialli, DS, Correa, JCF, Luna, NM, Brant, R, Greve, JMD, de Godoy, W, Baker, Richard and Lucareli, PRG 2014, 'Validation of GDI, GPS and GVS for use in parkinson's disease through evaluation of effects of subthalamic deep brain stimulation and levodopa', Gait and Posture, 39 (4), pp. 1142-1145.
  29. Thompson, JD, Hogg, P, Manning, DJ, Szczepura, K and Chakraborty, D 2014, 'A free-response evaluation determining value in the computed tomography attenuation correction image for revealing pulmonary incidental findings: A phantom study' , Academic Radiology, 21 (4), pp. 538-45.
  30. Tootell, AK, Szczepura, K and Hogg, P 2014, 'An overview of measuring and modelling dose and risk from ionising radiation for medical exposures' , Radiography. (In Press)
  31. Williamson, T, Kenney, LPJ, Barker, AT, Cooper, G, Good, T, Healey, J, Heller, B, Matthews, M, Prenton, S, Ryan, JM and Smith, CL 2015, 'Enhancing public involvement in assistive technology design research' , Disability and Rehabilitation: Assistive Technology. (In Press)

Good writing cannot turn 1* research into 4* research but it is certainly essential to ensure that the true quality of research is appreciated by the reader. The quality of writing can almost certainly lead to the difference of a whole star in the overall writing of the paper. This link takes you to a document suggesting how you can write to improve the perceived quality of your research.

Writing for REF

This document outlines the School of Health Sciences policy relating to the ROME panel.