Skip to main content

Table 3 Tasks sorted by participants in the Q-sort survey. Data are (overall rank) and mean ± SD rank of participants

From: The most important tasks for peer reviewers evaluating a randomized controlled trial are not congruent with the tasks most often requested by journal editors

 

Total

Clinician

Methodologist

Both

Other

(N = 203)

(N = 93)

(N = 72)

(N = 17)

(N = 21)

To evaluate the risk of bias of the trial

(1) 2.1 ± 2.0

(5) 1.3 ± 2.1

(1) 2.8 ± 1.8

(1) 3.6 ± 0.8

(1) 2.5 ± 1.6

To determine if the manuscript conclusion is consistent with the results

(2) 1.9 ± 1.9

(1) 1.7 ± 2.0

(3) 2.1 ± 1.8

(2) 2.4 ± 2.1

(3) 1.9 ± 1.9

To evaluate the adequacy of statistical analyses

(3) 1.8 ± 1.7

(6) 1.2 ± 1.6

(2) 2.6 ± 1.7

(3) 2.2 ± 1.6

(5) 1.6 ± 1.5

To evaluate if the control group is appropriate

(4) 1.4 ± 1.8

(8) 1.1 ± 1.7

(4) 1.9 ± 1.8

(8) 1.2 ± 1.4

(6) 1.4 ± 2.1

To check if all outcomes are adequately reported

(5) 1.4 ± 2.0

(9) 1.1 ± 2.1

(5) 1.7 ± 1.9

(5) 1.5 ± 1.6

(4) 1.9 ± 1.9

To evaluate the relevance of the primary outcome(s)

(6) 1.3 ± 2.1

(4) 1.3 ± 2.1

(6) 1.5 ± 2.1

(14) 0.6 ± 2.6

(8) 1.1 ± 1.7

To search for any attempt to distort the presentation or interpretation of results

(7) 1.2 ± 2.0

(11) 0.9 ± 2.0

(7) 1.3 ± 1.8

(4) 1.9 ± 2.5

(2) 2.2 ± 2.1

To evaluate the reliability and validity of the outcome measures

(8) 1.1 ± 2.0

(3) 1.4 ± 2.2

(11) 0.9 ± 1.9

(12) 0.7 ± 2.1

(9) 1.0 ± 1.9

To evaluate the importance of the study

(9) 1.1 ± 2.4

(2) 1.7 ± 2.4

(14) 0.6 ± 2.5

(17) -0.1 ± 2.0

(13) 0.7 ± 2.0

To evaluate if the abstract conclusion is consistent with the results

(10) 1.0 ± 1.9

(10) 1.0 ± 2.1

(9) 1.2 ± 1.7

(9) 1.2 ± 1.9

(20) 0.0 ± 1.7

To evaluate if the discussion is consistent with the results

(11) 1.0 ± 1.8

(7) 1.2 ± 1.7

(13) 0.7 ± 1.7

(13) 0.7 ± 1.9

(7) 1.2 ± 2.1

To check if all adverse events are adequately reported

(12) 0.9 ± 1.9

(14) 0.7 ± 2.0

(8) 1.2 ± 1.7

(6) 1.4 ± 1.9

(10) 0.9 ± 2.1

To check if the intervention is described with enough details to allow replication

(13) 0.8 ± 1.8

(16) 0.6 ± 1.8

(10) 1.1 ± 1.7

(10) 1.0 ± 1.8

(18) 0.2 ± 1.9

To check that limitations are adequately reported

(14) 0.7 ± 1.7

(17) 0.4 ± 1.8

(12) 0.8 ± 1.6

(7) 1.3 ± 1.7

(11) 0.9 ± 1.6

To evaluate the adequacy of the selection of participants and clinical setting

(15) 0.5 ± 1.9

(13) 0.7 ± 1.9

(15) 0.5 ± 1.8

(15) 0.4 ± 1.7

(17) 0.2 ± 2.0

To search for any inconsistencies or errors in the manuscript

(16) 0.3 ± 2.2

(21) 0.0 ± 2.1

(17) 0.3 ± 2.2

(11) 1.0 ± 2.4

(14) 0.6 ± 2.0

To evaluate the novelty of the study

(17) 0.2 ± 2.4

(12) 0.8 ± 2.5

(20) –0.1 ± 2.0

(21) –0.3 ± 2.1

(24) –0.5 ± 2.7

To check the sample size calculation

(18) 0.2 ± 2.1

(18) 0.4 ± 1.9

(18) –0.1 ± 2.4

(24) –0.5 ± 2.0

(15) 0.5 ± 2.1

To check if the authors reported all important outcomes and adverse events in the abstract

(19) 0.1 ± 2.1

(22) 0.0 ± 2.3

(16) 0.4 ± 1.9

(20) –0.2 ± 1.8

(23) –0.3 ± 2.3

To discuss the results in relation to other studies

(20) 0.0 ± 1.9

(19) 0.3 ± 2.0

(19) –0.1 ± 1.6

(30) –1.0 ± 2.1

(19) 0.1 ± 2.0

To evaluate if the manuscript can be suspected of fraud

(21) –0.1 ± 2.6

(15) 0.7 ± 2.7

(30) –1.1 ± 2.4

(29) –0.9 ± 1.6

(12) 0.9 ± 2.3

To provide recommendations on publication

(22) –0.2 ± 2.5

(20) 0.3 ± 2.8

(23) –0.5 ± 2.2

(22) –0.4 ± 2.3

(26) –0.7 ± 2.3

To check if all figures and tables are consistent with the text

(23) –0.2 ± 1.7

(23) –0.1 ± 1.7

(21) –0.4 ± 1.6

(19) –0.2 ± 2.0

(21) –0.1 ± 1.6

To evaluate clarity of presentation

(24) –0.5 ± 1.9

(26) –0.5 ± 2.0

(22) –0.5 ± 1.9

(28) –0.8 ± 1.6

(27) –0.7 ± 1.7

To check if the study reported ethics review board approval

(25) –0.5 ± 2.2

(25) –0.3 ± 2.3

(27) –0.9 ± 2.1

(23) –0.4 ± 2.1

(22) –0.3 ± 2.3

To search for plagiarism or imitation in the paper

(26) –0.7 ± 2.3

(24) –0.1 ± 2.4

(33) –1.4 ± 2.2

(32) –1.4 ± 2.0

(16) 0.2 ± 1.9

To compare information recorded in the trial protocol when provided by the authors and reported in the manuscript

(27) –0.7 ± 2.3

(27) –0.8 ± 2.2

(24) –0.6 ± 2.5

(18) –0.2 ± 2.2

(30) –1.0 ± 2.1

To check if the items requested by the CONSORT Statement are adequately reported by authors

(28) –0.9 ± 2.3

(29) –1.1 ± 2.2

(26) –0.8 ± 2.2

(16) 0.1 ± 2.6

(25) –0.5 ± 2.6

To check if the authors referenced all important studies

(29) –1.1 ± 1.7

(28) –1.1 ± 1.8

(25) –0.8 ± 1.7

(33) –1.6 ± 1.6

(33) –1.5 ± 1.2

To evaluate whether figures and tables can be understood without having to refer the text

(30) –1.1 ± 1.9

(30) –1.1 ± 1.9

(31) –1.2 ± 1.8

(26) –0.7 ± 2.0

(29) –1.0 ± 2.2

To check if items requested by the CONSORT extensions are adequately reported when appropriate

(31) –1.2 ± 1.9

(32) –1.5 ± 2.0

(29) –1.1 ± 1.8

(25) –0.7 ± 1.6

(28) –0.8 ± 2.0

To compare information recorded on a clinical trials register such as ClinicalTrials.gov and reported in the manuscript

(32) –1.3 ± 2.1

(31) –1.5 ± 2.1

(28) –1.0 ± 2.3

(31) –1.4 ± 1.7

(32) –1.5 ± 2.3

To read the journals’ recommendations to reviewers

(33) –1.4 ± 2.0

(33) –1.7 ± 1.9

(32) –1.2 ± 1.7

(27) –0.8 ± 2.5

(31) –1.1 ± 2.6

To evaluate all appendices when available

(34) –2.3 ± 1.6

(34) –2.2 ± 1.8

(34) –2.5 ± 1.4

(34) –1.8 ± 1.8

(34) –2.8 ± 1.1

To evaluate the adequacy of the language

(35) –2.9 ± 1.7

(35) –2.7 ± 1.9

(35) –3.0 ± 1.5

(35) –3.1 ± 1.3

(35) –3.4 ± 1.4

To evaluate if authors respect the requested format for references

(36) –4.0 ± 1.4

(36) –3.8 ± 1.6

(36) –4.0 ± 1.3

(36) –4.4 ± 1.2

(36) –4.0 ± 1.4