Consensus-based guidance for conducting and reporting multi-analyst studies
(1)
,
(1)
,
(2)
,
(3)
,
(4)
,
(3)
,
(4)
,
(5)
,
(6)
,
(7)
,
(4)
,
(8)
,
(9)
,
(10)
,
(11)
,
(12)
,
(6)
,
(13)
,
(6)
,
(14)
,
(15)
,
(16)
,
(17)
,
(4)
,
(18)
,
(19)
,
(19)
,
(14)
,
(20)
,
(21)
,
(19)
,
(4)
,
(22)
,
(23)
,
(6)
,
(24)
,
(13)
,
(25)
,
(26)
,
(4)
,
(27)
,
(21)
,
(6)
,
(28)
,
(29)
,
(30)
,
(31)
,
(32)
,
(25)
,
(33)
,
(34)
,
(35)
,
(3)
,
(6)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
Balazs Aczel
- Function : Author
- PersonId : 792930
- ORCID : 0000-0001-9364-4988
Barnabas Szaszi
- Function : Author
- PersonId : 792939
- ORCID : 0000-0001-7078-2712
Gustav Nilsonne
- Function : Author
- PersonId : 794676
- ORCID : 0000-0001-5273-0150
Jojanneke Bastiaansen
- Function : Author
- PersonId : 814792
- ORCID : 0000-0003-4831-6402
Daniel Benjamin
- Function : Author
- PersonId : 799920
- ORCID : 0000-0002-2642-5416
Udo Boehm
- Function : Author
- PersonId : 814793
- ORCID : 0000-0002-8677-0721
Rotem Botvinik-Nezer
- Function : Author
- PersonId : 802040
- ORCID : 0000-0003-2669-1877
Laura Bringmann
- Function : Author
- PersonId : 814794
- ORCID : 0000-0002-8091-9935
Niko Busch
- Function : Author
- PersonId : 814795
- ORCID : 0000-0003-4837-0345
Emmanuel Caruyer
- Function : Author
- PersonId : 740306
- IdHAL : emmanuel-caruyer
- ORCID : 0000-0002-8547-7726
- IdRef : 237499789
Andrea Cataldo
- Function : Author
- PersonId : 814796
- ORCID : 0000-0003-2787-224X
Nelson Cowan
- Function : Author
- PersonId : 779096
- ORCID : 0000-0003-3711-4338
Andrew Delios
- Function : Author
- PersonId : 764203
- ORCID : 0000-0002-6791-227X
Noah van Dongen
- Function : Author
- PersonId : 814797
- ORCID : 0000-0003-0387-7388
Johnny van Doorn
- Function : Author
- PersonId : 814798
- ORCID : 0000-0003-0270-096X
Ann Dreber
- Function : Author
- PersonId : 814799
- ORCID : 0000-0003-3989-9941
Gary Egan
- Function : Author
- PersonId : 814800
- ORCID : 0000-0002-3186-4026
Morton Ann Gernsbacher
- Function : Author
- PersonId : 814801
- ORCID : 0000-0003-0397-3329
- IdRef : 033498725
Rink Hoekstra
- Function : Author
- PersonId : 814802
- ORCID : 0000-0002-1588-7527
Sabine Hoffmann
- Function : Author
- PersonId : 799122
- ORCID : 0000-0001-6197-8801
Felix Holzmeister
- Function : Author
- PersonId : 794667
- ORCID : 0000-0001-9606-0427
Juergen Huber
- Function : Author
- PersonId : 794668
- ORCID : 0000-0003-0073-0321
Magnus Johannesson
- Function : Author
- PersonId : 794109
- ORCID : 0000-0001-8759-6393
- IdRef : 056756410
Michael Kirchler
- Function : Author
- PersonId : 794669
- ORCID : 0000-0002-5416-2545
Jean-Francois Mangin
- Function : Author
- PersonId : 183406
- IdHAL : jean-francois-mangin
- ORCID : 0000-0002-1612-461X
- IdRef : 126215499
Ben Newell
- Function : Author
- PersonId : 814803
- ORCID : 0000-0003-1898-205X
Russell Poldrack
- Function : Author
- PersonId : 783513
- ORCID : 0000-0001-6755-0259
- IdRef : 165846089
Don van Ravenzwaaij
- Function : Author
- PersonId : 814804
- ORCID : 0000-0002-5030-4091
Martin Schweinsberg
- Function : Author
- PersonId : 806189
- ORCID : 0000-0003-3529-9463
David Shanks
- Function : Author
- PersonId : 814806
- ORCID : 0000-0002-4600-6323
- IdRef : 050278002
Samuel St-Jean
- Function : Author
- PersonId : 768615
- ORCID : 0000-0002-8092-2974
Jelte Wicherts
- Function : Author
- PersonId : 814807
- ORCID : 0000-0003-2415-2933
Abstract
Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research.
Origin : Publisher files allowed on an open archive