Click here to purchase the entire book in PDF format. Randomization Whenever you're doing any listening test you have to be careful to randomize everything to avoid order effects. Subjects should hear stimuli in different orders from each other, and stimuli presentation (whether a sound is A or B in an A/B/X test for example) should also be randomized. This is because there is a training effect that happens during a listening test. For example, if you were testing codec degradation, and you presented all of the worst codec's to all the subjects first, they will gradually learn to hear the artifacts that are under test, while they're doing the test. As a result, the better codec will be graded more harshly because the subjects better knew what to listen for. If you randomize the presentation order and use that same order for all the subjects, then the training will not be as obvious, but you might get situations where one codec made then next one look really bad, and since everyone got them in that order, the second was unfairly judged. The only way to avoid this is to have each person get a different presentation order that is completely random. The moral of the story here... just randomize everything. There are exceptions. Let's say that you're doing a listening test to evaluate who should be on a listening panel. In this case, you want everyone to have done exactly the same test so that they're all treated fairly. So, in this case, everyone would get the same presentation order, because it's the subjects themselves that are under test, not the sound stimuli.