Click here to purchase the entire book in PDF format. Quantitative Descriptive Analysis (QDA) The biggest disadvantage of the RGT is that you get different sets of words from each of your subjects, do you can't go back one of the coffee shops and tell them that everyone rated their coffee more bitter than everyone else's, because some people used the word ``sharp'' or ``harsh'' instead of ``bitter.'' You can pretend to yourself that you know what your subjects mean (as opposed to what they say) but you can't really be certain that you're translating accurately... So, to alleviate this problem, you get the subjects to talk to each other. This is done in a different technique called Quantitative Descriptive Analysis (or QDA). In the QDA technique, there is a word elicitation phase similar to the knowledge elicitation phase in the RGT. Put all of your subjects in a room, each with a little booth all to themselves. Give them the four cups of coffee and ask them to write down all of the words that they can think of that describe the coffees. You'll get longs lists from people that will include words like sweet, bitter, brown, liquid, dark, light, sharp, spicy, cinnamon, chocolate, vanilla.... and so on... tons and tons of words. The only words they aren't allowed to use are preference words like good, better, awful, tasty and so on. No hedonic ratings allowed (although some people might argue this rule). Then you collect all the subjects' lists, sit everyone around a table and write down all of the words on a blackboard at the front of the room. With everyone there, you start grouping words that are similar. For example, maybe everyone agree that bitter, harsh and sharp all belong together in a group and that hot, cold, and lukewarm all belong together in a different group. Notice that these groups of words can contain opposites (hot and cold) as long as everyone agrees that they are opposites that belong together. Once you're done making groups (you'll probably end up with about 10 or 12 groups give or take a couple) you put headings on each. For example, the group with hot and cold might get the heading ``Temperature.'' You then pick two representative anchor words from each column that represent the two extreme opposite cases (i.e. extremely hot and extremely cold, or bitter and sweet). So, what you're left with, after a bunch of debating and fighting and meeting after meeting with your entire group of people, is a list of about 10 attributes (the headings for your word groups), each with a pair of anchor words that describe opposite ends of the scale for that attribute. Hopefully, everyone agreed on the words that were chosen by the group, but if they don't, then at least they were at the meetings and so they know what the group means when they use a certain word. Your next step is going back to the stimuli (the coffee) and asking people to rate each cup of coffee on each attribute scale. You can do this by giving the subject all cups of coffees, and asking for one attribute to be rated at a time, or you could give the subject one cup of coffee and ask for all attributes to be rated, or if you want to be really confusing, you give them all cups of coffee and a big questionnaire with all attributes. This is up to you and how comfortable your subjects will be with the rating phase. The nice thing about this technique is that everyone is using the same words. You can then use correlation analysis to determine whether or not people are rating the attributes unidimensionally (which is to say that people aren't confused and using two different attributes to say the same thing. For example, there might be a ``Bitterness'' attribute ranging from ``bitter'' to ``not bitter'' and a separate ``Sweetness'' attribute ranging from ``Not sweet'' to ``Sweet''. After you're finished your test, you might notice that any coffee that was rates as bitter was also rated as not sweet and that all the sweet coffees were also rated as not bitter. In this case, you would have a negative correlation between attributes, and you might want to consider folding them into one attribute that ranged from bitter to sweet. The results show that this is what people are doing in their heads anyway, so you might as well save people's time and ask them one question instead of two.... There is one minor philosophical issue here that you might want to know about regarding the issue of training of your subjects. In theory, all of your subjects after the word elicitation phase know what all of the words being used by the group mean. You may find, however, that one person has completely different ratings of one of the attributes than everyone else. This can mean one of two things: either that person understands the group's definition of the word and perceives the stimulus differently than everyone else (this is unlikely) or the person doesn't really understand the group's definition of the word (this is more likely). In other words, the best way to know that everyone in your group understands the definitions of the words they are using is if they all give you the same results. However, if they all give you the same results, then they become redundant. Consequently, the better trained your listening panel member is, the less you need that person, because he or she will just give you the same answers as everyone else in the group. This is a source of a lot of guilty feelings for me, because I hate wasting people's time...