Understanding your results.
The first thing you may notice upon receiving your results is the main chart. This chart is designed to help you get a sense of the whole by identifying the gists in your responses. As such, the main chart graphically depicts the relative proportionality of the most frequent words in your columns.
Title information. In the title area at the top of the chart are reported the names of the project and column being analyzed. Your project name is the name assigned to the project when it was uploaded to Raven's Eye. The name reported as the column being analyzed is derived from the label that as assigned to it during column classification.
Y-axis. The hash lines and numbers along the y-axis (or vertical axis) of the chart demarcate various degrees of overrepresentation (see discussion on the fifth column of the main table, below). By default, the scale of this chart depends upon the range of overrepresentation displayed among the 50 most frequent words in your response pool. However, you can also manually adjust the scale of the y-axis in the Settings page.
X-axis. Along the x-axis (or horizontal axis) are the 50 most frequent words in your response pool. These words are arranged in the order of decreasing frequency.
Message at a glance.
Beneath the main table is an area labeled Message at a Glance. In this area are listed the most frequent words in your response set, in prose format. As with the main chart, this area is designed to facilitate a sense of the whole through identification of various gists and their elaboration into concepts via inflections and synonyms. Depending on the specificity of the stimulus eliciting the response being analyzed, popular phrases and sentences may already be apparent in this section, which will eventually comprise your revelatory themes and essential structure.
Your results are initially displayed in a table containing 5 columns. The table will, by default, arrange data according to the proportion of the sample that each word comprises, which is displayed in the third column. Columns can be re-sorted by clicking on the title of a column.
The first column is entitled Word. This column lists each and every word in your data set.
The second column, # Sample, identifies the number of times (or frequency) that each specific word is found in your sample.
The third column, % Sample, identifies the proportion of the sample accounted for by each and every word in the sample, displayed as a percentage. Therefore, numbers to the right of the decimal indicate fractions of one percent. By default, this column orders all of the data presented in the table. It is arranged in descending order.
The fourth column, % Corpus, identifies the proportion of the background comparison corpus accounted for by each and every word in the sample, converted to percent. As with the third column, in this column numbers to the right of the decimal indicate fractions of a percent.
The fifth column, Overrepresentation, is the quotient resulting from dividing the third column (% Sample) by the fourth column (% Corpus), and represents the degree to which each given word is overrepresented in your sample, based on the proportion of that given word in the background comparison corpus. For instance, if “1.0” appears in this column, that means the word is used in the sample at the same rate that it is found in the comparison corpus, and is not then particularly associated with your question or stimulus. If, however, “10.3” appears in this column for a given word, that means that the word expresses a proportionality that is 10.3 times greater in your sample than it's proportionality in the background comparison corpus, and is highly associated with the question or stimulus at-hand.
As may be apparent, the numbers found in the Overrepresentation column serve to indicate those words or gists that are associated with the stimulus eliciting the response set being analyzed. If, for instance, the response set results from being presented with a question on a topic, these overrepresented words or gists will be those that most of your participants associate with the topic and utilize to answer the question. As such, they will be comprised of words or gists used to refer to the topic and its associated concepts, such as associated actions, feeling-states, time-frames, colloquialisms, and other parts of written language associated with the expression of the response to the particular question.
Because they are an expression of relative word proportionality, the quotients found in the Overrepresentation column serve and additional function: they act as estimations of confidence. For instance, in response to the first question on our survey on the meaning of life1, the word love had an overrepresentation quotient of 32.389. This means that it expressed a frequency that was proportionally a little more than 32 times greater than its frequency of appearance in the background corpus. Given that 346 participants provided responses that were on average 51.09 words in length, a total of 17,678 words comprised the response set to the question.
In order for the proportionality of the word love to become equal to its expression in the background corpus (i.e., in order for the Overrepresentation quotient to become 1.0), and thereby become not particularly associated with the stimulus producing the responses that contain it, we would need to increase our existing word pool by approximately 31 times its current size without acquiring any additional instances of the word. Given consistency in the average response length, this would necessitate acquiring an additional 548,000 words across 10,726 additional participants, each of whom must never mention the word in response to the question. Given the word's frequency in the current pool, however, it becomes quite apparent that such a result would be highly unlikely, and one can then maintain a high degree of confidence that the word love is associated with this particular question about the meaning of life.
If one form of a frequently used word is overrepresented in your sample, then often times that word’s inflections, determiners, and synonyms will also be overrepresented. For instance, if the word, “hurry,” is both frequent and overrepresented in your sample, then such word inflections as “hurries,” “hurried,” and “hurrying,” are also likely to be overrepresented. Similarly, such synonyms as, “rush,” and, “hustle,” are also likely to be overrepresented. These inflections, determiners, synonyms and other related terms together elaborate on your basic gists to form what we define as concepts.
This can be seen in the image to the right, which depicts the main table resulting from analysis of the first question from our survey on the meaning of life. In the image, the first few gists forming the concepts we and you are highlighted in blue and yellow respectively.
If your stimulus or question is specific and you have acquired a sufficiently sized sample, you will often be able to find phrases and complete sentences formed by reading downward the overrepresented words in the first column (the beginnings of such a sentence can be seen in the table to the right, which is based on responses to the question, “Why are you enrolled in this culture training?”). If such sentences are comprised of words from the first 50 or so listed by default in the table, then they likely represent the revelatory themes and essential structure of your response set, as expressed in the vocabulary of your respondents. We describe a standardized means of generating such themes and structures when discussing the features available in the Advanced menu button, below.
If the question or stimulus evoking the response is broad, as in, "Tell me about your life," or, if you are analyzing a large written work (such as a book) in its entirety, your responses will be comparatively broad. In these instances, it will be more difficult to quickly identify a well-constructed phrase or sentence in the first column, but the main gists and concepts will still be apparent.
If you require more specificity, you can either change your meaning units by subdividing your responses into subsections (such as sentences, paragraphs, or sections), or filter your cases by particular keywords and reanalyze your data, or ask a more specific question.
Menu bar additions.
When you receive the results of your analysis, five new buttons appear in your menu bar. These new buttons each correspond to different functional features available in Raven’s Eye. They are labeled Meta, Variables, Reclassify, Export Data, and Advanced.
We describe each button individually in this section, though several of their functions are also discussed in detail during subsequent pages of these Practicals, as well as in the steps of conducting a Quantitative Phenomenology with Raven's Eye.
Meta. The Meta button on the menu bar reveals information helpful in understanding the general context of your responses, or what we call geists. By clicking on the Meta button, the corpus you are using is displayed in a dropdown menu, with the following information about your response pool is revealed. If you have selected to combine your response columns, you will receive global information about the pool of natural language that you've uploaded. If, however, you have chosen to analyze your response columns separately, this information will subdivided according to each response column.
Reading ease—this is the Flesch Reading Ease Score of your sample, or if selected, your subsample.
Grade level—this is the Flesch-Kincaid Grade Level of your sample, or if selected, your subsample.
Sentence count—this is the total sentences in your sample, or if selected, your subsample.
Word count—this is the total word count of your sample, or if selected, your subsample.
Average cell word count—this is the average number of words per cell in your sample, or if selected, your subsample.
Syllable count—this is the total number of syllables in your sample, or if selected, your subsample.
Variables. By clicking on the Variables button in the menu bar, you can select particular states or classes of your variables to be compared against other states or classes of your variables. This is useful for comparing response patterns between such things as experimental conditions or demographic characteristics. The menu that appears when selecting this button is explained in detail in the Comparing your variables page of these Practicals.
Reclassify. By clicking on the Reclassify button, you can alter your previous column classifications. For example, data that was previously excluded can now be selected and included as a response to be analyzed. The process of reclassifying variables is explained in detail in the Working with previously uploaded projects page of these Practicals.
In order to utilize the features listed in the Advanced dropdown, you must first select a word in the main table. Additional words can be analyzed sequentially by first deselecting any currently selected words, then selecting a new word for analysis.
Word in Context—The Word in Context feature allows you to view each instance of a selected word or gist as it occurs in the context of your uploaded natural language sample. The context can be limited to the sentence in which the selected word occurs, or the entire response (or case).
CORVID—The CORVID feature automatically produces the network of words or gists associated with the selected word or gist. After selecting a word in the main table, then selecting CORVID, from the Advanced menu, a table appears that contains the most frequent first- and second-order words surrounding the selected word or gist.
Word Association—The Word Association feature facilitates performance of what is commonly referred to as the Key-Word-in-Context (KWIC) method or procedure. After selecting Word Association, you may then determine whether to group the association by sentence or by full response (e.g., one full case). Once you have determined this, you may utilize two dropdown menus to decide to include up to the 10 words preceding, surrounding, or following the word selected in the main table. After doing so and clicking on the Update button, a table appears that lists the frequency of words or gists meeting your set criteria.
More information about understanding and interpreting your results can be found in the Get a Sense of the whole, Constitute parts by identifying meaning units, and Transform data into revelatory statements pages of the steps in conducting a Quantitative Phenomenology with Raven's Eye. Additional methodological and theoretical details, can be found elsewhere in the Technicals.
1 The text of the question was, "Over the course of a short paragraph (i.e., 3-5 sentences), please describe in your own words: what is the meaning of life?"