Raven's Eye CAQDAS and natural language software acquires generalizable results from opinion polling

Er=Wpa + Ipi : The summation of panoramic wisdom and piercing insight.

Hillary Clinton or Donald Trump? An online survey of reasons for voter preference.

This report presents detailed information on the specific steps involved in conducting a quantitative phenomenology with Raven's Eye. Our presentation format has changes since this report was written. An example of our current presentation format may be found here.

From August 28th to September 1st, 2015, Raven’s Eye surveyed a stratified national sample of 396 Amazon Mechanical Turk workers about the reasons behind their voting preferences.

We asked:
1. If elections for the President of the United States of America were held today, and your choice was between the following two candidates, for which one would you cast your vote?
( ) Hillary Clinton ( ) Donald Trump.

2. Over the course of a short paragraph (i.e., at least 2-3 sentences), describe some of the main reasons why you would vote for the candidate you chose.

3. Over the course of a short paragraph (i.e., at least 2-3 sentences), describe some of the main reasons why you would NOT vote for the candidate that you DID NOT choose.
We then utilized Raven’s Eye to conduct a Quantitative Phenomenology on the responses. What follows is a summary of this study and its results.

Participants.

Of the 396 participants, 201 (50.8%) reported being female, while the remaining 195 (49.2%) reported being male. The largest reported political affiliation was Independent, with 159 (40.2%) participants, followed by 138 participants (34.8%) selecting Democratic, and 99 participants (25.0%) selecting Republican. The gender and party affiliation proportions in this study are, therefore, similar to those national proportions found in 2010 US Census data, and trends in related Pew surveys, respectively.

Results.

1. If elections for the President of the United States of America were held today, and your choice was between the following two candidates, for which one would you cast your vote?

A majority of participants selected Hillary Clinton as the candidate for whom they would cast their vote, were the elections for President of the United States of American held today, and the choice were between Hillary Clinton and Donald Trump. Tables 1 and 2 depict the percentage (and raw number) of participants selecting either Hillary Clinton or Donald Trump according to both gender (Female or Male) and political affiliation (Democratic, Independent, or Republican), respectively.

Table 1. Distribution of voting preference according to gender.


Total

Vote for Clinton
Vote for Trump
All
100% (396)
57.3% (227)
42.7% (169)
Female
50.8% (201)
63.2% (127)
36.8% (74)
Male
49.2% (195)
51.3% (100)
48.7% (95)

Table 2. Distribution of voting preference according to political affiliation.


Total

Vote for Clinton
Vote for Trump
All
100% (396)
57.3% (227)
42.7% (169)
Democratic
34.8% (138)
90.6% (125)
9.4% (13)
Independent
40.2% (159)
52.2% (83)
47.8% (76)
Republican
25.0% (99)
19.2% (19)
80.8% (80)

2. Over the course of a short paragraph (i.e., at least 2-3 sentences), describe some of the main reasons why you would vote for the candidate you chose.

Participants gave many reasons for voting for the candidate of their choice. We focused on the two most frequent reasons expressed that clearly differed according to the candidate selected1. To do so, we compared the reasons given among respondents who selected Hillary Clinton as the person for whom they would vote, to those reasons given among respondents who selected Donald Trump as the person for whom they would vote.

We then selected the two most frequent concepts (a group consisting of a word, its inflections, and its synonyms) that were highly overrepresented for each candidate (excluding candidate referent concepts, such as “he,” “she,” etc.) when compared to the concepts associated with the other candidate. Specifically, we looked for concepts based on gists (the most proportional inflection of a word) that were at least 4.0 times overrepresented (i.e., at least 4.0 times more proportional) in responses for the candidate in question, when compared to the responses for the other candidate.

Next, we derived revelatory statements from our concepts. We did so by utilizing the KWIC (Key-Word-In-Context) function of Raven’s Eye’s Word Association feature to acquire and compare the 5 words preceding, and the 5 words following, each instance of a targeted gist in the responses. The most frequent and overrepresented words associated with each gist in the responses are recorded, and phrases are constructed from them. These revelatory statements are variants on an essential thematic structure, which is in turn derived from the most popular and general concepts expressed in the revelatory statements2.

Reasons given for voting for Hillary Clinton.

Revelatory statements for concept 1: Experience.
Statements derived from associated concepts preceding the gist.
she has more political…
Hillary Clinton has foreign policy…
she has years of…
she has a lot of…
Statements derived from associated concepts following the gist.
… in government.
… in politics.
… in foreign policy.
Essential thematic structure of the natural language related to Concept 1: Experience.
Based on proportionality of words and frequency of association, the essential structure of the natural language related to Concept 1 is:
She has more EXPERIENCE.
Revelatory statements for concept 2: First.
Statements derived from associated concepts preceding the gist.
she would be the…
she has experience in government as former…
Statements derived from associated concepts following the gist.
… female president.
… woman president.
… lady.
Essential thematic structure of the natural language related to Concept 2: First.
Based on proportionality of words and frequency of association, the essential structure of the natural language related to Concept 2 is twofold:
She would be the FIRST female president.
She has experience in government as former FIRST lady.

Reasons given for voting for Donald Trump.

Revelatory statements for Concept 1: Business.
Statements derived from associated concepts preceding the gist.
he is a successful…
He is a good…
He has…
Statements derived from associated concepts following the gist.
… man.
… experience.
… savvy.
… know-how.
Essential thematic structure of the natural language related to Concept 1: Business.
Based on proportionality of words and frequency of association, the essential structure of the natural language related to Concept 1 is:
He is a successful BUSINESSman.
Revelatory statements for Concept 2: America.
Statement derived from associated concepts preceding the gist.
he will make…
Statement derived from associated concepts following the gist.
… great again.
Essential thematic structure of the natural language related to Concept 2: America.
Based on proportionality of words and frequency of association, the essential structure of the natural language related to Concept 2 is straightforward:
He will make AMERICA great again.

3. Over the course of a short paragraph (i.e., at least 2-3 sentences), describe some of the main reasons why you would NOT vote for the candidate that you DID NOT choose.

As with the previous question, participants supplied many reasons for not voting for the candidate that they did not select. We focused on the two most frequent reasons expressed that clearly differed according to the candidate not selected1. To do so, we compared the reasons given for not voting for Donald Trump among respondents who voted for Hillary Clinton, to those reasons given for not voting for Hillary Clinton among respondents who voted for Donald Trump.

We then selected the two most frequent concepts (a group consisting of a word, its inflections, and its synonyms) that were highly overrepresented for each candidate (excluding candidate referent concepts, such as “he,” “she,” etc.) when compared to the concepts associated with the other candidate. Specifically, we looked for concepts based on gists (the most proportional inflection of a word) that were at least 4.0 times overrepresented (i.e., at least 4.0 times more proportional) in responses for the candidate in question, when compared to the responses for the other candidate.

Next, we derived revelatory statements from our concepts. We did so by utilizing the KWIC (Key-Word-In-Context) function of Raven’s Eye’s Word Association feature to acquire and compare the 5 words preceding, and the 5 words following, each instance of a targeted gist in the responses. The most frequent and overrepresented words associated with each gist in the responses are recorded, and phrases are constructed from them. These revelatory themes are variants on an essential thematic structure, which is in turn derived from the most popular and general concepts expressed in the revelatory statements2.

Reasons given for NOT voting for Hillary Clinton.

Revelatory statements for Concept 1: Liar.
Statements derived from associated concepts preceding the gist.
she is a…
Hillary Clinton is a…
Statements derived from associated concepts following the gist.
… and should be convicted.
Essential thematic structure of the natural language related to Concept 1: Liar.
Based on proportionality of words and frequency of association, the essential structure of the natural language related to Concept 1 is:
She is a LIAR.
Revelatory statement for Concept 2: Liberal.
Statements derived from associated concepts preceding the gist.
she is too…
Hillary Clinton way too…
She is very…
her…
Statements derived from associated concepts following the gist.
… ideas.
Essential thematic structure of the natural language related to Concept 2: Liberal.
Based on proportionality of words and frequency of association, the essential structure of the natural language related to Concept 2 is:
She is too LIBERAL.

Reasons given for not voting for Donald Trump.

Revelatory statements for Concept 1: Experience.
Statements derived from associated concepts preceding the gist.
He has no…
He has no political…
Donald Trump doesn’t have governing…
Statement derived from associated concepts following the gist.
… in politics.
… in government.
Essential thematic structure of the natural language related to Concept 1: Experience.
Based on proportionality of words and frequency of association, the essential structure of the natural language related to Concept 2 is:
He has no EXPERIENCE in politics.
Revelatory statements for Concept 2: Racist.
Statements derived from associated concepts preceding the gist.
He is …
Donald Trump is sexist and also…
Statements derived from associated concepts following the gist.
… and he is sexist.
… and he is bigoted.
… and he is ignorant.
… and he hates women.

Essential thematic structure of the natural language related to Concept 2: Racist.
Based on proportionality of words and frequency of association, the essential structure of the natural language related to Concept 2 is:
He is RACIST.

Caveats.

We designed this survey primarily to demonstrate the abilities of Raven’s Eye in analyzing verbal information from surveys such as this. The constraints of this study should be considered before generalizing from our data. Our study was performed from August 28th to September 1st of 2015. As such, responses are likely affected by the information readily available to the participants during that time-period. Though the reported gender, location, and political affiliation of our participants largely represents national proportions, and we have sufficient sample size to generalize the main question to the overall U.S. population with 95% confidence at +/- 5 percentage points, we utilized Amazon Mechanical Turk workers to complete this survey. Because of this novel recruitment strategy, we recommend caution in generalizing these results. That being said, our data analyses certainly exceed those garnered through typical qualitative research methods in terms of geographic distribution, sample size, and inter-rater reliability.

Notes.

1In this report, we focus on deriving the top two concepts, their related statements, and their resultant essential thematic structure. There are certainly more concepts (and themes) in our existing data.

2Please see our Technicals and Practicals for more detailed information on the specific steps and procedures involved in conducting a Quantitative Phenomenology with Raven’s Eye.