Categories
Journalism Politics Technology

Is Twitter “representative” of public opinion? Media gets it wrong.

Pew Internet Research put out an analysis of Twitter conversation and compared that to its own public opinion polling. The results of their analysis? From the headline, “Twitter Reaction to Events Often at Odds with Overall Public Opinion.”

Pew’s search and analysis parter, Crimson Hexagon, took a three-day sample of tweets which contained words or phrases relevant to a given hot-button news item and analyzed them for positive or negative terms as described in their methodology:

The data on Twitter comes from an analysis of all publicly available Tweets. The time period for each event varied, but none included more than three days worth of reaction. For each subject, multiple search terms were used to identify appropriate tweets. For example, to find messages commenting on President Obama’s 2013 State of the Union Speech, Tweets were included if they appeared in the four hours following the start of his speech and used the words “state” and “union,” or “Obama,” or “SOTU.”

Unlike most human coding, CH does not measure each post as a unit, but examines the entire discussion in the aggregate. To do that, the algorithm breaks up all relevant texts into subsections. Rather than dividing each Tweet, paragraph, sentence or word, CH treats the “assertion” as the unit of measurement. If 40% of a story fits into one category, and 60% fits into another, the software will divide the text accordingly. Consequently, the results are not expressed in percent of Tweets, but rather the percent of assertions out of the entire body of stories identified by the original Boolean search terms.

But while we can argue about the efficacy of their methods (more on that later), the media seems to be willfully getting the results wrong. Check out a quick sample of the headlines:

Sample of conservative reactions by Twitterverse, at odds with the Daily Caller’s miopic understanding of reality. Source: Pew

This list even includes a majority of tech-savvy websites. The Daily Caller (ever the picture of reliable reportage) even took to interpreting the report as calling Twitter “a liberal, miopic, negative place.” This, despite the fact that the report clearly says that the Twitterverse occasionally breaks Conservative when public sentiment is Liberal. But there is a big difference between opinion on Twitter being “at odds” with general public opinion and not being a “reliable” indicator.

For a start, when 16% of Americans all share a common demographic bond – our affinity for Twitter – it should not be at all surprising that we share a common set of opinions. Neither should it be surprising that those opinions differ from a wider sample of the public.

Moreover, public opinion changes. It changes as people learn more about things and as facts present themselves. That very often takes more than three days for a lot of people. Twitter being heavily weighted to breaking news, tweeps have a tendency to be ahead of the curve.

We tweeps tend to “watch” the news unfold more or less together in real-time, so social reaction must also play its part. Twitter users have also been shown to be “influencers,” meaning we tend to voice our opinions to our friends more often than the average bear, you might say. It would be interesting to do the same sample, three days after a news break and then the following three days, to see if there is any change in the dichotomy between popular and Twitter sentiment.

But all of this presumes that Pew’s research is accurate. This is a very dicey affair, as indeed all public opinion polling is. But in this case, instead of speaking directly with tweeps, they’re using aggregation and analysis software to decide what is “positive” vs. “negative” or “conservative” vs. “liberal.” We are nowhere near a level of confidence in “Big Data” analysis of this type to consider this analysis anything other than hugely questionable.

The algorithms Crimson Hexagon uses would need to interpret tweets according to whether or not they’re really relevant to a given topic, whether the tweet was being sarcastic or some other form of humor, and whether the “negative” words are a function of genuine negativity or simple a reflection of language. Buffalo alone would be enough to give coders cold sweats, trying to interpret all that negativity.

And of course, it needs to be pointed out: Pew’s opinion polls do not reflect public sentiment any more accurately than Twitter, simply because Pew says they do. I am a big fan of Pew’s work – I cite it a lot, especially on (irony alert) Twitter. But by no means does this study reflect any kind of scientific fidelity.

Categories
Media Technology

Richer, better-educated and connected: the Digital Divide continues to languish

While the explosive growth of mobile technologies appears to span all income levels fairly equally, Pew Research’s Director Lee Rainie demonstrates in a recent presentation that not much at all has changed in terms of home access and the Digital Divide.

The numbers are shocking in their uniformity. For nearly every category of Internet access – from simply being on the Internet to broadband access – the same proportion of higher-income and better-educated respondents report high adoption rates while the lower-income and less-educated respondents report very low adoption rates. 97% of college-educated and 98% earners of incomes in excess of $75k are on the Internet, compared to 45% of those with no high school diploma and 65% of those earning less than $30k.

To some extent this is certainly a product of our upwardly-mobile society: because you don’t have a degree now does not mean you won’t in the future, at which point with a higher income, you will likely invest in broadband Internet access. However, for those whose incomes do not change or do not change much, this is a real trap. And as our society becomes increasingly dependent on the Internet, there is always the threat that this disparity could become self-perpetuating, with low-education consumers barred from access to the information store of the world.

That mobile adoption has increase at a rapid rate may or may not be a comfort to those concerned with Internet access disparity. While mobile apps and Internet have increased in their proliferation and usefulness, a phone or tablet simply does not offer the range of potential uses that a desktop or laptop does. Not to mention the bandwidth restrictions that come with using wireless broadband.

Digital Differences and Money | Pew Research Centers Internet & American Life Project.

LATE UPDATE:  Interestingly, Pew Internet just put out a poll today that says, of the one in five Americans who do not use the Internet, about half of them say they’re just not interested. How does this reflect on what we know so far about income and education disparity? I wonder what these numbers look like if you break them down by similar demographics?

Categories
Politics

Time to Think 2012? No, But Let’s Do It Anyway.

Nauseating, I know. But the pollsters are out in force today, testing the pulse of Americans – those willing to sit on the phone and answer survey questions, at least – for the likelihood of a second Obama term. Pew and Quinnipiac both have polling data released today, both showing roughly the same thing, but with remarkably different editorial conclusions.

Pew has a poll showing that less than fifty percent of the public would like to see the president elected to a second term, but points out that previous presidents have faired little better. Quinnipiac, for its part, has a poll showing that about fifty percent believes the president does not “deserve” a second term. Note that this is not what the question asked, exactly, its just what the editor chose to use in the summary. They point out that this is his lowest number since they’ve been tracking it and also that he loses by a slim margin to an Unnamed Republican Candidate.

So the numbers are around fifty percent with almost two years left to decide. My expert opinion is that “these numbers are likely to change.”