New Methods of Making Sense of Human Behavior

Virtually any city agency could benefit from getting accurate and useful neighborhood-level data about the public’s perception of community conditions. Doing so creates opportunities for more efficient allocation of agency resources; greater insight into what any given community cares about or prioritizes – be it crime, education, public or environmental health; and betterment of government-citizen relations (with the public sector demonstrating a real interest in the thoughts, opinions and actionable ideas of citizens).

At present, to get this data, local agencies throughout the country often use traditional attitudinal survey and polling methodologies, for instance telephone or mailed surveys, to gauge public perception. Notably, however, traditional surveys and polling create certain limitations that make obtaining an accurate, up-to-date picture of attitudes and conditions within a community difficult. Data-driven technological innovation, however, coupled with a proliferation of social media use, have resulted in a number of emerging techniques that could be applied in parallel or in place of traditional attitudinal surveys, potentially with less cost and burden.

Traditional Survey & Polling Methodologies

How Do They Work?

Traditional survey techniques often include “random sampling,” or targeting a random selection of participants within a desired community or population based on “probability theory.” The core concept of probability theory suggests that if respondents are chosen “randomly and appropriately from the larger population, the results from that random sample will be very close to what we would get by interviewing every member of the population.” Random sampling attempts to ensure that any adult in that given population has a chance of making it into the random sample. Yet “weighting” – a statistical adjustment to account for “the relative contribution of respondents” – is usually done after the fact in order to adjust the received data “to ensure that the sample more accurately reflects the characteristics of the population from which it was drawn and to which an inference will be made.”

Limitations

Traditional survey and polls can be subject to many sources of errors, ranging from “how well the questions were designed and asked to how well the interview was conducted to how well the sample design was implemented.” Specifically in the context of polling public perception of local government performance and service delivery, the following limitations also exist:

Inclusivity – Traditional polling, wherein sample sizes are designed and weighted to reflect accurate demographics in a city, tend to be ineffective at giving youth and minorities a voice. Because traditional techniques often rely on landline telephone polls, they disproportionately exclude low income and younger populations with mobile phones.

Sustainability – Traditional polling is rarely easily repeatable and almost never ongoing, instead providing only a snapshot in time of community perception.

Nuance – Traditional surveys tends to ask questions using degrees of sentiment, precluding unstructured or unanticipated responses that could help provide a more detailed picture of one’s opinion or experience.

Collaboration – Traditional surveys or polls are administered rather than co-created. This tends to mean those who typically fear or dislike the government voluntarily opt not to participate.

Relevant Innovations &Techniques

Traditional polling unquestionably provides meaningful insights into the attitudes of a community. However, the last two presidential elections – wherein Nate Silver’s advanced prediction models far outperformed attitudinal surveys conducted with the public – demonstrate a greater societal move toward more comprehensive, data-driven methods for identifying and predicting public opinions and preference. Furthermore, advancements in technology have enabled new methods to emerge for documenting and understanding community conditions without having to rely exclusively on traditional attitudinal surveys.

New techniques such as sentiment analysis or opinion mining, prediction markets, crowdsourced data gathering, cross-agency data sharingand social physics applications can help to develop a better and more accurate picture of community sentiment and situational conditions from a larger group of affected citizens. Initiatives like ipaidabribe.com also prove instructive as example innovations for ensuring an avenue for the historically under-represented or voice-less to get heard. Emerging incentive structures – such as those applied in many gamification projects – can also be leveraged to encourage members of a community to share their opinions, perceptions and experiences. Moreover, many of these new techniques can be combined or blended, including with more traditional methodologies, to create a more comprehensive picture of community conditions and public perception.

The “computational treatment of opinion, sentiment, and subjectivity in text.” An emerging field wherein computer science meets linguistics, sentiment analysis provides a technical means to interpret and understand large amounts of often-unstructured opinion or attitudinal data, such as that shared freely and publicly on personal blogs, Twitter, online reviews and Facebook pages or statuses. These data sources enable rich analysis of public perception and sentiment without the need for an inquiring party to pose pre-scripted questions. “[S]ocial media presents itself as a ‘big data’ source of citizen voice,” empowering government agencies to “constantly keep a tab on pulse of its citizens” to the end of providing better government services.[1]Sentiment analysis has been used in both the private and public sectors to improve product development and to help understand public sentiment following major community events.

In the public sector, sentiment analysis has proved a useful tool for the U.K. National Health Service (NHS) following U.K. riots in 2011. The NHS took data published by The Guardian that included over 2.5 billion tweets during the time of the riots. NHS then created a directed, unweighted network by intersecting the friends of all users with the list to better understand community sentiment following the riots. This network enables analysis of geographic neighborhoods, interest groups and media outlet companies at high, medium and low resolution during the riots. So far, it has also helped to produce an understanding of different types of engagement exemplified online – from active leaders to listeners – and has helped the NHS to form a deeper understanding of who are the influential voices outside of traditional interest groups. The work thus far has also provided a means of identifying which particular topics each group talked and cared about the most post-riots.

Sentiment analysis has potential to supplement traditional polling of public perception by a city agency, providing a much broader and more nuanced understanding of citizens’ attitudes toward their government. Furthermore, sentiment analysis provides a way to measure attitudes of large samples of individuals over time, creating the possibility of a more sustained approach to gauging city-wide agency performance in the eyes of citizens. Finally, the data sources from which sentiment analysis can be performed are often public and generated on an on-going basis, making it more easily accessible at low or no cost to a city.[2]

Sometimes referred to as “ideas futures,” prediction markets enable forecasting through the “elicitation and aggregation of individual judgments” and have been found to be effective in both political and sports predictions. Within prediction markets, the “stock price serves as an ongoing real-time forecast of future results associated with the question being asked.” Participants in prediction markets are typically asked yes/no questions regarding a potential future scenario or forecast. Answers are turned into probabilities and when a probability reaches a certain level, participants can then buy-in to that potential outcome and ultimately profit if that prediction becomes reality.[3]

One such prediction market, the Hollywood Stock Exchange, allows participants to buy and sell “virtual shares” of stock in actors and movies. When those movies come to fruition and do well at the box office, the stocks rise.

Prediction markets may provide certain learnings to a city agency, for instance, in helping to understand and predict future occurrences of crime or disease by neighborhood or to help the city better prepare for how holidays, major sporting events, or major storms will impact community conditions.

Crowdsourced data gathering is another technique that could help provide a more comprehensive picture of community conditions and perception about government performance and services. Today, many institutions seek to improve decisionmaking by better leveraging troves of newly available “big data” sources and finding ways to invite and empower citizens to play a role in both the collection and use of that data for social good.

In Boston, The Boston Research Map is an effort underway to combat urban crime. The Boston Area Research Initiative (a Harvard University-based collaborative of academics and city officials) began mining data from a Boston “hotline, website and mobile app for citizens to report everything from abandoned bicycles to mouse-infested apartment buildings” and is now using this data to create an “almost real-time guide” – a map – to the urban condition.[4]

Deploying such crowdsourcing techniques in any city could help provide a more comprehensive picture of community conditions. Crowdsourcing also provides an opportunity to gauge the sentiment of citizens who choose not to participate in traditional polling. Notably, the cost associated with many crowdsourced data projects remains relatively low – for example, any data storage and development costs are likely far less than the labor expenses incurred by an agency to staff ongoing data collection efforts in the field. Devolving data gathering responsibility to citizens who can at times use personal devices also mitigates the need for procurement of any high-tech sensor equipment.

Relevant and existing data sources that can help shed light on situational conditions,public sentiment and even crime typically exist outside of that collected by one particular city agency or department. This data could be responsibly shared across and among relevant actors to various degrees, depending on the sensitivity and privacy implications of the data.

Data sharing has proved particularly helpful in the public safety context in the United Kingdom. In fact, an evaluation showed that “judicious sharing of unique information about locations and times of violence derived from [Accident and emergency department] patients was a powerful and effective means of targeting police and other local resource to bring about violence reduction.” Of particular note, the evaluation uncovered prior discrepancies in data access across the agencies; in 1997 and 1998 for instance, the police only recorded about 50% of the assaults that had been recorded in the A&E department.

Data sharing, especially in relation to public safety or health could enable city agencies to better detect opportunities for crime prevention or enhance a more comprehensive situational strategy for neighborhood wellness.

Investing in social physics initiatives like Pentland’s has potential to generate new data-driven insights into how public perception of social conditions affect individual and collective behavior within a community. Furthermore, leveraging freely available, open or low-cost data sets for these purposes means the potential cost for initiating such a project remains low.

An emerging technique for incentivizing engagement, gamification refers to the application of “game design thinking to non-game applications to make them more fun and engaging.”[5] Initially used in the marketing context, gamification applications have increased rapidly in recent years, emerging in a variety of contexts from personal finance to exercise to education.

A San Francisco-based project Skillville applies gamification in the public sector. The initiative aims to combat unemployment and decreasing government budgets by allowing job seekers to complete government-created microvolunteering opportunities, through which they can earn badges for skill validation. Badges can then be redeemed for “real-world rewards” throughout the city, such as mentoring workshops, networking events and or job interviews.

Experimenting with emerging techniques like gamification in the public sector could help any city agency learn more about the shortcomings of traditional approaches to community outreach and about what incentives may work best to motivate individuals to meaningfully interact with public officials in a variety of contexts. Deploying gamification techniques may also help to develop better situational awareness neighborhood by neighborhood and help to empower citizens by devolving certain government responsibilities through a more sustained and interactive dialogue with the community.

Notably, present applications of polling/surveying layer traditional methodologies with new innovations to create a more comprehensive picture of community conditions and public perception. Blended techniques have been used in a variety of contexts to increase meaning and utility of sentiment and situational data.

A new interactive surveying platform, Agreeble, for example, launched in Beta version in January 2104 aims to combine traditional Likert scale poling of the public with more nuanced, unstructured opinion submissions to help gauge public sentiment around polarizing issues. Professor Nick Beauchamp at Northeastern University has combined sentiment analysis of large scale Twitter data combined with traditional election survey data to successfully predict changes in public opinion in state level elections. Combining less traditional techniques, Professor Arno Scharl of MODUL University of Vienna’s Department of New Technology created uComp, a program that uses both sentiment analysis, crowdsourcing and gamification techniques to better understand public perception of climate change.

Blending techniques creates potential utility to any city agency trying to develop a comprehensive and real-time understanding of public perception before and after major community events; in identifying and minimizing service breakdown hotspots; and in spotting and stopping dangerous, inefficient or ineffective neighborhood trends and forecasts.