Employing Sentiment Analytics To Address Citizens' Problems
It is easy to imagine the internet as a giant mirror that, in its own unique ways, reflects the society that we live in. The internet provides a massive information outlet for individuals, organizations or public bodies that are interested in understanding the pulse of the general public at any given time. As we know, phrases such as “globally trending” are commonly heard in today’s cultural lingo. The worldwide web provides a platform for people to voice their opinions about any topic. In a way, the internet involves people baring their true emotions or sentiments for others to see—something that such individuals may not do in the real world.
For several years, the concept of businesses organizations using sentiment analytics to learn about their customers’ likes, dislikes, product preferences and other factors has been prevalent. However, many of us believe that governments around the world could also use their resources to create the necessary infrastructure through which they would be able to ascertain the problems of their citizens before resolving them through a structured process.
AI-powered sentiment analysis systems can come in handy for the public sector bodies to understand the solvable problems of their people more closely.
The Role of NLP and Machine Learning
Natural Language Processing (NLP) and other linguistic and text-based tools play an instrumental role in AI-enabled sentiment analysis. NLP and machine learning allow AI-powered systems to detect and analyze the opinions and sentiments involved in a comment or any piece of text posted by someone on the internet. Firstly, NLP identifies and converts human language into machine language. NLP's semantic and syntactic capabilities allow the tool to understand what exactly someone means based on their online words.
Once the machine-understandable text is generated from the natural language text, the machine learning algorithms scan it to find patterns and generate forecasts, or determine what exactly the person who posted the comment wanted to convey. Advanced machine learning algorithms can also be trained to uncover any hidden subtext within each piece of natural language text.
Training an algorithm to identify the emotions behind a piece of text is a challenging task for AI researchers and analysts. Essentially, there are a few cues that are sought—such as text written in all caps, exclamation marks, emoticons—by an algorithm to gauge the “mood” of a comment. The sentiment of a text can be classified into three categories: positive, negative and neutral.
The process of model training in AI-enabled sentiment analysis systems involves the usage of thousands of text-based input datasets containing the above-mentioned cues and many more that are indicative of different moods. The algorithms in sentiment analysis systems gradually learn how to classify the text after going through several representative samples from each sentiment category.
For government-based sentiment analysis, machine learning and NLP’s capabilities can be utilized in the following use cases:
a) Information Extraction
Most of the information present on the internet is in the form of unstructured and incoherent text. The NLP tools in AI-enabled sentiment analysis systems are employed to find coherence and meaningful data from the text. Once that valuable information is extracted from online forums or other sources, it would help to boost the speed and accuracy of government services. Here's an example of NLP's information extraction prowess—police reports created in the immediate aftermath of a crime often feature poorly written or poorly phrased words and sentences. Such reports can be highly incoherent due to the sheer haste with which they were created. NLP can scan through such statements and determine crucial information such as the crime site, the weapon used, the names of people on the crime scene when it took place, and much more.
Although this example is not related to sentiment analysis, it paints a clear picture of how NLP can extract relevant information from even the most incoherent text.
b) Text Categorization
The complaints and grievances shared by general citizens on public forums, blogs or other online platforms may or may not contain clear categorization regarding the type of public department that would deal with their problem. A well-trained NLP or machine learning algorithm can identify and sort such text as per pre-defined taxonomies. So, any complaint regarding a certain issue—say, water contamination—will be identified by those tools before the concerned public body—such as the water supply board or the nearest health department—is notified about it.
c) Social Media Mining
Twitter, Facebook and other social media sites are some of the most common platforms for people to share their opinions on a wide variety of topics. So, naturally, governments can carry out data mining on such platforms to get a clear idea of their citizens’ real-time problems and complaints. This particular use case of NLP and machine learning can be useful in smart cities and other advanced urban jungles where microblogging sites like Twitter and applications such as Facebook are more commonly used.
One of the first things in social media data mining is to detect and separate racist, sexist or abusive posts from the other ones. This is done because such elements are generally found in tweets or posts from fake accounts or trolls. Such posts need to be classified into their own categories, and the other types of posts will be used for sentiment analysis.
Once the “cleaning” is complete, the posts need to be preprocessed before they are converted into machine language. Preprocessing is carried out to remove the noisy elements from social media posts and reduce their clutter so that vital information can be found in them more quickly. The preprocessing stage involves taking numeric features from social media text.
So, elements such as the “@” sign, incomprehensible words, punctuations, numbers and removable word repetitions are eliminated before the text is processed. This is followed by visualization and story generation to attain insights from the text.
Factors such as hashtags are analyzed and their relation to the tweet or post accompanying them is carefully assessed. As we know, trending hashtags have value if the post they accompany talks about a certain topic related to the trend. After classifying the posts, the necessary public department responsible for dealing with the issue is notified about it. The data mined from social media can be way more useful for such government departments rather than the information sourced from more utilitarian sources. Additionally, the data is collected faster and is more accurate. Hence, AI-enabled sentiment analysis is a highly effective option to understand public grievances from social media posts and other online sources before public bodies can address them.
Advantages of Using AI-Enabled Sentiment Analysis for Public Grievance Redressal
Several countries, like the U.S., currently involve AI in multiple federal government actions and operations. In the future, governments are expected to use AI, NLP and other modern tools to determine the sentiments of their citizens and act on them or resolve them. Additionally, the citizens will be able to send their grievances and complaints via multiple dedicated channels (social media posts, WhatsApp texts) to specialized public sentiment analysis teams in public offices. Two of the main benefits of using AI-enabled sentiment analysis to solve public grievances are:
a) Reduction of Corruption
Ideally, the sentiment analytics department in government offices should not involve too many political figures, just a dashboard and a few other screens on which information is collected and displayed for the various health experts, engineers, administrators and others in the most simple and comprehensible way possible.
The web-based dashboard displays an overview of complaints classified into different categories and also the resolution status of each complaint. The involvement of AI and NLP in public grievance detection and resolution reduces the need for government cronies and other shady middlemen to be present in such offices for the purpose. This automatically reduces the possibility of corruption, favoritism and bribery from the process to a great extent.
b) Increased Transparency
Complainants are constantly updated about the status of their complaints, or how long it would take for them to get resolved. Therefore, unlike regular public redressal procedures that may take days to get heard and resolved, AI-enabled sentiment analysis-based grievance redressal is more visible and trustworthy.
Disadvantages of Using an AI-based Approach
As stated at the beginning, the internet is the main source of information for sentiment analysis to flourish in the real world. Additionally, social media platforms across the board offer the largest share of information for AI-enabled sentiment analysis. Unfortunately, around 40% of the world’s population has no access to the internet. What’s worse, “only” half the world’s population has their own social media accounts, which leaves the other half out of the sentiment analysis coverage area. Additionally, smart cities are currently—and for the foreseeable future—very few in number, even in the wealthiest countries.
AI-enabled sentiment analysis seems like an idealistic dream, at least for a large majority of countries and people around the world. The over-reliance on smart city tech and social media platforms for attaining information is a problematic aspect of this idea. For this scheme to be successful, researchers and developers in the field of AI will have to come up with truly ingenious solutions.
Until then, the concept of AI-enabled sentiment analysis remains just that—a concept.