Sentiment Extraction: Measuring the Emotional Tone of Text

Overview

Lexalytics shipped the very first commercial sentiment analysis engine in 2004.   Over the past 10 years, we've been adding lots and lots of features for enterprise-class quality, ease-of-use and customizability.  From firsts like "emoji" smiley and "emoticon" support :), to other firsts like the ability to detect sentiment in hashtags like #ilovelexalytics, through complete control of negation, intensification and detection of buzz vs. actual experiences, Lexalytics continues to be on the forefront of sentiment analysis technology.

technical info

Details

Sentiment Extraction: Measuring the Emotional Tone of Text

Sentiment scoring is emotion!  Is the text positive or negative?  Good or bad?  How will you know if you don't look?  How can you look by hand if it's millions of comments per hour?

The obvious limitations of manual sentiment scoring and sentiment analysis led to the development of machine scoring. Once you have reliable, consistent machine-based sentiment scoring, there are a number of easy applications; reputation management (the problem every marketing person faces), “voice of customer” (listen to how they’re saying what they say, don’t constrain them to closed-ended questions), eDiscovery (was there a wave of negative emails before a certain crisis hit?), etc…

Lexalytics has provided text mining and sentiment analysis for over nine years now.  As such, we have a substantial amount of experience and know where it works well.  Unfortunately, many vendors haven’t been doing it as long as we have, so you need to be wary of over-reaching claims and confusing descriptions by the vendors rolling out early sentiment analysis engines.

These pages will demystify sentiment scoring and explain how the Lexalytics sentiment analysis engine works.  This includes a discussion of how and why we have extended the basic concept of document sentiment to the paragraph and entity level, and how this technology is being further extended to measure other indicators within content, including the assessment of threat, customer satisfaction and many other contextual indicators. 

How does Sentiment Scoring Work?

Humans seemingly have no trouble reading a sentence and mentally scoring the sentiment.  We humans use a process of reading and understanding the descriptors placed on the subject of a sentence.

Consider these sentences:

  • A horrible pitching performance resulted in another devastating loss.
  • Sub-par pitching and superb hitting combined to cost us another close game. 

They both have the same basic subject, the loss of a baseball game, but obviously (to you!) the first sentence is contextually much more negative. The keys humans that humans use to discern this are to focus on the emotive phrases "horrible pitching" and "devastating loss".  The sentiment system developed by Lexalytics does exactly the same thing.

Our software identifies the emotive phrases within a document and then scores these phrases (roughly -1 to +1) and then combines them to discern the overall sentiment of the sentence.  Importantly, our sentiment scoring will score those sentences the same every time they’re exposed to them – our software is not affected by whether or not it has had it’s morning coffee, or whether it is a fan of the team that lost (or that won!).  That’s consistency, and that’s important.

Part of Speech (PoS) Tagging and Sentiment Bearing Phrases

The first step in determining the tone of a document is to break the document into its basic parts of speech (POS tagging). POS tagging is a mature technology that identifies all the structural elements of a document or sentence, including verbs, nouns, adjectives, adverbs, etc.

We use well defined, well understood techniques that generate extremely high accuracy for tagging the various Parts of Speech.

Even though you don’t really realize you’re doing it, to determine the sentiment of a document, you’re mentally identifying the parts of speech within a document that indicate emotion. In most cases these are adjective-noun combinations like "horrible pitching" and "devastating loss". That’s how our software works, too. These combinations are called “sentiment-bearing phrases”.

The difference, of course, is that our software needs to actually assign a number to the sentiment – as opposed to you, when you’re reading it, just think “darn, they lost again”. What we’ve done is create a very, very large dictionary of sentiment bearing phrases and their relative scores. These scores are pre-determined by how frequently a given phrase occurs near a set of known good words (e.g. good, wonderful, spectacular) and a set of bad words (e.g. bad, horrible, awful).

Calculating Phrase Sentiment and Applying to a Document

We used an extremely large corpus of text (the web, via an internet search engine) to evaluate the nearness of known good and bad words to the phrase being considered. Consider the case of the phrase "devastating loss". It means something to you because you’ve come to associate that with something bad. That’s exactly the process that we go through – we check to see if we should associate that phrase with positive or negative sentiment, and just how closely we should associate it.

Thus we use search engine queries as so:

"(devastating loss) near (good wonderful, spectacular)"

"(devastating loss) near (bad, horrible, awful)"

Each query comes back with a hit count. These hit counts from are combined using a mathematical operation called a “log odds ratio” to determine the score for a given phrase. In this case the phrase "devastating loss" yields a phrase score of negative 0.56.  (Note that all of this operation has already occurred - this does not occur in real time for each document - it's simply explaining the basics of how we set up our dictionary of sentiment bearing phrases).

Here’s how this would work in a document:

Dan Wells joins Nautilus Footwear, America's fastest growing safety shoe company, as Vice President Nursing Division. "We have ended 2000 with record sales and earnings. Phase 3 of our strategic plan is to clearly separate our three business units of industrial, nursing and public safety. Dan will lead our Nursing strategy and drive the efforts of our nursing sales team. The clear separation of our Nursing business is a natural for us as we have been a leader in the nursing arena since beginning the company in 1996," said Wayne Easley, President / CEO. "Dan comes to us with an extensive background in the footwear industry most recently with Berkshire Hathaway's, Lowell Shoe Company. He has solid account and product knowledge that can clearly continue to deliver our 'ergonomic message' to the Nursing community," added Elsey.

The green phrases are the sentiment bearing phrases in the document (these all happen to be positive sentiment bearing phrases):

Sentiment Bearing Phrases

Whole Document Sentiment Scoring

We have an algorithm that we use to combine the phrase scores in the document based on an operation called “lexical chaining”. This operation is beyond the scope of this document, but is similarly consistent and repeatable. The overall document sentiment for this document comes out to 0.365 (mildly positive).

Named Entity and Theme Sentiment Analysis

Except for press releases, it’s rare to find a document that is  homogenously positive or negative. Sentiment is typically a localized phenomenon that is more appropriately computed at the paragraph, sentence or named entity level. Consider the following example:

"Julie Jones superb performance in the gubernatorial debate has all but assured her of victory in the upcoming elections. Unfortunately, the evening did not go as well for her opponent John Adams. Mr. Adams nervous and uncertain performance has put his entire political future into question."

The sentiment of this sentence is completely different for the two individuals described within, while the overall sentiment for the sentence averages out to roughly neutral.

Let’s look at the results of this snippet at the overall level and at the named entity level (red are negative sentiment bearing phrases, green are positive):

Sentiment Tagged Text

Sentiment Tagged Text

The tagged text yields a document sentiment of 0.11 which is squarely in the middle of the neutral range, and doesn’t really tell us anything about the real tone of this snippet.

Now let’s examine the named entity level sentiments for each person, and start by identifying every instance where the person is mentioned. Again, how we determine just what constitutes an entity is somewhat beyond this document, but isn’t important for understanding how we assign sentiment:

Entity tagged text

Entity Tagged Text

In the above-tagged text, notice that the system identifies not only the people by name, but also identifies the pronouns “her” and “his” (and will correctly associate the “her” with Julie Jones – an operation called “pronominal co-referencing”. Correctly including the pronouns significantly improves our software’s ability to measure the tone for each individual. Running this block through and computing sentiment analysis for each entity yields:

Julie Jones: Positive (+)0.22
John Adams: Negative (-)-0.11

This capability sets our core sentiment analysis software apart from other semantic analysis and text analysis tools, and enables our software to focus on the sentiment or tone of specific people, companies or products. The true value of sentiment analysis is in applying the measure to the people or products you’re concerned about. For most rich content sources, it’s much more important to measure and compare the sentiment of an individual named entity than it is to get the overall sentiment of the document. (Consider the case of a product review article, where one product gets panned and the other doesn’t – if you can’t discern which was which, then it doesn’t do you any good).

How we do Sentiment Analysis -- Conclusion

The determination of Sentiment or another indicator is another step in the process of converting unstructured content to structured content, so that the humans who interact with this ever-expanding sea of information can spot trends and patterns within the content.

The last few years have seen significant enhancements in the creation of metadata from content, including the extraction of entities (People, Places, Companies, and Products), key noun phrases, and subject classification. The Lexalytics Salience Engine has 9 years of experience in teaching computers how to consistently and reliably measure emotion, and more importantly, how to appropriately ascertain just to whom in the text that sentiment is directed.