Our Sentiment about Text Analytics and Social Media

Submitted by Seth Redmore on Tue, 2011-03-08 15:43

One of our partners up in the Great White North has taken the leap and is the first to roll out our French language pack. To really sum up what this brings to their customers: Now, an English-only speaker can have access to rigorous metrics from the French language content of interest. That's pretty cool! Link to MediaVantage release

Submitted by Seth Redmore on Tue, 2011-02-22 23:37

Quick quiz - how many of the companies in this article rely on Lexalytics for sentiment analysis technology? Computerworld: Sentiment Analysis Comes of Age Answer: 3/5 technology providers (including ourselves, duh), and 3/6 of the social media monitoring companies. Awesome-tastic. Oh - they're: Endeca Cymfony evolve24/Maritz Radian6 DNA13 (now MediaVantage)

Submitted by Seth Redmore on Tue, 2011-01-18 22:36

Our buddies over at MediaVantage http://www.mediavantage.com have just announced some significant improvements to their already really good service. Included with improvements to their content stream for social media (including both more stuff and better filtering) and an improved search interface; they've launched some stuff that's near and dear to my heart - automated tonality scoring. That would be where our stuff comes in. So, check out MediaVantage, and know that you're using the best sentiment scoring system available. Here's the release Here's a video about the new release

Submitted by Seth Redmore on Thu, 2010-12-23 19:46

I came across the following article today, our last day before the Christmas holiday break. If ever an article deserved a blog post, it's this one. Expert: Roseville Galleria paid no attention to social media for flash mob The article talks about a flash mob that occurred in a mall in Roseville, CA. This resulted in overloading in the food court, some "popping sounds", and the "floor shifting". There is a certain amount of question as to the "purity" of the flash behind this flash mob. (Given that the mall supposedly issued a press release about it 3 days ahead of time, along with other things). The author attempts to make the case that by monitoring social media, the mall could have gotten a handle on how many people were going to come and Do Something About It. Ok, I agree that monitoring for buzz is a useful tool to prepare yourself, and social media is a great way to keep your finger on the pulse. And, yes, if you were able to compare the amount of buzz this was getting compared to, say, some other event that you'd announced, you could plan for extra security and such. However, watching the pulse of social media does NOTHING for you if you aren't prepared in other operational ways to deal with situations. I think the question that should be asked is "Why wasn't the mall equipped to deal with a larger than expected crowd in ways other than 'evacuate everyone'?" I would suggest that they first need to really review their emergency preparedness plans. And perhaps part of a better process is a better listening strategy. I bet they listen to the weather, and I also bet that their parking lots are free of snow for the shoppers. That's because the operational weight is set behind the folks keeping the parking lot open for people to park. They need similar thinking for other aspects of their operations. We can give you really, really great data. We can help you get visibility into areas of your business that you haven't previously seen into. However, *you should have at least some ideas about what you plan to do with the results before engaging in an kind of monitoring/listening program*. Listening should not be a thing in and of itself. Even though listening is important, if all you do is listen, you aren't doing anything about what you're hearing.

Submitted by Seth Redmore on Tue, 2010-09-28 17:46

Nick Halstead has launched a new service called Datasift. Datasift exists to help you filter the twitter firehose, chopping things up along any vector you can think of (and more). For example, you can do simple keyword filtering, but across a polygon that defines a geographical locale around the tweet-er, and only includes positive tweets. And that's where we come in. They integrated Lexalytics' Salience Engine to provide our entity extraction and sentiment analysis functionality. In a matter (literally) of a few days of integration work, they had the power of Salience available for filtering the entire Twitter firehose.  

So, 20 different vectors are available from Salience... These include the ability to automatically extract company/people/product names (without having a list of them ahead of time), the ability to calculate tweet, entity, and "linked-content" sentiment, output lists of positive/negative entities, output quoted content and more. Example Tweet streams: * Movies getting positive reviews. * Coffee shops that are getting negative reviews within 10 miles of me (so that I can go sell them something to help them do better). * People mentioned in positive tweets about the US Government This is a really exciting new service that provides a super-sophisticated way to slice and dice the whole of the Twitter firehose...

Submitted by Seth Redmore on Tue, 2010-09-21 16:32

Salience is roughly split into two parts: the software itself, and the supporting data directories that have dictionaries, models, etc. This post discusses work specific to the data directory itself.

With the use of emoticons, abbreviations, and poor grammar; Micro-blog services, like Twitter, present a difficult task for natural language processing systems. We spent the summer teaching our software how to deal better with such content. We did pretty well before, but the improvements we made make processing this content significantly more useful.

1) Acronyms
We parsed thousands of tweets to get the common acronyms, and then made decisions on whether they were sentiment-bearing, acronyms to be expanded, or to be treated (from a part of speech perspective) as simply an interjection.

LOL: (Laugh Out Loud) does not carry sentiment, nor does expanding it add any value to the resulting lexical processing; so we treat it as an interjection
FTW: (for the win) carries positive sentiment
IDK: (I don't know) is useful when expanded out to its individual words

2) Emoticons
Some are obviously positive or negative, and others we use to push the sentiment towards neutral – not because they’re necessarily inherently neutral, but because they are used in a context that’s probably going to mess up sentiment analysis:

Positive includes:

Negative includes:

And faces like :P are used to push the content towards neutral because of the reasons above.

3) @ sign
We part-of-speech tag @ symbols as /MENTION - so that you can use this for further processing and reporting. In particular, calls for Entities will return @ tagged strings as people entities, with the associated sentiment, themes, etc.

4) # sign
We part-of-speech tag # symbols as /HASHTAG

The combination of all of these allow for much richer reporting on the conversations that are occurring around, about, and between different accounts on Twitter.

We did some other data cleanup work as well, most significantly in the "famous people" data file, adding more people and adding a wikipedia link to that person.

Submitted by Seth Redmore on Wed, 2010-08-11 16:22

In an earlier post, Jeff Catlin described analysis that we did around Bally's vs. Bellagio using publicly available customer reviews. We did this analysis using something called "categories", which is basically a fancy name for search strings. The important aspect of this analysis was in finding the sentiment associated with different important aspects of the hotel experience - using a known set of categories. In other words, the actual terms for each of these categories were defined, and then we determined the sentiment of each category.

This technique is highly useful for ongoing monitoring of known service areas, and provides highly reliable recall of those areas. Where it is less useful is for discovery applications - where you aren't comparing on known areas, but instead trying to find out what's outside of your analysis area. Discovery is inherently a different process from monitoring. Discovery means that you don't know what might or might not be out there. In monitoring applications, it's generally important to have higher accuracy and recall, so that you can correctly monitor trends. In discovery applications, accuracy and recall are paradoxically less important - largely because they're not really relevant concepts. You can't define accuracy or recall for things that you don't know about. Once you define them, then you can examine the statistical rigor of your system. But, I digress.

We took sample data from a publicly available review site and extracted the relevant themes. The following table shows the name of the institution, a sentiment score (for the theme), and the theme itself. It is important to differentiate between the sentiment score for the theme and for the entities resident in the text - consider the following sentence: "President Barak Obama is doing a great job with this awful oil spill." Now, whether you agree with that statement or not, you can see how the entity "President Barak Obama" would get a positive score, while "oil spill" would get a negative score.

Hotel BLOOM! Positive trendy hotel
Hotel BLOOM! Positive been decorated
Hotel BLOOM! Positive not mean
Hotel BLOOM! Positive main shopping
Hotel BLOOM! Positive botanical gardens 
Hotel BLOOM! Positive centrally located
Hotel BLOOM! Negative long flight
Hotel BLOOM! Negative next day
Hotel BLOOM! Negative not sure
Hotel BLOOM! Negative ordinary coffee
Hotel BLOOM! Negative stale smoke
Hotel BLOOM! Negative not require
Hotel BLOOM! Negative been given
Hotel BLOOM! Negative never stay

Some of these themes are clearly more useful than others. It is important to also note that the sentiment of the theme is not necessarily included in the theme itself - it is inferred from the language around the theme itself. If I was coming into this cold, and didn't have anything defined, I would be sure to track themes around "decor" and "location" on the positive side, and "smoking" and "coffee" on the other side. the other themes would potentially mean more when referenced to the text itself, but this is more just to give some quick examples.

Here's another example:

W Seattle Positive well decorated
W Seattle Positive romantic getaway
W Seattle Positive tastefully decorated
W Seattle Positive centrally located
W Seattle Positive even walked
W Seattle Positive top quality
W Seattle Negative wireless connection
W Seattle Negative outrageous prices
W Seattle Negative local taxes
W Seattle Negative even get
W Seattle Negative poorly lit
W Seattle Negative never stay


Again, the location seems to be nice and decor is also well taken care of. On the negative side, I would watch for complaints around communication issues and pricing. However, this is the W, so what they need to watch for (as we demonstrated in the post) was that they're providing value for money. If you're not providing a proportionately better experience relative to the cost difference, guests are going to end up being dissatisfied. There's more detail to how themes are determined to be relevant, but this is enough information to give you a taste of how to use themes to discover what you should be looking for (or looking out for!).

Submitted by Seth Redmore on Tue, 2010-08-10 13:49

For the past year, we've experimented with using a web service to provide low-cost text analytics services, and we've learned a lot doing this. Our core strength as a company is providing fully customizable text analytics that are applicable to any industry, for a wide range of text analytics solutions. Lexascope was designed to fit a single use-case, that of longer form media content - for low cost entry into the media analysis market.

What we've learned is that working to a single use-case is not what fits our customers the best. Our customers desire the fully custom nature of our Salience engine. We've also seen an increasing desire to analyze short-form content, an application for which Lexascope was not designed.

As such, we've decided to focus our engineering resources on coming up with a lighter-weight installed version of Salience, one that has all the power of our current Salience, but allows for more pricing flexibility for users who need to process content using the full powers of our current engine, but don't have the content volumes to justify an enterprise-class license.

We've stopped accepting new registrations for Lexascope, and will work with current Lexascope customers to come up with a solution to their needs. We think that this new solution fits much better with our core strengths as a company, and allows us to focus on our core business of providing the best possible text analytics functions; while simultaneously enabling a larger set of businesses to take advantage of the strengths of our super-flexible, very powerful core software engine.

Submitted by Jeff Catlin on Thu, 2010-07-22 14:21

Hotel Reviews represent one of my favorite uses of text analytics. About five years ago we built a site with FAST that measured hotel reviews to build a “consensus opinion ” of hotels in a narrow geographic area. The idea was to give users of the site (shown below) an idea of what people thought of various hotels in a given area (Manhattan for example). It’s a nice application because it plays to the strengths of sentiment scoring, where a group of reviews are rolled together to form a concensus opinion. Automated engines are very accurate in such a use case (possibly more accurate than people), and they can handle a large volume of content.  

Recently we revisited the scoring of hotel reviews, and dove a bit deeper this time. Rather than simply generating a score for each property we scored the reviews for various features of the hotel, like location and staff and dining. For this test we used reviews for a couple of hotels in Las Vegas, the Bellagio and Bally’s and we measured the following features for each: - Rooms - Price - Facilities - Location - Cleanliness - Service - Overall An important aspect of this analysis is that the hotels are basically in the same location (right across the street from each other). When you examine the results (below), you’ll see that the hotels scored nearlythe same on location. This is a good test that the results are indicative of reality.


Digging deeper into the results, I was surprised to see that Bally’s had higher scores than Bellagio because Bellagio is one of the 5-star properties in Vegas, so we dug a bit deeper to make sure we weren’t scoring the reviews wrong. We focused in on the most positive and most negative reviews and tried to figure out why Bellagio wasn’t scoring higher. The chart below shows that the “happy campers” were equally happy with Bellagio and Bally’s the unhappy visitors were really unhappy with the Bellagio.

When we dug into the reviews we discovered that people expected more for their money than they were getting at Bellagio. Through the simple application of sentiment analysis on publicly available information, we show that companies can make these comparisons with much higher reliability, at minimal incremental cost, and with an unprecedented ability to adjust categories on-the-fly, either based on these results, or to test out new hypotheses. In fact, using this technique, we can move beyond the limitations of traditional approaches by running additional analysis to discover new, previously unmeasured categories based on recurring themes within the data. What this means for brands is that those who are able to leverage sentiment analysis will remain at significant advantage over their competitors, able to anticipate and proactively respond to how customers perceive their brand, much faster, more comprehensively, and significantly cheaper than existing methods.

Submitted by Seth Redmore on Mon, 2010-06-28 17:57

One of the two major new features in Salience 4.3 (releasing around June 30th) is "opinion mining". Opinion mining expands our core technology to handle indirect quotes. We've been able to extract quote-mark delimited quotes for a while now, and you could perform further analysis on those quotes (which were attached to the speaker). Opinion mining means that Salience 4.3 can now handle sentences like: 1) Seth then asserted that this was a truly awesome feature. 2) Tim agreed that Bill was unduly angry. 3) Paul explained that the code was broken. In each case there is a speaker, a topic, and sentiment expressed. The "speaker" is always an entity - and it could be a place, person, or company. The topic can be either a theme or an entity. Sentiment is assigned to the topic. Thus, in sentence 1: Speaker: Seth Topic: awesome feature Sentiment: positive Sentence 2: Speaker: Tim Topic: Bill Sentiment: negative Sentence 3: Speaker: Paul Topic: code Sentiment: negative How does this work? I'm glad you asked. We have a data directory full of patterns for opinions. These basically come down to the following 3 classes: 1) "attributed" opinions (e.g. Paul said "This is great") 2) "cross sentence" opinions ("This is Great." Said Paul) 3) "unattributed" opinions (the examples above) Unattributed uses a list of verbs that are expected to express an opinion, and looks for certain patterns using those. There are roughly 200 verbs that clearly express opinion (acknowledge, accuse, add, admit, advise, affirm, allege, answer...) and roughly 200 that have additional requirements because they indicate opinion only in certain contexts: (accept, account for, address, agree, allow, analyze...). To give an example of how this works, consider the following: "Paul charged at George." vs. "Paul charged that George was incompetent." The "that" in the second sentence changes "charged" from being an action to indicating an opinion. For those who update to 4.3, check the data directory: /data/opinions/*.ptn for all the patterns. Try it out... we think that your opinion on this will be positive.