Is It Time To Reconsider Sentiment Scoring?

Definitely there’s a lot of very weak sentiment scoring going on out there, and when it’s not weak, it’s simplistic, reduction to a few summary numbers an analysis that should happen at the “feature” rather than the document level. Good sentiment analysis applies sophisticated natural language processing in order to resolve sentiment, including emotion, at that more granular level: Not, Did person X have a favorable opinion in a hotel review, but rather, What did X think of the cleanliness, service, location, prices, and so on, each separately. Call this semantic analysis if you wish: Many better tools do it as part of their sentiment analysis tools or services.

Seth

(Chair, Sentiment Analysis Symposium, http://sentimentsymposium.com)

In the sentiment reporting/tracking I do with my clients, we manually audit the reports each month, adjusting the sentiment and deleting the irrelevant posts. And we do separate analysis for review sites vs. Twitter and other platforms and track both the sentiment and issue of each consumer review. It can be a lot of work so we try and do a little each week to keep on top of it. It’s an inexact science, but it helps keep a finger on the pulse of what’s being said out there.

You’re right. Scoring a post for sentiment does not work in general. But that doesn’t mean we should abandon semantic analysis of what people are writing online; it just means we have to do it in a way that provides useful information. One way to do this is using strict rule-based analysis for specific products or categories. It takes a lot of customization work up front but it means you can achieve highly accurate (yep, I used that word) results for a narrow industry or product category. For instance, looking at the auto industry, you can find out how many people like or dislike a specific element, option or feature on a specific car. That sort of business intelligence is useful for marketing, customer service, product planning and a number of other areas.

Hey Paul, your link to The Problem with Automated Sentiment Analysis is dead. Is this the story you meant? http://www.freshnetworks.com/blog/2010/05/the-problem-with-automated-sentiment-analysis/

Doesn’t the fact that the majority of Social Media Monitoring services included in that post use Keyword-based as opposed to NLP-based tech cloud the water a bit? I’d be more interested in seeing their reaction to some of the smaller players in the space like General Sentiment or Crimson Hexagon (spoiler alert: I used to work at GS). There’s no question that automated sentiment scoring doesn’t work when you’re using keyword based systems, but, given my own personal experience, I think some of the smaller NLP outfits out there are doing a pretty good job. My two cents, in any case.