Sentiment Analysis

Introduction

Sentiment analysis (opinion mining) determines writer's opinion, emotion, attitude toward topic. Simple: positive/negative/neutral classification. Complex: fine-grained (1-5 stars), aspect-specific (restaurant review: food positive, service negative), emotion classification (joy, anger, sadness, surprise).

Applications: customer feedback analysis (product reviews, support tickets), brand monitoring (social media sentiment), market research, political analysis (election sentiment), content recommendation. Ubiquitous: reviews on Amazon/TripAdvisor, social media monitoring (Twitter sentiment), financial news analysis.

Challenges: sarcasm ("This movie is great!" when it's terrible), negation ("not good" vs. "not bad"), context-dependence (same word different sentiment in different contexts), domain variation (medical text vs. product reviews).

"Sentiment analysis reveals the unseen opinions in unstructured text,the voice of customer, citizen, patient,at scale impossible for humans alone." -- Bing Liu, Sentiment Analysis pioneer

Task Definition and Variations

Binary Classification

Positive vs. negative. Simplest setting. Example: movie reviews (thumbs up/down). Works for polarized domains. Discards neutral/mixed opinions.

Multi-Class Classification

Positive, negative, neutral, or 5-class (1-5 stars). Richer than binary. Neutral class captures objective statements ("The movie has 120 minutes duration").

Rating Prediction

Predict star rating (1-5) from review text. Regression or ordinal classification. Movie review text → 4.2 stars. More information than binary.

Aspect-Based Sentiment

Identify aspects (features) and associated sentiment. "The food was delicious but the service was slow." Food: positive, service: negative. Enables fine-grained analysis.

Emotion Detection

Classify emotions: joy, anger, fear, sadness, surprise, disgust. Richer than sentiment (positive/negative), psychologically motivated. Example: Twitter emotion classification for crisis response.

Intent Detection

Determine underlying intent beyond sentiment. "I want to return this product" (negative sentiment) + intent (return request). Useful for customer service routing.

Sentiment Levels and Document Structure

Document-Level Sentiment

Overall sentiment of entire document. "The movie was great overall, though the ending disappointed me." → mostly positive. Aggregates sentence-level sentiments.

Sentence-Level Sentiment

Sentiment of individual sentence. "The food was good" (positive), "Service was slow" (negative). Document aggregates sentences. Mid-level granularity.

Aspect/Target-Level Sentiment

"The restaurant's food is excellent, but prices are too high." Food: positive, prices: negative. Toward specific entities (attributes, features). Most useful for feedback analysis.

Opinion Extraction

Identify opinion holders (who), targets (what), expressions (how they feel). "Alice says the movie is terrible." Opinion holder: Alice, target: movie, expression: terrible. Structured representation enables reasoning.

Lexicon-Based Methods

Sentiment Lexicon

Dictionary of words with associated sentiment: "excellent" +2 (very positive), "good" +1, "bad" -1, "terrible" -2. Score text by summing word scores.

Example: "The food is excellent and the service is bad"excellent: +2good: +1bad: -1Score: 2 + (-1) = +1 (overall positive)

Lexicon Sources

• Manual: Humans annotate words (expensive, accurate). SentiWordNet, LIWC, OpinionFinder.
• Automatic: Derived from corpora (word co-occurrence with seed words). Cheap, noisy.
• Hybrid: Manual seed set, automatically expand (synonyms, word2vec neighbors).

Handling Modifiers and Negation

Negation flips sentiment: "not bad" opposite of "bad". Intensifiers amplify: "very good" > "good". Diminishers reduce: "somewhat good" < "good".

Negation window: words in N-word window after negation flipped"not [happy] at all" → happy score flippedIntensifiers: multiply score by factor (1.5x for "very")"very good": 1 * 1.5 = 1.5 vs. 1 for just "good"

Advantages and Limitations

Feature Engineering for Sentiment

Bag-of-Words

Represent text as word frequency vector. TF-IDF weights: frequent words downweighted (less informative). Simple, effective baseline. Discards word order (syntax matters: "not good" vs. "good not").

N-Grams

Bigrams/trigrams capture phrases: "very good" different from "good" alone. "not good" captures negation. Reduces sparsity (2-gram for "fantastic movie" even if "fantastic" unseen individually).

POS Tags and Syntax

Adjectives/adverbs carry sentiment information. "The film was disappointing" (adjective). Verbs less informative. POS-specific features: count adjectives, their polarity.

Features:- Adjective count and polarity- Negative words count- Intensifier count- Punctuation (! = strong emotion)

Linguistic Features

• Capital letters: "AMAZING" > "amazing" (emphasis).
• Exclamation marks: "Great!" (positive intensity).
• Emoticons: ":)" positive, ":(" negative.
• Hashtags: "#love", "#hate" sentiment signals.
• Repetition: "Soooo good" (prolonged sound, emphasis).

Supervised Classification Approaches

Naive Bayes

Baseline: P(sentiment | words) = P(words | sentiment) * P(sentiment). Assumes word independence (naive). Fast, simple, surprisingly effective for sentiment.

P(positive | "great movie") = P("great"|pos)*P("movie"|pos)*P(pos) / P("great movie")Works despite naive assumption: presence/absence of words predictive

Support Vector Machines (SVM)

Find hyperplane maximizing margin between sentiment classes. Sparse feature space (BoW): linear SVM effective. RBF kernel captures non-linear separability. Standard approach, strong results.

Logistic Regression

Probabilistic linear classifier. P(positive) = sigmoid(w·x + b). Interpretable (feature weights = importance), efficient, competitive with more complex methods for BoW features.

Ensemble Methods

Random Forest, Gradient Boosting combine multiple classifiers. Reduce overfitting. Typically outperform single models. Slower inference.

Performance Comparison

On movie reviews (binary): Naive Bayes ~80%, SVM ~85%, ensemble ~87%, neural ~90%. Improvements incremental with increasing complexity.

Neural Methods

CNN for Sentiment

Convolutional neural networks over text. Filters detect n-gram patterns (e.g., "very good" feature). Max-pooling captures most important features. Fast, competitive accuracy.

Input: word embeddings for sentenceConvolution: filters of width 3, 4, 5 (bigrams, trigrams, 4-grams)Pooling: max value per filter (most salient feature)Classification: fully connected layer → softmax

RNN and LSTM for Sentiment

Process words sequentially. LSTM captures long-range dependencies: "The plot was interesting but the ending was disappointing",ending sentiment modifies overall despite earlier positive. Slower than CNN but context-aware.

Attention Mechanisms

Identify important words for sentiment. Attention weights show which words influenced prediction. "extremely terrible" → attend to "extremely" and "terrible".

Transformer-Based Models

BERT fine-tuned for sentiment achieves state-of-the-art. Bidirectional context, pre-trained on massive corpus. 92-95% accuracy on standard benchmarks. Transfer learning highly effective.

Advantages of Neural Methods

• Automatic Feature Learning: No manual feature engineering.
• Context Awareness: Understand word interactions, negation naturally.
• Transfer Learning: Pre-trained models transfer to sentiment tasks.
• Performance: State-of-the-art results. 90%+ accuracy feasible.

Aspect-Based Sentiment Analysis

Aspect Extraction

Identify opinion targets (aspects): "food", "service", "ambiance" in restaurant reviews. Treated as sequence labeling (BIO tags) or aspect category classification.

"The food was delicious and the service was slow"Aspect extraction (NER-style):food: O B-ASPECTservice: O B-ASPECT

Aspect-Level Sentiment

Classify sentiment toward each aspect independently. Food: positive, service: negative. Models: aspect-specific sentiment, attention over aspects.

End-to-End Aspect-Based Sentiment

Joint extraction and sentiment classification. Single model extracts aspects and predicts sentiment simultaneously. More efficient than pipeline (extract then classify). Emerging approach with neural models.

Applications

Product reviews: identify aspects customers care about, measure satisfaction per aspect. Enables actionable feedback: "Users love design but complain about battery life."

Emotion Detection and Fine-Grained Sentiment

Emotion Classification

Emotions vs. sentiment: sentiment binary/ternary (pos/neg/neutral), emotions multi-class (joy, anger, fear, sadness, surprise, disgust). Psychologically motivated. "I'm so angry!" (anger emotion) = negative sentiment + anger specification.

Emotion Datasets

SemEval emotion dataset: tweets annotated with emotions. Emotions often co-occur (joy + surprise when receiving gift). Multi-label classification: example can have multiple emotions.

Fine-Grained Sentiment

Beyond binary: implicit sentiment ("This book is 500 pages long" - implicitly negative for those wanting quick read). Sarcasm detection ("Great job!" sarcastically = negative). Conditional sentiment (good for novel, bad for thriller).

Challenges

Emotion detection harder than sentiment (more classes, subtle distinctions). Sarcasm detection requires pragmatic understanding (linguistic patterns insufficient). Context-dependence: "small" negative (portion size) vs. positive (file size).

Challenges and Limitations

Sarcasm and Irony

"This movie is a masterpiece" sarcastically means negative. Surface-level analysis fails. Requires understanding intent, background knowledge. Remains unsolved challenge.

Domain Dependency

Model trained on product reviews fails on social media, movie reviews, medical text. Sentiment expressions vary by domain. Cross-domain transfer improves but doesn't eliminate gap.

Negation and Modality

"I don't think this is good" = negative. But negation scope unclear: "I don't think this is good or bad" (unclear sentiment). Conditional: "If you like horror, this is great."

Implicit Sentiment

"The battery lasted 3 hours" implicitly negative (short battery life). Requires domain knowledge. Hard to detect automatically.

Multi-Aspect Mixed Sentiment

Multiple contradictory opinions: "Great story but boring characters." Complex sentiment. Aspect-based methods help but require aspect extraction (additional challenge).

Bias and Fairness

Models reflect training data biases. Stereotypes encoded in sentiment lexicons/classifiers. Gender bias: "opinionated" positive for men, negative for women.

Applications

Customer Feedback Analysis

Automatically process reviews (Amazon, TripAdvisor, Yelp). Identify pain points: "Quality good but delivery slow." Dashboard: visualize sentiment trends, identify improvement areas.

Brand Monitoring

Track sentiment toward brand on social media, news, forums. Alert on negative spikes. "Rival company's tweet critical of us." Enables rapid response.

Market Research and Competitive Analysis

Sentiment toward competitors, products. "Our product rated higher but less reliable than alternative." Quantifies competitive position at scale.

Financial Applications

Stock price prediction from financial news sentiment. High negative sentiment → predict price drop. Imperfect but adds signal.

Political Analysis

Election sentiment from social media, news. Predict election outcomes. Understand voter concerns. Controversial (manipulation risk).

Healthcare

Patient satisfaction from reviews, surveys. Identify service issues. Mental health: detect depression/suicide risk from social media (controversial privacy implications).

References

• Liu, B. "Sentiment Analysis and Opinion Mining." Morgan & Claypool, 2012.
• Pang, B., Lee, L., and Vaithyanathan, S. "Thumbs up? Sentiment Classification using Machine Learning Techniques." EMNLP, 2002.
• Kim, Y. "Convolutional Neural Networks for Sentence Classification." EMNLP, 2014.
• Pontiki, M., Galanis, D., Papageorgiou, H., et al. "SemEval-2016 Task 5: Aspect Based Sentiment Analysis." SemEval Workshop, 2016.
• Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. "BERT: Pre-training of Deep Bidirectional Transformers." NAACL, 2019.