01WhatDid the news move the mood?
The question: how did topics, sentiment and public opinion about COVID-19 vaccines change over time, and which events caused the changes? The focus was on applying existing tools to real Twitter data rather than training new models. The dataset covers tweets about the vaccines used around the world — Pfizer/BioNTech, Moderna, Sinovac, Bharat Biotech, Sputnik V and AstraZeneca — from December 2020 to November 2021.
02HowClean, score, fit.
Tweets were lower-cased and stripped of hashtags, URLs, user handles, extra spaces, special characters, single characters and stop words; empty and duplicate tweets were dropped. NLTK scored the sentiment, and scikit-learn fitted linear and polynomial regressions to the trends.
03FindingsPeaks follow headlines.
- 2,905
- tweets on March 1, after the FDA authorized the J&J vaccine
- 2,151
- tweets on November 3, after the US said 5–11-year-olds could get vaccinated
- 2 : 1
- positive to negative tweets; neutral dominates
Both the number of vaccine tweets and the overall sentiment trend upward. Sentiment moved with events too: negative around January 16, when the CDC said we were in the early peaks of the pandemic, and positive on November 3. Correlation, not causation.
04HonestlyPartly a failed attempt.
The graphs and regressions worked, but the link between sentiment and events was weak: most extreme peaks came from just one or two tweets on that day, or from the same tweet posted by many accounts — like the negative spike on August 13. Cleaning was the real challenge: removing special characters also erased names like Johnson & Johnson, and 65% of the locations fell into “other” because they were written at city, state or country level, or not in English.
Next steps: clean the duplicates and locations properly, analyse sentiment at the state level, and try predicting the local COVID-19 situation from local tweets.
Adapted from my original write-up · more in the links above.