Conducting sentiment analysis on a broad topic: Analysing Brexit sentiment 18-months after the EU referendum

The vast scale of text content available from social media platforms provides rich opportunity for research and commercial analysis, but is only feasible using programmatic approaches. For a targeted topic such as the name of a person or product, sentiment analysis can relatively effectively produce meaningful insights but for broader topics there are significant challenges to understanding the meaning and nuance of opinion in freeform text content.

I found the opportunity to investigate sentiment analysis to be a highly interesting topic as it was an applied use of technology that felt closer to what I do professionally. The free tools we used for the analysis were rudimentary by modern software standards, though there are commercial alternatives that provide richer functionality and user experience.

I had expected the insights generated from sentiment analysis to be more conclusive – even for subjective and nuanced discussion topics such as Brexit. Relying on a static lexical approach to deduce meaning requires significant investment and subject matter expertise – both in the topic and the understanding of language patterns – and will surely be improved by a more dynamic approach to extracting meaning from text patterns more generally.

As I reflect in 2023 on the assignment, the 6 years since I wrote the sentiment analysis assignment has seen vast developments in Artificial Intelligence (AI) and the ability to programmatically make sense of content meaning and relationships. I would be interested to explore how advances in AI have affected automated context analysis tools such as sentiment analysis, especially in commercial software, and how this could be used in real-time to inform decisions.