What do you do with a ton of text data?

Pop quiz, hotshot. You’ve been given a big stack of unstructured text data - could be survey responses, social data, staff appraisals. You have to pull insights from this data, and explain to your organisation what’s going on.

What do you do?

Let's explore the most popular options, and why some of them are more popular than they should be...

Getting a human being to tag data

A common strategy. We love humans. Some of our best friends are humans. And to be clear, when you have a small amount of data, they’re absolutely the best option.

But if you have a lot of data, this strategy runs out of steam real quick. Apart from the obvious drawbacks (money, time, the fact that you’re landing your team with the most tedious task imaginable), there are also some subtle ones (the natural human traits of bias, inconsistency and h-anger).

Picking out individual posts

Again, very common. When we’re putting together a report at Wordnerds, picking out a couple of individual posts is great for underlining the narrative, and adding a bit of colour to your data.

But if that’s all you’re doing with text, then you’ll hit limitations pretty quickly. Not least: there’s no way of understanding the relative size of issues, how they’re changing over time, and how people feel about it as a whole.

In a world where big bosses need numbers to back up their decision, a one-off post from hotguy2739 will only get you so far.

Using the text insights from a major platform

If you have access to a big social listening or CRM platform, you probably trust their insights on your numerical data (volumes, engagements, influence etc).  And they offer text analytics (kind of), so why not use that for reporting?

In the most basic of reporting, this may well be what you need. But insights teams generally need more. And because they’re so good with numbers, the big platforms tend to treat words in the same way. And language, in all its sarcastic, colloquial, misspelled glory, requires a different view that you can only get by specialising. This is why almost all our customers use Wordnerds alongside a big quan platform.

Training your own neural network

Large organisations have a wide range of skills. You may have a technical whizzkid on staff who has looked into the big neural networks and would like to train one for you.

Should you go with them?

Training your own neural network is a bit like sewing your own clothes. It’s definitely possible, and for some people, it may even be the best option. But it’s extremely time-consuming, and you can easily end up with something a bit embarrassing.

This has been our only job for years. We’ve made all kinds of mistakes and misfires, on our way to building a platform that can read and understand text, and provide insights out of the box.

If you decide this is for you, then do reach out to us, we’d be delighted to help. But for most organisations, homemade NLP isn’t worth the stress.

Creating a word cloud

Look, we get it. Word clouds are pretty. They add a dash of colour to your reporting. They give you something visual to show off. If those are your criteria, may we recommend this photo of our team’s new puppy, Pip.

It does everything on this list and gives you exactly as much insight into your data as a word cloud.

The trouble with word clouds is that they work by completely removing words from their context. But words are their context. There is basically no word in the English language that always means the same thing and hundreds of ways of expressing each and every idea.

Counting the words will never give you the insight you need.

Doing nothing

Believe it or not, this is, in our experience, the most popular strategy of all. Text data tends to just sit around large organisations, gathering dust. The fact that you're even looking at text analysis puts you in a select group of professionals who care about extracting value from text data.

Could you do more with it? Are you ready for the next step? 

Don't worry if you aren't sure. Here's a wee hand for you - see where you sit on The Qualitative Data Journey.