Your Data Is Big But Is It Wide?

Big data is a term used to define information collected by companies in structured, semi-structured, and unstructured formats to be used for data mining and machine learning at a large scale. It is neatly classified using the 3V's; volume, variety, and velocity.

The volume of big data used in many environments is so large that it cannot be captured and processed efficiently by any single user, so it is handled by teams of users or via automation. The variety of big data comes from the many different sources for information, such as social media, news articles, video, audio, and text. Velocity can refer to how fast the data is handled and/or the speed with which big data is generated – it is sourced from many points at the same time or from aggregator sites.

With large volumes of data coming in at a high velocity and with ever-expanding variety, nuance is lost in data sets. This is where the data's width can be expanded by including qualitative data to complement it and improve the level of insight. This is sometimes called thickening the data.

Thick data is generated through primary and secondary research in the form of surveys, focus groups, interviews, questionnaires. It is gathered by ethnographers, anthropologists, and other experts at observing and analysing human behaviour and its underlying motivations. This enables them to reveal people’s emotional and thinking patterns, which can then be used to predict which products they are most likely to buy, what price they will pay, and so forth. Thick data is qualitative where big data is quantitative.

Lego: An Example of Thick Data at Work

Toy giant, Lego was on the verge of going burst in the early 2000s when it decided to conduct a major qualitative research project, studying children across five major global cities. Lego set out to understand why the emotional needs of children at play weren’t being met by its products.

Studying hours of video recordings of children playing with the Lego bricks and toys revealed that children were more passionate about the creative experience of play that engages their imagination than the instant gratification of toys like action figures. With this thick data, Lego abandoned action figures and went back to basics, marketing its traditional building blocks instead.

Benefits of Wide Data

Targeted Advertising

This has come to the forefront with usage of big data. Most companies are collecting data on how users navigate the internet and social media to infer their interests and distribute relevant ads based on this information. Gone are the days of spamming everyone on a given network with one type of ad. Targeting adds based on gathered data creates an increased opportunity for sales conversion due to buyer persona fit. 

Product Price Optimization

This is done by analysing data from customers and the market to determine the most effective pricing regime for your products that allows your business to meet its objectives i.e. increased profit margins, customer growth, or a blend of the two. This requires big data on customer trends and company costs, and furthermore, gathering qualitative or thick data that is specific to regions or ethnicities can help your business execute a more granular pricing regime.

Real-World Use Cases for Widening and Thickening Big Data

There are emerging terms and repurposed methodologies for enhancing the insight and power of big data as it stands today. Some of them are discussed below.

Wide Data Means More Inclusion.

One challenge that big data has faced over the last few years is the narrow definition of data parameters to be warehoused for a particular use case. This is due to some data types being inaccessible or challenging to use in certain use cases.

For example, a New York City non-profit was able to increase its effectiveness in helping homeless people by widening the scope of data that it collected. Initially, it only focused on new eviction cases but moved to predicting possible evictions through client targeting techniques that included the education level and experience with the shelter system. 

Thicken the Data and Get More Insight

Wide and thick data have helped companies like Netflix and Samsung better understand why their customer base is behaving in ways that the big data shows. Thick data allows companies to craft better product designs, marketing strategies, and ultimately increase their margins.

Netflix used qualitative data to understand that users were more prone to binge-watch shows, which helped to prove its model of delivering whole seasons at a time, unlike other services offered one episode a week. Samsung’s thick data revealed that customers viewed TVs not as electronics but as furniture, so designs were updated with this in mind.

Prevent Data Lakes from Turning into Data Swamps

However, there is an alternative challenge with over widening parameter scope, and that is the concept of data lakes turning into data swamps. A data lake is a very deep reservoir of data that is collected in an unstructured form only becomes structured at query time.

The challenge such data reservoirs face is that if the data is too jumbled (lacking metadata) for queries to make sense out of, it becomes a data swamp. Wide parameters can cause data lakes to turn into swamps if the use cases and connections between parameters are not well defined.

Is your big data wide enough?

Let the experts at ASB Resources guide you the infrastructure, systems and processes that your company needs to make the most of big data. Schedule a call with one of our experts today!