Data’s place in understanding our world and ourselves

We humans are predisposed to find patterns in the world around us. Our comprehension of the world we live in is fundamentally our understanding of how things interact with each other. Our minds observe repeating patterns in the world, from which the idea of cause and effect is born. This is the conclusion that our minds form when observing events that always occur one after the other. It’s this facet of our minds which enables us to form our judgements, predict outcomes and inform our interactions in the world accordingly.

The whole of scientific discourse is founded on the principle of finding cause and effect relationships within observable events. It’s from this that we posit theories and attempt how the world works. As a rudimentary example, the first person to heat water to 100°C would not have known that it would turn to steam. But through recreating the conditions and repeating the experiment several times, a causal link could be formed. Every time water is heated to 100°C, it evapourates; thus the conclusion can be drawn that heating the water to 100°C causes it to boil.


It’s from our understanding of cause and effect that we have developed complex theories of how the universe works. And it’s also from this understanding that we find patterns in seemingly unrelated events. We can be tricked into seeing faces in nature; we find patterns in meaningless noise.

Our quest to understand more complex patterns is the reason why science reduces objects, variables and events to data. The more data there is to analyse, the more detailed and reliable our predictions can be. Through science, we’ve reduced objects in the world to data; and we’re now reducing people to data too, in an attempt to understand ourselves.


We humans have always been creatures of habit. We have always had our rituals and routines. And it just so happens that our world now is characterised by digital data: our physical lives are mirrored in a digital realm. Everything we do leaves a trace of data behind. From purchasing goods to updating our Facebook statuses, our everyday actions are captured by monolithic computers designed to find patterns. Think about the data we sign away when we accept terms and conditions documents, the record of credit card transactions we leave, or the history of websites we visit. Every time you use a loyalty card, click a link in an email or watch Netflix, a machine somewhere is logging your actions. It’s collecting data that can be observed to find correlating events; it’s data that can be studied to find cause and effect relationships; it’s data that can be analysed to understand who we are.

In 2012, Andrew Pole, a statistician at US mega-retailer, Target, ran tests which analysed the purchasing habits of its customers. Target has what it calls its internal Guest ID system; it’s log of information on every individual customer ranging from what they buy, to their demographic, to their financial history. Pole sought out to find correlation amongst this noise. And what he found was that while people purchase different things at different times, they all form patterns. People’s purchasing histories form patterns so strong that he was able to spot the onset the second trimester of pregnancy in women to a 98% degree of accuracy. He discovered details of the private lives of these women simply based on what they bought, sometimes even before they knew it themselves.

Screen Shot 2014-02-22 at 22.44.19

This kind of insight is big business. Marketers have been quick to take advantage of the information that can be gathered from the data that people leave behind. In the case of Target, marketers bombarded the would-be mothers with offers of baby goods in the lead up to childbirth, armed with the knowledge that if they bought from Target before birth, they were far more likely to be loyal customers after. This ability to analyse data and act accordingly is what is known in the industry as ‘big data analytics‘; it’s a science of human behaviour.

Big data analytics is helping companies to understand people; the sheer amount of data we generate means that we are reducing ourselves to variables and objects for experimentation. Vast oceans of our data exist in machines across the globe, rich with information on who we are and what we do. So now, in the same way that science strives to collect as much data as possible in order to understand the natural world, companies are now collecting as much data as possible to understand us.

So the next time you accept online terms and conditions, the next time you register for a loyalty card, the next time you update your Facebook or do anything online, remember that you’re giving yourself up for analysis. It sounds sinister, but after all, we humans are predisposed to find patterns in the world around us.

Further Reading:

NY Times: How companies learn your secrets –


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s