by Petr Hájek, Information Management Advisor
What is intuition?
The old ‘wisdom’ says that managers spend their budgets on data and analytics, but finally, they make decisions based mostly on their intuition.
Intuition can be explained in many ways – some say that intuition is irrational as opposed to our otherwise rational way of thinking. Another explanation is that intuition is a product of our unconscious mind being served to our consciousness, which ultimately makes the decisions. Others suggest that intuition is something transcendental, coming to us from the outside. A more simple definition says that intuition is a knowledge acquired without understanding (without knowing “why”).
At first sight, it seems that the whole world of algorithms, data, and analytics stands exactly on the opposite side and is in direct contradiction to our mysterious intuition. Maybe this view is far from reality.
Recently, I finished reading a brilliant book by Yuval Noah Harari named Homo Deus and still cannot get it off my mind. It was one of those books that come to you at the exactly right time. They help you name ideas you unconsciously already had in your mind but were not able to rationalize them. (I could say that my intuition hinted at something, which I was not able to comprehend until reading this book. After having done that, I felt content, as for a moment, my intuitive self was in sync with my rational and conscious self…).
Frankly, I have always had doubts about the ability of machines and data to bring something more than just limited help to our day-to-day lives. I have witnessed several times how costly and timely data analyses eventually confirmed an already widely known piece of information.
The Evolution Race
The book, together with its prequel Homo Sapiens, explains a ‘brief history and future’ of humankind. The chosen perspective is the continuous development of our cognitive abilities. At first, our ancestors explained the mysteries of their small local worlds with animism, interpreting even inanimate objects as possessing a soul. Subsequently, humans realized the complexities of their world and turned to deism – the explanation that it must have been intelligently designed by one or more gods. Finally, we have arrived at humanism – the prevailing ideology of the present day. It is based on the belief that we, our lives, and their continuation are the utmost goal of the Universe and that the very depths of our minds are the best sources of answers to any question.
Like many times in history, people have learned that they are not in the centre of the Universe, that the Earth is only a grain of dust in a vast and mostly empty surrounding Space, the Sun does not circulate us but vice versa, etc. Similarly, history is not going to end with humanism. A new ideology has already emerged. The successor of humanism so-called dataism. This new term was coined by David Brooks from the New York Times in his article ‘The Philosophy of Data’.
Harari brings many proofs from the known past that any system (be it a cell, organism, nation, state or a company) gets the evolutionary advantage over other systems, which are not as efficient in data processing and data exchange. He also states that humans (more specifically human brains) have so far been the most efficient data processors in the known Universe and are therefore current leaders in the evolution race. But we have challengers.
Big Data vs. Small Data
Citing Brooks, dataism states that ‘our human intuition lies’. If we humans rely on our intuition, we make wrong decisions way too often. This could be avoided if we rely on data, particularly on big data. The truth is in data. Dataists (if I can call them that) bring a good example of actress Angelina Jolie, who underwent a radical surgery based on statistical knowledge telling her she was in high risk of fatal disease due to her genetic dispositions. Most likely, she felt perfectly healthy, and her intuition was silent, no inner voice kept telling her anything. But she decided rationally; she trusted big data and computer algorithms, which told her doctors that the risk of cancer was enormous in Jolie’s case.
Her intuition was not wrong; it only did not have a chance to process and learn from a larger amount of data in comparison to computers. It was limited to her individual experience, while the statistics were based on big data covering cases of many other women who suffered the disease – the more data, the better the decision. Jolie’s intuition was a closed and limited system. Closed systems lose in competition with open systems (e.g., when Russia lost the Cold War with the West, where people were free and open to new things).
Another example is a hypothetic mother warning her daughter not to marry a certain guy. I do not think these two would be having a rational discussion. Rather, it will be the mother’s intuition against her daughter’s feelings. But let’s imagine that time would show that the mother was right. Her intuition was nothing but learned experience of many similar cases (surely not her own, but from other families, newspapers, literature, etc.). In this example, a mother’s intuition is something like a data processing engine working over big data (many similar cases). Her daughter’s decision was also intuitive but using a smaller amount of data. Thus, according to dataists, the daughter would be wrong in her choice of husband.
I am certain you can find a number of other similar examples yourself.
Subsequently, we are dealing with a shift in our understanding of intuition. According to dataists, intuition does not come from a mysterious source. Human intuition is a big data engine, so far the best one we have in our universe. However, it remains an engine.
Beyond machines
One of the boundary conditions of dataism is that we people are nothing but algorithms.
This is nothing new, even Stephen Hawking in his last (posthumously released) book suggests we all are nothing but machines. Now, the dataist logic is clear: all you need to succeed in evolution race is to have an efficient data processing engine and operate over big data. Humans have the most efficient engine (the efficiency is determined mostly by several elements and unbelievable number of their connections in neural networks of our brains) but operate on rather smaller amounts of data. Present-day computers are not as efficient in data processing (their complexity is far below the complexity of our brains); however, they can process larger amounts of data. Thus, it is only a matter of time for computers to become more complex, connect, and operate over larger amounts of data. If humans integrate with computers in some form of symbiosis, they may continue to lead the race together with computers.
Human intuition seemed to be so mysterious only because we could not consciously cover it. In certain cases, we already struggle to understand how some contemporary computers with well-trained neural networks make their decisions. In simple terms, these computers have built something like an intuition of their own, exactly according to the definition: ‘knowledge without understanding’. There is no mystery behind it, just enormous complexity.
For some people, it may be a scary vision, for others an interesting theory. I like the theory as well – but my ‘gut feeling’ tells me it is missing something…
We all know the following hierarchy: symbols > data > information > knowledge > wisdom. There is no wisdom without learning; learning is acquiring knowledge through the information; information comes from interpreting of data; data is materialized in symbols. The more data (and time), the better are our neural networks trained for the tasks, which are important for us to survive.
Big data typically shows correlations. The current debate is about whether it can even determine causalities. The processing of big data and exchanging information can help us to see a larger part of the world, uncover new unrevealed relationships, look into details, understand better, be more efficient and finally succeed in the evolution race. We may better distinguish what is true and what is false by processing larger and larger amounts of data. But data itself does not bring insight or understanding. Our ambition to understand the world, and this understanding itself is the real mystery.
Recommended reading:
Yuval Noah Harari: Homo Deus: A Brief History of Tomorrow, HarperCollins Publishers, ISBN 9780062663177
Stephen Hawking, Brief Answers to the Big Questions, Hodder & Stoughton/Bantam Books, ISBN: 9781529345421