Before getting into this week’s essay, I just to alert you to a new undertaking of mine:
Concept testing on Linkedin.
From now on I’m going to be doing one post a day on the platform where I’ll be trying out new ideas and theories, having a chat about them, and then potentially expanding on them more deeply here in the newsletter.
If any of you are on the platform and would like to receive and chat about this stuff, then I’d love it if you gave me a follow / invited me to connect. I’ll always respond thoughtfully to any questions / builds / challenges in the comments – and over time hopefully we’ll get a nice little “talking shop” going on over there.
My profile is here, just go there, follow or connect, and hit the bell icon to get notified of new posts.
Right, now on with the business of the day…
___
The world of expertise is built on one gigantic lie.
Business consultants, doctors, coaches, lawyers, chess grandmasters – you name it and the lie is the same:
That good decisions are made by rigorous analysis of the facts at hand.
We have this idea that to solve a complex problem, as experts like these do on a routine basis, the formula is always roughly the same. First gather the data. Do your research, compiling all the solid objective information you can. Second, analyse the data. Have a good hard think about what you’ve learned, attempting to following it through to its logical conclusion. And then finally, arrive at “the answer”.
This belief in a conscious, rational, systematic process to deliver results is the driving force behind the popularity of trademarked “models” and “proprietary technology” that many professionals (especially in the business world) use to try and sell their wares. It’s why the medical industry is increasingly systematising its diagnostic processes; input symptoms here, output solution there. And it’s also why many people believe that sooner or later AI will take over all these fields – since if results are simply a matter of rigorous calculation, it stands to reason that a computer will eventually do it better than a human being.
Too bad then that this universally accepted belief is (almost) complete bullshit.
The truth is explained at (extraordinary) length by psychiatrist and neurologist Iain McGilchrist in his two-volume tome The Matter With Things – a part of which I’m going to attempt to brutally shorten and simplify for you here today.
In a nutshell what McGilchrist shows is that results in all fields of expertise (including business strategy) are not driven by conscious analysis, but are in fact almost wholly the result of unconscious intuition. In other words, great experts don’t really “think” about their problems at all – they more “glance” at them and then reach their conclusions instinctive “gut feel”. These conclusions might be post-rationalised to explain why they might be right, and to “sell” them to an outside audience, but the actual process by which they’re arrived at is nothing like the linear one that the post-rationalisation suggests.
So profound is this truth in fact, that not only is rational analysis and research relatively unimportant, but it actually lowers the accuracy of results when inserted into the process. In other words (to oversimplify slightly) experts will arrive at worse conclusions if they’re forced to think about the problem than if they just go with their gut.
How is all this possible? I’ll give you my crude interpretation, and then illustrate it with an example from the book which I found pretty astonishing.
In short the core idea is that the unconscious mind knows a hell of a lot more than the conscious.
Think about it like this. We can imagine reality as being comprised of an “infinite” amount of information. When analysing a problem there aren’t only, say, 5 pertinent “facts” (which is roughly the highest number our conscious mind can process), or even a million, but rather a sort of fluid continuum of stuff all of which is inter-related, and cannot be pulled apart or “reduced” for conscious analysis. Therefore there are far more variables to any situation than we can ever capture and think about it.
Now, couple this with the fact that of all the stuff our mind takes in, 99% is unconscious and 1% is conscious. Our brain only makes us “aware” of a tiny sliver of information it receives at any given moment in time. What that means then is that we “know” far more than we know we know. When a doctor observes a patient, for example, they might be consciously aware of 10 “facts” about them, but at the same time they are unconsciously aware of another 1,000 more.
As a result the unconscious mind is far better placed to accurately interpret reality, since it bases its decisions on a far fuller picture than the conscious mind does.
A powerful example of this can be found in the case of the “horse picker” Franck Mourier. His job is to basically give his opinion on which horse is likely to win a race, for the benefit of his clients who will then make their bets accordingly – and he’s very good at it, consistently beating the odds. What is amazing about his process however is not just that it’s intuitive, but that when he’s given seemingly pertinent facts about the horses (e.g. who won the last race or whatever), then his system falls apart and he can’t pick accurately – as if that single isolated piece of information has corrupted his otherwise holistic decision making process.
When asked to describe how he does it, he gives a very illuminating answer:
“I am really at a loss to explain what I am seeing, and have generally no real clue how I come to a percentage. It is definitely a combination of a lot of factors – but if I had to name a few, they might be: Does the horse look happy or worried to be on the racetrack? What is the quality of the relationship between the horse and the person that leads him to the race? Does the horse appear healthy and fit? Are the jockey and horse a good match? Does the horse trust his jockey and listen to him? Can the jockey understand the horse and communicate with him properly? But again these are just some impressions. It is not consistent and certainly not straightforward… Anyway, after the race we can immediately see the accuracy of my predictions. Then the process starts again in the next race.”
Do you see how what he’s describing is a much more complete view of reality than could be provided by a more “scientific” statistical analysis? How despite being what many would describe as “wooly”, it is in fact far more sophisticated than something more rigorous? The power of this process doesn’t only come down to the sheer volume of inputs it incorporates, but rather as McGilchrist notes: “part of the power of the unconscious mind is its capacity to deal with numerous different elements simultaneously – and, even more importantly, to see their interconnexions one with another”. Reality is a whole, and therefore an analytic process which breaks it down into “pieces” (as the conscious mind must) is always going to be inferior to one which takes it in as it truly is.
He concludes:
“Experts in any field need both to attend to the Gestalt, the picture as a whole, and, occasionally, to engage in (preferably limited) analysis. Why preferably limited? Because it has costs. It takes time, yes; but worse, it also changes its object, risks focus on areas that may be inessential, risks substituting something explicit and measurable for something much more important that is implicit and unmeasurable.”
What is funny about all this is that despite the mountains of evidence and studies which prove time and again that intuitive decisions are superior to analytic ones, we simply refuse to accept it. We refuse to accept a process that is opaque to us, even when it is unambiguously shown to be superior.
You will note that you never get consulting firms, or other similar sorts of professionals, selling their wares on the basis of intuition. McKinsey will never say “hire us because our intuition is really good”. No, there always has to be data capture and data analysis at the heart of any pitch – a process which McGilchrist would argue basically amounts to a myopic focus on a trivially small amount of information at the expense of the larger reality.
There is one good reason for this mistrust of intuition, and that is this:
Intuition is only reliable in the hands of experts.
If you are not comfortable in a given field, then you will probably not achieve superior results via unconscious intuition than with conscious deliberation. To take chess as an example, it has again been proven that great chess players are almost wholly intuitive. For instance when they perform exhibition matches where they play 50 different games against 50 different opponents simultaneously, their performance only fractionally drops in comparison to when they play a single game that they’re able to concentrate on. However if you are a novice player, this doesn’t apply. You actually have to think about what you’re doing very carefully, otherwise you’ll get slaughtered.
It’s the same, in a way, with this newsletter. I openly admit to my clients that despite there being elements of process, my work ultimately boils down to me observing their business as a whole and giving my intuitive conclusions. However this is only possible because I’ve thought about this stuff so much. For people still learning about the field, they’re going to need to think about things more deliberately if they want to make progress – and that’s why I write these pieces, to build up that base layer of understanding that will eventually allow effective intuition to be unleashed.
When all is said and done though, there is no analysis that can beat (well informed) intuition.
- There is no model which can mimic it.
- No computer which can touch it (no, not even the sainted ChatGPT).
- And no alternative accessible to the human mind which is going to yield better results.
We’ll never admit it, and we’ll always be happy to pay more for something that looks more rigorous, predictable, and technically correct… but that doesn’t stop it being true.
