How to generate insights

“…when it came to real-world complexities, the elegant equations and the fancy mathematics he’d spent so much time on in school were no more than tools – and limited tools at that. The crucial skill was insight, the ability to see connections.”

Taken from Complexity by Waldrop Mitchell, the quote described W. Brian Arthur, a pioneer in complexity economics.

Similar to Arthur, investors want insights. They want to know something that others do not. The most competitive institutional investors pursue insights in the form of differentiated data. Gabriel Plotkin, the owner of Melvin Capital (of GameStop short-selling fame), was an early user of credit card data when picking consumer stocks. Many investors interview vendors, employees, and product experts to uncover unique data.

If the puzzle of above-market returns is a walled castle, pursuing differentiated data is similar to a head-on attack. Accumulate armies of analysts and differentiated data, and swarm the castle’s defenses with sheer intellectual brilliance.

Yet a head-on attack is not without risk. Casualties abound (high turnover in analysts and data). The castle walls have to be re-built to prevent others from attacking (differentiated data is discounted by the market, demanding even more differentiated data sets for returns).

A better tactic would be to understand the needs of the town within the walls, and negotiate with its leaders for a surrender. No casualties. Nothing destroyed.

I label the alternative tactic as differentiated understanding.

The path to differentiated understanding begins with a hypothesis, which is tested against real-world data. Even when data is supportive, the hypothesis is never fully accepted. Statisticians use the term “cannot be rejected”. This means that other data, not yet considered or available, may reject the hypothesis. The analysis leaves room for further exploration, which, when repeated, leads to correct and deep understanding.

Compare what I just describe to the opposite. Start with differentiated data (instead of a hypothesis), determine patterns, then attempt to understand. This is what investors commonly do, and it leaves plenty of room for error instead of exploration. The patterns are likely to reflect correlation. Mistaking correlation for causation results in erroneous understanding, which is often expensive to correct in investing.

Yet data is often used as the starting point because it is relatively easier to obtain than a hypothesis. A credible hypothesis is abstract and creative, because it hasn’t been fully proven by data, isn’t known by others, and isn’t discounted by the market (hence containing the potential for profit).

More difficult than coming up with a hypothesis is refining it. The process demands more creativity and abstraction, more grounding by real-world data, and sometimes long feedback cycles. This is diametrically different from the touch-and-go nature of the typical data analysis process, in which analysts move quickly from one data set to another without proper understanding (hence creating an industry for more and more differentiated data).

Differentiated understanding is wisdom, whereas differentiated data is still data, just with unique sources. Dee Hock (founder of Visa) describes it best:

“Data, on one end of the spectrum, is separable, objective, linear, mechanistic, and abundant. Wisdom, on the other end
of the spectrum, is holistic, subjective, spiritual, conceptual, creative, and scarce.”

That statement is as profound as it is useful. The modern scientific pursuit is reductionistic. It distills the world to the few factors that mostly explains the observed phenomena. The resulting theory is thus indisputable and mechanical. Yet the theory only works in the defined environment, and can hardly be extended to the real-world containing more variables and complexity.

What wisdom attempts to achieve is results in the real-world. It discards the elegance and simplicity of reductionistic theories for utility. It sees the world not as linear cause-and-effect, but as a system comprising myriad interrelated nodes. Changes in a single node would reverberate through countless others.

Perhaps an example would aid in solidifying concepts. Say theory and experience have proven that mean reversion works in investing. Buy cyclical companies at trough valuations, sell them at peak valuations, repeat. Wisdom would require asking “and then what”. What happens when more and more investors realize and practice this strategy? Trough valuations would no longer be low when more investors buy. Peak valuations would no longer be high when more investors sell. What happens when more and more cyclical companies realize what investors are doing? They might adjust their business models to maintain stable valuations, which would aid as a currency for M&A and employee stock incentives. The entire game would evolve.

In another example, research has consistently proven that rising EPS is the key determinant of rising stock prices. Linear cause-and-effect logic would dictate one look for stocks with rising EPS and stagnant or declining prices.

This simple dynamic leads to complex behaviors. Consider that one should buy the stock before EPS rises. How early should investors be? Perhaps wait for indicators such as rising operating earnings that precedes rising EPS? If the market knows about rising operating earnings, the wise investor would have to look for earlier indicators yet to be discounted.

Here’s the kicker. What happens to the original linear cause-and-effect logic? The market has evolved such that actual rising EPS forms a weak (but not impossible) case for rising stock prices. When the market is looking for earlier and earlier indicators for rising EPS and yet ignores actual rising EPS (when it shouldn’t be), the market is strongly signalling that the rising EPS won’t last.

To achieve wisdom is to understand how the world really works. The real world evolves, but an experimental one does not. Little changes result in enormous shifts in the real-world, but are unlikely to be demonstrated as such in a controlled setting (it would make the setting uncontrollable).

This is not to say scientific inquiry is useless. If anything, wisdom recognizes both the utility and limits of scientific inquiry.

Every investor begins with a scientific mindset. There is no starting if you cannot count and theorize. But achieving great returns requires leaps beyond, and many non-scientific subjects would aid that effort. Dee Hock said it best again:

“Science has traditionally operated in the provinces of data … where measurement, particularity, specialization and rationality are most useful. It has often blithely ignored the provinces of understanding and wisdom.

Theology, philosophy, literature, and art have traditionally operated in the provinces of understanding and wisdom, where subjectivity, spirituality, and values are most useful. It has often blindly opposed the scientific way of knowing.

Data moves at the speed of light today (quite literally). Data, no matter how differentiated, is quickly discounted. Wisdom is difficult to gain but also hard for the market to discount quickly, and should henceforth be the insight upon which investors rely.