AI: The Indexation of Everything
Artificial intelligence is fast becoming the most significant technological development since the advent of the internet. Recent breakthroughs have brought AI out of obscurity and into the everyday lives of ordinary people everywhere. As with any such development, there are pros and cons. There are those who embrace this new reality, those who feel threatened by it, and those who proceed with caution.
There is so much being written about AI (much of which is being written by AI!) that I don’t intend to reproduce here. AI could do that better than me. Rather, I want to share a different perspective that comes from the investment world: AI can be better understood through the lens of indexation. Or perhaps indexation can be better understood through the lens of AI? Regardless, the index fund is essentially a very primitive form of AI.
The Essence of Indexation
An index is a representation of the average investment portfolio within a certain universe of publicly traded assets. The S&P 500, for example, is an index of the 500 largest public companies in America. The JSE Top 40 is an index of the 40 largest public companies in South Africa. There are indices for different regions and industries. Some are quite narrowly defined, while others are broader. There are also different weighting methodologies like market capitalization or price. Some indices are simply equally weighted. More sophisticated weighting methodologies are based on company fundamentals – so called smart-Beta indices. But in essence, every index is an average of some sort.
For this reason, every index is reliant on the availability of public information. This information has come about as a result of the work (or activity) of more traditional investment managers (also called active managers). These companies employ research analysts and other professionals to analyse potential investments with a view to determining a fair price for those assets. These ‘fair’ prices are then communicated to the public domain through trading activity on public exchanges. The prevailing market prices, which are determined by supply and demand, represent the ‘average’ opinion of these active investors.
Indexation’s Free Rider Problem
Because this information is publicly available, other investors who aren’t eager to devote resources to their own determination of a fair price, are happy to simply track an index (or an average) of some subset of the overall market, thereby achieving average results without incurring costs. This is called indexation, or passive investment. The obvious advantage of passive investment is that it will, by definition, always do slightly better than the average active manager operating in the same space, after costs.
The disadvantage is less obvious: Passive investment presents a classic free rider problem, where people choose to enjoy the benefits of some good or service, without contributing to the cost thereof. Unchecked, this eventually leads to market failure. In the US, passive investing already accounts for more than half of the stock market capitalization, up from 20% in 2009. At what point does the price discovery mechanism break down? Perhaps it already has, as a wave of indiscriminate buyers has dominated the purchasing activity of the major stock index constituents for more than a decade…
AI as an Extension of Indexation
But let’s move on to AI. Many of its most popular applications today are nothing more than sophisticated forms of indexation being extended to other industries, particularly those which are based on intellectual property. Artwork is a clear example of this: Why pay an artist to draw an image of a cow if AI can produce a generic cow at a fraction of the cost? The same is true for music and video. AI is already being used to write entire novels.
But where does AI get its ‘inspiration’? From data that’s been shared on the internet. Not an exact copy, mind you. That would be plagiarism. But an average of sorts. There are various ‘weighting’ methodologies that can be specified, but ultimately what you end up with is some kind of average representation of work that’s already been done. If you plagiarise everyone, does that still count as plagiarism? What we have here is another classic free rider problem. It’s the indexation of everything.
Lessons from the Investment Industry
So how will this play out? We can’t know for sure, but there are some important lessons we can learn from the investment industry, which serves as a kind of primitive case study into the potential effects of AI on other markets:
For industries most susceptible to AI mimicry, the supply of low-cost products and services is going to skyrocket, putting pressure on the prices of non-AI participants. This has pros and cons. One of the plagues of the investment industry in the 80s and 90s was investment managers milking their clients for doing nothing more than tracking the average anyway. Indexation has gradually forced active managers to become increasingly active (i.e. different from the average) and more cost-efficient. Likewise, one of the positive effects of AI will likely be to expose overpaid copycats in other industries. In this sense, AI has the potential to enhance creativity and efficiency rather than to kill it.
The obvious risk, however, is that you don’t only end up exposing the copycats but also killing the creative engine which makes this kind of AI possible in the first place. Some believe that AI could eventually replace that creative engine. That’s a scary thought if it’s true, but I’m not convinced that AI can be any more creative than a passive investor can determine a fair price for a stock. Some would argue that accurate price discovery in public financial markets is already failing because of indexation, but no one really cares as long as markets keep rising. It’s only on the downside that the problem will become apparent. The same problem could easily extend to other markets over time, with even further reaching consequences. Free rider problems, if left unchecked, have a tendency to end the ride eventually.
Much of the potential to harness AI sustainably hinges on our ability to mitigate the inherent free rider problem. Time will tell whether we do a good job of it not. In the meantime, whether you love it or hate it, it looks like AI is here to stay.









