Foundation models beyond the cloud
Opportunities for creating and owning new markets
Foundation models — which are manifested today in ultra-large generally pretrained generative models for text and image data, are a modern marvel. Retaining the value they generate is, however, a challenge. Foundation model developers are threatened by commoditisation, competition with open source, and enormous cloud bills.
To differentiate and retain at least some value, model developers (in the broad sense) follow the dominant narrative and draw inspiration from ”elastic” cloud-like services trying to wrap the deep learning magic into black-box APIs and thus build a ‘cloud on steroids.’ Inflection AI is building one of the world's biggest computing clusters to support the training and deployment of large-scale AI models. Hugging Face, on the other hand, is partnering with AWS . And it makes sense at first glance.
For those keeping pace with the tech industry’s rapid evolution, the current foundation models' wave might evoke a sense of déjà vu. It's a narrative reminiscent of how AWS provided a springboard for SaaS enterprises by making launching a SaaS startup much, much easier. Bringing easy access to AI through cloud-hosted foundation models is not dissimilar to how AWS reshaped the business of data storage and computing. Moreover, generative AI startups already syphon off about 10-20% of their total revenue to the cloud.
Aiming to become the ‘next AWS’ will backfire, however. The cloud, as a business, is susceptible to commoditisation. A Goldman Sachs study [1] points out that Amazon slashed AWS prices 42 times from its inception in March 2006 to March 2014. There has been a race to the bottom industry-wide: Google cut prices drastically in 2015 on both its Compute Engine and Cloud Storage to a consistent $0.26/GB, a decrease of 68% for most users.
Combining the commoditised business of the cloud with foundation models that exhibit similar tendencies makes for a weak overall proposition. Think of a race to the bottom on the software side due to open source, no moat on hardware since everyone is using the same chips from NVIDIA.
Perhaps model developers should be looking to the likes of Microsoft, Intel, Apple, and Google for inspiration. These companies successfully turned a variety of technologies (chip design, search algorithms, graphic user interfaces, etc.) into defendable non-commoditised businesses and created their own markets. They effectively built monopolies, following Peter Thiel’s motto — ‘competition is for losers.’
Microsoft, Google, and Intel were able to maintain a tight grip on their respective core markets for decades, with nearly 90% share, unlike AWS and other cloud vendors (Chart 1). The ambitious foundation model developers of tomorrow can learn from this.
Chart 1. Monopolistic vs duopolistic markets, operating systems, search vs cloud computing
Intel: Pentium and PCI
AI model development and microchip design don’t sound like similar businesses. However, they actually have a lot in common. Firstly, they share similar cost patterns (significant upfront investment, then mass production) and commoditisation pressures (open-source pressure in AI, intense competition with Japan in chip development).
Their respective roles in the technology stack are also similar. In the same way that a central processing unit (CPU) is the heart of a computer, not just a minor component, foundation models function are at the core of the modern data stack (data stack definitions vary, e.g. here and here. We use the broadest possible option). Furthermore, both the model and chip development industries show a pattern of frequent paradigm shifts and successive innovations (Chart 2).
Chart 2. Processors and foundation models have similar paradigm shift/iteration life cycles
Unlike regular software, foundation models cannot be continuously updated. Each model architecture must be completely replaced when a new one is developed, mirroring the lifecycle of a chip.
Consider an e-commerce company that digitises its entire product catalogue and trains a model on it; if the model struggles with a specific product category due to its architecture, it can't be swapped out easily. One either has to find a model with a very similar architecture to reapply embeddings to it, re-vectorise the whole catalogue, or wait until the next iteration of the model is rolled out.
When a product relies on the power of a large-scale, generally pre-trained model (even if indirectly, e.g. after fine-tuning towards a particular use case), it is far more than a casual software dependency. While the API could be preserved from one generation of the model to another, the intricate embedding space in which it operates and, even more, its behaviour might change almost arbitrarily. What’s more, such ‘version updates’ often take months of training on GPU-powered supercomputers and incremental and backward compatible model updates are not yet universally adopted.
Therefore, a dependency on a foundation model is more akin to a hardware dependency, now mostly a forgotten issue, which comes with all the usual platform-specific design choices, optimisations and, in the lack of an open standard, platform lock-in.
The only caveat is that hardware does not speak human language on its own. Time will tell whether prompting is sufficient for most applications and whether the low-level ‘language’ of embeddings can be bypassed. Perhaps, continuing the hardware/software analogy, we will see the emergence of Javas of the modern age that provide a reasonable abstraction layer easing the dependency on a concrete foundation model or a family of them.
In a world governed by Moore's Law and the scaling law in AI models (where we get better performance per unit of computation as models and their datasets get larger), there is always a new technology threatening to replace you. How one could encourage customers to become reliant on their technology and drive platform lock-in?
Intel's path offers useful insights. Intel became morphed into the architect of the PC industry by bridging its constantly evolving products with adjacent components, making itself a more and more critical piece of the tech stack over time. For example, Intel introduced the Peripheral Component Interconnect (PCI) in 1992, changing the internal architecture of PCs forever and encouraging future compatibility with newer Intel hardware. This was critical in making Intel the platform leader.
As model developers aim to emulate Intel's platformization playbook, data tools emerge as the potential linchpin to connect models with data in the same way that the PCI connected the CPU with peripherals. Improving the accessibility and integration of data into models could hold the key to making model developers indispensable, given that data is the primary chokepoint in the quest for better and more general foundation models.
Going deeper into the data stack also makes sense technology-wise, as ‘ …it seems unlikely that the [current data] schema design patterns we created for people are optimised for LLMs.’ The solution might be ‘... to favour schema design patterns that enable the big-brained computers more than they enable us feeble-brained apes.’ Designing and implementing schema models favourable to LLMs and building appropriate transformation layers, these and other endeavours in the data stack will entrench model developers and help them to capture value.
Case in point: AI21 Labs has launched the Contextual Answers tool to integrate customers' data/digital assets to enhance the efficiency and accuracy of information queries. Nomic AI's tool, Atlas, enables data sharing and provides AI operations such as vector search and topic modelling. DataOps platform Databricks acquired MosaicML, a fellow DataOps platform and model developer.
These early attempts of building data tooling around foundation models suggest a direction where the market may go. Instead of turning foundation models into commoditised cloud workloads, their developers might create unique combinations of algorithms, tools, and data sources. More audaciously, they may conquer an emerging data exchange market, currently being a fringe of the cloud (AWS runs a Data Exchange business, it does not disclose revenues, but there are only 300+ data providers there. Snowflake offers 2.1K data products on its marketplace).
Now, cast your mind back to the interconnect that made Intel's processors the beating heart of the PC platform. We're witnessing similar dynamics here. Tools and data exchange platforms are aligning, converging, and shaping models to become the centrepiece of the burgeoning AI ecosystem.
Windows, Apple, and Google: operating systems and app stores
Windows, iOS, and Android unlocked an abundance of opportunities for app developers by integrating new components and sensors into a unified system. Foundation models, owing to their incredible generalisability, similarly have brought an array of capabilities to app developers.
From language classification and translation to search, question answering, and conversation, a single large language model can do it all. Moreover, foundation models are about to enable the utilisation of previously underused sensors, such as EEG headsets.
By enhancing existing platforms, like mobile phones, and pairing them with new types of hardware, such as EEG headsets, foundation models could potentially evolve into a truly cross-device operating system.
However, this transition won't happen overnight, as similarly to Windows, model developers start from the interface layer. Windows started as a GUI for MS-DOS and gradually evolved into a full-fledged operating system. Initially, it used MS-DOS for file system services but had its own device drivers, for example.
Microsoft then evolved Windows into the full operating system, simultaneously increasing its value by offering its products for the emerging PC platform: ‘Microsoft's strategy was to rely on making its own complements — thus the development of applications like Word, Excel, Outlook, [and other components] embedded in Windows.’ Perhaps, while our attention today is taken by the results in language modelling, the next frontier is building the operating system analogue that doesn’t really care about data modalities or applications and knows how to deal with them in a unified mother-model or by cleverly orchestrating an ensemble of smaller models through a combination of natural language and vector embeddings.
Nowadays, when dozens of apps and multiple devices overload users, the need for a new interface to connect them all is apparent. Here foundation models step in. Adept AI, for instance, builds an overlay to the tools you use today, which may evolve into an operating system down the line too.
Meanwhile, OpenAI and Stability AI seem to be pursuing the app store route to dominance, much like Apple. Similar to Apple, these model developers released their own products first to gain traction and then invited third-party developers to join. Both create marketplaces and offer developers the ability to build plug-ins into their models and to connect with their userbases via plug-ins stores (Stability has not yet launched the store but mentions a plugin library). Connecting developers and users is not enough, though. As Apple demonstrates, building convenience on both sides is crucial.
So far, both interface and app store pathways to building an operating system out of models achieved some success. Adept AI and OpenAI raised strategic investments from Microsoft, and the latter’s ChatGPT had more than 735 verified plugins as of May 2023.
However, achieving operating system supremacy remains an uphill battle as the current operating systems build strong full-stack platforms (Apple - owns iPhone, Google – laid out the foundation via the T-Mobile-HTC partnership, and Windows partnered with Intel).
To secure a substantial foothold in the operation systems space, model developers might have to form partnerships in the hardware sector or branch out into emerging hardware categories beyond incumbents' control, like robotics or brain-computer interfaces.
Consumer platforms: Google Search & Facebook
The saturated consumer segment probably scared away model developers who seem to avoid building model-powered platforms there. Despite the dearth of pure consumer plays in the foundation models domain, CharacterAI stands out, allowing users to make their model-powered avatars public. This paves the way for a creator/consumer marketplace. Like Instagram used the limitations of early phone screens to create a new art form, the iconic square photo, CharacterAI could harness the quirks and limitations of early foundation models to establish a novel art form and own the platform where this art thrives.
Facebook's Instagram is just the tip of the inspiration iceberg when it comes to crafting a consumer platform enabled by foundation models. Enter Google, playing a different tune altogether. While Instagram curated a canvas where content blooms and resides, Google positioned itself as the bridge between content creators and the hungry audience. All of it orbits around Google's PageRank algorithm, a force that pulls advertisers, web developers, and consumers together.
On one end, therefore, there's the approach of pinpointing glitchy transactions—like an audience lost in the digital maze searching for a website — and seamlessly stitching them back together. Swing to the other side, there is crafting a novel strain of content. Its content is moulded by foundation models, much in the way square photos once mirrored the synthesis of mobile phones and the World Wide Web.
Conclusion
In a world where even our foundational AI models — trained on a mix of generic public and restrictive proprietary datasets — leave us awe-struck, the next act promises to catapult us straight into the realm of science fiction. This forthcoming wave has its sights set on dwarfing the magnitude of the cloud computing revolution.
To the architects of these foundation models, we say: don't merely emulate the likes of Amazon’s AWS or any other platform — learn from them, absorb their strengths, and then imagine and construct new types of platforms. Be audacious and innovative, and venture into markets uncharted and unexpected. Dare to imagine an entirely new data stack, foundation model-powered robotics, operating systems for brain-computer interfaces, and other things that don’t fit into the current cloud-powered SaaS paradigm.
📩 Drop us a line if you are developing foundation models and want to build foundational platforms of the future based on them.
✍️ Subscribe to hear our thoughts on other frontier technologies, themes, and business models.
***
Kudos to the following amazing humans for helping us with this article:
Boris Yangel, the Head of LLMs at Nebius. Check Boris’ thoughts on ML and LLMs here.
Patrick Ryan, a Co-Founder of Odin (an Approx.vc portfolio company). Check Odin out — it’s a great tool for angel syndicates, emerging managers and founders to raise capital from their network and manage their investor community.
Jacek Łubiński, a Partner at Market One Capital — a fantastic investor for anyone building a platform. Here’s his Twitter.
References
[1] Riding the cloud computing wave; RHT down to sell. Goldman Sachs Investment Research, Jan 2015.
[2] Global market share of personal computing platforms by operating system shipments, 1975-2012E. KPCB.
[3] Market share of leading desktop search engines worldwide from January 2015 to July 2023. Statista, 2023.
[4] Cloud provider market share trend. Synergy Research Group, July 2022.