![](https://cms.baincapitalventures.com/wp-content/uploads/2024/08/BinaryBlooms.jpg)
Selective State Space Models: Solving the Cost-Quality Tradeoff
As AI is increasingly used in production scenarios, costs are mounting. Are alternative architectures the solution?
All areas of infrastructure software change rapidly, but perhaps none of them quite as quickly as the web development stack. The rate of innovation in other categories of infra can be limited by a few things, including relative technical depth (e.g. databases) or reluctance in depending on new players (e.g cybersecurity). Web dev tools, on the other hand, can be built quickly, have a strong tendency towards open-source, and can be adopted, tested, and if necessary dropped within a day.
Another cause of rapid innovation in the space is the fact that end users directly interact with web applications, and thus the toolset and frameworks used to build those apps affect their experience. Delivering fast and delightful user experiences is top of mind for all software or software-enabled companies, and so the frameworks powering these experiences are in a constant state of improvement.
The Jamstack is the perfect example of user demands driving technical innovation. The Jamstack is a modern web architecture that delivers better application performance, easier scaling, stronger security, and better developer experience. At its core, the Jamstack is the use of modern build tools to serve static, pre-built content via CDN (content delivery network) with dynamic functionality added via serverless microservices and APIs.
In the traditional client-server architecture, a user hits your web page, the browser makes a request to the application server, the server makes any necessary database or CMS calls, renders the result page, and serves the built HTML to the user. This architecture requires building content on the fly and usually making multiple calls from client to server, both of which increase page latency. Another advantage of the Jamstack is improved SEO, given that web crawling algorithms often index information on a page quickly after first loading it, which may miss information if some of your content is still being rendered.
The frameworks, tools, and APIs shown below are many of the pieces used by developers today building applications with the Jamstack and modern practices:
Having researched the space for a few weeks and spoken with many industry experts, a few points about its current state and where it’s going stand out to me.
First, the Jamstack is in essence all about pre-building HTML files. Excitement about the architecture and its uses has caused the term to be thrown around at times more liberally than it should be, but only when HTML files are pre-built and extra functionality is added via third party APIs or distinct microservices can we truly refer to something as Jamstack. This is most often seen when a web app is mostly pre-built, but on a per user basis requires pieces of customization, so those customizations come from running client-side javascript to make requests. This paradigm really represents a traditional client-server architecture with client-side rendering, but with the amount of client-side rendering reduced by pre-building.
However, the problem described above can’t be fully addressed by a pure Jamstack architecture — how do you customize applications on the fly if the entirety of your site is pre-built? Logically, you can’t. For this reason, increasing mindshare in the industry is moving from static site generation to server-side rendering. Server-side rendered applications send a completely rendered page to the client, rather than sending bundled HTML, CSS, and JS to the client and having the client render.
Additionally, with server-side rendering, you can leverage the stale-while-revalidate caching pattern to store fully rendered pages in a CDN and not require every client request to go to the server, avoiding the render step all together. This is the piece of SSR that most closely resembles pure Jamstack. Another benefit of SSR over SSG is tied into this. When you have a cache miss, the client request will continue onto the application server, and imagine that you need to make a request as part of rendering a page to a third party API, like Twilio. We can better trust our application server to quickly connect to Twilio than we can an arbitrary client, but the Jamstack architecture requires a reliable and fast client connection to make that additional call.
Finally, we can notice that data layer tech for web dev moves to higher level of abstractions over time. In the not too distant past, it was common to write custom SQL statements for database access for even weekend-project type applications. More recently, ORMs have made it easy to manage and update data schema without worrying about specific db commands, and they’re even getting to the point where ORM becomes its own layer that’s decoupled from the storage layer.
As AI is increasingly used in production scenarios, costs are mounting. Are alternative architectures the solution?
Cube is the standard for providing semantic consistency to LLMs, and we are investing in a new $25M financing after leading the seed round in 2020.
In this edition of “In the Lab,” Amit Aggarwal explains why he’s building an AI startup in BCV Labs after selling his company The Yes to Pinterest.