In this comprehensive piece, we’ll share what we’ve learned about generative artificial intelligence in financial services these last several months. You can use the following hyperlinked directory to jump around. Note that an * on a company's name indicates it is a BCV portfolio company.

TL;DR — A Field Notes Directory

1. Traditional AI and generative AI will cohabitate ☄️
2. Financial services needs generative AI 🤝
3. Generative AI will be just part of the solution going forward 🧱
4. There are three models for leveraging generative AI in financial services: Construct, Configure and Adopt 🎂
5. Specific examples of financial services companies / fintechs using generative AI 👀
6. How to separate the Hope from the Hype from where something is actually Happening 🌈
7. Generative AI largely benefits the incumbents, but there are exceptions where we expect it to be disruptive — here’s how 🧨
8. Generative AI can help accelerate existing trends 🚀
9. Our advice for practitioners 📋
10. What we’re watching on the horizon 🚥

Setting the Stage

In January of this year, our partner Sarah wrote a piece considering the potential impact of the generative AI revolution on the financial services industry: “How Fintech Can Jump on the Generative AI Bandwagon.” At the time, the financial services ecosystem was eerily silent, even though ChatGPT had been released to the public a couple months earlier, and Twitter and articles all over the startup ecosystem were abuzz with grand prophecies of the profound impact generative artificial intelligence would have across the tech world.

She wrote:

As an investor who spends a lot of time in fintech, one thing that struck me about the coverage of generative AI has been how infrequently applications to financial services are discussed… Why is fintech not feeling the love?

Her piece was one of the early thought pieces on generative AI for fintech. In it, she identified the inherent incompatibility between the conservative financial services industry, which requires perfection for regulatory and business underwriting considerations, on the one hand, and the permissive nature of generative AI, on the other, which provides the “next best” token prediction and is by definition not precise or accurate. While this tension was true, she argued that there was still tremendous opportunity for generative AI to revolutionize the distribution, manufacturing and servicing of financial services. How? By not seeing generative AI as an end unto itself, but instead by “delivering generative AI as a component within the broader software or workflow process.” After we published we wondered: Would any of this actually come to fruition?

Our Field Notes

1. Neither evolution nor revolution: Traditional AI and generative AI will cohabitate within financial services organizations

Within financial services organizations, AI is nothing new. In his 2022 letter to shareholders, JPMorgan Chase CEO Jamie Dimon famously wrote that the bank was spending “hundreds of millions of dollars” on AI and seeing identifiable returns on its tech bets (spending $12 billion on technology in 2021 alone). Of course AI hasn’t been reserved for only the largest organizations either: a 2022 report published by NVIDIA on the state of AI in financial services described AI use as “pervasive,” reporting that across all sectors of financial services — capital markets, investment banking, retail banking, and fintech — over 75% of companies utilize at least one of the core accelerated computing use cases of machine learning, deep learning and high-performance computing.

Already high, this number is rising: NVIDIA has also found that ninety-one percent of financial services companies around the world are currently leveraging AI investments to drive critical business — a leading indicator for expanded AI usage going forward.

If AI is nothing new for financial services organizations, why is generative AI getting so much attention? The “traditional AI” that financial services firms have effectively deployed for the last 40 years is very good at some tasks, but has its limits. Simply put, generative AI can do things that “traditional” AI cannot because it’s more flexible, both in terms of the data inputs required and the tasks performed.

As generative AI has exploded onto the scene, we’re seeing that financial services organizations are experimenting with generative AI while maintaining their investments in more traditional forms of AI. Generative AI cannot replace the role of traditional AI within financial services organizations because:

1. The technology is not well set up to perform the same tasks;
   
2. Language models are not great at math — traditional models have proven to be much better for quantitative tasks like revenue forecasting, pricing predictions, etc.; and
   
3. The technology is more expensive, so you’d overpower the problem and waste resources. Another way to think about it: You don’t want to use a sledgehammer to drive a nail into a wall.

In this way, generative AI is neither an evolution nor a revolution. While evolution is the gradual development of something, especially from a simple to a complex form, revolution is the overthrow of the current system in favor of a new one. In both cases, the assumption is that it’s out with the old and in with the new. That’s not the case here. As we’re seeing play out today, we expect generative AI to coexist with and complement traditional forms of AI within financial services organizations for the foreseeable future.

Return to directory

2. Not discretionary for long: Why financial services organizations need generative AI

Our team’s initial piece argued that there was the opportunity for generative AI to play a role in financial services organizations. Based on the conversations of the past few months, it’s time to make a far bolder argument:

Financial services organizations need to leverage generative AI to fulfill their potential for customers, employees and shareholders.

As sand can fill the open space in a jar of stones, generative AI can fill the gaps within financial services organizations left unfulfilled by traditional AI. These gaps today represent the unfulfilled potential of financial services organizations to offer better, higher quality, less expensive experiences for customers, more rewarding and fulfilling experiences for employees and better earnings for shareholders. It may be that traditional AI has not been directed toward these problems before, or that traditional AI has been insufficient, as in the case of customer service.

As explored in #1 above, financial services organizations have already invested heavily in technology, including traditional AI. In spite of this significant investment, however, there is still so much low-hanging fruit to create better experiences for customers and employees and to improve organizational profitability. Financial services is one of the industries with the lowest customer satisfaction scores and highest service costs. How can both be true?

• Financial services products are in and of themselves complicated and highly regulated, and thus need to be distributed, manufactured and serviced with a high degree of precision and care. Corners can’t be cut, or else financial services firms risk regulatory action like losing their licenses and going out of business.
  
• To pick on customer service, one application vector: Traditional AI is leveraged heavily to take care of the 80% of requests that are straightforward (e.g., simple requests like “What is my balance?” or “When is my bill due?”) but not the 20% of requests that are bespoke and actually represent 80% of the cost (e.g., complex requests like “Should I refinance my mortgage now or wait until next year?” or “My account balance is lower than I expected. What happened?”). The latter requires customer service representatives, but they are slow, expensive and don’t always have the right information readily accessible to address the customer inquiry.
  
• Within financial services organizations today, traditional AI can be used for first (and second) order analysis, rules-based sorting and classification, and Q&A engagement, but it’s not sufficient to cover all edge cases, respond to never before seen problems or exercise judgment.
  
• Taken together we have a current system that is necessarily expensive, but also has low customer satisfaction.

How might generative AI change this low customer satisfaction and high service cost dynamic in financial services organizations today?

• Generative AI can be used to address the very issues that traditional AI struggles with, specifically having an answer for all questions, responding to never-before-seen problems and exercising judgment. In other words, generative AI provides an interface to simplify complexity.
  
• One example of complexity that is pervasive in financial services but rarely discussed is the “emotional side” of financial services, as Lightspeed’s recent post also explored. Even though financial products play a consequential role in our lives, they are poorly understood. We typically interact with our financial services organizations when we want to better understand our financial situations, address an unexpected and undesired outcome, learn what steps to take to improve our situations, and receive reassurance that things will be OK. Historically, this could only be done by humans, and only done well by a small subset of highly-trained, experienced, motivated humans. Now with generative AI, this paradigm is shifting. For the first time, generative AI enables financial services organizations to become more emotionally available to customers, employees and shareholders alike.
  
• Overall, the opportunity is two-fold: (1) the existing conversational AI bots will produce higher quality, more responsive, more resonant answers, and can be used in a higher percentage of cases; and (2) customer service reps responding to the remaining issues can be better equipped with better information so they can more completely and efficiently respond to these questions.
  • “BUT, BUT!” you might interject, “How do we know that generative AI will produce the precise response that’s required in this highly regulated industry?” Jump to #9 below if you can’t wait! 👇🏼

Return to directory

3. Generative AI is a fundamental building block that will be part of the way we interact with financial services technology going forward

We’ve heard the phrase “platform shift” thrown out a lot in conversations around generative AI. “Platform” is just a fancy word for the model through which you’re interacting with a computer or other form of technology. The platform shifts we’ve seen over time include desktop to mobile and on-premise to cloud.

Generative AI represents a platform shift because it offers a fundamentally a new way to interact with technology. Because the underlying large language models are trained on an Internet-scale corpus of English text, we can interact with them using plain, everyday English. We’ve never been able to do this before! In the past, we’ve had to learn programming languages to interact with technology through code. And even during the Software 2.0 era, a lot of work went into feature engineering and hand-picking different deep learning architectures for different tasks.

Today, transformer-based models generalize amazingly well to a wide variety of tasks, making self-supervised learning far more feasible. The upshot for most users is that we can now interact with AI in our native tongue. (And back to #2, the language based nature of the generative AI platform is also why for the first time the AI is emotionally available to us!) For a more detailed discussion of the implications of this paradigm shift, check out Simon Taylor’s recent post.

However, we do not think that language replaces our interaction interface with all technology, especially in financial services where #, $ and % rule supreme. As mentioned in #1, we expect generative AI to co-exist with and complement traditional forms of AI within financial services organizations for the foreseeable future, like sand seeping into the gaps among the stones that have been left by traditional forms of AI and other technology.

Going back to the list of applications of “traditional” predictive AI within financial services organizations covered in #2, each of these could be enhanced with generative AI. The question and answer interaction style of the popularized generative AI chatbot can itself be embedded into other software, communication and AI experiences so that the generative AI is simply just a part of the fabric of the software. Tools like Microsoft 365 Copilot in Excel or Arkifi allow financial analysts to utilize a Q&A chatbot to output financial models and/or summarize key trends found in financial/numeric data. These analysts will still need to guide the AI to generate the most effective output, which will require their continued understanding of #, $ and %. As such, at least in the financial services context, generative AI is best suited as an embedded technology — a fundamental building block that will be part of (most) of the finance applications we experience going forward.

And, looking forward, we are excited to see teams innovate on the user interface beyond chatbots to distribute generative AI. The core innovation of generative AI is the next token prediction for large language models, and there are alternative methods to expose the predictions. A rose by any other name would smell as sweet!

Return to directory

4. Defining the three models we’re seeing for leveraging generative AI within financial services: Construct, Configure and Adopt

Consider you’re hosting a birthday party and plan to serve birthday cake. What are your options?

• Do you buy the cake, ready-made, maybe even paying extra for customized writing with the birthday girl’s name and age?
  
• Do you buy a mix, just adding water, vegetable oil and an egg?
  
• Do you find a recipe and start from scratch — assembling the ingredients and combining them one by one?

It’s likely that what you choose to do depends on a few factors, such as your goal, your budget, the ingredients you already have at home, how much time you have available, when you’re thinking about this vs when the party is, any food sensitivities/allergies, and your skills. And it’s a certainty that not even the most ambitious, most able cooks are doing everything from scratch — we’re not going to be milking a cow and churning butter for this cake.

Similarly, there is a range of models available for a given use case of embedded generative AI in financial services. Over the course of dozens of conversations, we’ve seen three models emerge:

1. Construct (”Create from Scratch”): Creating your own LLM trained on a specific set of data, typically leveraging your own proprietary datasets.
   
2. Configure (”Assemble from the Kit”): Leveraging the foundation of ready-made LLMs (closed-source or open-source), and then customizing them for your specific use case, such as through embeddings, prompt engineering, fine-tuning, etc. Organizations may choose to customize directly, or through a flourishing layer of “tooling” apps (e.g., Contextual AI*).
   
3. Adopt (”Out of the Box”): Purchasing in full a generative AI application for a specific use case within the organization. The application is typically built on top of a closed- or open-source LLM, and then wraps workflow or additional AI algorithms around the generative AI to deliver a complete solution for the given use case, e.g., conversational chatbots for customer service requests. The adopting company may choose ‘flavors’ of the solution, but there’s little (real) customization.

Relating this back to the factors that direct this choice of investment:

• Goal – How does this use case fit into your company strategy? If the goal is building a proprietary product, then perhaps you lean toward construct. Or, if the goal is strictly use case performance, even if you have proprietary data, then you’ll choose to use one of the existing LLMs that has vast amounts of data in the adopt or configure model.
  
• Budget – How much do you have to spend on developing a proprietary instance of generative AI?
  
• Ingredients available – Do you already have your data assembled, structured and available?
  
• Time – Do you need a solution today, or can you afford the time to build for tomorrow?
  
• Food sensitivity/Allergy - How sensitive is your data to protect, and how important is data privacy to your organization and your regulatory context?
  
• Skills – Do you have the internal talent required to build? If not, can you find them? Can you recruit them? Can you afford them?

We expect that financial services organizations may simultaneously embed multiple use cases of generative AI throughout the organization. And, the way the use cases are implemented will probably span across the three models of Construct/Configure/Adopt. To extend the analogy: let’s say you’re also serving dinner at the party. You may choose to bake the cake from scratch, but order take-out pizza for dinner.

Return to directory

5. Specific examples of how generative AI is being used within financial services organizations

We’re often asked how financial services companies and fintechs are using generative AI today and wanted to share specific examples. We love answering this question because it shows both how far we’ve come and how far we have to go. H/T to our friends at a16z who have a great post on the five goals that generative AI is poised to fulfill for financial services organizations. In our field notes, we want to move beyond the theory and detail how financial services organizations are embracing generative AI in practice.

A few framing thoughts:

1. The below list is not comprehensive: More companies are adopting and developing generative AI applications every day, and furthermore, only a small fraction of generative AI applications and investments have been disclosed today.

• For example, below is the text thread with one of our friends who’s the head of product at a well-known fintech company. This company has not made any public announcements about using generative AI, and yet… this text interaction about sums up where we think many organizations are today.

• Why do some companies disclose while others do not? We’ve seen (public) disclosure as a means to (1) improve their valuation (in the private or public markets) by showing innovation, (2) broadcast their service-orientation and value to customers, and/or (3) announce their plans to competitors as a show of strength and competitive differentiation.

2. We’re including companies across multiple archetypes:

• Some of these companies are financial services providers (Morgan Stanley, Carta, Stripe) that are looking to embed generative AI into their own operations, and others are technology solutions (Parcha.ai, Materia) that sell to those providers.
• Some of these companies are generative-AI native (they were built in the last roughly 6-12 months directly on top of LLMs), and most are not.

3. In addition to the axis described above of Construct/Configure/Adopt, there’s a secondary axis that matters: whether a generative AI use case is domain-agnostic or domain-specific:

• Domain-agnostic applications: These are generative AI applications that can be used across industries, such as customer support, marketing, sales, AI assistants with search and productivity, design and advertising, web app builders and code completion.
  • Examples: Chime’s partnership with a stealth startup to train its own internal LLM to enable Chime’s engineers to more quickly and cheaply launch new products; Stripe’s partnership with OpenAI to power Stripe’s FAQ Docs
• Domain-specific applications: These are generative AI applications that are specific to financial services, like risk modeling, fraud detection, policy underwriting or purchase order reconciliation.
  • Examples: SlopeGPT – Slope Pay’s custom SMB risk underwriting model that’s built on top of OpenAI; Navan’s “Ava” virtual assistant that’s custom-built for expense reporting

Based on what we’re seeing, what are our predictions for how financial services/fintech organizations will embed generative AI going forward?

1. We expect that the best teams implementing generative AI will fall into multiple categories across Construct/Configure/Adopt. Already today some companies are in multiple cells of our matrix above; going forward, we expect this to be true for most, as different use cases warrant different implementation models. As Federato CEO Will Ross told us, “They’ll know what they’re good at and be good enough at it to know what they can’t do themselves.”
   
2. However, we expect that over time most of the generative AI use cases will fall into the “Adopt” category, but that most of these instances will not be disclosed (except if by the vendor that is being used). Many financial services providers may lack the technical resources or AI/ML expertise to build out features that are not necessarily part of their core competency.
   
3. These categorizations are not static over time. “Adopt” could be the way an organization trials a use case, and if it proves useful, to choose to move up to the “Configure” category and assemble it in-house. For example, we’re seeing many teams use ChatGPT to prove that something can be done and that it’s useful, and then proceed to figure out how to do it in a faster, cheaper, better way. As another top fintech founder told us (keeping him anonymous to protect his vendors!): “We’re being careful to evaluate build vs. buy since the landscape is moving so quickly.”

Return to directory

6. When you hear about a generative AI use case in financial services… is it real? How to separate the Hope from the Hype from where something is actually Happening

Most of us are familiar with the “hype cycle” popularized by Gartner to describe the adoption of emerging technologies, shown below. Following a “technology trigger,” there’s a surge in interest and intent to integrate the technology, and the visibility of the technology rises to the “peak of inflated expectations,” or the top of the hype cycle. The other dimension shown on the graph below is utility; it’s still low through peak hype as companies are experimenting with and learning how to use the technology. The tendency of human nature toward over promising and under delivering results in the “trough of disillusionment” as companies realize the technology is actually not the cure-all as promised. Underneath, though, the real, careful work to integrate the technology is continuing to build, and the technology finds fit in its appropriate subset of use cases as its visibility rises once again through the “slope of enlightenment” and finally reaches the utility crescendo during the “plateau of productivity.”

Applying this to generative AI in the context of financial services: We don’t need to rehash the well-discussed “trigger event” created by technological progress from the transformer and diffusion models, or the very effective and accessible chat form factor from OpenAI that quickly exposed so many people to the technology, leading to a steep ramp in visibility. Rather than a new technology in the background, generative AI is AI in your face — we can see it, touch it and feel it. The network effects are wild — the novelty, combined with the early promise of utility, combined with the bewildering nature of it all — leads users to share their experiences with others… “Can you believe this?” “This is so helpful!” “I don’t know what to believe!” “Which app should I use? They all seem the same.” It’s a cycle that feeds itself to a frenzy at the top of the hype cycle, and feels like about where we are now.

In other words, we’re still early. Or, as our partner Christina poignantly put it in the FinancialTimes recently:

If ChatGPT is the iPhone, we’re seeing a lot of calculator apps. We’re looking for Uber.

However, it would also be a mistake to overlook that utility is building all the while. The value of inflated expectations is that it creates the activation energy that organizations need to overcome inertia — and this is especially important for financial services organizations that are inherently conservative.

Today, we’d be hard-pressed to find a single financial services or fintech company that has not discussed generative AI at their board meeting. It’s the topic that everyone wants to have on the agenda for industry conferences. Most companies have created a committee. In fact, even those organizations that are not particularly technologically-progressive have had to care because (1) generative AI is so easy for employees to use (via ChatGPT and other B2C apps) and (2) well-governed organizations have concerns about this use. These concerns include data privacy, regulation, enterprise data security, cultural issues around job security, hallucinations and impact on the brand, IP infringement, competition for talent, lack of recourse when there are not humans-in-the-loop, and cost and availability of GPUs. Oh, my! Many are already successfully wading through these concerns, and we’ve seen many examples of real business value in financial services and fintech organizations ‘in the wild’ (see #5 above).

That said, all of these examples are not created equally. Through our conversations, we’ve learned how to separate out the hype from the hope from what’s actually happening, and here’s how you can, too:

• Hype: Companies tend to only have broad pronouncements around the intent to use generative AI, or the value of generative AI, or our favorite — the value of others using generative AI. Startups may include generative AI in a pitch deck, but can’t answer first order questions around the what, where, when, why, and how.
  
• Hope: Companies make announcements with specificity, such as the use case, the department, the leader, the data source, the users, and/or the partners (e.g., which LLM, which tools in the model stack, etc). Startups can articulate the components of their generative AI tech stack, and detail why they’ve selected each component, but struggle to identify why customers will choose them vs others with the same stack.
  
• Happening: Use cases actually in production, and with known and identified measures of impact, e.g., cost savings, higher pricing, new customers, time to value for new customers, cross-sell rate, etc. Startups have assembled the stack, are building the product, and have early proof of winning against similar competition due to defensible advantages.

We’ve been struck by how frequently investors and operators seem to blend these three categories, seemingly taking at face value the generative AI announcements. Just like not all use cases are created equally, not all communications about generative AI integration are equally valid (or at least not yet).

For those organizations interested in proceeding from hype to hope, where do you start? We’ve seen that financial services companies and fintechs have had the most success integrating generative AI within very specific workflows or tools that they’re already using (vs. inventing wholly new use cases). More on how to begin in #9 below.

In summary, the Gartner hype cycle is a useful way to look at the advance of new technologies, including generative AI in financial services. The skeptics say it’s all hype. The believers say it’s changing the industry forever. Both are true. More than anything, we know it’s a time to be humble: In most cases when you hear about a generative AI use case in financial services or fintech, it’s just too early to tell. We recognize that most of the home-building has to happen below ground before we see the frame rise above ground.

Return to directory

7. Generative AI largely benefits the incumbents, but there are exceptions where we expect it to be disruptive

Conversations about generative AI from a wide array of commentators routinely describe its powerful disruptive effects — e.g., BCG on March 7: “Generative AI has the potential to disrupt nearly every industry — promising both competitive advantage and creative destruction;” DataChazGPT, with close to 100,000 followers on Twitter on March 15: “The #AI disruption will happen faster than anything we’ve seen before;” Bill Gates on March 21: “Any new technology that’s so disruptive is bound to make people uneasy, and that’s certainly true with artificial intelligence;” and Goldman Sachs Asset Management on May 21: “We believe AI is a growth driver with a wide variety of applications and is on a path to disrupting entire industries as we currently know them.” But, these proclamations that generative AI is a disruptive technology in financial services are largely wrong.

If you’re only speaking to startups, it could seem that generative AI is a superpower that will enable David to topple Goliath. But, the startups are only part of the story, and we’re also speaking to the “incumbents” (here we mean both the large, established financial services companies, like Bloomberg, JPMorgan Chase and Charles Schwab, but also the established fintechs, like Stripe, AvidXchange* and Chime).

Typically, we use “disruption” to mean a radical change to the status quo, but the phrase “disruptive technologies” has a very specific meaning in business. In 1997, Clay Christensen published the “Innovator’s Dilemma,” defining the difference between “disruptive” and “sustaining” technologies. The Economist heralded the idea as “the most influential business idea of recent years.” It’s pretty clear when you return to Christensen’s original work that generative AI is primarily a sustaining technology. As Christensen defines:

• Sustaining technologies: Innovation that improves the performance of established products, along the dimensions of performance that mainstream customers in major markets have historically valued
  
• Disruptive technologies: Innovation that results in worse product performance, at least in the near-term, underperform established products in mainstream markets, but that have other features that a few (new) customers value, such as cheaper, simpler and more convenient

Generative AI is by and large a sustaining technology because incumbents can effectively integrate it into existing products to improve the performance of these products in ways that are generally accepted by customers, e.g., by providing higher quality and more responsive customer service, by tailoring marketing messages more directly to individual users, by delivering a more complete and lower cost solution that requires less human intervention, etc.

As we’re seeing generative AI is defensibly used as a component of a product or software, rather than the product itself, the advantage generally lies with the incumbents that already have distribution and know the customer. So, not surprisingly, the vast majority of instances of generative AI being used in financial services and fintechs today, as shown in #5, are in existing companies, and not brand new startups.

Generative AI as a (mostly) sustaining technology is creating what some have described as a generative AI “arms race,” where well-funded incumbents are racing to establish the best partnerships, hire the best teams and develop the best new technology built on generative AI with their proprietary data.

Disruptive technologies, on the other hand, are difficult for established companies to invest in because 1) they create simpler and cheaper products, which generally have lower margins and don’t create more profitability, 2) are commercialized in emerging or non-core markets, and 3) most profitable customers don’t want to or can’t use the products. For this reason, the rational and well-managed incumbent will explicitly ignore disruptive technologies because it would be irrational to invest in these technologies!

As Christensen puts it: “Rational managers can rarely build a cogent case for entering small, poorly defined low-end markets that offer only lower profitability.”

However, incumbents will not universally benefit: There are instances in which we expect generative AI to be disruptive within the financial services industry, and therefore are interested in companies building in these spaces. We can map out the initial outline of where it is not in the best interest of the incumbents to offer a generative AI product, specifically:

These are collectively the Achilles heel of financial services incumbents in generative AI. We expect to see new companies founded with these attributes across financial services domains. We’d love to hear from you if you’re building a generative AI-native product that takes advantage of one or more of these vulnerabilities.

Return to directory

8. Generative AI can help accelerate pre-existing trends within financial services, including open banking and embedded finance

Before generative AI came onto the scene, some of the biggest trends in fintech were embedded finance and open banking. These haven’t gone away and, in fact, generative AI further accelerates these trends.

Embedded finance is all about delivery financial services in real time and digitally. It’s key to automate the manual tasks that are part of financial services distribution, and to help increase execution speed. Specifically, generative AI can help with fraud detection and risk management, personalized user engagement, customer support and knowledge retrieval. We’re particularly excited to see how generative AI can advance the embedded financial services promise in areas that have been more resistant to the embedded wave, such as insurance.

In the case of open banking, we believe generative AI will create more utility from open banking APIs. Startups (and incumbents) now have the opportunity to prompt better, leveraging real, specific, personal data. For example, Curl founder Mike Kelly recently demonstrated on LinkedIn how easy it is to connect generative AI and open banking when he created a “Bank-GPT” plug-in, which can tell customers their balance, find transactions, discuss budgeting and make payments. Consider this the beginning!

Return to directory

9. Practical advice for companies looking to leverage generative AI within their organizations (and cautionary tales)

The worst place to start is nowhere; the best place to start is anywhere. While we need to be humble about sharing any “best practices” given how this entire field is developing as we speak, we have seen enough processes to share some practical advice. We’ve seen that how an organization begins will look different for every organization, but most simplify to a few steps:

1. Determine the use case: What is the purpose of integrating generative AI, and how will you know it’s successful? Also, identify the “why” — is it defensive competitive action? Or offensive? Is it to expand revenue? Or lower costs? We are observing that service-intense areas of the business are a natural starting point for many.
   
2. Assess your organization’s readiness for generative AI: As described in #4 above, consider your goals, budget, existing data sophistication, time and talent/skills. Based on these answers, select the appropriate model among Construct/Configure/Adopt.
   
3. Select your vendors: Review the landscape to identify who are possible partners for your use case, stage, scale, industry, and relative level of sophistication. Many companies leverage existing networks and relationships.
   
4. Build with agility: Start small. Do more of what works. Adapt or kill what doesn’t work. Ask the “why” behind everything that works or doesn’t so every experiment generates valuable learnings.
   
5. Encourage bottoms-up learnings: As you’re running through a tops-down playbook, consider inviting more creativity and experimentation from across the organization, such as through hack-a-thons and Slack channels. Celebrate creativity and emphasize a culture of new ideas.

The best way to learn what works is by engaging with others who are also on the journey. For example, at BCV we host an annual Fintech CEO Summit in addition to a Fintech Demo Day that brings together entrepreneurs (including several of our portfolio companies) building AI innovation, along with prospective partners within and across the financial services community. It should come as no surprise that AI has dominated these conversations over recent months. BCV also communes entrepreneurs sitting more squarely in the AI/ML arena; we recently launched BCV Labs, our technical community and incubator based in Palo Alto that’s focused on AI/ML innovation at both the application and infrastructure layers. Further, we’ve formed an AI/ML advisory board that meets regularly, and host events for our AI community in San Francisco and New York. (Please reach out to any of us if you’re interested in joining!)

Along the way, we’ve heard some common concerns from financial services organizations and fintechs — even among those that are currently in progress to embed generative AI. What follows is a summary of the most commonly cited concerns (albeit non-exhaustive), and some solutions (definitely non-exhaustive).

• Data privacy
  • Concern sounds like: “Generative AI systems require the processing of huge volumes of data, but this is sensitive given customer data use is sensitive to substantial rules and regulations.”
  • Solutions: (1) Construct your own LLM using proprietary data; (2) If you’re planning to configure an existing LLM, work with the LLM partner to devise a custom contract that does not require you to share back proprietary data; (3) Leverage a partner in the generative AI tooling layer that can keep your data partitioned and appropriately protected; (4) Continue to invest in your cybersecurity and compliance teams and best practices.
    
• Data context
  • Concern sounds like: “LLMs are trained on open internet data, and that’s not financial data, so they’re not going to get to good (enough) answers for our context.”
  • Solutions: (1) Prepare your data for inclusion through pre-processing and ingestion of structured data (such as with Unstructured); (2) Input processing, e.g., prompt engineering; embedding vector databases, which serves as an extended knowledge base for the LLM; (3) Fine-tuning on proprietary data adjusts the actual model parameters in the LLM (such as with Cleanlab).
    
• Hallucinations
  • Concern sounds like: “LLMs are trained on open internet data, and may not respect the basic rules of the road that are required in all of our customer interactions based on requirements from our regulator.”
  • Solutions: (1) Can leverage the regulatory guidelines themselves to create guardrails for LLM model output — think of it like adding bumpers to a bowling lane; (2) Can be done through prompt engineering or vector databases; (3) Can lean on models taking a more safety-oriented approach, e.g., Anthropic Constitutional AI or with Contextual AI*; (4) Fine-tuning may reduce but will not eliminate the risk.
    
• Black box
  • Concern sounds like: “I’m uncomfortable not understanding how the recommendations of a model are made, and my regulators are prohibitively against this. LLMs are a black box, so they’re a no-go in our environment.”
  • Solutions: (1) Can use a Chain of Thought prompting process to generate proper reasoning, or go a step further and introduce reasoning + acting, via ReAct prompts and fine-tuning (code from Langchain), or train a model under the Step by Step paradigm from OpenAI; (2) Can leverage model as a service companies, like Elemental Cognition, that provide “provably-correct” formal reasoning methods.
    
• Cost
  • Concern sounds like: “GPU costs are too expensive!”
  • Solutions: Query concatenation, caching questions (e.g., using a vector database), LLM cascading.

Let us leave you with one final idea: Doing nothing isn’t an option.

Return to directory

10. What we’re watching carefully in this space right now…

• What it takes to reach early adopters is different from what it takes to appeal to the majority. The space is growing and evolving so quickly, so the buzzy solutions are those that attract a lot of usage quickly. They become big enough to “pop” out amidst the noise. For example, even though there were many foundational research models, OpenAI’s decision to launch the B2C ChatGPT last November, which exploded in popularity at rates never before seen, was (among) the reason it’s become the best well-known model. However, playing with ChatGPT to pick out wine for dinner doesn’t mean that the sommelier at my neighborhood restaurant is going to pay for access to it. As the buzz wears off, we’re starting to see that what it takes to get the first X users isn’t necessarily the same thing that it takes to get the next X+1 user. Or that what it takes to get a user to use a new application once isn’t the same thing as it takes to make usage recurring or habitual. For founders, business leaders and investors, the implication is to be humble about declaring victory on product-market fit with a new generative AI application.

• Increased regulatory action: Just last week the EU released the draft of the EU AI Act, the world’s first comprehensive AI law. The purpose is to “make sure that AI systems used in the EU are safe, transparent, traceable, non-discriminatory and environmentally friendly. AI systems should be overseen by people, rather than by automation, to prevent harmful outcomes.” (Great research by the Stanford Center for Research on Foundation Models shows how much work there is to do to comply with these guidelines.) The U.S. is further behind in rule making, still in the hearings and announcements stage, but has similar intentions. The ways that government chooses to regulate generative AI will have broad-sweeping implications on its adoption cycle, cost, usage, and impact. As investors, we’re also aware that it will impact where value accrues within the generative AI tech stack. We know the regulatory burden will be real and complex, but unfortunately it’s too early to define its impacts.
  
• Development of domain-specific models: Domain-specific models are custom-trained LLMs that leverage domain-specific data. They are likely to be less expensive than the primary LLMs in their specific context, but also be less flexible than the primary LLMs. In the financial services context, there is a significant volume of data that is unlikely to be absorbed by the primary LLMs over time because it’s non-public, proprietary, and regulated. For this reason, we expect to see domain-specific models emerge in the financial services industry — and we’re already seeing early examples in the “Construct” category in #5 above. Some domain-specific models will be proprietary, available only to the company who constructed or configured them, while others will be available “for rent,” available to be adopted by any other company (see OpenAI’s recent exploration of an app store for AI software as a sign of things to come).
  
  What’s TBD is (1) how long it takes for this to play out — especially as a function of how quickly the key players in the tooling layer emerge, to what extent context windows can be large enough to effectively specify one of the primary LLMs with sufficient domain knowledge and obfuscate the need for domain-specific models, and how quickly companies with sufficient proprietary data build the sophistication to construct domain-specific LLMs, and (2) whether small domain-specific models can be more (or at least just as) performant as the large best-in-class models (for the specific, relevant set of use cases for which they’re built).
  
• Which layer of the generative AI tech stack is the most valuable, and most defensible? Generally speaking, there are five layers of the generative AI tech stack: 1) Infrastructure, e.g., NVIDIA GPUs, 2) Cloud platforms, 3) Foundation models (closed-source, open-source, or domain/app-specific), 4) Tooling, and 5) Apps. Each layer is changing rapidly, with evolving competitive dynamics among players driven by funding, research and go-to-market strategies. Some are more wide open than others (e.g., there are only a few players who can provide the infrastructure required, whereas there seem to be new tooling companies founded daily!) What makes it so exciting, and so uncertain, is that each layer impacts the other, and nothing is static. For example, will the advance of open-source models combined with the sophistication of the tooling layer break the supremacy of the closed source models, and transfer value in the stack from Layer 3 to Layers 4 and 5? We have early hypotheses on how this will play out differently in financial services than in other industries.

Return to directory

For anyone that wants to add more to this, or has ideas, or wants to compare notes, please get in touch! You can reach us at: shinkfuss@baincapital.com, cmelaskyriazi@baincapital.com, kzhang@baincapital.com, aco@baincapital.com, tdimitrova@baincapital.com, chowe@baincapital.com and mharris@baincapital.com.