The Road to Enterprise-Scale Adoption of Generative AI: Challenges and Considerations
The road to enterprise-scale adoption of generative AI remains arduous as businesses struggle to harness its potential. While those who have moved forward with generative AI have reported improved business outcomes, with 15.8% revenue increase, 15.2% cost savings, and 22.6% productivity improvement on average, according to a Gartner survey, the reality is that 80% of AI projects in organizations fail, as noted by Rand Corporation. Additionally, Gartner’s survey found that only 30% of AI projects move past the pilot stage.
Four Key Challenges
While some companies may have the resources and expertise to build their own generative AI solutions from scratch, many underestimate the complexity of in-house development and the opportunity costs involved. The following four key challenges can thwart internal generative AI projects:
1. Safeguarding Sensitive Data
Access control lists (ACLs) – a set of rules that determine which users or systems can access a resource – play a vital role in protecting sensitive data. However, incorporating ACLs into retrieval augmented generation (RAG) applications presents a significant challenge. RAG, an AI framework that improves the output of large language models (LLMs) by enhancing prompts with corporate knowledge or other external data, heavily relies on vector search to retrieve relevant information. Unlike traditional search systems, adding ACLs to vector search dramatically increases computational complexity, often resulting in performance slowdowns.
2. Ensuring Regulatory and Corporate Compliance
In highly regulated industries like financial services and manufacturing, adherence to both regulatory and corporate policies is mandatory. This applies not only to human employees but also to their generative AI counterparts, who are playing an increasing role in both front-end and back-end operations. To mitigate legal and operational risks, generative AI systems must be equipped with AI guardrails that ensure ethical and compliant outputs, while also maintaining alignment with brand voice and regulatory requirements, such as ensuring compliance with FINRA regulations in the financial space.
3. Maintaining Strong Enterprise Security
In-house generative AI solutions often encounter significant security challenges, such as protecting sensitive data, meeting information security standards, and ensuring security during enterprise systems integration. Addressing these issues requires specialized expertise in generative AI security, which many organizations new to the technology do not possess, raising the potential for data leaks, security breaches, and compliance concerns.
4. Expanding Across Use Cases
Building a generative AI application for a single use case is relatively simple, but scaling it to support additional use cases often requires starting from square one each time. This leads to escalating development and maintenance costs that can stretch internal resources thin.
Conclusion
The issue with in-house generative AI projects is that often companies fail to see the complexities involved in data preparation, infrastructure, security, and maintenance. Scaling AI solutions requires significant infrastructure and resources, which can be costly and complex. Most organizations that run small pilots on a couple of thousand documents haven’t thought through what it takes to bring that up to scale: from the infrastructure to the types of embedding models and their cost-precision ratios.
FAQs
Q: What are the four key challenges in implementing generative AI in an organization?
A: The four key challenges are safeguarding sensitive data, ensuring regulatory and corporate compliance, maintaining strong enterprise security, and expanding across use cases.
Q: What is the success rate of AI projects in organizations?
A: According to Gartner, only 30% of AI projects move past the pilot stage, and 80% of AI projects in organizations fail.
Q: What are the benefits of using a pre-built generative AI platform?
A: A pre-built platform can provide scalability, security, and compliance, reducing the need for internal resources and expertise, and enabling organizations to focus on their core business.
