Traditional Computational Drug Discovery Relies Almost Exclusively on Highly Task-Specific Computational Models for Hit Identification and Lead Optimization

Adapting these specialized models to new tasks requires substantial time, computational power, and expertise—challenges that grow when researchers simultaneously work across multiple targets or properties.

The Rise of Generalist Models

While specialized models remain widely used, the rise of generalist models has ignited the hope that these versatile frameworks can acquire a useful amount of chemical intuition, tackling diverse drug discovery tasks and uncovering solutions and patterns that specialized models often overlook.

Introducing SAFE-GPT

The recently introduced SAFE-GPT model represented a paradigm shift in AI-driven molecular generation by introducing a chemically intuitive framework aligned with how medicinal chemists approach molecule design. By using the Sequential Attachment-based Fragment Embedding (SAFE) representation, SAFE-GPT addressed critical limitations in earlier molecular generation models to fully capture the flexibility and modularity of molecular structures. This enabled SAFE-GPT to outperform SMILES-based generative models, graph neural networks, and early fragment-based models for a variety of drug discovery-related tasks.

The Limitations of SAFE-GPT

While SAFE-GPT was transformative in its time, it has notable limitations to its efficiency, scalability, and adaptability for diverse drug discovery tasks.

Comparing SAFE-GPT and GenMol for Drug Discovery Tasks

GenMol, a recently introduced model, presents a new approach to molecular generation, addressing some of the limitations of SAFE-GPT. This article compares the strengths and weaknesses of each model, highlighting their importance for drug discovery.

SAFE Overview

The choice of molecular representation is critically important for the accuracy, efficiency, and versatility of computational models in molecular design and must align with user chemical intuition to become widely adopted.

Example GenMol Inference Code

The GenMol NIM microservice and its companion notebooks simplify inference requests by enabling you to input varying SAFE or SMILES and mask strings, with de novo generation requiring only a pure mask and the desired molecule count:

Comparing SAFE-GPT and GenMol for Drug Discovery Tasks

GenMol and SAFE-GPT represent two distinct approaches to AI-driven molecular generation, each with unique strengths and limitations (Table 1).

Molecular Generation and Exploration of Chemical Space

SAFE-GPT uses a GPT architecture with sequential, autoregressive decoding, generating molecules fragment-by-fragment. SAFE-GPT, combined with the fragment order-insensitive nature of the SAFE representation, can be applied to de novo and fragment-constrained generation of molecules.

Computational Efficiency

SAFE-GPT’s sequential generation and reliance on reinforcement learning objectives make it computationally intensive, particularly for large-scale or high-throughput scenarios.

Conclusion

The importance of these molecular generation models goes beyond just how molecular generation is done. It also explains why it needs to be reimagined.

Frequently Asked Questions

Q: What is the difference between SAFE-GPT and GenMol?

A: SAFE-GPT is a general-purpose AI model for molecular generation, while GenMol is a more specialized model designed for goal-directed lead optimization and hit generation.

Q: What are the strengths and weaknesses of SAFE-GPT and GenMol?

A: SAFE-GPT excels in motif extension and scaffold generation with strict fragment constraints, while GenMol is better suited for more flexible, goal-directed lead optimization and hit generation. Both models have their own unique strengths and weaknesses.

Q: How does GenMol improve upon SAFE-GPT?

A: GenMol improves upon SAFE-GPT by offering enhanced computational efficiency, adaptability, and task versatility, making it a more suitable choice for diverse drug discovery applications.

Q: Can I use GenMol for hit generation and lead optimization?

A: Yes, GenMol is designed for goal-directed lead optimization and hit generation, making it an excellent choice for these tasks.

Q: Can I use GenMol for motif extension and scaffold generation?

A: While GenMol can be used for these tasks, it is not as well-suited as SAFE-GPT, which excels in these areas with strict fragment constraints.

Q: How can I get started with GenMol?

A: You can start by testing GenMol as an NVIDIA NIM or exploring code examples on GitHub to learn more about using GenMol for goal-directed hit optimization, lead optimization, and more.

Q: How does GenMol compare to other molecular generation models?

A: GenMol outperforms SMILES-based generative models, graph neural networks, and early fragment-based models for diverse drug discovery-related tasks, making it a valuable addition to the field.

Q: Can I use GenMol for large-scale or high-throughput scenarios?

A: Yes, GenMol is designed to be scalable and efficient, making it suitable for large-scale or high-throughput scenarios.

Frequently Asked Questions

For two weeks in July, the All England Lawn Tennis Club (AELTC) hosts Wimbledon, the most prestigious tournament in the sport. IBM has been partnering with the Club for more than three decades, enhancing coverage of The Championships and engaging fans with rich data-driven insights. This year, some of the most compelling stories of the tournament will be told with the help of IBM watsonx, the enterprise-ready generative AI platform.

How watsonx keeps Wimbledon fans up to date

The new Catch Me Up feature, available in the Wimbledon app and website, provides AI-generated summaries for every singles player—describing what happened in their last match and the challenges they face in their next one. Users can personalize their Catch Me Up list by adding their favorite players. If matches are live or video highlights are available, Catch Me Up provides relevant links.

Along with showing app users the match results of their favorite players, the Catch Me Up feature factors in recency, geolocation, and rankings to present additional relevant content. It also generates a summary card, capturing major storylines, highlights, and previews of upcoming matches.

Behind-the-scenes benefits

The Catch Me Up feature saves the Wimbledon editorial team hours of time and frees them up to write more stories and manage higher-value tasks. Behind the scenes, the process begins by collecting a huge volume of data from player rankings and momentum to games and sets played. This structured and unstructured data is managed by IBM watsonx.data, a data store built on an open lakehouse architecture that enables flexible, scalable access by applications across the Club’s hybrid cloud servers.

To generate the Catch Me Up summaries, the team uses a powerful, open-source large language model (LLM) called IBM Granite, which was trained on thoroughly reviewed, fully transparent, enterprise-grade data. The model boasts 13 billion parameters and is optimized for dialogue use cases such as virtual agents or chatbots. Using IBM watsonx.ai — a next-generation studio for building and training generative AI models for business use cases — the model was tuned with trusted Wimbledon data, and trained to generate stories in Wimbledon’s unique editorial style and tone of voice. The entire workflow is managed and monitored using IBM watsonx.governance to deliver reliable, integrated performance.

Enhancing fan-favorite features through ongoing collaboration

Wimbledon stays at the forefront of sporting event technology through its collaboration with IBM Consulting. Each year, teams from IBM and the AELTC collaborate and co-create (using the IBM Garage methodology) to bring the beauty and excitement of the tournament to life for millions of fans all over the world.

A shining example is the IBM SlamTracker. “SlamTracker has been where you get your scores and stats for almost 20 years,” says Sidell. “Over the last 18 months, we’ve been working with Wimbledon to create a new SlamTracker experience. We’re showcasing generative AI capabilities on top of it—some of the snippets you see as part of the player cards will now also live within SlamTracker, with three bullet points for match previews and recaps, generated by watsonx.”

This year, Wimbledon’s implementation will also enhance SlamTracker with new forms of head-to-head data including past matchup information, and improved server-receiver visualizations.

An enterprise-wide technology roadmap

What’s next on the technology roadmap? “This year, we’re doing a proof-of-concept with multimodal AI models—that is, a variety of computer vision models that understand both videos and images—so we can translate that into text,” says Sidell. “Our goal is to capture more of what happens between each point, during changeovers, and between play to create more contextual narratives around the players.” So perhaps next year, watsonx will help describe injury timeouts, weather delays, and other play interruptions such as interactions with umpires, ball girls and ball boys, and activity in the Royal Box.

The collaboration between IBM Consulting and the Wimbledon teams extends beyond the fan-facing digital platform, into enterprise-wide transformation. “Some of Wimbledon’s sustainability leads saw a demo of the IBM Envizi platform, showing how it could improve the monitoring and reporting of their carbon emissions and energy consumption data,” says Sidell. “That led to us implementing an instance of Envizi for the AELTC to monitor the tournament this year.” As the sole grand slam tournament played on natural grass, it seems only fitting that Wimbledon should stay ahead of the curve on environmental concerns.

Conclusion:
The collaboration between IBM and Wimbledon has resulted in a more engaging and informative experience for fans, while also providing valuable insights and time-saving benefits for the editorial team. The use of IBM watsonx has enabled the creation of AI-generated summaries, improved the SlamTracker experience, and laid the groundwork for future innovations.

FAQs:

Q: What is IBM watsonx?
A: IBM watsonx is an enterprise-ready generative AI platform that enables the creation of AI-generated summaries, stories, and narratives.

Q: What is the Catch Me Up feature?
A: The Catch Me Up feature is an AI-generated summary for every singles player, describing what happened in their last match and the challenges they face in their next one.

Q: What is the IBM SlamTracker?
A: The IBM SlamTracker is a fan-favorite feature that provides scores and stats for the tournament. This year, it will be enhanced with new forms of head-to-head data and improved server-receiver visualizations.

Q: What is the future roadmap for IBM and Wimbledon?
A: The future roadmap includes the development of multimodal AI models that can understand both videos and images, and the implementation of the IBM Envizi platform for monitoring and reporting of carbon emissions and energy consumption data.

Challenges and Best Practices for Building Large Language Models

In recent years, large language models (LLMs) have achieved extraordinary progress in areas such as reasoning, code generation, machine translation, and summarization. However, despite their advanced capabilities, foundation models have limitations when it comes to domain-specific expertise such as finance or healthcare or capturing cultural and language nuances beyond English.

Overcoming Limitations

Overcoming those limitations can be achieved with further development using continued pretraining (CPT), instruction fine-tuning, and retrieval-augmented generation (RAG). This requires high-quality, domain-specific datasets, a robust AI platform (software and hardware stack), and advanced AI expertise.

iGenius and Colosseum 355B

iGenius, an Italian technology company specializing in artificial intelligence for enterprises operating in highly regulated sectors, such as financial services and public administration, aimed to develop a state-of-the-art foundational LLM within a tight timeline. They faced challenges in accessing large-scale GPU clusters (thousands of GPUs) and securing support for highly scalable training frameworks.

Collaboration with NVIDIA

During this engagement, iGenius developed the Colosseum 355B LLM, designed and developed for highly regulated environments, which provides businesses with confidence in the accuracy of the model output and security, knowing none of their information or IP is ever compromised. iGenius chose to collaborate with NVIDIA to accelerate their LLM development for Colosseum 355B.

DGX Cloud Environment

Improving LLM reasoning capabilities requires a robust, distributed hardware and software solution, where accelerated compute, networking, storage, and libraries must seamlessly work together. Any bottleneck in the system can significantly slow or even stop the entire training process. NVIDIA DGX SuperPOD negates the risk and complexity of this by providing a fully optimized solution that is designed, built, and validated by NVIDIA before handing over a ready-to-go system to their customers.

CPT and Alignment

iGenius used large-scale CPT and alignment on specific domains to build Colosseum 355B, a foundational LLM developed using NVIDIA DGX Cloud infrastructure and NVIDIA AI Enterprise software with NVIDIA NeMo Framework. They reduced computational costs and improved efficiency through CPT in FP8 precision.

Challenges and Best Practices

As training scales, minor issues become critical. Running the Colosseum 355B training job on 3K GPUs can take 15–20 minutes just to load the checkpoint, which consumes 5 TB of memory footprint. A storage system performing well with multiple small jobs across thousands of GPUs may struggle when a single workload requires all GPUs to read and write checkpoints simultaneously, causing delays and potential timeouts.

Best Practices

Here are some best practices and lessons learned when running LLM training at scale:

• Explore the basics at a reduced scale
• Monitor effectively and track at scale
• Explore the basics at reduced scale
• Progressive scaling is key
• Small debug model: Exploring large-scale training configurations is highly challenging
• End-to-end process testing
• Robust checkpointing
• Expansions from minimal to large-scale distribution
• Dataset testing
• Monitor effectively and track at scale
• Accurate experiment tracking
• Infrastructure observability
• Predefined tests

Conclusion

By using large-scale CPT and alignment on specific domains, iGenius built Colosseum 355B, a foundational LLM developed using NVIDIA DGX Cloud infrastructure and NVIDIA AI Enterprise software with NVIDIA NeMo Framework. iGenius reduced computational costs and improved efficiency through CPT in FP8 precision.

FAQs

Q: What are the limitations of foundation models?
A: Foundation models have limitations when it comes to domain-specific expertise such as finance or healthcare or capturing cultural and language nuances beyond English.

Q: How can these limitations be overcome?
A: These limitations can be overcome with further development using continued pretraining (CPT), instruction fine-tuning, and retrieval-augmented generation (RAG).

Q: What is Colosseum 355B?
A: Colosseum 355B is a foundational LLM developed by iGenius using NVIDIA DGX Cloud infrastructure and NVIDIA AI Enterprise software with NVIDIA NeMo Framework.

Q: What are the best practices for running LLM training at scale?
A: The best practices include exploring the basics at a reduced scale, monitoring effectively and tracking at scale, exploring the basics at reduced scale, progressive scaling, and more.