Red Hat Expands AI Capabilities with New Linux Platform for Large Language Models -- Pure AI

Red Hat Expands AI Capabilities with New Linux Platform for Large Language Models

By John K. Waters
12/06/2024

Red Hat, one of the world's leading providers of open-source solutions and a subsidiary of IBM, has announced the general availability of the latest iteration of its Red Hat Enterprise Linux AI (RHEL AI), a specialized version of its flagship Linux platform tailored for building, testing, and deploying artificial intelligence (AI) models, including large language models (LLMs).

The RHEL AI 1.3 release introduces significant advancements, including support for the newly open-sourced Granite 3.0 LLMs, expanded data preparation tools, and enhanced capabilities for hybrid cloud deployments. It also incorporates support for Intel’s Gaudi 3 AI accelerator, positioning Red Hat as a key player in enabling enterprise AI innovation.

Building on a trend highlighted by the industry watchers at IDC, which found that 61% of enterprises plan to adopt open-source foundation models for generative AI, Red Hat is emphasizing tools and features to support smaller, open-source-licensed LLMs. The platform includes fine-tuning and inference optimization features that cater to enterprises looking for customizable and efficient AI

"Red Hat Enterprise Linux AI continues to be at the forefront of supporting enterprise AI development, providing organizations with flexible, open-source tools to build and optimize generative AI models," the company said in a statement.

Central to this release is the Granite 3.0 8b model, an 8-billion-parameter converged model supporting over a dozen natural languages, code generation, and function-calling capabilities. These features are currently available in developer preview and will be expanded in future releases. RHEL AI 1.3 also integrates InstructLab, an IBM initiative aimed at accelerating open-source generative AI contributions, enhancing RHEL’s utility for enterprises.

The platform now supports Docling, an open-source project from IBM Research designed to simplify document processing. Docling converts formats such as PDFs and slide decks into developer-friendly formats like Markdown, easing data ingestion for model fine-tuning. The tool’s context-aware chunking capabilities improve the coherence and relevance of AI-generated responses.

RHEL AI 1.3 extends its support to Intel’s Gaudi 3 accelerator, which boasts up to 2.3 times the power efficiency of NVIDIA’s H100 GPU, while also promising faster LLM training. The release builds on Red Hat’s existing support for GPUs from NVIDIA and AMD, offering enterprises diverse hardware options.

Red Hat OpenShift AI, a Kubernetes-native platform for AI applications, now includes support for vLLM runtimes. This upgrade enables parallelized serving across multiple nodes, allowing LLMs to handle multiple real-time requests while optimizing performance through sharding or quantization.

As enterprises increasingly turn to open-source models for their AI strategies, Red Hat’s RHEL AI aims to deliver an efficient and scalable foundation for generative AI development. The platform is expected to attract organizations seeking robust, cost-effective solutions for deploying LLMs across cloud and on-premises environments.

The latest release underscores Red Hat’s commitment to driving innovation in AI infrastructure while leveraging IBM’s resources to remain competitive in the rapidly evolving AI landscape, the company says.

About the Author

John K. Waters is the editor in chief of a number of Converge360.com sites, with a focus on high-end development, AI and future tech. He's been writing about cutting-edge technologies and culture of Silicon Valley for more than two decades, and he's written more than a dozen books. He also co-scripted the documentary film Silicon Valley: A 100 Year Renaissance, which aired on PBS. He can be reached at [email protected].