Why Models-as-a-Service architecture is ideal for AI models

The Models-as-a-Service (MaaS) platform leverages Red Hat OpenShift AI, Red Hat 3scale API Management, and Red Hat Single Sign-On to create a secure and scalable environment for AI models. OpenShift AI provides the foundational platform for the AI/ML lifecycle, 3Scale manages API access and security, and Red Hat SSO ensures centralized authentication and authorization. The vLLM powers model execution with its efficiency and speed. The architecture supports AI governance, zero-trust access, and hybrid cloud flexibility, creating a cohesive and high-performing ecosystem for deploying and managing AI models effectively.

Follow this series:

1. Part 1, 6 benefits of Models-as-a-Service for enterprises, is an introduction to MaaS for enterprises.
2. This article explores broad architectural details and why enterprises need MaaS.
3. Part 3 explains how to implement MaaS in an enterprise and its various components.
4. Part 4 discusses inference optimization, scalability, and security aspects for large model deployments.

The MaaS architecture solution

Building a scalable and efficient MaaS platform demands a thoughtfully constructed and resilient architecture that integrates a diverse array of critical components. This architecture is not about throwing technology together. It’s about creating a cohesive and high-performing ecosystem.

The core of this robust MaaS solution stack consists of a combination of leading-edge technologies. OpenShift AI serves as the foundation, providing a comprehensive platform for the entire AI/ML lifecycle. It handles everything from model training and development to deployment and monitoring. Complementing this is the 3Scale API Gateway, a crucial component of Red Hat 3scale API Management. The API Gateway is essential for managing access, controlling traffic, and ensuring the security of the AI models as they are exposed as services. Red Hat SSO further bolsters security by providing centralized authentication and authorization, enabling simplified and secure user access to the platform.

Powering the actual model execution is an AI inference server, specifically vLLM. The vLLM is known for its speed and efficiency, making it an excellent choice for handling the demands of real-time AI inferencing. This combination offers a complete and end-to-end solution. It brings together AI governance, ensuring that models are used ethically. 

Zero-trust access is also a key point. Every user and device is rigorously authenticated and authorized before gaining access. Hybrid cloud flexibility is another benefit, allowing the platform to operate seamlessly across different environments, whether it's on-premises, in the cloud, or a mix of both. All of this is delivered on a single, unified platform with consistent tooling, which greatly simplifies management and operations.

For a clear visual representation of the Model-as-a-Service solution, refer to the high-level architecture diagram Figure 1. This diagram provides an overview of the MaaS solution, illustrating how each component fits together and interacts to create a functional and scalable platform. 

It is important to note that Red Hat OpenShift, acting as a robust and consistent Kubernetes layer, forms the de facto platform for deploying MaaS across diverse environments (public clouds, private data centers, or at the edge). This provides the flexibility needed in modern hybrid cloud strategies. 

Hybrid cloud advantages

The MaaS architecture's reliance on a hybrid cloud model unlocks a suite of benefits for enterprises. The following advantages offer operational efficiency, security, and cost optimization:

• Unparalleled freedom of choice and portability: The MaaS platform empowers businesses with the freedom to deploy models wherever needed. This eliminates concerns about vendor lock-in and ensures that AI workloads are fully portable and adaptable to changing business requirements.
• Fortified security and policy consistency: By leveraging OpenShift as the consistent Kubernetes foundation alongside Red Hat SSO, the MaaS solution guarantees uniform security policies across all hybrid AI deployments. This ensures secure access to large language models (LLMs) regardless of their location and maintains consistent policies across all environments, providing a robust and unified security posture.
• Optimized costs and enhanced resource utilization: The MaaS approach focuses on reducing costs by centralizing model inference services and avoiding the costly duplication of resources. This model allows enterprises to offer open source models and the necessary AI technology stack as a shared resource accessible across the entire organization. Self-hosting addresses critical data privacy concerns associated with relying on third-party models, which can incur substantial expenses when deployed at scale.
• Strengthened data privacy and security: Organizations can uphold compliance with existing security, data, and privacy regulations by avoiding the use of third-party hosted models, which might inadvertently expose sensitive enterprise data to external entities.
• Scalable and granular access management: Features, such as SSO for all internal AI portals and advanced session management for regulatory compliance, facilitate scalable and fine-grained access management across distributed hybrid environments. The architecture supports multi-tenancy, logically isolating environments through shared resources, which allows for the efficient management of multiple tenants, tenant administrators, and user access to APIs and administrative portals.
• Streamlined operations and governance: IT departments gain the ability to consistently manage APIs across both cloud and on-premises environments through OpenShift integration and deploy dedicated API gateways for private LLM instances. This results in enhanced AI management and robust oversight and governance with features such as versioning and regression testing, leading to a more controlled and reliable deployment process.
• Accelerated innovation cycles: The synergy of 3Scale, Keycloak, and OpenShift AI fosters accelerated innovation through managed access and APIs. Automating the 3Scale configuration via its operator, significantly streamlines the process of deploying and exposing new models, resulting in a quicker time to market and rapid deployment of innovative AI solutions.

The integration of these components enables enterprises to develop a highly scalable and manageable MaaS platform, empowering developers to seamlessly integrate AI capabilities into their applications across a broad spectrum of diverse and distributed infrastructures. This facilitates a more agile, secure, and cost-effective approach to AI deployment and utilization.

Next up

You can refer to the detailed MaaS architecture. You may also want to review this arcade video presentation. In part 3 of this series, we'll discuss the various components of MaaS and explore how to implement it for an enterprise.