vLLM

This is a repackaged open source software product wherein additional charges apply for cloudimg support services.

Overview

vLLM is a high-throughput, memory-efficient inference and serving engine for large language models. Its PagedAttention scheduler delivers state-of-the-art serving throughput, and it exposes an OpenAI-compatible REST API so existing OpenAI SDK code works unchanged. This image delivers vLLM fully installed and configured as a system service on an NVIDIA GPU instance, so a private, self-hosted LLM inference endpoint is running within minutes of launch. The current release available is vLLM 0.22.

GPU Accelerated

This image is built and shipped for NVIDIA Ampere+ GPU instances (g5, g6 families). The NVIDIA datacenter driver is preinstalled and verified on real hardware during the build, and vLLM auto-detects the GPU for accelerated inference. Launch on a GPU instance type and the model serves on the GPU out of the box.

Application Stack

vLLM runs in a dedicated Python virtual environment as an unprivileged service account on the loopback address, with an nginx reverse proxy fronting it on port 80. A systemd service starts the server on boot and restarts it on failure. Model weights live on a dedicated, independently resizable storage volume kept separate from the operating system disk, and a small open-weights model is pre-downloaded so the API responds immediately.

Secure By Default

vLLM's native API-key authentication is enabled. This image generates a fresh API key, unique to your instance, on its first boot and writes it to a root only file. The public health endpoint stays open for load balancers; the OpenAI-compatible inference endpoints require the key as a bearer token. No shared or default key ships in the image.

Ready To Use

Call the OpenAI-compatible endpoints from the OpenAI SDK, LangChain or LlamaIndex by pointing base_url at your instance and passing the API key. Serve a different model by editing the model name in the service environment file.

cloudimg Support

24/7 technical support by email and chat. Help with vLLM deployment, model selection, GPU sizing, throughput tuning, quantization, the OpenAI-compatible API, TLS termination and scaling.

Use Cases

High-throughput private LLM inference in your own VPC for teams with data residency or compliance requirements. A drop-in OpenAI-compatible backend for RAG and agent applications. Cost-efficient batch and online serving of open-weights models.

All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.