Base
Hardened build
minimal ports, security patches applied at build time
vLLM
AWS
Azure
Artificial Intelligence (AI)
2 variations
vLLM, a high throughput OpenAI compatible LLM inference engine, GPU accelerated and secured by an API key on first boot.
Variations
Choose the operating system, cloud and version you need. Every variation is the same hardened vLLM image, built and supported by cloudimg.
Overview
vLLM is a high throughput inference and serving engine for large language models with an OpenAI compatible API, built around PagedAttention for efficient GPU memory use. It serves model completions to applications at scale.
Why the cloudimg image
The cloudimg image is hardened and fully patched with vLLM running as a service behind an nginx reverse proxy on port 80 and the NVIDIA datacenter driver preinstalled and verified on real hardware for GPU inference on Ampere and newer instances. A unique API key is generated on first boot and stored in a root only file, with 24/7 cloudimg support included.
Common uses
- LLM inference serving
- OpenAI compatible APIs
- GPU accelerated AI
More in Artificial Intelligence (AI)
AnythingLLM
Artificial Intelligence (AI)
An all in one document chatbot and RAG workspace, ready to ingest and answer over your own files on launch.
Di
Dify
Artificial Intelligence (AI)
Dify, a source available platform for building LLM apps, agents, RAG and workflows from a visual console.
Flowise
Artificial Intelligence (AI)
A visual low code builder for AI agents and LLM workflows, ready to run behind a unique login on first boot.
In
Infinity
Artificial Intelligence (AI)
Serve text embeddings and reranking privately, behind one API with a per VM credential and no default login.
Ie
Infinity Embeddings
Artificial Intelligence (AI)
The Infinity GPU server for text embeddings and reranking, with an OpenAI compatible API on port 80.
Ls
Label Studio
Artificial Intelligence (AI)
The Label Studio data labeling platform for machine learning, secured behind nginx with per instance credentials.