Infinity Embeddings Server

This is a repackaged open source software product wherein additional charges apply for cloudimg support services.

Overview

Infinity is a high-throughput, low-latency server for serving text embedding and reranking models. It serves models such as BGE, GTE, E5, Sentence Transformers and mixedbread with dynamic batching and an OpenAI-compatible embeddings API, so existing OpenAI SDK code works unchanged. This image delivers Infinity fully installed and configured as a system service on an NVIDIA GPU instance, so a private, self-hosted embeddings endpoint is running within minutes of launch. The current release available is Infinity 0.0.77.

GPU Accelerated

This image is built and shipped for NVIDIA GPU instances (g4dn, g5, g6 families). The NVIDIA datacenter driver is preinstalled and verified on real hardware during the build, and Infinity runs on the GPU for fast, batched embedding generation out of the box.

Application Stack

Infinity runs in a dedicated Python virtual environment as an unprivileged service account on the loopback address, with an nginx reverse proxy fronting it on port 80. A systemd service starts the server on boot and restarts it on failure. The embedding model lives on a dedicated, independently resizable storage volume kept separate from the operating system disk, and a small open-weights model is pre-downloaded so the API responds immediately.

Secure By Default

Access is gated by HTTP Basic Authentication at the nginx reverse proxy. This image generates a fresh password, unique to your instance, on its first boot and writes it to a root only file. The public health endpoint stays open for load balancers; the embedding and reranking endpoints require the password. No shared or default credentials ship in the image.

Ready To Use

Generate embeddings from the OpenAI SDK or the native API, and feed them into a vector database such as Weaviate or Chroma for retrieval augmented generation. Serve a different embedding or reranking model by editing the model name in the service environment file.

cloudimg Support

24/7 technical support by email and chat. Help with Infinity deployment, model selection, GPU sizing, batching and throughput tuning, the OpenAI-compatible API, TLS termination and scaling.

Use Cases

The embeddings backend of a private, self-hosted RAG pipeline in your own VPC. High-throughput batch embedding of documents. Reranking for search and retrieval. A drop-in OpenAI-compatible embeddings endpoint for teams with data residency or compliance requirements.

All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.