Self-hosted llama.cpp Server on Ubuntu 24.04 LTS by cloudimg. Upstream llama.cpp's own single-binary CPU inference engine - a built-in browser chat UI plus an OpenAI-compatible REST API (/v1/chat/completions), no GPU required. Bundled with a small redistributable demo model (Qwen2.5-1.5B-Instruct, Apache-2.0) ready to chat with on first boot. nginx on port 80 with a per-VM HTTP Basic Auth password generated on first boot. MIT licensed.
llama.cpp is the reference CPU-first inference engine for GGUF-quantized large language models - no GPU required. This appliance installs upstream's own llama-server binary (the official prebuilt CPU release, not a third-party wrapper), which serves two things from a single process: a built-in browser chat UI and an OpenAI-compatible REST API (/v1/chat/completions, /v1/completions, /v1/models). llama-server has no login system of its own, so the cloudimg image locks it down for a marketplace appliance: llama-server is bound to loopback only, and nginx does all customer-facing work on port 80 - a per-VM HTTP Basic Auth gate protects both the chat UI and the API, and an unauthenticated /healthz endpoint serves load balancer probes. A small, genuinely redistributable demo model - Qwen2.5-1.5B-Instruct, Apache License 2.0 - is pre-downloaded onto a dedicated data disk so you can start chatting or calling the API within a minute of first boot, with no internet access required at runtime. Swap in a larger GGUF model at any time by dropping a new file onto the data disk and editing the systemd unit - documented step by step in the user guide. Licensing: MIT (llama.cpp) + Apache-2.0 (bundled demo model), FREE - no per-request or per-seat fee. Backed by 24/7 cloudimg support.