serving

FreeToken

Serving engine that aims to run massive models locally on a desktop (Apache-2.0).

Open on GitHub → More in Local Models & OCR

Listed by the community and not security-audited. Read the repo before you install it.

More in Local Models & OCR

airllm Python library for running large LLM inference on a single 4GB GPU (Apache-2.0).… colibri Hierarchical inference engine in C for large mixture-of-experts models: hot experts on GPU (CUDA, Vulkan), war… Unlimited-OCR Baidu's OCR model for one-shot, long-horizon document parsing (MIT).…