Hierarchical inference engine in C for large mixture-of-experts models: hot experts on GPU (CUDA, Vulkan), warm in RAM, cold streamed from NVMe.
Listed by the community and not security-audited. Read the repo before you install it.
Hierarchical inference engine in C for large mixture-of-experts models: hot experts on GPU (CUDA, Vulkan), warm in RAM, cold streamed from NVMe.
Listed by the community and not security-audited. Read the repo before you install it.