jundot

omlx

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

Trusted Project Python
13.8k stars via trending-python
View on GitHub
← Back to Trending Repos