vmlx
installs/mo · pypistarsforks
How to install vmlx?
jjang-ai/vmlx
$git clone https://github.com/jjang-ai/vmlx---
README.md
| 1 | <p align="center"> |
| 2 | <picture> |
| 3 | <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/jjang-ai/vmlx/main/assets/logo-wide-dark.png"> |
| 4 | <source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/jjang-ai/vmlx/main/assets/logo-wide-light.png"> |
| 5 | <img alt="vMLX" src="https://raw.githubusercontent.com/jjang-ai/vmlx/main/assets/logo-wide-light.png" width="400"> |
| 6 | </picture> |
| 7 | </p> |
| 8 | |
| 9 | <h3 align="center">MLX Inference Server for Apple Silicon</h3> |
| 10 | |
| 11 | <p align="center"> |
| 12 | Self-hosted inference server for LLMs, VLMs, and image generation on Apple Silicon.<br> |
| 13 | OpenAI + Anthropic + Ollama compatible HTTP API. Self-hosted; no third-party API keys required.<br> |
| 14 | Native MTP artifact detection and family-specific cache policy gates keep speculative/cache settings explicit and model-safe. |
| 15 | </p> |
| 16 | |
| 17 | <p align="center"> |
| 18 | <em>Looking for a native Swift macOS app or Swift inference engine? See <a href="https://osaurus.ai">osaurus.ai</a>.</em> |
| 19 | </p> |
| 20 | |
| 21 | <p align="center"> |
| 22 | <a href="https://pypi.org/project/vmlx/"><img src="https://img.shields.io/pypi/v/vmlx?color=%234B8BBE&label=PyPI&logo=python&logoColor=white" alt="PyPI" /></a> |
| 23 | <a href="https://github.com/jjang-ai/vmlx/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-Apache_2.0-green?logo=apache" alt="License" /></a> |
| 24 | <a href="https://github.com/jjang-ai/vmlx"><img src="https://img.shields.io/github/stars/jjang-ai/vmlx?style=social" alt="Stars" /></a> |
| 25 | <img src="https://img.shields.io/badge/Apple_Silicon-M1%2FM2%2FM3%2FM4-black?logo=apple" alt="Apple Silicon" /> |
| 26 | <img src="https://img.shields.io/badge/Python-3.10+-3776AB?logo=python&logoColor=white" alt="Python" /> |
| 27 | <img src="https://img.shields.io/badge/Electron-28-47848F?logo=electron&logoColor=white" alt="Electron" /> |
| 28 | <a href="https://ko-fi.com/jangml"><img src="https://img.shields.io/badge/Support-Ko--fi-FF5E5B?logo=ko-fi&logoColor=white" alt="Ko-fi" /></a> |
| 29 | </p> |
| 30 | |
| 31 | <p align="center"> |
| 32 | <a href="#quickstart">Quickstart</a> • |
| 33 | <a href="#model-support">Models</a> • |
| 34 | <a href="#features">Features</a> • |
| 35 | <a href="#image-generation--editing">Image Gen</a> • |
| 36 | <a href="#api-reference">API</a> • |
| 37 | <a href="#desktop-app">Desktop App</a> • |
| 38 | <a href="#advanced-quantization">JANG</a> • |
| 39 | <a href="#cli-commands">CLI</a> • |
| 40 | <a href="#configuration">Config</a> • |
| 41 | <a href="#contributing">Contributing</a> • |
| 42 | <a href="#한국어-korean">한국어</a> |
| 43 | </p> |
| 44 | |
| 45 | --- |
| 46 | |
| 47 | > **JANG 2-bit destroys MLX 4-bit on [MiniMax M2.5](https://huggingface.co/JANGQ-AI/MiniMax-M2.5-JANG_2L):** |
| 48 | > |
| 49 | > | Quantization | MMLU (200q) | Size | |
| 50 | > |---|---|---| |
| 51 | > | **JANG\_2L (2-bit)** | **74%** | **89 GB** | |
| 52 | > | MLX 4-bit | 26.5% | 120 GB | |
| 53 | > | MLX 3-bit | 24.5% | 93 GB | |
| 54 | > | MLX 2-bit | 25% | 68 GB | |
| 55 | > |
| 56 | > Adaptive mixed-precision keeps critical layers at higher precision. Scores at [jangq.ai](https://jangq.ai). Models at [JANGQ-AI](https://huggingface.co/JANGQ-AI). |
| 57 | |
| 58 | <table align="center"> |
| 59 | <tr> |
| 60 | <td align="center"><img src="https://raw.githubusercontent.com/jjang-ai/vmlx/main/assets/chat-tab.png" width="500" alt="Chat interface" /></td> |
| 61 | <td align="center"><img src="https://raw.githubusercontent.com/jjang-ai/vmlx/main/assets/agentic-chat.png" width="500" alt="Agentic coding chat" /></td> |
| 62 | </tr> |
| 63 | <tr> |
| 64 | <td align="center"><em>Chat with any MLX model -- thinking mode, streaming, and syntax highlighting</em></td> |
| 65 | <td align="center"><em>Agentic chat with full coding capabilities -- tool use and structured output</em></td> |
| 66 | </tr> |
| 67 | </table> |
| 68 | |
| 69 | --- |
| 70 | |
| 71 | ## Quickstart |
| 72 | |
| 73 | ### Install from PyPI |
| 74 | |
| 75 | Published on [PyPI as `vmlx`](https://pypi.org/project/vmlx/) -- install and run in one command: |
| 76 | |
| 77 | ```bash |
| 78 | # Recommended: uv (fast, no venv hassle) |
| 79 | brew install uv |
| 80 | uv tool install vmlx |
| 81 | vmlx serve mlx-community/Qwen3-8B-4bit |
| 82 | |
| 83 | # Or: pipx (isolates from system Python) |
| 84 | brew install pipx |
| 85 | pipx install vmlx |
| 86 | vmlx serve mlx-community/Qwen3-8B-4bit |
| 87 | |
| 88 | # Or: pip in a virtual environment |
| 89 | python3 -m venv ~/.vmlx-env && so |
People who liked this also viewed
6 picksType
- mcp-proxy-for-awsAWS MCP Proxy ServerMCP ServersJul 2026
- pydantic-ai-backendFile Storage & Sandbox Backends for Pydantic AI: console tools for file operations, Docker-isolated sandboxes for safe execution, and permission system with presets for access control. Enables secure multi-user handling and testing in agents via in-memory, local, or containerized storage.MCP ServersJul 2026
- mcp-for-argocdAn implementation of Model Context Protocol (MCP) server for Argo CD.MCP ServersJul 2026
- heroku-mcp-serverHeroku Platform MCP Server using the Heroku CLIMCP ServersJul 2026
- ghost-mcpA Model Context Protocol (MCP) server for interacting with Ghost CMS through LLM interfaces like Claude. Allow you to control your Ghost blog by simply asking Claude etc.MCP ServersApr 2026
- Google DriveUpload, organize, and share files in the cloud. Manage folders, set permissions, and search across stored documents.MCP Serversrecently
