OpenAI gpt-oss:20b runs on a Mac mini

The gpt-oss:20b model, which has about 21 billion parameters, fits on 12 GB of GPU memory and can be run without a Docker container. The author measured a total processing time of 16.28 seconds, an input speed of 260.42 tokens/second and a response speed of 38.80 tokens/second. The model can also be accessed through a browser interface via Open WebUI, which the author installed with Docker.

The post highlights the benefits of local inference—avoiding cloud API fees, reducing data leakage risk, and allowing custom integration through Python or HTTP APIs. It also notes that the model can be combined with RAG for document‑based queries, though the author has not yet set up the Open WebUI RAG API.