oMLX and local LLMs with Posit Assistant in Positron - Getting Started & Configuration

Posit Assistant
1

Hi.

I'm kicking the tyres of running local LLMs on my Macbook Pro (M4Pro 24Gb RAM) and trying oMLX with Gemma 4-12b model, or Qwen2.5. I believe that oMLX is a preferred way to run models locally on Macs. A little web searching shows that oMLX works like OpenAI and serves models at https://localhost:8000/v1. I've tried signing in with my API key in Posit Assistant, but I'm having trouble seeing the running model when I get there.

I'd welcome any tips to get started...
Mike

2

Hi there, @MikeKSmith!

I was able to get oMLX to work by using http://127.0.0.1:8000/v1 as the base URL.

Here is my Positron info:

Positron Version: 2026.08.2 build 4
Code - OSS Version: 1.124.0
Commit: 747dd536c5913adad71a568a57679bb2c5e5cd71
Date: 2026-08-20T19:13:05Z
Electron: 42.2.0
Chromium: 148.0.7778.97
Node.js: 24.15.0
V8: 14.8.178.14-electron.0
OS: Darwin arm64 25.6.0

I couldn't get Gemma 4-12b to run (I'd get an HTTP 507 error, documented here: omlxurus/docs/TROUBLESHOOTING.md at main · deesatzed/omlxurus · GitHub), but I was able to get Qwen2.5-7B-Instruct to run.

I hope that that helps! Let us know.