RStudio + LM Studio (local model): Posit Assistant context window stuck at 4096 regardless of LM Studio config

Posit AI: Feedback & Discussions
,
2

Follow-up for anyone hitting the same issue: I got it working. Posting the solution here since this configuration path isn't in the official docs yet.

Root cause: Posit Assistant doesn't auto-detect the actual context window for custom/local providers like LM Studio — it falls back to hardcoded defaults (4096 context / 2048 output tokens) unless you explicitly declare the model's capabilities yourself, with models.discovery disabled so it doesn't override your settings.

The fix: edit ~/.posit/ai/providers.json and declare LM Studio as a custom provider (not the built-in lmstudio provider type at the top level), with an explicit model list:

{
  "providers": {
    "custom": {
      "my-gateway": {
        "type": "lmstudio",
        "baseUrl": "http://localhost:1234/v1",
        "models": {
          "discovery": "off",
          "custom": [
            {
              "id": "<identifier>",
              "name": "<name>",
              "maxContextLength": <contextlength>,
              "supportsTools": true,
              "supportsImages": false,
              "supportsToolResultImages": false,
              "supportsWebSearch": false
            }
          ]
        },
        "endpoint": "http://localhost:1234/v1"
      }
    }
  }
}
  • <identifier> = the exact "id" field returned by curl http://localhost:1234/api/v0/models
  • <name> = whatever display name you want
  • <contextlength> = the context length you set for the model in LM Studio (check with the same curl command, field loaded_context_length)

Steps:

  1. Edit ~/.posit/ai/providers.json as above.
  2. Restart Posit Assistant: close the Posit Assistant pane, then Session → Restart R, and reopen the pane.
  3. Go to Configure LLM Providers, select my-gateway, and click Configure.

A few things worth noting for anyone debugging this themselves:

  • The positron.assistant.models.overrides.<provider> setting documented for Positron/VS Code does not apply here — it's a different settings mechanism.
  • The field names differ too: it's maxContextLength here, not maxInputTokens/maxOutputTokens.
  • curl http://localhost:1234/api/v0/models (LM Studio's own API) and LM Studio's Developer Logs were very useful for confirming that the actual LLM backend had no problem handling large contexts — the limit really was coming from Posit Assistant's model metadata, not from LM Studio itself.

Credit to GaryR for the syntax, found in the posit-ai.pdf doc draft shared here.

If anyone finds a simpler or more elegant way to do this, I'd love to hear it!