LM Studio started life as a desktop chat app, but since version 0.4.0 (announced January 28, 2026) it also runs without a window. A setup guide published October 30, 2026 lists 0.4.27 (released October 29, 2026) as the newest confirmed build, and every build serves an OpenAI-compatible API at http://localhost:1234/v1 with no authentication by default. This tutorial turns that default into a headless, token-protected LM Studio server that scripts, editors and other machines on your network can call.
The timing matters because developers now treat model endpoints as plumbing. Stack Overflow’s 2025 developer survey press release put it plainly: “31% of developers reported that they currently use AI agents.” Agents hammer an endpoint far harder than a person typing into a chat box, so a local server needs to survive concurrent requests, restarts and a few careless clients.
If you still need the desktop basics (installing the app, picking a first model), our LM Studio setup walkthrough covers them. This guide starts where that one stops. You will install the lms command-line tool, run the headless daemon, download and load a model from the terminal, call the API from curl, Python and Node.js, add structured output and tool calling, lock the server behind an API token, keep it alive with systemd, and finish with a smoke-test script you can drop into any project.
Why Run LM Studio Headless Instead of Through the Desktop App
A desktop app is fine for chatting. It is a poor fit for a server. It needs a logged-in session, it grabs GPU memory for the interface, and it dies when someone closes the lid. LM Studio 0.4.0 changed that. According to the official release notes, it added parallel requests with continuous batching, a non-GUI deployment option, a stateful REST API and a redesigned interface. The llama.cpp engine bundled with it graduated to version 2.0.0 in the same release, which brought support for concurrent inference requests to the same model.
Those three pieces (headless mode, concurrency, a scriptable CLI) are what make LM Studio usable as shared infrastructure. One mini PC or a spare workstation with a decent GPU can serve your whole team’s editors and scripts. The model weights never leave your hardware, and the per-token bill drops to electricity.
Be honest about the cost math, though. OpenAI’s pricing page lists GPT-6 Luna at $0.10 per million input tokens and $0.50 per million output tokens. At those rates a light workload can be cheaper on a hosted API than on a GPU you have to buy and power. Local serving wins when the data cannot leave the building, when you run thousands of long agent loops a day, or when you want a fixed, predictable cost. Gartner forecast worldwide generative-AI spending of “$644 billion in 2025, representing a 76.4% increase from 2024”, and part of that spend goes to metered API usage that a local server can absorb for some workloads.
Two clarifications before you start. First, OpenAI-compatible means the request and response shapes match. It does not mean OpenAI’s models run on your machine. The official results confirm that identifiers such as gpt-6-sol belong to OpenAI’s hosted API, and nothing shows they can be downloaded into LM Studio. Second, LM Studio now has sibling products. LM Studio Bionic, announced July 16, 2026, is an agent for open models that can use local models, LM Link or LM Studio Secure Cloud, and cloud billing needs an account. None of that is required here. Everything in this tutorial runs locally.
By the end you will have a server with these properties:
- It starts at boot and loads a named model without anyone logging in.
- It answers on the OpenAI-compatible routes, so existing SDK code works after a one-line URL change.
- It returns schema-valid JSON and tool calls when the model supports them.
- It rejects requests that lack a bearer token.
- It reports its own health through a script you can schedule.
If you are still deciding between runtimes, our comparison of Ollama, LM Studio and Jan lays out the trade-offs. Pick LM Studio for this walkthrough if you want a model catalog, a polished CLI and a GUI you can fall back on.
Prerequisites and Versions
You need a machine that stays on, a terminal, and about an hour. The table lists the tested baseline. Newer minor versions should work, but keep LM Studio at 0.4.27 or later so the commands below match.
| Component | Version or spec | Why you need it |
|---|---|---|
| LM Studio | 0.4.27 or newer (October 29, 2026 build confirmed) | Provides the lms CLI, the llmster daemon and the API server |
| OS | Ubuntu 24.04 LTS, macOS 15, or Windows 11 | The headless steps use Linux, the rest works on all three |
| Python | Not listed | Runs the client examples and the smoke test |
| openai (Python SDK) | Latest 1.x release | Talks to the OpenAI-compatible endpoints |
| Node.js | 22 LTS | Runs the JavaScript client example |
| RAM | 16 GB minimum, 32 GB comfortable | Weights, KV cache and the OS all share it |
| GPU (optional) | NVIDIA, AMD or Apple Silicon | Speeds up generation, CPU-only works for small models |
| Disk | 20 GB free | Room for two or three quantized models |
Check the basics from a terminal before you go further.
python3 --version
node --version
nvidia-smi --query-gpu=name,memory.total --format=csv # NVIDIA only
free -h # Linux RAM check
df -h ~ # free disk spaceA word on licensing. A secondary guide reports that LM Studio dropped its separate commercial-license requirement on July 8, 2025, so the old claim that it is free for personal use only is outdated. That report is not an official statement, so read the current terms on the vendor’s site before you deploy it for a company. Model licenses are a separate matter. Each model carries its own license, and some limit commercial use.
Pick a Model Size Your Hardware Can Actually Hold
Most failed local setups come from one mistake: loading a model that does not fit. Memory use has three parts. The first is the weights, which you can estimate as parameters times bits per weight divided by eight. The second is the KV cache, which grows with context length and with the number of parallel requests. The third is overhead from the runtime and your operating system. The table shows the weight arithmetic for the three Ministral 3 sizes, which LM Studio has supported since version 0.3.33.
| Model size | Weights at 4-bit | Weights at 8-bit | Realistic RAM or VRAM target |
|---|---|---|---|
| 3B | about 1.5 GB | about 3 GB | 8 GB machine |
| 8B | about 4 GB | about 8 GB | 16 GB machine |
| 14B | about 7 GB | about 14 GB | 24 GB machine or 16 GB GPU |
Treat the last column as a planning number, not a promise. It leaves headroom for context and for whatever else the machine runs. Official LM Studio material says the distilled DeepSeek R1 0528 model at 8B runs on as little as 4 GB of RAM, which fits the 4-bit arithmetic above, but long contexts will push you past that floor quickly.
Quantization deserves a few sentences of its own. A 4-bit file trades a small quality loss for a model roughly half the size of the 8-bit version, which is why most local users start there. If your task is extraction, classification or short summaries, the loss is usually invisible. If your task is long-form reasoning or code generation, test an 8-bit build of the same model against the 4-bit one on ten of your own prompts and keep the smaller file only if the answers hold up. The KV cache is the other lever. It scales with context length, so halving the context from 16,384 to 8,192 tokens frees real memory without touching the weights.
Larger catalog entries show where the ceiling sits. LM Studio’s own announcement reportedly estimated that DeepSeek V4 Flash 0731, a mixture-of-experts model under the MIT License, needs at least 156 GB of system memory. A listing for NVIDIA’s Nemotron 3.5 Bolt 74B-A9B describes a 2-bit vision-language model built to fit in roughly 24 GB. Both come from single sources, so treat the figures as indicative. The lesson is that mixture-of-experts models are cheaper per token than their parameter count suggests but still demand memory for every expert.
For this tutorial pick an 8B model if you have 16 GB, or a 3B model if you have 8 GB. Both give you fast feedback while you build the server. You can swap in something larger once the plumbing works. Our ranking of open-source LLMs and the wider AI and machine learning coverage track which open-weight models are worth the disk space this quarter.
Install LM Studio and the lms Command-Line Tool
Step 1: Install the App and Bootstrap lms
Download LM Studio from the vendor’s site and install it as you would any desktop app. On macOS and Windows that is all you need to do for a workstation. The lms CLI ships with the app and lives in the LM Studio folder inside your home directory. On first run you may need to put it on your PATH.
# Linux/macOS: add the CLI to PATH for this shell
export PATH="$HOME/.lmstudio/bin:$PATH"
# One-time: register lms in your shell profile (LM Studio must have run once)
lms bootstrap
# Confirm the CLI works
lms --help
lms versionlms --help prints every subcommand your build supports. Keep that command handy, because flags change between releases and the help text is the authoritative list for your version. If lms reports command not found after bootstrapping, open a new terminal so the profile reloads.
Step 2: Install the Headless Daemon (llmster)
The desktop app is optional on a server. The headless daemon, called llmster, runs the same engine without the interface. The vendor documents a one-line installer for Linux and macOS and a PowerShell equivalent for Windows. Read the script before you pipe it into a shell, and confirm the current command in the developer docs on lmstudio.ai, since install URLs occasionally move.
# Linux/macOS (confirm the current line in the LM Studio developer docs first)
curl -fsSL https://lmstudio.ai/install.sh | bash
# Start the daemon, then check it
lms daemon up
lms daemon statusOn a machine that already has the desktop app, skip this step and use the app’s own engine. Running both at once is the first pitfall later in this guide. On a dedicated server, install only the daemon. It keeps memory free for the model.
Windows and macOS servers follow the same logic. On Windows, run the PowerShell installer from an elevated terminal only if the vendor’s docs say so, then manage the daemon with a scheduled task or a service wrapper instead of systemd. On macOS, a launchd agent plays the role that systemd plays on Linux. The CLI commands in the rest of this guide do not change between platforms, only the way you keep them running.
Download and Load a Model From the Terminal
Step 3: Search and Download a Model With lms get
Downloading from the CLI means you can script a fresh server build and never open a GUI. lms get searches the catalog and Hugging Face, shows quantization options and fetches the file you pick.
# Interactive search, pick a quantization when prompted
lms get ministral
# List what is on disk
lms lsChoose a 4-bit quantization (the names usually include Q4) for the best balance of size and quality on a modest machine. GGUF is the file format used by the llama.cpp engine on every platform. On Apple Silicon you can also pick MLX builds, which LM Studio has supported since version 0.3.4 and which tend to run faster on Macs. The lms ls output lists each model’s identifier, which you will need in the next step.
New architectures need an updated runtime before they load. A third-party report says llama.cpp merged support for Microsoft’s Mage-VL-Cowork and that ggml-org published an official GGUF conversion on November 7, 2026, per ModelFit’s memory write-up. In LM Studio you pick up that kind of support through runtime updates. The 0.4.x line, for example, updated the bundled llama.cpp runtime to 2.50.0. Our llama.cpp tutorial explains what that runtime does underneath LM Studio.
Step 4: Load the Model With Explicit Settings
Loading is where you set the knobs that matter for a server: how many layers go to the GPU, how long the context window is, and how long the model stays in memory when idle.
# Replace MODEL_ID with the identifier shown by `lms ls`
lms load MODEL_ID
--gpu max
--context-length 8192
--ttl 3600
--identifier local-main
# Show what is currently loaded
lms psThe --identifier flag gives the loaded model a stable alias, so your clients can ask for local-main no matter which model file sits behind it. That makes swapping models a one-line change on the server instead of an edit in every client. The --ttl value is the idle time in seconds before LM Studio unloads the model to free memory. Start with 8192 tokens of context. Raising it multiplies KV cache memory, and a context that does not fit is the most common cause of slow generation and crashes.
lms ps lists loaded models with their identifiers and status. The exact columns vary by build, so the next block is only a sketch of what to look for.
IDENTIFIER MODEL STATUS CONTEXT
local-main MODEL_ID IDLE 8192Start the Server and Test the API
Step 5: Start the LM Studio Server and List Models
lms server start --port 1234
lms server status
# The first sanity check: ask the server what it has loaded
curl -s http://127.0.0.1:1234/v1/models | python3 -m json.toolA healthy server answers with a JSON list of model objects. If your alias appears, the server is running and the model is loaded.
{
"data": [
{ "id": "local-main", "object": "model", "owned_by": "organization_owner" }
],
"object": "list"
}The output above is a representative shape. Your identifiers and extra fields will differ by build. The point is the /v1 prefix. The default base URL is http://localhost:1234/v1, and the DevelopersDigest API compatibility reference documents the same base URL and the limits that come with it.
Step 6: Send Your First Chat Completion With curl
curl -s http://127.0.0.1:1234/v1/chat/completions
-H "Content-Type: application/json"
-d '{
"model": "local-main",
"messages": [
{"role": "system", "content": "You are a terse assistant."},
{"role": "user", "content": "Name three uses for a local LLM server."}
],
"temperature": 0.2,
"max_tokens": 200
}' | python3 -m json.toolThe response follows the OpenAI shape: a choices array with the message, a usage block with prompt and completion token counts, and a finish reason. Read usage. It tells you exactly how much context each call burns, which helps when you tune the context length from Step 4.
Four request fields cover most tuning. temperature controls randomness, and values between 0 and 0.3 suit extraction, classification and code. top_p trims the tail of unlikely tokens and rarely needs changing once you have set the temperature. max_tokens caps the reply and protects you from runaway generations. stop takes a list of strings that end generation early, which helps when you ask for a single line. If your build supports a seed value, use it for repeatable tests, and remember that identical seeds only give identical text on identical builds and hardware.
Beyond /v1/chat/completions, official results and secondary guides describe support for /v1/models, /v1/responses, legacy /v1/completions and /v1/embeddings. The Responses endpoint was documented by at least October 30, 2026, and official release notes mention a 0.3.39 update for Open Responses support. If a client only speaks Chat Completions, you can ignore the rest.
Call LM Studio From Python and Node.js
Step 7: Use the OpenAI Python SDK Against the Local Server
The compatibility layer means you change one URL and keep your existing code. Install the SDK in a virtual environment and read the base URL and key from environment variables so the same script works against localhost today and a LAN address tomorrow.
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade openai
export LMSTUDIO_BASE_URL="http://127.0.0.1:1234/v1"
export LMSTUDIO_API_KEY="lm-studio" # any non-empty string until you enable authimport os
from openai import OpenAI
client = OpenAI(
base_url=os.environ.get("LMSTUDIO_BASE_URL", "http://127.0.0.1:1234/v1"),
api_key=os.environ.get("LMSTUDIO_API_KEY", "lm-studio"),
)
models = client.models.list()
print("Loaded models:", [m.id for m in models.data])
reply = client.chat.completions.create(
model="local-main",
messages=[
{"role": "system", "content": "Answer in two sentences."},
{"role": "user", "content": "Why serve an LLM on a local network?"},
],
temperature=0.3,
max_tokens=200,
)
print(reply.choices[0].message.content)
print("Tokens used:", reply.usage.total_tokens)Output on a working setup looks like this. The text will vary with your model.
Loaded models: ['local-main']
A local server keeps prompts and documents on hardware you control, and it gives every device on the network one shared model. It also removes per-token fees for high-volume jobs.
Tokens used: 71Note the dummy API key. The SDK refuses to start without one, and LM Studio ignores it until you turn authentication on in Step 12. Never reuse a real cloud key here, because a misconfigured base URL would send that key to the wrong server.
Set a client-side timeout and a small retry count in anything that runs unattended. A cold model answers more slowly than a warm one, because the first request may have to load weights from disk. The SDK accepts a timeout argument and a max_retries value on the client, and the smoke test later in this guide sets a 120-second timeout for exactly that reason. In a batch job, log the model alias and the token usage for each call, so you can trace a slowdown to a prompt that suddenly doubled in length.
Step 8: Call the Same Server From Node.js
// save as client.mjs, then: npm install openai && node client.mjs
import OpenAI from "openai";
const client = new OpenAI({
baseURL: process.env.LMSTUDIO_BASE_URL ?? "http://127.0.0.1:1234/v1",
apiKey: process.env.LMSTUDIO_API_KEY ?? "lm-studio",
});
const res = await client.chat.completions.create({
model: "local-main",
messages: [{ role: "user", content: "Give me a one-line shell tip." }],
max_tokens: 80,
});
console.log(res.choices[0].message.content);Editors and chat front ends work the same way. JetBrains documents LM Studio as a custom LLM provider for its Junie agent, and a web interface such as the one in our Open WebUI setup guide accepts the same base URL. Anything that lets you type an OpenAI-compatible base URL can point at this server.
Streaming, Structured Output, Tool Calling and Embeddings
Step 9: Stream Tokens and Force JSON With a Schema
Streaming cuts perceived latency because the first token shows up while the rest is still generating. Structured output solves a different problem: making the model return JSON your code can parse without regex surgery. LM Studio has offered a Structured Outputs API since version 0.3.0, so you pass a JSON schema and the server constrains generation to match it.
import json, os
from openai import OpenAI
client = OpenAI(
base_url=os.environ.get("LMSTUDIO_BASE_URL", "http://127.0.0.1:1234/v1"),
api_key=os.environ.get("LMSTUDIO_API_KEY", "lm-studio"),
)
# 1) Streaming
stream = client.chat.completions.create(
model="local-main",
messages=[{"role": "user", "content": "Count from one to five in words."}],
stream=True,
)
for chunk in stream:
delta = chunk.choices[0].delta.content
if delta:
print(delta, end="", flush=True)
print()
# 2) Structured output
schema = {
"type": "object",
"properties": {
"title": {"type": "string"},
"severity": {"type": "string", "enum": ["low", "medium", "high"]},
"needs_human": {"type": "boolean"},
},
"required": ["title", "severity", "needs_human"],
"additionalProperties": False,
}
resp = client.chat.completions.create(
model="local-main",
messages=[{"role": "user", "content": "Ticket: the nightly backup failed twice in a row."}],
response_format={
"type": "json_schema",
"json_schema": {"name": "ticket", "strict": True, "schema": schema},
},
temperature=0,
)
ticket = json.loads(resp.choices[0].message.content)
print(ticket)A good run prints the numbers as words, then a dictionary such as {'title': 'Nightly backup failed twice', 'severity': 'high', 'needs_human': True}. Set temperature to 0 for extraction jobs. You want the same answer every time.
Step 10: Add Tool Calling
Tool calling lets the model ask your code to do something, such as read a file or query a database. The model does not run the tool. It returns a structured request, your code runs the function and you send the result back. This only works with models trained for tool use. The LM Studio catalog flags those models, and an official LM Studio release note on 0.3.15 mentions improved tool use in the API.
import json, os, shutil
from openai import OpenAI
client = OpenAI(
base_url=os.environ.get("LMSTUDIO_BASE_URL", "http://127.0.0.1:1234/v1"),
api_key=os.environ.get("LMSTUDIO_API_KEY", "lm-studio"),
)
def disk_free_gb(path: str = "/") -> float:
return round(shutil.disk_usage(path).free / 1e9, 1)
tools = [{
"type": "function",
"function": {
"name": "disk_free_gb",
"description": "Return free disk space in gigabytes for a path.",
"parameters": {
"type": "object",
"properties": {"path": {"type": "string"}},
"required": ["path"],
},
},
}]
messages = [{"role": "user", "content": "How much free space is on / right now?"}]
first = client.chat.completions.create(model="local-main", messages=messages, tools=tools)
call = first.choices[0].message.tool_calls[0]
args = json.loads(call.function.arguments)
messages.append(first.choices[0].message)
messages.append({"role": "tool", "tool_call_id": call.id, "content": str(disk_free_gb(**args))})
final = client.chat.completions.create(model="local-main", messages=messages, tools=tools)
print(final.choices[0].message.content)If tool_calls comes back empty, the model most likely lacks tool-use training. Swap models before you debug your code.
Embeddings and the Native API
For retrieval work, load an embedding model next to your chat model and call /v1/embeddings with the OpenAI SDK’s embeddings.create. LM Studio also has a native REST surface. Version 0.4.0 introduced a stateful /v1/chat endpoint alongside model load and unload endpoints under /api/v1, which lets you manage models over HTTP instead of the CLI. One current LM Studio Hub project notes that plugin integrations go through the native /api/v1/chat path, not the OpenAI-style /v1/chat/completions endpoint. For plain chat clients, stay on the OpenAI-compatible routes.
Lock It Down With an API Token and Network Rules
Step 11: Decide Who Can Reach the Server
A model server with no authentication on a shared network is an open door. It can burn your GPU, leak whatever the model has in context and, if you wire up tools, run actions on your behalf. By default LM Studio listens on the local machine only, and the safest way to reach it from elsewhere is an SSH tunnel. The tunnel adds encryption and uses your existing SSH keys.
# From your laptop: forward local port 1234 to the server's loopback port 1234
ssh -N -L 1234:127.0.0.1:1234 you@llm-server.lan
# Now http://127.0.0.1:1234/v1 on the laptop reaches the server
curl -s http://127.0.0.1:1234/v1/modelsIf many clients need direct access, LM Studio offers a serve-on-the-network option (documented since the 0.3.0 release notes). Turn it on in the server settings, then restrict who can connect at the firewall. Our Ollama API gateway tutorial shows the same gateway pattern for a different runtime, and the firewall rules carry over.
# Ubuntu: allow only your LAN subnet to reach the LM Studio port
sudo ufw allow from 192.168.1.0/24 to any port 1234 proto tcp
sudo ufw status numberedTreat a home or office Wi-Fi network as hostile for this purpose. Guest devices, smart TVs and a laptop with malware all sit on the same subnet as your server. A firewall rule narrows the audience, the token narrows it again, and TLS stops anyone on the path from reading prompts. None of the three is enough alone. If the server holds sensitive documents in its context, keep it on loopback and use the tunnel.
Step 12: Require an API Token
Authentication is off by default but can be switched on. In the server settings, enable the authentication option, create a token and copy it somewhere safe. Menu labels differ between builds, so look for the token or API-key control in the Developer or server section. After that, every request needs a bearer header.
export LMSTUDIO_API_KEY="paste-your-token-here"
# Without the header the server should refuse the call
curl -s -o /dev/null -w "%{http_code}n" http://127.0.0.1:1234/v1/models
# With the header it answers
curl -s -H "Authorization: Bearer $LMSTUDIO_API_KEY" http://127.0.0.1:1234/v1/models | head -c 200Expect a 401 for the first call and JSON for the second. Your Python and Node clients already read LMSTUDIO_API_KEY, so they pick up the real token without code changes. Keep the token out of shell history and version control. A .env file with mode 600 works for a small team.
Keep It Running With systemd
Step 13: Run the Daemon as a Service
A server that needs a human to restart it after every reboot is not a server. This unit file starts the daemon at boot, loads your model and starts the API server. Adjust the user name and the path to lms for your machine. The subcommands match the ones used earlier. Run lms --help to confirm your build supports them.
# /etc/systemd/system/lmstudio.service
[Unit]
Description=LM Studio headless server
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
RemainAfterExit=yes
User=llm
Environment=HOME=/home/llm
ExecStart=/home/llm/.lmstudio/bin/lms daemon up
ExecStartPost=/home/llm/.lmstudio/bin/lms load MODEL_ID --gpu max --context-length 8192 --ttl 0 --identifier local-main
ExecStartPost=/home/llm/.lmstudio/bin/lms server start --port 1234
ExecStop=/home/llm/.lmstudio/bin/lms daemon down
[Install]
WantedBy=multi-user.targetsudo systemctl daemon-reload
sudo systemctl enable --now lmstudio.service
systemctl status lmstudio.service --no-pager
journalctl -u lmstudio.service -n 30 --no-pagerA healthy status shows active (exited), which is normal for a oneshot unit that hands the real work to the daemon. The --ttl 0 setting in this unit is deliberate for a dedicated server. Check lms load --help to see how your build treats it, and if a zero value unloads the model right away, drop the flag and rely on the default idle behavior. Then reboot the machine once and run the curl check from Step 5. A service you have never rebooted is a service you have not tested.
Complete Working Project: A Smoke Test for Your Local LLM Server
This script exercises every feature the tutorial set up and prints a pass or fail table. Run it after every LM Studio update, model swap or config change. It exits with code 1 on any failure, so it drops straight into cron or CI. It needs only the openai package.
#!/usr/bin/env python3
"""Smoke test for an LM Studio server (OpenAI-compatible API)."""
import json
import os
import shutil
import sys
import time
from openai import OpenAI
BASE = os.environ.get("LMSTUDIO_BASE_URL", "http://127.0.0.1:1234/v1")
KEY = os.environ.get("LMSTUDIO_API_KEY", "lm-studio")
client = OpenAI(base_url=BASE, api_key=KEY, timeout=120)
results = []
def pick_model() -> str:
wanted = os.environ.get("LMSTUDIO_MODEL")
if wanted:
return wanted
ids = [m.id for m in client.models.list().data]
chat_ids = [i for i in ids if "embed" not in i.lower()]
if not chat_ids:
raise RuntimeError("no chat model loaded")
return chat_ids[0]
def run(name, fn):
start = time.perf_counter()
try:
detail, status = fn(), "PASS"
except Exception as exc: # report every failure, keep going
detail, status = str(exc)[:70], "FAIL"
results.append((name, status, time.perf_counter() - start, detail))
MODEL = pick_model()
def t_models():
return f"{len(client.models.list().data)} model(s) listed"
def t_chat():
r = client.chat.completions.create(
model=MODEL,
messages=[{"role": "user", "content": "Reply with the single word: ready"}],
max_tokens=40,
temperature=0,
)
text = (r.choices[0].message.content or "").strip()
if not text:
raise RuntimeError("empty content (raise max_tokens for reasoning models)")
return f"{r.usage.total_tokens} tokens"
def t_stream():
start = time.perf_counter()
first, chunks = None, 0
stream = client.chat.completions.create(
model=MODEL,
messages=[{"role": "user", "content": "List five fruits, one per line."}],
max_tokens=120,
stream=True,
)
for chunk in stream:
if chunk.choices and chunk.choices[0].delta.content:
chunks += 1
first = first or time.perf_counter() - start
if not chunks:
raise RuntimeError("no streamed content")
total = time.perf_counter() - start
return f"first token {first:.2f}s, ~{chunks / total:.1f} chunks/s"
def t_json():
schema = {
"type": "object",
"properties": {"ok": {"type": "boolean"}, "word": {"type": "string"}},
"required": ["ok", "word"],
"additionalProperties": False,
}
r = client.chat.completions.create(
model=MODEL,
messages=[{"role": "user", "content": "Return ok=true and word=hello."}],
response_format={"type": "json_schema", "json_schema": {"name": "t", "strict": True, "schema": schema}},
temperature=0,
)
data = json.loads(r.choices[0].message.content)
assert set(data) == {"ok", "word"}, "schema keys mismatch"
return "schema respected"
def t_tools():
tools = [{"type": "function", "function": {
"name": "disk_free_gb", "description": "Free disk space in GB for a path.",
"parameters": {"type": "object", "properties": {"path": {"type": "string"}}, "required": ["path"]}}}]
r = client.chat.completions.create(
model=MODEL,
messages=[{"role": "user", "content": "How much free space is on / ?"}],
tools=tools,
)
calls = r.choices[0].message.tool_calls
if not calls:
raise RuntimeError("model returned no tool call")
args = json.loads(calls[0].function.arguments)
return f"tool call ok, path={args.get('path')}, free={shutil.disk_usage('/').free // 10**9} GB"
for label, fn in [("models", t_models), ("chat", t_chat), ("stream", t_stream),
("json schema", t_json), ("tool calling", t_tools)]:
run(label, fn)
print(f"Server: {BASE} Model: {MODEL}")
print(f"{'TEST':<14}{'STATUS':<8}{'TIME':<8}DETAIL")
for name, status, secs, detail in results:
print(f"{name:<14}{status:<8}{secs:<8.2f}{detail}")
sys.exit(1 if any(r[1] == "FAIL" for r in results) else 0)Save it as lmstudio_smoke.py and run it.
python3 lmstudio_smoke.py; echo "exit code: $?"Here is illustrative output. The timings are an example from a small model on a mid-range GPU, not a benchmark, and your numbers will differ.
Server: http://127.0.0.1:1234/v1 Model: local-main
TEST STATUS TIME DETAIL
models PASS 0.01 1 model(s) listed
chat PASS 0.84 31 tokens
stream PASS 2.10 first token 0.42s, ~38.5 chunks/s
json schema PASS 1.12 schema respected
tool calling PASS 1.35 tool call ok, path=/, free=412 GB
exit code: 0A FAIL row names the feature that broke, which is half the diagnosis. A tool-calling failure with everything else green points at the model. A failure across the board points at the server, the token or the network. To run it as a health check, add a cron line such as */30 * * * * /home/llm/.venv/bin/python /home/llm/lmstudio_smoke.py || mail -s "LLM server down" you@example.com.
Extend the script as your needs grow. Add an embeddings check once you load an embedding model, a concurrency check that fires four chat requests at once and reports the slowest, and a latency budget that fails the run when the first token takes longer than your threshold. Keep each check small and independent so a failure points at one cause. Write the output to a log file with a timestamp, and you gain a cheap history of how updates and model swaps changed speed over time.
Common Pitfalls to Avoid
These mistakes cause most of the support threads about LM Studio servers. Skim them before you blame the software.
- Forgetting the /v1 suffix. Clients that point at
http://host:1234get a 404 on every call. The base URL must end in/v1. - Running the desktop app and the daemon together. Two engines fight over the same port and GPU memory. Run one or the other. If you use LM Studio Bionic as well, note that it can coexist with LM Studio and borrow it for local inference, but it needs Bionic 1.1.8 or newer with LM Studio 0.4.26 or newer.
- Setting the context too high. A 32,000-token context on a laptop-class GPU can eat more memory than the weights. Start at 8192 and raise it only when your prompts need it.
- Ignoring reasoning tokens. Reasoning models spend part of
max_tokensthinking. A cap of 100 can leave the visible answer empty. Raise the limit or turn the reasoning effort down. - Exposing port 1234 without a token. Serve-on-network with no authentication hands your GPU to anyone on the Wi-Fi. Use the token from Step 12 and a firewall rule.
- Sending a real cloud key to a local URL. A copy-pasted config can leak a paid key to the wrong host. Use a throwaway string until auth is on.
- Expecting every model to support tools or schemas. Both depend on model training and chat-template support. Test with the smoke script before you build on a model.
- Skipping the reboot test. A service that works after a manual start can fail at boot because of PATH, user or ordering problems. Reboot once and run the smoke test before you call the job done.
Troubleshooting: 12 Problems and Fixes
When something breaks, match the symptom in the first column, then work through the fix. Run lms log stream in a second terminal to watch server logs while you retry.
| Symptom | Likely cause | Fix |
|---|---|---|
| Connection refused on port 1234 | Server not started, or daemon down | Run lms server status, then lms server start –port 1234. Check lms daemon status |
| 404 Not Found on every request | Base URL missing /v1 | Use http://host:1234/v1 as the base URL, not the bare host |
| 401 Unauthorized | Authentication enabled, header missing | Send Authorization: Bearer YOUR_TOKEN or set LMSTUDIO_API_KEY |
| model not found error | Client asks for an ID that is not loaded | Call /v1/models, copy the exact ID, or reload with –identifier |
| Empty reply, finish reason length | Reasoning model used the token budget | Raise max_tokens, lower reasoning effort, or switch to a non-reasoning model |
| Out-of-memory error or crash on load | Model plus context exceeds RAM or VRAM | Pick a smaller quantization, cut context-length, or lower –gpu offload |
| Very slow generation | Layers spilled to CPU or the context is huge | Check lms ps and GPU memory, reduce context, use a smaller model |
| tool_calls empty or malformed | Model lacks tool-use training or template | Use a model flagged for tool use in the catalog, update runtimes |
| JSON does not match the schema | Old build or model ignores constrained output | Update LM Studio, set temperature 0, validate with a JSON schema library |
| Works locally, fails from another machine | Loopback-only binding or firewall | Use the SSH tunnel, or enable network serving and open the port for your subnet only |
| lms: command not found | CLI not on PATH | Add the LM Studio bin folder to PATH, run lms bootstrap, open a new shell |
| Service starts, then model unloads | TTL expired or flag unsupported | Remove or raise –ttl, confirm with lms ps after boot |
If none of these match, update LM Studio first. The 0.4.26 to 0.4.28 series added Windows-on-ARM CUDA 13 support, refreshed the llama.cpp runtime to 2.50.0, improved runtime discovery and fixed several compatibility problems, according to the official changelog. A runtime bug from an older build is a surprisingly common culprit.
Advanced Tips for a Faster, Safer Server
Use continuous batching on purpose. Since 0.4.0, one loaded model can serve parallel requests. Test with two or four clients at once and watch latency. Throughput per request drops as concurrency rises, so find the point where your slowest acceptable response time sits and cap the number of agents you run against one GPU.
Give every use a separate alias. Load one small model as fast for autocomplete and one larger model as smart for chat, if memory allows. Clients choose by name, and you can retarget an alias without touching a client. Our vLLM setup guide covers the next step up, a dedicated serving engine, once a single LM Studio box no longer keeps pace with your traffic.
Pin and record versions. Write the LM Studio version, the runtime version and the model file name into a small text file next to the systemd unit. When quality or speed changes after an update, that note tells you whether the engine or the model moved. The 0.4.x releases ship often, so keep the previous installer and the previous model file as a rollback plan. It saves an afternoon the first time an update misbehaves.
Put a reverse proxy in front for TLS. If clients connect over a network you do not control, terminate HTTPS with Caddy or nginx and keep LM Studio on loopback. Disable response buffering for the streaming route, or tokens arrive in one lump at the end.
Use speculative decoding where it exists. LM Studio shipped speculative decoding in 0.3.10, which pairs a small draft model with a large one to speed up generation. Gains depend on how well the draft model predicts the main one, so measure with your own prompts using the smoke script’s streaming timing.
Watch the logs and the context. Use lms log stream to see prompts and timings. If your use case is document search, a larger context costs more than a retrieval step does. Chunk documents and use the embeddings route instead.
Re-run the smoke test after every update. The ecosystem moves weekly. The official model catalog was updated on November 10, 2026, and tooling such as an OpenAI adapter in the LM Studio Hub was updated on November 8 and 9. A five-second test catches regressions before your users do. If you are weighing this setup against another runtime, our LM Studio vs Ollama benchmark gives measured comparisons.
FAQ
Does LM Studio’s server work with the OpenAI SDK without changes?
Yes, for the Chat Completions, Responses, models and embeddings routes. Change the base URL to http://localhost:1234/v1 and use any non-empty API key until you enable authentication. Features that depend on a specific model, such as tools and JSON schemas, still need a model that supports them.
Can I run LM Studio without the desktop app?
Yes. Version 0.4.0 added a non-GUI deployment option, and the lms CLI manages models and the server. Step 2 and Step 13 cover the daemon install and the systemd service.
Is LM Studio free for commercial use?
A secondary guide reports that the separate commercial-license requirement was removed on July 8, 2025. Confirm the current terms on the vendor’s site before relying on that, and check each model’s own license, which can restrict commercial use independently.
How much RAM do I need to serve an 8B model?
Plan for 16 GB. The weights of an 8B model at 4-bit take about 4 GB, and the KV cache, the runtime and your operating system take the rest. Longer contexts and several parallel requests raise the total.
How do I expose the server to other computers safely?
Prefer an SSH tunnel, which needs no extra open port. If clients must connect directly, enable network serving, turn on API-token authentication, allow only your subnet through the firewall and add TLS through a reverse proxy.
Why does the model return an empty answer?
Reasoning models spend part of the token limit on hidden thinking. If max_tokens is too low, nothing is left for the answer and the finish reason reads length. Raise the limit or switch models.
Can I run OpenAI’s GPT models in LM Studio?
No evidence from the vendors’ pages shows that OpenAI’s hosted GPT-6 models can be downloaded and run in LM Studio. LM Studio serves open-weight models you download, and it copies OpenAI’s API shape so existing code works unchanged.
Does LM Studio Bionic replace the local server?
No. Bionic is a separate agent product, and the local OpenAI-compatible server keeps working. Bionic can use LM Studio for local inference when both are installed, provided the versions meet the minimums listed in the pitfalls section.