The plan was simple. Pi would stay on my Mac, next to my files, and a PC with an RX 7900 XT (20GB of VRAM) would do the inference in LM Studio.
Qwen3.8-27B loaded at 128K context with all 64 layers set for GPU offload, and I used it through Pi to build a React and ECharts visualization. Below is the setup that worked, plus the things I haven't measured yet.
The setup
LM Studio runs the model on Vulkan rather than ROCm and serves an OpenAI-compatible API on my local network.
Pi on Mac
-> LM Studio API on PC (Vulkan runtime)
-> Qwen3.8-27B
-> RX 7900 XT 20GB
These are the values from LM Studio:
| Setting | Value |
|---|---|
| Model | Qwen3.8-27B |
| GGUF | Qwen3.8-27B-UD-Q3_K_XL.gguf |
| Model source | unsloth/Qwen3.8-27B-GGUF |
| Runtime | Vulkan |
| Context length | 128,000 |
| GPU offload | 64 of 64 layers |
| Flash Attention | On |
| K cache | Q4_0, on GPU |
| V cache | Q4_0, on GPU |
| CPU thread pool size | 4 |
| Evaluation batch size | 512 |
| Physical batch size | 1024 |
| Concurrent predictions | 1 |
| RoPE | Default |
| Speculative decoding | Off |
| MTP | Off |
The official Qwen3.8-27B model card lists 27 billion parameters, 64 layers and a native context of 262,144 tokens. I stopped at 128K. That's more than I usually need for coding, and it fits.
Why Q3_K_XL
Unsloth publishes several GGUF quantizations of Qwen3.8-27B. These are the ones I looked at:
| Quantization | File size |
|---|---|
UD-IQ3_S | 12GB |
UD-Q3_K_XL | 13.1GB |
UD-IQ4_XS | 14.3GB |
UD-Q4_K_M | 16.5GB |
UD-Q5_K_XL | 20.9GB |
Sizes are from the Unsloth GGUF repository. I went with UD-Q3_K_XL: at 13.1GB it leaves more headroom than the 4-bit files, without dropping to the smallest 3-bit ones.
I picked it for fit. I'm not claiming it's the best quality. I haven't run UD-Q3_K_XL and UD-IQ3_S on the same tasks, so I can't say which is better at coding or tool use.
Loading it in LM Studio
I set the runtime to Vulkan. Before loading anything, I checked the memory estimate:
lms load --estimate-only <model-key> --gpu max --context-length 128000
You get the model key from lms ls. It isn't necessarily the GGUF filename. According to LM Studio's load command docs, the estimate accounts for context length, Flash Attention and vision support.
The 13.1GB file is only part of what goes into VRAM. Runtime buffers, the vision components and the KV cache all need room, and a 128K cache is a lot bigger than an 8K or 16K one. I kept the Q4_0 K and V caches on the GPU, sharing the 20GB with the model layers.
LM Studio's model loading API can move the KV cache into system RAM with offload_kv_cache_to_gpu. That can buy you a longer context, but it's slower, and some compute buffers stay in VRAM anyway. More system RAM helps if the cache or some layers live on the CPU side. It won't make a 20GB card behave like a bigger one.
I wouldn't trust the estimate as the final word. Context length, where the cache lives and batch size all change what actually loads. The LM Studio panel after loading is the number to write down.
Connecting LM Studio on the PC to Pi on the Mac
Pi reads and edits files on the Mac, and LM Studio does inference on the PC. LM Studio's local server defaults to port 1234, so Pi points at the PC's LAN address:
http://<PC-LAN-IP>:1234/v1
Before touching Pi's config, I asked LM Studio for the model ID:
curl http://<PC-LAN-IP>:1234/v1/models
The endpoint is covered in LM Studio's OpenAI-compatible models API. The active ID was qwen3.8-27b. My first guess, qwen/qwen3.8-27b, was wrong.
Pi reads custom providers from ~/.pi/agent/models.json:
{
"providers": {
"lmstudio": {
"baseUrl": "http://<PC-LAN-IP>:1234/v1",
"api": "openai-completions",
"apiKey": "lm-studio",
"models": [
{
"id": "qwen3.8-27b",
"name": "Qwen3.8-27B",
"reasoning": true,
"input": ["text"],
"contextWindow": 128000,
"cost": {
"input": 0,
"output": 0,
"cacheRead": 0,
"cacheWrite": 0
},
"compat": {
"maxTokensField": "max_tokens",
"supportsDeveloperRole": false,
"thinkingFormat": "qwen-chat-template"
}
}
]
}
}
}
Pi's custom model docs explain the models.json format and the openai-completions provider type. The apiKey above is just a placeholder for a server with no authentication. If you turn authentication on, keep the real token local and out of Git.
pi --list-models | grep qwen3.8-27b
That printed lmstudio qwen3.8-27b 128K. If LM Studio listens on anything beyond localhost, turn on authentication and keep it off the public internet.
The coding test: a monochrome Internet graph
A chat reply didn't feel like a real test, so I had Pi build a monochrome network visualization in React with Apache ECharts.
The finished demo draws an Internet topology dataset with 17,184 nodes, 29,340 edges and 31 classes. It uses ECharts GL, a ForceAtlas2 layout, a grayscale palette and a small HUD. The nodes start out inside a triangle and then the force simulation pulls them into the graph.
The model worked inside an existing Vite project and loaded an external data file, and later edits had to keep the WebGL interaction working. The demo works. I didn't log its tool calls, failed edits or retries, though. It was a real project, not a controlled AI agent benchmark.
What I didn't measure
I didn't record tokens per second, prompt processing speed, peak RAM or load time, and I never did a controlled 64K vs 128K comparison. So the setup works, but these results tell you nothing reliable about its speed or quality compared to other quantizations.
Next time I'll start at 64K and repeat the same test at 128K. For each load I'll record:
- Final GPU-offloaded layer count.
- Model load time.
- Prompt-processing speed using the same long prompt.
- Generation speed using the same output limit.
- Peak VRAM and system RAM.
- Whether a fixed multi-step Pi task completes without manual recovery.
I also want to put UD-Q3_K_XL against UD-IQ3_S. The smaller file might free up room for more context or GPU layers. What I don't know is whether that costs anything in coding and tool use reliability.
Where this leaves me
For this project the split worked: Pi on the Mac, inference on the PC. Before I call it fast, or trust it for long sessions, I need a proper controlled run.