Bonsai 2 27B

Ternary · llama.cpp · 128K context · RTX 2000 Ada 16 GB

Prism ML's ternary Bonsai 2, built on Qwen3.6-27B: a 27B model packed small enough for 16 GB, with vision, tools and thinking.

– tok/sdecode

– tok/sprefill

128Kcontext window

Earlier checkSep 21, 2026

Run it

With Omarchy Local AI it is one button. By hand, it is three steps: the weights, the config, the container. The server then answers on http://localhost:8000/v1.

1 Start the server

docker run --rm \
  --gpus all \
  -p 8000:8080 \
  --entrypoint /opt/llama/llama-server \
  ghcr.io/0xsero/deploy-llamacpp-bonsai-ptq1-rtx-2000-ada-16gb@sha256:9a646183bf0dd199504bea9b6f47d9ec2aa07a3bdf1a4827382f449e94292f02 \
  --model /opt/models/Ternary-Bonsai-2-27B/Ternary-Bonsai-2-27B-PTQ1_0.gguf \
  --mmproj /opt/models/Ternary-Bonsai-2-27B/Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf \
  --alias Ternary-Bonsai-2-27B-PTQ1_0 \
  --host 0.0.0.0 \
  --port 8080 \
  --ctx-size 131072 \
  --cache-type-k q4_0 \
  --cache-type-v q4_0 \
  --n-gpu-layers 99 \
  --jinja

What it passed

✓ loadLoads and serves within an hour

✓ chatAnswers a plain question and stops on its own

✓ reasoningThinks separately and gets 17 × 23 right

✓ toolsCalls a tool with the right arguments and uses the result

– contextRecalls a code buried in a prompt that fills 85% of the window

– speedDecodes at 15 tok/s or more

Passed the older check (loads and chats); a full six-check run is pending.

Details

Weights: inside the image
Image: ghcr.io/0xsero/deploy-llamacpp-bonsai-ptq1-rtx-2000-ada-16gb@sha256:9a646183bf0dd199504bea9b6f47d9ec2aa07a3bdf1a4827382f449e94292f02
Engine profile: llama.cpp-bonsai-2-27b-ptq1-0-128k ›
Recipe file: registry/recipes/nvidia/rtx-2000-ada-16gb/bonsai-2-27b.llama.cpp.128k.json ›
Model released: Apr 21, 2026
Tested: earlier acceptance