LLM2Jev is an independent open-source project. It is not affiliated with or endorsed by Jev or TypeSafe.

• October 11 - v0.1.0: give your LLM / MLLM Jev-style decision capabilities with zero training. Supports SGLang, Transformers, and MLX, with a Python client that extends the OpenAI SDK.
• October 8 - Valen Sokoban evaluation: added a visual Sokoban demo and single-step action evaluation based on Valen.
• September 26 - JevBench evaluation: LLM2Jev accuracy and latency on 231 public items; P50 latency is below one tenth of the Jev official result.
• September 23 - MLX backend: added text and image scoring, candidate batching, bounded prefix reuse, and a compatible System One HTTP service.
• September 22 - Multimodal inputs: added text-and-image requests for SGLang, Transformers, and the System One HTTP API.
• September 21 - Web and Snake demos: added interactive examples for composing mixed questions and model-driven decisions.
• September 21 - Prefix reuse on cold requests: added staged candidate submission for reusing SGLang's Radix Cache, with architecture, usage, and benchmark documentation.
• September 20 - SGLang and System One API: added the SGLang scoring backend and a compatible POST /v1/systemone endpoint.

• No training or fine-tuning: use existing model weights to make Jev-style decisions with models supported by SGLang, Transformers, or MLX on Apple Silicon.
• Chat and decisions share a service and model: with the SGLang backend, chat and /v1/systemone use the same inference engine and loaded model.
• Chat and decisions share a client: Client and AsyncClient extend the OpenAI Python SDK with systemone.create(), preserving its existing interfaces, authentication, timeouts, and retries. Endpoint availability depends on the backend.
• Prefill only: read yes/no logits and assemble probabilities directly, without generating an answer or parsing generated JSON.
• Text and image decisions: combine text and images in state or instructions for classification, evaluation, and action selection.
• Three question types in one request: mix Choice, Score, and Noul to get option probabilities, rubric scores, and yes/no probabilities as typed Python objects or JSON.
• Reuse shared prefixes: reduce repeated computation across candidates with SGLang's Radix Cache or MLX's explicit prefix prefill, including within a cold request.

Candidates share state, and candidates for the same question also share its instructions. The SGLang backend first scores a real criteria candidate to establish the prefix cache, then submits candidates that can reuse it. Each candidate is scored once, reducing repeated computation for long inputs with many candidates. The MLX backend explicitly prefills shared prefixes before scoring candidate suffixes.

Both backends preserve the same binary scoring interface. See the Usage guide for details.

Learn how it works: From Jev Request to LLM Request → Shared-prefix design.

On Linux with a supported NVIDIA GPU, start one SGLang service for chat and decisions:

git clone https://github.com/Yinsongxu/LLM2Jev.git
cd LLM2Jev
uv sync --extra sglang
source .venv/bin/activate
llm2jev-serve --backend sglang --model-path /path/to/model \
  --served-model-name local-model --host 127.0.0.1 --port 30000

Replace /path/to/model with a model directory supported by SGLang. In another terminal, use the same Python environment to call both endpoints with one client:

from llm2jev import Client, Choice

client = Client(base_url="http://localhost:30000/v1", api_key="unused")

decision = client.systemone.create(
    model="local-model",
    state="My card was charged twice.",
    questions={"department": Choice(criteria={
        "billing": "Charges and billing", "shipping": "Delivery",
    })},
)
print(decision.choices["department"].probabilities)

chat = client.chat.completions.create(
    model="local-model",
    messages=[{"role": "user", "content": "How can I request a refund?"}],
)
print(chat.choices[0].message.content)

Use your service's API key if authentication is enabled. See the Usage guide for async calls, client lifecycle, offline inference, and the MLX and Transformers services. For image serving, see Multimodal inputs.

See Installation for environment requirements, SGLang, Transformers, and MLX dependencies, and uv or pip installation.

See the Usage guide for complete examples:

• Offline: Python API
• Online: HTTP service
• Online: Python client
• Choosing between staged and all
• Multimodal inputs

Web demo	Snake demo
MuJoCo pick-and-place demo	Valen Sokoban demo

See Performance benchmarks for the Qwen3-1.7B / RTX 5090 measurements, test conditions, and comparison of staged and all across cold and warm caches. Gains depend on input length, candidate count, and cache state.

This test covers 231 public JevBench items. LLM2Jev with Qwen3.5-4B measured 76.2% accuracy.

The recorded LLM2Jev latency is P50 48 ms / P95 346 ms, while the Jev 1.13.0 values are P50 652 ms / P95 722 ms. See the JevBench report for the data and reproduction commands.

The single-step evaluator measures tie-aware optimal-action accuracy on 500 image-conditioned Sokoban questions from Valen. It uses the multimodal Choice pipeline with the SGLang backend.

• More benchmarks across model sizes, datasets, and workloads, covering decision quality, latency, and throughput.
• An interactive web demo for submitting questions and inspecting probabilities.
• Initial local-image support for Transformers and SGLang.
• More multimodal tasks and demos.

python -m unittest discover -s tests -v

This project is licensed under the Apache License 2.0.