standard-ai-v0.1

RigRoute Labs' first local-LLM-inference methodology. Measures prompt-processing and generation throughput at a fixed context depth, across an ~8B and an ~14B Q4-family model, using llama-bench's own repetition/statistics support.

Why methodology is versioned

Methodology versions are immutable. If anything that affects comparability changes — model, quantization, context, prompt/generation size, repetitions, or other benchmark settings — that becomes a new methodology id (e.g. standard-ai-v0.2), never a silent edit of this one. Every published result records which methodology version produced it, and two results are only ever compared when that version matches.

Engine: llama.cpp / llama-bench

Labs does not implement its own inference engine. It orchestrates llama-bench, part of llama.cpp, which already measures throughput reliably and reports the statistics this methodology needs natively. Every accepted result in this version of the methodology used the identical build:

Fixed test parameters

"Context depth" is llama-bench's -d flag: the number of tokens already in context before the timed prompt-processing/generation test begins. llama-bench has no separate context-size flag — this is the correct, native way to test throughput at a given context length.

Models and quantization

Tested models are ~8B and ~14B parameter classes at a Q4-family quantization. Every accepted result records the model file's real SHA-256 (computed from the file's own bytes) and the declared identity (architecture, parameter count, quantization) as reported by llama-bench's own GGUF parsing — never inferred from a filename. See Data for exact hashes.

Prompt-processing and generation measurement

Each test measures one of two things at the fixed context depth above: prompt-processing throughput (feeding 512 new tokens) or generation throughput (producing 128 new tokens). llama-bench repeats each 5 times and reports the individual repetitions, mean, and standard deviation natively — Labs relays these directly rather than recomputing them.

GPU-acceleration verification

A benchmark that silently fell back to CPU would be worthless — or actively misleading — if reported as GPU performance. Every accepted result is checked against llama.cpp's own load_tensors: offloaded X/Y layers to GPU log line, parsed from the real captured output of that run, not assumed from the hardware being present. On a machine with more than one GPU-class device, the specific device benchmarked is also explicitly targeted and recorded — never left to an ambiguous default.

PASS semantics and failure classifications

A benchmark run always ends in exactly one status. Failures are first-class results, not discarded errors: PASS, FAIL_MODEL_LOAD, OUT_OF_MEMORY, BACKEND_UNSUPPORTED, DRIVER_UNSUPPORTED, RUNTIME_UNAVAILABLE, CPU_FALLBACK, TIMEOUT, CRASH, NOT_TESTED, UNKNOWN_FAILURE. Only PASS results are published on this site; a real failure would still be preserved as evidence in the source repository, with its own honest classification, not silently hidden.

UNKNOWN and UNAVAILABLE

A fact that could not be determined is recorded as the literal string UNKNOWN. Telemetry (power, temperature, accelerator memory usage) that could not be measured — because the tooling to read it wasn't safely available — is recorded as UNAVAILABLE. Neither is ever silently converted to zero, and missing telemetry never blocks a valid throughput result.

Raw-evidence retention

Every accepted run retains its exact command line, full raw llama-bench JSON output, and captured stdout/stderr — the acceleration-verification line above is parsed live from that same raw evidence, not summarized separately. See Data for what is published.

What this methodology does measure

Prompt-processing and generation throughput, in tokens/second, for a specific model file, at a specific context depth, on a specific accelerator, using llama.cpp. That is the whole scope.

What this methodology does NOT establish

• It is not MLPerf or any other industry benchmark suite, and does not claim that scope or rigor.
• It does not produce a universal "AI score" — no single number summarizes general capability.
• It does not measure interactive, end-to-end user experience: time-to-first-token and full serving latency under real request patterns are a different, later methodology, not this one.
• It does not establish that one system is universally better than another — only these specific, measured numbers, on this specific workload.
• It has not yet tested gaming or image-generation workloads.