# Malbec performance results: the files

Every number the site shows comes from the files in this folder and from `../bench.json`.

- `../bench.json` holds one summary per test: the benchmarks (Unigine Heaven 4.0, 32-bit; Unigine
  Superposition 1.1, 64-bit) and the games we measured by playing them (today Overwatch). It is
  drawn on the home page, /compare/, /performance/ and a measured game's own page.
- `overwatch-2026-10-22.json` holds every Overwatch map we counted on 22 October 2026, for Malbec
  and for CrossOver, and the frame-time trace drawn on /performance/#overwatch.
- `frames/overwatch-2026-10-22-<setup>-<map>.csv` holds the frame times of one map: one line per
  frame, `frame_ms` (the time since the frame before it) and `gpu_ms` (the GPU's time for it), in
  milliseconds, as the Metal Performance HUD listed them. Every other Overwatch number is computed
  from these, except average FPS, which is the HUD's own frame counter: `counter_frames` over
  `counter_seconds` in `overwatch-2026-10-22.json`.
- `results.csv`, described further down, is for runs of games one line each; it doesn't exist yet.

## bench.json

| Key | What it holds |
|---|---|
| `demo` | `true` keeps every test off the pages. `"demo": true` on one test keeps only that test off: the pages then name it as being measured, next to the tests that are measured. |
| `engine_name` | The name the pages use for Malbec's engine. |
| `mac`, `macos`, `date` | For the benchmarks: the Mac, its macOS version and the day of the runs (YYYY-MM-DD). |
| `tests[].id`, `kind` | A short name, and `benchmark` or `game`. |
| `tests[].name`, `settings` | The test and the settings every setup ran with. |
| `tests[].claim` | The row of our claims ledger, for example `L-02`. A measured test needs one. |
| `tests[].note` | Optional: a line under the test's results, for example why a setup isn't compared. |
| `tests[].results[]` | One per setup: `backend` (`Malbec`, `CrossOver 26.2 (D3DMetal)`, …), `fps` (the median of the counted runs), `runs`, `spread` (the quickest run minus the slowest), and `min` and `max`. A setup with fewer than three runs is named as being measured. |

A game test (`"kind": "game"`) also has `game`, `scene`, `mac`, `chip`, `memory_gb`, `macos`,
`date`, `data` (its file in this folder), `average_basis`, `headline` (the metrics shown as big
numbers), `cache` (a sentence on the shader caches, printed after the measured-on line) and
`metrics`: one entry per number (`key`, `name`, `unit`, `better`: `higher` or `lower`), and in
each result `metrics.<key>` with the `median`, `min` and `max` of its runs. The metrics are
`avg_fps` (frames counted by the HUD over wall time), `low_1pct_fps` (the frame rate over the
slowest 1% of frames), `stutters_over_50ms_per_min` (frames that took longer than 50 ms, per
minute) and `max_ms` (the slowest single frame).

Each test is drawn as bars from zero to the median of each setup's runs, with a line from its
lowest run to its highest. When the ranges of two setups overlap, the result says "no
difference".

## overwatch-2026-10-22.json

The facts of the test (`game`, `scene`, `settings`, `cache`, `mac`, `chip`, `memory_gb`,
`macos`, `date`, `claim`), `average_basis` (`counter`: average FPS is the frames the HUD counted
over wall time, used for every map when the HUD listed fewer than 99% of the frames in any of
them), and `setups`: one per setup with its `backend`, `engine`, `setting`, `note`, `cut` (how its
maps were cut: at the tester's key presses, or at the loading screens between maps), whether it
is `compared` (a setup marked `false` would be shown for the record but left out, with its
`reason`), and `runs`: one per map, in the order played, with `map`, `seconds`, `frames`,
`avg_fps`, `counter_frames` and `counter_seconds` (the HUD counter's frames and seconds; with
`average_basis` `counter`, `avg_fps` is the first over the second), `low_1pct_fps`,
`low_01pct_fps`, `median_ms`, `p99_ms`, `max_ms`, `stutters_over_50ms`, `stutters_over_50ms_per_min`,
`gpu_median_ms`, `gpu_bound_frac` (the share of frames whose GPU time was at least 80% of the
frame's), `listed_coverage` (the share of counted frames the HUD listed) and `frames_csv`. A
compared setup's `trace` is the median map by average FPS: `slowest` lists the slowest frame (ms)
in each `slice_ms` of the map.

## results.csv: the columns

`results.csv` has one line per run. The "Results from games" tables on
[malbec.app/performance](https://malbec.app/performance/) are built from it, and every number there
links back to a line in this file and to the frame times it comes from. Until the file exists, the
page has no such section.

| Column | What it holds |
|---|---|
| `claim` | The row of our claims ledger this result belongs to, for example `L-02`. |
| `game`, `scene` | The game, and the part of it we played, for example a map and mode. |
| `settings` | The in-game settings: preset, resolution and anything we changed. |
| `mac`, `chip`, `ram_gb`, `macos` | The Mac model, its chip, its memory in GB and the macOS version. |
| `app`, `app_version` | What ran the game: `Malbec`, `CrossOver (D3DMetal)` or `CrossOver (DXMT)`, and its version. |
| `engine` | The engine that version used. |
| `run`, `run_kind` | The run's number, and `warm` (shaders already built) or `cold` (the first run after clearing the caches). |
| `avg_fps` | Frames divided by seconds, over the measured part of the run. |
| `p1_low_fps` | The frame rate over the slowest 1% of frames. |
| `stutters_per_min` | Frames that took longer than 50 ms, per minute. |
| `frames`, `duration_s` | How many frames, over how many seconds. |
| `date` | The day of the run, YYYY-MM-DD. |
| `raw` | A link to the run's raw capture. |
| `frames_csv` | Optional: a file with one frame time per line (a `frame_ms` column), drawn as a chart. |

Each table shows the median of the warm runs for each setup, with its slowest and fastest run.
When the runs of two setups overlap, the difference is reported as "no difference". A result needs
at least three warm runs per setup.

The frame times come from the Metal Performance HUD that macOS has built in.