GPT-6.1 Sol, timed day by day

OpenAI's @thsottiaux posted on 30 September that added capacity would bring "almost twice the speed compared to what we served yesterday". On 1 October @sama posted: "Should be much better now!" On 2 and 4 October posts on X still called it slow. I timed it again on 3, 4, 5 and 6 October. On 5 October @thsottiaux posted that the change "should be felt within the next two hours"; I timed it before that post, right after it, in a retest inside the two hours and after them. The group since 5 October's retest starts at that retest, the first sitting inside the two hours. I set it after seeing the 5 October runs, and every later sitting joins it, whatever it shows.

Each run starts Codex CLI 0.159.0, OpenAI's command-line coding tool, on a fresh copy of a small Python project in the bench's Docker container. GPT-6.1 Sol, at medium effort, reads the project's 53-line file and explains in plain English how it adds up monthly totals and what happens with a bad date, without changing anything. The clock runs from starting Codex until it exits, and the runs go one at a time. A run passes when its answer contains both "month" and "ValueError". All 40 runs passed. Two test runs of the updated timing script on 5 October, between the sitting right after the post and the retest, are not charted. They took 18.4 s and 11.7 s, streamed at 35.4 and 42.4 text pieces a second, 36.2 and 43.4 by OpenAI's token count, and both passed.

On launch day Codex also loaded the apps and plugins that come with a ChatGPT login, and I have switched them off since. On 1 October I ran four runs with them on and four with them off: 15.8 to 19.7 s and 13.5 to 19.6 s, all slower than launch day, so turning them on did not bring back launch day's times.

Both rates are timed from the stream: each answer text event OpenAI sends is timestamped as Codex receives it, and a rate is a count over the seconds from the first text event to the last, pauses included. Text pieces/s counts those events; a piece is not a verified token, so it is labeled a proxy. TPS counts OpenAI's own output tokens for the answer reply, logged for the twelve runs since 5 October's retest and two script tests before it. That count may include a few non-text tokens (it was 4 above the pieces on every run with both) and includes the first output, so it is a rate observed at my machine for the answer, not the engine's generation speed. Each run is one short answer on a ChatGPT subscription, so neither compares with API benchmarks. OpenAI's server-side time between tokens is a separate engine measure and is not shown. The CSV also has tokens a second of waiting: all output tokens divided by the seconds from each reply opening until it completes, added up.

The answer's stream is under half of every run. Launch day's answers were shorter, 157 and 152 pieces of text against 166 to 203 since 5 October's retest, so at 34.0 and 34.1 a second they streamed in 4.6 s and 4.5 s, inside the 3.3 to 5.1 s since. Every launch run made two model requests. The six runs since that made two finished in 10.3 to 12.4 s, against launch day's 10.7 to 12.8 s; the six that made more took 15.1 to 20.8 s.

The Quick suite page times every model on this task and keeps the 30 September runs. The run records are in results.json, under timing_runs and speed_recheck_runs, model gpt-6.1-sol.