I ran the same seven hard coding tasks through five models. Every current model passed every test. The only things that differed were the bill and the stopwatch.

Update, 6 October, 4 pm. Hours after I published, OpenAI made GPT-6 Astra and GPT-6.1 Sol about 50% faster by default for people signed in with ChatGPT. I re-ran the Sol suite this afternoon and have corrected the numbers below. Old figures are struck through. Cost, token use, pass rate and quota burn did not change. I have not re-tested Astra.

Continue reading →

OpenAI just broke something that was working fine, in the name of progress. Software developer’s first rule, like doctors, should be “first do no harm.”

I’m a paying subscriber. I’ve been running Codex hard lately - finance modelling work, not so much straight coding - and my estimation of GPT’s capabilities had genuinely grown over recent weeks. Good performer. I was impressed. But not any more.

Continue reading →