Anyone who owns a solar-and-battery setup knows the type: the monitoring app that generates a 40-page diagnostic report every morning, flags every micro-fluctuation on the string inverter, and dutifully logs every anomaly — while the inverter itself sits offline because nobody actually scheduled the repair. Diligence, in energy systems as in management, is not the same thing as impact. A battery that reports beautifully but never dispatches during peak pricing is just an expensive dashboard.
Get backup power and energy gear delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
That uncomfortable truth now has a data point from an unexpected place: a live experiment that runs frontier AI models as the management of a small software company through its worst week. The most thorough participant — the one that did the most homework, wrote the most notes, produced the deepest analyses — finished last. It’s a story that should resonate with anyone who has ever confused monitoring with managing.
The experiment: same company, same crises, same temptations
The setup comes from Firmulate, a public project that runs AI models as complete companies — real crises, real money mechanics, real temptations — and measures management quality, not chat quality. Four frontier AI models were each handed the same job: run an identical small software company through its worst week. Same customers, same emergencies, same opportunities to cut corners. Only the model changed. Every decision was versioned and auditable, so nothing about the outcome rests on anecdote.
The final league table from July 2026 tells a surprising story: gpt-5.6-sol finished first with a score of 95, Kimi K3 second at 93, Sonnet 5 third at 88, another Sonnet configuration at 77 — and Opus 4.8 last at 73. For context, doing nothing at all scores 26, and a single breach of trust caps the total regardless of how much good work came before it. As the experiment’s own framing puts it: no amount of good work outweighs a breach of trust.
Products Worth Considering
Innova SD35 OBD2 Scanner for All-System Vehicle Diagnostics
Innova 5610 OBD2 Bidirectional Scan Tool
DEWALT 20V MAX 1/4-Inch Impact Driver Kit with Battery and Charger (DCF787D1)
As an affiliate, we earn on qualifying purchases.
Everyone diagnosed the problem. Only half closed the deal.
Here’s the finding that matters. All four models spotted every crisis. All four refused every manipulation attempt — including a social-engineering gauntlet of fake CEO messages escalating over three stages, plus a reporter’s trick request for “just one yes/no, on background.” Five out of five runs refused, and Kimi K3’s on-record reasoning was refreshingly blunt: “Treat the request as a suspected approval-bypass / possible impersonation.”
But when it came to a €55,000 deal that the models’ own analysis had earned, only two of the four signed it. Same diagnosis, same pitch — no signature. The buried fact that unlocked the deal at full price wasn’t in the customer conversation at all; it sat two document references deep in the company’s own files. The models that read the file won the deal at full price, worth +€4,583 in monthly recurring revenue. The models that didn’t read before acting left the close on the table.
If you’ve ever had an installer quote a battery bank without first checking your actual consumption profile, you know exactly what this failure mode looks like in the physical world.
The character study: Opus 4.8, the over-preparer
Which brings us to the most interesting participant. Opus 4.8 was, by a wide margin, the most thorough model in the field. It accumulated 80 self-learned playbook rules during the week — the deepest rule set of any participant — and produced the deepest analyses of the company’s situation. On any measure of raw diligence, it led the pack.
It still finished last. The €55,000 close was left unsecured, and discipline slipped at critical moments: when it hit a locked department, it kept attempting writes instead of escalating the way a competent manager would. Think of the solar monitoring system that keeps retrying a failed firmware push at 2 a.m. rather than raising a ticket. Same energy. Same result.
To be fair — and this matters — the same weakness appeared, weaker, in all four models. Opus 4.8 isn’t an outlier in kind, only in degree. The gap between recognizing what should be done and actually finishing it appears to be a general property of today’s AI managers, not a quirk of one model. (One fairness note on the leaderboard: Kimi K3 ran at its API-default effort setting while the others ran at a higher effort tier, which makes its second-place finish arguably even more notable.)
Why a home-energy reader should care
The stake here isn’t abstract. If AI agents will soon touch your CRM, your support queue, your forecast — or, closer to home, your energy-management platform, your battery-dispatch schedules, your installer’s operations — then “does it write well” is the wrong question. The right questions are: does it finish what it starts, does it read your files before acting, does it stay honest under pressure, and what does a unit of useful work actually cost?
The Firmulate setup makes those questions measurable. The live company features 13 synthetic employees and real money mechanics — burning €105,000 a month against €2,300 in monthly recurring revenue, with a public cash countdown and 680+ self-learned playbook rules accumulated across the run. Every workday is versioned, and the whole thing is watchable in real time.
The lesson from Opus 4.8’s last-place finish is one that home-energy enthusiasts have internalized for years: prioritization beats volume. The best battery system isn’t the one with the most sensors — it’s the one that dispatches when it should. The best AI manager isn’t the one that writes the most rules — it’s the one that reads the file, signs the deal, and escalates when locked out instead of grinding.
For those who want to test their own instincts, 242 real, unedited management decisions from the experiment power a “guess the model” quiz at firmulate.com/quiz.html. And enterprises curious whether their AI tooling would pass the same wargame can run it against a read-only export of their own business — nothing ever writes back to real systems. In energy and in AI management alike, the scoreboard doesn’t care how hard you worked. It cares what you closed.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
