We gave nine models the same forty briefs and had the marketing team grade them blind. The ranking was not the leaderboard's.
Rayyan Darugar//1 min readThis is a placeholder post. It exists so the blog shows its real layout; Rayyan replaces it, or deletes it, before launch.
Placeholder text for this section. The finished post will make one point here, in plain language, with the numbers or the example that backs it up.
Placeholder text for this section. The finished post will make one point here, in plain language, with the numbers or the example that backs it up.
Two models that score the same on a benchmark can cost ten times apart per correct answer. Here is the number a finance team should look at instead.
Every company we talked to had built an agent that worked in the demo. None of them had one in production.
The proof of concept is the easy part. The last two months, where security, permissions, and the systems of record get involved, are where projects go quiet.
We’ll give you automated workflow agents.