Meta and Microsoft swapped models in months, and a set of your own graded tasks lets you do the same safely

Reports dated 5 Oct say Microsoft cut its projected Anthropic spend, at least $1B this year, by more than a third, and that Meta's internal Claude Code users fell from about 60,000 to 30,000. Can your company swap the model under its agents that fast without breaking things? Yes, if you hold a set of your own real tasks with a written expected result for each, because without one nobody can say what "as good as before" means. I think both buyers could move that fast because they own the harness around the model: the prompts, the tool permissions and the review steps. A company that rents that harness from its model vendor pays for the same move in rework.

Two buyers moved their staff inside months

The reports, carried by ROIC and Yahoo Finance on 5 Oct 2026, put two numbers on the table. Microsoft's projected Anthropic spend, at least $1B this year, was reportedly cut by more than a third. That is more than $330M of projected spend cut, with staff moving onto GitHub Copilot CLI. At Meta, internal Claude Code users reportedly fell from about 60,000 to 30,000 as its own coding tools and Muse Spark models took over.

Two things stayed where they were. Microsoft's investment in Anthropic of up to $5B is in place, and so is Anthropic's $30B Azure commitment. These are reported figures, and I use them as reported.

Meta's count went from about 60,000 users to about 30,000 within months, and Microsoft's spend moved by more than a third in the same window.

Owning a rival tool explains the move

Microsoft sells GitHub Copilot. Meta builds its own models and its own coding tools. A company that sells a coding assistant using its own coding assistant is the plain reading of the news, and it leaves Claude's quality an open question.

The same reports say customer spend on Claude through Azure and Bedrock keeps growing. Microsoft still hosts Claude for paying customers and still holds its $5B investment.

The boring explanation covers every reported fact: two firms with rival products pointed their own staff at them. The transferable lesson sits in how fast they could do it, and that depends on who owned the layer around the model.

Owning the harness makes a swap fast

The harness is the layer of prompts, tool permissions and review steps wrapped around a model.

My read is that the staff who moved to GitHub Copilot CLI kept working inside a harness Microsoft controls. Which commands an agent may run, which changes need a human look, how a prompt gets assembled: all of it belonged to the buyer. Swapping the model underneath meant changing a setting inside a system they already ran. I read Meta the same way, with Muse Spark models and Meta's own coding tools on its own plumbing.

Now take a company that built its agent workflow inside its model vendor's tooling. The system prompts live in the vendor's format. The tool definitions follow the vendor's schema. The approval steps are configured in the vendor's console. Switching models means rewriting each of those, and the rewrite lands on the same engineers who are supposed to be shipping the product. Rented harness, same switch, paid in rework.

The model sits behind a harness, and whoever owns the harness and the eval set decides how fast a swap happens and what it costs.

A swap without graded tasks is a guess

A graded task set is a fixed list of real tasks, each with its input and the output your team accepted, written down.

Take a coding agent that opens pull requests. You change the model. The same system prompt now gets read differently, so the agent skips a convention it used to follow. It calls the test-runner tool with different arguments. It returns diffs in a slightly different shape, and a downstream script that parsed the old shape quietly drops a field. The demo looks fine. Reviewers approve the first ten pull requests.

Three weeks later the failures arrive as tickets: a missed lint rule in one repo, a malformed changelog in another, a migration file without its rollback. Each one is small. Each one lands on a different person, and the cause is spread across a prompt, a tool call and an output format.

Then comes the retro. One engineer says the new model is better. Another says it is worse. Without a written expected result per task, both are offering opinions, and the argument ends when the loudest voice gets tired. Nobody can state whether the new model matches the old one, because "as good as before" was never written down anywhere that a script can check.

A written expected result per task turns that retro into a table: task, old output, new output, pass or fail. A table settles in minutes what opinions take weeks to settle.

Build the test set from finished work

Fifty tasks is the starting size. Pick the one agent workflow you can least afford to lose, and take 50 tasks your team already completed on it. For each, keep the input and the output you accepted. The accepted output is your expected result, and it already exists in your pull request history, your ticket system or your reviewed documents.

Mix the 50 on purpose. Include the routine tasks that make up most of the volume, a handful of hard ones that needed two review rounds, and the tasks that went wrong once and got fixed. The ones that went wrong are the most valuable rows in the set.

Then put the model name behind one config value. A thin wrapper takes a task and a model name, runs the full harness with that model, and returns the output. A grader compares the output to the accepted one: an exact match where the output is code with tests, a rubric written by your best reviewer where the output is prose. With that wrapper, any model can run the same 50 tasks the same way.

Refresh the set every month from real failures. Each ticket that traces back to the agent becomes a new row with the corrected output as the expected result. The set stays small and stays current, which is what keeps it honest.

Run a second model through it this week

Run your current model through the set first, to get a baseline. Then run a second model through the same 50 tasks with the same harness. Record two numbers for each: the pass rate and the cost per task.

Sort the failures by cause. Some will be prompt behavior, where the new model needs a rewritten instruction. Some will be tool calls, where arguments or ordering differ. Some will be output format, where a parser needs a patch. Put an hours estimate next to each cause, then multiply by your engineers' loaded rate.

Whatever fails is the switching bill, written down in hours and dollars. A company that has this table negotiates differently. A founder looking at an annual commitment sized to one model's pricing can see what leaving would cost, and what the second model's cost per task would save at current volume. A CTO can answer a board question about vendor concentration with a spreadsheet instead of a feeling.

Either result is worth knowing before a pricing change or an outage forces the question on someone else's schedule.

What would prove this reading wrong

The first objection: two companies with rival products say nothing about a buyer who has none. The swap needed an owned harness, so the work for everyone else is to own the harness and the eval set. A company with no rival product can still build both, and the 50-task set is the cheapest piece of it.

The second objection: evals drift, and 50 tasks miss edge cases. Which is why the set gets refreshed from real failures each month. A thin but current set still turns a swap from a guess into a measured result, and a measured result with known gaps beats a confident guess with unknown ones.

The reading can also be tested against the market. If Microsoft's Anthropic spend is back above its original $1B-plus run rate by mid-2027, the cut was a pause and the lesson shrinks. If Claude consumption through Azure and Bedrock falls over the next two reported quarters, the case for model-level churn gets stronger than this angle assumes, and I would say so in print.

Pick one workflow and test a second model this week

Pick the single agent workflow your company can least afford to lose. Pull 50 tasks your team already finished and attach the accepted output to each. Put the model name behind one config value so the harness runs on any model. Run a second model through the set, record the pass rate and the cost per task, and write down how many hours each failure takes to fix. If you skip it, the first time you learn your switching cost is the week a vendor changes its price or its terms, and by then every hour is already urgent.