Is the Model the Problem, or Am I?

Last weekend I was really disappointed with Claude (using Fable and Opus). I recently started a new project that researches businesses online, finds key decision makers, validates their email addresses, and checks whether their LinkedIn and Instagram accounts are still active.

Claude wrote some of the most unreadable, overcomplicated code I’ve ever seen. The number of wrong assumptions it made turned the whole thing into a mess.

It probably seems strange that I’d care this much about the code, or that I’d ask for such specific things instead of just letting Claude handle it, especially since bugs are easy to fix these days. But when it comes to the core logic of a project, I need it to be right. I have a clear mental model of how it should work, and I don’t want to run it and get misleading results. This was one of the few times I decided to sit down and read the code instead of ignoring it.

After that weekend, I felt off. Why was I the one reading code and making sure it matched what I wanted? Will I still be doing this once the models and harnesses improve? And was the mess actually my fault, or Claude’s?

Unrelated, I was also listening to the latest episode of “Raising An Agent” from the Amp team, and it got me thinking about testing. I’ve always resisted testing because it’s hard to do well, and most of the time you end up testing the obvious, like checking that “1 + 1” equals “2.” But now I think it’s worth trying again.

Testing lets you verify correctness without reading the code yourself. Asking Claude to write a test case for “a LinkedIn profile only counts as active if it has 50+ connections” is a simple way to get confidence that the code works as expected.

When I run into problems with Claude or any coding agent, it feels more productive to assume I’m the problem rather than the model or the harness. And if reading code is the symptom, testing is the fix: write the test cases, and you remove the need to read the code in the first place.