This is the problem with agentic coding
Simon Willison has a new blog post. It's a minor detail in the post, but he includes some Claude output:
I see the question “count the notes” is probably asking me to tally up blog notes. It could also mean published notes or drafts, so there’s some ambiguity there. I’ll need to figure out the total number of notes, likely by querying the count for both published notes and drafts to get a clear answer. Let’s execute that count!
There are 151 notes.
That in a nutshell explains the "productivity gain" of an LLM. In many cases it produces lines of code faster and with less effort than the programmer typing it out by hand. But where did the gain come from? A big part of it was the agent making a decision on the definition of a note.
That's great if the goal is to minimize the effort you put into it. You even get a number (151) that you can report to someone. The imprecision of the prompt means there are multiple correct responses. The LLM speeds up the process by not requiring the human to specify precisely what they want. It arbitrarily chooses one of the possible solutions that is consistent with the prompt.
I'm sure the agentic coding bros will say this is not a problem of AI coding, it's a problem of using the LLM wrong. Just like the problem was never C, it was programmers using C incorrectly. Sure, you can add tests and evaluations and various types of overhead that were invented so that agentic coding engineers can justify their position. That adds work and cost. There's no business value in allowing sloppy prompts and then creating a bunch of work to patch the problems. Programming languages force the user to think through what they want before using the keyboard.