Leftover Thoughts

Dear Slop Cannon Engineer

Someone posted this confident assertion online:

In a year or three from now, no one will be writing code by hand or reading it manually.

I won't deny that it's possible that it's true, but it seems extremely unlikely.

One reason is that it doesn't mention cost. When someone makes a broad claim like that and there's no careful argument showing how cost will make it a rational, sustainable equilibrium, you should feel free to stop reading. It's not a serious comment if they don't address cost. There's a lot to say about this, but you can find tons of content about it already out there. Heck, AI coding at scale is expensive even when it's subsidized.

The other thing it's omitting is that testing large, 100% AI-written codebases sucks. You have to consider the thousands of different ways the code can fail if you don't want to write or read it. This is nothing new. For decades, you've been able to get a job testing code even if you didn't know how to program. How is thorough testing better than reading and writing code? That's left unanswered.

Now, obviously, you need to write tests even if you write the code yourself. It's just that it's a lot uglier if the code is black box. You might not even know what to test, because since you don't understand the plumbing, it's hard to know what might fail.

It's a lot easier to write small tests as you go, correcting problems as they show up in the codebase. It's also a lot easier to fix the problems when you find them.

More than likely, you have programmers like the commenter not writing code, not reading code, and not seriously testing it. Because they're being paid whether or not they take the lazy approach. And that's when they're running a slop cannon rather than developing software.

Edit: Upon further thought, it might sound like I'm saying you need to either write/read code or do extensive testing because the LLM might make mistakes, and I suppose you do. A story from my childhood will do a better job making the point.

When I was in school, cheap pocket calculators were becoming widely available to the point that everyone had access. Some kids protested at having to learn to do mathematical calculations when calculators would trivially do them correctly every time. The teacher told them something like this. Imagine you have to calculate 3+2. You accidentally type 3-2 or 33+2. If you've never learned to calculate, those answers look correct and you'll happily use 1 or 35. If you know how calculations work, you'll immediately know something's wrong. You can trust the machine to give you the correct answer to the calculation you tell it to do, but you can't trust yourself to give the machine the calculation you need.

With LLMs, it's a stretch to trust them to always produce correct code, but even if they do, you can never trust your prompt to be 100% correct.