Bad LLM Code, 4,000 Lines or Less
I didn’t get bad code because an LLM can’t write software. I got bad LLM code because I gave it too much freedom.
I was working on a new feature for TrueMath this past week. I’d already written it before, but it lived behind an admin wall, and I wanted to elevate it for customers. The change came with some structural overhead, but at its core this was mostly a lift-and-shift exercise.
So I started a session with my trusty LLM, fed it a bunch of code, and said, “Here’s the basic structure, here’s what I did before—write this in the new style.”
The original code was about 2,000 lines. The LLM wrote 4,000 lines and still wasn’t done. And the code was spaghetti. There is no way I could have supported that long-term.
After a few days, I stopped writing code, started a new chat with the same LLM, and changed my instructions. This time I explicitly told it to lift and shift within the new structure and to write as little new code as possible.
It nailed the core in a single response. The rest took me a couple of days to finish up. The result? Clean, easy to update—and about 2,200 lines of code. Perfect.
One of the beautiful things about using an LLM to write software is that there’s a clear way to check whether what it did was correct. We run code through a compiler. We inspect outputs. We write automated tests and run them again and again.
Most things we ask LLMs to do don’t work like that.
We ask it to do research, and unless we double-check everything, we have no idea whether it made parts of it up. We ask it to draft legal documents without knowing whether it missed important laws in our jurisdiction.
We ask it to do math.
Did it use the right formulas?
Did it use the right data?
Did it make assumptions it never told us about?
Software gives us compilers, tests, and repeatability. Other domains—research, law, and especially math—don’t. Until they do, we’re still guessing. Just faster.
Reach out: elia.freedman@truemath.ai
Learn more: truemath.ai
Sign up for early access: https://app.truemath.ai/signup
