Happy Friday!

Look, I know everyone is probably pretty tired of all the AI talk, but I wanted to share my impressions of using Claude Code for a day on my reko.day app. There are still many things for me to learn about how to use Claude effectively, but first impressions are still useful.

First Contact

Unlike a lot of other AI coding tools, Claude Code runs in your terminal and operates more like a really junior developer. You give it basic instructions instead of crazy engineered prompts, and it gets to work.

I had it first look at my stuff and summarize its findings. After a while, a summary popped out that was really quite impressive. It figured out my monorepo layout, the technologies, and the overall architecture.

I asked it to put its findings in the Claude.md file so it would keep this understanding.

Oh, did I mention it told me my codebase was really well-written and tested? It did, and it is.

A Simple Refactor

Well, even if it's well-written, that doesn't mean it can't be improved. I pointed Claude at something in my codebase that I never liked and asked it to suggest a refactoring.

What it suggested was reasonable, but it had some trade-offs it didn't acknowledge—either way, I wanted to let it cook and see how it went.

It made the code changes and then broke the project file.

I fixed the project for Claude, and it attempted to run the project again. However, it ran the wrong configuration, and I had to stop it before it began affecting production.

I asked it to run the tests. It tried, but again with the wrong configuration, and I corrected it again.

My impression was that this was a junior developer who spent a lot of time studying code, but had a really poor sense of working on any kind of software project.

A Small Feature

With that little trial finished, I asked Claude to create a plan to add a very small feature to the application: Entering a preferred email address. It went through and analyzed all of the code again and came up with a plan that I had problems with, but would work technically. After making some changes to Claude.md to hopefully address the issues with running in production, I provided the final instructions.

"Execute your plan, but you must follow TDD."

Like most developers, it did not follow TDD. It wrote all of the tests all at once, checked to make sure they all failed, then tried to write all the implementation at once and see if the tests passed.

They didn't.

This started a death spiral for Claude as it began to try to rapidly whack-a-mole the failures. It tried rewriting tests so they never fail, second-guessing its overall plan, and creating strange levels of duplication across files to try and patch up holes it created.

I intervened, and we made progress again, slowly. Although it had a hard time following my instructions, I had to interrupt it repeatedly.

Remember when it messed up the project file? Well, it spent well over an hour trapped in a loop around this, too. It would write code to read the project file multiple times, switch languages from Python to Ruby when that didn't work, then it would write code to write the project file. Finally, after all of that, it left the project file broken, and it wanted to reset its changes and try again.

My session expired while this happened.

When I had time again, I told them never to add or remove files again and just ask me to do it instead.

Progress continued.

After about four solid hours of this, we completed two-thirds of the work, although I will delete half of it because it is not good.

My Thoughts

What makes Claude Code unique isn't its capability; it is its design. It can utilize agents and MCP systems to incorporate vast amounts of context and operational modes.

Unfortunately, the core problems of it, hallucinating, making poor choices, doubling down, and failing to recognize issues, are the same as ever.

I want to continue using it to see what changes over time, and I also want to conduct a few experiments. In particular, I want to see if it can build a model of the code using tests faster or better than what it does now.

If you're using it or considering Claude, think of it as someone who is highly educated but at their first job.

Sincerely,

Ryan