LLM experiences

I was impressed by Claude Code, what was most interesting was the fire and forgot nature. Just telling it to do something and then waiting. This was great for a side project, as I could mind my kids or watch TV at the same time. I imagine it will be terrible in work as one has to keep switching focus.

For its impressiveness though it was also really terrible at things. I'd have to review everything it did and anything slightly challenging it would usually butcher. Anything algorithmic though, particular if likely done many times before, was super easy. It integrated a maze library and it converted the digital representation of a maze into a PDF design in a few minutes, which I wasn't even prepared to think about how to do. It also built to various vendor APIs with ease, I only had to fix minor pieces.

I'm not fearing for my job yet, while it solves a lot of easy work, it still regularly got things wrong and didn't build things cleanly. It also didn't do well after the first one or two tries at something. I see this in the long run speeding up software development and enabling small more bespoke software to be created more cheaply.

I was overwhelmed with the amount of choice here, I used fal .ai and there was a lot I could pick from with significant variance in price and speed. A lot were poor at first attempt. My need was creating images for the start and end of the maze. I used animals to start and it did these very strangely. I then changed approach and write a prompt for Claude that had it created the two prompts for the start and end image. This did much better, so a learning there is to have LLM write the prompts you want to use in your system. I also found cost for images is also much higher then text, very much proving the point that a pictures worth a thousand words.

I started this all because I didn't like the idea of installing an app or going to a website to use LLMs. I found the experience slow and sometimes I just wanted to ask a question and come back to it later. It occurred to me email is perfect for that, as you don't expect immediate email responses. After building it, it does work well for that but to be fair it's also not fast enough for actual conversation. Ultimately what it did expose is that all the value in the LLMs are in the tools built around them by the likes of Claude and Open AP. Those they don't want to share by API, since they are probably heavily subsidizing the cost which API usage would expose. It will be interesting to see how this evolves.

So what I built will definitely be useful to me, but in reality the most useful was Claude Code for building them. The tools that I built that use AI could easily be built to not rely on AI if I wanted (except the direct LLM email one). What I've come to see is that AIs have made building tools easier but I'm not sure they themselves are the tool, and there is still large amounts of work needed to build them. There is changing coming but for now at least it will only require more effort and no less, and certainly isn't something that will bring coding to the masses.

I'll probably cancel Claude Code soon unless I think of other things to build. I had toyed with some idea of a remaining system, or something to help me aggregate my board game ideas, but neither feels very clear. For now being able to easily post to my blog is the best thing.