This is a post that really only talks about what folks call vibe coding. This is not about writing books, creating movies, or drawing pictures with LLMs. (And don't get me started on the tedious stuff that is currently the mode de jour for 3D model crowdfunding campaigns)

As I've mentioned in the past two blogposts, I've been trying to figure out how this vibe coding thing works. The experiment is now concluded. you can have a look at the Obsidian Vault Compiler and the Obsidian2Typst plugins on GitHub.

I chose these particular projects to scratch a personal itch: Getting the nearly 53 thousand words roleplaying game book that was spread across 387 files reliably and repeatedly into a coherent document that I can pass around for proofreading and use in printed form at my own table.

Of course I looked at existing Obsidian plugins and tools, but none were really suited to my personal purpose without a massive reorganisation of my files and writing workflow.

The reasons I chose vibe coding was that

1. I am not a developer by a long way, so writing it myself was out of question.
2. let's face it, the particular use case is obscure enough that chances are slim someone else picks it up out of sheer excitement
3. although I am living comfortably, I do not have the extra cash to pay a developer the two to three weeks worth of work for this
4. I have developer colleagues and friends who do work with LLMs. Any job advert for Product People asks for these skills, and I felt I needed to understand what this is all about, how it actually feels working like that, and what the capabilities are.

The reasons that deterred me from doing all this up to now are clear enough, I wrote this in the last post:

My leftist and green gut instinct says to go all-in as a Luddite and declare Butlerian Jihad on all things AI.

How are these thing trained, who benefits from it, who gets hurt, whats the ecological, social, and equality damage? These are hard questions, and the answers to all of them aren't entirely clear, but certainly not good in a leftist-green sense.

Still, the need to have something working for my workflow was getting bigger (my editor wanted a printout), so I went at it.

And now, in the aftermath, there's also the taste question: But what about slop code?

The short answer: I do not care.

Years ago, when I was still with polypoly, my boss always asked prospective new developers the same question during the interview:

Do you think that developing software is an artistic or an engineering task?

The point then wasn't the actual choice, but how they reacted and discussed this with us. Because none of the candidates had the answer ready at hand, and even our own answers varied from interview to interview.

Today, my own answer is that the actual writing of code is mostly an engineering task, but the decisions that lead up to actually writing if..then clauses and so on are rather artistic, political and social science based.

The question that we did not ask, but maybe should have, and that is relevant to the "is LLM written code slop?" question is "How do we measure code quality?"

What is code quality?

Again, I am not a developer, I am a product person. From my point of view, quality code is something that

• satisfies the user story
• does not harm the compute environment nor the user
• does its job with at least a moderate amount of, but ideally minimum of resources spent
• can be maintained - meaning security updates and feature changes or additions can be implemented without too much fuss.

As a product person, code readability, the thing being somewhat aesthetically pleasing, is not a concern for me. I don't care how the code for a working-for-my-userstories system reads or looks, as long as me asking for security updates or feature changes is not hindered. I cannot read the code anyway, nor appreciate any aesthetic concerns.

And I daresay that most non-developer end users feel the same.

Readability and aesthetic merits of code are a quality measure that is important for teams of developers, so they can collaborate and take over each others work effectively. If they have to go through unreadable spaghetti code from a predecessor that creates friction that delays the process or even makes it nigh impossible to fix something without breaking something else.

When your development team consists of just one person, and you never plan to add someone else to that team, whatever this person likes, is the chose standard.

We have common standards because we want the onboarding of new people to be as painless as possible, and we want to make sure that the project can survive a changing of the guard.

That is, in my humble opinion, and again, not a developer, the sole reason for the code quality.

Code Quality and LLMs

Now, I have used LLMs to create two Obsidian Plugins, and I have not once even looked at the code. I have not the foggiest idea how it looks, what exactly it does, how it does that, or if there are any security issues I should have looked for.

I do know though, for example, that the plugin code works fine when I don't connect my machine to the internet, and that it doesn't open any connections at runtime, because I made that a requirement of the code, and the agent made me test that.

I do know that it fulfils all the requirements I threw at it, from "this is how things should be put together" to "these things should not be touched or changed"

And frankly, when I work with human software teams, that is the exact same outcome: I get code functionality that does what I asked for, and I do not look at the code ever.

I also have a corpus of user stories, tickets and test descriptions that are human readable and that I can check and give another LLM agent to work with. And yes, I tested that with a different agent, and it did indeed pick up pretty seamlessly where the other left things.

The experience (as a Product Person)

For these "just for me projects", this is, I'll be honest, the ideal working environment. I work as a product person as usual, and get something that does what I want. Specifying how the process to get there should look like, what to observe doing it, how to document, and so on, is exactly like my day-to-day work. The only difference, from my perspective, is that it is even more strict on the development side:

The LLM, given a thought-through harness, did not forget to apply a test, walked through the jobs step-by-step, and wrote down any errors it made to prevent itself from doing them again.

Frankly, without wanting to throw shade on any of the developers I've worked with in the past and present, this was quite the productive and positive experience for me. Mostly because the feedback loop between us was so fast!

Specifying something, the LLM interviewing me about the peculiarities, then writing user stories, tickets, tests, code and finally running the tests and code, giving me an artefact to review usually took way less than an hour, instead of days.

Martin Seeger dared a look ahead:

AI is reducing the cost of software development so drastically that custom solutions—which would have been completely unthinkable just a few years ago—are suddenly becoming economically viable.

This could lead to a renaissance in custom software and call into question the business model of large segments of the established software industry—especially in areas where vendors have traditionally relied on the fact that in-house development was too expensive for their customers.

Based on this experiment, I daresay he is right. Of course, he is also right when he followed up with the issue of maintaining, operating and securing said custom software. And we still haven't tackled the ethical questions about where the models get their training from, who got exploited, and who got rich.

But from a more practical point of view, I do not see a way to put this particular genie back into the bottle, nor do I really want to.

Yes, I do want my cake (developing with LLMs) and eat it too (have an equitable, green, and fair society where everyone flourishes).