I work at a medical research institute that has worked on "artificial intelligence" long before the rise of LLMs. That's not what I work on here, but I get exposed to their work and have conversations.

I also have friends (because I'm a nerd) and colleagues (because I'm a Product Owner/Manager) who are programmers, and some of those are using LLMs to develop software.

And of course I have had coworkers that went early all-in and started to ask ChatGPT about everything.

Oh, and I have friends who are professional writers and illustrators and other artist types who are genuinely furious about the copyright infringements and afraid for their livelihoods.

On top of that, we have lots of conversations in some friend groups about the ecological, financial, social, and political effects all this has.

So, when I think about or discuss this new technology with others, things are.. complicated.

• My leftist and green gut instinct says to go all-in as a Luddite and declare Butlerian Jihad on all things AI.
• My inner sense of wonder looks at the possibilities of the technology and can't help but ask "what if?"

First: Testing - what can this actually do?

Because I am a curious person, I have tried out LLMs to

• create pictures
• answer random questions
• create document scaffolds for my work
• write whole documents for my work
• create roleplaying game scenarios
• create code

Yes, there are way more things we can do with LLMs and other AI tools, but those are the ones I actually did myself.

How did that go?

Creating pictures

Yeah, that'll work, and they are usable, but I find no artistic merit in them, and haven't come across one that spoke to me in any special way.

On top, there is often a very noticeable "look" to them, which is immediately noticeable

But even if they weren't: For anything that I would want to publish or use commercially, I'd rather ask a friend or even pay someone to do it.

The only upside is that tasking an LLM to create a picture taught me a bit more language of what I want from a picture, so I unintentionally got better at explaining a human artist what I want from them.

Answer random questions

Usually useless to me, nothing a proper websearch can't do at least as good, often even better. Tried it, dismissing it

Create documents scaffolds

This actually can be useful - especially if it's in a field or a format I'm not too familiar with. The more I already know about the thing, the less upside I get from the LLM, but if it is a format I have no idea of, I'll get a useful starting point to put my own texts in.

Write whole documents

For me? Useless. I have a "voice" when writing texts, and teaching the LLM that voice is more work than it saves me - and I don't write enough similar documents that it is worth to teach that skill in a more permanent way, because different document types get different voices, and.. ach, no.

Also, it's not just the voice, it's also the content. I don't write boilerplate documents - the ones I write are usually pretty specific for a certain audience about a topic that I have just researched or have specific domain knowledge in.

Create roleplaying game scenarios

I wrote about that in a separate blogpost. The gist? The system isn't going to write an adventure for me. But it can keep me engaging during the thinking process and make the process a lot faster. At least for certain types of scenarios.

Create code

hoo boy. I talk to Isotopp regularly, and he has been deep-diving into this lately, and published a lot of blogposts about his discovery process. This one for example, about LLM-driven development.

So, I piggypacked on his insights and had a usecase: Compiling the pile of files for my new roleplaying game (all written by myself without any LLM) into one coherent markdown file.

At first, I tried this in an interactive session with the free tier of Lumo, and got to some.. acceptable results.

Then I installed OpenCode and used a different model and went to set up the harness, context, instructions for the agent, and so on. All in all about 30 minutes of setup work, plus about an hour of back and forth with the agent, slowly filling my work folder with user stories, tickets, ADRs, test descriptions...

...and when I asked it to actually build the thing, I ended up with an immediately working Obsidian plugin. Not working perfectly, but I had actually working code that did what I told the system to build.

I have now spent a few more hours on this, and the result works exactly as if I'm talking to a remote team of developers that work of my human-readable documentations and requirements and produce again human-readable tickets, ADRs, release notes, and so on.

There is now an Obsidian Community Plugin in my vault that probably would have taken a human developer a week to make, maybe longer. The agent has nearly a 100 tests that it checks against before deploying new code, and I haven't even once run into the "here, I fixed this" - "no, you didn't" - "you're absolutely right, but NOW I fixed it." - "actually, no, you..." loop.

I gotta say it: This will not make developers obsolete, but it will change how developers will work. It requires a good knowledge of how to design software and what specifications are, but it absolutely works.

And for things that I didn't do, but others: Have a look at CorridorKey That is a tool, generated with the help of AI tools, that cleanly separates very complex images from a green- or bluescreen background. That has, between the filming of the original Mary Poppins and now been something that was very hard to do manually. Or things like Chatto, or the ArtCraft tool suite.

The ethical problems

I'm not gonna sugarcoat this: The whole field is rife with medium to big ethical problems. These include questions like these:

• Who's really benefiting from these things?
• Who has their work stolen from them by LLM training on unauthorized copies without compensation?
• Whose work is devalued?
• Where is the electricity and other resources for the data centers coming from?
• What is the stock market AI-frenzy doing to our global economy?
• What is the datacenter craze doing to our environment?
• How can we ensure that the gains, money, time savings, whatever are distributed in any sensible way to everyone, not just the billionaire class?
• How does education need to change so that people don't become mere meat proxies, and will still be able to do all kinds of things without resorting to an LLM?
• How can we make LLMs good internet citizens, so that agents and training data ingestion systems don't put too much load on every web server, and only ingest data that has been expressively cleared for ingestion? (Some systems report that nearly all of their load comes from crawlers instead of people...)
• what does doing things with an LLM actually cost? How much energy etc. do I actually burn? This is pretty opaque, especially with cloud providers.

Those are not trivial questions, and more often than not, the current answer skews heavily into "LLMs rely on stolen work and only benefit the billionaire-class"

On top of that, even if the training material for the LLM training were ethically sourced (all indications say they are not), the other costs the rest of us bear directly or indirectly are considerable.

Answering these questions is not a technical task. It is a political one, and one that most current politicians and governments are, to my knowledge, fail to acknowledge, let alone tackle. I have a good idea which directions the answers should take, but I'm really not qualified here.

Looking to the future

I am cautiously optimistic that at least the data center issues will be resolved in the medium to long term, as models are getting a lot better at being executed on local computers instead of relying on specialised data centers.

I am quite pessimistic about the political, economical and as a result regulatory landscape. Where politicians talk about "AI", they talk either about it in terms of a space race, where any given nation or bloc needs to get "ahead of others in AI", whatever that means, or they regulate the very end of the output chain: How do we mark pictures or videos, so people don't get tricked by them?

That said: My own experiments tell me that humanity has created a very, very powerful tool. One that can and will transform knowledge work, and probably also add tons of little improvements to other sectors. And yes, those will include potentially very harmful things, like more or less autonomous weapons.

Thus it is imperative that we talk to our politicians to get them to tackle the whole problem at a more fundamental level.

For now, I will personally not use AI to replace or create published creative work (writing, visual arts, etc.), and will not even attempt to earn money with products created through those means. (Published meaning: Visible in a public or near-public space)