AI is overhyped, misused, and a bubble waiting to burst.

Is it coming for us?

No. AI is not going to replace programmers.

But the CEO of Nvidia said…

No. AI is not going to replace programmers. Mr Huang profits massively from perpetuating the Myth of an AI Utopia, as do many other CEOs. Given this huge conflict of interest, their words should be met with suspicion.

Should everyone learn to code?

Everyone should learn to code. No one should learn to code. As is often the case with two opposing extremes, neither is correct.

There are very few specialist skills that everyone would benefit from learning. Does the point really need to be argued? If no one learns to code, modern society collapses. If everyone learns to code, other specialist skills will get left behind. Why annoy a promising doctor with mandatory coding lessons that detract from their core passion and skills? It’s more important that we give everyone who is passionate about a subject the opportunity to explore it.

Stephen, are you an AI hater?

No. I do not hate AI. Quite the contrary. My university dissertation was on Neural Networks, and how they are more powerful than the Naive Bayes classifier. The constant praise of AI, every project supposedly needing AI, and people never shutting up about it, is what prompted me to write this post. AI is just a tool, so please treat it as such.

Commander Data is probably my favourite Next Generation character. I loved the Terminator movies when I was younger, and the question of whether or not human consciousness can be replicated by a machine has always fascinated me. Could an AI run essential services and production while we relax and explore our own interests? Is it conscious, and if so, would we be creating a race of slaves? Or perhaps, the machines would enslave us, like in the Matrix.

There are a lot of complex questions in a world with strong Artificial General Intelligence, but we aren’t there yet, and personally, I’m far from convinced that it’s close.

ChatGPT was revolutionary!

Yes it was, and I use it to this day (it generated the thumbnail for this post). ChatGPT was a giant leap forward. The idea of having a human like conversation with a program, one that is a subject matter “expert” in everything, is insanely useful.

Long conversations about code often lead me to learn new syntax, or consider other approaches, though I rarely find it useful to copy code verbatim. The code such models generate is more like a guide post, one that should be treated with skepticism.

Just glancing at my conversations, I’ve asked it everything, from mixing spices for chicken, how various subcultures of music developed, and how a suit should fit properly at the shoulders (I may have watched John Wick just prior to that conversation).

ChatGPT and similar systems are a superior Google search, that contain a huge amount of knowledge and can tailor responses in a very powerful way. So long as you keep your critical thinking hat on, large language models are an amazing tool and it’s not surprising that they’ve made waves.

Vibe Coding

This one hurts. The amount of b***sh*t on the internet about this is off the charts, especially on LinkedIn.

I can just picture the marketing team at Anthropic, rubbing their hands as they read posts from people pushing for full teams of AI agents, or claiming that AI is better than Senior Developers and completely replaces Juniors. I currently consider such notions to be ridiculous, but I’m open to changing my view on that when and if AI becomes substantially better than it is now.

A Vibe Coding Success

So. After getting sick of hearing about it, I paid for one month of Claude Code. The idea of interacting with an AI via the CLI was rather appealing, and Claude Code was meant to be very good at generating code.

I run Arch Linux on my main desktop, but I didn’t trust letting Claude Code loose on that, so I setup an Ubuntu machine using virtual box, and ran Claude inside that. I gave it a relatively simple task. I told it to make me a Rust equivalent of the Python HTTP server. After five minutes of it working and asking me permission to change a few files, I had a working server.

This surprised me. I was expecting the program to not compile, or require manual changes. I then asked Claude to add support for basic auth, detailing what I wanted, and what the command line interface would look like. It worked for another five minutes, and again, got it right first time.

At this point I was seriously impressed. While it’s not a hard project, it certainly would have taken me much longer than ten minutes to get a working file server in Rust with support for basic auth.

As much as I am negative about AI in a lot of this article, it must be admitted that this kind of capability simply didn’t exist five years ago, and was borderline science fiction ten years ago.

A Vibe Coding Failure

Spurred on by the initial success of Claude, I decided to give it a more complex, less standardised challenge. You see, I was concerned that anything web-server related would be a bad test, as there is a huge amount of potential training data online for anything web server related, though that is less true for Rust than something like Node.

I asked Claude to create me a small text adventure game, with about fifteen minutes worth of game-play. The core idea was that the player would explore their environment, but have to die, and come back to life multiples times, each time retaining more memory to ultimately understand their situation.

Claude was given an excruciating level of detail about the requirements. The actions the player could take, the concept of rooms, the plot, the hidden secret, the complex death and rebirth mechanic, everything.

Things did not go smoothly… After over half an hour of generating code, it gave me a program that would not compile. As part of my experiment, I was unwilling to write code myself, but I told it where it had gone wrong, and it managed to fix it. Unfortunately, the very first terminal output when running the game, gave away the main plot, that the player is in-fact a clone.

I asked Claude to change the text to not reveal this detail, and to modify any other user facing text that would give this away before the correct time. It managed to do this, though rather funnily it also asked my permission to change a bunch of variable names that the player would never see.

So, after redacting the plot reveal, I tried to play the game. I went North, then West. I fought a Robot and won, then another in a different room, but I quickly realised I was trapped. There was the classic locked door, but no way to get through it, at least none that I could find. For the first time, I looked at the games code, and realised that Claude had rather sneakily put the key for the door, in a room that is beyond that door…

I gently explained this logical impossibility, and asked it to move the key into a room that’s accessible right from the start. This took one or two tries and further explanation before it fully understood it’s own trickery.

Death’s a b*tch

Playing the game again, I managed to get the key and progress further, but was killed by a more powerful Robot after getting through the locked door. This is intentional, as each time you die you regain more information and become stronger. However, where do your items go on death? A crowbar is required to complete the game, but if the player picks it up, then dies, the crowbar is gone, and there are none remaining.

I wanted flexibility here. The player should be allowed to pick up another crowbar from the original box of crowbars, but also have the option of looting their corpse. This would be a trivial change for a developer to make, but it took nearly an hour and lots of retries before Claude finally managed to create a solution where only one crowbar could be obtained, and always re-obtained after death.

I was forced to explain in agonising detail, multiple times, all the possible combinations that could occur, and how to handle each, until it finally lucked into a reasonably valid solution. At least I think so. After testing it, I think it may still be buggy here.

At this point I was getting annoyed. I’d spent far more time messing around with this than I had wanted to, but I really wanted to know if I could make it work without writing any code myself. After eventually solving the crowbar problem and adding a subtle nod to Gordon Freeman from the Half-Life series, I played again.

This time, I didn’t die enough. I discovered it was possible to defeat all enemies before dying the minimum number of times. The game had hard-coded a number of required deaths before accessing the final secret. This felt like it should be trivially solvable. Instead of random damage, we need a system that appears random. A system that guarantees the minimum number of deaths. A game developer would understand this very quickly. Claude never gained this understanding. After another full hour of back and forth, it never managed to implement such a system, and it was disappointing that it failed to do this after maxing out all of my tokens in a nearly three hour session.

Advice

AI is a powerful tool, but not one that should code your applications for you. I recommend using it as a more powerful search engine. If asking about code, you must have some knowledge yourself, at least enough to review its solution. If you are asking for code that you couldn’t write yourself, you’re on dangerous ground.

There is enormous potential for security vulnerabilities by using AI tools. People accidentally upload secret keys in their prompts, or copy and use insecure code without even looking at it. No matter how much training you give people, a double digit percentage still click the phishing link in an email. If we assume a similar percentage just copy code and use it, without understanding it, we are in for a rough road.

At the time of writing, there have been a large number of hacks recently in the UK, most infamously Marks & Spencer. Senior developers know all too well, how complex modern build systems and environments can be. All it takes is one bad docker file, one malicious dependency, or a badly configured CI/CD setup, and you’re toast.

Trusting AI to handle any of this without oversight is a recipe for disaster, and I foresee an increasing number of hacks due to poorly configured Vibe coded solutions that leak like a sieve.

Do not replace Juniors with AI. Juniors learn by solving problems themselves, and become Seniors.

Do not let your skills atrophy by gorging on AI.

Use it to enhance your productivity while still running the show yourself.