In reply to elm.

Since you made the Omelas reference, I find this version maps even better to our grim meathook present: https://clarkesworldmagazine.com/kim_02_24/

Omelas 1.0 didn't deliver the expected benefits. I'll try it again when they have maybe a dozen or a hundred kids or maybe all of them.

In reply to Tony.

Jesus.

Also, every one of these high-profile tech engineers should be forced to take an anthropology course. It’s honestly pathetic to hear them talk about 'humanity' when they clearly don't understand the first thing about it.

Since the 'original sin'—the theft of our collective soul—is already fully consummated, we might as well use the machine to fight fire with fire. Shouldn't we reprogram the T-800?

UPDATE:

This post has become catnip for people who want to share their anecdote of the form, "I spent a bunch of time letting an LLM do my thinking for me and it was so bad at it!!" I am now deleting all of those as they come in.

As if that was the only, or even primary problem.

You have missed the entire point of the cited article.

You have decided that using these deeply unethical tools would be acceptable if only they worked better.

You're the kind of person who is complaining about ICE being poorly trained, as if the problem with the SS officers who ran the Nazi camps was lack of training.

You are willing to look past, to quote the cited article:

ICE using LLMs as accountability sinks for waving extremists through their recruitment processes, generated abuse, how chatbot-mediated alienation seems to be pushing vulnerable people into psychosis-like symptoms, that "AI" is designed to be an outright attack on labour and education, using the works of those being attacked -- without their consent -- as the tools for dismantling their own communities and industries, all done in overt collaboration with the ultra right. That "AI" is a right-wing political project built on disregarding consent, being applied to dismantle public infrastructure and institutions.

Even as you say "but it is bad!" you are still attempting to negotiate the level of abuse that you find acceptable. Preventing abuse is not on your agenda.

Fuck that and fuck you. Make better choices.

Some Stanford PhD emailed me (unsolicited email, my favorite) to say he was "designing AI [sic] tools to help therapists" and did I want to spend (uncompensated) time talking with him to help him make it better.

I wrote back "Absolutely not.  LLMs should be nowhere near therapy, for therapists or clients. Please pick someone else's field to 'help' and leave mine alone."

That was the least-hostile version of my drafted responses.

In reply to Not Frank.

I can see it now

AI: making rockstar syndrome available to the lumpen prole. Yes! for a mere £200/month and thirty soul-shavings an hour you too will be able to drive your very own Rolls-Royce into your swimming pool[*].

[*] Rolls-Royce will be virtual, swimming pool may be imaginary.  Terms and conditions apply and are available on receipt of four fingers.  Oath of loyalty to the company required.  Offer may be terminated at any time.  Not available to women and foreign people.

I mean it's practically socialism if you look at it the right way.

In reply to elm.

It has already happened with cryptocurrency "smart contracts".

If generative AI for coding worked well, I'd expect the AI companies to be using it internally.

If the AI companies were using well-functioning AI for internal use, I'd expect them to be using it as part of developing their crawlers and other infrastructure.

The high number of badly malfunctioning AI crawlers out there seems to imply to me that the coding models they are using are really bad, and shouldn't be used for anything serious.

In reply to ayy.

It's the singularity / superexponential thing, isn't it?  In the old days, commercial fusion reactors were always 30 years away.  But today we're much closer to the singularity, snd AGI is always two weeks away.  Quite soon the Next Big Thing will be always be a few seconds away.  Nothing, you understand, will actually change, except the pitch of the bullshitters' voices.

In reply to jwz.

Apparently it should all and only be addressed by politics.

But not the kind of impolite politics that identifies problems and has feelings about them.

Not the kind of politics that raises its voice or blocks traffic or even says harsh words about slop generation and its utility in generating fascist rhetoric.

The polite and calm and dead-inside politics that thinks "law and funding priorities" are effective. The kind of politics that loves Blue Ribbon Panels and Subcommittees on Paramilitary Murderer Reform.

Surely you agree that ICE would be fine if it reported to the honorable Mr. Gavin Newsom, right?

If Generative AI "gets better" there always still needs to be some real live humans shovelling coal into the furnace hopper somewhere. Just like the Waymo drivers in the Philippines. They just won't be well paid and can be kept conveniently out of sight, and beyond any pesky legal consequences.

A quote from Brandon Sanderson (author):

“The book, the painting, the film script is not the only art. It’s important, but in a way it’s a receipt. It’s a diploma. The book you write, the painting you create, the music you compose is important and artistic, but it’s also a mark of proof that you have done the work to learn, because in the end of it all, you are the art. The most important change made by an artistic endeavor is the change it makes in you. The most important emotions are the ones you feel when writing that story and holding the completed work. I don’t care if the AI can create something that is better than what we can create, because it cannot be changed by that creation”

And from Cory Doctorow:

"Code is a liability. Code's capabilities are assets. The goal of a tech shop is to have code whose capabilities generate more revenue than the costs associated with keeping that code running. For a long time, firms have nurtured a false belief that code costs less to run over time: after an initial shakedown period in which the bugs in the code are found and addressed, code ceases to need meaningful maintenance. After all, code is a machine without moving parts – it does not wear out; it doesn't even wear down."

In reply to wwarner.

[...] mass produced automobiles [...] those things are bad [...]

Yes they are

[...] because thugs use them

No, not really.

(Yes, I have one, yes I feel bad about it.)

Quite apart from these things just being evil[*], programmers who want them are just being stupid.

For most people programming is interesting, debugging less so, reviewing code really not at all.  If LLMs work (if, note) your job will now be the last two, not the first one.  So, well done.

There is more: whatever else becoming good at anything involves, it always involves a lot of practice.  And that practice has to be recent: you are not, today, good at the skill you last used a decade ago: you can probably become good again but you're not now.  So people, not getting any practice, will be less good programmers.  Less able to find bugs in LLM-written code, less able to judge its quality.  Pretty soon new people will appear who never were proficient programmers at all because they never had to put in the hours.

More of the code LLMs are trained on will be LLM-written, and of declining quality.  Leading to model collapse.

[*] Every time I have written 'evil' or 'malign' or ... in the last five years I realise that all the times I hyperbolically used terms like that, on usenet or wherever, are just coming back to bite me.  We were all the boys who cried 'wolf': now there is a wolf, and it's got radioactive blood and poisoned canines, and we should all feel stupid for what we did.

In reply to adamrice.

If you "debate" a fan of the text generators, you should expect auto-generated text in response.

In reply to ipman.

Business Analyst

To debate someone is a sign of respect. If you know from the git-go you don’t respect their position, why bother?

In reply to cheide.

The first slop-coded bank accounting system will be a literal gold mine for anyone who bothers to explore it.

In reply to Cordwainer Fish.

That's never been proven to be anything more than an urban legend.

As far the evidence goes, it was likely made up by fans of the film grasping at straws and/or miscontextualising quotes from the Wachowskis.

In reply to Peter H.

The end state that I've seen some people propose is that the source code becomes a complete black box and nobody need ever look at it again, it'll all be managed by the AI. Just write your tests and keep reprompting until it passes the tests, and tada, you're done! Everybody's job is just 'prompt engineer' now.

To me that just seems like a way to get code so riddled with kludges and weird edge cases that it cannot be properly tested to begin with, and it'll be chock full of weird bugs and security holes.

In reply to John White.

What is a BA?

In reply to Peter H.

So those developers won't be needed over time.

AGI, two more weeks!
Hard doubt. Model would collapse when 51% of inputs are unsupervised. This will be perpetual rent on everyone using the tool.

I had some long screed that would be preaching to the choir about AI (and how it's actually even worse than Magda Gobbels's strudel) that I accidentally killed, so instead I'll just focus on this:

chatbot-mediated alienation seems to be pushing vulnerable people into psychosis-like symptoms

I really think this is basically the same thing that drives the billionaires insane: a perfect sycophant who will never say no. The billionaires get them in bulk and as humans, of course, but apparently even at the small scale it's enough to send folks off the deep end, perhaps aided by the fact that the AI will never tire of telling you how smart you are.

It's all very funny as well as being very dystopian.

It is literally impossible to get the autocorrect to produce even vaguely useful code without feeding in all existing code for context + a very verbose BA doc + a high quality QA testing plan. Meanwhile, companies that are sucking the AI roulette-revolver are laying off not just devs but BA and QA experts.

BA's are just expensive, often female versions of Gemini's meeting summary robot, right? Who needs those? And the AI can do any QA required based on what devs spat it it, surely?

On the other hand, if you have a job with seniority in a nation with actual labour protections, the superior Chinese AI models are quite useful for simple tasks. I have masturbated on work hours more this year than in all of Q1 2024. But, it's not really that satisfying, I probably need to do something more productive with my newly found free time.

In reply to Robert.

Any time one of you amoral motherfuckers pops in with "but it can be useful" all I hear is "Magda Goebbels made a great strudel."

In reply to CSL3.

It can be useful for very specific things. Like debugging code if you don't know every single piece of errata to explain a poorly documented IDE/IDF error.

A thousand examples on github of the specific thing you want it to do for it to plagiarize. If it doesn't have that then instead of returning, "IDK lol," it will use $20 worth of tokens to write garbage code, you feed it the compiler errors and it will offer to comment out the code and tell you that it's fixed the problem. $30 later it's wasted an hour and no work has been done.

The cheap AI tends to be trained on corporate eavesdropping. When I tried github copilot all the models tended to evade questions or directions and try to counsel me instead of doing a thing, "You seem frustrated."

Some things they are very good at is counting. Identifying and analyzing obscure signal protocols from logic captures and data format patterns. They're also good at refactoring some isolated snippets into time optimized assembly macros.

Mostly its purpose is to trick you into feeding it quarters.

In reply to Zygo.

The writers wanted to use the human compute farms concept.  The suits said "we're idiots and we can't understand that and we refuse to believe the audience is any smarter than we are; have them be, I dunno, fucking batteries or something".

Risk assessment for AI:  Don't share AI output with your team, because the harm will spread beyond just you.  Don't use it for anything more consequential than, say, a spam filter.

Cost assessment for AI:  Don't use AI for a spam filter, either.  If traditional spam filters aren't good enough, then humans are cheaper, and use less power and water for equivalent quality output.

The machines in The Matrix were on to something:  at some point on the output quality/power usage curve, switching from racks of GPUs to racks of humans in pods would result in a net power saving for the machines.  I can imagine a deleted scene, where the machines are in a meeting, saying, "we switched a data center from nvidia cores to humans, and now we have enough unused generating capacity to expand our population/territory/etc!" and the humans mishear that as "the machines are using humans as part of their power supply."  Yes, that line has bugged me since 1999, because the physics are nonsense without this additional context, thanks for asking.

In the real world, AI models that might be capable of replacing junior engineers today have usage fees on par with the salaries of senior engineers--but nobody really knows, because the cost of testing that theory would be astronomical.  We'd still need qualified people on staff to filter out the slop and catch the mistakes, so we can't do the layoffs that offset our new AI costs.  Or, if we ignore the warnings and do the layoffs anyway, our new lost business, business errors, and liability costs from slop and uncaught mistakes.

I used to have a joke like this:  Safe use cases for AI so far (assuming you're already OK with the copyright infringement and resource usage issues):

1. intellectual wanking
2. actual wanking
3. end of list

However, that was before it turned out that use case #1 sometimes kills people, and it seems nobody wants use case #2 to work except the end users of the AI systems (who, to be fair, have wanted to use every new technology for actual wanking since the invention of fire and the wheel).

In reply to wwarner.

I have no idea what you are trying to say with this weird collection of sentence-shaped objects.