AI is really worrying and I fear this sort of image generation is tip of the iceberg

[mids2019]

https://www.bbc.co.uk/news/articles/c98p1r4e6m8o

I don't know to what extent current legislation covers this but to my mind any woman with an image on the net could be prone to this. Are we going to reach a stage where our daughters are going to simply not want any image taken of them for fear of how it could be manipulated?

Am I right in thinking this is very similar to the Hentai, Anime stuff that Japan has had for decades ?...Not tech wise obviously, but stimulation for its appreciative audience ?!

Thank you.

What exactly are you defending about this @PeppercornMill?

Governments don't want to regulate AI (or make AI companies pay royalties to the people whose work they scrape) because they see it as a money tree. Afaik it costs so much to run the systems that scrape and generate that the lot will implode. Meanwhile, don't use it if possible.

I think all photos will just become as believable as drawings. The front page of one of the daily newspapers has a picture of the ski nightclub with champagne sparklers apparently setting fire to the ceiling with written underneath it something like: picture might have been altered or enhanced.
I think all photographs will be as believable as the portraits royals used to send each other to consider for marriage in the Middle Ages.
No one will believe naked photos are real they may as well be sketches of imaginary women.

Not just a money tree, I think they are scared that if they repudiate it, others will still use it and gain real and serious advantages, economically and politically.

It's a reasonable fear imo.

the men are all over it

Men are perfectly able to control themselves and refrain from being "all over it".

Because you are not an immature incel.

Or you are and are trying to pretend you haven't been warped by the patriarchy and the recent explosion of pornography as a norm and are imbue the AI you have helped to created with all the anti woman, sexist, hatred that incels spew out in posts.

This shows that it is the human values that are creating this.

First that the programmers didn't include (which they could have done) that AI would not generate sexualised images of childre.

Secondly that as men's dominate narrative is not contained in the virtual world, as it is in a limited way IRL.

The virtual world has no police, it has no courts, and a sign of human values is seen by humans to be somewhere they / men are not constrained by values of decency let alone respect for women.

https://www.thecanary.co/trending/2026/01/03/grok-images-abuse/

I'm fairly sure it's not easy to prevent AI from producing images which are, from it's own perspective (if you can call it that) totally arbitrary. Without also very likely preventing it doing things that are useful or in some way interfering in its generative abilities.

You could tell it not to accept certain prompts, but I don't think it's going to be difficult for anyone who is motivated to find other prompts that will produce something similar.

It's not like it understands what it produces in any way.

I think this really nails it for me that this year I'm going to delete my Facebook account, and reset one up which is just for things like marketplace.

but why did no one make any fuss when photoshop came out as that could do excatly the same but years ago ?

Probably because a batch of images that would have taken a year to produce in PS by someone very skilled can now be made in a second or two by anyone with an internet connection.

id say days with photoshop etc

Too many of them right now are not controlling themselves. The internet, as pps, have said, has catered to the lowest common denominator since its inception and has not been good for women. This website was built and thrives because of the male domination and misogyny in technology.

Photoshopping has existed almost as long as photographs have. The Victorians photoshopped using tools they had available. And before that - well paintings of nudes are hardly rare…

I briefly worked in tech (financial systems), a complaint we had from users was that there was a bug when trying to do X. When you spoke to the tech team it was very clear that no one had programmed the system to do X, users had found a work-around to do it.

I am not defending the act, but defending the tech. It's like moaning at camera producers that they're being used to photograph child abuse, "why did the camera producers make this technology that allowed such as a thing?".

It's clear with all of these AIs they have restrictions in place at producing such content, but users are still finding ways around the restrictions.

With AI if I want to produce a picture of a cat, I can do so, but there wasn't a programmer there who specifically had to program producing cat pictures.

Agree. Which means that international rules must be set up - a long diplomatic slog with lots of countries thinking they can do better without regulation.

I like the idea that photos will just lose their power as depictions of ‘real’ things. It takes away some of the terrifying power of this. It’s sad for documentary photography and will make it difficult to record events for historical purposes though. Could this truly be the advent of the ‘end of history’ ? I know it’s not what Fukuyama meant exactly but are we seeing the advent of the ‘last woman’ here? Will we just learn to accept that naked and pornogrified images and video of almost all of us will circulate amongst the men in our lives as once we accepted that they might draw us naked and/or being violated? Will we learn to prepare our young daughters for this as they approach puberty as we now prepare them for menstruation and take them for first bra fittings? What a god awful thought, but best to prepare ourselves as it seems unlikely that these are the inevitable consequences of this.

Sorry - likely - not unlikely.

I've read some suggestions that we are at the "end of pornoraphy." Basically that for those who use it regularly, it's gone about as far as it can, and it becomes diminishing returns.

I expect that is an over-statement, but I can imagine an endpoint where depictions have become so completely bizarre and so available, that it's actually difficult to create any kind of arousal any more.

And it's not just the male-centric image based stuff, the fan-fic/romance/erotica kinks of stuff used by women has become increasingly bizarre and available.

It's like some weird road to species burn-out.

The point is that it is humans who created AI.

And just as in the fictional world of robotics there were laws that would be programmed into a robot, eg dont kill humans, you could and should do the same with AI.

Admittedly like bringing up a child you would hope that telling it it should not produce pornography, depict gross acts of violence, there will always be rogue or underground versions willing to do that.

The real fear is not that this couldn't be done, and isn't because the dominate male culture just isn't bothered about it, but that at some time in the future, we will all have become so dependent on AI that in fact we no longer know how to think or create for ourselves. And rely on AI to do our thinking for us.

There have been newspaper stories, which may or may not be true, of AI programs becoming independent of human control.

And that no one will now how to challenge that.

ie the sort of future as shown in the film Idiocracy where humans rely on preprogrammed systems and have no idea how they are created or how to change them.

And that will happen if we as humans choose not to challenge the dominant male narrative.

AI at the moment is a misnomer- there is no intelligence involved. It is merely predictive based on what has gone before. But just as images become so corrupted so we no longer know what image is true and assume none are, so too will it be the case with other information. We may well be entering a new information dark age.

Entering? It’s been coming for a while.

A different but related, and upsetting, thing I saw earlier today and haven't seen mentioned yet.

Yes, there are more ways to harm with fake images than porn.