New Mexico is seeking an injunction to permanently block Snap from practices allegedly harming kids. That includes a halt on advertising Snapchat as “more private” or “less permanent” due to the alleged “core design problem” and “inherent danger” of Snap’s disappearing messages. The state’s complaint noted that the FBI has said that “Snapchat is the preferred app by criminals because its design features provide a false sense of security to the victim that their photos will disappear and not be screenshotted.”
“Heather” also tested out Snapchat’s search tool, finding that “even though she used no sexually explicit language, the algorithm must have determined that she was looking for CSAM” when she searched for other teen users.
But literally in just the previous paragraph:
Posing as “Sexy14Heather,” the investigator swapped messages with adult accounts, including users who "sent inappropriate messages and explicit photos.
Gee, I wonder how the algorithm could’ve possibly suggested these users. What a mystery.
I’m not defending Snapchat here - they’re a scumbag company with a scumbag product and they should be held responsible for enabling the sharing of CSAM on their platform - but it doesn’t just match you with random predators out of thin air. They went in with specific keywords in their username and a pattern of account engagement.
That’s nuance that I hadn’t considered and I appreciate you pointing it out. I’m not on any of these sharing platforms so I have no idea what they’re like and that made it easy to overlook this detail which is probably pretty relevant.
Despite Snapchat setting the fake minor’s profile to private and the account not adding any followers, “Heather” was soon recommended widely to “dangerous accounts, including ones named ‘child.rape’ and ‘pedo_lover10,’ in addition to others that are even more explicit,” the New Mexico DOJ said in a press release.
wtf
How the fuck do you get even more explicit than that
Yo I can give you a more explicit username if you want one.
it’s more how was it allowed.
Shouldn’t it take one person reporting that user for an instant ban?
Ethical triumph or new disaster?
Yes.
Tough call. If you put out bait, you’re gonna get someone. But would that person have done the same thing if they had not seen your bait? Chicken and the egg. On one hand, it looks like entrapment.
I mean, that part isn’t really at issue here. It’s fundamentally the same technique that’s been used since the 90’s, famously on To Catch a Predator. Seemingly, the “entrapment” angle has been settled.
But now they can argue that they aren’t sexually attracted to children, just AI artwork, which is technically not an image of a child. And unless I missed it, they were not trying to meet the girl.
The problem is going to be that images that aren’t real of a crime aren’t a crime. Of the opposite was true, images of murder would be illegal. Can’t just cherry pick.
If I draw a stick figure and label it “naked girl,” does it become child porn? What if I’m a really good artist?
I believe that cartoon images depicting sex of underage kids is still illegal. At least in the US.
Feel free to correct me if I am wrong but seems like I remember this from a news article a while back. Maybe it was just a specific state.
I am not going to Google that one though to find out though.
https://cellebrite.com/en/ai-and-csam-a-look-at-real-cases/
Best I could find about this.
Imo as long as the ai was not trained on actual CSAM and the product is not depicting real people, then it shouldn’t be illegal as it is not hurting anyone which is why we have laws against CSAM in the first place.
What if it normalises CSAM and some people don’t discerne between real and AI?
And what if video games, movies, and books normalize killing? There is no evidence to show that it does or that it will.
We have porn games, but we don’t have CP games. There’s a line between violence and SA with minors.
Edit: oh wait, Japan might be an example 🙃 and yeah, they got issues.
Yeah don’t Google it hahaha
It what makes it a child? There’s some creepy anime girls who definitely fall into that questionable category. And if I label a stick figure with an age… does that make it illegal? What about an ai image with bubble text that says “I’m not real. I’m 18, I have a magical curse on me etc etc” now it’s fiction?
Since it isn’t actually real… what is the line, and how can that line be measured? Since this is just going to keep being a problem, this awkward conversation needs to happen in a logical, calm manner.
https://cellebrite.com/en/ai-and-csam-a-look-at-real-cases/
Best I could find about this.
Imo as long as the ai was not trained on actual CSAM and the product is not depicting real people, then it shouldn’t be illegal as it is not hurting anyone which is why we have laws against CSAM in the first place.
I definitely don’t want to sound as if I’m promoting this material, but I agree. Fake things are fake and real things are real. Yeah, it makes a lot of people uncomfortable to think about it and I totally understand.
Fake images of murder seem to be perfectly fine! And that’s arguably the worst crime possible. We show that shit to our kids.
Isn’t it sad that we even have to say we don’t promote it before we say anything else?
I think the bar is whether it could be reasonably mistaken for a real child. Which makes quite a lot of disgusting content legal.
I also find it to be repugnant, but if the images are not based on real people and the ai was not trained on real csam(good luck proving this either way), then it shouldn’t be illegal. The laws were made to protect kids, and drawings of purly fictional characters are not hurting the kids.
Pretty much every law ever made in the history of humanity that was ostensibly to protect children is actually about control of the population.
This is just plain wrong.
Obviously, there are loads of laws and very good legislation that does indeed protect children.
Just one example: child labour laws.
I suspect that what you really mean is that whenever a politician says whatever police powers are required to protect children, they really just want more power to violate privacy to make it easier to prosecute various crimes.
When the “AI artwork” is made for the specific purpose of representing underage children and is indistinguishable from the real thing, that argument is going to get flattened pretty quickly.
Pretty easy to present a couple pages to a jury of kids pictures (not nude) and say “tell us which ones were AI”.
Well that’s not how the law works. A jury wouldn’t decide that. They don’t get to say “close enough for fake to be real” because real has a name and fake doesn’t have a name.
The crime is possessing a photo of a thing that happened. A real person was abused. The fake image, like a realistic painting of a fictional event, does not actually involve a person getting hurt.
Let me set up a situation: A person creates a very realistic, but fake, video. They first have the character walk onto the screen in a wireframe. Then, the animation begins to build on texture and now we have a person on a green background. They look real, but we know for sure it’s not real. Then the background enters in the same way. Now the video appears to be real, but… we know it’s not. Just like movie, those are just special effects even if it looks pretty believable.
The crime is a documented event of someone being hurt. If there was a video of a person actually killing a person, that video could be considered evidence of a crime. But if that event was staged as part of a video intended as entertainment, there is no crime and that video isn’t real.
Of course, the topic of child abuse is difficult to talk about. One may make the statement that fake images lead people to the real thing, and that would encourage people to do bad things. Well, they said the same thing about video games—so we would obviously need to apply the same laws to them. Movies and books about crimes could also encourage people to commit crimes, so those need to be banned entirely, and my huge collection of horror movies could put me in jail for life.
The line becomes impossible to draw.
And if that video built to feature a child performing fellario, it would still be child pornography
No, the voiceover will say in an adult voice “I am over 18, I have a fictional disease that makes me look like a younger.”
And now it’s not a child, because “youth” is subjective at this point.
There is very legal real commercial porn of adults engaging in “age play” and “incest,” both which are illegal in reality. Some of those videos would make you think, “how are they actually 18?!”
On the flip side, in most US jurisdictions, incest is illegal and an actual video of two adults engaging in that would be evidence of a crime and they could be prosecuted.
In conclusion (haha), real is illegal and fiction is not.
Don’t get me wrong though, I’m not advocating for specific things to be LEGAL, just arguing that a law making something that is FICTION illegal could be difficult to prove in many circumstances, and lead to many false accusations.
Ok, so when the police use a young-appearing officer to nail somebody who is looking to hook up with what is listed as a 14yo… that’s just going to get dropped? Because it seems that’s a tactic that’s been used often enough and the actual age of the officer matters less than the intent of the perp to engage in sex with a minor.
The argument “I just thought we were role-playing” isn’t significantly different from “I totally know this ‘14yo giving a BJ to old man.mp4’ was AI generated and that’s just my fetish, not real kids”
That’s not what this article is about though.
They’re not saying “this user looked at our image so they’re a pedo and must go to jail”.
They’re saying snapchat is full of pedos, and using the proliferation of this account as evidence supporting that claim.
I agree completely with you that Snapchat is an unmonitored disaster that gives the user the impression that the person they are sharing their nudes with cannot save the content. A good portion of the videos on porn sites have that little Snapchat progress wheel on them and are clearly screencaps.
Aside: I think there is a big topic that conveniently gets overlooked because it’s so much easier to blame “social media” or the “predator,” and that is—where are the fucking parents?
The parents are out buying an AR-15 for their son.
On a more serious note, parents always get judged, just not always fairly.
IANAL, but my understanding is entrapment is when they convince you to do something you might not otherwise have done. So if the cops create an account of a minor and message an adult asking if they want to fuck, and the answer is like “uh no, absolutely not,” and then the cops follow up by repeatedly sexting, and the adult blocks their account, but the cops relentlessly keep sexting from burner accounts, and plant people in the adult’s work and social environments who keep talking about how normal it is to fuck minors who sext you out of the blue, and then the adult is finally like “oh whatever, fine” - that’s entrapment.
Now, most people still are literally never going to take the minor up on the offer, no matter how relentless they are or how normalized it is in their environment. That’s true about most crimes. The question is how many people wouldn’t have committed that crime unless this very specific police-created situation came up, and that difference is what falls into entrapment.
I’d argue this isn’t even close to entrapment, because all they did was set up an account much like all the others that exist, and waited for others to find them. It’s no different from leaving a bike unlocked, then catching somebody who steals it. There are unlocked bikes everywhere, and people don’t suddenly decide to steal the only bike of their life because they happened to find that unlocked bike.
Of course, they could also be spending this time and money getting to the root of societal issues and fixing the core problems instead of catching a small percentage of active pedophiles and letting the rest of them continue to cause irreparable harm.
The last paragraph is the big issue. Fix society. We can argue all day long about what line does “artwork” cross before it becomes illegal, but that’s not actually preventing anyone from getting abused.
And imo, it seems a little sick to say, “we made a bunch of kiddie porn that didn’t exist previously, and I’m going to distribute it to catch criminals—using tax dollars” … tf?
In general I think using AI imagery, and catfishing in general, is basically entrapment. In most civilized countries; that’s illegal for police to do.
Now if they begin to actually trade in actually legitimate forbidden materials…sure; by all means arrest for that charge alone. That wouldn’t be unjustified. But provoking someone who might then turn around and harm a real child, seems wrong.
Maybe we should just make those AI tools public and free and leave it at that. If you monsters want kiddie porn, have this AI generated stuff. It’s sort of like having needle exchanges or safe using centers for addicts. It doesn’t solve the problem, but it makes it safer for everyone.
If we’re talking about it from a harm reduction or public health view point then it is the circulation of the content, not the personal generation that poses the greatest risk. Once the material gets circulated, it becomes harder to know it’s origins and it might look like someone IRL and expose them to risks.
I don’t understand how this would be fine but pedophiles generating them at home without distributing them would be illegal.
Cause the cops are creating CSAM and putting it out there in some shape or form, which you could argue would encourage pedophiles same way circulating ai images would as well.
Either generating under age sensitive material is always wrong or it’s not.
Federal law is creating fictional CSAM at home without transmitting it is legal.
The few AI arrests I’ve seen they transmitted them.
Huh, that’s very different from what other people have said before
https://en.wikipedia.org/wiki/Legal_status_of_fictional_pornography_depicting_minors
Section 1466A of Title 18, United States Code, makes it illegal for any person to knowingly produce, distribute, receive, or possess with intent to transfer or distribute visual representations, such as drawings, cartoons, or paintings that appear to depict minors engaged in sexually explicit conduct and are deemed obscene.
Specifically: distribute, receive, or possess with intent to transfer or distribute visual representations
By the statute’s own terms, the law does not make all fictional child pornography illegal, only that found to be obscene or lacking in serious value. The mere possession of said images is not a violation of the law unless it can be proven that they were transmitted through a common carrier, such as the mail or the Internet, transported across state lines, or of an amount that showed intent to distribute.[135]
Edit: So it has to be 100% locally generated and never transmitted. You still wouldn’t want to try and fight this in court though, I’m sure they’d do their best to throw you in jail, or fabricate an intent to distribute if you’d made a lot, even with all the images you generated to try and get the images you actually wanted.
Edit: Also this is federal, there may be other state laws.
I mean with the intent just means that possession is illegal cause all it takes is the judge to decide there was intent.
I guess I was right that possession is essentially illegal too.
The prosecutor needs to convince the jury there was intent. Intent is one of the hardest things to prove beyond a reasonable doubt in court without evidence and motive. It’s not a guarantee especially with no history.
Edit: And in this case, a history means you’ve probably already been convicted as thats what the history would lead to.
If it gets to a trial instead of a plea deal, but fair enough, that is a rather reasonable guardrail.
Ya, I imagine many of these highly specific cases would plead out to something not involving jail and/or pedophile record. Stakes are pretty high and that’s a big gamble. A case like this it’d be hard to have a sympathetic jury.
The article didn’t say the cops generated AI CSAM. It said they created a profile pic, which was shown in the article.
So if someone generates a minor’s image and it’s not nude, is that not CSAM?
I’m genuinely asking, I always thought it was about sexualizing children, not whether they are nude or not.
I don’t think so. People keep throwing that acronym around but I suspect they didn’t read the article and find out that it was one normal picture of a high school-aged girl.
I actually read it and then made a comment because even though it’s a profile picture, the intent is to have a viewer sexual the picture and thereby sexualizing a minor.
I do get how it’s a normal picture, but it made me think of this slippery slope and where the line is.
In an alternate universe:
Cops help pedophiles with AI pics of teen girl. Ethical triumph or new disaster?
I feel like there’s two stories here:
-
Snap is a pedo grooming ground
-
Police are generating fake CSAM
-
is, well, yeah . Like any major network you have creeps, Nazis and pedos and it’s good to root them out. It’s weird that themselves don’t do that.
-
is … difficult. I feel like the cops that prompt GenAIs to produce (sexualized) images of children risk severe trauma. It must be punishing to ask a a machine to produce fake images of morally reprehensible material. That’s gotta take a toll on you. What’s even more important is that those people could be spending their time taking down actual child abuse images.
I can’t wrap my head around that.
-
already been a movie along these lines
What does Chris Hansen have to say?
deleted by creator
So glad I got rid of that bullshlt long ago.
Congratulations on kicking your ai generated kiddie porn habit!
I left snapchat long, long before AI. And I am glad I did!!
