You must log in or # to comment.
TL;DR: The new Reimage feature on the Google Pixel 9 phones is really good at AI manipulation, while being very easy to use. This is bad.
This is bad
Some serious old-man-yelling-at-cloud energy
It’ll sink in for you when photographic evidence is no longer admissible in court
Photoshop has existed for a bit now. So incredibly shocking it was only going to get better and easier to do, move along with the times oldtimer.
Photoshop requires time and talent to make a believable image.
This requires neither.
But it has been possible, for more than a decade
You said “but” like it invalidated what I said, instead of being a true statement and a non sequitur.
You aren’t wrong, and I don’t think that changes what I said either.
Lmao, “but” means your statement can be true and irrelevant at the same time. From the day photoshop could fool people lawyers have been trying to mark any image as faked, misplaced or out of context.
When you just now realise it’s an issue, that’s your problem. People can’t stop these tools from existing, so like, go yell at a cloud or something.
Well yeah, I’m not concerned with its ease of use nowadays. I’m more concerned with the computer forensics experts not being able to detect a fake for which Photoshop has always been detectable.
As the cat and mouse game continues, we ask ourselves, is water still wet?
Just wait, image manipulation will happen at image creation and there will be no “original”. Proving an image is unmanipulated will be a landmark legal precedent and set the standard for being able to introduce photographic evidence. It is already a problem for audio recordings and will be eventually for video.
I really don’t have much knowledge on it but it sound like it’s would be an actual good application of blockchain.
Couldn’t a blockchain be used to certify that pictures are original and have not been tampered with ?
On the other hand if it was possible I’m certain someone either have already started it, it is the prefect investor magnet “Using blockchain to counter AI”
How would that work?
I am being serious, I am an IT and can’t see how that would work in any realistic way.
And even if we had a working system to track all changes made to a photo, it would only work if the author submitted the original image before any change haf been made, but how would you verify that the original copy of a photo submitted to the system has not been tempered with?
Sure, you could be required to submit the raw file from the camera, but it is only a matter of time untill AI can perfectly simulate an optical sensor to take a simulated raw of a simulated scene.
Nope, we simply have to fall back on building trust with photo journalists, and trust digital signatures to tell us when we are seeing a photograph modified outsided of the journalist’s agency.
Yep, I think we pictures are becoming a valuable as text and it is fine, we just need to get used to it.
Before photography became mainstream the only source of information was written, it is extremely simple to make a fake story so people had to rely on trusted sources. Then for a short period of history photography became a (kinda) reliable sources of information by itself and this trust system lost its importance.
In most cases seeing a photo means that we were seeing a true reflection of what happened, especially if we were song multiple photos of the same event.
Now we are arriving at the end of this period, we cannot trust a photo by itself anymore, tampering a photo is becoming as easy as writing a fake story. This is a great opportunity for journalists I believe.
There has never not been a time when photography was not manipulated in some way, be it as simple as picking a subject and framing it in a specific way can completely change the story.
I really enjoy photography as a hobby, however I find it a bit embarrasing and intrusive to take photos of other people, so my photos tend to look empty of people.
I will allways frame a picture to have no or as a very few people in it as possible.
In general I don’t edit my photos on the computer, I just let them speak for themselves, even if that story is a half truth.
We have never been able to trust photographs completely, though you make a good point about truth in numbers, that won’t go way just because of AI.
The big issue now is how easiy it is to make a completely believably faked photo out of an existing photo, we have been able to do this for decades, but is has been way, way harder to do.
As for the blockchain making photos valuable, we tried that, NFTs as a concept is dumb and has failed, I don’t believe that NFTs will be the future of ownership.
Image manipulation has always been a thing, and there are ways to counter it…
But we already know that a shocking amount of people will simply take what they see at face value, even if it does look suspicious. The volume of AI generated misinformation online is already too damn high, without it getting more new strings in it’s bow.
Governments don’t seem to be anywhere near on top of keeping up with these AI developments either, so by the time the law starts accounting for all of this, the damage will be long done already.
On our vacation 2 weeks ago my wife made an awesome picture just with one guy annoyingly in the background. She just tucked him and clicked the button… poof gone, perfect photo.
But it’s never been this absolutely trivial to generate and distribute completely synthetic media. THAT is the real problem here.
Yep, this is a problem of volume of misinformation, the truth can just get buried by one single person generating thousands of fake photos, it’s really easy to lie, it’s really time consuming to fact check.
That’s precisely what I mean.
The effort ratio between generating synthetic visual media and corroborating or disproving a given piece of visual media has literally inverted and then grown by an order of magnitude in the last 3-5 years. That is fucking WILD. And more than a bit scary, when you really start to consider the potential malicious implications. Which you can see being employed all over the place today.
Honestly yeah I agree. Many mainstream social media platforms are infested with shitty generated content to the point of being insanity.
Okay so it’s the verge so I’m not exactly expecting much but seriously?
No one on Earth today has ever lived in a world where photographs were not the linchpin of social consensus
People have been faking photographs basically since day one, with techniques like double exposure. Also even more sophisticated photo manipulation has been possible with Photoshop which has existed for decades.
There’s a photo of me taken in the '90s on thunder mountain at Disneyland which has been edited to look like I’m actually on a mountainside rather than in a theme park. I think we can deal with fakeable photographs the only difference here is the process is automatable which honestly doesn’t make even the blindest bit of difference. It’s quicker but so what.
It used to take professionals or serious hobbyists to make something fake look believable. Now it’s at the tip of everyone’s fingers. Fake photos were already a smaller issue, but this very well could become a tidal wave of fakes trying to grab attention.
Think about how many scammers there are. Think about how many horny boys there are. Think about how much online political fuckery goes around these days. When believable photographs of whatever you want people to believe are at the tips of anyone’s fingers, it’s very, very easy to start a wildfire of misinformation. And think about the young girls being tormented in middle school and high school. And all the scammable old people. And all the fascists willing to use any tool at their disposal to sow discord and hatred.
It’s not a nothing problem. It could very well become a torrent of lies.
It used to take professionals or serious hobbyists to make something fake look believable.
Your point being…?
I mean…we can all see those are inanimate, right? But that doesn’t even change my point. If anything, it kinda helps prove my point. People are gullible as hell. What’s that saying? “A lie will get halfway around the world before the truth has a chance to pull its boots on.”
A torrent of believable fakes will call into question photographic evidence. I mean, we’ve all seen it happening already. Some kinda strange or interesting picture shows up and everyone is claiming it was AI generated. That’s the other half of the problem.
Photographic evidence is now called into question readily. That happened with photoshop too, but like I said, throw enough shit against the wall—with millions and millions of other people also throwing shit at the wall—and some is bound to stick. The probability is skyrocketing now that it’s in everyone’s hands and the actually AIgen pictures are becoming indecipherable from photo evidence.
That low effort fairy hoax made a bunch of people believe there were 8in. fairies just existing in the world, regardless of how silly that was. Now, stick something entirely believable into a photograph that only barely blurs the lines of reality and it can be like wildfire. Have you seen those stupid Facebook AI pages? Like shrimp Jesus, the kids in Africa building cars out of garlic cloves, etc. People are falling for that dumbass shit. Now put Kamala Harris doing something shady and release it in late October. I would honestly be surprised if we’re not hit with at least one situation like that in a few months.
Come on, science fiction had similar technologies to fake things since 40s. The writing was on the wall.
It didn’t really work outside of authors’ and readers’ imagination, but the only reason we’re scared is that we’re forced into centralized hierarchical systems in which it’s harder to defend.
I mean, sure, deception as a concept has always been around. But let me just put it this way:
How many more scam emails, scam texts, how many more data leaks, conspiracy theories are going around these days? All of these things always existed. The Nigerian prince scam. That one’s been around forever. The door-to-door salesman, that one’s been around forever. The snake oil charlatan. Scams and lies have been around since we could communicate, probably. But never before have we been bombarded with them like we are today. Before, it took a guy with a rotary phone and a phone book a full day to try to scam 100 people. Now 100 calls go out all at once with a different fake phone number for each, spoofed to be as close to the recipient’s number as possible.
The effort input needed for these things have dropped significantly with new tech, and their prevalence skyrocketed. It’s not a new story. In fact, it’s a very old story. It’s just more common and much easier, so it’s taken up by more people because it’s more lucrative. Why spend all of your time trying to hack a campaign’s email (which is also still happening), when you can make one suspicious picture and get all of your bots to get it trending so your company gets billions in tax breaks? All at the click of a button. Then send your spam bots to call millions of people a day to spread the information about the picture, and your email bots to spam the picture to every Facebook conspiracy theorist. All in a matter of seconds.
This isn’t a matter of “what if.” This is kind of just the law of scams. It will be used for evil. No question. And it does have an effect. You can’t have random numbers call you anymore without you immediately expecting their spam. Soon, you won’t be able to get photo evidence without immediately thinking it might be fake. Water flows downhill, new tech gets used for scams. The like a law of nature at this point.
Wise people still teach their children (and remind themselves) not to talk to strangers, say “no” if not sure, mind their own business because their attention and energy are not infinite, and trust only family.
You can’t have random numbers call you anymore without you immediately expecting their spam.
You’d be wary of people who are not your neighbors in the Middle Ages. Were you a nobleman, you’d still mostly talk to people you knew since childhood, yours or theirs, and the rare new faces would be people you’ve heard about since childhood, yours or theirs.
It’s not a new danger. Even qualitatively - the change for a villager coming to a big city during the industrial revolution was much more radical.
That’s exactly what I meant when I said:
It’s not a new story. In fact, it’s a very old story.
And you just kinda proved my point. As time has gone on, the great of deception has grown with new technology. This is just the latest iteration. And every new one has expanded the chances/danger exponentially.
What I really meant is that humanity is a self-regulating system. This disturbance will be regulated just as well as those other ones.
The unpleasant thing is that the example I’ve given involved lots of new power being created, while our disturbance is the opposite - people\forces already having power desperately trying to preserve their relative weight, at the cost of preventing new power being created.
But we will see if they’ll succeed. After all, the very reason they are doing this is because they can’t create power, and that is because their institutional understanding is lacking, and this in turn means that they are not in fact doing what they think they are. And by forcing those who can create power to the fringe, they are accelerating the tendencies for relief.
I don’t think this is the power redistribution you’re implying it is. I’m not actually sure what you mean by that. The power to create truths? To spread propaganda? I can’t think of any other power this tech would redistribute. Would you mind explaining?
The new technique distorts reality in a much larger way. That hasn’t been there before. When everybody has this in their smartphones, we will look at manipulated pics on an hourly basis. That’s unprecedented.
deleted by creator
There was actually a user on Lemmy that asked if the original photo for the massacre was AI. It hadn’t occurred to me that people who never heard of the 1989 Tiananmen Square protests and massacre would find the image and question if it was real or not.
A very sad sight, a very sad future.
Photoshop has existed for years. It’s no different than a student in 2010 being shocked at the horrors of man and trying to figure out how it could be faked with a computer. People have denied the Holocaust for generations!
This argument keeps missing that it is not only the quality but mainly the quantity of fakes which is going to be the problem. The complete undermining of trust in photographic evidence is seen as a good thing for so many nefarious vested interests, that this is an aim they will actively strive for.
It is different. The old Photoshop process took a lot of time. Now an image can be manipulated incredibly quickly and spread almost as fast before anyone has time to do anything about it.
Were they from the .ml instances?
How is it sad? If they’re young and/or don’t have the best schooling, it’s not their fault they haven’t heard of it. And then they encounter an absurd picture and approach it with skepticism? That’s not sad at all. Healthy skepticism is good, especially with the influx of AI generated content
Awful title.
Clickbait 101
It’s the verge, after all. Nobody should read their slop
Relevant XKCD. Humans have always been able to lie. Having a single form of irrefutable proof is the historical exception, not the rule.
Regarding that last panel, why would multiple people go through the trouble of carving lies about Ea-Nasir’s shitty copper? And even if they did, why would he keep them? No, his copper definitely sucked.
The obvious conjecture is that they were trying to commit fraud and get free copper
interesting thought. we haven’t had photos in history, and people didn’t need them. also, we’ve been able to produce text deepfakes all throughout history (and people actually did that - a lot) and somehow, humanity still survived and made progress. maybe we should question our assumptions whether we really need a medium to communicate absolute truth.
humanity still survived and made progress
Humanity never needed truth, for all of that. Only a good enough illusion.
It’s just that, most of the times the illusions are not good enough and the truth comes out.
We literally lived for thousands of years without photos. And we’ve lived for 30 years with Photoshop.
The article takes a doomed tone for sure but the reality is we know how dangerous and prolific misinformation is.
The Nazis based their entire philosophy on misinformation, and they did this in a world that predated computers. I don’t actually think there’s going to be a problem here all of the issues that the people are claiming exist have always been possible and not only possible but actually done in many cases.
AI is just the tool by which misinformation will now be spread but if AI didn’t exist the misinformation would just find another path.
Except it was way harder to do.
Now call me a “ableist, technophobic, luddite”, that wants to ruin the chance of other people making GTA-like VRMMORPGs from a single line of prompt!
You know that’s not possible right?
if I as an anti-AI person said that, I’d be called out for posting FUD…
have you considered just not listening to AI bros and not letting their opinions upset you
What are you talking about lol
This is a hyperbolic article to be sure. But many in this thread are missing the point. It’s not that photo manipulation is new.
It’s the volume and quality of photo manipulation that’s new. “Flooding the zone with bullshit,” i.e. decreasing the signal-to-noise ratio, can have a demonstrable social effect.
It seems like the only defense against this would be something along the lines of FUTO’s Harbor, or maybe Ghost Keys. I’m not gonna pretend to know enough about them technically or practically, but a system that can anonymously prove that you’re you across websites could potentially de-fuel that fire.
Them - and F2F.
Meh, those edited photos could have been created in Photoshop as well.
This makes editing and retouching photos easier, and that’s a concern, but it’s not new.
Something I heard in the photoshop VS ai argument is it makes an already existing process much faster and almost anyone can do it which increases the shear amount that one person or a group could make almost how a printing press made the production of books so much faster (if you’re in to history)
I’m too tired to take a stance so I’m just sharing some arguments I’ve heard
Making creating fake images even easier definitely isn’t great, I agree with you there, but it’s nothing that couldn’t already be done with Photoshop.
I definitely don’t like the idea you can do this on your phone.
Exactly, it was already established that pictures from untrusted sources are to be disregarded unless they can be verified by trusted sources.
It is basically how it has been forever with the written press: Just like everyone now has the capability to manipulate a picture. Everyone can write we are being invaded by aliens, but whether we should believe it is another thing.
It might take some time for the general public to learn this, but it should be a focus area of general schooling within the area of source criticism.
almost how a printing press made the production of books so much faster
… and we all know that lead to 30 years of bloody war, btw
We’ve had fake photos for over 100 years at this point.
https://en.wikipedia.org/wiki/Cottingley_Fairies
Maybe it’s time to do something about confirming authenticity, rather than just accepting any old nonsense as evidence of anything.
At this point anything can be presented as evidence, and now can be equally refuted as an AI fabrication.
We need a new generation of secure cameras with internal signing of images and video (to prevent manipulation), built in LIDAR (to make sure they’re not filming a screen), periodic external timestamps of data (so nothing can be changed after the supposed date), etc.
I am very opposed to this. It means surrendering all trust in pictures to Big Tech. If at some time only photos signed by Sony, Samsung, etc. are considered genuine, then photos taken with other equipment, e.g., independently manufactured cameras or image sensors, will be dismissed out of hand. If, however, you were to accept photos signed by the operating system on those devices regardless of who is the vendor, that would invalidate the entire purpose because everyone could just self-sign their pictures. This means that the only way to effectively enforce your approach is to surrender user freedom, and that runs contrary to the Free Software Movement and the many people around the world aligned with it. It would be a very dystopian world.
It would also involve trusting those corporations not to fudge evidence themselves.
I mean, not everything photo related would have to be like this.
But if you wanted you photo to be able to document things, to provide evidence that could send people to prison or be executed…
The other choice is that we no longer accept photographic, audio or video evidence in court at all. If it can no longer be trusted and even a complete novice can convincingly fake things, I don’t see how it can be used.
There’s no need to make these things Big Tech, so if that’s why you are opposed to it, reconsider what you are actually opposed to. This could be implemented in a FOSS way or an open standard.
So you not trust HTTPS because you’d have to trust big tech? Microsoft and Google and others sign the certificates you use to trust that your are sending your password to your bank and not a phisher. Like how any browser can see and validate certificates, any camera could have a validation or certificate system in place to prove that the data is straight from an unmodified validated camera sensor.
What in the world is going on with Elsie’s hand in the “second of the five photographs?”
I was thinking about those pictures! The garden is magic enough without there needing to be fairies at the bottom of it. I’m not sure if the saying is linked to these forgeries, but I always kind of thought it was.
Even a few months ago it was hard for people with the knowledge to use AI on photos. I don’t like the idea of this but its unavoidable. There is already so much misinformation and this will make it so much worse.
I don’t believe there’s misinformation because we fail to discern the truth though. Misinformation exists because people believe what they want to believe.
These photoshop comments are missing the point that it’s just like art, a good edit that can fool everyone needs someone that practiced a lot and has lots of experience, now even the lazy asses on the right can fake it easily.
Yeah, it is going to be mainly a quantity issue rather than a quality one. The quality of faked photos has already been high since photoshop. Now a constant growing avalanche of high quality fakes (produced by all sorts of different vested interests with their own particular purposes) is going to barrage us on a daily basis, simply because it is cheap and easy
Photography manipulation existed almost since the invention of photography. It was only much harder see the famous photo edition https://www.history.com/news/josef-stalin-great-purge-photo-retouching
Great point. But tools that make it so a 10 year old can manipulate photos even better than your example in several minutes, are in fact fairly new.
Hell they can generate photos that fool 70% of people on Facebook, though now that I say that, maybe that bar isn’t too high…
This is only a threat to people that took random picture at face value. Which should not have been a thing for a long while, generative AI or not.
The source of an information/picture, as well as how it was checked has been the most important part of handling online content for decades. The fact that it is now easier for some people to make edits does not change that.
Your comment somehow just made me realize something: When we see/read news, we have to trust the one who’s telling them to us. Since we weren’t there in person to see it with our own eyes. Therefore, it’s always about a “chain of trust”.
This is true no matter whether photos can be manipulated or not. People have been able to lie since humanity exists. Nothing has really changed. Photography, just like globalization, has only brought everything closer together, making it easier to have a more direct, straightforward relationship to other people and events. With the beginning of AI, this distance between you and an event is going to increase a bit, but the core mechanics are still similar.
I kind of wonder, how do we really know that something is true? Do atoms actually exist? What if we’re being lied to by our authorities. You say “of course not”. But what if? I mean, if we blindly trust authorities, we end up like the republicans, who believe everything that fox news tells them. How, then, do we discern truth?
How, then, do we discern truth? I guess we have to give “proof” for everything, in the sense of mathematical proof. Something that everybody can follow, using only their fundamental assumptions and deduction. I guess that is what science is all about.
People can write things that aren’t true! Oh no, now we can’t trust trustworthy texts such as scientific papers that have undergone peer review!
The Verge are well versed on writing things that are untrue
