The Importance of Watermarking and PhotoGuard: Navigating the Challenges of AI-Generated Content

As we continue to explore the capabilities of artificial intelligence, particularly in the realm of image generation, it's essential to address the issue of authenticity. The advent of watermarking technologies like PhotoGuard has sparked a crucial conversation about how to verify the legitimacy of images created using these models. Watermarking, essentially, relies on the fact that legitimate developers implement these security measures with the intention of preventing unauthorized use. However, with the open-sourcing of these models, there's a growing concern that anyone can access and exploit them, rendering the watermarking system ineffective.

This raises significant questions about the responsibility that lies with those who develop and deploy these AI-generated content tools. Can we truly trust that developers will adhere to best practices, or will they succumb to the temptation of exploiting this technology for personal gain? The implications are far-reaching, and it's crucial that we recognize the risks associated with unchecked access to these models.

One of the primary concerns is the potential for malicious actors to train their own versions of these models using readily available resources. This would allow them to bypass existing security measures and create counterfeit images that could be used to deceive or manipulate others. The notion that anyone, regardless of expertise or resources, can replicate this technology with relative ease sends a chilling message about the vulnerability of our systems.

Moreover, as AI-generated content continues to proliferate, we're faced with the daunting task of keeping pace with its rapid development. The field is advancing at an unprecedented rate, making it increasingly challenging to predict the consequences of these technologies. While it's impossible to anticipate every possible scenario, it's essential that we acknowledge the potential risks and work towards mitigating them.

Dealing with the current technology as it exists, while also considering the future implications, presents a formidable challenge. As researchers and developers, we're caught in a paradoxical situation where we strive to improve these models for the greater good while also acknowledging their limitations. The question remains: how can we balance innovation with responsibility?

One area of concern is the open-sourced nature of these models. While this has enabled researchers like ourselves to explore the capabilities and limitations of AI-generated content, it also raises the specter of unauthorized exploitation. If anyone can train their own model on a relatively low-cost setup, then the security measures we've implemented become increasingly irrelevant.

This brings us back to the importance of legitimate developers taking responsibility for implementing these security features. It's not enough to rely solely on the benevolence of individuals; we need concrete steps to ensure that these models are used responsibly. This might involve developing more robust systems or establishing regulatory frameworks that hold those who develop and deploy these tools accountable.

As we move forward, it's essential that we consider the broader implications of AI-generated content. The potential benefits are undeniable, but so too are the risks. By acknowledging these challenges and working together to address them, we can ensure that these technologies are used for the greater good. This requires a multidisciplinary approach, involving researchers, policymakers, and industry leaders who are willing to collaborate and prioritize responsible innovation.

Ultimately, the future of AI-generated content holds much promise, but it also demands our attention and commitment to addressing the challenges that come with it. By engaging in open and honest discussions about these issues, we can work towards creating a future where these technologies are harnessed for the betterment of society, rather than exploited for personal gain.

As we look ahead, it's clear that the next few years will be critical in determining the course of this technology. Will we find ways to mitigate the risks and ensure that AI-generated content is used responsibly? Only time will tell. But one thing is certain: the conversation about the implications of these technologies has just begun, and it's an essential dialogue that we must continue to have.

"WEBVTTKind: captionsLanguage: en- Okay, Hadi, so we areworking in this field of AIfor a couple of years now,but it seems that, you know,AI is always exciting, but itbecame even more so recently.So what are some of thethings that really get you,like interested and evenmore and more excitedabout the future right now.- Yeah, no, the past likeyear, couple of years,have been very, veryexciting in this field.Like we are observing really,really powerful AI models,whether with, you know, ChatGPTand large language models recently,and even like slightly before thatwith like these amazing diffusion modelsthat allow us to like,amazing generative AI modelsthat allow us to like,very easily generate super,highly realistic imageswith a very, very, you know, easy waywith just like textual prompting.You can now any one of us, very easilyinteract with these modelsand generate amazing images.You want the image of a dog, you know,in New York on a skateboardthat will give you an amazing,you know, image, super realistic.And you can imagine how,what amazing stuff you can do with this.Basically translating our imaginationand everything that wethink of into actual imagesand maybe in the future videos as well.It's really awesome.Like, this is very, very exciting for me.- So just walk me through.So essentially like imagine I am an artistor I'm not an artist, Ijust want to, I have a hobbyor I just want to have a nice illustrationfor my Christmas card.I just type in what I wantand one of these modelsjust give this to me.- Yeah, exactly.Like previously, like you have to likemaybe learn Photoshop, liketo create like an amazing,you know, nice, like avery nice photo or like,you want to, like, you should hire someoneto like create like a nicecard for you for Christmas,for example or like anyother celebration like,now you just, like, whatever in your mind,you just talk to the computer,literally talk to these AI models,tell them I want the imageof a blah and a blah.Like you can createwherever you want literallywith very, very ease.Like, everyone can do that.This is the amazing thing.Like, it's super easy and super good.Like, it's amazing howgood it is right now.- So you mentioned Photoshop,so it's not only about likecreation something completely new,but you can also modify existing photos.- Yeah, yeah, definitely,certainly like thesemodels are not only about,yeah, generating things fromscratch and your imagination.You can actually bring photos,like the photo of you and your wife,you would like to goto Hawaii for example,but you can't afford that, for example,you can just like Photoshop thatin a very easy way right now.Like you don't have to knowanything about Photoshop.You can create amazing photos.You can edit your photos with your wifeor with your friends in avery super realistic way.And very easily you just specify,oh, this is the photo of me and my wife.I want like to change thebackground to be, you know,in Maui or Hawaii or anywhere,like anywhere in Hawaii,for example, anywhere in the world.You will do it immediately,like in one second,literally it's amazing.- Okay, so that's super exciting.I'm sure my wife will beexcited about this as well.She will definitely prefer thatto actually going to Hawaii.So, but like now that startsto get me a bit worried, right?So essentially like we are not usedas a society to this kind of, you know,ease of creation of likevery realistic imagesor modification of imagesthat anyone can do.So aren't you worriedabout it a little bit?- Yeah, it's certainly an amazing,but also like, certainly like very riskyand like it definitely worries me,like, although we can likedo amazing stuff with it,we can also like do amazinglygood bad stuff with it.Like we can easily like imaginenow people using these toolsto like create, you know, fake images,spreading them online,spreading misinformation.They might create, you know,a fake image of some explosion somewhere,which will cause like, you know,stock markets to react to this,you know, fake image or video.It'll cause like maybetension between countriesand even on a more mundane level,like people might be ableto like can right nowmanipulate other people like their friendsor their like otherpeople's images in a bad wayand maybe like blackmail themif they're actually bad peopleto give them money, otherwisethey will spread these,you know, maliciouslymodified or edited images.All of us have imagesonline right now, right?So like everyone literally,if there's a maliciousactor without any experiencewith like Photoshop or anything,they can bring these images,download them from wherever they are,use these AI models to editthese images maliciously,make us look very bad,make us do something that'slike we don't want to doand then blackmail us to actually,you know, give them money,otherwise they will spread,so it's pretty risky,like, there's so manythings that can go wrongwith this technology basically.- Okay, so that's a lot.And actually quite scary,so let me unpack this.So first of all, you mentioned kind of,you know, kind of comingup with some fake images,some explosions, actuallyrecently this happened.- Yeah.- There was an explosion,well purported explosion at Pentagonthat kind of went onTwitter and now it's X.- Exactly.- And essentially this caused some ripplesbefore people realizethis is a fake image,but like what you're saying isactually even more maliciousand even more inside you.So you are saying, okay,now if I want to take anyphoto of let's say you online,you know, you have it onInstagram, on Twitter,on your webpage.- Yeah.- And then I can use it usingthis AI advances to create,like to put you in somecompromising situation.And I can say, look, Hadi,if you don't pay methis much of a Bitcoin,I will just spread it around.And yes, you know, thiswill be a fake photo,but you know, just the damage of youhaving to explain it to everyone and so onjust might be not worth.And you say, okay, I will pay you.And of course it might notwork if actually, you know,if I'm your friend becausethen you know who I amand there will be a lot of social costs.But I guess you canimagine that like scammers,like professional scammerscan just do it online,like can do this at scale becauseyou essentially don't needhuman in the loop in that.And essentially that'ssomething that happens.- Yeah, I think like this,like doing it at scaleis in particular very dangerous.Like before, like ifthere's a malicious actor,they have to like be veryprofessional, you know,at Photoshop, et cetera.And like they have to do itlike one image at a time,or they have have to recruitmany people to do it.Now, a person, a single person,single malicious actor can do it at scalethat can modify theimages or edit maliciouslythe images of thousandsof people immediately.This is very dangerous.Like it's ease of use,it's scalable natureis very, very, it makesit actually in particular,you know, dangerous ina much more serious waythan like how Photoshopis dangerous, for example.- Yes, so I completelyagree with that actually.Like I was testifying in Senateand one of the other witnessesthere was actually a womanwho was essentially a, youknow, she was a victim of a scambased on essentiallylike someone called herand played to her, the clipof her daughter saying,okay, I got kidnapped.And then they kind of, these people say,okay, you need to give us ransom,otherwise something willhappen to your daughter.Of course, as you can imagine,this was an extremelytraumatic experience except,and that's the good part,is that it turned out thather daughter is totally fineand this clip of herdaughter kind of sayingthat I was kidnapped wasactually fake using AI.So this is kind of a bitmore individualized effort,but yes, above the kind of thingsthat we used to trust implicitly,now we need to learn not to trust.And again, this scale of it,that's something that makes meworried quite a bit as well.So future is grim, is thereanything we can do about it?- It's a very, very seriousproblem and that's why we like,we have to do something about it.And that's what, you know,why we are trying todo something about it.And this is exactly what wetry to do in our, you know,recent project, recent research paper,you know, PhotoGuard.We're trying to attempt tolike solve this problem.We're trying to see if there's anythingthat we can do there.So in this project, what we tried to do,or the idea that wewanted like to go afteris like is there a waywhere we can actually preventthis from happening by,you know, maybe modifying the image,our images that we upload onlinein a way such that they, you know,hinder any attempts ofmanipulating them by these models.So this is like the idea thatlike we were thinking of like,oh, is there anything we can do there?And indeed we tried andwe actually have somethingthat seems to be working andit's a prototype of course,but like, it's just likeshows that we have stuffthat we can do there.We are able to like, you know,protect our images in a specific way.- Yeah, so let me kind ofunpack it a little bit.So first of all, you know,before we actually explain in more detailwhat this is doing,actually there's a nice story about this.- Yeah.- So can you kind of tell ushow actually this project came about?Because I think that's pretty cool.- Yeah, totally, totally.So like, you know, I was watching,like I'm a big fan of Trevor Noah.I was watching the Daily Show,you know, before it stopped,back like in December.And there was this like, you know,episode with like Trevor Noahinterviewing Mira Murati,the CTO of OpenAI.And they were discussing,basically it was like the timewhere DALL-E too came out.And it's like an amazing, you know,very powerful AI modelfor like generating,you know, images and also like edit.It has like the capabilityof editing images.It's one of these like verypowerful generative AI models.So they were discussingthis and Trevor asked Mira,it was like, he was likeexactly discussing this, like,oh, this is amazing,but this is super scary.Like he was literallymentioning all of these risksthat we were talking about right now.And he was like, oh, isthere anything we can do?And Mira responded, of course,like, we have to do thingsand you know, you know,things have to happenand like they're tryingto do things, et cetera.But like what we were talkingabout maybe with watermarkingand other things, but then I was like,oh yeah, I think I know oneway that might be able to like,you know, fix this problem.And we, given that like wehave worked a lot on like,you know, adversarial examples,I got like this idea of like,why don't we use these things?And I can explain later aswell what these things are.Why don't we use this technologyto actually protect our images?Like it might be actuallyvery easy to do that.So that's where the idea came from.I was literally having dinner with my wifeand literally watching this episodeand we just came up with this idea.- Okay, so let's back up a little bit.So what we're trying to do in PhotoGuardis kind of develop this procedurefor kind of immunizing your photosthat you upload online from, you know,them being edited using this a AI models.Okay, so that's the idea.Somehow we would like to make surethere's some like protective coatingthat doesn't really change the image to usand if humans looking at it,but somehow this AI modelshave really hard timeor it's impossible for themto actually realisticallychange the image.So that's the goal.So now you mentioned this thingcalled adversarial examples.So can you give us a bit of a hintof what adversarial examples areand how they kind of fit the picture here?- Yeah, totally.So adversarial examples islike an interesting phenomenonin machine learning.It's these small perturbationswhich we can add to images,which we feed to ourmachine learning models,which will disrupt,totally disrupt the machinelearning model behavior.So for example, imagine youhave the image of a dog, okay?And imagine there is a classifier,a machine learning modelthat recognizes this dog withvery, very high accuracy.It's amazing at recognizing this dog.An adversary example isbasically a small modificationto this image of a dog,which keeps it for us humansas the image of a dog.So for us humans, we don'tsee anything that changed.It's small changes in the pixelssuch that it actually leads to an imagewhich will be recognized bythis machine learning modelas totally different thing.It'll think that this image of a dog,which for us humans lookslike a dog is actually a cat.So the model will justlike totally misunderstandwhat the image is.- So you're saying, okay,so a real example is essentially this wayin which you can take an existing imageand change it in a way that'simperceptible to humansby modifying the pixels a little bit.So as the any machine learning modelthat kind of works withthis image will, you know,will kind of think something different.You can essentially like steerthe behavior of this modelor disrupt the behavior ofthis model on this image.Okay, how does it fit to?- And it has been andof course you can seehow this is like a dangerous thing.Like it has always been observed thatadversary examples are bad things, right?Like we have a machine learning modeland people now can actually attack it,what we call it adversarial attack.We can attack this model bychanging small, you know,by doing small imperceptible changesto the inputs of this modeland can break our model.So it has always been to observe thatthese things are bad things.What we did in our projectin PhotoGuard is actually,you know, repurposing these, you know,this phenomenon to actuallymake it something good.This phenomenon adversarialexamples is very good at likebreaking machine learning models, right?So why don't we use itto actually break these,you know, AI generative modelsif they try to manipulate our images.So that's where theidea came out from like,we know that this tool exists,this thing actually this phenomenon sorry,exists in machine learning.Let's use it in the right way.Let's use it to actually, youknow, reinforce what we want.Let's use it to actuallyprotect our images.So let's add this, you know,small adversarial imperceptibleperturbations to our imagesbefore uploading them online,such that if they actually get pastby some malicious actorto some diffusion model,so that they edit them,they will not work.It'll basically these perturbations,these small immunizations that we add,would disrupt this AI model.So this is the overall idea basically.- I see, so essentiallywe are kind of usingthose perturbations as alike this protective layerbecause it'll essentiallylike yes to us humans,this will still be the same picture,but any model that tries to act on itwill kind of get confusedand not work well.- Exactly.- Great, so here we havethis idea, this solution.So is it ready to go?Like can we just start using it right now?- So what we did is basically a prototype.As a research lab, you know,we do prototypes anddemonstrate that this thingis actually, you know, feasible,it actually disrupts, youknow, existing models.And we actually got it to work.We actually got it tolike disrupt, you know,a few open source models, youknow, stable diffusion models.But for this to actuallymake be a practical solution,there definitely islike a couple of thingsthat we need to take into account.And before of course, it's atool that everyone can use.So the first thing isthat it's very importantto actually make theseimmunizations robust to, you know,tampering attempts by malicious actors.If I immunize my image and put it online,someone might actuallyattempt to remove this,reverse engineer this immunizationby doing some tricks, youknow, like cropping the image,maybe like adding something to it,maybe filtering it, maybe like,you know, taking a screenshot of it.So we have to build these,you know immunizationsto be robust to such phenomenons.So this is the firstthing that has to happen.And the second thing is thatonce we immunize our imagesand put them online,we have to guarantee in some waythat they will work forfuture generation models.Like they might actuallywork for current models,but we have to have someway to make sure thatif there's a new model thatwill come in the future,these immunizations will work for it.So we have to like makethem future proof basically,or forward compatible with future models.And for this to happen, it'svery hard to do this thingas a user personally,it has to come from the actualdevelopers of these models,because they are the ones who knowwhat the future models will look like.They can guarantee thatthese immunizationswill actually also like immunize imagesagainst future modelsthat they will build.So these are like two of the core thingsthat in my opinion has to happenbefore this is a practical tool.- Okay, so let's go the one by one.So for this question of likebeing resistant to tamperingafter the images is put online,like what can we do?Like how would you go about, you know,like making them resistantto this tampering?- Yeah, it seems like a tough problem,but luckily we have ahuge amount of literatureon exactly solving this problem,but like in a different context.So there's like few years of workon exactly making adversarial examples,robust to such tampering effects.So generating robustadversarial perturbations,we can utilize all theresearch that happens thereto exactly apply it to this problemand this will actually like work,like it's basically the same phenomenonin different contexts.So we are lucky to have likethis huge amount of workthat previously was like,oh, to make that tax robust,but now it translates naturallyto making the defenses robust basically.- I see, so that's very like, interesting.So people were working hardto make these attacks even more nasty.- Exactly.- But we can use themto make our immunizationto be much more nastyin a sense that like,you know, essentially much,much harder to a counteract.- Exactly.- That's great.So now let's talk about this other issue.So we're saying, okay, you know,one thing that's important tounderstand about PhotoGuardis that it's kind of,it has to know what kindof models we are trying to,you know, we are tryingto protect the, you know,like the image against,and of course we might do itfor all the existing models,but you know there will be new modelsdeveloped in the futureand there's no guarantee thatwhatever we do at this pointwill also work against them.But what you're saying, saying oh,that sounds like a hopeless propositionand saying, okay, maybenow our image is protected,but in two years it mightnot, what you're saying,okay, so one way to approachthis question saying,oh, actually any legitimatedeveloper of this model,probably, you know, is on boardwith having this kind ofprotections work because,you know, that's definitelynot against the business,it actually helps them, you know,counteract some of the problems there.So you could actually imaginethere would be some cooperation of themwhen they say to actually make surewhen they design the modelthat the new models inreleased in the futureare also like, you know,essentially this is funnythat they make sure they are susceptibleto these perturbations,which means that this kind of protectionswill continue to work.- Exactly, exactly.Like, it's funny here that like nowwe have the incentive to make our modelsmore susceptible to these, you know,attacks or adversarial perturbations,which before we didn't wantbecause we want to make ourmodels robust to these things.But like in this particular,you know, scenario or situation,these companies have theincentive to make their models,you know, like, breakableby these immunizations.Like they have the incentive to do thatbecause if people want to usethese models in a good wayto manipulate these image in a good way,they must be able to,but if they want to protect their images,they also have to be able to do that.And this is like how these companiescan actually like get in thereand like provide these,you know, APIs for example,to immunize our images againsttheir models basically.- Okay, so I guess thefuture would be that maybeit's all happening automatically,each time you upload yourimage from your iPhone,it just happens in the backgroundand as usual don't haveto think about that.So that's great.That's a great visionthat we might go towards,but kind of, I just wanted togo back to one of the conceptsthat you mentioned in the passing recentlybecause it's kind ofcomplementary is the watermarking.So what is watermarkingand in what way it can helpor not help the scenariosand kind of how does it differfrom the type of functionalitythat PhotoGuard wants to provide?- Yeah, very good question.So like watermarking,we're hearing aboutwatermarking everywherein the context of, youknow, language models.We wanna want watermark, you know,like the output of these modelsso that we can track them.We are hearing about italso like in the contextof these diffusion models.So watermarking basically is a waywhere you actually add some, you know,like something to the imagethat's generated by these modelsso that you can detect thatit's actually a fake image,not an actual or real image.And it definitely is a legit wayto like go towards, you know,like keeping track of likewhat images we are generatingbecause it's important thatthe images that we generateor other people generate are tracked,whether it's they're fake ornot fake for many reasons,like for, you know, likemisinformation tracking,for many, many different reasons.And this is a very, very good,you know, direction of course,it differs from PhotoGuardin the fact that like,it's comes like as in, youknow, a post, you know,like it's not a preventive solutionof the edit from happeningat the first time.It's like a post hoc maybelike that's what you call it.Like it's a post hoc solution,like it comes afterwards.We generate a fake image andnow we can track it if it's,we can detect it if it's fake or not.The problem that might happen,purely detecting fake imagescan sometimes be not enough.For example, if there is the imageof some explosion somewhere,even before it's flagged,whether it's like,you know, fake or not, theeffect might happen already.Like stock markets might reactlike before even people double checkor you know, fact check whetherit's, you know, fake or not.Imagine there's like anedited photo of a childin some school, which makes this childlook embarrassed, for example.The effect has happenedalready before, you know,even after, you know, theschool announces that,oh, it turns out this is a fake photo.So there's like preventingthese edits from happeningat the first place was, you know,one of our main goalsas well in this tool.It's like, yeah, it's a preventive measurecompared to like, you know,detective measure basically.And both of them arecomplimentary, of course.Like if something is not prevented,hopefully we can detect it basically.- Sounds great, so essentiallyyou are saying that like,it's like actually you could use both,like they're complimentary.The problem with watermarkingis that essentially yes,you can check that this photo is fake,assuming you can do thewatermarking robustly enough.- Yeah.- But like the photo is already there.- Yes.- Where we are trying tonot let the photo happen to begin with.- Exactly.- Okay, so this is kindof all very interestingand we are very kind of happythat we can provide this ideasinto the space, but like, you know,as you look at thelandscape moving forward,like, you know, are youoptimistic at where we are going?Are you worried about where we are going?What is your feeling here?- AI as a field is moving very, very fast.And like a year ago or two years ago,we didn't imagine that like the progresswill happen this fast,whether it was with theseamazing generative modelsor even with like ChatGPT and like,things are moving very, very fast.So it's hard to predictwhat's coming next,but even without anything new coming next,we have already like very serious risksthat accompany our AI tools right now.So I think a major challengeis actually, you know,dealing with the current technology as is,while thinking that likethis is just improvingand improving in the future more and more.For example, one thingthat I worry really aboutis like the open sourcenature of these models.Open source is an amazing thing,of course as a researcher,as a PhD student, I love it.Like, you know, I can likeplay with these models,I can have access to everything,but we can imagine likeif these very powerfulgenerative models are accessible by anyoneto even train them veryquickly on like small machineson one GPU for example,then this problem is not solvable anymore.Like if we reach that situation,no one can solve this problem.Like, that's it.Anyone in us and on his laptopcan train these amazing generative modelsin a way that they, you know,fool whatever, you know,immunization layers we haveand that's it, game over.Like I don't think there isa solution there anymore.So this is one of the thingsthat is very, very riskyin my opinion, for example.- I see, so, so just to actually like,so that's an important pointthat you make saying likekind of implicitly inactually both watermarkingand the you know, andthe PhotoGuard, you know,relies on the fact thatthere are developersthat are kind of like legitimateand they are willingto do the right thing.- Exactly.- Implement the watermarking,implement PhotoGuard.- Yeah.- But now if everyonecan train their own modelin the basement, then youknow, that's up to them.If they are not legitimate,they will just do all these things.You can have some law thatcompels them to do it,but in the end the enforcement of thatmay be much, much harder.- Much harder, yeah.- So, okay, so it sounds likethere's a lot of excitingstuff moving forward,there's a lot of danger.I think our society has tothink really, really hardabout like, you know, what's to come.And as usual, we mightnot be able to predictall the implications what's to come.I think, you know, the policyhas to catch up with this as well.This is no longer just somethingthat only engineer can solve.But yeah, so I guess we will seewhat the next couple of years will bring.- Yeah, very exciting.I like, I'm super excitedabout what's coming next,like, as an AI researcher,like this is all what I like to do.Like, I just like to likedevelop these models,make them as powerful as possible.Like I love doing that,but at the same time I also likeworry about like their implications.So I work a lot on like, you know,we as a team work a lot like on, you know,making them robust, makingthem, you know, trustworthy,making them, you know,deployable in a confident way,basically like making us likeconfident when we deploy them,figuring out like all of these questions,it's super exciting but alsosuper, super challengingbecause like the advancements in AIis moving really, really fast.- Yeah, well looking forwardto continue to work with you on just that.- Yeah.- Thank you for tuning into the discussion about PhotoGuard.I hope you found it interesting.- Yeah, we're happy that you'relistening to our discussionand if you have any questions,of course, reach out.Thank you so much for listening.\n"