In addition to the possible business threat, forcing OpenAI to identify its use of copyrighted data would expose the company to potential lawsuits. Generative AI systems like ChatGPT and DALL-E are trained using large amounts of data scraped from the web, much of it copyright protected. When companies disclose these data sources it leaves them open to legal challenges. OpenAI rival Stability AI, for example, is currently being sued by stock image maker Getty Images for using its copyrighted data to train its AI image generator.
Aaaaaand there it is. They don’t want to admit how much copyrighted materials they’ve been using.
The EU’s stance is concerning. Their coming laws would benefit unlawful AI devs backed by dictatorships. (Edit: They’ll do whatever they want to research and build more powerful AIs while devs in EU struggle due to heavy restrictions.) Currently, big techs are still learning how to build strong AIs, and giving dictatorships huge advantage like this is dangerous.
I think it’s a basic requirement that the data upon which a large language model is trained be publicly disclosed. It’s the same as the requirement of writing the ingredients in packaged food. Or in knowing where your lawyer got their degree from. You want to know where what you’re using is coming from.
I disagree with the “they’re violating copyright by training on our stuff” argument, but I’ve turned against generative AI because now automation is taking art from us, and we’re still slaving away at work, when automation was supposed to free up time for us to pursue art.
I’m with OpenAI here. Screw copyright and greedy publishers.
You wouldn’t be saying that if it was your content that was being ripped off
As if OpenAI was run by people not caring about copyright.
“Ripped off”? That isn’t how LLMs work.
if you read a copyrighted material without paying and then forgot most of it a month later with vague recollection of what you’ve read the fact is you still accessed and used the copyrighted material without paying.
Now let’s go a step further, you write something that is inspired by that copyrighted material and what you wrote become successful to some degree with eyes on it, but you refuse to admit that’s where you got the idea from because you only have a vague recollection. The fact is you got the idea from the copyrighted material.
FWIW that’s pretty much exactly how all creative works are created. Especially music.
Except the illegally obtaining the copyrighted material part, which is the main point. And definitely not on this scale.
Except that nobody has a superhuman ability to create endless amounts of content almost instantly based on said work.
People throw this “artists/writers use inspiration to create X” argument all the time and it just totally ignores the fact that we’re not talking about some person spending 10s/100s/1000s of hours of their time to copy someone’s working style.
It’s a piece of software churning it out in seconds.
If I would create a very slow AI that takes 10 or 100 hours for each response, would that make it any better in your opinion? I do not think calculation speed of a software is a good basis for legislation.
If analyzing a piece of art and replicating parts of it without permission is illegal, then it should be illegal regardless of the tools used. However, that would make every single piece of art illegal, so it’s not an option. If we make only the digital tools illegal then the question still remains where to draw the line. How much inefficiency is required for a tool to still be considered legal?
Is Adobe Photoshop generative auto-fill legal?
Is translating with deepl.com or the modern Google Translate equivalent legal?
Are voice activated commands on your mobile phone legal (Cortana, Siri, Google)?All of these tools were trained in similar ways. All of these take away jobs (read: make work/life more efficient).
It’s hard to draw a line and I don’t have any solution to offer.
That’s still not how LLMs work. I can’t believe everybody who is upset with them doesn’t understand this.
The LLM has no idea what it’s reading. None. It’s just doing a word association game, but at a scale we can’t comprehend. It knows what arrangement of words go together, but it’s not reproducing anything with any actual intent. To get it to actually output anything that actually resembles a single piece of material it was trained against would require incredibly specific prompts to get there, and at that point it’s not really the LLM’s making anymore.
There’s plenty of reasons to be against AI. Such as the massive amounts of data scraping that happens to train models, the possible privacy invasions that come with that, academic cheating, etc. But to be mad at AI for copyright infringement only shows a lack of understanding of what these systems actually do.
The training process of LLMs is to copy the source material word for word. It’s instructed to plagiarize during the training process. The copyrighted material are possibly in one way or another embedded into the model itself.
In machine learning, there’s always this concern whether the model is actually learning patterns, or if it’s just memorizing the training data. Same applies to LLMs.
Can LLMs recite entire pieces of work? Who knows?
Does it count as copyright infringement if it does so? Possibly.
The training process of LLMs is to copy the source material word for word. It’s instructed to plagiarize during the training process. The copyrighted material are possibly in one way or another embedded into the model itself.
No it isn’t. That;s not how neural networks work, like at all
In machine learning, there’s always this concern whether the model is actually learning patterns, or if it’s just memorizing the training data. Same applies to LLMs.
It’s learning patterns. It’s not memorising training data. Again, not how the system works at all
Can LLMs recite entire pieces of work? Who knows?
No. No they can’t.
Does it count as copyright infringement if it does so? Possibly.
That’d be one for the lawyers were it to ever come up, but it won’t
Here’s a basic description of how (a part of) LLMs work: https://huggingface.co/learn/nlp-course/chapter1/6
LLMs are generating texts word for word (or token by token if you’re pedantic). This is why ChatGPT is slowly generating the response word by word instead of giving you the entire response at once.
Same applies during the training phase. It gets a piece of text and the word it’s supposed to predict. Then it’s tuned to improve its chances to predict the right word based on the text it’s given.
Ideally it’s supposed to make predictions by learning the patterns of the language. This is not always the case. Sometimes it can just memorize the answer instead of learning why (just like how a child can memorize the multiplication table without understanding multiplication). This is formally known as overfitting, which is a machine learning 101 concept.
There are ways to mitigate overfitting, but there’s no silver bullet solution. Sometimes it cannot help to memorize the training data.
When GitHub Copilot was new people quickly figured out it could generate the fast inverse square root implementation from Quake. Word for word. Including the “what the fuck” comment. It had memorized it completely.
I’m not sure how much OpenAI has done to mitigate this issue. But it’s a thing that can happen. It’s not imaginary.
That’s, uh, exactly how they work? They need large amounts of training data, and that data isn’t being generated in house.
It’s being stolen, scraped from the internet.
If it was publicly available on the internet, then it wasn’t stolen. OpenAI hasn’t been hacking into restricted content that isn’t meant for public consumption. You’re allowed to download anything you see online (technically, if you’re seeing it, you’ve already downloaded it). And you’re allowed to study anything you see online. Even for personal use. Even for profit. Taking inspiration from something isn’t a crime. That’s allowed. If it wasn’t, the internet wouldn’t function at a fundamental level.
I don’t think you understand how copyright works. Something appearing on the internet doesn’t give you automatic full commercial rights to it.
An AI has just as much right to web scrape as you do. It’s not a violation of copyright to do so.
It’s not an AI webscraping. It’s a commercial company deciding to do a mass ingest.
Your first comment and it is to support OpenAI.
edit:
Haaaa, OpenAI, this famous hippies led, non-profit firm.
2015–2018: Non-profit beginnings
2019: Transition from non-profit
Funded by Musk and Amazon. The friends of humanity.
Also:
In March, Open AI co-founder Ilya Sutskever told The Verge that the company had been wrong to disclose so much in the past, and that keeping information like training methods and data sources secret was necessary to stop its work being copied by rivals.
Yeah, he closed the source code because he was afraid he would get copied by other people.
With replies like this, it’s no wonder he was hesitant to post in the first place.
There’s no need for the hostility and finger pointing.
he was hesitant to post in the first place.
Was he hesitant? how do you know that?
He created his account 2 weeks ago and didn’t post until tonight.
Read the whole thing. The reason OpenAI is opposing the law is not necessarily copyright infringement.
One provision in the current draft requires creators of foundation models to disclose details about their system’s design (including “computing power required, training time, and other relevant information related to the size and power of the model”)
This is the more likely problem.
Given their name is “OpenAI” and they were founded on the idea of being transparent with those exact things, I’m less impressed that that’s what they’re upset about. The keep saying they’re “protecting” us by not releasing us, which just isn’t true. They’re protecting their profits and valuation.
Maybe, but I believe that the AI model of this level should not be shared with dictators like the CCP, at least for now.
If I do a book report based on a book that I picked up from the library, am I violating copyright? If I write a movie review for a newspaper that tells the plot of the film, am I violating copyright? Now, if the information that they have used is locked behind paywalls and obtained illegally, then sure, fire ze missiles, but if it is readily accessible and not being reprinted wholesale by the AI, then it doesn’t seem that different from any of the other millions of ways we use data in every day life. Just because a machine learned it instead of a human, I don’t believe that it makes it inherently wrong. I am very open to discussion on this, and if anyone has a counter-argument, I’d love to hear it, because this is a new field of technology that we should all talk about and learn to understand better.
Edit: I asked GPT-4 what it thought about this, and here is what it said:
As an AI developed by OpenAI, I don’t access any copyrighted databases, subscription sites, confidential information, or proprietary databases for my learning or response generation. I was trained on a diverse range of internet text, which includes publicly available data in multiple languages and formats. The training also involves learning from large-scale datasets that contain parts of many kinds of texts (books, websites, scientific articles, etc.). However, I don’t know specifics about which documents were in my training set or have access to any specific documents or sources.
Your question seems to pertain to how copyright law applies to AI. This is a complex and evolving area of law and policy, largely because AI systems like me do not “read” or “use” material in the same way a human would. A human might read a book and then consciously recall information from it when writing or speaking. In contrast, AI models do not have consciousness or awareness, they analyze statistical patterns in the data they were trained on and use these patterns to generate responses.
When a human uses copyrighted material to create something new, the law often considers this to be a derivative work. It is typically necessary to obtain permission from the copyright holder to create derivative works. However, AI doesn’t use or recall copyrighted material directly; instead, it learns patterns from a huge amount of text data to predict or generate text based on a given prompt.
Dissemination of copyrighted material, such as through forum posts or Q&A platforms, without proper licensing or within the bounds of fair use, can be a violation of copyright law. AI doesn’t ‘disseminate’ copyrighted material; it generates responses based on patterns it has learned.
That said, the legal interpretation of how copyright law applies to machine learning and AI is still evolving, and there can be a lot of complexities and nuances. Therefore, the information here should not be taken as legal advice.
As of my last update in September 2021, this was the case. However, I recommend checking the most recent legal standpoints and opinions, as this area is still evolving rapidly.
I am very open to discussion on this, and if anyone has a counter-argument, I’d love to hear it, because this is a new field of technology that we should all talk about and learn to understand better.
That’s very cool and all but while we have this debate there are artists getting ripped off.
You aren’t having a debate. You’re blindly claiming that artists are getting ripped off, because maybe they are a bit, or maybe they’re latching onto any reason that lets them still have professional careers in 30 years.
I’m not making blind claims. And I won’t point you to the sources either. I’m not making any homework for anyone today. Dig the subject and post us some information if you are really into the debate thing.
If you can provide some sources with real data from people that have proven a loss of income due to getting “ripped off” by AI, I’d love to look over it. Until then, it’s a witch hunt.
I can provide you with reddit posts from artists who are replaced by AI.
Would you like it served with a cup of tea and some sandwiches?
If you have some that have actual proof in them, sure. That’s exactly what I’m looking for. However, if it amounts to nothing more than hearsay, then no, I don’t think I want them.
Had you spent a minimum of time digging the subject you would know exactly what I’m talking about.
I’m not making your sandwich for you, you will have to make your sandwich yourself.
Burden of proof being what it is, I’ll leave the sandwich making to those with the meat and bread.
I won’t take any burden for your majesty. Do your homework.
This is one of those cases where copywrite law works opposite as intended; in that it should drive innovation. Here we have an example of innovation, but copywrite holders want to (justifiably) shut it down.
I think this is actually a case where copyright works correctly. It is protecting individuals of getting their work, they provided for free in many cases, ‘stolen’ by a more powerful party to make money from it without paying the creators of their work.
