If scientists made AI, then it wouldn’t be an issue for AI to say “I don’t know”.
But capitalists are making it, and the last thing you want is it to tell an investor “I don’t know”. So you tell it to make up bullshit instead, and hope the investor believes it.
It’s a terrible fucking way to go about things, but this is America…
It’s got nothing to do with capitalism. It’s fundamentally a matter of people using it for things it’s not actually good at, because ultimately it’s just statistics. The words generated are based on a probability distribution derived from its (huge) training dataset. It has no understanding or knowledge. It’s mimicry.
It’s why it’s incredibly stupid to try using it for the things people are trying to use it for, like as a source of information. It’s a model of language, yet people act like it has actual insight or understanding.
You do not understand how these things actually work. I mean, fair enough, most people don’t. But it’s a bit foolhardy to propose changes to how something works without understanding how it works now.
There is no “database”. That’s a fundamental misunderstanding of the technology. It is entirely impossible to query a model to determine if something is “present” or not (the question doesn’t even make sense in that context).
A model is, to greatly simplify things, a function (like in math) that will compute a response based on the input given. What this computation does is entirely opaque (including to the creators). It’s what we we call a “black box”. In order to create said function, we start from a completely random mapping of inputs to outputs (we’ll call them weights from now on) as well as training data, iteratively feed training data to this function and measure how close its output is to what we expect, adjusting the weights (which are just numbers) based on how close it is. This is a gross simplification of the complexity involved (and doesn’t even touch on the structure of the model’s network itself), but it should give you a good idea.
It’s applied statistics: we’re effectively creating a probability distribution over natural language itself, where we predict the next word based on how frequently we’ve seen words in a particular arrangement. This is old technology (dates back to the 90s) that has hit the mainstream due to increases in computing power (training models is very computationally expensive) and massive increases in the size of dataset used in training.
Source: senior software engineer with a computer science degree and multiple graduate-level courses on natural language processing and deep learning
Btw, I have serious issues with both capitalism itself and machine learning as it is applied by corporations, so don’t take what I’m saying to mean that I’m in any way an apologist for them. But it’s important to direct our criticisms of the system as precisely as possible.
Uh, I understand the sentiment, but the model doesn’t know anything. And it’s legit really hard to differentiate between factual things and random bullshit it made up.
Was gonna say, the AI doesn’t make up or admit bullshit, its just a very advanced a
prediction algorithm. It responds with what the combination of words that is most likely the expected answer.
Wether that is accurate or not is part of training it but you’ll never get 100% accuracy to any query
It’s not actually deciding anything, the AI thinking is marketing fluff really. But yes that’s called confidence rating and it does. But at the scale of something like chatgpt that uses a snapshot of the entire internet and is non mutable there’s no way to train it for every possible question. If you ask about a topic 99% of the internet gets wrong it’ll respond the wrong thing with 99% confidence
If it has been trained using questionable sources, or if it’s training data includes sarcastic responses (without understanding that context), it isn’t hard to imagine how confidently wrong some of the responses could be.
Yeah, no one can make it say “I don’t know” because it is not really AI. Business bros decided to call it that and everyone smiled and nodded. LLMs are 1 small component (maybe) of AI. Maybe 1/80th of a true AI or AGI.
Honestly the most impressive part of LLMs is the tokenizer that breaks down the request, not the predictive text button masher that comes up with the response.
Honestly the most impressive part of LLMs is the tokenizer that breaks down the request, not the predictive text button masher that comes up with the response.
Yes, exactly! It’s ability to parse the input is incredible. It’s the thing that has that “wow” factor, and it feels downright magical.
Unfortunately, that also makes people intuitively trust its output.
It “knows” as in it has access to the information and the ability to provide the right info for the right context.
Any part of that process the AI can just “bullshit” and fills in the gaps with random stuff.
Which is what you want when it’s “learning”. You want it to try so it’s attempt can be rated, and the relevant info added to its “knowledge”.
But when consumers are using it, you want it to say “I can’t answer that”. But consumers are usually stupid and will buy/use the one that says “I can’t answer that” the least.
And it’s legit really hard to differentiate between factual things and random bullshit it made up.
Which is why AI should tell end users “I don’t know” more often.
Do you not understand how “answer unavailable” is a better answer than taking a small percent of strips of paper at random and filling in the rest with words that sound relevant?
taking a small percent of strips of paper at random and filling in the rest with words that sound relevant?
It’s like a mad libs
Right. They’re text generators. That’s the technology. It can’t do what you’re demanding because that’s not how it works. LLMs aren’t magic answer machines. They don’t know when to say “answer not available”. They don’t know what they’re being asked. They don’t know anything.
You know that answer unavailable is better because you have real intelligence, an LLM is just some mathematical functions so it can’t do that. If it could it would be getting much closer to actually being AI.
This has nothing to do with scientists vs capitalists and everything with the fact that this is not actually “AI”. Someone called it T9 (word prediction) on steroids and I find that much more fitting with how those LLMs work. It just mimics the way humans talk, but it doesn’t actually converse intelligently or actually understands context - it just looks like it does, but only if you take it at face value and don’t look deeper into it.
It’s just short for automatic transmission, opposed to manual transmission. I think Americans call manual cars sticks though. But they’re not sticks, because sticks are wood and cars are almost always metal. Not metal like the music though.
Edit - thinking on it you could play metal through the car stereo though.
I know the difference between an automatic & manual car & transmission. The analogy just doesn’t make sense, because when you say “automatic / manual car” you’re still referring to something within the car, the transmission system - you’re not actually calling the car to be “automated” or whatever. Calling LLMs “AI” however is nothing but a misnomer and that analogy simply does not compare at all.
It is made by scientists. And we don’t know how to make the model determine whether or not it knows something. So far, we only have tools that tell us that something probably wasn’t in the training set (e.g. using variance across models in a mixture of experts setup), but that doesn’t tell us anything about how correct it is.
What’s your view of the fizbang Raspberry blasters?
Gpt ‘I’m not familiar with “fizbang Raspberry blasters.” Could you provide more details or clarify what they are?’
It’s a drink making machine from china
Gpt ‘I don’t have any specific information on the “fizbang Raspberry blasters” drink making machine. If it’s a new or niche product, details might be limited online.’
So, in this instance is didn’t hallucinate, i tried a few more made up things and it’s consistent in saying it doesn’t know of these.
Chatgpt and gpt4 are two different things. Gpt4 is like the engine and chatgpt is like a car. In early version they were pretty much the same thing, but nowadays they have implemented so much in chatgpt.
On top of that chatgpt4 is constantly trained for these scenarios, it is no longer a base model.
Oh ok thanks i thought this thread was about AI LLMs in general.
Weird i was downvoted for demonstrating the very thing that apparently (according to these very learned comments) AI can’t do, actually doing it well. Seems like irrational bubble hate to me, common on reddit but getting more so on Lemmy it seems. “that guys asking topic based questions that make our comments look poorly thought out and potentially wrong, burn him”
This is a thing that is true of all LLMs, but it seems like you’re misunderstanding the core issue. It CAN give outputs like that sometimes. What we CAN’T do is force it to give outputs like that ALL the time.
It will answer “I don’t know” if its predictive text model guesses that the most common response to this would be “I don’t know”. To do that, to simplify a little, you could imagine that it reads your question, compares that to all the text in its training data, and tries to find the conversation that looks most like the question you asked, then answers whatever the person in the training data answered. But your exact question wasn’t in its training data, so if you took that mental model, and instead had it compare to 1000 similar looking things in its training model and average them, then it would hopefully do a better job of coming up with something at least close to what you actually asked. Now take it to a million, or a billion.
When we’re asking questions about the real world, we would prefer for it to answer based on knowledge about the real world. But what if it “matches” data from a work of fiction? Or just someone who doesn’t know what they’re talking about? Or true information, but about a different subject?
It doesn’t know anything. It doesn’t understand anything you say. It just looks at patterns that it learned from the training data and tries to guess what words are most likely to be said in that case. In other words, “here’s one case where it didn’t hallucinate” and “it will never hallucinate” are not the same thing at all.
Edit: To clarify, it doesn’t search its training data to answer your question, so asking “was this in the training data” is impossible. By the time you interact with it, the data is long gone. It was just used for training.
Ok very long and detailed response, i was responding to the initial comments that explicitly said if you give ai a made up thing it will definitely hallucinate. Which i demonstrated to be false in (multiple times). I’m not suggesting it doesn’t hallucinate a lot of the time still, but the comments were making out its 100% broken, and it clearly works for many queries very effectively, despite its limited applications. Im just suggesting we don’t throw the baby out with the bathwater.
I think the trouble is, what baby are we throwing out with the bathwater in this case? We can’t prevent LLMs from hallucinating (but we can mitigate it somewhat with carefully constructed prompts). So, use cases where we’re okay with that are fair game, but any use case (or in this case, law?) that would require the LLM never hallucinates aren’t attainable, and to get back earlier, this particular problem has nothing to do with capitalism.
If scientists made AI, then it wouldn’t be an issue for AI to say “I don’t know”.
But capitalists are making it, and the last thing you want is it to tell an investor “I don’t know”. So you tell it to make up bullshit instead, and hope the investor believes it.
It’s a terrible fucking way to go about things, but this is America…
It’s got nothing to do with capitalism. It’s fundamentally a matter of people using it for things it’s not actually good at, because ultimately it’s just statistics. The words generated are based on a probability distribution derived from its (huge) training dataset. It has no understanding or knowledge. It’s mimicry.
It’s why it’s incredibly stupid to try using it for the things people are trying to use it for, like as a source of information. It’s a model of language, yet people act like it has actual insight or understanding.
you’re so close, just why exactly do you think people are using it for these things it’s not meant for?
because every company, every CEO, every VP, is pushing every sector of their companies to adopt AI no matter what.
most actual people understand the limitations you list, but it’s the capitalists at the table that are making AI show up where it’s not wanted
Imagine searching your computer for a PDF named “W2.2026”…
Would you rather the computer tell you it’s not in the database? Or would you prefer a random PDF displayed with the title “W2.2026”?
This isn’t a new problem.
You’re getting hung up on “know” instead “has relevant information in it’s database and can access it”.
But besides all that and the other things you got wrong:
It’s still about capitalism for the reasons I just said
You do not understand how these things actually work. I mean, fair enough, most people don’t. But it’s a bit foolhardy to propose changes to how something works without understanding how it works now.
There is no “database”. That’s a fundamental misunderstanding of the technology. It is entirely impossible to query a model to determine if something is “present” or not (the question doesn’t even make sense in that context).
A model is, to greatly simplify things, a function (like in math) that will compute a response based on the input given. What this computation does is entirely opaque (including to the creators). It’s what we we call a “black box”. In order to create said function, we start from a completely random mapping of inputs to outputs (we’ll call them weights from now on) as well as training data, iteratively feed training data to this function and measure how close its output is to what we expect, adjusting the weights (which are just numbers) based on how close it is. This is a gross simplification of the complexity involved (and doesn’t even touch on the structure of the model’s network itself), but it should give you a good idea.
It’s applied statistics: we’re effectively creating a probability distribution over natural language itself, where we predict the next word based on how frequently we’ve seen words in a particular arrangement. This is old technology (dates back to the 90s) that has hit the mainstream due to increases in computing power (training models is very computationally expensive) and massive increases in the size of dataset used in training.
Source: senior software engineer with a computer science degree and multiple graduate-level courses on natural language processing and deep learning
Btw, I have serious issues with both capitalism itself and machine learning as it is applied by corporations, so don’t take what I’m saying to mean that I’m in any way an apologist for them. But it’s important to direct our criticisms of the system as precisely as possible.
You don’t seem to understand. There is no database.
https://www.merriam-webster.com/dictionary/analogy
https://en.wikipedia.org/wiki/False_equivalence
Uh, I understand the sentiment, but the model doesn’t know anything. And it’s legit really hard to differentiate between factual things and random bullshit it made up.
Was gonna say, the AI doesn’t make up or admit bullshit, its just a very advanced a prediction algorithm. It responds with what the combination of words that is most likely the expected answer.
Wether that is accurate or not is part of training it but you’ll never get 100% accuracy to any query
If it can name what the most likely combination is, couldn’t it also know how likely that combination of words is?
It’s not actually deciding anything, the AI thinking is marketing fluff really. But yes that’s called confidence rating and it does. But at the scale of something like chatgpt that uses a snapshot of the entire internet and is non mutable there’s no way to train it for every possible question. If you ask about a topic 99% of the internet gets wrong it’ll respond the wrong thing with 99% confidence
If it has been trained using questionable sources, or if it’s training data includes sarcastic responses (without understanding that context), it isn’t hard to imagine how confidently wrong some of the responses could be.
Yeah, no one can make it say “I don’t know” because it is not really AI. Business bros decided to call it that and everyone smiled and nodded. LLMs are 1 small component (maybe) of AI. Maybe 1/80th of a true AI or AGI.
Honestly the most impressive part of LLMs is the tokenizer that breaks down the request, not the predictive text button masher that comes up with the response.
Yes, exactly! It’s ability to parse the input is incredible. It’s the thing that has that “wow” factor, and it feels downright magical.
Unfortunately, that also makes people intuitively trust its output.
It “knows” as in it has access to the information and the ability to provide the right info for the right context.
Any part of that process the AI can just “bullshit” and fills in the gaps with random stuff.
Which is what you want when it’s “learning”. You want it to try so it’s attempt can be rated, and the relevant info added to its “knowledge”.
But when consumers are using it, you want it to say “I can’t answer that”. But consumers are usually stupid and will buy/use the one that says “I can’t answer that” the least.
Which is why AI should tell end users “I don’t know” more often.
If you feel this is a simple solution, I strongly suggest you write up exactly how you do this and make yourself a billion dollars.
It doesn’t, though, any more than you have access to the information in a pile of 10 million shredded documents.
Right, in this case that we’re talking about…
Do you not understand how “answer unavailable” is a better answer than taking a small percent of strips of paper at random and filling in the rest with words that sound relevant?
It’s like a mad libs
Right. They’re text generators. That’s the technology. It can’t do what you’re demanding because that’s not how it works. LLMs aren’t magic answer machines. They don’t know when to say “answer not available”. They don’t know what they’re being asked. They don’t know anything.
You know that answer unavailable is better because you have real intelligence, an LLM is just some mathematical functions so it can’t do that. If it could it would be getting much closer to actually being AI.
This has nothing to do with scientists vs capitalists and everything with the fact that this is not actually “AI”. Someone called it T9 (word prediction) on steroids and I find that much more fitting with how those LLMs work. It just mimics the way humans talk, but it doesn’t actually converse intelligently or actually understands context - it just looks like it does, but only if you take it at face value and don’t look deeper into it.
Removed by mod
Removed by mod
It’s just short for automatic transmission, opposed to manual transmission. I think Americans call manual cars sticks though. But they’re not sticks, because sticks are wood and cars are almost always metal. Not metal like the music though.
Edit - thinking on it you could play metal through the car stereo though.
I know the difference between an automatic & manual car & transmission. The analogy just doesn’t make sense, because when you say “automatic / manual car” you’re still referring to something within the car, the transmission system - you’re not actually calling the car to be “automated” or whatever. Calling LLMs “AI” however is nothing but a misnomer and that analogy simply does not compare at all.
Removed by mod
Removed by mod
It is made by scientists. And we don’t know how to make the model determine whether or not it knows something. So far, we only have tools that tell us that something probably wasn’t in the training set (e.g. using variance across models in a mixture of experts setup), but that doesn’t tell us anything about how correct it is.
Just put this into GPT 4.
What’s your view of the fizbang Raspberry blasters?
Gpt ‘I’m not familiar with “fizbang Raspberry blasters.” Could you provide more details or clarify what they are?’
It’s a drink making machine from china
Gpt ‘I don’t have any specific information on the “fizbang Raspberry blasters” drink making machine. If it’s a new or niche product, details might be limited online.’
So, in this instance is didn’t hallucinate, i tried a few more made up things and it’s consistent in saying it doesn’t know of these.
Explanations?
Chatgpt and gpt4 are two different things. Gpt4 is like the engine and chatgpt is like a car. In early version they were pretty much the same thing, but nowadays they have implemented so much in chatgpt.
On top of that chatgpt4 is constantly trained for these scenarios, it is no longer a base model.
Oh ok thanks i thought this thread was about AI LLMs in general.
Weird i was downvoted for demonstrating the very thing that apparently (according to these very learned comments) AI can’t do, actually doing it well. Seems like irrational bubble hate to me, common on reddit but getting more so on Lemmy it seems. “that guys asking topic based questions that make our comments look poorly thought out and potentially wrong, burn him”
This is a thing that is true of all LLMs, but it seems like you’re misunderstanding the core issue. It CAN give outputs like that sometimes. What we CAN’T do is force it to give outputs like that ALL the time.
It will answer “I don’t know” if its predictive text model guesses that the most common response to this would be “I don’t know”. To do that, to simplify a little, you could imagine that it reads your question, compares that to all the text in its training data, and tries to find the conversation that looks most like the question you asked, then answers whatever the person in the training data answered. But your exact question wasn’t in its training data, so if you took that mental model, and instead had it compare to 1000 similar looking things in its training model and average them, then it would hopefully do a better job of coming up with something at least close to what you actually asked. Now take it to a million, or a billion.
When we’re asking questions about the real world, we would prefer for it to answer based on knowledge about the real world. But what if it “matches” data from a work of fiction? Or just someone who doesn’t know what they’re talking about? Or true information, but about a different subject?
It doesn’t know anything. It doesn’t understand anything you say. It just looks at patterns that it learned from the training data and tries to guess what words are most likely to be said in that case. In other words, “here’s one case where it didn’t hallucinate” and “it will never hallucinate” are not the same thing at all.
Edit: To clarify, it doesn’t search its training data to answer your question, so asking “was this in the training data” is impossible. By the time you interact with it, the data is long gone. It was just used for training.
Ok very long and detailed response, i was responding to the initial comments that explicitly said if you give ai a made up thing it will definitely hallucinate. Which i demonstrated to be false in (multiple times). I’m not suggesting it doesn’t hallucinate a lot of the time still, but the comments were making out its 100% broken, and it clearly works for many queries very effectively, despite its limited applications. Im just suggesting we don’t throw the baby out with the bathwater.
I think the trouble is, what baby are we throwing out with the bathwater in this case? We can’t prevent LLMs from hallucinating (but we can mitigate it somewhat with carefully constructed prompts). So, use cases where we’re okay with that are fair game, but any use case (or in this case, law?) that would require the LLM never hallucinates aren’t attainable, and to get back earlier, this particular problem has nothing to do with capitalism.
Yes, i agree
It is made by scientists. The problem is that said scientists are paid by investors mostly, or by grants that come from investors.