Google rolled out AI overviews across the United States this month, exposing its flagship product to the hallucinations of large language models.
You must log in or register to comment.
Can we swap out the word “hallucinations” for the word “bullshit”?
I think all AI/LLM stuf should be prefaced as “someone down the pub said…”
So, “someone down the pub said you can eat rocks” or, “someone down the pub said you should put glue on your pizza”.
Hallucinations are cool, shit like this is worthless.
I don’t even think hallucinations is the right word for this. It’s got a source. It is giving you information from that source. The problem is it’s treating the words at that source as completely factual despite the fact that they are not. Hallucinations from what I’ve read actually is more like when it queries it’s data set, can’t find an answer, and then generates nonsense in order to provide an answer it doesn’t have. Don’t think that’s the same thing.
I don’t even think it’s correct to say it’s querying anything, in the sense of a database. An LLM predicts the next token with no regard for the truth (there’s no sense of factual truth during training to penalize it, since that’s a very hard thing to measure).
Keep in mind that the same characteristic that allows it to learn the language also allows it to sort of come up with facts, it’s just a statistical distribution based on the whole context, which needs a bit randomness so it can be “creative.” So the ability to come up with facts isn’t something LLMs were designed to do, it’s just something we noticed that happens as it learns the language.
So it learned from a specific dataset, but the measure of whether it will learn any information depends on how well represented it is in that dataset. Information that appears repeatedly in the web is quite easy for it to answer as it was reinforced during training. Information that doesn’t show up much is just not gonna be learned consistently.[1]
I understand the gist but I don’t mean that it’s actively like looking up facts. I mean that it is using bad information to give a result (as in the information it was trained on says 1+1 =5 and so it is giving that result because that’s what the training data had as a result. The hallucinations as they are called by the people studying them aren’t that. They are when the training data doesn’t have an answer for 1+1 so then the LLM can’t do math to say that the next likely word is 2. So it doesn’t have a result at all but it is programmed to give a result so it gives nonsense.
Yeah, I think the problem is really that language is ambiguous and the LLMs can get confused about certain features of it.
For example, I often ask different models when was the Go programming language created just to compare them. Some say 2007 most of the time and some say 2009 — which isn’t all that wrong, as 2009 is when it was officially announced.
This gives me a hint that LLMs can mix up things that are “close enough” to the concept we’re looking for.
No, hallucination is a really good term. It can be super confident and seemingly correct but still completely made up.
It is, but it isnt applicable in at least the glue-pizza situation as the probable source comment has been found on reddit.
A better use of the term might be how when you try to get Bing’s image creator to make “Battletech” art, you just mostly get really obvious Warhammer 40k Space Marines and occasionally Iron Maiden album art.
It’s a really bad term because it’s usually associated with a mind, and LLMs are nothing of the sort.
Anthropomorphication is hard to avoid in AI.
Many worthy things are difficult.
But is anthropomorphism of AI particularly worrying?
It is when the people tends to give more credence to entities that appear sentient and to have agency.
The head of Google *Search right now is the same guy that was head of yahoo search when it was dying. To put all of this in perspective.
The head of Google Search
FTFY
TBH I hate the term “hallucination” in this context. It’s just more BS anthropomorphizing. More marketing for “AI” (also BS). Can’t we just call it like garbage or GIGO or something more accurate? This is nothing new. I know that scientific accuracy is anathema to AI marketing but just saying…
We don’t choose. It’s decided to be the term for this. Computer bugs aren’t bugs. Etc etc. It’s just what the scientists called it
Does anyone have a realistic idea of how this happened? I get Google has been fallen off for awhile but they’re still a multi billion dollar company.
I’m probably late, but in this case this is the combinations of 2 things.
- The usual capitalistic incentives ruined yet another company. There was a recent article about how Google pushed out the people who builded and maintaned search on favor of MBA growth focused assholes. Like they put the guy that was Yahoo’s CEO while Yahoo search was crumbling, in charge of Google search to get him to increase the amount of searches they serve, and ads obviously. People keep suggesting to use DDG, or Kagi, or some other comercial product. And for now, we must because Google is basically useless right now. But just give time to the other companies to fall in the same trap hahaha.
- LLMs are not smart, not even close. They are just a parlor trick that has non technical people fooled. There is a lot of evidence to me, but to me the most obvious one is that they don’t have anything resembling human short term memory. Like the way they make them look like they are having a conversation is by providing the entire conversation up to that point, including their own previous responses lol, as input/context so the bot autocompletes the conversation. It literally can’t remember a single word of what you said on it’s own. But sureee, they are just like humans lol.
So what we have here is obvious, we have a company trying to grow like cancer by any means necessary. And now they have a technology that allows them to create enough smoke and mirrors to fool non technical people. Sadly, as part of this they are also destroying the last places of the internet not fully controlled by corporations. Let’s hope lemmy survives, but it’s just a matter of time before they flood this place too.
including their own previous responses lol, as input/context so the bot autocompletes the conversation. It literally can’t remember a single word of what you said on it’s own.
Chatgpt has had memory from previous conversations for about a month now and it’s context window is no longer fixed. Additionally it has the ability to assign sentences to memory on its own. So if it “thinks” what you said is important it saves it.
Can you point me to the paper/article/whatever where this is being discussed please? I’m actually interested on learning about it. Even if I don’t like the way they are using the technology, I’m still a programmer at hearth and would love to read about this.
To the point of the conversation, honestly man that was just an example of the many problems I see with this. But you have to understand that people like you keep asking us for proof that LLMs are not smart. But come on man, you are the ones claiming you solved the hard problem of mind, on the first try no less hahaha. You are the ones with the burden of proof here and you have provided nothing of the sort. Do better people or stop trying to confuse us with retoric.
I mean it’s just the release notes. Go to their website. I have used the memory feature myself on the app so know it’s working and as for the context window it can actually tell you what it is for each session.
But you have to understand that people like you keep asking us for proof that LLMs are not smart.
Where? Where have I asked that? Don’t strawman me, I am not your punching bag and won’t defend something I didn’t say. You can “come on man” all you want but it won’t change my answer. I have made zero claims if this thing is smart or asked anyone to weight in on the issue either way.
I pointed out two features it has now, which I don’t think anyone can dispute that it does have those features. It has a larger context window and memory that it can update. That is all I said, a very small claim that you can prove for yourself in under five minutes by going to their website.
Oh, you are talking about this https://help.openai.com/en/articles/8590148-memory-faq hahahaha. I’m sorry man, but you are a moron or arguing in bad faith. That’s yet another feature where they inject even more shit in the context/input to make it feel like the thing has memory. That’s literally yet another example of what I was pointing out, so thanks for confirming my suspicions. Seriously dude, do better if you really want to have a conversation. Your response made me waste my time, and on top of that you insult me hahaha.
It’s just a fucking Chinese Room
And humans aren’t?
