Maybe it can. If you find a way to port everything to text by hooking in different models, the LLM might be able to reason about everything you throw at it. Who even defines how AGI should be implemented?
A LLM is basically just an orchestration mechanism. Saying a LLM doesn’t do reasoning is like saying a step function can’t send an email. The step function can’t, but the lambda I’ve attached to it sure as shit can.
ChatGPT isn’t just a model sat somewhere. There are likely hundreds of services working behind the scenes to coerce the LLM into getting the right result. That might be entity resolution, expert mapping, perhaps even techniques that will “reason”.
The first initial point is right, though. This ain’t AGI, not even close. It’s just your standard compositional stuff with a new orchestration mechanism that is better suited for long-form responses - and wild hallucinations…
Source: Working on this right now.
Edit: Imagine downvoting someone that literally works on LLM’s for a living. Lemmy is a joke sometimes…
I think many in the AI space are against the current state of how AI is being pushed, probably just as much as the average tech person.
What is ridiculous is that Lemmy prides itself as both forward-thinking and tech focused, and in reality it is far more close-minded than Reddit or even Twitter. Given how heavily used Mastodon is in tech spheres it makes Lemmy look like an embarrassment to the fediverse…
Yeah, there is every reason to be sceptical of the hype around AI, in particular from the big tech companies. But to a significant part of the Lemmy userbase saying “AI” is like saying “witch” in 17th century Salem. To the point where people who are otherwise very much left wing and anti-corporate will take pro-IP/corporate copyright maximalist stances just becuase that would be bad for AI.
And how does reasoning work exactly in the human body? Isn’t it LLM/LAM working together with hormones?
How do you know that humans aren’t just doing something similar? Your mind tricks you about a lot of things you experience, how can you be sure, your "reasoning” is just sorta LLM in disguise?
Language models are literally incapable of reasoning beyond what is present in the dataset or the prompt. Try giving it a known riddle and change it so it becomes trivial, for example “With a boat, how can a man and a goat get across the river?”, despite it being a one step solution, it’ll still try to shove in the original answer and often enough not even solve it. Best part, if you then ask it to explain its reasoning (not tell it what it did wrong, that’s new information you provide, ask it why it did what it did), it’ll completely shit it self hallucinating more bullshit for the bullshit solution. There’s no evidence at all they have any cognitive capacity.
I even managed to break it once through normal conversation, something happened in my life that was unique enough for the dataset and thus was incomprehensible to the AI. It just wasn’t able to follow the events, no matter how many times I explained.
Maybe the grown up human LLM that keeps learning 24/7 and is evolved in thousands of years to make the learning part as efficient as possible is just a little bit better than those max 5year old baby LLM with brut force learning techniques?
The 5 year old baby LLM can’t learn shit and lacks the ability to understand new information. You’re assuming that we and LLMs “learn” in the same way. Our brains can reason and remember information, detect new patterns and build on them. An LLM is quite literally incapable of learning a brand new pattern, let alone reason and build on it. Until we have an AI that can accept new information without being tolled what is and isn’t important to remember and how to work with that information, we’re not even a single step closer to AGI. Just because LLMs are impressive, doesn’t mean they posses any cognition. The only way AIs “learn” is by countless people constantly telling it what is and isn’t important or even correct. The second you remove that part, it stops working and turns to shit real quick. More “training” time isn’t going to solve the fact that without human input and human defined limits, it can’t do a single thing. AI cannot learn form it self without human input either, there are countless studies that show how it degrades, and it degrades quickly, like literally just one generation down the line is absolute trash.
Nope, people are quite resilient. As long as it’s not a literal new born, the chance of survival isn’t THAT low. Once you get past 4 years and up, a human can manage quite well.
Also dying because no one takes care of you and you fail to aquire food and dying of a stroke/seizure are 2 very different things.
This is because of semi hardcoded stuff using the mechanics of hormones that interact with the neurons in the brain, I would say. They are hardcoded by the instructions provided by the DNA, I believe.
About the learning differences between human and LLM, there I believe that a sub-“module" of the brain functions very similar to how the LLMs work with just a way better/efficient learning algorithm that is helped by the other modules in the brain like the part that can simulate 3D space and interpret other sensory data like feeling touch, vision, smell etc
Current LLM models are being used in static manner without ability to learn in real time, so of course it can not do anything it has not learned yet.
It is just a theory and it can not be proven wrong since the understanding of neurons is not advanced yet.
Well, or at least, I did not hear a good argument that proves that theory 100% wrong.
Nope. Autists reason entirely without words/language *. Neurotypicals are capable of that too, but it’s generally more convenient for them to bridge over words in conscious reasoning. They are basically fooled into thinking, ‘thinking’ is based on language.
* have to “translate” thoughts for conversation, which is exhausting
Well, maybe I should have written "neural network” instead of LLM/LAM… Our brains, like LLM work by hardening paths which the data goes through the nodes. In LLM we simulate the chemical properties of the neurones using math. And we have already prototype of chips that work with lab grown brain tissue that show very efficient training capabilities in machine learning (it already plays pong)
🤷🏻♀️ think about that how you want, we will see anyway
Why is it impossible to build a text-based AGI model? Maybe there can be reasoning in between word predictions. Maybe reasoning is just a fancy term for statistics? Maybe floating-point rounding errors are sufficient for making it more than a mere token prediction model.
Not all the time. I can think about abstract concepts with no language needed whatsoever. Like when I’m working on my car. I don’t need to think to myself “Ah this bolt is the 10mm one that went on the steering pump”, I just recognize it and put it on.
Programming is another area like that. I just think about a particular concept itself. How the data will flow, what a function will do to it, etc. It doesn’t need to be described in my head with language to know it and understand it. LLMs cannot do that.
A toddler doesn’t need to understand language to build a cool house out of Lego.
Well, you just have to give the LLM (or better said to a general machine learning Algorithm) a body with Vision and arms as well as a way to train in that body
I’d say that would look like AGI
The key is more efficient training algorithms that don’t need a whole server centre to train 😇I guess we will see in the future if this works
This poster asked some questions in good faith, I don’t understand the downvotes when there’s a legitimate contribution to the conversation because that stifles other contributions.
Except LLMs don’t actually have real reasoning capacity. Hooking in different models that can translate more of the world to text could give the LLM a broader domain, but not an entirely new ability beyond its architecture. That might make it more convincing, but it would still fail in the same ways as it currently does.
You’re doing reasoning based on chemical reactions. Who says it can’t do reasoning based on text? Who says it’s not doing that already in some capacity? Can you prove that?
If you genuinely think LLMs are anyway capable of even basic reasoning despite all arguments towards the contrary, I honestly don’t want to try convincing you anymore. You’re asking for a miracle out of me—to explain consciousness itself, even—while you can just say “but there’s a chance” even though LLMs can’t get basic facts right.
Maybe it can. If you find a way to port everything to text by hooking in different models, the LLM might be able to reason about everything you throw at it. Who even defines how AGI should be implemented?
The LLM is just trying to produce output text that resembles the patterns it saw in the training set. There’s no “reasoning” involved.
A LLM is basically just an orchestration mechanism. Saying a LLM doesn’t do reasoning is like saying a step function can’t send an email. The step function can’t, but the lambda I’ve attached to it sure as shit can.
ChatGPT isn’t just a model sat somewhere. There are likely hundreds of services working behind the scenes to coerce the LLM into getting the right result. That might be entity resolution, expert mapping, perhaps even techniques that will “reason”.
The first initial point is right, though. This ain’t AGI, not even close. It’s just your standard compositional stuff with a new orchestration mechanism that is better suited for long-form responses - and wild hallucinations…
Source: Working on this right now.
Edit: Imagine downvoting someone that literally works on LLM’s for a living. Lemmy is a joke sometimes…
they’re very very anti ai and crypto. I understand being against those, but lemmys stop caring about logic when it comes to those topics.
I think many in the AI space are against the current state of how AI is being pushed, probably just as much as the average tech person.
What is ridiculous is that Lemmy prides itself as both forward-thinking and tech focused, and in reality it is far more close-minded than Reddit or even Twitter. Given how heavily used Mastodon is in tech spheres it makes Lemmy look like an embarrassment to the fediverse…
Yeah, there is every reason to be sceptical of the hype around AI, in particular from the big tech companies. But to a significant part of the Lemmy userbase saying “AI” is like saying “witch” in 17th century Salem. To the point where people who are otherwise very much left wing and anti-corporate will take pro-IP/corporate copyright maximalist stances just becuase that would be bad for AI.
You might be interested in Nim then when you get a chance. Talk about orchestration
https://developer.nvidia.com/nim
And how does reasoning work exactly in the human body? Isn’t it LLM/LAM working together with hormones? How do you know that humans aren’t just doing something similar? Your mind tricks you about a lot of things you experience, how can you be sure, your "reasoning” is just sorta LLM in disguise?
Language models are literally incapable of reasoning beyond what is present in the dataset or the prompt. Try giving it a known riddle and change it so it becomes trivial, for example “With a boat, how can a man and a goat get across the river?”, despite it being a one step solution, it’ll still try to shove in the original answer and often enough not even solve it. Best part, if you then ask it to explain its reasoning (not tell it what it did wrong, that’s new information you provide, ask it why it did what it did), it’ll completely shit it self hallucinating more bullshit for the bullshit solution. There’s no evidence at all they have any cognitive capacity.
I even managed to break it once through normal conversation, something happened in my life that was unique enough for the dataset and thus was incomprehensible to the AI. It just wasn’t able to follow the events, no matter how many times I explained.
Maybe the grown up human LLM that keeps learning 24/7 and is evolved in thousands of years to make the learning part as efficient as possible is just a little bit better than those max 5year old baby LLM with brut force learning techniques?
The 5 year old baby LLM can’t learn shit and lacks the ability to understand new information. You’re assuming that we and LLMs “learn” in the same way. Our brains can reason and remember information, detect new patterns and build on them. An LLM is quite literally incapable of learning a brand new pattern, let alone reason and build on it. Until we have an AI that can accept new information without being tolled what is and isn’t important to remember and how to work with that information, we’re not even a single step closer to AGI. Just because LLMs are impressive, doesn’t mean they posses any cognition. The only way AIs “learn” is by countless people constantly telling it what is and isn’t important or even correct. The second you remove that part, it stops working and turns to shit real quick. More “training” time isn’t going to solve the fact that without human input and human defined limits, it can’t do a single thing. AI cannot learn form it self without human input either, there are countless studies that show how it degrades, and it degrades quickly, like literally just one generation down the line is absolute trash.
A human not trained by other humans also just dies…
Nope, people are quite resilient. As long as it’s not a literal new born, the chance of survival isn’t THAT low. Once you get past 4 years and up, a human can manage quite well.
Also dying because no one takes care of you and you fail to aquire food and dying of a stroke/seizure are 2 very different things.
This is because of semi hardcoded stuff using the mechanics of hormones that interact with the neurons in the brain, I would say. They are hardcoded by the instructions provided by the DNA, I believe.
About the learning differences between human and LLM, there I believe that a sub-“module" of the brain functions very similar to how the LLMs work with just a way better/efficient learning algorithm that is helped by the other modules in the brain like the part that can simulate 3D space and interpret other sensory data like feeling touch, vision, smell etc
Current LLM models are being used in static manner without ability to learn in real time, so of course it can not do anything it has not learned yet.
It is just a theory and it can not be proven wrong since the understanding of neurons is not advanced yet.
Well, or at least, I did not hear a good argument that proves that theory 100% wrong.
Nope. Autists reason entirely without words/language *. Neurotypicals are capable of that too, but it’s generally more convenient for them to bridge over words in conscious reasoning. They are basically fooled into thinking, ‘thinking’ is based on language.
* have to “translate” thoughts for conversation, which is exhausting
Well, maybe I should have written "neural network” instead of LLM/LAM… Our brains, like LLM work by hardening paths which the data goes through the nodes. In LLM we simulate the chemical properties of the neurones using math. And we have already prototype of chips that work with lab grown brain tissue that show very efficient training capabilities in machine learning (it already plays pong) 🤷🏻♀️ think about that how you want, we will see anyway
PS: 😁 I am most likely neurodivergent as well 🙌🏻
You’re doing that too from day one you were born.
Besides, aren’t humans thinking in words too?
Why is it impossible to build a text-based AGI model? Maybe there can be reasoning in between word predictions. Maybe reasoning is just a fancy term for statistics? Maybe floating-point rounding errors are sufficient for making it more than a mere token prediction model.
Not all the time. I can think about abstract concepts with no language needed whatsoever. Like when I’m working on my car. I don’t need to think to myself “Ah this bolt is the 10mm one that went on the steering pump”, I just recognize it and put it on.
Programming is another area like that. I just think about a particular concept itself. How the data will flow, what a function will do to it, etc. It doesn’t need to be described in my head with language to know it and understand it. LLMs cannot do that.
A toddler doesn’t need to understand language to build a cool house out of Lego.
Well, you just have to give the LLM (or better said to a general machine learning Algorithm) a body with Vision and arms as well as a way to train in that body
I’d say that would look like AGI
The key is more efficient training algorithms that don’t need a whole server centre to train 😇I guess we will see in the future if this works
This poster asked some questions in good faith, I don’t understand the downvotes when there’s a legitimate contribution to the conversation because that stifles other contributions.
Except LLMs don’t actually have real reasoning capacity. Hooking in different models that can translate more of the world to text could give the LLM a broader domain, but not an entirely new ability beyond its architecture. That might make it more convincing, but it would still fail in the same ways as it currently does.
You’re doing reasoning based on chemical reactions. Who says it can’t do reasoning based on text? Who says it’s not doing that already in some capacity? Can you prove that?
If you genuinely think LLMs are anyway capable of even basic reasoning despite all arguments towards the contrary, I honestly don’t want to try convincing you anymore. You’re asking for a miracle out of me—to explain consciousness itself, even—while you can just say “but there’s a chance” even though LLMs can’t get basic facts right.
LLMs can’t reason about anything, ever.
_