Benchmarks used to rank AI models are several years old, often sourced from amateur websites, and, experts worry, lending automated systems a dubious sense of authority
The article makes the valid argument that LLMs simply predict next letters based on training and query.
But is that actually true of latest models from OpenAI, Claude etc?
And even if it is true, what solid proof do we have that humans aren’t doing the same? I’ve met endless people who could waffle for hours without seeming to do any reasoning.
I think I know enough about these concepts to know that there isn’t any conclusive proof, observed in output or system state, to establish consensus that human speech output is generated differently to how LLMs generate output. If you have links to any papers that claim otherwise, I’ll be happy to read them.
I’ve just diagonally read a google link where the described way humans work with language appears for me to be very similar to GPT in rough strokes. Only human brain does a lot more than language. Hence the comparisons to the mechanical Turk.
I’m not saying humans and LLMs generate language the same way.
I’m not saying humans and LLMs don’t generate language the same way.
I’m saying I don’t know and I haven’t seen clear data/evidence/papers/science to lean one way or the other.
A lot of people seem to believe humans and LLMs don’t generate language the same way. I’m challenging that belief in the absence of data/evidence/papers/science.
You’re actually incorrect in regards to Russell’s teapot in this instance. The correct approach is to admit to yourself and others you don’t know. Not to assume a negative became you can’t prove a positive, if you can’t prove the negative either.
The article makes the valid argument that LLMs simply predict next letters based on training and query.
But is that actually true of latest models from OpenAI, Claude etc?
And even if it is true, what solid proof do we have that humans aren’t doing the same? I’ve met endless people who could waffle for hours without seeming to do any reasoning.
Information theory, entropy in Markovian processes. Read up on these buzzwords to see why.
I think I know enough about these concepts to know that there isn’t any conclusive proof, observed in output or system state, to establish consensus that human speech output is generated differently to how LLMs generate output. If you have links to any papers that claim otherwise, I’ll be happy to read them.
What? Humans, ahem, collect entropy every moment of their existence.
I mean I have an opinion too; what I’m seeking is evidence.
Evidence for what?
I’ve just diagonally read a google link where the described way humans work with language appears for me to be very similar to GPT in rough strokes. Only human brain does a lot more than language. Hence the comparisons to the mechanical Turk.
Also Russell’s teapot.
I’m not saying humans and LLMs generate language the same way.
I’m not saying humans and LLMs don’t generate language the same way.
I’m saying I don’t know and I haven’t seen clear data/evidence/papers/science to lean one way or the other.
A lot of people seem to believe humans and LLMs don’t generate language the same way. I’m challenging that belief in the absence of data/evidence/papers/science.
Like going out and meeting a dino - 50% yes, 50% no. It’s a joke.
Russell’s teapot again.
You’re actually incorrect in regards to Russell’s teapot in this instance. The correct approach is to admit to yourself and others you don’t know. Not to assume a negative became you can’t prove a positive, if you can’t prove the negative either.