Altera is back

[JPortici]

altera should be henceforth be known as altuna

did you try asking it why altuna?

[Nominal Animal]

You should spend some time with GPT4 before drawing conclusions based on 40-year-old concepts like Kohonen nets.

That is where I started, KE5FX, not how I draw conclusions.  I'm pretty well aware of all of the math side involved in LLMs, down to details on sentence embedding vectors and their similarity models.  I'm simply not blinded by their apparent brilliance like you are.

You should spend some time with the history of the field, to truly understand the situation, and not be blinded by the apparently wonderful results.
LLMs and other artificial neural networks are utterly amazing at complex pattern matching, that's all.

The only thing that has changed, is that now we have enough memory and computational power –– especially with the so-called "AI accelerators" or NPUs that use integer and/or half-precision (IEEE 754-2008 Binary16) floating-point operand types for multiplication, addition, and subtraction –– to apply large enough neural networks to boost the complexity to levels where their behaviour seems "intelligent" to humans, and enough data one can harvest for free (if one ignores copyright issues, like these model developers tend to do, unless it is their own copyright) to train the networks.  Everything else, especially the math side, has seen only normal incremental development.

The results that seem emergent are really just emerging out of the hugely increased complexity, and therefore correlation capabilities, of the networks.

A neural network is a perfect example of combinatorial explosion.  If you keep the inputs and outputs at fixed size/complexity, the memory and correlation capabilities of the network increase exponentially when the size of the network increases, because of the combinatorial explosion in the number of possible input-to-output correlations.

The reason I believe artificial general intelligence is just as far in the future as before, and not related to LLMs, is that general intelligence is at its core the ability to solve problems not encountered before; i.e. not within the training set.  In psychometrics, this is called the g-factor, and is behind all the predictive power of "IQ tests".  (Not all IQ tests manage to obtain results reflecting the g-factors, however, possibly explaining why increased education seems to slightly boost IQ scores.)
Memory and recall is not involved in g-factor per se, and you cannot significantly raise your g-factor by reading or learning more.  For humans, g-factor seems to be fixed by early childhood, whereas the related "IQ" can be slightly boosted by education and training.

Simply put, general problem solving ability as modeled by the g-factor is not a transformation on experience, or a pattern matching skill, the two things neural networks do well.  Exactly what is needed to get to true general intelligence, the ability to successfully solve problems never encountered before, is an open question.  After all, the animal brain is a neural network.  From species we consider "intelligent", the ability to interact in complex ways seems crucial.  (Whether also requires things like theory of mind is unclear; but logic and reasoning and the ability to model and predict results of interactions seem strongly correlated, when comparing e.g. humans to corvids.)

[KE5FX]

The reason I believe artificial general intelligence is just as far in the future as before, and not related to LLMs, is that general intelligence is at its core the ability to solve problems not encountered before; i.e. not within the training set.

Just curious, what are your thoughts on this?

If it answered 100% of those questions correctly instead of 39%, would you still call it a mindless pattern-matching technique?

How about 45%?  50%? ... I'm just trying to understand where your goalposts actually are.

[nctnico]

The reason I believe artificial general intelligence is just as far in the future as before, and not related to LLMs, is that general intelligence is at its core the ability to solve problems not encountered before; i.e. not within the training set.

Just curious, what are your thoughts on this?

If it answered 100% of those questions correctly instead of 39%, would you still call it a mindless pattern-matching technique?

How about 45%?  50%? ... I'm just trying to understand where your goalposts actually are.

A better question is: can AI deal better or worse with new problems compared to the average human? My observation is that there are quite a few humans around which can't deal with new problems. Some are literally remote hands attached to a phone with somebody giving instructions at the other end. You might be able to replace the person giving instructions with AI even at today's level.

[Nominal Animal]

The reason I believe artificial general intelligence is just as far in the future as before, and not related to LLMs, is that general intelligence is at its core the ability to solve problems not encountered before; i.e. not within the training set.

Just curious, what are your thoughts on this?

It maps knowledge, not intelligence.

If it answered 100% of those questions correctly instead of 39%, would you still call it a mindless pattern-matching technique?

Of course.  That is memory and correlation, not problem-solving.

We need to use written text cognitive tests that were not described or discussed in the training set.  Considering large swathes of the internet are used as the training set, it is pretty difficult to do.

Even then, we don't expect a pure transformer to yield a 0% score.  The real floor is random choice.  Any tendency towards correct solutions indicates either intelligence, or that the problem type is related/correlated to a problem within the training set.

For corvids, rats, and other animals, physical puzzles involving tool use and phenomena like adding water or a rock to a vessel containing water to raise the water level to bring a reward within reach, have been used; and levers, ropes, and push-sticks.  These need to be used in combination or in sequence.  The number of attempts needed to find a solution is part of the intelligence estimate.

In particular, intelligence is not measured directly, but comparatively and statistically.  (For the g-factor, using e.g. factor analysis.)

The exact same problem is with IQ tests on humans.  It is true that a single IQ test has cultural and whatnot biases, and that only by examining a random set of tests and their results (across many people) can the g-factor for each individual be estimated.  Here, we have something/someone that has basically access to all of the web, and we want to find out whether they can actually solve problems, or only look up the problem or similar problem, and derive an answer.  One is related to "intelligence" (problem-solving without prior experience or training), and the other to "memory" and "pattern-matching".

If you were to accept that "intelligence" ≠ "memory" and/or "pattern-matching", it would not mean LLM's would suddenly become useless.  Just less 'dazzling'.
To me, the pattern-matching capability is the key.  It is extremely useful, but it must not be confused with problem-solving.  Pattern-matching answers the question "is there a solution to this type or pattern of a problem listed in the training set?", whereas true intelligence is required to solve a problem previously unknown (including the problem pattern, not just the details of the problem).

(It is also why I mentioned Kohonen networks: they were the first ones with true end-user practical use cases like OCR, back when best flatbed scanners were made by HP and had SCSI interfaces, and computers just didn't have the RAM or computational speed to use much larger networks.  If you add a perceptive filter, turning bitmaps into glyph skeletons or vector models, you can make it much more powerful.  Different sentence embedding approaches have similar effects for LLMs.)

Thus, asking an LLM to solve a problem, is equivalent to asking an oracle whether the problem pattern and solution is listed in a database (the LLM training set).  If it is, the oracle produces it.  If it isn't, the oracle produces correct-looking garbage.  It is extremely difficult to tell which is which, unless you derive or verify the solution for yourself; and this is the reason I don't like playing with them that much.
To find whether the oracle is intelligent, one needs to find out whether it can solve problems it has absolutely no prior experience with.

[KE5FX]

Of course.  That is memory and correlation, not problem-solving.  We need to use written text cognitive tests that were not described or discussed in the training set. ... Thus, asking an LLM to solve a problem, is equivalent to asking an oracle whether the problem pattern and solution is listed in a database (the LLM training set).  If it is, the oracle produces it.  If it isn't, the oracle produces correct-looking garbage. 

Sounds like you didn't even skim the article.  You should; it's interesting.  The whole idea was to benchmark with hundreds of original graduate-level problems that could not possibly have appeared in any training set.  GPT4 wasn't (yet) competitive with domain experts, but outperformed PhDs from unrelated domains.

[Nominal Animal]

The whole idea was to come up with original graduate-level problems that could not possibly have appeared in any training set.

It is not enough for the particular problem to not exist in the training set.  It is the pattern of the problem existing in the training set that is the problem.

Remember, the sentence embedding turns the input into a vector.  Individual words do not need to match, if the pattern, after whatever transformations is done to it, matches.  This is where neural networks excel.

[KE5FX]

What might be an example of a question that would meet your criterion of not relying on pattern-matching?

[Nominal Animal]

As to "undergraduate level" having anything to do with intelligence, some of the graduates and PhD's I've met have had excellent memories and social skills, but the logical, deductive, and problem-solving skills of a wet paper bag.  I don't think it is anywhere near an appropriate yard-stick.

What might be an example of a question that would meet your criterion of not relying on pattern-matching?

I'm not sure I am intelligent enough to come up with one that has no pattern analog on the web already.
(I am pretty sure there are others either doing this already, or who will do it within the next few years.)

I suspect the practical way to find an answer to this question would be to use a carefully limited dataset in training a purported AI model, that omitted one narrow sector of human knowledge, and then asked the purported AI questions whose answers would fill in the blanks.

For cognitive tests, I personally place much more weight on the nonverbal ones (like Raven's Matrices and similar ones) than verbal ones.  Me fail English often, too.

I was thinking about variants of the Sally–Anne test, perhaps with added decoy actors to confuse the pattern, but it turns out Microsoft already did that last year, and it is more about theory of mind (and self-awareness) than intelligence.  (I most likely remembered reading about it.)

Perhaps if one could invent an algorithm to solve a previously unsolved logical problem, for example like a math one I did here, and instead of publishing it, create a sequence of questions to see if a purported AI could replicate the discovery if prompted step-by-step?  The key discovery there was to replace the x axis with the same value written in binary in the reverse order, i.e. turning 101100111₂ into 111001101₂.  There are theoretical 'hints' that prompted me to try it, so I did not deduce it; I only discovered it.  Diligent work, not brilliance.
Considering many things are discovered and rediscovered again and again, and often described in one domain before "discovered" anew in another by unrelated people, this may not be very reliable, as there is no proof the pattern does not already exist in analogous form in the training set.

Regardless of whether one labeled such logical extensions as "intelligence" or not, it would add another facet beyond pattern-matching to neural networks.
I am conflicted on whether I would be surprised about it.  On one hand, discoveries in one domain often spur discoveries in other domains, thus creating analogous 'patterns', so some extensions to a partial knowledge base could be deduced based on pattern matching, and not surprising.  On the other hand, it would extend pattern-matching to applying analogous patterns, and that itself would be extremely useful (in addition to pattern matching alone).
The problem is determining whether it can do so reliably.

My biggest peeve with software is software that silently garbles data, when it could have told the user something happened that might mean the data was garbled.  (It is because some people think run-time error checks are waste of effort, because "if that error did occur, they'd already have bigger problems than anything to do with this program; and besides, it has never happened on my system, so must be extremely rare and not worth checking for anyway".)
The problem with neural networks is that they cannot qualify their output as to how reliable or likely it is to be correct.  There just isn't such a possibility in neural network models.  The output is what you get; that's it, it is neither right nor wrong.
Compare to a human, who might choose to start with "This is my current understanding.  It is not my core field of expertise, but I've dabbled in this – consider me a hobbyist – so I could be wrong.  I don't think so, though."  An LLM cannot do that, and always will state the answer as if it were a fact.  Whether it is or is not, is not available from the model.

[glenenglish]

ChatGPT or other generative AI -  cannot make analytical distinctions, only well chosen regurgitation.

[pcprogrammer]

A teacher from long back told me that you can know the formula but if you don't know how to use it, it is worthless. I believe that to be very true. You can have all the knowledge in the world but when you lack understanding of it and don't know how to use it, it is indeed worthless. Sure you might sound intelligent when you regurgitate the knowledge but when it does not match the question you actually look very stupid.

Another saying I believe still holds up is "If the human brain was so simple that we could understand it, we would be so simple that we couldn't".

In the basis it might well have neural networks, but probably more complex then the simple multiply and add factoring used in today's AI networks.

As to intelligence tests I concur with Nominal Animal that the nonverbal ones shed light on actual intelligence and ability to solve problems then the more common ones that include language skills and social knowledge, like who was president from some country in some period of time. But even for these nonverbal ones, one has to learn or at least understand a sample to deduce the rest.

But I also believe that intelligence comes in many forms. Some shine at math, where others shine in music or art, etc.

[Nominal Animal]

A teacher from long back told me that you can know the formula but if you don't know how to use it, it is worthless.

That's exactly why I too push people to use simple tools like dimensional analysis.  Fancy name, but just means you ignore numeric values and check the units only through the formula, to see if it makes any sense whatsoever.  Next step, simple powers of ten with human-scale known results, to see if it produces garbage.  Takes only minutes, but tells a lot about the formula.

I do not bother to remember any formulas off-hand.  It just isn't worth the effort and risk (in misremembering).  It is much better to understand where and how things apply.  For example, in physics, it is useful to know that momentum is conserved in "elastic collisions", when no internal deformation or change occurs as if the objects were infinitely hard spheres; and that total energy is conserved in all isolated systems.  You can quickly find the relevant formulas and be able to apply them.  For things like "sentence embedding" in large language models, you need to understand it is the numerical vector representation of the input textual data or words, that can be used as an input to the actual numerical neural network; the details of exactly how it can be done, how two sentence embeddings can be compared to each other, and what differences the different approaches have, you can look up whenever you need.

In programming, it means there is no use in trying to remember interface details, like the parameter order for memset() for example.  Just keep a terminal window open, ready for man -s 2,3 function or a browser window open for Linux man pages online (which contains the standard C documentation, and mentions which systems/standards provide each, so isn't "just" for Linux), or whatever documentation you're using.  You'll soon become so efficient and fast at looking up the exact details, so that memorizing the interfaces (and occasionally remembering them wrong, wasting precious development time) becomes counterproductive.  And you only need to understand when to use which interfaces, not remember nitty-gritty details.

In such terms, neural networks and large language models are not difficult to understand.  The only big step is discarding any preconceptions one might have, and keep ones mind open for truly understanding, instead of trying to force the new information about it into the preconceived notions.  Typically, this is called "thinking outside the box", but really, it is more like an attitude you can learn to ignore your own preconceptions and assumptions.  It hurts to admit, yes, but our current 'knowledge' often limits what 'new' we are ready to learn.

It is a typical human error to confuse existing knowledge (wisdom) with the ability to solve problems (intelligence).

Another saying I believe still holds up is "If the human brain was so simple that we could understand it, we would be so simple that we couldn't".

Exactly.  For example, I might sound "clever", but that's all just the result of hard effort; I'm not that intelligent.

It is extremely difficult to design an intelligence test for those more intelligent than the test maker in the g-factor sense; the IQ tests that measure ability and knowledge are easier.

It is even more difficult to design a test to measure something you cannot exactly describe or define, like "intelligence".
"Wisdom" is much easier, and because of the common error of conflating the two, many tests claiming to test for "intelligence" or problem-solving ability, actually just test for "wisdom" or the breadth of experience in problem solutions.

But I also believe that intelligence comes in many forms. Some shine at math, where others shine in music or art, etc.

We also lack proper terms to describe these.  I like to use "intelligence" specifically in the g-factor sense, as in the ability to solve new problems not previously encountered (even in analogous forms).  For the others, I use various different terms.

True creativity, or creating something new instead of "creating" something "new" by mixing existing things, is a very, very big one.
I do not have a word for it, but creating something interesting by mixing existing things is also "creative", and some of the neural networks can do this extremely well.  This, too, is useful in practice, but I don't like to conflate it with the truly-new-creative term.  Needs a new word; I don't have one.  Synthesizy?

Intuition, and especially intuitive leaps, are another: it is like a transformation function on a pattern that allows you to match it to a completely different one, like Fourier, Laplace, or Z-transforms applied in signal processing.  Sentence embedding in large language models is itself such a transform, with results similar to how a Fourier transform on the digits of two multiplicands allows you to calculate their product with a simple sum and an inverse Fourier transform.

Even "simple" pattern matching, especially if adaptive (i.e. does not look for exact copy, but uses an analog of the abstraction-filtration-comparison test used for similarity in the copyright sense in the USA), is extremely useful.  Many, many human jobs have a large pattern matching component as a core part: for example, you have a set of rules, and you need to apply them.

The problem in using LLMs and transformers in pattern matching is that there is no way to know how accurate the pattern match is, because the matching itself is the product of the entire network.  A completely separate mechanism is needed to check the results for accuracy or applicability.  Current use cases require us humans to do that check ourselves.  To anyone familiar with human nature, that means the checks are rarely, basically never, done.  And we can see the results: lawyers citing cases that do not exist because they used an LLM, and so on.  Ugly.

I myself enjoy the discovery process, the act of solving the problem in a manner I can show how I found the result.  I believe that being able to show how the result was obtained, and derive the rules when the result applies from that, is about half the worth of the entire solution.  I dislike the current use patterns of LLMs exactly because they discard this half, as if it was worthless and not needed anyway: as if the quick wrong answer is worth just as much as a slow, proven-correct one.  I do not like that, because to me, it is exactly equivalent to just applying formulas blindly, and presenting the results using similar language as verified results tend to be presented.  It is too close to intentionally lying to me.  At least humans are sometimes held accountable for how they apply the rules they're supposed to pattern-match against.

However, that does not change the fact that neural networks are an extremely powerful tool.  Once again, it is just a question of how us humans use it.  I wholeheartedly support many use cases –– including things like summarizing existing texts ––, and object to some.  I'm also a bit peeved that some profit hugely off of models they ripped the training data from off the web; I'm not sure that is fair.  I want all interactions to be mutually beneficial.

[pcprogrammer]

It is extremely difficult to design an intelligence test for those more intelligent than the test maker in the g-factor sense; the IQ tests that measure ability and knowledge are easier.

Certainly true, and these latter test do have their use. Still possible to get valuable information from it when looking for the right person for a job, but it should not solely be based on those results.

In the Netherlands there was a TV program once a year called the "National IQ test". An assembly of all sorts of groups, like students, athletes, etc in different sections and a bunch of so called BN'ers. (Known Dutch people like actors, politicians, etc) The presenters asked all sorts of questions about different kind of skills. Think of language, politics, science, etc, but also memory based things at the end of the program. Like what was the color of something shown during the musical intermezzo.

In one show as part of language skills a multiple choice question was "wat is swaffelen".  Yes you guessed it right, I did not know the answer to that one. See here on what it means and ask your self in what society such a word becomes the word of the year. 

Even though the tests did not measure actual intelligence it did show the bell curve results across the attendees. And some celebs needed to hide in shame. A talk show hostess only scored 77. Ouch.