the publication system is broken because publication rather than quality is in the self interest of a large fraction of the community.
Yes: I also believe that is the largest contributing factor.
True AI? I am not sure we can even define what intelligence is.
The g-factor as used in psychometrics, measured by the statistical ability to solve problems one has not encountered before, is a useful definition.
I tend to default to that one, until a more useful definition happens to crop up.
Calling the large memory models Babel babble generators overstates the case.
I disagree, obviously, but here is the reason: the models construct sentences based on a massive set of internally weighed statistical relationships. (Look up machine learning
transformer model for a better description of that.)
LLMs do not consider the content of any word –– technically speaking, the exact difference between the token at hand and its nearest neighbours. At best, you can claim they roughly model the relative
magnitude of such differences.
One definition of
to utter meaningless words is
to babble. I am using that as the technical exact definition for the output, since the LLMs
cannot know the meaning of the individual words, only their relationships to each other. The latter is also why the output
seems intelligent, but is not. (A comparison to a very powerful search engine over its source material using fuzzy matching is also apt.)
I do recognize that the human language acquisition process also
starts with a similar charting of relationships. However, by interacting with others (especially face-to-face, so that body language and microexpressions will affect our own understanding of each term) we refine the meaning the word has for ourselves. Similarly, sentence construction, word order, and so on are built in interaction, with each interaction adding meaning (based on the difference in original intent versus reaction observed) on top of those associations.
I would agree that the people who are truly superior at something and truly cannot communicate are rare. But there is a broad spectrum of this capability just as in any other area of human performance.
Sure: just look at my own output. I often fail English. Like LLMs, my output is verbose and typically well-structured, yet I still fail, because of lack of face-to-face use (which leads to the lack of direct feedback bypassing my conscious mind to my language understanding), and failure to predict how specific terms and sentences are understood/perceived/assigned meaning by others.
I do believe that if we develop LLMs into a tool that can track its sources, and use models generated from controlled datasets, we
can build tools that would help a lot with especially scientific communication.
In simple terms, that corresponds to creating LLMs that can translate jokes and anecdotes, perhaps even poems, across languages while still tracking the reasons for its choices (as, for example, references to the strongest source materials affecting its choice).
To continue my gun analog, those would correspond to bolt guns, dart guns with a variety of medical substances available, guns designed for shooting blanks at short range safely (for use in entertainment), and so on. We just are not there yet. Nobody seems even remotely interested in developing such, in fact. Instead, LLMs are used as if they were already 'there', with the end result that they only make it easier for those who do not have anything meaningful to say to couch that non-message in attractive outer shape. Thus, my opinion that they just cause more shit to be generated.
(It is interesting to compare LLM proponents' assertions and beliefs to those of explosive and weapon inventors. Belief that sufficiently efficient killing machines would prohibit wars and save lives, due to the excessive cost in human lives, has been common. But perhaps this comparison is too 'angry', and something like adding tetraethyl lead to gasoline to aid gasoline engine efficiency, would be more apt. Or perhaps that too is 'too negative' for the LLM proponents.)
One of my hobbies is looking for science fiction stories with interesting storylines. I'm not that interested in the characters per se, I'm mostly interested in the
events depicted. Many aspiring authors are now using LLMs to "flesh out their ideas", and the output (from
my point of view) is so crap and waste of my time, that I've
started to avoid looking at the output of new authors altogether. Granted, perhaps my view of LLM use is overly negatively colored because of this, but having an academic background myself, I do not see scientific authors behaving any different.