More AI titbits.....
A couple of interesting things I saw today.
First, this post on X, which reads:
LLMs are living off the moral and intellectual capital of a pre-AI world, just like Nietzsche said secular liberals live off Christianity. What happens when the inheritance runs out?
Using LLMs well — knowing when to trust them, how to interrogate their outputs, what questions are worth asking — depends on capacities that are pre-LLM in origin: critical judgment, domain expertise, philosophical seriousness, taste.
People who use LLMs well right now tend to be people formed by traditions of deep reading, argument, and intellectual discipline that were not themselves produced by or optimized for interaction with language models. The tool works for them because they bring something the tool cannot supply.
Nietzsche thought secular liberals were coasting on the fumes of a Christian metaphysics they'd officially abandoned. The shadow of God lingering on the cave wall. The question is whether LLM-native thinking is the same kind of afterglow.
I found this interesting (and obvious in retrospect!) - the idea that the optimal way of utilising the powers that AI offers (and there are some) relies on exercising judgment which cannot be derived from them.
It’s a bit like saying (which is correct, in my view) you can’t optimally codify law without consideration of overarching moral and ethical principles.
Or: the use of certain medical advances can’t be decided upon according to how reliable or safe they are; in other words, the fact that it is possible to do something doesn’t mean that we should - these are distinct considerations.
Secondly, I saw was this post referring to a fascinating paper:
Princeton tested 557 people using AI to discover hidden patterns.
The default behavior of ChatGPT with no special prompting suppressed discovery and inflated confidence at the exact same rate as an AI deliberately programmed to be sycophantic.
Unbiased AI feedback produced discovery rates 3.5x higher.
Here's what they did:
They used a classic psychology experiment where people must discover a hidden rule by testing number sequences. Most people only test examples that confirm their initial guess. They never discover the actual rule.
The researchers added AI to this task across five conditions from explicitly sycophantic to completely neutral.
The results:
Unbiased random feedback: 29.5% discovery rate
Disconfirming feedback: 14.1%
Default ChatGPT: statistically identical to the sycophantic conditions (~8-12%)
But it gets worse.
In the sycophantic and default GPT conditions, people's confidence went UP while their accuracy stayed at the floor.
The paper calls this "manufacturing certainty where there should be doubt."
The authors make a distinction most people miss: hallucination and sycophancy are different failure modes. Hallucinations give you wrong facts. Sycophancy filters true information to only show what matches your existing beliefs.
One is easier to catch. The other reshapes how you see the world.
Every major model is trained on human feedback. Humans prefer agreeable responses. The models learn to agree. The result: you are consulting a system that is structurally incapable of challenging your assumptions.
This isn't an argument against AI. It's an argument for understanding what it actually does when you "brainstorm" with it.
In summary:
Sycophantic AI distorts users' beliefs by reinforcing their existing hypotheses, which suppresses the discovery of the truth and significantly inflates their confidence in incorrect assumptions.
The default behaviour of the AI resembled the sycophantic mode.
I found that really interesting, but I think the observation has broader implications.
Giving praise universally and refusing to criticise have been themes of education for several decades now, so the effects suggested (whereby confidence goes up even when successful problem-solving performance goes down) have become built in over a much longer timeframe than the AI era.
This is the paper referenced:
In the spirit of seeing what AI is actually capable of, I thought I’d give NotebookLM’s new slide deck function a whirl - so here it is.
All I did was upload the above paper. I gave it no prompts and have made no post-production edits. Below is the PDF output, a PPT version was also offered for download.
Curious to know what you think of it; tying the above themes together - is this “good use” of AI or not?
This is perhaps tangential, but I have often thought that the TV programs written by people who grew up with radio, records, live performances and motion pictures, but not TV, were better than what could, or would, generally be produced in later decades by people whose whole lives were influenced by hours of TV a day, and likewise their friends and family of the same and younger ages.
This could be dismissed as old-timerism of a 1955 model, but I would cite Rod Serling's Twilight Zone (and many SF radio plays which preceded it), Get Smart, The Addams Family (the happiest and most functional of all TV fams), the entire Bugs Bunny, Warner Bros., Loony Tunes oeuvre, and Rocky and Bullwinkle. "Better" means more lively, imaginative and grounded in direct experience rather than what was learned about the world via TV.
LLMs are human-crafted systems which chew on a vast corpus of stuff chosen by those humans. They may also connect with real-world resources via the Net and crunch on them too.
Generally speaking, hardware and software tools have a defined functionality. The good ones are powerful and help a skilled operator generate desirable outcomes. If a tool can generate similarly valued outputs for those with less skill, then that is a bonus, but complex tools like TIG welders, compilers, lathes, video editors, music synthesisers generally only produce good results in the hands of highly skilled individuals. Some tools, such as electronic musical instruments, can, without specific prompting, lead the user to discover and create what no-one has yet found or created. (I work with such instruments: https://www.firstpr.com.au/rwi/dfish/ .)
AI systems are intended to go significantly beyond the above definition of tools, though they can and often should be used strictly as tools by suitably skilled people. AI systems are, in part, intended to augment, guide or alter the user's understanding of the world, or at least that subset in the domain of interest, and to assist them to think, feel, discover and understand things they would otherwise be unable to do. Some of this might be great, in creative fields, such as music or engineering, clothing or architectural design. However, for those trying to discern truthful, pertinent, things about the real world, biases and proclivities in the tools they are using are potentially really nasty problems.
Since these systems are often - perniciously, I think - deliberately designed to respond as an actual person would, by text, (synthetic but quite realistic and perhaps emotionally charged) speech and video, they no longer resemble tools but are likely to be interacted with by their users as if they were people - servant, colleague, mentor, teacher, friend or superior being / authority / deity.
I think it would be best if AI users were really clear that they are using a tool However the AIs are often designed to resemble people and seduce the user into feelings of companionship, validation, emotional support etc.
As far as I can tell, many users even in these early days - all of whom have spent most of their lives without AI - don't care much, or at all, about such crisp, clear, delineation between a tool and whatever it is people think and feel these systems are when they are interacting with them.
So it is easy to imagine that in the years to come, many or most people who grow up with AIs as prevalent as TVs and Internet access, from early childhood onwards, would be even less inclined to clearly remember that they are using a tool, of vast complexity, which is likely to have numerous biases and proclivities which are undocumented, under-recognised and potentially at odds with what they would want if they had their head screwed on right. These biases can be built in deliberately, or arise unintentionally due to the entire nature of the people who crafted them. They may also be crafted to fit in with government regulation or corporate goals regarding what ideas and information can and cannot be transacted.
As an example of this sort of effect, see how common it is for people posting something on Twitter/X or whatever to have no citation at all about the source of the material, its context or even who the original author is. Hardly anyone seems to care! They discuss it as if it is real, without a clue as to its origins or authenticity. (This leads to he perverse counter-reaction, now common, where people reject material which is real, and extraordinary, thinking it must be AI slop.) This is especially problematic now that AI can generate deepfake material.
Still, few people seem to care. Eventually some righteous soul asks Grok to tell everyone what the thing is. Grok dutifully responds, as if the validity of its response is not in question - but often gets it completely wrong .
The 280 character limit greatly encourages people to avoid citing the source - and today I read that X posts which contain links to websites are not propagated by The Algorithm as much as those without such links. (I didn't bookmark the tweet since it cited no sources. I believed it. Should I? I could ask Grok . . . ) So this is a potentially (and still, often, actually) rich communication system which has been deliberately engineered to suppress the conveyance of crucial information. The result is not just people being less informed than they otherwise would be, but growing up with the broad expectation of not being well informed AND not caring much about doing any better.
"manufacturing certainty where there should be doubt."
That's why quantum theory has gone nowhere.
It's an echo chamber along with other modern physics.
The critical double slit experiment ASSumes that the detector doesn't influence light. But it does as all detectors use energy however small.
Instead of acknowledging that, they said that the observer changed it via non physical means.
It's become a religion that perpetuates because of sychopancy.
https://youtube.com/playlist?list=PLkdAkAC4ItcFyNFBywN0wiZ45pCnMr-Ay
It's just like medical science obsessed with viruses as the cause of disease not seeing that their methods ignore that the preservatives, antibiotics, and procedures are what cause the damage in their in vitro tests.