The most obvious consequence of this is the simpish behavior LLMs exhibit. Ask if men are more manipulative than women and it twists itself in a 'Women are Wonderful' shaped knot.
Only watched the video so far, but it was another good one. Dare I say even a bit cathartic.
Interestingly, but probably not surprisingly, this phenomenon of "tool-assisted mental atrophy" has been relevant to the programming space for a while. Even before we started farming the actual code writing out to Machine Learning systems, we have been abstracting ourselves into forgetting our own craft with the very languages we use to program. There are lots of programmers who learned on languages like Python and JavaScript who are able to get *a* solution to problems, but struggle to figure out *good* solutions to problems because they never trued to figure out what those "easy to use" abstractions were doing under the hood and are thus divorced from the "reality" of what they're actually telling the computer to do.
But even then, since those are deterministic abstractions, it's possible to figure what they are doing and why, then use them in a mentally "full" way that does stretch your mental muscles while still allowing you to work more efficiently and ultimately go farther than you would have without the tool.
This is the good path man can walk with his tools. And I liken it loosely to how cars may take away the need to walk, but you won't find a fat person in a Formula 1 car.
Our current "AI" systems, on the other hand, are built to intentionally obscure how they work to create an illusion that they are something they are not. If people used them for what they truly are, statistical analysis machines, they would find their healthy place in society just fine.
But no, people are putting a shoddy flesh mask on them and parading them around like they're their the next king to be in the "evolutionary race." So, now those systems can't be "mastered" like our other tools, because they are being used to solve the wrong problems and thus will never give the kind of predictable behavior that allows a person to *extend* his own mastery *through* the tool. Instead you must fight the tool or let the toolmaker decide for you what it will do...
Anyway...
I'm feeling now that I really oughta finish my article going into the physics of why machines *can't* have souls and *can't* "think."
I remember hearing a programmer tell me a few years ago that people were starting to forget programming languages. This had come up in a conversation about Chinese and Japanese people forgetting how to write characters and English-speakers forgetting how to spell. The point was that if the language is forgotten, how much does the person lose?
I've heard people talk about using AI for statistical analysis, machine problems. I have an AI engine in my video editing software that creates smoother effects by analyzing video data. But, as you aptly said, giving it a flesh mask is where things become troubling. Using AI for data retrieval (like a google search) is all well and good until the AI starts trying to talk to you like a librarian. Several of the above studies showed that if the AI was "trusted" then it could lead its users around by the nose. Or if the AI said the same thing over and over again the user would believe it. There is an element of human use there, of course. But ultimately, why does the internet search tool need to speak to me in sweet nothings and tell me how interesting I am?
I would be interested to read that article when you finish it.
I need to read this article in depth, but I wonder if the conclusion, the logical one, is that language itself is under a type of threat, but it remains to be seen if humans, language bearers, will defend and nurture it in a way that transforms it to something more human--transformative rather than utterly destructive. I can't quite get at what I'm actually trying to say or think on this. I really enjoy these comments and articles. To me, this is social media at its best!
It's widely accepted in China that many people can recognize, pronounce, type, and tell you what a character means but cannot write it by hand. 提笔忘字. They type the pinyin and select characters from a list. That's how I learned Mandarin. I know some teens here in US who cannot write or read cursive... :(
What really surprises me is how many people personify LLMs and think that LLMs think and have cognitive processes. It's a word predictor. It doesn't have a logical unit, it cannot reason.
And on topic of cognitive atrophy, even spellcheck has caused it from what I've seen. People always mixed up its/it's, they're/their/there, then/than, affect/effect, but these days I see lose/loose, border/boarder, paid/payed, viscous/vicious. These only started popping up in my experience in last 15 years, and I think it's because people blindly trust spellcheck.
I started thinking that, too. In fact, I noticed that since I had the checker turned on in my word processor (so the misspelled word was underlined), my own spelling started to suffer. So now I have that turned off and run a spell check when I'm done. Because of that I noticed that my awareness of whether or not a word was correct or spelled correctly started to go up. My typing accuracy also improved. We don't realize how much we lean on the tool when it's there until the muscles we previously had become so weak that we find ourselves NEEDING the tool.
I had a long conversation last Sunday night with a good friend. We talked about A.I. Neither of us could come up with anything that A.I. has actually made better in modern society. We both agreed it's just literally made everything worse. :-| This too shall pass.
LLMs specifically have no use so far. AI in general has use. For example neural nets are used by meat processing machines to grade carcasses (https://marel.com/en/products/auravcs/). Difference is there neural nets are trained for a single specific purpose, and because they're not huge, they don't need a whole lot of power but run locally.
You have similar usages in machining also, neural nets trained on data for a specific purposes, usually some form of quality control.
Yeah I mean, of course there are areas in which AI (machine learning) is totally useful. Take the analysis of medical scans, to help diagnose conditions. That's super useful and totally should be used.
I think me and my friend were more discussing the impact of AI on culture and society. AI has lots of small use cases where it's good. I don't even mind the use of LLMs as research assistants or advanced search engines. I think we can all get behind the idea of an AI like the ships computer on the starship enterprise. A helpful AI assistant that can search archives for you, instead of you doing it manually.
But imagine if in Star Trek, the AI had replaced the human crew of the enterprise, and turned the crew into slaves that serve the AI. That's basically the Borg, and the way some modern companies and individuals (Sam Altman, cough cough) talk about and are attempting to use AI, you'd think there ultimate goal IS the Borg.
That first study basically says that people who rely on LLM’s are happy and dumb, and I think that betrays exactly why these clankers are being pushed.
Your eloquently articulated assertions leave me disturbed as usual, though I think in a mostly beneficial manner. I have reflected on many of these same ideas since reading/watching your articles and videos on female “literature”, the decline in literacy, and fan fiction. Most especially I have looked into Postman’s book and the many issues with the rise of the entertainment regime, though I don’t recall if you have mentioned him by name before now. I appreciate the link to a free archive; thank you.
I especially agree with your points on television and social media. I may not have been born into financial abundance, but I am deeply grateful, and even more so in light of my recently growing awareness of the atrophy of reason in modern society, that my parents restricted me from both aside from sparing occasions under supervision until my adulthood. Of course, I have fallen into heavy social media use, but I can only imagine how much worse it would be if I had grown up using Tik Tok and YouTube Kids from the age of four.
Still, I have difficulty forcing myself to embrace difficulty and struggle, even as I can feel my own ability to articulate rational thought decline because of it. I graduated high school with the English departmental award (not particularly impressive in a graduating class of perhaps fifty students, but an achievement nonetheless), and I don’t think my skill in writing has improved significantly in the several years since. I completely failed at university, due in part to my ongoing struggle with YouTube, and even now I continue to watch far more videos than I have any reason to.
I have made some progress in cutting social media down, but even Substack is a social platform as well. Not only in the tab of the exact same kind of scrolling short videos as Tik Tok, but the engagement it promotes with essays is fundamentally identical to Tik Tok, Shorts, and Reels. I make an effort in engaging more thoughtfully with fewer essays than the Substack homepage is designed to promote, but I resent that doing so is intentionally designed against.
Regardless, I appreciate your opinions, however they might unsettle me. They have forced me to realise the harm I am causing myself with my own habits around social media, and even around embracing difficulty and struggle more generally, and have strongly motivated me to halt and reverse them.
I’m honestly surprised there was next to no difference at all between llm and llm+notes. You would think that there would be something even if it was still inferior to notes alone. Makes me wonder if they werent just writing down what the llm told them.
Thank you for taking the time to compile this article.
Some of the results from the studies mentioned mirror a few observations I've made in 2025. I've read quite a bit on the subject, so I'm not claiming my ideas are original, even though I've thought things through independently.
For instance, I'm convinced AI can be great for those tasks where as adults we don't have expertise, such as creating short films or music, but if we're technical experts (ie already know how to play an instrument or create cgi shorts, if such a thing exists), it degrades our competencies. Likewise summaries are great only on those subjects we're not inclined to pursue more than occasionally, while their use by people whose work involves analyzing leads to missing details or key information. The issue is further compounded down the line as entire patterns are missed. Here on ss I've noticed more than one writer get very defensive if their use of LMs in the process is called out.
All in all I'm satisfied those younger than me won't be stealing jobs because they're setting themselves up for failure by using LMs. It goes without saying all humans are lazy and enjoy taking shortcuts.
Thanks again for writing and compiling this list of articles.
Going to read that 2003 study as soon as I find the time. I already know that people are delusional, but really want to know just how bad it is. Hardly anyone bases their behavior on hard evidence from a large dataset. But, to be fair, random violence is extremely rare, yet 2 people I know were killed completely randomly in the past few years.
Please consider including The Fordham Experiment in the Television section.
To stay true to yourself and keep your own sacred mind pure is the only way to create without said creation being spoiled – like when cooking food, if you add a rotted ingredient, not only will it leave a bad taste in your mouth, but it could kill you. And in the literary sense, kill the progress you have been making, not only to improve your work, but to improve your mind. Never let what is fake enter and contaminate your world – your world is exclusively your own – be sure and keep it that way!
"Tonight AI came up. I recently joined an online book club facilitated by a
person named Hilary Layne. Meanwhile, I've been exploring other topics and books she's written on, namely. . .
The idea for me is that humans should not use AI to do their thinking for them,
but use as a tool. I'm using it to help me do some research on some writing topics (my cancer experience, as you know), but I'm not letting it write for me!"
And:
"I held off on using it because I want to maintain my brain skills as I become an old man who has had an experience much like the man in the Job narrative. I see a massive Red Flag when people without critical thinking skills turn to AI as if it is a trusted advisor. Goes without saying that AI is not human. AI has no intrinsic moral system. AI is presented as human. Okay then, 'Alexa, "Turn off Gemini.'"
This article is extraordinarily well written. I love that you have your videos in written form as well. You have become one of my go to channels.
When it comes to AI I have noticed that my friends that use it for fiction writing have issues formulating their sentences otherwise and putting their thoughts in order. Their conversational skills have not suffered much, but their vocabulary has become more limited.
Now it is easier to notice the AI writing pattern of two negatives and "just/only this". As well as the favorite words the LLMs use. Since they solve written text as an equations they will follow these patterns. And it is getting annoying bordering on infuriating.
I hope we escape from this corner AI is trying to put us in.
Edit:
I remember that while teaching (English as a second language) for a bit I noticed the students didn't want to write down the new words. Instead they took photos of the whiteboard. I let them do it and the next lesson they could not spell or use them in the exercises. The never actually practiced them and similar to T9 they only thought they new the words. After a short battle they started to write them down, but neglected the homework and practice of repeatedly writing the words so they could memorize the spelling and meaning. Technology tends to make us lazy and we lose skills or simply never develop them.
“This illuminating tidbit reveals that AI’s number one source for information is Reddit. Number two? Wikipedia”
Gods above how horrifying.
The most obvious consequence of this is the simpish behavior LLMs exhibit. Ask if men are more manipulative than women and it twists itself in a 'Women are Wonderful' shaped knot.
Don't mince words, Hilary. Tell us how you really feel. 🤣
Only watched the video so far, but it was another good one. Dare I say even a bit cathartic.
Interestingly, but probably not surprisingly, this phenomenon of "tool-assisted mental atrophy" has been relevant to the programming space for a while. Even before we started farming the actual code writing out to Machine Learning systems, we have been abstracting ourselves into forgetting our own craft with the very languages we use to program. There are lots of programmers who learned on languages like Python and JavaScript who are able to get *a* solution to problems, but struggle to figure out *good* solutions to problems because they never trued to figure out what those "easy to use" abstractions were doing under the hood and are thus divorced from the "reality" of what they're actually telling the computer to do.
But even then, since those are deterministic abstractions, it's possible to figure what they are doing and why, then use them in a mentally "full" way that does stretch your mental muscles while still allowing you to work more efficiently and ultimately go farther than you would have without the tool.
This is the good path man can walk with his tools. And I liken it loosely to how cars may take away the need to walk, but you won't find a fat person in a Formula 1 car.
Our current "AI" systems, on the other hand, are built to intentionally obscure how they work to create an illusion that they are something they are not. If people used them for what they truly are, statistical analysis machines, they would find their healthy place in society just fine.
But no, people are putting a shoddy flesh mask on them and parading them around like they're their the next king to be in the "evolutionary race." So, now those systems can't be "mastered" like our other tools, because they are being used to solve the wrong problems and thus will never give the kind of predictable behavior that allows a person to *extend* his own mastery *through* the tool. Instead you must fight the tool or let the toolmaker decide for you what it will do...
Anyway...
I'm feeling now that I really oughta finish my article going into the physics of why machines *can't* have souls and *can't* "think."
I remember hearing a programmer tell me a few years ago that people were starting to forget programming languages. This had come up in a conversation about Chinese and Japanese people forgetting how to write characters and English-speakers forgetting how to spell. The point was that if the language is forgotten, how much does the person lose?
I've heard people talk about using AI for statistical analysis, machine problems. I have an AI engine in my video editing software that creates smoother effects by analyzing video data. But, as you aptly said, giving it a flesh mask is where things become troubling. Using AI for data retrieval (like a google search) is all well and good until the AI starts trying to talk to you like a librarian. Several of the above studies showed that if the AI was "trusted" then it could lead its users around by the nose. Or if the AI said the same thing over and over again the user would believe it. There is an element of human use there, of course. But ultimately, why does the internet search tool need to speak to me in sweet nothings and tell me how interesting I am?
I would be interested to read that article when you finish it.
Just remembered this. I did end up finishing the article:
https://www.arscorvi.com/p/on-ai-the-toaster-will-never-have
Don't think it's a bulletproof argument, but I think it gets a good start at the line of reasoning I'm exploring.
I need to read this article in depth, but I wonder if the conclusion, the logical one, is that language itself is under a type of threat, but it remains to be seen if humans, language bearers, will defend and nurture it in a way that transforms it to something more human--transformative rather than utterly destructive. I can't quite get at what I'm actually trying to say or think on this. I really enjoy these comments and articles. To me, this is social media at its best!
It's widely accepted in China that many people can recognize, pronounce, type, and tell you what a character means but cannot write it by hand. 提笔忘字. They type the pinyin and select characters from a list. That's how I learned Mandarin. I know some teens here in US who cannot write or read cursive... :(
Thank you for sharing. I look forward to reading the article.
What really surprises me is how many people personify LLMs and think that LLMs think and have cognitive processes. It's a word predictor. It doesn't have a logical unit, it cannot reason.
And on topic of cognitive atrophy, even spellcheck has caused it from what I've seen. People always mixed up its/it's, they're/their/there, then/than, affect/effect, but these days I see lose/loose, border/boarder, paid/payed, viscous/vicious. These only started popping up in my experience in last 15 years, and I think it's because people blindly trust spellcheck.
I started thinking that, too. In fact, I noticed that since I had the checker turned on in my word processor (so the misspelled word was underlined), my own spelling started to suffer. So now I have that turned off and run a spell check when I'm done. Because of that I noticed that my awareness of whether or not a word was correct or spelled correctly started to go up. My typing accuracy also improved. We don't realize how much we lean on the tool when it's there until the muscles we previously had become so weak that we find ourselves NEEDING the tool.
I had a long conversation last Sunday night with a good friend. We talked about A.I. Neither of us could come up with anything that A.I. has actually made better in modern society. We both agreed it's just literally made everything worse. :-| This too shall pass.
LLMs specifically have no use so far. AI in general has use. For example neural nets are used by meat processing machines to grade carcasses (https://marel.com/en/products/auravcs/). Difference is there neural nets are trained for a single specific purpose, and because they're not huge, they don't need a whole lot of power but run locally.
You have similar usages in machining also, neural nets trained on data for a specific purposes, usually some form of quality control.
Hey Katriga! Thanks for your input.
Yeah I mean, of course there are areas in which AI (machine learning) is totally useful. Take the analysis of medical scans, to help diagnose conditions. That's super useful and totally should be used.
I think me and my friend were more discussing the impact of AI on culture and society. AI has lots of small use cases where it's good. I don't even mind the use of LLMs as research assistants or advanced search engines. I think we can all get behind the idea of an AI like the ships computer on the starship enterprise. A helpful AI assistant that can search archives for you, instead of you doing it manually.
But imagine if in Star Trek, the AI had replaced the human crew of the enterprise, and turned the crew into slaves that serve the AI. That's basically the Borg, and the way some modern companies and individuals (Sam Altman, cough cough) talk about and are attempting to use AI, you'd think there ultimate goal IS the Borg.
That can't be a good goal for society. :P
I'm just being pedantic about the term. People are now conflating AI with LLM, but AI has a wider meaning and has legitimate uses.
I agree that LLMs are garbage and have no use case.
That first study basically says that people who rely on LLM’s are happy and dumb, and I think that betrays exactly why these clankers are being pushed.
Your eloquently articulated assertions leave me disturbed as usual, though I think in a mostly beneficial manner. I have reflected on many of these same ideas since reading/watching your articles and videos on female “literature”, the decline in literacy, and fan fiction. Most especially I have looked into Postman’s book and the many issues with the rise of the entertainment regime, though I don’t recall if you have mentioned him by name before now. I appreciate the link to a free archive; thank you.
I especially agree with your points on television and social media. I may not have been born into financial abundance, but I am deeply grateful, and even more so in light of my recently growing awareness of the atrophy of reason in modern society, that my parents restricted me from both aside from sparing occasions under supervision until my adulthood. Of course, I have fallen into heavy social media use, but I can only imagine how much worse it would be if I had grown up using Tik Tok and YouTube Kids from the age of four.
Still, I have difficulty forcing myself to embrace difficulty and struggle, even as I can feel my own ability to articulate rational thought decline because of it. I graduated high school with the English departmental award (not particularly impressive in a graduating class of perhaps fifty students, but an achievement nonetheless), and I don’t think my skill in writing has improved significantly in the several years since. I completely failed at university, due in part to my ongoing struggle with YouTube, and even now I continue to watch far more videos than I have any reason to.
I have made some progress in cutting social media down, but even Substack is a social platform as well. Not only in the tab of the exact same kind of scrolling short videos as Tik Tok, but the engagement it promotes with essays is fundamentally identical to Tik Tok, Shorts, and Reels. I make an effort in engaging more thoughtfully with fewer essays than the Substack homepage is designed to promote, but I resent that doing so is intentionally designed against.
Regardless, I appreciate your opinions, however they might unsettle me. They have forced me to realise the harm I am causing myself with my own habits around social media, and even around embracing difficulty and struggle more generally, and have strongly motivated me to halt and reverse them.
I’m honestly surprised there was next to no difference at all between llm and llm+notes. You would think that there would be something even if it was still inferior to notes alone. Makes me wonder if they werent just writing down what the llm told them.
It makes me think that if it's there, the mind will automatically defer to it to SOME degree.
Is this on the list?
https://arxiv.org/abs/2506.08872
Yes.
Well, I’ll stop using any Ai for anything.
I found it dumb anyways.
Thank you for taking the time to compile this article.
Some of the results from the studies mentioned mirror a few observations I've made in 2025. I've read quite a bit on the subject, so I'm not claiming my ideas are original, even though I've thought things through independently.
For instance, I'm convinced AI can be great for those tasks where as adults we don't have expertise, such as creating short films or music, but if we're technical experts (ie already know how to play an instrument or create cgi shorts, if such a thing exists), it degrades our competencies. Likewise summaries are great only on those subjects we're not inclined to pursue more than occasionally, while their use by people whose work involves analyzing leads to missing details or key information. The issue is further compounded down the line as entire patterns are missed. Here on ss I've noticed more than one writer get very defensive if their use of LMs in the process is called out.
All in all I'm satisfied those younger than me won't be stealing jobs because they're setting themselves up for failure by using LMs. It goes without saying all humans are lazy and enjoy taking shortcuts.
Thanks again for writing and compiling this list of articles.
Going to read that 2003 study as soon as I find the time. I already know that people are delusional, but really want to know just how bad it is. Hardly anyone bases their behavior on hard evidence from a large dataset. But, to be fair, random violence is extremely rare, yet 2 people I know were killed completely randomly in the past few years.
Please consider including The Fordham Experiment in the Television section.
https://en.wikipedia.org/wiki/Fordham_Experiment
When watching a screen that emits light, rather than being projected (TV/phone vs theater), the participants reported:
Comments on a feeling of a loss of sense of time rose from 6% to 40%
Comments on a sense of total involvement rose from 15% to 64%
Comments on a sense of total emotional involvement rose from 12% to 48%
To stay true to yourself and keep your own sacred mind pure is the only way to create without said creation being spoiled – like when cooking food, if you add a rotted ingredient, not only will it leave a bad taste in your mouth, but it could kill you. And in the literary sense, kill the progress you have been making, not only to improve your work, but to improve your mind. Never let what is fake enter and contaminate your world – your world is exclusively your own – be sure and keep it that way!
I'll reshare here:
To a dinner group my wife and I attend:
"Tonight AI came up. I recently joined an online book club facilitated by a
person named Hilary Layne. Meanwhile, I've been exploring other topics and books she's written on, namely. . .
The idea for me is that humans should not use AI to do their thinking for them,
but use as a tool. I'm using it to help me do some research on some writing topics (my cancer experience, as you know), but I'm not letting it write for me!"
And:
"I held off on using it because I want to maintain my brain skills as I become an old man who has had an experience much like the man in the Job narrative. I see a massive Red Flag when people without critical thinking skills turn to AI as if it is a trusted advisor. Goes without saying that AI is not human. AI has no intrinsic moral system. AI is presented as human. Okay then, 'Alexa, "Turn off Gemini.'"
This article is extraordinarily well written. I love that you have your videos in written form as well. You have become one of my go to channels.
When it comes to AI I have noticed that my friends that use it for fiction writing have issues formulating their sentences otherwise and putting their thoughts in order. Their conversational skills have not suffered much, but their vocabulary has become more limited.
Now it is easier to notice the AI writing pattern of two negatives and "just/only this". As well as the favorite words the LLMs use. Since they solve written text as an equations they will follow these patterns. And it is getting annoying bordering on infuriating.
I hope we escape from this corner AI is trying to put us in.
Edit:
I remember that while teaching (English as a second language) for a bit I noticed the students didn't want to write down the new words. Instead they took photos of the whiteboard. I let them do it and the next lesson they could not spell or use them in the exercises. The never actually practiced them and similar to T9 they only thought they new the words. After a short battle they started to write them down, but neglected the homework and practice of repeatedly writing the words so they could memorize the spelling and meaning. Technology tends to make us lazy and we lose skills or simply never develop them.
Excellent work! Just a minuscule correction, Neil Postman’s Amusing Ourselves to Death was originally published in 1985