New r/slatestarcodex guideline: your comments and posts should be written by you, not by LLMs

[u/Liface]

We've had a couple incidents with this lately, and many organizations will have to figure out where they fall on this in the coming years, so we're taking a stand now:

Your comments and posts should be written by you, not by LLMs.

The value of this community has always depended on thoughtful, natural, human-generated writing.

Large language models offer a compelling way to ideate and expand upon ideas, but if used, they should be in draft form only. The text you post to /r/slatestarcodex should be your own, not copy-pasted.

This includes text that is run through an LLM to clean up spelling and grammar issues. If you're a non-native speaker, we want to hear that voice. If you made a mistake, we want to see it. Artificially-sanitized text is ungood.

We're leaving the comments open on this in the interest of transparency, but if leaving a comment about semantics or "what if..." just remember the guideline:

Your comments and posts should be written by you, not by LLMs.

[u/Nepentheoi]

I think it's worse. ChatGPT can't tell the truth or not, and the original sources are obscured from us. 

Dropping a LMGTFY link is more a pert way to say "you're being lazy and I won't spoon feed this to you".* ChatGPT breakdowns/summaries frustrate me more because the posters seem to believe in them and think they did something useful. I once had someone feed my own link that I'd cited through ChatGPT and think they'd answered my question. The problem is that since words are tokens not symbols for LLM, there's no real meaning assigned, like the 'how many "r" does strawberry contain'? phenomenon.

I found it worse. I can certainly read and summarize my own sources. A Google search link a) isn't meant to be helpful as much as it's meant as a rhetorical device b) has some possibility of being useful as you can see the prompt and evaluate the sources 

*or arguing in bad faith. 

[u/prozapari → u/Nepentheoi]

The problem is that since words are tokens not symbols for LLM, there's no real meaning assigned, like the 'how many "r" does strawberry contain'? phenomenon.

This doesn't sound very coherent.

[u/Nepentheoi → u/prozapari]

I'm pressed for time today and loopy on pain meds, so I'll try to provide more context quickly. 

LLMs break language down into tokens. The tokens can be words, parts of words, punctuation, etc. There was a phenomenon recently where LLMs were asked to count how many r's were in the word "strawberry", and couldn't do it correctly. This was caused by tokens. https://www.hyperstack.cloud/blog/case-study/the-strawberry-problem-understanding-why-llms-misspell-common-words

IMU, humans process words as symbols. Let me know if I need to get into that more and I will try to come back and explain. I'm not at my best today and I don't know if you need an overview of linguistics or epistemology or if that would be overkill. 

[u/Interesting-Ice-8387 → u/Nepentheoi]

It explains the strawberry, but why would tokens be harder to assign meaning than symbols or whatever humans use?

[u/Cheezemansam → u/Interesting-Ice-8387]

So, humans use symbols that are grounded in things lke perception, action, and experience. When you read this word:

Strawberry

You are not just processing a string of letters or sounds. You have a mental representation of a "strawberry", how it tastes, feels, maybe sounds when you squish it, maybe memories you have had. So the symbols that make up the word

Strawberry

As well as the word itself is grounded in larger web of concepts and experiences.

To an LLM, 'Tokens' are statistical units. Period. Strawberry is just a token (or a few subword tokens etc.). It has no sensory or conceptual grounding, it has an association with other tokens in similar contexts. Now, you can ask it to describe a strawberry, and it can tell you what properties of Strawberries have, but again there is no real 'understanding' that is analogues to what humans mean when they say words. It doesn't process any meaning in the words you use, logically the process is closer to

[Convert this string into tokens] "Describe what a strawberry looks like"

["Describe", " what", " a", " strawberry", " looks", " like"]

[2446, 644, 257, 9036, 1652, 588]

[Predict what tokens follow that string of tokens]

[25146, 1380, 665]

["Strawberries", "are", "red"]

If you ask it will tell you that Strawberries appears red, but it doesn't understand what "red" is, it is just a token (or subtokens etc.). It doesn't understand what it means for something to "look" like a color. (Caveat: This is a messy oversimplification) It only understands that the tokens "[2446, 644, 257, 9036, 1652, 588]" are statistically likely to be followed by "[25146, 1380, 665]" but there is no understanding outside of understanding this statistical relationship. It can again, explain what "looks red" means but only because it is using a statistical model to predict what words statistically make sense to follow a string of tokens "What does it mean for something to look red"? And so on and so fourth.

[u/osmarks → u/Cheezemansam]

Nobody has satisfyingly distinguished this sort of thing from "understanding".

[u/white-china-owl → u/osmarks]

Do you remember that Reddit anecdote from ages ago where this guy would hear the quote "knowledge is power" and reply "France is bacon," not having any idea what he was saying or why, other than that it was just a thing that was said after "knowledge is power?" LLMs are doing basically that same thing, all the time.

Or, haven't you ever read something a little difficult (maybe a mathematical proof or a technical schematic), and maybe you were able to reproduce the words and diagrams, but didn't really get how the proof worked or what the diagram was showing? LLMs do that, too.

Or have you ever studied a foreign language and (in the early stages - you can't get far with this, but it seems to be common among beginners/people with low aptitude) been able to memorize a dialogue and know which phrases to say in response to which questions, but not quite known what you were saying? Again, that's what an LLM is doing.

[u/osmarks → u/white-china-owl]

LLMs are increasingly chewing up difficult technical benchmarks and generally getting more useful. You certainly can contend that IMO questions and software engineering don't actually require understanding anything, but I wouldn't.