> My brain knows where it's going to go by the end of the sentence as well
How would you know that?
Sincere question, because to me it feels like my brain is improvising word by word when typing out this sentence. I often delete and retype it until it feels right, but in the process of typing a single sentence, I'm just chaining words one after another the way they feel right.
In other words, my brain doesn't exactly know the sentence beforehand - it improvises by chaining the words, while applying a fitness function F(sentence) -> feeling that tells it whether it corresponds to what I wanted to say or not.
In my case it's because my internal dialogue is saying the sentence before I get to the end of it. I usually have the entire sentence in my inner dialogue before I even start typing. Will I edit during typing? Sure, but I have a first version in my head before I start.
> I usually have the entire sentence in my inner dialogue before I even start typing
Interesting. Perhaps the question then becomes, does your inner dialogue simply chain the words one after another, or does it come up with sentences as whole?
I have a friend who listens carefully and starts to mouth and even mumble his brain’s predictions for the next words I am going to say. So, I fuck with him by unexpectedly (even to myself) changing my words (or even the sentence) half way through when I catch him doing it. Predictably, I don’t like being predictable.
The fact that ChatGPT does so well is perhaps a sign we do somewhat generate sentences on the fly. Obviously we mostly listen and read sequentially.
One could test it by using an external random bell to signal you should try and make a significant sentence change e.g. from English to Spanish. How much practice would it take?
From 1984: In the middle of Hate Week, the speaker is halfway through a sentence about hating Eurasia, he is given a note, and he continues the sentence except now Oceania is not, and has never been, at war with Eurasia, and it is Eastasia that is now hated.
It's impossible to form meaningful sentences without having a model of the sequence of the words, this is as true for a human as for a markov chain, but that doesn't mean it then follows that human sentences are just sophisticated markov chains, the nature of sentences means that they must be a sequence of words regardless of the implementation details behind the hood of the sentence creator.
Why would it? It would make more sense to start with concepts ("dog", "brown"), and then pull it into words ("That dog is brown") - given that we're trying to convey a thought that seems to be far more likely than us chaining things - we start with the intent to communicate something, the markov chain does not.
Your eyes are telling you what color that shade of red is on the wall. But if you put it next to a different color, you’ll suddenly get a different answer from your eyes.
I think if our eyes can deceive us at a fundamental level, it’s arrogant to think we aren’t deceived by our thoughts.
Well then those who think they are coming up the sentence one word at a time are equally as easy to fool, no? They could have the whole sentence figured out and their attention is only coming to a single word at a time, possibly revising what's in their field of mental view if they come up with a better word. That's the fun with these sorts of doubts, if we assume nothing, then we can form an argument for anything.
> Sincere question, because to me it feels like my brain is improvising word by word when typing out this sentence. I often delete and retype it until it feels right, but in the process of typing a single sentence, I'm just chaining words one after another the way they feel right.
That’s your conscious experience, but it doesn’t necessarily match what your subconscious mind has actually been doing. I’d hazard a guess that it’s thinking several moves ahead, like a good chess player. What you end up being consciously aware of is probably just the end product of multiple competing models - including ones that want to stop writing altogether and go do something else.
I don’t generally think in individual words when writing. When I want to write a sentence, the first thing I think about is the idea that I want to convey. Then clusters of words start to come into my mind, as ways of expressing that idea, and I select and organize those fragments into a sentence, sometimes fine tuning words to avoid undesirable connotations or emphasize desired ones. Finally I glue the tuned fragments together with syntax and connecting words. Throughout the process I’m constantly iterating.
I don’t think that what ChatGPT does is anything remotely like what I do to communicate. But maybe I’m weird.
I definitely just start saying things or hammering out words and then figuring out midway that I need to reword the paragraph at some point or at least choose to sacrifice legibility for my own laziness. It reminds me of improvised music or comedy. You just gotta roll with it and build a structure out of what you’ve already said. Like 60s amphetamine beat poetry epics.
> I'm just chaining words one after another the way they feel right.
I think we know where we are going in a conceptual sense, the words start feeling right because they are taking us to that destination, or not.
If I leave a sentence in the middle for some reason, when I return I often have zero idea how to finish the sentence or even what the sentence fragment means.
Here's a theory. Thoughts exist on certain time scales. Everything about perception exists on certain time scales. The brain wave frequencies we use to represent things is directly related to the time scale of the thing it represents. The end of the sentence which you are trying to reach departed on its internal journey though the brain around the same time as whatever triggered the thought. It just so happened to take a more circuitous route, delaying it in preference of other words until it was ready. The longer the distance of logical relations between parts of the sentence, the larger the relevant time scale, the lower the relevant brain waves used to represent it.
Its like a juggling act. The ball with the conclusion is thrown up highest, a bunch of other balls are thrown up in between, and they should all start arriving back in your hands, in the correct order, one at a time, without having known the exact sequence to expect when they were first thrown.
Sometime the juggler misjudges and the train of thought is scrambled and lost.
> Its like a juggling act. The ball with the conclusion is thrown up highest, a bunch of other balls are thrown up in between, and they should all start arriving back in your hands, in the correct order, one at a time, without having known the exact sequence to expect when they were first thrown.
"experience tranquility"
--zenyatta overwatch
Yeah, that metaphor works. ;) As an extremely ADHD person, every thought comes with extra bonus thoughts (and parentheticals!), and the trick is knowing when to introduce each supporting point without re-introducing concepts needlessly but also try to have my bizarre brain make sense. Internet arguing and trying to preemptively address counterarguments with supporting points has seriously broken my brain and it leads to very longwinded posts. Keeping it short and coherent is specifically something I really have to work at because I love to write and people don't want to read a novel every comment. It's a matter of effective communication though.
Personally the description of the transformer as "writes words and then edits the output as a unit once the words are complete" really describes my writing at both a sentence and paragraph level. I'll go back and edit a comment a ton to try and tune it and clarify exact meaning/nuance with the most precise language I can.
A ton of people read my comments and are like "did an AI write this!?" and yeah only the finest biological neural net.
Another friend described it as "needing to slow his brain down" and perhaps a similar metaphor would be a database pivot - taking sparse facts and grouping them into a clustered dense representation as an argument. It's an expensive operation especially if there's more there than you thought.
How would you know that?
Sincere question, because to me it feels like my brain is improvising word by word when typing out this sentence. I often delete and retype it until it feels right, but in the process of typing a single sentence, I'm just chaining words one after another the way they feel right.
In other words, my brain doesn't exactly know the sentence beforehand - it improvises by chaining the words, while applying a fitness function F(sentence) -> feeling that tells it whether it corresponds to what I wanted to say or not.