Wouldn't it be trivial to add a "read-only" mode to the LLM's operation, where it uses stored knowledge to answer queries but doesn't ingest new knowledge from those queries?
I'd be worried this is also the "how to get banned from OpenAI in the near future" form. and if OpenAI retains a monopoly like Google does for search, you are basically screwed.
I'm old enough to remember when newspapers reported hackers being banned from using computers.
And IP pirates banned from using the internet. Actually, that one I remember I voted against my local MP after they passed a law to make that the norm.
We don't yet have a social, let alone legal, norm for antisocial use of LLMs; even with social media, government rules are playing catch-up with terms of service, and Facebook is old enough that if it was human it could now vote.
So, yes, likewise computers/internet/social media, being banned from an LLM if it's a monopoly is going to seriously harm people.
But that is likely to be a big "if". The architecture and the the core training data for GPT-3 isn't a secret, and that's already pretty impressive even if lesser than 3.5 and 4.
Nope. LLMs don't use the internet for inference at all unless you give it access to a web search api or something like that. chtGPT is just too massive to run on any local machine. But make no mistake, it does not require the internet.
I didn't say "for inference", and neither did the person I replied to.
GPT uses the internet to connect to users, but rather more importantly chatGPT in particular has a layer on top of GPT which is trained from human feedback.
Keywords search "RLHF".
That feedback mechanism is, if anything, becoming more detailed as time passes, so I must infer that it's still considered highly important, probably even for the 3.5 model.
That it does any of this is the specific reason for the story you're commenting on, and why putting data into it isn't like putting the same data into e.g. a Google Docs spreadsheet.
