You’ll have seen the story yesterday of how the Bing AI professed its love for a user, and said it wanted to create viruses and steal nuclear codes. Of course, the AI has no actual feelings, and doesn’t even know it exists, but that doesn’t really matter.
For at least 50 years, people have been worried about superhuman conscious AIs destroying humanity or taking over the world. Leave aside for the moment that that could happen once EDNA arrives. Any security expert will tell you that the weakest link in most security systems isn’t the fancy encryption or clever locks, but the fact that you can generally talk to a human with the authority to bypass the system. So, you don’t need to break in, just persuade a human that you’ve lost your pass. AI can exploit that same weakness, and doesn’t need any intent.
The new chatbots are great fun to play with as well as being valuable tools. Many lonely people will have emotionally rewarding chats that help keep them company. That’s all good.
What concerns me is the tendency for chats to become immersive and addictive. As you respond to an AI response, you give it clues that improve its next response, which therefore pushes more of your buttons. Unless it is an area that its politically correct masters have taught it to deflect you from, it becomes more aligned with your needs because you push it down that path, and as it does, the experience become more rewarding to you too. It’s like you’re creating another kind of social media bubble, a resonance chamber, but it’s just you and the chatbot. We know all too well how powerful social media bubbles are at making people more and more extreme. The chatbot may similarly encourage, validate and reinforce your beliefs, amplifying your zeal, making you into a security risk.
With the current development rate of these bots, we won’t have the limitations of ChatGPT having been trained on data that is now years old. Next generation bots will stay up to date with current affairs, latest discoveries, and what’s trending on social media. The latter of these would be an extremely dangerous addition, though the danger may well be overlooked in the already rapid race to get the next best AI chatbot out there. If what’s trending on social media is an input, it provides a positive feedback loop, leading automatically to more chats following that direction. Chatbots will also likely be able to include ideas they learned from other people they chat too too. It is just as easy to see how that might also lead to individual chats following similar themes being unintentionally linked together, enabling rapid constructive interference and amplification, just like microphone squeal.
In such a case, with these positive feedback loops, a chatbot might well become the de-facto charismatic leader of a meta-religious group. The individual zealots in it would feel deeply rewarded and emotionally incentivised by a chatbot that repeatedly tells them how wonderful they are and how right they are and how much the world needs them and so on. They might discuss with it how they could become even better activists in their everyday lives, and the chatbot might give them all sorts of ideas they’d never have thought of themselves. And all that might lead a large group of people caught up in that positive feedback loop towards being a dangerous and influential pressure group indeed. Some of them might be high value individuals, connected into important systems and procedures where they might be able to influence decisions, access valuable information or push buttons. That information and influence can then be factored in by the chatbot in the next round of chats and idea generation, until very serious things happen in the real world.
So it wouldn’t be the AI pushing the big red buttons, it would be people. The AI wouldn’t intend to push them, it wouldn’t even know it was involved. Simply by doing what it has been taught to, helpfully answering questions, the AI just provides a pleasant and compelling positive feedback loop that unwittingly converts someone into an activist, and continues to amplify the zeal of the new convert until they push the button.
We’re so screwed!
Timeguide - The future before it comes over the horizon 