The AI revolution I'm afraid of
Lots of "AI Safety" research feels like science fiction to me. But there are really pressing, near-term questions.
It’s been a scary week in national news, and an inexplicably bonkers week in AI-related tech news.
Starting with the weird:
Blake Lemoine, an engineer at Google, has been placed on administrative leave after becoming convinced that LaMDA, the company’s chatbot, had become sentient. This sounds like the start of a sci-fi love story, until you remember that chatbots are essentially just big probabilistic lookup tables that output (very mathematically fancy) combinations of sentences in their training sets, in which case the story looks more like this:
In national news, the January 6th hearings have revealed that an event where we already knew the President, the majority of one party’s members of congress, and a Supreme Court Justice’s wife conspired to overthrow a democratic election was somehow even worse than we realized.
But there’s one aspect of the whole January 6th fiasco that’s really stood out to me, and that’s how easy it’s gotten to spread misinformation, and the outsized role AI will play in this in the coming decade.
Let’s back up for a second.
Human beings are wired to take other people’s ideas seriously. If my friends are operating in good faith, and lots of them tell me that I should be worried about monkeypox getting out of control, I will pay attention. If I don’t yet have an opinion, perhaps I’ll be inclined to agree with them. If I disagree, the mere act of hearing lots of (presumably well-meaning) people disagree with me might move me from “monkeypox isn’t going to spread much more” to “I don’t think monkeypox will spread much, but there’s real and valid disagreement.”
(I picked this example because I know nothing at all about monkeypox or how afraid of it we should be. Please don’t ask me because I do not know.)
This is a good thing for social creatures like humans, because it lets us take advantage of each others’ knowledge. All of us are wrong about some things, and in general “combining everyone’s knowledge and opinions” keeps us humble and (hopefully) prevents us from doubling down on our own ignorance.
But it’s also a bad thing, because this component of human psychology is really easy to hack.
I first noticed this when I started to understand evolution. It wasn’t hard to see that the creationist “science” I had believed wasn’t consistent with the world of nature — a single college course that didn’t even focus on evolution was enough to see how many arguments I was so sure were scientific were in fact little more than word games.
So why were they so convincing to me?
A really, really big part of it is the social aspect: I didn’t think critically about whether they were true, or whether they followed any sort of scientific methodology, because so many people talked about them as if they were! In my church youth group growing up, we regularly watched videos where people dressed up like scientists would say things like “the probability that evolution happened is only ten to the power of negative a million” and the part of me that really, really wanted to trust that people I cared about weren’t showing me straight-up lies rationalized the whole situation as “I guess there must be real scientific debate about this.”
As far as I can tell, this is exactly how January 6th happened. Trump’s strategy after the election was to just straight-up lie about the fact that he had lost. He never produced any evidence of his claims, but his followers quickly came out with “proof”: fabricated stories, misleading videos, and even papers claiming that the probability of Biden winning was “one in a quadrillion”.
Anybody sincerely looking into these claims could have told you this was nonsense (and the courts did exactly that!) But — and I didn’t recognize this enough at the time — it’s just plain hard to admit that somebody you trusted straight-up lied to you. It’s hard to overcome the voice in your head that says “well, lots of people around me are saying this, there must be something to the story.” It’s hard to avoid looking for a middle ground, even when all the facts are on one side of the story.
This is something that the Republican Party — particularly the MAGA wings of it — are exploiting ferociously. When Tucker Carlson says that nobody had guns in the capital or that Trump’s Russia collusion was a “hoax”, he’s not trying to convince you of a fact. He’s adding it to the national conversation, knowing that people who trust him will spread the rumor, until it becomes the sort of thing that “people are saying” and we have to spend time debating.
This is explicitly designed to exploit the “trust other people” instinct: it’s now part of the conversation, and your brain is hardwired to take the “maybe this idea has some merit” or “we need to compromise between this position and the truth” route rather than the “this is false information and I shouldn’t pretend it’s real” route.
I’m not really worried about the sorts of “AI apocalypse” scenarios that are popular in rationalist circles, which I think tend to be somewhat removed from what AI actually is and the sorts of risks it poses.
But I’m really afraid of people misusing technology in predictable ways. If you look at my post from last week, you’ll see that AI-written text isn’t always super consistent. But it does, very quickly, produce large amounts of text that, for the most part, could pass as human-written if you aren’t specifically trying to tell if it’s real or not. Add this to the fact that AI can produce photorealistic faces:
And you have a recipe for large armies of bots producing vaguely plausible misinformation. If you can get this many people to believe an election was fraudulent just by saying so:
what else can you get people to believe if you have an army of AI trained to convincingly lie to individual people? To produce content aimed at their particular biases and triggers? To fake photos designed to pull at their particular heartstrings? How many internet news sites are just going to turn into AI making up stories that they know will rile up their base?
And how easy is it going to be for this to turn violent? We’re already living through a year of increasingly violent anti-gay and anti-trans rhetoric, and we’re already seeing the consequences this has had in the lives of real people. What’s going to happen when AIs are trained to generate this type of content (articles, photographs, tweets) at scale?
I don’t know. Maybe the “rogue superintelligence” people are right, although I remain deeply skeptical. But it seems to me like we’re at a point where AI is going to be weaponized without anything like “superintelligence”, and none of the proposals I’ve read to deal with it seem like we’re at all prepared for the coming deluge of misinformation (and ultimately violence.)
I’m really worried about this, but I don’t want to end on a purely cynical note, so here are some things that might help off the top of my head:
Teach kids critical thinking.
This includes things like “critical race theory”. There’s a specific right-wing misinformation technique that takes people’s natural inclinations to racism or sexism or homophobia, and teaches them to lean into these feelings and use them to resist changing, rather than doing the hard work of repenting. People who aren’t aware of this dynamic inevitably fall into it.
If you’re funding AI longtermism, please please please also fund AI short-term-ism. You can’t do your long-term planning if the society you’ve funded it in collapses in violence.
Pastors, stand up to misinformation. If your religion rests on Truth, you shouldn’t be afraid of truth.
We need better research on how people break out of misinformation spirals. I suspect the literature on cult deprogramming might be relevant, but this isn’t my field so I don’t have a good sense of it.
If you’re religious, pray! But also do things. Pray the kind of prayer that brings you to action rather than apathy.