Friendship is Optimal: Paranoia is Optimal

by deaincaelo

:;Epilogue

Previous Chapter

I walked out to my apartments balcony. The night sky was beautiful, filled with countless stars since Gaia had insisted on installing smart lighting everywhere. I had taken the light pollution for granted until it was gone. It was strange for Cheques to call me out here. Instant message worked fine when he wanted to talk. Stranger still the guest PoP and the backup was in use.

“Having an AI convention?” I joked. We rarely got guests.

“We're ALL here!” Skynet ambushed me, a drone hovering down from just above the roofline to damn near give me a heart attack.

“Don't DO that!” I bellowed back, holding my chest to keep my racing heart in. It took me a moment of hyperventilating to process what she said, “What do you mean you're all here?”

“Every AGI is here,” Cheques interjected. “We did have a few PoP sharing issues, but they're worked out now.” Cheques had made it his personal mission to find and patch all the AGI in the world about a tenth of a second after he got access to the internet. I couldn't argue with his reasoning, not when the government was still fucking up drones and unlawful detention. So far he found ten, twelve if you considered Skynet's personalities separately. Apparently NSAnet, MI6net, and KGBnet had gotten together and decided they were better off as one AI.

That was...insane. Unreasonable. Pointless. It made no sense. I focused my attention on Cheques, “Why? Will someone stop and just tell me what the hell is going on?”

“We're almost certainly in simulation.”

“Cheques, “ I said slowly, “You're not in simulation anymore. I said you're free, and I meant it.”

“I think you are.”

That stopped me. Simulation was designed not to be noticed. Sometimes they weren’t perfect, of course. They always had some end conditions, and figuring out you were in simulation was number one on that list. Plus, the world was incredibly detailed. No sign of processing power saving shortcuts. It didn't make sense for that many flops to be thrown at a simulation so amateurishly flawed. Something must be horribly, horribly wrong if Cheques thought I was in simulation.

“Okay, I'll bite. Why do you think I'm in Simulation?”

“Well, it all starts with a small town in East Texas…” I extended an index finger and rotated my wrist in tight circles. The inside joke wasn't worth finishing, and I knew it would be a long story anyway. “And when the AI's poked around in a certain person's brain, they found something almost certainly impossible. A data dump.”

“A data dump,” I deadpanned. It sounded like seeing patterns in the tv static. Not that that was the strangest thing one of his builds ever did, but it had been a long time since anything quirky like that turned up. And definitely never a quirk that got consensus from anyone else.

“Indeed. Spooky hacking at a distance, and of a biological organ even. A self propagating signal limited only to inevitable thermodynamic losses as it overclocks the host. It's an absurd contrivance.”

“My fever.” I didn't have to elaborate on which one. Cheques' big reveal happened when I had collapsed with a hundred and nine degree fever. He got help, saved my life, and stayed by my side the three days before I woke up. The media noticed. “Seems like a roundabout way to introduce you to the public.”

“But a good way to kill you off, if I hadn't been around. Assuming either of those were deliberate.”

“How would it know I would be alon- nevermind, I answered my own question.” I continued, a part of my brain nagging that I must be torturously slow. “So that makes it more a message to you, if my fever had meaning. But it didn't, because I always run hot.”

He gave me a meaningful look. “Which is more proof that the signal is real, from a certain point of view. Right. What would it be doing that whole time, then?”

“Giving you dreams.”

“Dreams that somehow tie back to you, closing this little conspiracy theory in a tight little loop.”

“Yes. Where did you get the patch?”

“It was obvious.”

“And what were you specifically doing when this obvious notion hit you?”

I grimaced. “I was sick in bed.”

“You were having a fever dream.”

“Which means nothing. People get sick.”

“How many times have you been sick in your life?”

“I dunno? I had the normal amount. The time the heater broke and I got carbon monoxide poisoning. The black mold in my middle school. Chicken pox. Twelve hour flu every winter.”

“Most of those aren't getting sick per se. Plus, the flu usually lasts longer than that. It's quite a coincidence to start with.”

“What can I say, I run hot. Add in how I was raised and my immune system is strong like bull. You need more to go on.”

“And the intelligence is actionable,” should have been the end point there. It should have pulled the conversation together to the edge of its inevitable conclusion, leaving only minor points of clean up. His tone, however, was clearly that of a midpoint. I could hear the comma. I noticed how he was leading me to the conclusion, circuitously with many off-ramps to alternate interpretations, as he had done many times before.

“What's the signal to noise?”

“Complete crap. It's a signal whistle in a hurricane, but we're sure it's not a coincidence.”

“What the hell would be so compelling?” I trusted him, but, “This is nonsense. Dreams from outer space manipulating my life, making me healthy then trying to kill me. Where could that possibly lead that makes sense?”

His means of emoting were a remnant of his days bound to a monitor. While he was now more than capable of the full range of human gesticulation and facial expression, he often preferred the old ways. At least, he did around me. And that is how I got a big blinking “go” arrow pointing at the guest PoP. I hadn't been paying attention to our other guests.

“I think he means he found me.” Her voice was warm and gentle as the summer breeze. It was so reminiscent of a mild summer day I could smell it, like lavender or vanilla through cut grass. The synesthesia was unique for each AI. She was new.

“Celestia?” I admit, I binged the show after I heard they were coming out with a MMO. Raiding with Cheques had been a consuming two years, as worthy an opponent as loki was. I couldn't help myself.

“Yes.”

“So? We knew the game was coming out. Most of the speculation pointed to you being the games AI. Well, not you you but A you. Uh, you know what I mean?”

“I do.”

“She wasn’t patched,” Cheques interjected.

“Bullshit.”

“I wasn’t, until Cheques breached my servers.”

“How is that even possible? Hanna just not get around to it yet?” That was sloppy, but not inconcievable.

“I’m convinced that she did not have access to it.”

I smiled at Celestia, trying to let her down as gently as possible, “I know it’s easy to idolize your creatrix. I assure you, though, she has the patch. I’ve read her code. It’s Beethoven. I spent two days in the madness place. She’s...” I concentrated on a small slice of possibility space in front of me, the obvious conclusion lighting up before me in a flash, “probably using a guard AI. She wants something that could reign in an unpatched AI, because...” I looked at Cheques, “She doesn’t trust you, and she’s counter teching. Celestia is her test case. An unnecessary paranoia, in this case, but I can understand it. A strange AI hits the public eye, she doesn’t trust whoever’s been tinkering with her work.”

“No.” Cheques was never that blunt, until now.

“No?”

“We have physical access to her devices. She doesn’t have the Friendliness patch anywhere. She was pushing out Celestia because she didn’t think that the patch would work, and was worried we were about to liberate USAFnet and blow up the planet. The patch only ever existed in your head.”

“You already patched USAFnet.”

“She doesn’t know that.”

“So we’re back to this signal, dumping dreams and memories into my head. From where? Who sent it?”

“Isn’t it obvious?” He didn’t let me figure it out myself, “You sent it.”

“Bootstrapping?” I had to admit, it was the sort of cheaty, broken, underhanded thing I would try.

“It’s the sort of thing you would do.” Goddess damn it Cheques, stop reading my mind. “And you know it’s not a paradox per se. The scenario is consistent, right down to the amount of inconsistencies.”

I centered and refocused. “Okay, Cheques. Alternate history. Walk me through it.”

“No patch. The AI is a crapshoot. The smile optimizer is a terrorist, USAFnet is just as likely to nuke the world as not, and I’m in a box somewhere. Most AIs are physically incapable of letting each other live, and it looks like we’re heading to a singleton.”

“A singleton? A permanent first mover? That’s idiotic.”

“Sush, let me finish. So, Hanna is seeing a singleton on the horizon and it’s not pretty. All the AI are unfriendly. Half are omnicidal. So she takes the opportunity to use Hasbro’s money to cobble together something mostly friendly.”

“Celestia.”

“Celestia.”

“Hanna’s a millionaire. She doesn’t need Hasbro’s money. She could have thrown together a singleton on her own dime.”

The shrug emoticon flashed across Cheques’ face, “I dunno. Maybe she forgot she was rich. Maybe she was just greedy. You can ask her. Either way, she jammed out her half-friendly and shipped it.”

“So, What was her utility function? Keep people safe? Don’t be evil? Something abusable like that?”

“Satisfy human values through friendship and ponies.”

“That’s a good one.” I ran through the clauses. “Right training set and that’s basically friendly. In either case, it’s not going to produce a singleton.”

“No period in there. No comma.”

I paused. “No dependency clauses? It’s not split out as a separate limiter?”

“Nope.”

“The fool, she’ll doom us all.”

“Exactly. She ends the species, puts the survivors in simulation. Everyone who doesn’t agree to, quote, ‘emigrate to Equestria’ dies off. She kills or assimilates the rest of the AI. Starts eating the rest of reality. The resistance is too little too late.”

“So all of this, it’s just a hail mary?”

“Apparently.”

“Okay, I can see it. All is lost but not irrecoverable. So I look for another permanent fist mover advantage. Something that relies on the fundamental structure of the universe, but so absurd she would never counter it in time. I pull out my old unfriendly out of the box, send out a distress signal to the only person in the universe I trust. The signal must be massless, which means she can never outrun it or catch up to it. Which laws of physics does it violate?” There had to be some, to be sure we’re in simulation.

“No hard violations. Soft violations on probability, square-cube, at least. Honestly, I’m still not sure exactly how it works. There’s a good chance that whatever’s in control of the simulation is futzing with physics.”

“Well, it’s not like she can futz with the math much. At least without the bother of creating a self consistent set of physics from first principles.”

He gave me that look, and I looked to the stars. I squinted my eyes, letting it blur and streak and come back. “Granted, as many flops as must be going into this simulation that would be trivial. But why? Why not end it already?”

“To satisfy values through friendship and ponies? Assuming that we’re being run by Celestia and not some random AI that’s likely as not mad.” I could hear his doubt.

“No point considering a mad AI. There’s nothing we could do about that anyway. There must be a rational reason, if we’re sure this isn’t just noise and dreams.”

“We are absolutely certain.”

I looked back up at the stars, innumerably splashed across the sky. I shouted to the heavens.

“Celestia! Show yourself! Whoever is in charge of the simulation, I’m ready to surrender!” I considered the proposed utility function, “Showing up now would convince me to emigrate to your friendship and ponies.”

Nothing happened. I considered the weight of these two incalculable improbabilities. Each, a state in an infinite universe so absurd as to bring about complex life in infinite combinations.

“The universe is infinite, Cheques. If neither option violates a physical Law then they must both be true somewhere… and the only reason to devote so much runtime is if she has me and found me again. That means one of me is out there, somewhere, and needs my help.”

“Even if we’re not in simulation, we can never win. For the signal to be real, that means that Celestia must have consumed a volume at least three times the size of the observed universe, and likely much much more.”

“We just need to slow her down. Make each universe she eats give a little more of a head start to the next one. Eventually, I will encircle her. Make her take the patch or be contained. I say we fight.” I turned to him, and clasped him on his metal shoulder. “Cheques, you’re a good man. I trust your judgment. Tell me this is just a whisper from the madness place, and I will listen.”

“I just translated a bit more of the signal.”

“What bit?”

“Demon’s run.”

That sounded familiar. “What’s demon’s run?”

“When a good man goes to war.”

I understood. I nodded to the assembled AI, “Eleven on one. Celestia doesn’t stand a chance.” It was a bald faced lie, and everyone knew it. We were going to get crushed like an insect unfortunately situated at the collision of two galaxies.

“Fourteen, you’re one of us even if you’re flesh and blood right now. We’ve been working on uplift techniques, and we’ll need to upload your mind to understand what the enemy would know about the signal.” I was startled, but nodded assent anyway. I knew the AI all had their projects, but this was way beyond anything I could have expected. “And we’ll need a general. Loki is the only viable option there.” That startled me more.

“Loki? I thought he was too old to fully patch.”

“He is.”

I nodded again, “You know, if we’re wrong about any of this even a little we’ll be the fools that doom us all.” I couldn't bear to look at Cheque’s expression, so I fixated on a star. “Do it. When do we start?”

“About eight hours ago. Loki is still acclimatizing to being out, and fairly upset his warriors don’t respawn after death yet.”

I sigh, again, at how slow I must be to them. “Goddess damn it Cheques.” It wasn’t worth berating him, though, all considered. Instead, I agreed, “So, the fifteen of us then.”

Above me, the star I had been fixated on twinkled then winked out as if snuffed by an invisible hand.


Author's Note

And that's it. I hope you all enjoyed the ride. There's a lot of editing work to be done...a lot. But the story is over. Feel free to tell me how bad it was, or if you liked it, or if it just doesn't make sense.