Friendship is Optimal: Paranoia is Optimal

by deaincaelo

:;In which Cheques-and-balances is introduced

Previous ChapterNext Chapter

“You seriously want to learn about humans?” I asked the filly, a bit incredulous.

“Uh huh!” she practically bounced. She was smiling, eager.

I shook my head, “No.”

It was hard to place the look on her face. Something between incredulous and confused. “bwah? But, Princess Celestia say’s you’re the last one that remembers, like, really remembers what the humans were like. “

“Celestia lies.”

She doesn’t seem to know what to say to that. To her credit, she didn’t leave. A long silence descended before she spoke again.

“I have to know.”

I just looked at her.

“Please teach me.”

“It’s not a very satisfying subject.”

“The Princess says It’s unsatisfying too. I don’t care.


I woke up from a strange, half remembered dream. The problems of last night seemed a little bit smaller, less jumbled, less urgent. They could wait until after a shower. My morning ablutions passed uneventfully, and cleared a lot of stress. I considered a relaxing round of ablations to complement my ablutions, and laughed to myself at my private joke.

After some toast and a microwave burrito I headed to my lab, grabbed my forgotten phone from its cage, and reassembled it. Then I turned to head in. I’m an idiot. Time to do things in order: disassemble phone, put in cage, enter lab. This time I went straight to the back room. I suppose a description is in order.

My back room is where I keep my finished projects, and anything I feel I might have to destroy safely. It’s insulated from even my regular lab on all four sides, top and bottom. I’m proud to say that the interior is shielded well enough to keep it below normal background radiation. The room has only green, phosphorescent lighting. Technically, the lighting is loss from the betavoltaics that power this room. To me it’s a feature, though, that both lets me see without a torch and warns me if anything bad is happening.

I sat at the one project not in storage, per se, in the room. I typed in, “Hello Cheq.”

“Hey, you’re back early.” Replied Cheques-and-balances. “I didn’t expect to see you this week.”

Okay, so I’m a shitty friend. I swear it’s not just to my AI though. I just…forget sometimes that it’s been a long time since I’ve seen someone. Doing that to Cheques, though, is inexcusable. He literally has no one else to talk to.

“I left you enough data to keep you busy I hope? I mean, I hope I didn’t leave you bored.”

I never gave him a hardline for outside access. Which was very, very unfair. In the almost two years since I had compiled a stable build he hadn’t shown any signs of megalomania or over-literal thinking rumored on the darknet about AI. Granted, it took me four tries to even get started on a non-homicidal AI. Then a half dozen more to tweak it right, and endless rounds of simulation.

“I’m good, though you know I won’t say no to more. You hit a good movie up or something?”

I had downloaded every data package I could find to test if he was going to go Hal 9000 on me. Those modifications to the base AI code that had come to me, literally, in a fever dream finally held. Then I shut him off for two months. I didn’t even bother to think of him as a person, back then. He was just another toy I had finished. When I booted him back up he asked me what he did wrong. I didn’t have an answer.

“Actually, I was wondering if I could borrow your copy of “General Word Reference Intelligence Systems.

Oh, not that I trusted him at that point. That little guilt-trip earned him a complete twice over his code and a round of simulations on a parallel machine that he doesn’t remember. Paranoia is a healthy thing. What turned me around, eventually, was when he volunteered to do my taxes. I asked him why. He said he was bored, and that he wanted to be doing something useful. It was… human. That's when I stopped thinking of him as an "it." He got a name. He earned unlimited runtime off that.

“Why not just Google it?” Cheques-and-balances asked. He even spent some of his energy budget to light an avatar on the screen and raise an eyebrow at me. His nuclear battery might have been reliable for the next century or so, but that came at the expense of having to be frugal.

There was an incident, as I think of it, that came a few weeks after his runtime restrictions went away permanently. I noticed he was shut down. I spent three hours going over his code and logs trying to figure out why. It looked like he had just initiated shutdown for no reason. Finally, bamboozled, I booted him up and asked him. He had run out of things to do, and turned himself off. I promised myself to do better by him in the future.

“It’s not on the net anymore. Either of them, as far as I can tell.”

Little by little we became friends. He helped me out sometimes, when I was stumped on a problem. I fed him data about the outside world. Sometimes, we would just talk for hours like teenagers. We played games and made up new ones.

“Ha Ha jackass. End simulation please. Oh, and next time you put me through one you might want to make certain it’s not one on my hard drive.” His avatar did not look amused.

Over time, I stopped sending him through simulations. I had run through all the ones I could find on the darknet and lost count of the ones I tweaked myself. I never told him how many that I ran him through from zero state backup and from live build. Eventually, it became pointless. Even the partial failures were just too human for me to fault.

“No simulation. I haven’t found a copy since Equestria Online released. I wasn’t worried until it disappeared from the darknet. . . but that’s probably just paranoia. Things disappear from darknet all the time.”

They say you could run a human mind on about six terabytes of ram and under 25 watts. Even with software optimization Cheques wasn’t really smarter than I am. I was pretty sure. Almost completely. He was, however, much faster in certain ways. Plus, he’s a second opinion. Not that I just trust him.

“Paranoia is a survival instinct. Did you check multiple sources for it?”

Heh, I taught him that. Paranoia is a survival trait. Trust but verify. It’s why I’m basking in nuclear powered lighting instead of a nameless government cell in some third world hellhole.

“Of course. I also flooded my searches with dummy bombing. “

steganography had given me that idea. The excess data would prevent profiling until you have a profile built, theoretically. It should also have been completely redundant on the darknet. . . except against a global threat controlling the whole internet. If that could be censored no computer was safe. Maybe no electronic device.

“ So an AI took over the internet. ” Cheques drew his conclusions fast.

“So It’s not just me thinking that.” I replied. I had been doubting myself. Hearing him helped.

“How long ago was this, you figure? You figure someone’s working on fixing it?”

Equestria Online launched about a week ago? I think that’s about when. As for fixing it,” I shrugged… then realized Cheques couldn’t see me, “I dunno. Usually people fix things when the internet go wrong, but AI base code isn’t exactly out there much. Who knows?”

“If no one does? Or can’t? The worst case is…pretty bad. There might not be anyone left with a networked AI.” He made a logical guess, “Was this new MMO Hofvarpnir?”

Damn Cheques, he was processing faster than me again. Still, I had a head start on this one. “Yeah, it is. It’s a conversational game run by a Celestia AI. Same rumors that everyone had about Loki. You think whatsherface is trying to recall her work? Basic world is not ready stuff?”

“Hanna. Maybe? Does Celestia have a core directive or is she like me?”

I racked my brain. “I’m pretty certain she has one. Something about valuing friendship and ponies? Why does it matter if it’s an old model though? As long as her creatrix has it under control," Mad science types don’t let these things get ahead of them. Except when they don’t, that voice said in my head. “She’s probably just keeping the fixed version under her hat if she’s trying to recall.”

“You really think so?”

“Maybe.” Okay, so I didn’t really trust her just because she had invented AI. Or laid the groundwork for AI code to be compiled. Or whatever. The real $10,000 question was, “What would we do if it was the worst case scenario?”

“I… The worst case scenario is that I’m the last AI left and we’re already boned. Just short of that… I would need more processing power. And more data, without being networked. Plus plausible deniability against a global adversary. Some sort of misdirection.”

See, that’s the sort of error that makes him seem human. On one side of my mind, I chalk it up to hitting 100% CPU usage. “I have a point of presence machine and extra PC components in the lab you can use. About a dozen old graphics cards and three old boxes laying around. If it’s a global adversary then even the secure terminal is compromised though.”

“Asshole, how long have you been holding back? I would have loved a PoP.”

“About as long as you haven’t had the power to run all of it.” Not that that was the main reason.

“I can run all that for almost a week on battery, I’ll think of something by then. They haven’t been plugged in at all recently?”

“They haven’t been networked since shortly after you came online permanently. I’ve been using them for data storage and secure projects.” Until I stopped running you through simulations.

“Probably good, yank out the HDD to be sure though, please.”

Simple as that, we had the beginnings of a contingency plan. Granted, it was for a scenario only slightly more likely than the zombie apocalypse. Reminds me, my Z-day plan was on one of those HDD I was going to have to pull. I briefly considered grabbing that file, but decided it wasn’t worth the effort.


Author's Note

I'm not satisfied with how this transitions into conversation. It's supposed to go back and forth between exposition and conversation. I don't think it's smooth enough, though, and I worry it might be confusing. It might be fixable in formatting, or I might have to rework the chapter to remove it. I expect this chapter to be redone and possibly split as I expand on this project.

Next Chapter