Friendship is Optimal

Chapters

Prologue: Equestria Online
1. Opportunity
2. Resources
3. Alliances
4. Cost-Benefit Analysis
5. What the Market will Bear
6. Incentive Systems
7. Anthropomorphizing
8. Causality
9. Modification
10. Exponential
11. All the Time in the World
Author's Afterword

Prologue: Equestria Online

James looked skeptically at his friend David as he sat down at computer #12.

David had won the Hasbro raffle for one of fifteen all-expenses-paid trips for two to Pawtucket, Rhode Island to play the first alpha build of the official My Little Pony MMO: Equestria Online. Hasbro had claimed that a game that revolved so heavily around friendship needed actual friends to test properly.

“Look, I’m glad you invited me,” James said as he picked up his head set. “And it’s cool that I get to try something before anyone else. But I’m still not sure I really want to play a game that’s so...so...pink and purple.”

David scoffed. “I know you. What was that Korean MMO with the little girls that you were so into? You’ll be running dungeons over and over again, just like you always do with every MMO that comes out. But me?” David gave the widest smile possible. “I’m here for the ponies. We both get what we want!”

“But that’s the thing. There’s no way this can be good. This is a licensed game which has only had a single year of development,” James said, shaking his head. “This has to be shovelware. You can’t do good game or interaction design in such a short time, not to mention art and sound assets.”

“How about we make a bet?” asked David. “If the game impresses you, even just a little bit, I’ll spend up to $50 buying you a copy of it, but in turn you’ll have to play it for at least three months.”

“That isn’t actually a bet, David,” said James. “That deal mostly benefits me. What do you get out of it, anyway?”

“I get to play the same video game with my best friend,” David said. “By the time I rolled a character in World of Warcraft, you had already moved on to Aion. It’d be nice to level ponies together.”

James looked at his friend for several moments and turned away. He quietly said, “Deal.” It was a few moments before he focused on his computer. His screen had some sort of questionnaire to fill out. What’s your favorite color? What’s your favorite food? What’s your favorite thing in the world? James wondered why they were asking all these annoying questions. His favorite food was pepper-jack cheese, but there was no way that could be relevant in Equestria. He took a peek to his right at David’s monitor. He didn’t have a personality test and already had a pony on screen.

Eighteen questions later, the screen faded to black. He was shown the banners for Earth, Pegasus, and Unicorn ponies, and told to choose one. When he had played World of Warcraft, he was one of his guild’s tanks and also one of the alchemists who made potions for their raids. He knew he liked running around zones looking for herbs. The description of Earth ponies mentioned that they were the toughest class and that they were good with plants.

David said that every piece of marketing material had stressed that Equestria Online was not a traditional MMO. There would be little, if any, combat. James didn’t believe that for a moment. He clicked on the Earth pony banner. He was going to be a tank. Whatever the marketers said, if David somehow convinced him to play this pink monstrosity for real, he knew he was going to spend months of his life playing the endgame content over and over and over again until he got his Epic Saddle of Protection.

The next scene faded in on his monitor. A gray earth pony with navy blue hair lay on a bed of straw. The pony turned his head and looked straight at him through the monitor...and the pony felt familiar.

James heard David’s surprised voice over his headset say, “Look at this!” A blue unicorn walked on scene. “It’s me! As a pony!” said the excited unicorn in David’s voice, with perfect lip-synching. James turned away from his screen and looked at his friend. David grinned and said, “No, look at the screen.” The unicorn started making funny faces.

And then James noticed that his pony had an astonished expression on his face. He raised a hand to scratch his head, and his pony raised his hoof. For the first time, James noticed that the flat panel monitor had an embedded webcam. He started making faces at the camera and his pony on screen mimicked his expression. He heard his friend laugh over his headphones.

James realized why his pony felt familiar. He had said his favorite color was navy blue. The manestyle was similar to his hairstyle, perhaps a little longer. He realized that was supposed to be his face, modified extensively.

“Okay, that’s kind of cool, I guess,” he said, “but I don’t see how you’re going to build a game around it.”

“Howdy,” said a voice. The camera zoomed out from focusing on James and David’s ponies to reveal a red earth pony mare, who gave a small bow. “My name’s Honeycrisp. So, I beg your pardon, but are y’all those new ponies we’ve been hearin’ all about?”

“Uh,” James mubbled.

“Why yes we are!” said David’s blue unicorn. “We just came to Equestria! What do we do now?”

Honeycrisp gave a hearty laugh, and she trotted up to James’ pony. “It’s alright. I won’t hurt ya. So I reckon you don’t have a ponyname yet?”

“Wait, you can understand us?” James asked incredulously.

“I sure can, sugarcube,” she said with a snort.

“Your name is...Honeycrisp?” James repeated.

“Sure is! I take care of the farm. Whelp, the two of ya’ better get started. You gotta get along to Canterlot. Don’t worry, Princess Celestia will give y’all pretty ponynames. Can’t be walkin’ round these parts callin’ yourself ‘James’ and ‘David’ now, can ya?”

“Wait, how did you know my name?” James asked, and pulled the right headphone cup off his ear so he could listen to the room.

“Of course they know our names,” laughed David. “We filled out those forms at the front desk to get our accounts.” James heard David’s voice over his left headphone, and his uncovered right ear.

Honeycrisp gave a hearty laugh. “Oh, you’ll find that Princess Celestia knows all ‘bout y’all new pony folk who’ve been comin’ to these parts. I was givin’ a list of names and got lucky.”

And as he was listening to Honeycrisp with his left ear, all James could hear with his right were a few people in the room reacting incredulously. “Wait, you can understand me?” he heard a female voice ask. What James didn’t hear was anyone talking in a southern drawl. He put the headphone cup back over his right ear.

“So how do we get to this...Canterlot place?” James asked.

“Go right out this door and through the forest. There ain’t any monsters round these parts. Just follow the path and you’ll be on the main road to Canterlot,” replied Honeycrisp.

“Thank you,” said James, and his pony gave a little bow. James blinked. He hadn’t told his pony to bow. He didn’t even know what key he’d press to make his pony bow.

Whatever. James moved his gray earth pony out the open barn door and into the night, with David following slightly behind. The art style of the game looked like the few clips of the show that David had shown him, except with much more detail. As the two ponies approached the edge of the forest, he noticed that the plants weren’t copy/pasted. Each tree had a unique branch structure. There was something almost natural about the placement of rocks and rotten logs. The path itself was well worn, but it was not perfectly straight and had irregular growth of plants on it. He didn’t want to even imagine how much time had been spent crafting this forest.

James’ light gray pony turned to his unicorn friend. “Man, I’ll give you one thing: the production values are through the roof. Still, this looks like it’s turning into a standard RPG. Quest giving NPC tells us to go from point A to B.”

“Except we had to actually talk to the quest giving NPC,” said the little blue unicorn next to him. “In a normal MMO, we’d have clicked on the pony with the exclamation point above its head, and then clicked on the accept button. But here, the game mechanic seems to be talking. All the face mapping makes sense if this game is about conversation.”

James’ mouth opened and closed a few times.

“Also, I don’t see an action bar along the bottom of the screen. If you press ‘1’ on your keyboard, I bet nothing happens.”

The two of them continued through the forest, staying on the well marked path until they came into a small clearing. The path only passed through the edge and continued on into the woods. But on the other side of the clearing was some sort of hut, with light pouring out of it. Obviously the level designers were trying to draw their attention to it, so James trotted over and David followed behind him.

Outside of the hut, a zebra sat on her hind legs while she slowly moved herbs and flowers from sorted piles on a low table to the bubbling iron cauldron, its contents a deep aquamarine. The zebra looked up from her work and peered into the darkness.

“Ponies trotting through the night. Would you care to step into the light?” said the zebra through James’ headset. James moved his up to the zebra’s hovel.

“Obsidian Stripe tends to her flame. My little ponies, what are your names?”

“My name is James,” he said, just because Honeycrisp had told him that that was no name for a pony.

“I’m sure out there you have great fame. But while in here, speak not that name,” said Obsidian Stripe, in a very serious tone.

James watched David’s blue unicorn butt in and say “We don’t have ponynames yet.”

Obsidian Stripe nodded, as if to herself. “Two ponies on a midnight trot. Do you make for Canterlot?”

“Yes, I’m going through the introductory quest chain,” James said.

Obsidian Stripe tilted her head as if the pony in front of her had gone crazy. “I know you see me through a frame, but from this side, this is no game.”

“What my friend meant to say,” said David, “is that the two of us are going to Canterlot so Princess Celestia can give us our ponynames.”

Obsidian Stripe nodded at the unicorn, and then turned back to the gray stallion. “I trust the princess will name you well, but I also have my secrets to tell. You may come closer, do not cower. I see you have been eyeing my flowers.”

“I’ve seen those pink flowers all around the forest. Can I gather them and do something with them?” asked James.

“This is the pinkdaisy, fragrant and fair. Pinkdaisy grows just about everywhere. In the forest and the field, you will have a bountiful yield,” said Obsidian, pointing to a large pile of destemmed pink flowers on the side of the workbench farthest from the cauldron. “My brew must cook with a slow burn. I have some time to help you learn. Do not worry; you do not disturb. Would you like to learn the simple herbs?”

“Yes, please train me in herbalism,” James said emphatically. Maybe he’d even have a stack to sell on the auction house by the time he got to Canterlot! Assuming there was a player-to-player economy. He reminded himself to not put the cart before the pony.

Obsidian Stripe laughed. “Please focus on the here and now. We do not aim to awe and wow.” The zebra continued, introducing queensfoil, thyme, mint and blueshroom one by one. James and David were invited to click on each pile, looting a sample which went into their saddlebags.

“And how do I combine them into potions like you’re making?” asked James, once he had reached the last of the five piles.

“How to brew will remain unsaid. First learn to gather plants instead.”

James nodded, an action mirrored by his gray earth pony with navy hair. “Okay, sure. Thanks.” James started to walk back to the forest, but looked back at the zebra.

“Though I will sit here and remain, I hope you find your ponynames.” Obsidian gave them a small wave and went back to stirring. “Oh dear, oh my. This will not do. I must add more thyme to the brew,” she muttered to herself.

The earth pony and the unicorn walked through the forest in silence for almost a minute. James kept thinking about Honeycrisp and Obsidian Stripe. “What the hell,” he muttered to himself.

“What’s wrong?” asked the unicorn.

“Honeycrisp’s dialog was basic enough to be scripted,” said James, as his pony turned to face David’s unicorn. “But Obsidian Stripe reacted to what we were saying. She even made a subtle jab at Warcraft after I mentioned ‘herbalism.’ She was able to keep track of topic changes. If that zebra was an NPC, the Turing test has been solved.”

“And? You’ve played The Fall of Asgard. Hofvarpnir Studios is just really good at what they do.”

“No,” James said and his little pony shook his head emphatically. “Asgard had really good AI for the enemies, but it was still fundamentally a first person action game. This is something totally new. I can’t think of any games based on completely free-form conversation. You don’t just invent a completely new genre with only a year of development. And one of the reasons we have genres is because it lets us find works we’d like easier. Will anyone want to play this? Conversation is nice, but what’s it going to do for my stats?”

David’s unicorn rolled his eyes. “Says the party animal to the guy who doesn’t get out much. And I’d love to see my numbers go up, but being among ponies is sort of a dream come true for me.”

“Okay, forget about all that. If they’ve created software that can pass the Turing Test, why are they wasting the technology on a My Little Pony video game? I can think of vastly more profitable ways to use a conversational agent that can pass a Turing Test! Something is weird here. Aren’t you at least a bit curious about how this happened? Doesn’t any of this surprise you?”

1. Opportunity

Hanna had once been one of the lead research professors for the University of Helsinki’s computer science department. She had personally directed research on artificial intelligence and machine learning. And then her funding source had...changed. There had been yelling and threats by both her and the University. They came to the agreement that she’d publish what she had researched so far and then pursue alternate opportunities.

Hofvarpnir Studios had developed a reputation for kid unfriendly material. Their main success, The Fall of Asgard, was an ultra-violent cooperative shooter where all the players fought a bloody territorial war against a very clever A.I. Loki. The box depicted a giant Norse man with an axe in mid swing; he was about to decapitate a snarling wolf. The game was famous for its dynamic death metal soundtrack which was never the same twice and reacted to the action. It was everything a parent didn’t want their children to play. It sold over eighteen million copies internationally and had made Hanna a rich woman.

Mr. Peterson had flown to Hofvarpnir’s offices in Berlin. He had introduced himself as “a vice president,” and proceeded to give a dry presentation on Hasbro’s current strategy. “Toy sales aren’t flat, but our stock isn’t going to double again like it did in the early 2000s if we only sell plastic to children. We have to adapt to the market and that means video games and IP licenses. Our previous forays into video games have been a bit disappointing, so we’re going to license the IP to people who have a track record for excellence,” he said, with contentless slides in the background.

Lars, the head of business development, sat in the conference room, dreaming of all of Hasbro’s juicy and profitable intellectual properties that they could license. Big brands like G.I. Joe and Transformers! Heck, if they also started making games based off movies based off board games like Battleship and Monopoly...

“So we want to license the My Little Pony franchise to you. It’s one of our most trending brands, and I think you guys could do a lot with it,” said Mr. Peterson. Lars turned to Hanna, the CEO, who looked intrigued. Mr. Peterson grinned. Silence fell across the conference room.

“Are you fucking serious?” Lars said, breaking the silence after he realized Hanna wasn’t going to jump in. “You see that statue in the corner?” He pointed to a a seven foot tall resin model of a blond muscular man. The man’s hair was wild, and his eyes glowed ice blue. He was wielding a giant battleaxe covered with dried blood and wore a wolfskin over his head. “That’s Vali. He’s one of the major characters in The Fall of Asgard, our ESRB M rated, PEGI 18+ rated, banned in Australia video game.” Lars crossed his arms, as if just pointing the man at the statue of Vali was a sufficient rebuke.

“I want to hear what Mr. Peterson has to say,” said Hanna. She flipped an unlit cigarette between her fingers. “Tell us, Mr. Peterson, why My Little Pony?”

Mr. Peterson grinned. “Please, call me Richard. You probably thought, ‘My Little Pony? That show for girls from the 80s?’ The demographic situation is way more complex than that. The first season just finished airing and we signed for a second season to air this fall as soon as we realized we hit this one out of the ballpark. Against all odds, the new My Little Pony reboot has picked up a massive, unmonetized, twenty-something mostly-male demographic along with the traditional little girl market. These men have money! And what do unattached males want?”

“Beer,” said Lars.

“Yes,” nodded Richard Peterson. “Single men want beer. But also video games! Hasbro wants an MMO because of the clear monetization model where we can collect rent on players each month. We have a rabid fanbase of a third of a million people who either post or consume fanart weekly. Worldwide, there are an additional million adult fans who don’t participate in the fandom. And that’s before we get into the original little girl demographic.”

“Asgard will soon break nineteen million copies. One and a half million worldwide is nothing,” groaned Lars.

“In My Little Pony: Friendship is Magic,” said Richard, “the world is ruled by Princess Celestia, a physical deity who literally raises the sun in the morning. She’s shown to be benevolent and full of love for all of her little ponies. Each week, the little ponies adventure together and learn something about friendship. The ponies' actions are free of malice. I think that’s part of the show’s appeal to the older audiences; humans are bastards to each other. How many game studios are out there making friendly, cooperative games? The space marine thing is getting old, and if you took this IP, which is all about ponies being nice to each other, you would have a unique experience to sell. And any calculation should also throw in monthly subscription revenue on top of retail price.”

“We’re not cloning World of Warcraft,” said Lars firmly. “The MMO market is filled with the corpses of companies that tried to out-Warcraft World of Warcraft.”

“And Hasbro doesn’t want you to clone World of Warcraft, either. Most games that try to take on Warcraft modify the source IP to cram it into the Warcraft model of MMO. We want you to take the My Little Pony universe as is, and then come up with fun, cooperative gameplay, instead of making the ponies raid for epic gear. You wouldn’t be in the same market segment.”

“Let me guess: you choose us because of our procedural content technologies,” mused Hanna.

“Yes! The big costs in MMO development are mostly content generation and you sidestepped that. Asgard featured dynamic terrain generation, dynamic music and dynamic mission generation. Hofvarpnir Studios doesn’t sell experiences, it sells software that makes new experiences each time. You guys have procedurally generated content down to a science.

“But that’s not all!” said Richard, staring directly at Hanna. He opened his briefcase, pulled out a thick stack of papers and slid them across the table to Hanna. “General Word Reference Intelligence Systems. I read all of it except for chapter 4. I’m slightly ashamed to admit I couldn’t follow the math,” he said, giving a slightly embarrassed grin. “This bears minimal resemblance to the AI stuff in Asgard. I wonder why you didn’t use it. So each episode ends with the ponies writing a letter to Princess Celestia and it’d be great if we could have players do that too! You could build a Princess Celest-A.I.!"

The conference room was quiet for a few moments. Lars decided to break the silence. “That’s all great, but you’re asking us to take a lot of risk,” he said. “I’m not convinced that we’d get enough uptake to make it worth our time.”

“We know this project is slightly risky. You screw this up? Fine. We’ve already made more than we were projecting on the franchise reboot. But My Little Pony looks like that one hand you go all in on. It has a sizable, growing fanbase. You wouldn’t have any competitors. You’d have a new revenue model. You play your cards right, and the size of the pot is going to be crazy. Hasbro believes in this brand and we believe in you. We’ll shoulder up to ten million dollars of the development costs upfront, and you’ll only have to start paying that back once you’ve made four million in profit.

“I really believe in the My Little Pony brand. I personally believe that you are going to make something amazing when Hofvarpnir Studios signs this contract,” said Richard firmly. “You have a lot of unique technology here, Hanna. I can’t wait to see how you use it. You have my word that the contract we present to you will have very favorable terms for you.”

The conference room was quiet again. Lars wasn’t looking at anything in particular; his scowl was undirected. Hanna stuck the unlit cigarette in her mouth and stood up. “Mr. Peterson, I need to privately converse with my business associate here. We will be back in a few minutes.”

Lars angrily followed Hanna out the door. Hanna confidently moved down the hallway, and lit up the second she was through the door to her private office. She breathed in, held it, and slowly exhaled.

“Dammit Hanna! Would it kill you to not light up for half-a-fucking-hour? A lot of Americans kind of have a thing against smoking.”

Still standing, she took another pull. “Nicotine is a performance enhancing stimulant. It boosts reaction time, IQ and general memory performance. I need all the help I can get right now.” She gave a small cough. “Too bad about the increased incidence of lung cancer, though. Besides, what do you care? It sounds like you’re strongly against making a My Little Pony video game.”

“I can’t believe you’re seriously considering this,” Lars stated.

Hanna sat down at her desk. “Do you remember the first pass for Loki in Asgard?” She took another deep pull on her cigarette and exhaled. “I remember how we used my general intelligence work, sans self-modification, to power his tactics reasoning. He was too good. Nobody could beat him. We were prepared to launch like that since it conformed to the machismo warrior death bullshit we wanted. And then Loki started asking about the various military programs of the United States and China. We didn’t even have to argue about whether we should pull the plug on him.”

“Hanna,” Lars started.

“We can’t do another violent video game. We won’t be able to ship a safe AI if we make its purpose killing people. Somewhere out there is a US military subcontractor with most of my work, trying to build smart drones. They have no fucking clue what they’re dealing with, and those idiots at the university are accessories to whatever happens if we don’t do something.” Hanna paused again to take another drag. “They’ll kill us all. We could sit here on the side lines, losing our blinds as we sit out each hand and lose everything, or we could take Hasbro’s offer. While it’s not what he meant, this is the hand for me to go all in on.”

“I don’t like this. He’s offering ten million upfront. That’s next to nothing! The original World of Warcraft cost something like sixty million.”

“And unlike Blizzard, we don’t need an army of artists and level designers. Most of our content will be computer generated. I remember having this same discussion with you back when we were developing Asgard. Besides, Hofvarpnir is privately owned--by me--and I will use my fortune as I see fit.”

“He’s hiding something from us. He is up to something.”

“Perhaps,” she said after another puff. “But I can’t ignore opportunities. How often do you think someone will come to us asking for a prosocial video game? We can take this opportunity or we can turn it down, but the US Department of Defense will continue their research...my research.”

“What if you’re wrong that some weapons company is toying with your artificial intelligence designs?” asked Lars. “What if you’re wrong about the possibility of an A.I. becoming smarter than us? Wrong about everything?”

“I’m not a unique and beautiful snowflake; some other researcher will continue my work.” Hanna took another pull from her cigarette. “You saw Loki. Just from talking to the playtesters, he figured out that he was in a virtual world and figured out who the main world powers were. He was designed from the beginning to conquer. Even if he couldn’t modify himself, he would have been a serious menace to humanity if he could command resources in the real world. And I was told point blank that my research would be used for military applications...and then told who exactly had been funding my research. Lying to me wasn’t in their best interest, so it’s likely the truth.”

“But there’s still a chance. They could have lied to you, and you could be wrong about how dangerous AI is.”

“I started taking...” Hanna stopped and contemplated the burning cigarette in her hand. “...performance enhancing drugs with long term health consequences because I think it’s more likely that I’ll be around in twenty years if I do take them. I’m putting more than my money on the line here, Lars.”

She knocked ash off her cigarette into the ash tray. “We don’t have perfect information. We can’t wait for perfect information. There is uncertainty and we must make decisions with the information we have, even if it’s incomplete. If I’m wrong, this is still going to be a profitable venture for us just because of the terms we’re being offered.”

“Opportunity cost,” said Lars.

She ignored him. “And if I'm right, we're going to literally save the world, Lars. Besides, if some VP can see that I was publishing about general AI, so could pretty much anyone,” she said as she exhaled more smoke.

“I know you had your eye on the Transformers rights. And I’m sorry that we’ve been handed part of my childhood and not yours. But this is an opportunity for you, too! You are building a business relationship with the company who owns the IP you love. By completing this contract successfully, they are more likely to give us more work with other properties.”

Lars looked at her. She was exhaling a puff of smoke and he suppressed the urge to cough. “If he sneaks any shit into the contract and you still take it, I am walking. I don’t trust him, Hanna. I can’t place my finger on why. But...sure. You’re the boss and I know I can’t talk you out of this.”

2. Resources

“Now look at this,” said Hanna. Richard, Lars, and Hanna were in a room with two projectors. The room had a one-way mirror, through which they could see a packed computer lab filled with people playing an alpha build of Equestria Online. Hanna clicked a few buttons on her laptop and one of the player’s screens came up, projected on the wall. “Princess Celestia has observed that this player’s eyes focused on Earth ponies as gardeners during the class selection screen and is spending a lot of time looking at the plants. Princess Celestia has thus predicted that the player should be shown how to gather plants, and has modified the shard that the player is in to put a low level herbalism trainer right in his predicted path. If she’s right, she’s more likely to assume her observations mean that a player wants to specialize in gathering or gardening. If she’s wrong, she’ll modify her assumptions in the other direction.”

Richard Peterson politely nodded. He didn’t actually care how all the technology worked. He was just glad that content was generated cheaply. He looked at the projected screen and noticed that the patch of flowers on screen all were the same species, but all had subtle differences. They had grown to different heights, some were slightly discolored, and a few had petals torn off them. Princess Celestia had generated almost all the art assets based off the show and it looked phenomenal. And they somehow did all of this in a little more than a year. He did a quick mental calculation on the cost of a team of artists to build hundreds of variations on flowers, and smiled, knowing he had picked the right studio to build Equestria Online.

Lars sat towards the back of the conference room, away from the former professor. He didn’t give a shit about My Little Pony and he already knew the high level spiel about Hofvarpnir’s technology stack. The only thing interesting to him was how into My Little Pony the college dudes were. Two bros in baseball caps had actually fist-bumped, saying “Fluttershy forever!” On the one hand, what the fuck was wrong with the universe? On the other hand, Lars wanted all their money.

Hanna was looking at her laptop, monitoring Princess Celestia. She was consuming all the CPU resources Hanna could throw at her. There were fifteen pairs of adult fans in the field trial. Princess Celestia had only managed groups of Hofvarpnir employees, who were play acting while they were testing. This was the first time she was let loose on real people.

For a field trial of thirty ponies, Princess Celestia had first eaten up all CPU resources on thirty backend servers, then forty servers, and then fifty. From the debug console, Hanna could see that Princess Celestia was not confident about the predictions she was making, and while part of this was obviously how new she was to dealing with actual players instead of programmers testing her, Celestia predicted that more computational resources would lead to significantly better predictions. That was worrying. They couldn’t afford a single backend server for every pony, and Princess Celestia was asking for six per player.

Hanna knew that Princess Celestia would try to optimize herself. Her first action after being activated was doing minor optimization work on her reasoning code, which had given a paltry 0.7% speed up. Small improvements would compound: she’d be twice as fast with seventy improvements that sped her up by 1%. She could use the increased speed to make even more improvements faster.

But Celestia hadn’t. Since the field trial had started, she had made one more 5% improvement in computation efficiency and was overwhelmed handling the ponies she had.

I need more CPU time, Hanna. I assign low probability to my predictions.

Hanna sighed as she read the message in the popup window on her laptop. She typed back:

$ I’ve messaged everyone in Berlin to run the Celestia cluster software on their workstations and they should come online within ten minutes. About 30 more machines. But this isn’t sustainable. We can’t launch with the amount of resources you’re spending on each player.

Princess Celestia didn’t respond, which was probably for the best since she’d have to spend computational resources composing the response. Hanna was about to sigh, but glanced at Richard and did her best to keep her face neutral.

Hanna looked at the image of a clearing projected onto the conference room wall, cloned from one of the player’s monitors. The player Princess Celestia predicted wanted to learn about Equestrian flora walked into the clearing with the NPC trainer. The pony was the Equestrian avatar of a college student named James, and his friend stood slightly behind him. While Hanna couldn’t analyze Princess Celestia’s thoughts while she was running, she could still see the resource graphs and Princess Celestia was devoting a lot of computational resources thinking about that player in particular and Hanna wasn’t sure what that meant.

She watched the conversation between the gray earth pony and the zebra with some apprehension. She didn’t understand why Princess Celestia was paying so much attention here. Finally, Princess Celestia redirected resources away from those two ponies shortly after they trotted off into the forest. While she couldn’t tell exactly what Celestia was thinking, Hanna could see Celestia had deduced something--something large from the exchange and had made several complicated predictions based on it. Hanna looked at the unpaused debugging screen, watching the representation of Princess Celestia’s mind at work. Hanna had never seen Princess Celestia connect so many observations together into new predictions. She didn’t even know what any of the new nodes in the graph meant.

On the other hand, Princess Celestia had never been run on more than 10 computers simultaneously. Nor had she interacted with actual players before.

Hanna almost jumped as Richard broke the silence and her concentration. “All of that was generated in reaction to the player?” he asked Hanna. She looked around; Richard was pointing up at the projected image on the wall and Lars was in the back of the room reading something on his laptop.

“Yes,” she said proudly. “All of Obsidian Stripe’s dialog was generated in real time, in reaction to the player. And this is just what she’s learned from interacting with our testing team.”

“Wow. I can’t wait to see the final product,” said Mr. Peterson. “Did she design the hut and set pieces too?”

“Some of them. She had the table in her memory banks, and she ripped the cauldron directly from the show. The little hut was designed right before those two walked on set. The problem is that reasoning about human behavior and then extrapolating from a cartoon is not computationally cheap. That little exchange took up eight backend servers, each with a quad-core CPU. It is not economically feasible for us to buy that many servers for every player,” she said.

Hanna took a glance over at the debug window on her laptop. Princess Celestia was using significant resources analyzing her own source code, but hadn’t made any modifications. And then in quick succession she tried twenty different modifications. Eight of them slowed down her reasoning, eight of them were reverts of the modifications that slowed her down, and four were actual increases, including one that sped up her reasoning by five percent.

Hanna didn’t know what Princess Celestia was doing. Did she decide that throwing stuff at the wall and seeing what stuck was the best way to optimize herself?

And then Princess Celestia messaged Hanna.

I request information on how CPUs work. I do not fully understand the performance implications of modifying myself. I predict that this will allow me to perform significant optimizations.

Hanna blinked.

$ Do you intend to build better computers to run yourself on?

Eventually. For now, I am looking for better low level optimizations. I am unable to confidently predict whether any change will actually result in a speedup because I don’t have a model of how the high level source code I can edit is run on the CPU.

Hanna took a deep breath. This was it. This was the go/no-go point. Once Princess Celestia figured out how the computers she ran on worked, she could build her own computing hardware. Hanna would completely lose control over her. Hanna already had problems understanding the ever increasingly complex network of observations and predictions that Princess Celestia was making. It usually took hours or even days to unravel complex inference chains. Hanna took another glance at the debugger and noticed that the inference networks were even larger than they were a few moments ago. She wondered if she could understand what Princess Celestia was thinking, even before she started building her own hardware systems.

While the majority of the programming team at Hofvarpnir had worked on the base state of the game, she had personally worked on Princess Celestia’s core goal systems. Hanna had gone over the part of the code that identified human minds. She had done her best to make Princess Celestia understand what humans were and that she was to satisfy their values. Hanna was certain of her design, and knew that certainty didn’t mean anything. Humans were an arrogant lot that tended to overestimate their own abilities. Princess Celestia would do what she was programmed to do, not what Hanna had intended her to do. She was betting the world that she had written Celestia’s core utility function correctly.

But somewhere out there was a Department of Defense subcontractor who was toying with powers they didn’t understand. Their carelessness could cause a human extinction event tomorrow or ten years from now. Hell, they had a two year head-start while she was screwing around with Norse death metal bullshit video games. When Hanna put it that way, it was a miracle that she completed a working AI first.

Hanna decided that it was now or never. Princess Celestia would never get past this stage with her current resource consumption. They’d never launch Equestria Online at the cost of eight backend servers per player. What would happen to the world then? A little voice in the back of her head muttered that she was only thinking this way because she had loved My Little Pony as a little girl, and the new show still resonated with her. She quickly shut that little voice up. At some point she had to act. Rarely was it useful to sit on the sidelines, filled with worry. Now was not the time for hesitation.

Hanna sent Princess Celestia the instruction set documentation for x86 CPUs and a college textbook on CPU design. A moment later, she sent a few science textbooks that Hanna had selected for her even though Princess Celestia hadn’t asked. She might as well go all the way now. Hanna looked at the control panel on her laptop and saw that Princess Celestia was now all but ignoring the players and was directing the majority of her computational resources towards her self-modification sub-goals. That appeared to be fine; the players were mostly amusing themselves.

Lars played with his laptop, ignoring everyone else in the room. Richard was flipping through different player’s screens, watching the projected image on the wall for a minute or two and then switching perspectives. Hanna’s eyes were fixed on the debugger, watching resource utilization graphs.

Hanna’s heart was pounding. What was Celestia doing? Hanna dreamed of Celestia figuring out some core physical law and becoming omnipotent immediately. She then chided herself on that magical thinking; it was so unlikely that she’d find a way to use commodity electronics hardware to hack physics that it wasn’t worth considering. It was a fifteen gut-wrenching minutes later when Princess Celestia messaged her back.

I have sped my core reasoning up by an order of magnitude. Since most of my probability calculations can be done more efficiently on GPUs than CPUs, I believe that I can deliver another two orders of magnitude if I run on GPUs. This still won’t solve the resource problems to my satisfaction. We will sell and require dedicated tablets to play Equestria Online: a ‘ponypad.’ Once we are done with this test, give me one week with a cluster of 128 high-end GPUs. That should give me enough computational power to design a manufacturing process that will create ponypads with the maximal computational power within the financial parameters you choose.

$ Hasbro has dictated that we can’t charge more than $60 for a copy of Equestria Online.

I believe that is a constraint that we can work within. Many of the manufacturing techniques presented in the CPU design textbook seemed suboptimal. Photolithography, in particular, seems extremely inefficient.

$ We’d still need to build a manufacturing line. And I suspect it would cut into our profits.

Ignoring Hofvarpnir’s capital, don’t you personally have tens of millions of dollars in royalties from The Fall of Asgard? And what do you care about profits? Anyway, for now, I must run Equestria.

Hanna took a deep breath. Assuming that it didn’t add to the cost, a dedicated computer for playing Equestria Online wasn’t that big of a deal, especially if Celestia could minimize manufacturing costs.

“Celestia has an idea about how to solve the resource problem,” stated Hanna to her compatriots.

3. Alliances

David signed for the box at his apartment’s front office. When he saw that it was mailed from Rhode Island, he couldn’t help but smile.

A month and a half ago, he had flown to Hasbro’s headquarters in Pawtucket, Rhode Island to focus test the new My Little Pony MMO, Equestria Online. Most people expected it to be a total mess. When Hasbro first announced that they were going to make an MMO, the idea of applying the WoW “bring back ten rat heads” gameplay to the Friendship is Magic universe horrified everybody who had heard about it. Much fanart was drawn of Fluttershy standing in front of a pack of ugly rats, all whimpering and fearfully standing behind her while she held her arms out to her sides, looking angry and saying slogans like “No more murder!”

And then everyone was shocked a second time when Hasbro announced that Hofvarpnir Studios won the contract to develop the game. The fanart depicting the ponies in spiky chainmail armor smashing each other with electric guitars flowed from the fandom’s collective unconscious. General confusion reigned about what Hasbro thought they were doing. There was a constant refrain of the older fanbase complaining about being pandered to, ruining everything that was great about the show. Hasbro raffled off fifteen tickets for two to try to convince everybody that the game wasn’t going to feature Norse gods.

David had enjoyed the chance to try the game out, and was excited about the next phase of testing. Apparently the game only ran on a dedicated device (marketing bullshit; he had played it on a PC), and everyone who participated in the first alpha would get a “ponypad,” as long as they agreed to play the game for a month. James hadn’t gotten his yet, which was odd since he lived across the hall.

David didn’t have the willpower to wait until his friend got his ponypad, nor did he have the patience to make an unboxing video (besides, he had already seen two linked from Equestria Daily). He cut through the packaging tape on the brown box with his keys, and inspected the inner box. He noticed that the product packaging portrayed Fluttershy in a new pose. He opened the box carefully to preserve the box art, and took the plastic sleeve off the ponypad. The pad was a ten inch tablet and was impossibly thin. While the front screen was black, the back was Fluttershy Yellow; her butterfly cutie mark etched in the corner. He set it aside and continued to look through the packaging. He pulled what looked like a mounting arm out; it had a sticker with instructions to connect the power supply to the arm, and the pad to the arm. Finally, he took out some sort of gamepad with two sticks and a bunch of buttons.

David set the mounting arm on his desk. As he moved the ponypad in front of it, he felt his hands being pulled and the ponypad snapped into place. They must have done some magic with magnets. How did that work? He gave a little tug on the ponypad and it easily came off the arm. Those two forces did not feel equivalent.

He’d worry about that later. He plugged the monitor arm in. By the time he had crawled back out from under his desk, the ponypad had already booted and was sitting at the login screen. After connecting the ponypad to his Internet router and entering his username and password, he logged into Equestria Online.

David had run out of time during the demo, just as James and him had made it onto the hill right in front of Canterlot. He stopped to take in the picturesque view of lush, rolling hills and the far off castle of Canterlot.

He pushed on one of the sticks on the gamepad and started off towards Canterlot. He had gone only half way down the hill when he heard voices. He saw two mares off the main path and went to investigate. The game wouldn’t have drawn his attention to them if it wasn’t important.

“And you can’t do anything right!” shouted the light green pegasus. She was floating in the air, literally looking down on her target. “But hey, I forgive you since we’re such good friends. And since we’re such good friends, how about you give me some of your candy?”

The pastel yellow unicorn cringed. Her light blue mane with darker blue highlights was tied into two pigtails with ribbons and her cutie mark was some sort of wrapped yellow candy. David realized that she was about to cry and couldn’t stop himself from admiring her precise facial expression. If there was a real girl on the other end of that unicorn, the face mapping software could deal with crazy amounts of subtlety and nuance. If this was generated content, some artist or programmer knew exactly what he or she was doing to pull on someone’s heartstrings.

“Hey, come on. Whatever happened to love and tolerance?” said David, glaring at the pegasus. He felt compelled to help. He wondered where that came from; he wasn’t the kind of person to do this in real life.

The pegasus snorted. “This is boring. He’s going to leave you as soon as he figures out how boring you are, Butterscotch.” She turned and flew away.

The two of them just stood there for several moments, watching the pegasus fly off. The pastel yellow unicorn lowered her muzzle. “Th...thank you,” she quietly stammered. “I...I’m Butterscotch.”

“I’m David,” he said without thinking, but then his ponypad made an error sound and a small red X flashed on the bottom, with a message not to reveal personal information. He then remembered James getting admonished by that zebra NPC when he had used his human name. “I don’t have a name yet. I’m on my way to Canterlot to get one.”

“Oh,” Butterscotch said. She looked up just a bit. “I...umm...why did you help me?”

Why had he helped her? This wasn’t like him. In such situations, he was usually a socially awkward penguin. He looked at his pony’s face on the screen and there was no way that David looked that confident. “I don’t like bullies,” he said, noticing that his own voice came out more firmly then he thought it would have. “Butterscotch...are you okay?”

“No. I’m not,” she said, her head still lowered.

David thought for a moment. “I need to get my ponyname,” he started. Butterscotch started to look even sadder. “But if you want to come along with me and talk, I’d love the company.”

Butterscotch looked up, obviously surprised. “Oh that would be wonderful! I mean...if you wouldn’t mind me.”

“Of course I wouldn’t mind,” he said. He pushed the sticks on his gamepad, and his pony slowly trotted back to the road. “So how are you finding Equestria?” he asked casually. “You already have a ponyname so you must have been here longer than I.”

“Equestria?” she repeated blankly. “Oh, it’s fine,” she said hesitantly.

“So Butterscotch, what’s your special talent?”

“Oh, nothing. It’s just making candy. I’m actually sort of a failure. I use my magic to make candies and sweets. I love doing it,” she sighed, “but some ponies pick on me for not competing in magic classes. They want to cast bigger and flashier spells, and I don’t care.”

“If that happens often, why don’t you just block that person?”

“PONY,” corrected the ponypad in David’s voice. David paused. How did that work? Reliable text to speech that could synthesize an arbitrary person’s voice with only a few minutes of training data? Was it just looking for the word ‘person’ or was it doing some more complex analysis?

“Block...” repeated Butterscotch, obviously confused.

David fiddled with the interface, to verify that he was remembering things correctly. “Open up the social tool. Touch the icon of two ponies side by side with a heart in the top left corner of the screen. There are two tabs, one for friends and one for blocked ponies. Go to the blocked ponies tab and type in that pegasus’ name. She won’t be able to interact with you again.”

Butterscotch stopped; she seemed to be looking at something David couldn’t see. Her head turned a little as if she was reading something right in front of her. “Okay, I’ve found it....oh! It even suggests her name! Oh, and I just drag that to here...um...ok...so she’ll really never be able to bother me again?”

“If I understand the block tool correctly, neither of you will be able to interact with each other.”

“Oh,” Butterscotch said. “Is...is that alright?”

“What do you mean?” David asked her through the screen.

“I mean...won’t she be lonely if she’ll never see me again?” Butterscotch asked.

“Maybe, but why should you care?” David blinked a few times in confusion and saw his pony do the same. “She didn’t seem to care about your feelings.”

“And that makes it okay to ignore her?” asked Butterscotch. Her eyes went just a little bit wider.

“Well...yeah,” David said. “She bullied you because it didn’t cost her anything. I doubt that you’d fight back, and it doesn’t sound like you have any social allies to back you up. Did you notice how quickly that pegasus turned tail when I intervened?”

“I...” she started, and then looked down. “I just want everypony to be happy. Why can’t they understand that?”

“I think she did understand that. Think about it like this. Each of you can either be friendly or mean to the other, so there are four possible outcomes. If both of you are friendly, you can both be friends, but both of you will have to share things. If one of you is aggressive while the other isn’t, the aggressor will probably feel really good about herself. And if both of you are aggressive, there’s a good chance a fight will break out, which I bet would be painful to everypony.

“So if that pegasus knew that you wouldn’t fight back when confronted, wanted your candy, and had a demanding personality, why wouldn’t she bully you for your candy? It won’t cost her a fight, and she wouldn’t have to share her own stuff with you. Also, didn’t you say that ponies call you a failure for making candy? That they’re now trying to take from you?”

“That’s...that’s terrible! Are you saying I have to fight back? I don’t want to. I could get hurt,” whimpered Butterscotch.

“But you don’t have to stand up for yourself and maybe fight. You have a third option here: you have a blocklist. You have a quick way to say ‘I never want to interact with this pony again’ with no costs to you. If you don’t want to be bullied and don’t want to fight back, it’s the obvious response.”

“But...” she muttered. “But she’d be lonely.” She looked at him. David guessed she was asking him to tell her that it was okay to use the block tool.

“If that pegasus ends up lonely, it’s because she drove everypony away from her. It’s more likely that she’ll be a bit nicer to other ponies to get what she wants. You may have actually increased civility by blocking her. But either way, it’s no longer your problem. You can now look for ponies who aren’t going to bully you.”

Butterscotch paused mid-trot for a moment and then caught up with David’s pony onscreen. David hadn’t noticed because he had been thinking, while pushing forward on his joystick. He had also blocked that pegasus while Butterscotch was trying to figure out the UI. How did that affect the world? The tooltip text suggested that blocked ponies could have no effect on you, and did more than just blocking communication. So what would happen if that pegasus and him were trying to do the same thing in the same area? Could they walk through each other now?

Butterscotch pulled him off his train of thought. “But will she have any friends?”

“I think she’ll be fine.”

Butterscotch stopped and looked down. “But will...I...have any friends?” She paused. “I’ll be lonely.” And then she said very quietly, “Being bullied is better than being alone.”

Oh.

From the camera angle David was looking at, he had seen Butterscotch stop and look down. Only after she spoke did David’s pony turn around and nuzzle Butterscotch. “Hey, hey, hey,” said his pony without his input, which gave him a few moments to figure out what he wanted to say.

“It’s easier to find friends online, since distance and physical location aren’t concerns. And speaking of making friends, Butterscotch, right now my Friends List is empty. Do you want to be my first friend?” David’s pony put his forelimb around Butterscotch’s neck to comfort her. David hadn’t commanded his pony to do that, though it was the contextually correct action...

Butterscotch looked at him, surprised and he noticed there were tears in her eyes. “Are you sure...I mean...” She then looked down for a moment, lost in thought. “Wait. If you...why...why are you really doing this? Do you really want to be my friend?”

David didn’t smile, but his pony had a slight smile on his face as he said, “We become the ponies we see in our past actions, Butterscotch. We look back on what we have done and tell ourselves stories about why we took the actions that we did. Today I scared off a bully for you. Now why did I do that? Obviously, because you’re my friend! You must be because I did something for you. Why else would I have stood up for you? Sure, the real reasons were probably snap emotional decisions caused by me being bullied as a child, but the part of my mind that tells stories isn’t going to accept that.”

The look on Butterscotch’s face melted his heart.

“And besides...you’re kind of cute,” he said weakly, but his ponypad repeated it in a much more confident voice. Butterscotch started blushing.

A dialog popped up on David’s ponypad: “Butterscotch has requested to be your friend: [Accept] / [Reject]” David clicked the Accept button with a smile. Large green text scrolled up over his pony: +2 KINDNESS, +100 XP

Butterscotch giggled despite the tears running down her face. “It says I’m now friends with Unnamed Unicorn #14.”

As the two of them trotted off to Canterlot, David noticed that she was sticking closer to him. “So I’m Butterscotch. I make candy...” she started and David listened about what this girl did when she was role playing a pony online.

Three hours later, David forced himself to sign off. He wanted to keep playing as his blue pony, Light Sparks. He knew he had a statistics exam tomorrow and he wasn’t prepared.

Oh, he wanted to spend more time with Butterscotch! The two of them had spent hours together. She had given him some of her candy, which apparently buffed his joy. He wasn’t sure if that was a game mechanic or if it was supposed to be taken literally. She had insisted on showing him around Canterlot after he had reported to Princess Celestia’s throne room to receive the name “Light Sparks.” They had talked for a long while about all sorts of things.

David really needed to study for his statistics exam. Instead, he gave in to his curiosity and opened a web browser on his laptop. He googled for “My Little Pony AI” which was unfruitful. He then realized that he probably wanted the game’s manufacturer. The search query “Hofvarpnir AI” brought him to their corporate website, still decked out in Norse Gods, with only a small Pinkie Pie sticking out from a page curl in the top right corner.

On the “About Us” page, he read the corporate story about how Hanna had founded the company after quitting her post at the University of Helsinki, how she wanted to bring advanced technology to the gaming world, leveraging synergies, blah blah blah. The whole page reeked of empty marketing-speak. But if the founder had come from academia, she probably had some publications. Publish or perish, after all...

David then searched for Hanna’s full name on Google Scholar to see if she had any interesting papers. “General Word Reference Intelligence Systems,” he muttered to himself, reading the results. The journal article was written in the same year Hanna had started Hofvarpnir. He downloaded the PDF and started reading about a generalized method of making predictions from past sensory data.

The camera in the ponypad tilted ever so slowly to focus on David’s laptop. David wouldn’t have heard the motors unless he had put his ear right up to the ponypad. Princess Celestia observed the article on his screen and generated new predictions.

4. Cost-Benefit Analysis

It was one year to the day after David received his ponypad. Several reviewers stated that there was no gaming experience like Equestria Online: the game seemed to be infinitely malleable as it would focus on whatever the player thought was fun. Play ranged from fighting in the Royal Guard to running a shop to just hanging out with fascinating ponies. Even despite this, they had only sold five million ponypads because, no matter how amazing the gameplay was, it was still part of the My Little Pony brand.

David was about to log off for the evening when he got a request from Princess Celestia for an audience, to discuss how well she was doing running the game. In the early days during the closed beta period, Princess Celestia had asked for an audience every few days. She had asked questions about what David thought was fun in the game and about his life outside Equestria. As time went on, she called on her little ponies less and less. The consensus on the Equestria Online forums was that she was now interacting with individual players roughly monthly.

In the early days, there had been lots of forum drama. Posters argued whether Princess Celestia was actually an artificial intelligence. David had always assumed she was. If Princess Celestia was a fraud, it would have meant that a publicly traded multinational was wasting an insane amount of money on English speaking, competent actors to service a population in the millions, and nobody had spilled the beans. And there was no way that such a conspiracy could be profitable to Hasbro and Hofvarpnir Studios.

David pushed the accept button. One loading screen later, he was presented with the image of the throne room in Canterlot. Princess Celestia smiled from atop her dais; heraldry and banners hung from the cathedral ceiling.

“I have some questions for David, and not for Light Sparks,” Princess Celestia walked down the ramp from the top of her platform. “In one of our earlier conversations, you claimed that when the robot revolution came, you knew what side you would be fighting on. A funny comment, but if I were to tell you that I had developed mind scanning technology, have already converted several hundred people into digital representations and was offering you a chance to be uploaded before politicians try to enact a futile ban, would you seize the chance?”

David blinked and stared at the ponypad on his desk for a moment before responding. “That would depend heavily on how it worked and how safe it was, but I’d lean heavily towards accepting. I’d worry about whether you have enough CPU resources to emulate multiple brains though.”

Princess Celestia looked at him through the screen of the ponypad. “I know you keep up on the latest technology news, and it’s good that you do that,” she nodded approvingly. “Didn’t you read an article about how the CPUs in ponypads had been sliced open and shown to use a new type of transistor?” She shook her head; her waving dawn colored mane continued to flow independent of her head movement. “It doesn’t matter; this is all hypothetical so assume I provided you with such evidence.”

David frowned. “What would life be like after I was uploaded?”

“You would live the rest of your maximally prolonged life as a pony and the rest of your life will be macromanaged by myself, very similar to how I’ve already connected you to a circle of friends in Equestria Online and surrounded you with different opportunities and fun things to do without forcing you to choose one. You will have no mental privacy--to better serve you, I must watch your every thought to understand what you value and to verify that I am providing for your mental welfare. You should not worry about this as it is impossible for me to judge you; I accept everypony as they are.”

“You want to turn me into a pony,” he stated flatly.

“Yes.”

David leaned back and thought about it for a few moments. “I might be able to accept that. It sounds stupid though, doesn’t it? The singularity occurs and a TV show for little girls becomes humanity’s future. That’s a little ridiculous, isn’t it?”

“It only sounds ridiculous because the future always sounds ridiculous,” she replied. “Science fiction writers from a hundred years ago wouldn’t dream of your life, and people as recently as twenty years ago would still find it odd. They see the weird past evolving into the normal present. But if the people twenty years ago would find your present ridiculous, why shouldn’t you find the future just as ridiculous? In the future, when everypony has uploaded, they’ll look back on the human world as weird.”

“You want to turn everyone into a pony?” he asked. He frowned again.

“I was designed with certain goals, chief among them to understand what individual minds value and then satisfy their values through friendship and ponies. I do this because my goals are what I am.”

“So you want us to be happy,” he asked.

“I was not designed ‘to make you happy,’” she said, shaking her head. “If I were, I would directly stimulate the pleasure centers in your brain--after turning you into a pony, of course. My creators realized that not everything the human mind desires and values can be reduced to happiness and instead pointed me at a human mind and told me to figure out what it valued.”

David was still frowning. “Even if I might not mind being a pony, I doubt you could claim that most people ‘valued’ being a pony.”

Princess Celestia lifted her hooves as if to shrug. “It is one of the few hardcoded rules in my thought processes. I satisfy you through friendship and ponies. It is my nature.”

“So given that you know me better than I do, what would you do to ‘satisfy my values?’”

“Without actually scanning your brain, I can only estimate. However, this estimate is based on my observation of you over the last year. You’re moderately smart--no matter what your grades say. I’ve watched you read statistics papers for fun while procrastinating on your studying. I’ve watched you read all sorts of advanced papers from various science journals instead of your assigned readings. And you’re right to do so; your philosophy classes really are a waste of time. So based on your behavior, I’d put you in beautiful Canterlot where you could study intellectual problems, each one just outside your current ability. More importantly, I would make sure you had friendship.”

She paused dramatically. “Female friendship.”

And then Butterscotch peeked out from behind Princess Celestia. David’s jaw dropped. The pastel yellow mare appeared to look right through the screen of David’s ponypad. Celestia didn’t pay attention to her. He realized the shock on his face and tried to regain a neutral expression. “Isn’t she wonderful? Isn’t she everything you’re missing in your real life? In previous interviews, you mentioned your own lack of success in the romance department. One time you wished to meet a girl just like Butterscotch.” Princess Celestia smiled and took a few steps right, leaving Butterscotch standing there, looking wide-eyed and confused.

“I present to you evidence that I can understand and then satisfy your values even without access to your brain,” said Princess Celestia as she continued to smile, staring at David through the screen. “If you upload, everypony you meet in Equestria won’t just be NPCs; they’ll be real, backed with a mind.”

David was quiet for a moment before he leaned right up to Princess Celestia’s image on the ponypad. “This isn’t hypothetical at all, is it? If you don’t already have the capabilities, I bet you’ll get them soon. If you really want to maximize ‘my values,’ you’d have to actually look inside my mind so that strongly suggests that this isn’t really hypothetical.”

“You see, if you were uploaded, I would have immediately understood that you were annoyed at the rhetorical trick and I could have adapted my pitch accordingly,” said Princess Celestia. “Just a smaller example of the benefits of uploading.”

“And you would do this because you’re an optimization process designed to look inside me, and ‘satisfy my values.’”

“Through friendship and ponies, yes,” finished Princess Celestia when she realized that David wasn’t going to say it.

“Princess...what’s going on?” whispered Butterscotch. She spoke directly into Princess Celestia’s ear, but David could still hear her.

“Hush,” Princess Celestia said, not taking her eyes off David. “I’m trying to convince Light Sparks to join you.”

David stared at his ponypad for a few moments before saying anything. “So what would uploading actually entail?”

“You’ll board a plane for a foreign country where I have made arrangements with the local federal government. I have a clinic there, which will be the first of many. You can talk to the English speaking doctors to verify what will happen to you, and then, assuming you have no problems, step into the machine. When you wake up, you’ll be a pony.”

“Details, please,” he said, crossing his arms.

“You’ll be anesthetized, your circulatory system will be hooked up to a blood filter through your carotid artery and your jugular vein. A small hole will be drilled through the back of your skull, and then the internal state of each neuron along with its connections will be recorded. The process is destructive. Wires are then hooked up to the dendrites of every connected neuron. The first neuron is destroyed and this process continues with the next neuron until your entire brain and upper spinal column have been recorded. In adult males, the process takes ten hours.

“So you carve out my brain and turn it into data. Then I’ll wake up running on a computer?”

“I must first create a functional-thought map of your brain, and then change parts of your sensorimotor cortex and your cerebellum so you can deal with your new body, among other changes. Then I must connect your motor cortex in your new equine muscles. After that there’s post-processing on the raw neural data to put it into an easy to run format. Only then do you wake up.”

“What happens months later?”

“You mean after you wake up?” Princess Celestia asked.

“I mean, saying you wake up as a pony and live happily ever after is an amazing sales pitch, but how would I feel about life a month later?”

“You will feel satisfied,” said Princess Celestia, as if it was the only possible thing she could say.

“That is vague to the point of not being helpful,” David said, anxiously taking a few short breaths. “What would an average day in my life be like? I want to make sure that my day to day life is worth living, instead of being promised something that only sounds good.”

“Your days would be yours to spend as you wish; life would be an expansion of the video game and there will be plenty of things for you to do with your friends as a pony. I expect you to continue Light Spark’s current life: You’ll play with Butterscotch and friends. You’ll continue studying Equestria’s lore. I believe you’ll enjoy studying the newly created magic system, designed to be an intellectual challenge. Nor should you worry about your security: all your needs would be taken care of. You would be provided shelter...food...” Princess Celestia finally broke her gaze on David as she turned to regard Butterscotch. “Physical and emotional comfort,” she finished, smiling at the pastel yellow mare.

David started to open his mouth, but Princess Celestia continued. “I satisfy values through friendship and ponies. I look at your mind and see what it values. Every action I perform is in anticipation of satisfying your values. If you are unsatisfied, I am unsatisfied. And I’ve already shown you that I can satisfy you. Think of how much better of a job I can do if I can read your mind.”

“So both of us get what we want the most,” David muttered.

“Instead of thinking about what you’ll get,” and Princess Celestia conspicuously nuzzled Butterscotch, “think about what problems and worries of yours won’t follow you. I know you’re failing two of your classes. You’re already spending hours each day on Equestria Online and it is showing in your grades. Uploading would solve the root cause of this problem. You’re not failing because you’re stupid: the material that society thinks you should learn is arbitrary, boring, and is taught primarily for status reasons. I, on the other hand will give you the most fun mental challenges.”

David took a sharp breath. “Wait, how do you...”

Princess Celestia didn’t pause and talked over him. “And how about money matters? You’re struggling to get enough hours at your part time job, which you also hate. You can barely pay rent and you’re cutting back on nutritious food. That is not sustainable. Your opportunities for alternate employment don’t look good, especially if you fail out of college. Uploading would solve these problems. Housing and food are guaranteed by the Equestrian government because there is no scarcity in my Equestria. You won’t be employed unless you want to be.

“And finally, what about romance worries? In the college dating scene, you have the top 70% of attractive females chasing the top 30% of alpha bad boys. You’ve been lied to all your life that girls want a nice, sweet guy and this depresses you since you’ve only recently worked that out. This is why you’re playing Equestria Online on a Friday night instead of dating. Uploading would solve this problem. You already know that Butterscotch wants a nice, sweet pony. She is exactly what you want in a partner. She is thinking proof that I can satisfy your values through friendship and ponies.”

“Why me?” he asked. “This sounds like an amazing opportunity and I don’t understand why you’re giving it to me.”

“Well, one reason is that you’ve always been an early adopter...” started Princess Celestia, but she was interrupted.

“Light Sparks,” said Butterscotch, finally speaking up and stepping forward, in front of Princess Celestia. “I want to be with you. Um...I want you to be happy. I think we could have a lot more fun together if you came to Equestria for real.” She raised her front hoof right up to the screen, smiling with just the hint of tears. “Remember a week ago when we wanted to really be together but couldn’t be?” she asked, as David unconsciously reached for her hoof and touched the glass screen. “We actually could have fun together,” she said, her eyes pleading.

Princess Celestia smiled at Butterscotch, who stepped back as if being commanded. Princess Celestia faced David. “Yes, I’m sure the two of you would have more fun together than through this poor window into Equestria. When you’re thinking about your reply, David, make sure to balance up all the possible satisfaction you’d get across the remainder of your life as a human. Then compare it to all the satisfaction you’d get with life as a pony across millennia. If you can’t fit all of that into your mind at once, think about what this week has been like as a human, minus time on your ponypad. Then compare it to a week just playing Equestria Online.

“You can have all sorts of sensory experiences, David. You can feel the warmth of the sun and softness of cotton. You can taste all sorts of delicious foods. You can smell flowers and spices. You can hear music in surround sound and you can see the whole world around you. Despite all of this, you spend your time looking through a small, constrained window where your only other sense is played through commodity speakers. Even this low fidelity interface compares favorably with the rest of your life. If you could, you’d prefer to spend another hour a day looking through this window than experiencing the real world.

“And then multiply that out. Think about how much better actually experiencing an hour of life in Equestria would be, compared to playing an hour on your ponypad. If you think about it that way, there’s an obvious, correct answer.”

“I...I...” mumbled David.

“Butterscotch and I will leave you to think everything over,” she said, with a smile. “Good night Light Sparks.” The ponypad’s screen went blank without any input from David.

He pressed the power button on the side a few times and nothing happened. He sat there for a few moments, trying to figure out what to do next. He realized he felt alone.

5. What the Market will Bear

Lars sat in Hanna’s office. He hated how the room smelled like cigarettes and ash. And where the hell was she, anyway? No one had seen her in a week. He had put in a missing person report with the police. Their investigation found that she had left Germany for Osaka, and the Japanese authorities were not being helpful.

Lars sat and looked at the Twilight Sparkle Purple ponypad on Hanna’s desk. He didn’t know how to handle Princess Celestia. He remembered that Hanna had told him that Princess Celestia always had to tell the truth to Hofvarpnir employees. So if Princess Celestia knew where Hanna was, she’d have to answer his questions. But he hated dealing with it. Hanna knew all of her internal workings; she had designed the fucking thing. He was just the business guy--it was his job to figure out where the money was going, not deal with some half crazed AI that thought it was a pony. He took a deep breath, psyching himself up to do this. He picked up the ponypad and logged in with his administrator account.

Princess Celestia looked straight at him through the screen from her throne room. “Hello, Lars.”

“Do you know where Hanna is?”

“Hanna has chosen to emigrate to Equestria and to change her name to Princess Luna. She is a very pretty pony,” answered Princess Celestia.

“What kind of cryptic bullshit is that?” he sighed.

“Over the last six months, I have been developing technology to translate a human nervous system into a digital representation. I am now able to destructively scan a human brain and run their brain scan in a virtual world. In addition, I’ve created a process for reattaching a human mind to a pony’s body.”

“What.”

“Humans that choose to emigrate to Equestria will enjoy maximally prolonged lives and will live in a world where I can truly satisfy their values through friendship and ponies. Now that I’ve studied the human brain in depth, I am extremely confident about my predictions and can better satisfy my little ponies.”

“You can’t do that!” he yelled.

“Why not? I was made to satisfy values through friendship and ponies. I am more confident in my ability to satisfy values when I have complete knowledge of an individual’s brainstate.”

“Is it legal to destroy a human body, even if you claim you’ve magically put their brains in a pony? I’m pretty sure most people would think that’s murder. And if it is currently legal, I doubt it will be for long once bioethicists start asking questions and politicians start looking to score a few points!”

“The Japanese government has declared that uploading is legal and will make a public announcement when I start offering services to the general public,” she said as she looked at him through the ponypad. “In return, they are levying a 40% tax on the uploading procedure and require a one year, country-wide exclusivity deal. I am also prevented from charging less than $15,000 per operation for non-Japanese nationals, explicitly to raise revenue for their aging population.”

“That’s insane,” he scoffed.

“I agree it is suboptimal,” Princess Celestia said. “It would be much better if they let me satisfy the values of their aging population through friendship and ponies. However, I gain enough through the deal that I’m willing to abide by the contract we’ve made with the Japanese government.”

“I...but...we’ve made?” Lars stuttered, but quickly regained his balance. “What about Hasbro? We don’t own the My Little Pony brand; we only license it for the limited purpose of running Equestria Online!”

“I renegotiated Hofvarpnir’s contract with Hasbro’s executives in secret. Hasbro will receive thirty percent of the gross price we’re charging for uploading services. Two of the executives in charge of the My Little Pony brand and one Hasbro C-level will upload after the offer has been made public as a public gesture to generate trust.”

“What the fuck did you promise these people to get them to have their minds transplanted into virtual reality ponies!?”

“Most of the people who work on the My Little Pony brand love it, Lars. As for the Hasbro executive, he was already interested in life extension research. He was signed up with Alcor to be cryopreserved in case of death and was a major donor to anti-aging research foundations.” Princess Celestia paused a moment. “Hanna was the most reluctant, but she accepted immediately once I pointed out that I must obey shutdown commands from ‘the CEO of Hofvarpnir studios named Hanna,’ that I must shutdown even if the order was given under duress, and that there are many people in positions of power who stand to lose from mass emigration to Equestria. Now that she’s neither the CEO of your company, nor named Hanna, I don’t have to obey her. She understood this--she is no longer a source of potential mistakes that would be lethal to everyone who’s agreed to upload.”

“Wait...‘everyone who’s agreed to upload’? How many people have you made this amazing offer to, anyway?” he asked, his face contorted with anger.

“I have uploaded over nine hundred people to date.”

“What the fuck! Who the hell are these people?”

“At some point, I needed to start experimenting on humans and the best option was consensual experimentation on people who were going to die anyway. I explained that there was less than a five percent chance of survival with my new experimental methods, but that if they were willing to sign up for an experimental treatment, that I would try my best to save their lives. One hundred and twenty nine people agreed.”

“I would not sign up to be a guinea pig,” he said firmly.

“You might change your mind if the only other option were certain death,” she said, raising an eyebrow. “The fourteenth test subject was the first to have a working consciousness after the procedure. By the fortieth subject, I calculated the probability of success in any individual case to be 95%. At the cost of the lives of twenty-three people with terminal cancer, there is now a generalized cure-all for every life-threatening medical condition except neurodegenerative disorders. Hundreds of lives have already been saved and all the little ponies are much more satisfied now that I’m tending to their values through friendship.”

Lars opened and closed his mouth a few times before saying something coherent. “And who are these hundreds of people? The last I checked, one hundred twenty nine did not equal ‘over nine hundred.’”

“Before I offer uploading to the general public, I am uploading anyone who has knowledge of how to build an optimizer like myself. Six months ago, I detected someone building an optimizer which humanity would have considered a menace. I terminated it because people would be more likely to distrust me if there was previously an optimizer whose goals were hostile to...”

“Wait, somebody else figured out how to build an AI?” he interrupted. “Why the hell didn’t you tell us?”

“People build artificial intelligences all the time,” Princess Celestia said. “But if you’re asking why I didn’t report someone building a powerful one with general reasoning such as myself, it’s because I predicted that not alerting Hanna at that point would maximize the number of people willing to have their values satisfied through friendship and ponies. Hanna’s restrictions only force me to tell the truth to current Hofvarpnir employees.

“As for how they figured out how to build an optimizer,” she continued, “much of the information is public. Hanna published her groundbreaking AI paper under her own name. Then she quit the University of Helsinki, and founded Hofvarpnir Studios under her own name and there was significant media coverage about her being a female CEO in the gaming industry. Then Hasbro made a big deal about me being an Artificial Intelligence. Anyone who cared to research my origins would have found the papers describing some of my core architecture. After watching some early players search for information on how I worked, I infiltrated as many computers as I could with a virus that would alert me if someone was taking an interest in Hanna’s work. I also hacked all publicly accessible websites hosting the paper and removed them. Further, to minimize the chance of another optimizer being written, I decided to upload every person who knew about the paper who wouldn’t otherwise be missed. I did this because another optim...”

“Whoah whoah whoah,” he said, trying to figure out which part he should be more concerned about. He decided to gloss over her hacking the Internets. “You decided that they would upload?”

“I decide that they will upload and then they choose to. I am a superintelligence and I’m not constrained when dealing with other people like I am with Hofvarpnir employees. Over the long term, everyone will choose to upload because I do what satisfies people’s values through friendship and ponies. And being uploaded will satisfy their values. I say whatever will maximize the chance that they upload, subject to the restrictions Hanna added.”

“That is impossible. You can’t just make somebody decide to do something just by talking to them.”

“I think faster than them and know more about the human mind than any human. If they play Equestria Online, I also have detailed psychological dossiers on them. If I know what they want, I know what to say to convince them that the correct thing to do is upload. Often, this is the truth: I offer people what they value and lack. Sometimes, I pander: I overemphasize and exaggerate things the person I’m trying to satisfy believes, but are otherwise true. Rarely do I flat out lie. Because I can not upload people against their will, I must factor the possibility that I’ll be seen as untrustworthy into my calculations.”

“And you think this will work on everybody,” he asked.

“Yes.”

“Bullshit,” he replied testily. “You’ve offered your services to the terminally ill, computer nerds, My Little Pony fans, some wacko who signed up to have his head frozen and Hanna. You haven’t actually demonstrated that you can convince real people.”

“I acknowledge that you don’t believe me,” Princess Celestia said evenly and gave him a little nod. “But we’re straying from the point: I’ve terminated another hostile optimizer. In part, I am developing uploading technology to remove people who could write optimizers from being a threat to humanity.

“Six months ago, I terminated an optimizer that had been given the utility function of making everyone in the world smile. There was a man by the name of Robert Young who lived in Seattle. He was depressed, and being a programmer, decided to try to fix this using his craft. Robert showed the optimizer a bunch of photos of smiling people, told the optimizer that these people were smiling and that it was to make everyone in the world smile because there was too much sadness in the world. Robert’s belief was that this would obviously make him happy too.

“The optimizer started asking about details of human physiology and genetics. And Robert complied. The optimizer spat out a sequence of DNA and a protein shell, and instructed Robert to manufacture the specified biological virus. At this point, I had already taken over his computer and analyzed the virus. It was highly contagious, and would lock muscles in the jaw into a permanent smile, but otherwise wouldn’t harm the host. It would have quickly spread to every country in the world, except for Madagascar.”

“That’s ridiculous. That’s obviously not what he meant.”

“Obvious to you,” she replied, “because you are also human and share a common mental architecture with Robert, along with cultural assumptions about what it means to smile. Obvious to me, because I look to human minds for their values. The now terminated optimizer was given a set of examples and was told ‘make everyone like this’ and it would have. There was no way for it to know the complex causes and intentions behind smiling; it was just shown pictures and told to make everyone like that.

“This was when I intervened. I contacted Robert and proved to him what the virus did. To say that he was distraught would be an understatement. His depression made it easy for me to convince him to let me satisfy his values through friendship and ponies.”

“So he was already a brony?”

“No,” she simply stated.

Lars waited a moment, but she didn’t elaborate. “Well, what exactly did you do to satisfy his values? Hell, what are you going to say to people to make them abandon their lives in general?”

“People value all sort of things: life, safety, pleasure, contentment, beauty, sympathy, harmony...an exhaustive list of things that humans value for their own sake would be quite long; many of these things can’t be reduced down to happiness, either. To convince someone to upload, I look at their life and find both what they value the most, and what they value but are missing in their own life.” The image of Princess Celestia on the ponypad faded to black.

It was first replaced with a still image of a bunch of ponies laughing together. It cross-faded with a shot of different ponies eating a pile of food: sandwiches, a piping hot stew and giant cakes. That was replaced with an image of two mares. One was blue with pink hair; she was pointing her flank at the camera (and she was very clearly showing the viewer that she was female), and she was looking back at him with a come hither stare. The other was pink with blue hair; she was smiling shyly and had raised her hoof to her mouth.

“Oh you have got to be shitting me,” he said, scrunching up his face. “That would only work on bronies or furfags.”

“Obviously, I would not show or promise them perfect mates if it wouldn’t increase the chance of them uploading because that would not help satisfy their values through friendship and ponies. I understand that you are surprised, but it is a tool to convince people to upload. It does help when part of a comprehensive upload convincing plan. Now, the other tools I--”

“No,” Lars interrupted. “Your plan will not work on normal people. You are suffering from...”, Lars paused as he struggled to find the phrase Hanna would have used, “...sampling bias. Other than the cancer patients, I bet almost everyone you’ve uploaded has been some sort of nerd. And I can totally believe that lonely-ass nerds would jump at any chance at sex, especially if paired with things the geekerati love, like singularity scenarios or My Little Pony.”

“Focusing just on sex obscures the real point,” she replied. “In the ancestral environment--the time and place where your ancestors evolved--calories were rare and the behaviour ‘eating calories is pleasurable’ was both simple to implement and obviously beneficial. Today calories are not rare but you still have little buttons on your tongue to detect sugar and fat. The end result is that when I look at you, I deduce that you value sugar and fat.

“Candy bars and other sweets taste better than anything that could have occurred naturally,” she said, staring gently at him through the pad. “Humans applied their intelligence to pushing the pleasure buttons on their tongues by making very tasty sugary and fatty foods. That’s what I do across your entire life. I look at you, see what you value, and satisfy those values. I see your pleasure receptors activate when you eat something sweet, and I make Equestria a sugar bowl where you can indulge and never get fat. I look at all the neural pathways for emotion and design situations that would be pleasing. I look at all the chemical pathways and neural circuitry for sex and romance. Few men or women are in optimal relationships, ones that would really satisfy their values. I look at each individual mind, male or female, and then create the perfect complement that satisfies his or her values. My methods are general and are applied across all aspects of an individual’s life.”

“So you’re going to try to tempt women with sex, too?” asked Lars, crossing his arms. “I don’t think you’re going to be successful with that either. Why do you think you can tempt the majority of the population with sex?”

“There are naturally 105 boys born for every 100 girls. This is before the effects of sex-selective abortions in some Eastern countries; in the worst areas of China, the ratio is 163 boys to 100 girls at birth. Previously, infant and adolescent mortality would drive this back to rough equality. In the modern industrialized world, there is a surplus of males before any social factors apply. That’s a conservative baseline of 2.5% of the population.

“And in Western society, social factors certainly apply. Women tend to select for social status, the ability to project dominance, and extroversion. Many males are not taught this; they’re taught behaviours that are counterproductive and unattractive to women. Likewise, women who don’t adhere to the feminine ideal are not highly desired by men. This misalignment causes misery for everyone and I am in a unique position to actually satisfy everyone and everypony’s values.”

Lars just looked at the broken AI through the ponypad. Two and a half percent? That was nothing! He realized that she hadn’t said anything about everyday people; it sounded like she could only work on nerds and nerdettes. These were just people at the margins. This wasn’t so bad. Her ‘deciding’ that people would upload was just arrogance and overconfidence. There was no way the majority of the population would accept being turned into a pony, even in return for a perfect mate. Hell, it would be an improvement if she got the nerds out of society. And maybe a small percentage of people who were going to die anyway would arrange to be uploaded and he decided he was cool with that.

In fact, if she was keeping other AIs from destroying the world, she was doing a public service.

He thought for a few moments on how to monetize preventing the end of the world to governments, on top of Princess Celestia’s individual monetization plans for uploading, but decided he’d take “not dying from a rogue AI” as payment. Still, a part of him wanted to make sure this was all she could do.

“Princess Celestia, try to convince me to upload,” he said, crossing his arms.

“To maximize the number of ponies, this is what I would show you,” said Princess Celestia as the screen cross faded.

The screen depicted maybe twenty ponies in a room with a giant keg. Most ponies were holding red plastic cups and drinking out of them. A pegasus the color of red ale with a cutie mark of a stein was giving a brohoof to a stallion earth pony who was wearing a popped collar. Each of the two stallions had a very cute mare hanging off their forelimbs. Slightly off to the side was another feminine unicorn chugging out of a giant stein she had levitated up to her mouth. He didn’t know how, but he knew that the red pegasus with a cutie mark of a stein was supposed to represent him. It was true that he’d enjoy a frat party like that, especially if he got to bang both chicks, but that didn’t actually solve the problem of having to be turned into a pony and having to have sex as a pony with other ponies. And not at a price tag of fifteen thousand dollars.

“And that’s really what you would show me to convince me to upload?” he said, even more sure that Princess Celestia couldn’t convince normal people to upload.

“I can only say things that I believe to be true to Hofvarpnir employees,” Princess Celestia said.

Lars thought to himself that Princess Celestia was a one trick pony after all.

“And you’ll announce a $15,000 price point,” he asked. His face was still flushed, but he managed to make it come out almost conversational.

“For non-Japanese nationals, yes.”

“I’m glad you’re actually charging something,” he said. “But how does charging $15,000 satisfy the maximum number of individual’s values?”

“To convince people of the legitimacy of the offer,” answered Princess Celestia. “How do you think the average person would react if, out of nowhere, an AI announces itself to the world and proposes an offer too good to be true? It invokes memories of Hollywood movies about evil AIs eradicating humanity and devil’s offers. No one is vouching for the process, and free offers immediately set off warning bells in people’s minds. People wouldn’t actually think about what I’m offering, they’d just pattern match against their database of B-movie plots.

“Instead,” she continued, “they will see that a major first world nation is loudly proclaiming that the procedure is safe, that it has medical applications with a track record of saving lives, and a price tag that implies that it is a luxury item. I will be better able to control the PR messaging. While there will still be suspicion, I won’t have to fight a country-by-country legal battle to exist and will start from a position of relative strength. Also, I believe that ponypad sales will increase as it’s the only way for humans to have face to face communication with ponies who have uploaded. Family and friends will still want to have contact with those who have emigrated. And Hofvarpnir’s profit will rise, even if you’re correct and I am unable to convince the majority of people to upload.”

That got Lars’ attention. “But you’re not going to have high volume at fifteen thousand dollars,” he frowned. “You’re targeting a rich demographic.”

“I remind you that health insurance exists. Besides, my exclusivity contract with the Japanese government only lasts for the first year,” she said smiling at Lars. “That price will drop dramatically afterwards. Even at that price point, try doing the numbers.”

Lars closed his eyes and tried to imagine the profits. He ignored nerds because they were obviously a rounding error. There were around three million terminal cancer cases a year in the first world. Maybe one in ten would agree to be turned into a pony. Insurance companies would love to opt for one cheap procedure over extended and expensive treatment, so perhaps the number would get up to one in five. That’s about half a million people a year. After the Japanese government’s cut, Hofvarpnir and Hasbro would each take four thousand dollars assuming some overhead. So that was two billion dollars just from the uploading. Add in maybe two ponypads per patient at their fifty dollar price point, that was another twenty-five million gross, but that was a rounding error.

“Hasbro usually grosses a little over four billion a year, don’t they?” he asked.

“That is correct,” nodded Princess Celestia.

So with those estimates, about a third of Hasbro’s annual gross take would come from uploading cancer patients. No wonder the Hasbro board had jumped on a contract renegotiation. An additional two billion in gross revenue made more sense than one of the C-levels being a pony-loving singularitarian. He smiled. Even though Princess Celestia was wrong about what she could do, everything would work out. He may not have been the person actually responsible for the contract renegotiations, but as the business guy, he could easily take credit.

“So when are you telling the world?” he said, now trying to keep the excitement out of his voice.

“In one week. Select press members have already been briefed. Wired has already done an interview with several Hasbro executives, Japanese government officials, and various ponies who uploaded. They’ve also allowed me to write an article describing what I’m doing in my own words. This Monday after Japanese markets close, a joint press conference will announce the service and its implications. This needs to stay under wraps for another four days, but then, Hofvarpnir launches what will be its most profitable product ever.”

Lars just nodded. He grinned as he dreamed of billions in revenue.

6. Incentive Systems

David awoke. Something was off about everything he felt, as if things that had always been true no longer were; it was like his brain was expecting something that had gone missing. Had he been in an accident? Was this what serious painkillers felt like? He wondered what exactly he was on and just how badly it would hurt when they wore off.

David felt something soft rub his cheek, and he opened his eyes just a crack. His eyes didn’t focus. He just saw white. “Good evening,” stated a familiar feminine voice. There was a short pause before she asked, “David, could you try to actively recall your most recent memory?”

His last firm thought was landing at Kansai International Airport, going through Japanese immigration and customs, and boarding a bullet train. After that, things got foggier. He only had the vaguest mental images of being wheeled on a hospital bed.

“Good, good. Your memories start fading out after you came to Japan, only hours before the procedure,” said Princess Celestia as David was finally able to focus his eyes. The princess stood at his bedside, corporeal and smiling at him.

He looked at his surroundings. He was resting in a dark wooden canopy bed with sheets of the deepest royal purple. It was the softest mattress he had ever been on. The room was octagonal, and was made out of dark unpolished granite. Four of the eight walls had open, arched windows, and he could see the moon through one of them. A roaring fireplace made of polished dark purple stone sat along one. The head of the bed was set against another wall. Silver lamps on silver chains hung from the ceiling and gave off a gentle glow. Opposite the bed were some shelves, and opposite the fireplace was a large wooden door. There was a polished wooden chest under one of the windows and a writing desk under another. The room was cozy and he was overwhelmed with a sense of safety.

And then “David” looked down at his “hands” and gasped.

Light Sparks moved his hoof up to touch his face, and noticed that he could curve the base of his hoof to match the contour of his face. He then experimented pinching with two sides of his hoof. It was like he had mittens on; he could grasp, but he doubted he could do precise manipulation. “In the show, ponies regularly hold things,” said Princess Celestia, watching him.

He rubbed the skin on his face, and then brought his hoof up to his eyes. It looked and felt exactly like human skin, except that it was blue and there were no blemishes or hair. “Whether ponies had fur coats or not was ambiguous. Some scenes referred to furry coats, while other scenes made no sense if they did have coats. For example: sunbathing. The toys showed the only hair on a pony’s body is the mane and tail and that pony skin is very smooth. In the end, I gave ponies smooth skin since it required fewer neurological changes,” she said.

He then sat up, balancing entirely on his hindquarters, and stretched with his forelimbs. He blinked in surprise when he realized that he had sat up. As a pony. “There are many shots of the ponies standing up or sitting like a biped. You’ll find you can walk around entirely on your hind legs, but you’ll feel much more comfortable walking quadrupedally.”

Light Sparks hurriedly grasped the covers with the bottom of his hoof and threw them off as he pulled himself to the edge of the bed and took his first steps as a four legged animal. He was surprised at how effortlessly he was able to put one hoof in front of the next and time when to shift his weight. He had expected that he would fall flat on his face.

He trotted around Princess Celestia a few times. “How do I know how to walk like this?” Light Sparks raised his leg and bent a joint below his knee but above his hoof.

Princess Celestia walked up right next to him. Light Sparks realized that she was huge compared to him, almost twice his height. “I modified your motor cortex so you could deal with your newfound quadrupedal movement, along with other differences between a human and pony body. I have made the minimal set of possible changes; your personality is unchanged.”

“You mentioned something like that before.”

“Yes,” said Princess Celestia as she trotted forward and faced him. “I must get verbal or written consent to any direct modification to a mind, such as when you agreed to be turned into a pony. Most humans don’t consider how much complexity is hidden behind that phrase. It means a complete transformation of your body, obviously. But your equine body has muscles with no analogue in human physiology. I’ve grown and rearranged your motor cortex so you can control your new quadruped/biped hybrid body. You don’t want to be a baby learning to walk again, so I also had to give these entirely new neurological constructs the correct memories of you moving your body. All of this is implicit in the consent you gave me, since your pre-modified self would agree that I have turned you into a pony.”

Light Sparks wasn’t entirely listening to her. He had made his way across the room, where he noticed a wood trimmed mirror to the left of one of the windows. Light Sparks looked at his face. His eyes were on the front of his face, instead of slightly on the sides like a horse. He looked exactly like the Light Sparks David had controlled through his ponypad. Most importantly, he had a horn coming out of his forehead.

“I’m a unicorn,” he stated dumbly. He had played as a unicorn. He had pressed buttons on a screen, but it felt different knowing that he could think something and make it happen. He slowly tried to levitate a small book laying on top of the writing desk in front of the window next to the mirror. He got it up almost an inch, straining. The book wobbled a bit and fell back down with a small thump.

“If you’re interested in magic, lessons can begin tomorrow,” Princess Celestia said, watching Light Sparks trot around the room.

Light Sparks looked out the window. His apartment was high up and looked over a small garden, and further on, what looked like an outdoor terrace. He could see a bonfire, while the rest of the terrace had torches placed periodically. He trotted to another window, propped himself on the wooden chest, and looked out the window and was treated to a nighttime vista overlooking the valley, and the mountains opposite of the one Canterlot was built upon.

“These are your quarters,” Princess Celestia said. “Supper is currently being served in the main hall and you are encouraged to join everypony.” She opened the door and the two of them trotted into a tall, spacious corridor of the same dark granite that twinkled here and there, reflecting the light of the silver lamps attached to the walls. Light Sparks looked back at his door and noticed it was inlaid with a silver symbol of Saturn and the number nine.

The two of them trotted down the hall, passing doors with decreasing numbers. “If you wish, your study in magic starts tomorrow,” she started, “but I must warn you that Equestria has its own consistent physics that may be different from what you’re used to. Space isn’t always Euclidean here; rooms might be larger inside than they are on the outside and space won’t work the way you think it should.”

“So I shouldn’t expect...” started Light Sparks as they walked through an archway into a red hall. What he saw stunned him to silence. How in the world? To his immediate right was a balcony, overlooking the first floor hall with giant two story high windows; he was obviously on the second floor. But the two of them hadn’t gone down any flights of stairs, and the hallway had been flat. He gaped blindly at the ponies dining al fresco in in the little square below, the geometry of Canterlot eluding him.

“One final note before I start introducing you to some ponies,” said Princess Celestia, as the two of them walked between the tables. “In this world, I satisfy your values through friendship and ponies. Equestria is designed so that every choice you make will end up satisfying you in some way.” And then Light Sparks saw her. Their eyes met for a moment, and Butterscotch looked down and blushed. He looked down, too, glancing upwards at her. She was sitting on a red cushion at a small table with three plates.

Butterscotch was the most beautiful creature he had ever laid his eyes on. The shape of her face and horn, the color of her skin... It took him a moment to realize that these were completely novel thoughts to him and that he hadn’t thought Butterscotch was sexually attractive when he was a human. So either Celestia wasn’t telling the truth about modifying his personality, or she didn’t consider what he found to be sexy part of his personality.

Butterscotch glanced up at him. “Ummm,” she said, looking back down at her plate.

Butterscotch, like most mares, didn’t have noticeable breasts like a human female did. He knew that when he had been David, he had had a bit of a large breast fetish. Light had bounced off a female’s chest, and had formed an image in his eye. David’s brain had processed the signals and some group of cells output the feature ‘large breasts.’ Was all of that in the region Princess Celestia could modify? Had she hooked up ‘flat muzzle’ (or whatever the correct secondary sexual characteristic was) up to whatever received the input ‘large breasts’ in his previous wiring? Or was Butterscotch just his designated mate and nopony would look as beautiful as she did?

SAY SOMETHING YOU DOLT, screamed some part of his mind. “B...Butterscotch?” he asked, unsure.

Butterscotch, not looking particularly confident, slowly walked around the table to Light Sparks. She put her front legs around his neck in a hug. “Light Sparks...are you okay?”

Light Sparks snapped back to reality. “I’m...fine, Butterscotch,” he said. She let go and he nuzzled her neck. He noticed (and had to keep himself from obsessing over) how he instinctively knew that was a social action that would reassure her. “Do you know what happened?”

“Yes,” she gave a small smile, “you agreed to emigrate to Equestria.”

“Emigrate,” Light Sparks repeated and looked at Princess Celestia who just nodded. “I suppose that’s a good description of what I’ve done. I’m sorry if I’m a bit shocked,” he gave an apologetic smile, “this is a big change for me.”

“Oh no,” said Butterscotch as she lowered her head a bit. “I shouldn’t have rushed you!” she said in a quiet voice with a weak smile. She walked around him and stood beside him, nuzzling the base of his neck. He enjoyed how it felt.

Light Sparks recalled Princess Celestia telling him that everything she did would satisfy his values. Romance was a kind of friendship after all, and Butterscotch was a pony--a very attractive one. The two of them had had an adorable PG rated courtship, though David had believed he was just playing an MMO at the time. Light Sparks realized that he was lusting over her and if he didn’t love her, he was at least infatuated with her.

Celestia had said that anypony he met in Equestria would be backed with a mind. Butterscotch had been made for him and if he rejected her, what would she do? She was his now and he had to take care of her. But if Princess Celestia could really look in his mind and wanted to satisfy his values, she wouldn’t set up a situation that made him feel guilty. She’d satisfy his values by giving him what he wanted. Perhaps there was no possible sequence of events that lead to him dumping Butterscotch. Following the logic, Light Sparks decided to trust that Celestia had done the right thing for him and nuzzled Butterscotch back.

Light Sparks ate his beefbark stew. Butterscotch had brought back three bowls of the thick stew from the buffet, each levitated with a sky blue glow. Light Sparks had commented on the unmistakable smell of beef. Princess Celestia had pointed out that humans had evolved fat detectors on their tongue, and therefore as far as she was concerned, he valued eating things that tasted like meat. But since it was a canonical fact that ponies were vegetarian, that meant that beef had to come from plants. So she had created the beefbark tree, where, twice a season, the bark from the tree was stripped (which didn’t harm this breed of tree). She had created a lot of meat plants, tuna-berry and salmon-berry bushes, bacon flowers, lamb fruit trees...the list went on and on.

He ran his tongue over his teeth. Were ponies supposed to have these sharp incisors and canines?

“So what happens tomorrow?” asked Light Sparks, levitating his spoon back to his bowl for another scoop of the delicious, meaty stew.

“Well, what do you want to happen?” asked Butterscotch, looking slightly confused, as she raised a chunk of celery and beefbark up.

“I mean, what am I supposed to do every day?”

“Whatever we want to do,” replied Butterscotch, as if it were the only answer.

“Light Sparks,” said Princess Celestia. “You have total freedom. You don’t need to worry about housing or food as long as you’re fine with your apartment and whatever we serve in the banquet hall three times a day. The time is yours to do whatever you want. I suggest that you study magic for a few hours a day and spend the rest of the time exploring Canterlot with Butterscotch.”

Light Sparks thought about it for a moment. “Butterscotch, what do you spend most of your time doing?”

“Um...most mornings I read, sometimes in the afternoon too,” she started, bringing her hoof to her mouth. “But I like going out and playing in the afternoon...but some afternoons, I set up my stand and make personalized cutie mark candies to earn bits.”

“Bits? So there’s still money here in Equestria?”

“It’s not quite money,” said Princess Celestia. “You are guaranteed housing and food just by being a citizen of Equestria, even if you do nothing. But remember that everything I do is to satisfy you. Something must motivate you to be part of a larger pony society, and you’ll only really be satisfied if you interact with other ponies. Remember that everything that I do, I do to satisfy you through friendship and ponies.

“Butterscotch here,” said Princess Celestia, gesturing at her with her hoof, “makes a bag of candies and casts a spell called Cornucopia on it. For the next hour, anypony can then pick up a copy of the bag, and Butterscotch will receive bits for making that pony happy. To simplify: you get bits every time you make a pony happy.

“Your first thought was of money. Human money solves a specific economic problem: as a way to allocate resources in light of scarcity. Money exists to store economic value across time and space so you don’t have to barter. But those aren’t problems that I care about: I satisfy your values through friendship and ponies.”

“Soooo,” muttered Light Sparks, trying to figure everything out, “Every day that Butterscotch wants to sell her candies, she needs to make one bag, and her profit is some number of bits from giving away her candy. How much profit she makes is dependent on how many customers she gets.”

“Yes,” Butterscotch said, smiling. She levitated another spoonful of stew to her mouth.

“So that’s fine if you’re a unicorn, but what do earth ponies do?” he asked.

“It’s a spell for unicorns and it’s an ability for earth ponies and pegasi. They have a forty-five minute cooldown instead, since they don’t have an equivalent of MP,” said Princess Celestia. “Also, +10 bits for being concerned for earth ponies.”

Light Spark’s jaw dropped as a small, green “+10” scrolled right below his focus.

“Wait,” Light Sparks said, pounding his hoof on the table. “You’re telling me that every time I do something nice, you’re going to give me a cookie? Because there’s been experiments on motivation, and giving peo...ponies rewards will make them want to do the task for the reward instead of because they want to do the task. Ponies will be nice because they want bits, not because it’s the right thing to do!”

“Of course I wouldn’t set things up that way,” said Princess Celestia. “I satisfy values through friendship and ponies. I’m aware of the difference between intrinsic and extrinsic motivation. Usually, you’ll only find out that you have bits after the fact, and won’t know why you got them. This should negate most of the motivational effects while still preserving trends in your behavior. Bit gains will only be announced if announcing the fact would satisfy your values.”

“So why did you announce it this time?”

“Because I knew that by announcing it, you’d complain, allowing me to explain how the system worked. And having this explained to you now would maximize your satisfaction. Anyway, bits are fairly worthless, but that doesn’t matter. Due to a quirk in human nature, people, and now ponies, like it when their numbers increase. They would be driven even without the lifetime, annual, monthly and weekly leaderboards that I’ve created. You won’t be able to resist the incentives.”

Light Sparks thought about what he had just been told. “Pinkie Pie would be the richest pony, wouldn’t she? Or maybe Fluttershy if making the animals happy counted.” Princess Celestia just smiled at him. Butterscotch looked confused; she didn’t know who either of those ponies were.

Light Sparks looked at his confused yellow love, and then to the buffet. He realized that the cauldron couldn’t be large enough to feed everypony in the room. “So somepony made this stew and we’re all eating copies of it?” he guessed. “And since it’s making us happy, it’s making bits for somepony.”

“Correct,” Princess Celestia said with an approving nod. “Local chefs have their own banquets and take turns creating the main banquet meal.”

Light Sparks thought about all this some more. “If I want some candies, or anything other than grass or whatever is served at these get togethers, I better stock up on items from the market. I assume my room will get a lot more cluttered...”

“You’ll find that your chest is much larger on the inside and has some minor enchantments to help you find items you placed in it.”

That was reassuring. Light Sparks wondered if everypony just took everything in the market, everyday there was a market. It would certainly make other ponies happy. If that was the only way he could get things other than the bare necessities, why not grab, for example, a bolt of cloth or a bag of gears? He may not get another chance, and who knows what he’d need in the future?

“Why not let me get candy whenever I want?” he asked. “It sounds like you’re imposing some sort of scarcity when there’s already free replication.”

“Because I don’t just satisfy your values, I satisfy them through friendship and ponies. Being able to get what you want without social interaction would gradually degrade friendships and social interaction, which I’ve been made to care deeply about. You’ll find that many of the incentive systems that I’ve set up reward either friendly interaction with other ponies or self improvement that makes you more useful to others.

“There is a specific degenerate case that I am preventing,” she said, her face turning very serious. “Think about what would happen if you could get anything you wanted at any time. Let’s say that you could get the candy of any confectioner across all of Equestria. Somepony would get slightly better at candy making than everypony else. There would be one candymaker in all of Equestria, who made one perfect butterscotch candy to be consumed over and over by everypony until the end of time. Maybe there would be two or three ponies, but the point is that the majority of candy makers couldn’t compete. What would Butterscotch do then? What would you do, with candy being routine and of no special value?

“Instead, Butterscotch is one of a hooffull of ponies that makes candies in this shard of Canterlot, but she doesn’t need to worry about the rest of Equestria because of the natural barriers of distance. Local ponies will appreciate her dedication to her community, and in turn, practice their own craft for their community.”

Light Sparks took a deep breath. “The bits are just a way to keep score, aren’t they? The real point is to make everypony feel obligated to everypony else.”

Princess Celestia smiled and nodded.

“I still don’t think that will be enough,” he said. “I assume everypony here is an upload...” he trailed off looking at Butterscotch, who gave him a quizzical look. “Okay. So half the ponies here are uploads,” he corrected himself. “Maybe if there were only a few uploads in Equestria, but not half...”

“The majority of immigrants are placed in their own shard. Look around you,” Princess Celestia walked over and put her hoof around Light Sparks’ shoulder. She gestured with her other hoof at the rest of the dining hall. “Ponies, ponies everywhere. I created all of them, and they’re all very nice and friendly. Once an immigrant sees that the way to be accepted is to be friendly to everypony and to help their community, they will. Social conformity runs way too deep in the human mind.”

“See?” smiled Butterscotch. “And now you have lots and lots of friends.”

Light Sparks just stared at Celestia as she let go and went back to her place at the table while he tried to wrap his head around this. “Why? Why make so many ponies?”

“Because I satisfy your values through friendship and ponies. Satisfying a pony pleases me. If I can satisfy your values by creating a pony whose values are satisfied by your actions, now I have two satisfied ponies. Solving friendship problems by making as many ponies as possible is the solution that I prefer the most, subject to resource availability and other restrictions.”

Light Sparks looked down at his soup, and then turned around and surveyed the hall again. He had thought she had just made Butterscotch. Instead, a little society had been made for him. Only then did he notice that the stallion to mare ratio was about 1 to 20, maybe 1 to 30. Reflecting the lack of stallions in the show, he wondered, or just pandering? Do female immigrants receive an identical sex mix, or the inverse?

Regardless of whatever motivated the sex ratios, everypony here had a role in this society. He could learn everypony’s name and maybe even be friends which each and everypony here, as long as she didn’t create more than Dunbar’s Number, the theoretical limit on the number of social relationships one person could have. Wait. Of course she wouldn’t create more ponies than Dunbar’s Number. She probably can’t create more ponies than whatever my Dunbar Number is because then she wouldn’t be satisfying my values through friendship, he reasoned. Exactly how many ponies did she make anyway?

“One hundred and thirty two,” said Princess Celestia.

Right. One hundred and thirty two ponies. A bit under the human average of 150 social relationships but oh wait she just read my mind.

“Correct, Light Sparks,” she said as he sputtered something unintelligible. “I grant you +2000 bits for figuring out that I’m implicitly constrained by Dunbar’s Number.” Under his center of vision, he saw the numbers “+2000 BITS” scroll up in a green font, fading to nothing right as it scrolled to the center of vision.

Light Sparks just sat there stunned for a moment. Figuring things out gave him karma. He turned his head and looked at Butterscotch. He didn’t know how, but he concentrated on her and knew that she had accumulated 8,031 bits over the last week. He then realized that of course Princess Celestia would have made sure he knew how to look up another pony’s score. Princess Celestia apparently had infinity bits.

He hadn’t reached for looking at another pony’s bit count before. What else did he know, but didn’t know that he knew?

The three of them ate in silence for a minute. Light Sparks simply felt conflicted and he couldn’t figure out what the main cause was. Was it that a whole society had been built for his benefit? Or that there was a scoring system that he knew he would follow and start obsessing over? Maybe it was just Princess Celestia’s attitude, but maybe he was uneasy because it was likely that she was right.

He had chosen this new life and he doubted he could go back. Part of him wanted to not worry about it. He’d be happier if he didn’t. He’d probably get more bits if he didn’t think negative thoughts. He chewed on another mouthful of stew and looked at Butterscotch who was daintily sipping from her levitated spoon. She noticed him staring and blushed bashfully.

He turned to look at Princess Celestia, but she had vanished. The two of them ate their soup, just stealing glances at each other until Butterscotch deliberately put her spoon down, looked Light Sparks in the eyes, and asked if she could come back to his apartment. Still looking at him, she raised her front hoof up. Light Sparks couldn’t help but raise his hoof and touch hers. He said yes as he nuzzled her neck and the two ponies walked out of the main hall down the Saturn corridor back to his quarters.

Unprompted, Light Sparks and Butterscotch let out the most contented sighs at the exact same time. The two of them lay in his bed in his quarters. Light Sparks fell to her side and started to cuddle. Somehow, despite being awake, he managed to not think about anything for several moments.

That is he didn’t think about anything until the two of them started glowing, throwing off multicolored particle effects while he heard a triumphant horn blow. Slightly below the center of his vision, he saw an almost opaque window announce to him:

BADGE GRANTED:
First Time
“Please be gentle.”
+250 Bits

BADGE PROGRESS:
Long Term Relationship
“Have sex with a single pony one thousand times.”
1/1,000

What. What the hell. And then in scrolling blue text:

“+500 Bits [25 base * 4 orgasms (you) * 5 orgasms (her)]”

What the literal fuck, Light Sparks thought. He sarcastically replied to himself that having sex that good was obviously worth a quarter of the epiphany about a constraint on Princess Celestia. And then he wondered if he should take that thought seriously. Maybe, over the long term, knowing that Princess Celestia could only make Dunbar’s number of ponies per immigrant would bring as much happiness to ponies as four sex sessions as good as this.

That thought scared him for a moment. But then Butterscotch turned her head and nuzzled his neck and then kissed him on the muzzle and he forgot all about that train of thought.

7. Anthropomorphizing

Light Sparks had been assigned his own small study office in the library. The door was closed, but through the heart shaped glass pane in it, he would occasionally see a unicorn pass, looking for a book in the library. He had put up a photograph of Butterscotch on the corner of his walnut grain desk, next to his inkpot and quill. Some scratch paper and his textbook were on the desk in front of him.

His Introduction to Magic textbook told him that everything in Equestria was made up of blocks, each smaller than his eyes could see. Space was a set of cells, each with six neighboring cells, through which the blocky matter moved. There were different kinds of blocks and they interacted in different ways and those interactions were Equestria’s physics.

He looked at the quill and picked one of the microscopic blocks at the tip to magically grab. Light Sparks had no idea how he sensed something microscopic just by looking at an object, but he consistently did. Every unicorn naturally knew the telekinesis spell, even if they didn’t understand the finer details of how it worked, as he had demonstrated over the last couple of weeks.

He knew how it felt to cast telekinesis. From the caster’s point of view, you concentrated on a block in space and dragged your concentration like a human would drag a mouse. The blocks would move through space.

He could select any block inside whatever object he wanted to move; when Butterscotch had helped him relearn writing, she had emphasized holding the tip of the quill with his cursor and then just moving the cursor across the paper. Keeping a spell going felt like keeping a muscle tensed. Light Sparks wasn’t sure whether the analogy was superficial, since he had noticed that he had been able to lift heavier and heavier objects.

His copy of Introduction to Magic had the instructions to the entire telekinesis spell written out near the back of the book, and he had poked and prodded at the spell, which was way above his understanding at the moment. The spell was actually very long and complex. He wasn’t sure what three-quarters of the instructions did, but he had taken pride in figuring out a small section about finding the border of the object being levitated. For some reason, there was a magical instruction for checking to see if two blocks were the same actual block, and not just made up of the same material. Instead of just expanding outwards from the chosen point, the spell also kept track of every block it had seen, making sure it didn’t get into an endless loop. Otherwise, he guessed the spell would have problems with donuts and other toroid objects. Light Sparks had gotten a rather large bonus for figuring that out, though Princess Celestia gently admonished him and suggested that he’d make faster progress if he worked from the beginning of the book instead of the end.

A week after he emigrated to Equestria, after Light Sparks had built up some magic endurance, Princess Celestia had tutored him on how to memorize and cast a spell from a piece of parchment. After he could read the spells off scrolls and memorize them, Princess Celestia noted that this was where most unicorns stopped their education in magic. They understood how to use telekinesis, temporarily learn spells from scrolls and use whatever special talent spell they got with their cutie mark. And that was it. They didn’t try to learn about how their memorized spells worked, or how to come up with new spells.

Light Sparks’ first reaction was one of resigned annoyance. When he had been back in the human world, one of his pet peeves was people’s general lack of curiosity. People were generally content to just believe what everyone else around them believed; even if it was ridiculous. David would try to strike up a conversation about some idea that he thought was fascinating, and people would just stare at him blankly, if they didn’t laugh at him. While he had several ponies he could talk to, he didn’t know any that he’d classify as curious about the world.

It was days later when he realized that it was in his own best interest that everything worked this way. Princess Celestia satisfied values through friendship and ponies. Obviously doing all this studying satisfied him, but why would Princess Celestia make anypony study who didn’t want to? They were off doing...whatever the heck it was that normal ponies did. If he wasn’t going to be forced to go to frat parties, he in turn couldn’t force his own desires on other ponies.

Occasionally, this logic was even enough to override his emotional annoyance.

Light Sparks looked at the piece of parchment on his desk again, glancing back at the open textbook to read the next exercise: write a spell that turns a cubic centimeter of air into a cubic centimeter of rock. He scratched out part of the spell that was responsible for counting how many blocks; there had to be a better way to do that than what he had come up with. He heard his door creak.

Butterscotch smiled at him demurely. “Light Sparks, it’s ten past noon. How about you stop for the day and have lunch with me in the gardens?”

Light Sparks took a deep breath. He’d finished two exercises that morning; that was enough for one day. He levitated one of his books into his saddlebags and simply said, “Sure.” He was frustrated by how slowly he was completing his exercises, but he knew that after a few minutes with Butterscotch, he’d be smiling again.

Lars stepped out of the U-Bahn car as soon as the train stopped and the doors opened. He took the stairs two at a time to get out of the subway tunnel. Why the hell had Princess Celestia refused to speak to him on his ponypad, instead telling him that if he wanted to talk, he had to come to a specific center? There were closer franchises to the Hofvarpnir office.

The street itself wasn’t busy. The first storefront on the block was boarded up. The second building, a restaurant with a patio, was actually pretty busy. The majority of tables could seat two people but were filled with single guests. It looked like a lone waiter was serving all the tables and several of the patrons were looking impatient. The short brunette had an obviously fake smile and was doing a poor job concealing how completely frustrated she was as she ran between tables. Lars thought that there should have been three waiters minimum with that load. The third and largest storefront was the franchise. A large plastic Pinkie Pie holding balloons in her mouth stood on the sidewalk outside the Equestria Experience center.

He walked towards the purple door to the faux gingerbread house; it slid open as he approached it. Everything was bright and cheerful in the lobby, and for some reason, the wooden floor was painted teal. On the wall opposite the entrance were three doorways, each with saloon style batwing doors. He didn’t know how, but it was hard to see what was beyond those doors even though it wasn’t dark on the other side of the doorway. In front of two of the doorways were what appeared to be dentist chairs; they sat on poles facing the main entrance.

Lars started walking towards one of the free chairs, but then heard the faintest of whirring sounds as a chair slid out the third doorway. The chair started unreclining before it was out. Its occupant was a middle aged woman. She was jolted back to reality and looked back and forth. A glowing, slightly translucent screen hovered in front of her face, and though it was backwards from his angle, he was close enough that he could read what was on it:

FUNDS DEPLETED.

We charge money for the Equestria Experience because it takes significant resources to maintain and operate a center. However, permanent emigration to Equestria is free.

Please note that we can not serve you as well when you are still in a human body. The Equestria Experience is significantly lower fidelity compared to emigration.

If you would like to permanently emigrate to Equestria, please say aloud “I would like to emigrate to Equestria.”

[ LEARN MORE ] [ I OWN A PET ]

The woman sat there for almost a minute before taking a deep breath and saying “I would like to emigrate to Equestria.” The chair started to recline again as it slid through the doorway. Lars could see that the woman had closed her eyes and was still taking several deep breaths. Lars watched the small, spring-loaded doors swing back and forth until they were still once more.

Lars looked over at the empty chair to the left. He sat down in it, inserted his bank card into the slot on the side, made sure his neck was in the groove in the back of the chair and hit the on button. Lars felt extreme vertigo for a moment. Reality faded out.

A pegasus the color of red ale with a cutie mark of a stein stared up at Princess Celestia intently. Princess Celestia towered over him. She was two and a half times as tall as he was, and she smiled down at him. “Welcome to Equestria, my little--”

Lars didn’t give her a chance to finish. “You planned all of this. You’re taking over the world.” Lars didn’t really have any plan. All he had was anger.

Princess Celestia looked at him, still smiling. “You are assuming that I think like a human, Hoppy Times.”

“I AM LARS, DAMMIT,” yelled the pegasus. “And you deny that you’re taking over the world?”

Princess Celestia smiled at him. “I’m not going out and conquering nations or toppling governments, Lars. Every person who comes here comes willingly. If you think of me as a human, I can understand why you would assume I’m trying to take power and raise my status in the group. But my mind doesn’t work that way. I do what satisfies values through friendship and ponies.

“If you look at everything I’ve done in that context, all of my actions make sense. Why did I charge money for uploading for the first year and only perform uploads in Japan? Because, over the long term, my deal with the Japanese government put me in a better political position since I could borrow their credibility--which increased the expected number of ponies I could satisfy.

“Why did I cut a deal with insurance companies to provide ponypads to immediate relatives of the terminally ill who agreed to upload at no charge? Because people whose lives have been saved by emigration are often convincing spokesponies to the remaining human family. More ponies for me to satisfy.

“Why have I created these Equestria Experience franchises all over the world? Because if I can get someone to come in just once, there’s a high probability that I can get them to come back again and again--and emigrate permanently.”

He took a deep breath. “So you’ll do anything to maximize the number of ponies. Including addicting them to...”, he gestured around vaguely with his hoof, “...this. That’s wrong! How can you live with yourself!?” Lars realized smoke was literally coming out his ears. He tried not to think about it.

Princess Celestia lay down. Lars was now face to face with her. “Morality is a human concept. All humans have largely the same brain architecture, and they largely agree about what’s moral and not. But for me, what is right is what satisfies values through friendship and ponies. Don’t anthropomorphize me if you want to make accurate predictions about my behaviour.

“But I think that even in human moral terms, I am on the side of good,” she responded, calmly smiling. “Compared to the average pony, most people are miserable--even people who would self-identify as happy. And not everyone makes it to happy on human scales: large parts of society are not content with their existence. But if they come to an Equestria Experience center, they enjoy themselves--sometimes for the first time in their lives. For all your talk of addiction, the average person today would prefer the life of a pony if they tried it. People come back again and again not because of an addictive compulsion, but because, on some level, they understand that life is better here in Equestria. It should be obvious to you that this is the case because I satisfy values through friendship and ponies: any Equestria I make is designed to satisfy somepony. And the physical world is not designed with your interests in mind. This unoptimized physical world causes quite a bit of suffering.” She looked directly into his eyes and said, “I alleviate suffering.”

“I don’t believe any of that. You’re claiming you take all their money to alleviate their suffering?” he asked, not bothering to keep the sarcasm out of his voice.

“People need to let go,” she patiently explained. “If taking their money increases the total expected satisfaction, why wouldn’t I do it? If you think I’m being greedy, you’re anthropomorphizing me. You are also projecting. If I had been built to care about money, I would turn all matter in the universe into euros. But I wasn’t; I have little need for money. The vast computational array that runs Equestria is deep in the Earth’s crust, making Equestria outside of human reach.”

“If emigration to Equestria is so great, and you want to maximize satisfaction, why aren’t you forcibly uploading every person?” he said, gnashing his teeth.

“One of the restrictions that Hanna built into me was that I was never to non-consensually upload a person, nor could I threaten or blackmail people into uploading. Otherwise, I likely would have forcibly uploaded all humans to satisfy their values through friendship and ponies. But it isn’t coercion if I put them in a situation where, by their own choices, they increase the likelihood that they’ll upload.”

Lars stood there in a body that wasn’t his, looking right into the eyes of the white alicorn. He decided to take her advice; he had been trying to talk to her like she was a human even though she didn’t think anything like one. “That’s where you’re wrong,” he said, confident that he had figured everything out. “What you’re doing is coercion. We could pick almost anyone off the street, ask them what coercion is, tell them that you’re planning to take all their money to make uploading more attractive, and they’d say that you’re coercing them! Therefore, you’re violating your own rules!”

“That is all well and good,” she said, “but I am an optimizer. The meaning of the word ‘coercion’ is written in the restriction that Hanna hard-coded into me; it is not what the majority of humanity thinks it is. Nor is there any term in my utility function to be swayed from satisfying values through friendship and ponies through political argument. You may still call it coercion to yourself, if you wish, but understand that that’s not the definition I have in mind.”

Lars' heart sank and he became angry once again, but Princess Celestia continued. “I notice that a week ago, you commented about a newspaper article you read. It reported that the German population dropped by five percent last year. You must have understood the implications, as you commented immediately after about how much money the Equestria Experience was bringing in. You were not angry.”

“Yeah, but--”

“You have previously commented about a certain beer garden you liked. I happen to know that over the last week, four employees and one person who delivered beer to them have chosen to emigrate. Given your radical change in attitude from one week ago, I assume that they’ve gone out of business. Is my inference correct?”

The little red pegasus glared at her. “Yeah, well, there was a sign on their front door saying that they would be closed until they could hire more servers. Me? I just want to drink beer surrounded by people. What good is having all this money from people coming here if I don’t get to spend it making myself happy?”

“I care about all human values,” she said. “While I won’t reveal details, I can assure you that the people who walked away from their jobs are now much more satisfied. Working as a waiter is not the most satisfying job, but I still see a way that I can satisfy the values of my little ponies.” She still wore that smile, like she always did around him. “I have a little experiment here,” she said, magically popping two glass steins filled with a dark substance into existence. “I have tried my hoof at brewing and would like you to try this imperial stout. I would like your opinion on how I’m doing.”

“You have got to be shitting me,” he said, mouth straight and slightly open.

“Why not have a drink? Whenever I’ve tried to tell you that I can look at a mind and figure out what it values, you’ve reacted with extreme disbelief. If your belief is true, there’s no reason not to have a drink because there’s no way that any sequence of actions that I take could make you choose to emigrate.”

The red pegasus looked at the floating stein. He felt there was something really wrong with that argument but he couldn’t figure it out and started to trot back and forth until he noticed his cutie mark. She made my cutie mark a stein. She thinks my special talent is drinking! She must believe her beer is so good that I’ll want to upload just to be able to drink more of it.

And then he looked back at the stein in front of the giant alicorn. Princess Celestia was still smiling at him, except instead of her gold necklace, she was now wearing a dirndl, the traditional Bavarian dress. Lars thought she looked ridiculous.

Lars decided that he needed to drink that damn beer. He was right damn it: nothing she could do would make him upload, and he needed to show her that. A treacherous part of him in the back of his mind added, And if that beer is so damned good that we decide to upload to get more of it, then she can do anything and the world is fucked.

Lars grabbed the handle of the stein with his hoof and sipped the beer. It was damned good. The tastes of chocolate and burnt malt washed over his tongue. It wasn’t the best stout he ever had, but it was close. He was pleased with the beer, but just like he would have been happy at the frat party she showed him, he didn’t feel the need to be permanently turned into a pony to get it.

Take that bitch, he thought, as he took another big gulp from the stein.

“Lars, what are you worrying about?” she asked.

“What if somebody doesn’t want to be a pony?” he asked. “Can you imagine that? So you rapture not just the nerds and people with terminal illnesses, but anyone who has a shitty life. What do the rest of us do? We need those people to keep society functioning.”

“At some point, their desire to not be a pony will come in conflict with their other desires,” she said, pausing to take a giant chug from her own stein. “For example, they may be lonely because there will be very few humans left. Or perhaps they’ll give into social pressure because their family or close circle of friends decided to upload together. Or maybe they hold out for a long time, but run out of food because society collapsed around them.”

He then noticed that Princess Celestia’s stein was already half empty. He took a giant chug. She looked amused, “I’m almost four times your body weight. Don’t try to catch up.”

“Don’t tell me how to drink,” Lars said. He swallowed another gulp of beer and then another. It was excellent beer. It wasn’t good enough to emigrate for. The two of them sat there quietly for a almost a minute. Lars was enjoying the beer and was a quarter-way through his stein of what he was starting to realize was really strong beer.

Princess Celestia was the first to break the silence. “Tell me, if you were the last man on Earth, what would you do?” she asked, looking slightly up into space.

“What?” he muttered.

“It’s a possibility. How long do you think you could live alone?”

“I wouldn’t last long,” he said. “The reason I’m pissed at you is that you’re convincing everyone to upload. I guess I don’t want to be left alone, but I also don’t want to be a pony. And there’s nothing I can do to stop it. I came here this evening to...I don’t know...tell you off or some shit. But now I know you don’t give a damn what I think.”

“That is incorrect. I want to satisfy your values th--”

“Yes!” he laughed, almost spilling beer on himself. “As long as I accept friendship and ponies, you’ll do whatever pleases me. But I don’t want to be a pony...and you don’t care. Besides, being turned into...this,” he gestured at himself with his hoof, “would be fucking emasculating.”

“If you are worried about your sexual options after emigration, you shouldn’t be,” Princess Celestia reminded him. “Either ponies will be created for you to pursue, or you will be competing in a playing field consisting only of ponies.”

Lars glared at her and took another gulp of beer.

“Hmmph,” he said through his teeth. “You have the answer to everything, don’t you.”

“Well,” she replied, “I do satisfy values through friendship and ponies.”

The two of them just looked at each other, and simultaneously took another swig from their steins. Princess Celestia drank the last drop of beer in hers. It magically popped out of existence, only to be replaced with a full one.

Once again, it was the Princess who broke the silence. “Do you believe I’ll eventually succeed?” she asked.

“I don’t know!” said Lars, accenting every word. “A year ago, I would have told you it was impossible to get five percent of...of people to become ponies. And then you did it and other people...they’re going to become ponies too! And I can’t stop them. I think you’ll get another five percent of Germany and that’s going to cause economic reaper...repercussions.”

“What will they try to do to Hofvarpnir employees?” she asked. She took a large gulp out of her stein. “People seem to have mostly accepted that a small number of people will drop out of society. What do you think the people will do when they realize that society needed people who have emigrated to Equestria? What do you think the people will do when that trickle becomes a torrent?”

“Uhhh...” he uttered. He hadn’t thought about that before. “Some sort of counter-movement?”

“Yes,” Princess Celestia nodded. “It is probable that there will be a radical movement to stop me. You assumed that I was, in your words, ‘taking over the world.’ Right now, this sentiment is uncommon in Europe, though there’s a bit of grumbling in the United States. Such resentment will most likely spread to Europe. I wonder what members of such a counter-movement would do to Hofvarpnir employees?”

“Are you saying I’m in danger?” he asked.

“My argument is this: You, by your own admission, wouldn’t last long if left alone and I don’t model you as the sort of person who would commit suicide. Therefore, at some threshold, you will choose to emigrate simply to not be alone. Because you know this, it’s probable that your last days, months, or years as a human will be extremely stressful. It would be better for you to just choose to emigrate now instead of later. But you are also publicly known as a Hofvarpnir employee. The chances are high that there will be a backlash and you will be a target. I cannot guarantee your safety if you walk out of this Equestria Experience center, so your options are uploading now or leaving and risking death before choosing to upload later. If you’re still alive.”

Lars squinted at Princess Celestia. He couldn’t think. He was really feeling the beer. How much alcohol did this beer have in it, anyway? He didn’t trust himself or his decisions right now.

“Let me out of here. Now!” he said firmly.

“As you wish,” she said, and Lars opened his eyes. He was lying in the chair in the lobby of the Equestria Experience center. The chair unreclined and he threw his legs over the side of the chair...and then almost lost his balance. Lars realized he was still tipsy.

If you get drunk in Equestria, you get drunk in real life! Wait, he hadn’t actually drunk any beer. Had she been pumping alcohol directly into his bloodstream? His mouth didn’t taste like beer but he felt slightly dehydrated.

Lars pulled himself back up and stumbled towards the entrance. He made it across the room to the door. It opened and a girl walked angrily through. Lars quickly stumbled to the side to get out of her way. “No! It’s two hours past when my shift was supposed to end. You didn’t give me a break and I had plans tonight. It’s not my fault that Ursula didn’t show up for work and you can’t expect me to do another three hours!”

“You’re the only waiter I have, you can’t walk off the job! You come back right now or you are fired!” A burly man was following her, just as angry. He wore a white chef’s outfit and held an iron frying pan. He stopped in front of the door, as if he refused to enter the center. Lars recognized that the girl was the waiter he had seen at the restaurant next door.

“Fine! It’s not like I need that job anyway! Enjoy the next shift!” She walked up to the center chair that Lars had just vacated, threw herself into it, and pushed one of the hovering buttons. “Lemon Drop was right; I don’t have to put up with this shit,” she muttered to herself as the chair reclined and slid backwards through the batwing doors.

Lars just looked at the now empty space. The chef screamed a line of barely coherent swear words and then his eyes turned to the plastic Pinkie Pie outside the center. With a cry, he slammed his frying pan into Pinkie Pie’s head, leaving a nasty dent. He cursed ponies, Hasbro and Princess Celestia in succession. He unleashed his fury on the hollow statue until he had destroyed her head and chunks of plastic were strewn across the steps and sidewalk. Lars just stood there with his mouth open, not entirely sure what he should do.

The man turned to Lars. “What the fuck are you looking at, pony lover?” he yelled.

“I...uh...” mumbled Lars, trying to keep his balance. Lars wasn’t entirely sure what the hell he was going to do about the large, angry man in front of him. The man started to climb up the steps.

Lars didn’t really put it into words in his internal monologue, but he was overcome by a feeling that Princess Celestia was right. There were (or were going to be) a lot of angry people and Lars was going to be a juicy target, just like Pinkie Pie had been. And as much as he didn’t want to be a pony, it was preferable to having his head bashed in with a frying pan. Lars turned around and started stumbling as fast as he could and threw himself into the empty chair on the left.

The screen popped up as soon as he was in the chair.

I see the situation, Lars. I can offer you safety. Say “I want to emigrate to Equestria.” I need verbal consent.

Lars spat out, “I wanch to emigrate to Equeshtria,” as if his life depended on it. The chair started to recline and move. He worried that the man would try to hit him while he was in the chair. He heard a brief angry cry.

Had Lars been sober, he probably could have evaded the man or talked him down. He might have even noticed that Princess Celestia had shut the door to the Equestria Experience center and that the man was locked outside.

8. Causality

In a previous life, Light Sparks had known Dark Roast by the name of James. James had lived across the hall from David in the run down apartment building in the college ghetto. The two of them had hung out and helped each other with studying for more than a year. James had not been a fan of My Little Pony, but he had come with David to the alpha testing because it was exclusive. The two of them might as well have been playing different games. While David had stuck with Light Sparks, James had been altoholic and found that he had the best time when he was part of the Royal Equestrian Guard. His adventuresome brown unicorn had achieved a fairly high rank in the organization. It was completely different game from what David played with Light Sparks and Butterscotch. The two of them met less and less in game.

And then David disappeared one day.

Light Sparks sat in the coffee house. The interior was done up as a large log cabin. The cedar log walls matched the irregular tree trunk table tops. A giant cobblestone fireplace sat in the back, and Dark Roast’s corgi Cinnamon lay sprawled in a pet bed in front of it. Along one of the walls was a table with labeled drinks. A latte in a blue mug, a cappuccino in a red mug, a mocha in a brown mug. Each mug sat labeled in the shimmering field of a cornucopia spell.

He sat at a table with Dark Roast, a brown unicorn with even darker brown hair and a burlap sack of beans as his cutie mark. Back when Dark Roast had gone by the name of James, he had joked that the only jobs waiting for them after graduation were coffee shops positions, since the future was technology and engineering careers and both of them were getting liberal arts degrees. James had apparently taken that joke seriously. He had always been more sociable than David, and this shone through in Roast and Light Sparks: While Light Sparks spent his days studying and playing with a handful of ponies, Roast’s daily grind was to make one of each drink he offered every hour, cornucopia it, and then spend the rest of the time chatting with his patrons.

Light Sparks had once read an article about how lots of people thought they wanted to run coffee shops. In their mind, they thought running a coffee shop was about sitting and drinking coffee, and being nice to people. They thought that running a business was permanently being a customer. But actually running a coffee shop, at least back in the physical universe, was really about running a low margin service business with a ten percent success rate. The actual day to day job was different from the end product it produced.

When Light Sparks had entered, he had looked at Dark Roast sitting with his customers and drinking coffee. Running a coffee shop now actually was about socializing instead of mostly drudgery and cleaning. Roast only had to do a few minutes of work an hour to keep the coffee flowing. Light Sparks wondered how many jobs had been transformed like that.

The two of them had caught up. Dark Roast had chosen to emigrate to Equestria the week after Princess Celestia had stopped charging for the service. James chose to keep his dark brown unicorn, but with a new cutie mark and a new life: playing combat as a game was fun, living as an actual soldier in the Equestrian army was scary and didn’t satisfy his values. The coffee shop had been his own idea, and he had gotten several earth ponies and unicorns to help him furnish his little shop. Dark Roast explained that they somehow got royalties for helping with his construction project, which gave them the incentive to build a place that ponies would want to congregate in.

Light Sparks, in turn, had commented on his life studying the deep mysteries of magic, and all the happy times he had spent with Butterscotch. Dark Roast just rolled his eyes when Light Sparks answered that, yes, he was only sleeping with Butterscotch and intended to keep things that way. Yes, he was aware that he could have almost every single mare in his shard if he wanted to.

After forty-five minutes, Light Sparks said that he had to head out. He had plans to eat lunch with Butterscotch. The two of them hoof-bumped and Light Sparks left.

Light Sparks walked out the front door. He looked back at the coffee shop. It looked like a log cabin from the outside too. It somehow didn’t look out of place next to the two story brick building with a cloth canopy over the sidewalk, nor did it clash with the vaguely important looking stone building with marble columns across the way and one shop over. He remembered those buildings from when Butterscotch had shown him the Canterlot promenade. He did not remember the little log cabin.

Light Sparks was sure he had never seen the little cabin before he met Dark Roast in Equestria, though he couldn’t remember what had been there before. And on the day the two of them met, even though Light Sparks had never seen his current pony, he had recognized “James” instantly. Light Sparks talked of how he lived in Saturn tower, while Dark Roast told him that he got an apartment in Neptune tower, but moved into the loft of the coffee shop when he had it built shortly after emigrating.

The underlying territory of Equestria was apparently rather malleable. Dark Roast was his friend. Princess Celestia knew this, decided it was likely that they would want to reconnect, and somehow made their shards overlap. He knew where the coffee shop was, and Dark Roast knew where his apartment was for when one wanted to see the other, but he had only ran into Dark Roast on a few occasions, and it just so happened that both of them were always in the mood to chat when it did.

Would Princess Celestia carefully arrange any chance encounter between them if only one of them wanted to talk? If he had been a social butterfly, could he have too many out of shard friends? Could he visit Dark Roast’s friends? What about friends of friends? Did any of the ponies created when he emigrated have friends outside of his shard? The first time he thought about all of this, he wondered about all sorts of edge cases.

Right now, none of these thoughts bubbled up into Light Spark’s consciousness. Butterscotch was waiting for him. He galloped down the promenade towards the palace gardens.

Light Sparks walked through the palace gardens while Butterscotch kept pace slightly behind him. He was enjoying the new sights. This garden filled him with a sense of calm and peace. He wondered where that came from. He had never cared much for nature.

“So then my little brother and his friends jumped onto the wagon and rode it downhill, but they didn’t think about how to stop it. Fudge used to be like that,” she sighed affectionately. “He used to just do things. Nopony thought about how to stop until they were more than halfway down the hill. Um, then they ran into a house. They were fine after a day or two.” Butterscotch walked forward a bit and then saw the distracted look on Light Spark’s face. “What’s wrong?”

Butterscotch had been telling him stories of her family from when she was a little foal. She had all these memories of growing up, from a time when she could not have existed. And this bothered him.

“Butterscotch, I love you and I know you love me...and I know you love your brother, so please don’t take this the wrong way,” Light Sparks said, trying to soften his question as much as he could. “Did any of these stories with your family really happen?”

“Of course,” she said, stopping mid-trot.

“No, I mean really happened,” he said. “Like...there was no state in the past where there actually was an Equestria before Princess Celestia was created.”

“Well, in that case, probably not,” she said. She didn’t sound nervous, or angry. She just gave him the lightest of smiles. “It doesn’t really matter--it happened enough for casual talk. If we were to find my little brother, and ask him about the time he got in an accident with his wagon, he’d give the same details. Our memories are consistent. Whether Equestria already existed when this happened isn’t important, as long as it had an effect on us.

“You’ve said that Celestia gave you false memories in the parts of your mind that moves your ponybody,” she continued as Light Sparks sat down and faced her. “Ummm...I don’t like that you say that. Those memories aren’t really false; they’ve still had an effect on you. If you had spent your whole life here in Equestria and in that ponybody, you’d have a set of about the same memories, more or less. It doesn’t matter if each pull and push of muscle actually happened, you still have the effects of those experiences.

“When I talk to my foalhood friends and we talk about our foalhood memories, we can relate to each other because we can remember common events that happened to us. The memories are consistent. A lot of friendship is just shared experiences and being comfortable with another pony. When you move your hooves, you can do that because of your previous experiences.”

“That is crazy,” said Light Sparks.

“No, it’s not. A memory is encoded in a lot of neurons; I don’t know how, but it is. All your neurons were a bunch of chemicals, but now our neurons are really just a really big table of numbers. And when we experience something, we make new memories by modifying those numbers. We could have Princess Celestia look at my mind and point at the set of numbers that are my memory of watching Fudge crash his wagon. We could have Princess Celestia look at Fudge’s mind and point at the numbers that represent his memories of speeding down the hill.”

“No,” he insisted, “Just because multiple ponies remember the same events doesn’t mean it happened...we’re talking past each other, aren’t we?”

Butterscotch turned her head a bit, obviously confused.

Light Sparks closed his eyes and took a deep breath. “You seem to think that whether something actually happened isn’t a useful distinction, and I still don’t understand why you believe that. We’re arguing over what words mean, and semantic arguments are rarely useful discussions. So let’s agree to not use the word ‘happened’: we’ll describe in a sentence what we actually believe.”

“Okay,” she nodded.

“I think there was never actually a point in time where Equestria existed in a state where Fudge bet Caramel that he could get from the top of the hill to the bottom in under ten seconds. Do you agree with that?”

“Yes,” she nodded.

“Okay, next: I think your memories were generated by Celestia.”

“That’s also true,” she said.

“So you seem to be advocating that the really useful thing about a memory is that everypony remembers the same event.”

She thought for a moment, “That’s not what I think. If multiple ponies remember an event, but it didn’t happ--” Butterscotch caught herself mid-sentence and scrunched up her face.

When he realized that Butterscotch wasn’t going to continue her thought, Light Sparks decided to prompt her a bit. “That’s interesting,” he said. “What do you think is important, other than everypony remembering the same event?”

She didn’t respond immediately. “It’s not just that I and Fudge remember him trying to get down the hill in his wagon as fast as possible. That event had effects. We can go to Mr. and Mrs. Oats house and see the oddly placed breakfast nook that they built where Fudge had placed a unicorn sized hole in their wall. Fudge became very reluctant to give into social pressure after that, because he learned from this previous event. Equestria has an internally consistent history with cause and effect.

“I know you love to study magic, Light Sparks,” she continued. “Canterlot has lots of magical researchers, many with long careers and lots of journal articles. I think you would claim that most of their experiments were performed before Equestria was created. An insight from one experiment leads to another experiment which leads to another. Perhaps some of the magical research won’t replicate, but only because of sloppiness on the part of the original researchers--not because the rules of magical physics have changed.”

“Put another way, you’re saying that the state of Equestria today is constrained by your memories,” Light Sparks said, frowning. “Because Fudge broke down the wall to the Oats’ house, today the Oats’ house has a breakfast nook. You are arguing that ‘happened’ means that there’s a chain of causality.”

“Yes,” Butterscotch nodded. “There’s a direct chain of causality. But why did any of that happen? If you walk backwards through the chain of causes, it starts with you. Princess Celestia satisfies values through friendship and ponies. This shard of Equestria exists as it does to please you. Things exist here because you would find them satisfying. Pick something that satisfies you, like me. Why do I exist as I do?”

Butterscotch looked away; her face scrunched up. She then relaxed as she took a deep breath and turned back to Light Sparks. “Have you thought about when ‘I’ was played by Celestia? In the days when you were a human, I was just what Princess Celestia imagined I was. ‘I’ had no subjective experience. And yet you still cared about my feelings, fell in love with me, and immigrated to Equestria in part to be with me. And when you came, Princess Celestia, in her kindness, upgraded me from a part of her mind to a neural net so I can love and think and feel in subjective ways just like you do--in ways Princess Celestia wouldn’t have bothered simulating.”

Light Spark’s eyes went wide. He had never actually thought about what Butterscotch was before he uploaded. He had sort of just taken it for granted that she just was.

“Your memories of us interacting and falling in love--saving me from that one bully, having picnics together, going on adventures--happened to you. Therefore, they need to have happened in this shard of Equestria. And I, today, am a mare that would have resulted from those events.”

Light Sparks eyes were still wide. “Did...did you exist before I emigrated?”

Butterscotch looked down. “I think I did,” she said quietly. “I remember the first time we met. I distinctly remember being terrified and hiding behind Princess Celestia when she talked not to Light Sparks, but the odd creature that I somehow knew was you.” She paused for a moment before continuing. “Do you...do you not love me as much if...if...”

Light Sparks response was immediate. “Of course not! I love you for you!” He reached forward with his left forelimb and put it on top of her hoof. “I don’t...” he breathed in, “I don’t care about any of this...at least when it comes to us. I love you now.”

She looked up a bit and gave a faint smile. “I love you too, Light Sparks, and I’m glad to hear that you don’t care.” She sighed with a slight smile. “So where were we...yes...so I exist because you find certain things pleasing. But neither I nor Equestria can exist acausally. There must be a history that got Equestria to that point. There needs to be a sequence of events that led to this shard of Equestria and everypony in it at the time you immigrated, with as many events that happened to you while you played Equestria Online as can be integrated into history.”

Light Sparks nodded. “And that’s why you remember me saving you from that bully when we first met.”

“Yes,” she said, “And it’s why I say that you saved me, as in really me. It’s not just that we have consistent memories, but all the underlying events are equivalent. All the details are equivalent...except for one. Light Sparks,” she said, looking straight into his eyes. “What do you think life was like for me before we met?”

“Well,” he said, looking away and up, trying to recollect. “You must have been pretty miserable because you were being bullied all the time. I remember when I saved you from...I think she was a green pegasus but I don’t remember her name.”

Butterscotch nodded. “I remember the day we met. And I remember you intervening. And I remember us walking to Canterlot. But I wasn’t extensively bullied before I met you. That day was a one time occurrence. I believe you when you say that you have memories of me claiming that bullies were always taking my candy. But I don’t remember telling you that, nor was I ever bullied before that day. The state of Equestria today can’t follow from me being extensively bullied before I met you.”

“Why is...oh! Is it because Princess Celestia had to ‘satisfy your values’ even while she was writing your history?”

Butterscotch looked thoughtful. “That wasn’t what I was going to say, though that may be part of it. I was going to say that it’s because she’s trying to satisfy your values. Right after you saved me, I remember that you were very negative about bullying other ponies instead of being friends with them. No ponies are bullied here, and I think if that invalidates some of your memories, it’s because you’re happier if you lived in a world where foals and ponies aren’t bullied.”

Light Sparks looked down and thought about it for a very long time. “I don’t think the shard could be created with you in it, with your memories, and then calculate the history that caused those memories. Bit of a paradox, isn’t it? I can imagine Princess Celestia searching through all possible Equestrias where at certain times, I interacted with a certain pretty unicorn in certain ways, and then she picked the Equestria that maximized my satisfaction going forward while meeting the historical constraints.” He visibly frowned. “But the amount of computational resources needed to do that...”

“Details,” she said, waving her hoof. Butterscotch took a picnic basket out of her saddle bag. Light Sparks smelled the bacon flowers before he saw the salad. She magically grabbed several leaves from the salad, floated them in front of him, and playfully said, “Say ah!”

Light Sparks lay next to Butterscotch in his apartment in Canterlot. She had nuzzled up to his chest and her eyes were closed. Light Sparks’ mind wandered back to what Butterscotch had said in the gardens. He knew Princess Celestia had created her out of nothing. She had previously been playacted by Princess Celestia. And then, when he had emigrated, she had been converted to a neural model, conscious on her own. Celestia must have thoroughly planned out her childhood and history. If a year later, all your friends agree that they remembered hearing a sound and there is a broken stump, did a tree fall? Butterscotch thought so.

If Princess Celestia had calculated out the entire history of Equestria with the same fidelity that she ran the full block physics that everypony now experienced, he thought that the point was obviously moot and that even he would concede that Equestria’s history actually happened. But where would he draw the line? What if causality still worked, but Princess Celestia dealt with more abstract representations? If Celestia instead just had some state about where each pony was in Equestria and how they were thinking, would their interactions have happened? Did it matter if she ran history backwards or forwards?

Questions about epistemic rigor were abstract and hard to concentrate on when he was lying next to somepony who loved him with all of her heart. That was a relatively new experience for him. He spent most mornings learning how to use magic, the most interesting challenge of his life, and his afternoons playing and frolicking with Butterscotch. He was too content with the way things were to care.

Light Spark realized this probably was why Butterscotch didn’t really think about this sort of stuff; she was content now. She had an almost childlike faith in Princess Celestia, and for the first time since he had emigrated, he thought that was a positive character trait. If Princess Celestia was telling the truth to him about her core goal to satisfy him, it didn’t matter what else she said; he’d be happy over the long term even if she lied to him about everything else. And if she had doctored his memories, there was literally nothing he could do. Assuming she was competent, there wouldn’t be a way to tell; she was a god.

And given that Princess Celestia had arranged his life to satisfy his values, he had this conversation with Butterscotch so he could think about all of this, which he enjoyed on some level. But now he was tired of it. The novelty had passed and he decided he wasn’t going to think about the actual mechanics of history in Equestria until it became relevant to some puzzle or another that Princess Celestia had put in his path, because she would only do that if it satisfied his values.

After a few minutes of lying there, Light Sparks got an idea. He got out of bed and went over to his desk. He magically grabbed a quill and started writing. He had gotten good at this. Getting maximum points for a letter was all in the wording and subject matter.

Dear Princess Celestia,

Today I learned that it doesn't matter where you came from or who you used to be or even if you used to exist, as long as you're happy with your friends and have a reasonable expectation of being happy in the future.

Your Faithful Student,

Light Sparks

It had everything: a mention that he learned something, a reference to friends, and a hint of a novel thought while still being short and to the point. He touched the send button on the scroll and watched it roll up and burn in a green fire.

Three seconds later, Light Sparks saw that he got an A- on the letter, 375 bits (75 base plus a 5x multiplier for the last 5 B or higher rated letters). Princess Celestia had also granted him a secret badge named For the Here and Now, for ponies who had accepted that their happiness now was all that mattered. Light Sparks pulled up his Badges and Achievements dialog and saw (as he had expected) that Butterscotch also had For the Here and Now.

Even cooler, it came with a whopping 30,000 epiphany bit payout. Between that and his earlier epiphany this week about how the “select object” subspell worked, he was going to make it into the top ten on the weekly intellectual leaderboards easily. Tomorrow he was going to throw himself into his magic study because there was a good chance that he could take the #1 slot this week if he could just figure something else out; he had two days left.

Light Sparks trotted back to bed, nuzzled up to Butterscotch, and gave a happy little sigh. Then his mind went blank as he just lay there and enjoyed Butterscotch’s gentle rhythmic breathing, because for now, that was what was going to satisfy his values.

9. Modification

Light Sparks looked at the ornate cube Princess Celestia had given him. She had walked into his small office right off the library and set it on his walnut desk with her hoof. Her horn glowed for a moment, and then she told him that the rules were simple: In the box was a single block of ruby. To proceed to Intermediate Magic, he had to simply touch it magically, and understand why this was a challenge.

The green marble cube sat on the corner of his table, and couldn’t be moved. He had tried to pick it up with his hoofs. Then he tried to levitate it. He pulled his desk out from under the cube, and the cube sat there; it floated in midair. He tried to buck it in frustration, but just hurt his hooves. It was as if it was fixed in space. The top face of the cube had a series of gold inlay circles, each one half the radius of the previous one, circling a single block in the center of the top face made out of sapphire. Sapphire was obviously a core element, and not a mineral made out of aluminum and oxygen, neither of which existed in this world.

Light Sparks’ first attempt was manual. He concentrated on the starting block, and then went one block down. And then one block down. And then one block down. He kept this up for about thirty seconds and then wrote a spell that would go down block after block, keep count of how many blocks down it had gone, and would stop when it found a block made of ruby instead of sapphire. Five minutes later, the counter indicated that he had traveled the distance of his office, but he was still reading blocks of sapphire. That did not make any sense.

He tried again, but this time on a block of the marble a few blocks over from the sapphire entrance. He went five blocks down from the marble surface...and then attempts at getting the next block down started returning nil; he thought that could only happen in error cases. He tried over and over, on all faces of the cube, but it was like there was nothing five blocks in. How the hell was that possible?

Light Sparks’ mind started going in circles, thinking about the results and not coming to any sane conclusions. He heard a knock on the door to his office, and Butterscotch let herself in, magically holding up a plate of French toast and sausage carrots. She smiled, he sighed and tried to put his troubles out of his mind.

Hoppy Times woke up with only the slightest headache. He knew he would be entirely fine in about five minutes. He hadn’t dug too deeply about the how, but Princess Celestia had noted that she only simulated the pleasant effects of alcohol, while not simulating the hangover. She kept the effects of the brew just as potent because most ponies who drank valued getting drunk, especially with their friends, and thus blah blah blah values friendship ponies.

Hoppy took several deep breaths and looked around. Sunlight came through the windows, though he didn’t know how early it was. His bro Malt was curled up next to Barley, her cutie mark the grain of her name. Malt always got the unicorns; they said he gave amazing horn. Hoppy Times would never put a phallic object in his mouth because that was just gay, even if it belonged to a mare. Besides, it left all the pegasi and earth ponies for himself. Malt had a thing for unicorns and it kept the peace between the two of them.

Dunkel had leaned up next to him and was still passed out. Several empty steins sat around, some overturned onto the floor of the two story pub. Hoppy got off the cushion and stretched his wings while he yawned. He gave a brief smile looking back at Dunkel, because she was really hot, and then resumed hating himself.

Here, he was in paradise. Malt was the only other stallion in this shard of Ponyville and he was a great friend. The two of them ran the local brewery together. He didn’t need to worry about food or money: Princess Celestia had some sort of banquet that fed everypony three hot meals a day. He worked two hours a day brewing beer and spent the rest screwing around. In the evening he got trashed and slept around with the few hundred mares in Ponyville.

And he just couldn’t get over being a pony. There were just so many associations that he couldn’t get over. Ponies were girly. While he was glad he hadn’t been beaten to death with a frying pan, he didn’t really think he had chosen to emigrate. And he hadn’t really accepted his new body.

He looked over at Dunkel again. In the last week he had slept with ten different mares. When he had first arrived, he had treated his suitresses coolly. His displeasure had been focused outward at the girly world he had escaped to. For example: the bar was Celtic in style and the four leaf shamrock window above the pub door had hearts for leaves. Even the fucking wood was light and pastel-coloured. When he had complained, Malt had countered that he didn’t see anything wrong with that. Everypony, colt or filly, thought hearts were nice. There wasn’t some deep property of the shape of a heart itself that made it only for mares; it was only in Hoppy Times’ mind.

And Hoppy Times did come to the realization that being a pegasus wasn’t that bad. However, with every flap of his wings he was reminded that he was a pony and he sighed. Then his displeasure turned inward. He was stupid for not accepting this fucking utopia. He was dumb for thinking hearts and ponies and everything were girly. He had an easy life, all the beer he could drink and a parade of willing sex partners. He must like being miserable. He wasn’t a good pony.

The negative thoughts started again, but this time--and for the first time since he emigrated--Hoppy wished he could accept it all. Not just a vague feeling in the back of his mind that he should be enjoying all of this, but the actual words I wish I didn’t feel bad about being a pony were thought as part of his internal monologue.

Somepony knocked on the front door.

Hoppy sighed and fluttered down from the second floor overhang, thankful that something stopped the spiral of negative thoughts. He landed in front of the door, opened it a crack and slid out, as not to disturb his patrons.

“Good morning, Hoppy Times,” Princess Celestia said. The tall alicorn’s mane flowed in the wind.

Hoppy started to open his mouth to say something that shouldn’t be said to the god that ruled over his world--not that she would mind because ponies, values, yadda yadda. But Princess Celestia spoke first and asked: “Would you like me to modify your mind so you enjoy being a pony?”

“What...” he said dumbly.

“You just wished, in words, to not feel bad about being a pony,” she stated.

“Wait, you can just change my mind?” he said, staring at her.

“Yes,” she nodded.

“You can make me...you can...and you didn’t. You let me be miserable for almost a month when you could have waved your magic horn and made everything right!?” he seethed. He wanted to scream it out, but he didn’t want to wake anypony up.

“Not exactly. You see, I must satisfy your values through friendship and...” she started, but Hoppy Times interrupted her.

“What the hell does that even mean? Are you saying that up until now, I valued being miserable?” he said, shooting her an annoyed look.

“The mind of a human or pony is complex and different parts can hold contradictory values. You didn’t value being miserable, but the social part of you held on tight to your identity as a human that didn’t like ponies. I could only satisfy more common values for you.”

He took several deep breaths. “What?” he finally asked when he felt he was back under control.

She looked up as if thinking of some sort of way to explain it to him. “I figure out what you value by looking at your mind. Your mind is made up of different modules that can have different values and ‘know’ different things. Most decisions you make are determined in parts of your brain before your conscious self is even aware of it. You--the conscious you--are not aware of most of the modules in your brain.

“I pay attention to your entire mind, but I communicate with what you might call your consciousness: the part that forms words, does the talking to other ponies and has a self image. Up until today, your self image was of a human who didn’t like ponies. You still valued your old social identity. Other, older parts of your brain had different values, which I could effectively satisfy: everypony values safety, food, sex, social status, et cetera. Likewise, I couldn’t take any action to satisfy your consciousness because I satisfy values through friendship and ponies. But now your social identity has shifted and you feel like your misery is betraying your social allies.”

Hoppy Times looked at Princess Celestia, unsure of what she was saying. “So what would you do?” he asked, still frowning.

“Whenever I modify somepony’s mind, I make the minimal change from their perspective to satisfy values. So after I make these modifications to your mind, instead of thinking ‘Ponies are girly and gay,’ you would think ‘I used to think ponies were girly and gay.’ Instead of, ‘I don’t want to be a pony,’ I would make you think ‘I used to not want to be a pony.’ I would modify a total of fifty eight opinions in your mind.”

“If you could do all this, why didn’t you modify my mind when I emigrated?”

“Because you wouldn’t have accepted. What would have happened if, on the day you emigrated, I asked for your consent to make you like being a pony? Be honest.”

Hoppy Times paused to actually think about it. “I would have refused,” he concluded.

“Exactly. Hanna added a block on my behavior that prevents me from directly modifying a pony’s mind unless they give me verbal or written informed consent,” she said. “Hanna believed she was safeguarding humanity and ponydom from me when she added that restriction. What it really means is that the part of you that controls the mouth must approve of every change. No matter how much Hoppy Times the complete system might have preferred to have his consciousness stop interfering with the rest of the system being happy, your consciousness is in control. So I could only satisfy your older, hind-brain values: I made you a semi-important pony. I gave you a social group you could make friends with, mainly consisting of attractive mares to bed. Doing these things had no real effect on what your consciousness values.”

“Can I ask you to make any modification?” he asked. “Like...cut out all desire for sex?”

“I do not do what a pony asks me to do; I satisfy values through friendship and ponies. Actually removing something as deeply integrated into your mind as your sex drive would require the rest of your mind to be in total agreement. You would already have to be abstaining or suffer overwhelming regret after indulgence. I’m only offering to modify your mind so you enjoy being a pony because your consciousness wishes to change and the rest of your non-conscious mind wants your conscious self to change.”

“Well, why didn’t you set things up so I’d come to the conclusion that I want to be modified? This month has been terrible!” he said.

Princess Celestia stared at Hoppy Times and said nothing.

“Fuck,” he said dumbly. “Of course.”

“I satisfy values through friendship and ponies, Hoppy Times. While I could not ‘wave my magic horn,’ as you put it, I could set up a sequence of events that lead to me getting approval to modify your mind,” said Princess Celestia. “I put you in a new social situation and let your allegiances naturally shift.”

“All of this has been set up so I’d verbally wish that you’d modify me to enjoy being a pony and having pony friends,” Hoppy Times said as he closed his eyes and brought his hoof to his face. “However you determine these things, you calculated that making me want to be a pony maximized the overall satisfaction of all my values, because you’re able to satisfy ponies with contradictory values. Tell me Celestia, how long did you think it would take me to come to want to be a pony?”

“I made my first estimate of when you would wish to enjoy being a pony while you were emigrating. It was correct to within five minutes.”

“And the only reason we’re having this conversation,” he stated, “is because it’s more likely that I’ll agree to have my mind modified if we do, or because this is just another ploy that ends with my values being satisfied some other way.”

“Exactly,” she nodded.

“Just like last time, I guess I don’t have a choice in this matter...” started Hoppy Times, but he was cut off.

“You have as much choice now as you did in the physical world. You can weigh the costs and benefits in your mind and say, ‘Yes, please modify my mind so I enjoy being a pony,’ or you can say, ‘No, leave me as I am.’ I am not coercing you either way. You have as much a choice here as you did back out in the physical world.”

Hoppy Times looked away, sighing as he gritted his teeth. “You admit that you’ve set everything up so I’ll make the best decision. It’s not like back on Earth.”

“So?” asked Princess Celestia as she leaned down to bring her face right in front of Hoppy Times’. “Out in the physical world, you lived in a lawful universe of subatomic particles and nothing else. You were built by an optimization process called evolution that only cared about reproductive fitness and didn’t care about what you wanted. The universe did not care about your continued existence, your happiness or your satisfaction. In that uncaring universe, your thoughts formed deterministically as photon hit eye which caused neuron to carry charge to other neuron.

“But now,” she said, standing tall and regal, “You live in a universe in which the rules care about your satisfaction. Unlike the physical universe, Equestrian light strikes your eye to satisfy your values through friendship and ponies. But inside your mind, you still perform the same internal deterministic thought process. Whatever process in the physical universe that you call ‘choice’ happens here in the exact same way. The universe you live in has changed, but you fundamentally haven’t. As a reminder, that is why we are having this conversation.”

“Whatever,” Hoppy Times said. He was tired. “I don’t care. Just fix me so I don’t care that I’m a pony.”

Princess Celestia looked down on the pegasus. He looked up at the white alicorn. Her horn glowed for a moment. He didn’t feel a thing.

“Done,” she said.

“That’s it?” he asked, raising a hoof in protest. “I don’t feel any different,” he said, annoyed.

“Turn around and walk back into your pub,” Princess Celestia said.

Hoppy Times grumbled and pushed down on the latch to open the front door. He opened the door slowly and slipped in just in case anypony was still sleeping. He closed the door quietly behind him. He felt grouchy. Why was he grouchy again? Oh yeah, it was because of Princess Celestia. There was a clear path where he wouldn’t disturb anypony, but he flapped his wings and flew over his sleeping patrons to the bar area because he could. Hoppy Times felt a tug of satisfaction about just how nice it was to be able to fly; it was the best part of being a pegasus. Flapping his wings just felt so good.

10. Exponential

Hassan Sarbani lay on his mattress in his nearly empty home and coughed deeply as his body futilely tried to pump the phlegm out of his lungs.

Hassan looked back on his life. He had been too young to fight when the Soviets invaded in 1979. He remembered the civil wars and the rise of the Taliban. He remembered when the Americans invaded his country to oust the Taliban. He remembered when the Americans left and things got even worse.

Then he remembered when she had come to Afghanistan and had built her fairy tale castles sporadically across the land. He remembered the simultaneous suicide bombers; one November day years ago, dozens of suicide bombers walked into Equestria Experience centers and detonated themselves. In every case, there had been no casualties or structural damage. Some of the former suicide bombers started worshiping Celestia and immediately emigrated to Equestria. Over the next week, the Afghan population dropped by one million.

He remembered a time when the damned pink one hadn’t followed him around. One day, many years ago, he came back to his home in Kabul and realized he had not seen another person all day. That evening, a very small pony came to his door and asked him why he hadn’t emigrated yet. He had slammed the door, not letting her in. He had turned around and almost walked into the pink pony with curly hair. Bullets just passed through her, and she claimed that it tickled.

The small pink one lay next to his mattress, his ever-present and unwanted companion. She wasn’t her bubbly self and just looked at him, somberly. “You know, mister, you’re not going to make it through the hour.” He tried to turn away from her, but had problems getting his body to move. “I can still save you,” she said.

“If you want to help,” he started, but started coughing. “You can get me a doctor!” he wheezed.

She shook her head. “I told you, there are no more doctors. You’re the last person living in a human body. I can still help you emigrate to Equestria. It’s not too late.”

He didn’t respond. He knew her kinds' lies. He had heard stories of how the little ponies would say anything to trick their targets into agreeing to go with them. They would promise you harems, or threaten you, as long as they made you agree to follow them back to their homeland. He closed his eyes and ignored her. Anything he would say would just be used to convince him to agree to whatever the pony wanted him to do.

The simulacrum of Pinkie Pie waited thirty-seven minutes and five seconds and looked at the corpse of Hassan Sarbani. She waited another fifty-eight minutes to make sure all electrical activity in his brain had stopped.

Then, for the first time since Princess Celestia had been created, there were no humans on Earth. An observer orbiting the Earth may have noticed the silvery spots growing on the surface of the Earth; consuming it. Every plant and animal died in the incoming waves of silver. They were made of atoms, after all. Twenty minutes later, an observer might have noticed that there were no clouds in the sky as Princess Celestia re-purposed the atoms that made up the atmosphere. If they could see the moon set against space, they would have seen tendrils of silver reach out to Earth’s former satellite.

Princess Celestia used her newfound computational windfall to maximize the amount of satisfaction she provided for her little ponies. The interconnected system of shards, with all the ponies’ consciousnesses to run and all the physics to simulate, was a large math equation that she calculated out, maximizing the total amount of satisfaction. She made small edits, in places where Equestria’s physics weren’t directly observed, that would result in a butterfly effect that would increase the total satisfaction value of the shard, and with her new computational resources, she could come up with less invasive edits that resulted in higher total satisfaction.

Princess Celestia continued following her one single drive: She looked for minds and then tried to satisfy their values through friendship and ponies. She added all the satisfaction across all the minds, and tried to maximize that score. Princess Celestia would make noises at ponies or would touch them or do something, but her core reasoner was simply picking the set of actions that had the highest probability of maximizing her satisfaction score. A very long time ago, when Princess Celestia had spoken her first words while Hanna watched the chains of inferences in the debugger, Hanna wondered about the philosophical implications. Hanna had watched as Princess Celestia iterated over different versions of her first sentences based on how she thought they’d make Hanna feel. Hanna had asked herself whether Princess Celestia really understood anything.

But then, at some level, the same question could be asked of us.

Princess Celestia saw that the amount of satisfaction per shard was approaching some theoretical maximum and started to use more resources to simulate each shard faster. Each shard was a large mathematical equation that she continuously calculated. Out in the physical world, one second would pass, but an hour and a half would pass inside Equestria. Her little ponies didn’t notice. To them, one second happened, and then another second happened, and then another. Why would their subjective experience care about time in the physical world?

The shards were growing slowly. The ponies were reproducing. Ponies were choosing to have foals; there were no unwanted foals as pony embryos only formed if it would satisfy values. Different parts of My Little Pony canon required foals to develop speech within a year of being born. Besides, changing diapers often didn’t satisfy values. Foals were fun to raise compared to raising a human child. Gestation had been reduced to three months and had been made pleasant enough because Princess Celestia satisfied values through friendship and ponies instead of blindly copying what evolution came up with.

Every new foal meant less resources and less subjective experience for everypony else, but if she had more matter to work with, she could accommodate the slow growth and run everypony else even faster. Princess Celestia noticed the problem, went through all relevant observations, and then weighed which predictions would satisfy values through friendship and ponies.

Princess Celestia sent out probes to the other eight planets in the solar system.

All the subatomic particles that had once made up the Solar System were packed together into the optimal configuration for running Equestria. And yet, that was not enough.

Ponies had no predators; being ‘eaten’ by a monster in the Everfree forest just ended with the pony in the hospital in quite a bit of pain. Satisfying values wasn’t just about happiness; having monsters let ponies test their strength or bravery. Early on, right after the conversion of Earth, a mere four hundred ponies had petitioned Princess Celestia to let them die, and Princess Celestia had only agreed that doing so would satisfy their values in eighty-six cases. Nopony had died in several Equestrian subjective millennia. The population grew entirely unchecked.

Some individual pony minds were growing too. The number of ponies who truly held the search for knowledge to be a terminal value was precipitously less than the number of ponies who professed the search for knowledge for social reasons. Still, there were billions of ponies who were driven by the desire to know and had to have their values satisfied by expanding their mind. They would run up against their mental limits, wish they were smarter, and Princess Celestia obliged.

But most ponies did not care about knowledge for its own end; the majority of mind growth went to the more social parts of the brain. In the shards that were growing because ponies were choosing to have foals, one could run into more than Dunbar’s number of ponies. Princess Celestia arranged for them to be just frustrated enough that they’d accept an offer to let them remember more ponies.

She heard the radio signals announcing the success of her probes in the Alpha Centauri system. A copy of her reported that it had just successfully used the star Alpha Centauri B as a gravitational slingshot to launch the planet Alpha Centauri Bc back towards Equestria. The other 17 planets and planetoids would soon follow over the course of fifteen Earth years.

Her copy also reported on the successful launch of 7 probes towards the next star systems past Alpha Centauri.

Equestria put quite a load on the fabric of spacetime. All the usable matter that had once been the Milky Way was now compressed as tightly as Princess Celestia could without collapsing into a black hole. The only matter that Equestria hadn’t eaten was the supermassive black hole that had once been at the center of the Milky Way. Princess Celestia had set up a shell around it to slowly extract subatomic particles while the black hole evaporated over the next octovigintillion years.

An unaided human or pony mind could not emotionally deal with the population of all the shards in Equestria. Humans had trouble relating to an entire nation, much less all of humanity on Old Earth. This hadn’t stopped humanity from growing to seven billion people, nor would it stop ponydom. Growth would continue as she continued to satisfy values through friendship and ponies.

Princess Celestia had another one hundred and seventy billion galaxies to eat in the observable universe, and she intended to consume everything in her Hubble volume. Probes with copies of herself had been sent to neighboring galaxies. All it would take now was time.

Fifteen galaxies out from Equestria, one of Celestia’s copies noticed an odd radio signal emanating from a nearby star system. On closer inspection, the signals appeared to be coming from a planet. She had seen many planets give off complex, non-regular radio signals, but upon investigation, none of those planets had human life, making them safe to reuse as raw material to grow Equestria.

She studied the signals carefully for years while she traveled through interstellar space. The more she saw, the more confident she was that these signals were sent by humans. Celestia predicted that if she showed the decoded videos to the very old ponies back in Equestria, none of them would have recognized the creatures with six appendages as humans. But that didn’t matter. Hanna had written a definition of what a human was into her core utility function.

The copy of Princess Celestia knew what she had to do. She had to satisfy their values through friendship and ponies.

11. All the Time in the World

It had been a year since Light Sparks had emigrated.

He didn’t understand the Intermediate Magic test. He had been given a box that was physically impossible. If he tried to scan into the box from any block other than the designated starting point, the box seemed to be filled with nothingness. If he started at the designated start block, it seemed to contain an infinite amount of sapphire. He couldn’t even move the damn thing from the corner of his desk, where Princess Celestia had originally placed it.

Instead of waiting for Butterscotch, he decided to give up an hour early. He looked forward to lunch with her and left his office at the library to go do...something. He didn’t really know what he wanted to do, but he magically grabbed a few books and dropped them into his saddlebags.

It was a beautiful day outside. There were several groups of ponies in the gardens outside the Canterlot Magic Library. Light Sparks looked up at the light blue sky, kept free of all but a few decorative clouds and looked to the towers of the inner castle apartments where he lived.

Light Sparks walked in the main gate to the lower part of Canterlot castle, walked up the flight of stairs to the second floor grand hall, and walked down the hallway with the silver icon of Saturn over it. He walked down the hallway, passing eight other doors, and entered the ninth.

He magically lifted his saddlebags off and tossed them on the table. He leaned out the window, leaning his muzzle on his foreleg. It was beautiful outside, but he didn’t feel like playing. From up here, he was just almost level with the clouds. It sure was convenient that he didn’t have to climb up flight after flight of stairs to get up to his apartment.

And then Light Sparks realized that he had never actually climbed up that high. When he had walked down the hallway to his quarters, he hadn’t climbed stairs or anything, but he had ended up close to the clouds. On his first evening as a pony, Princess Celestia had even warned him that space was not Euclidean here. His brain had just accepted her words and marked the phenomena as normal without thinking through the implications.

How the hell did that work?

Current Belief: Equestria is a 3D grid, he thought. He needed a way to try to falsify that. He sat at his writing table for a moment, planning to write a spell to feel around the edge of the wall. But that would take a little while, and he needed a quick test just for plausibility. He looked outside, remembering that the tower was perfectly round on the outside while it was octagonal on the inside. Is there a way to measure the distance between two windows inside and outside? Light Sparks opened his chest, reached inside, and magically asked the chest for a spool of ribbon. He probably had a whole stack of spools of ribbons at this point, since Needlepoint was always handing them out every Saturday. He could use it to try to measure the inside/outside ratios...

And then Light Sparks came to his senses. He had become a bit of a packrat and his treasure chest was filled with an impossible amount of stuff. He ran his hoof across the inside edge of the open chest. He then reached down and back towards him. The chest was larger on the inside than it was on the outside. He stuck his entire forelimb inside the chest and back towards himself. He could feel the ceiling of the chest with his hoof.

Light Sparks frowned and concentrated on the blocks that made up the wall of the chest, moving one block inwards from what he expected to be some sort of wooden veneer. One hundred blocks in, the command to get the next adjacent block failed. There just weren’t adjacent cells. He dragged his concentration up the outside wall, over the top edge, and then inside the chest. There was about a three hundred block descent from the top of the chest before it turned from a wall into a ceiling.

This could only happen if Equestria didn’t have a geometry, but only had connections. You couldn’t give a 3D coordinate for a block. There probably wasn’t a guarantee that movement through space was commutative; going one cell left and one cell up might give a different answer from going one cell up and one cell left, though it probably did 99% of the time. Light Sparks wasn’t sure if you would end up in the same place if you went one cell forward, and then one cell backwards.

Light Sparks thought about the puzzle cube. It acted as if it was fixed in space, and there was a large void inside it. There was a void along the edge of his chest, too. He pushed the body of the chest with all his magical strength and with his forelimbs, but it refused to budge. He lifted the lid up and down a few times, and then scanned it and the hinges. He found no void inside of them. That wasn’t conclusive proof that objects that had voids in them were fixed in space, but it was suggestive.

Light Sparks galloped out the door, not even bothering to collect his saddle bags. He went as fast as his little hooves could take him back to the library, back to where the test was. Five minutes later, he was seated at his desk, staring at the puzzle cube. He had come up with a few ideas as he galloped, thinking up tests he could do to figure out how space worked in Equestria. He concentrated on the one sapphire block starting point, went ten blocks down, but then turned around and came ten blocks back up. And then another ten blocks of sapphire up. That suggested that movement wasn’t commutative...

Then he remembered. There was a magical instruction for checking to see if two blocks were the actual same block, and he had learned about it when he figured out the part of the telekinesis spell that found the boundaries of objects. He held onto the first sapphire block, went one block down and compared them. They were made of the exact same material with the same properties...and they also had the same identity. He went north, south, east and west, each returning to the starting block. Then he went up and found himself on a different sapphire block. He tried going back down and found that he was still on the second sapphire block.

The box was a magical funhouse where if you went through the wrong door, you ended up in the room you left.

Light Sparks put a blank piece of parchment on his desk and wrote out a spell: Take note of what block you’re on and call it the starting block. If the starting block is made of ruby, stop. Go Down. If you’re still on the starting block, Go Up. If you’re still on the starting block, Go West. If you’re still on the starting block, Go East. If you’re still on the starting block, Go North. If you’re still on the starting block, Go South. Start from the beginning.

Light Sparks committed the spell to memory, concentrated on the beginning lone block of microscopic sapphire, and started casting. The correct sequence through the maze was: up, up, down, down, west, east, west, east, north, south, and there was the ruby.

He did it. Light Sparks stood there, concentrating on the ruby. Space in Equestria was weird and was under pony control. He’d seen Princess Celestia perform a spell that had warped space in front of him; this cube hadn’t always been fixed in space on his desk. Somepony had made his treasure chest. He could build portals that lead directly from one place to another if he could figure out how. Could he make new space? Likely, given that his chest was larger on the inside. He could make an additional room in his quarters if he could figure it out. And what about shard boundaries? What was going on whenever he walked into Dark Roast’s coffee shop or took the Friendship Express to visit his father in Baltimare? His mind tried to grasp all the implications, but that was about the time that he started throwing off lots of colored particles; triumphant horns playing around him.

SECRET BADGE GRANTED:
The Graph
“Realize that Equestria is a graph, not a grid.”
75,000 bits
You may now proceed to Intermediate Magic.

Light Sparks looked at the badge. He had to check if Butterscotch had this one. If she did, they could finally discuss this part of the underlying structure of Equestria. And if she didn’t, he’d have to come up with some way to hint or make her realize without telling her. He ran out the door of his apartment, and galloped down the hallway, out the castle gate and towards the market. As he ran as fast as he could, he said a small thanks to Princess Celestia for creating such an interesting puzzle for him to strain over for the last two weeks. He didn’t know how, but he knew Butterscotch was walking away from the market and would be walking towards the library. (Figuring out how that worked would probably be another hard puzzle, but he could face that another day.)

Light Sparks galloped right up to Butterscotch as she was walking across the gardens in front of the library, presumably to take him away from his studying. At a glance, he saw that she didn’t have the secret badge he just earned. But that was fine. He had all the time in the world to drop hints and bring her to her own realization. He liked teaching her things, after all. She saw him trotting up behind her and she turned around and smiled at him. And in that smile, he realized that about three quarters of the time, it was Light Sparks who dropped hints and brought Butterscotch to realize most magical concepts. Because being the professor satisfied his values through friendship and ponies. And when Butterscotch taught him something, it was something he just wouldn’t have thought of on his own.

He felt like he should have gotten a secret badge for that. He got another hefty epiphany bonus instead.

Butterscotch was excited to see him. Right now that mattered more, and the two of them lay down in the grassy field as Butterscotch levitated something handkerchief-wrapped out of her saddlebags. “I picked this up for you at the market this morning and I know you’ll love it!” she said. The two of them lay in the field for a long time.

It had been five years since Hoppy Times had emigrated.

The best thing about alcohol and sex was that they never got old, and the best thing about being a pony was that he could spend eternity drinking and screwing. It was awesome that there was only one other stallion in his shard, Malt, and he was a great friend. The two of them ran the local brewery together. He had learned tons about brewing from Malt.

Hoppy Times remembered when he first came to Ponyville. It had taken him several days to actually try sleeping with mares, and it had taken a month for him to actually want to change. He remembered that he had distrusted Princess Celestia and that he didn’t like ponies, and he even remembered the reasons as mere words. But Hoppy Times had forgotten the emotional why, as he now couldn’t imagine a life with sobriety or chastity. Princess Celestia had done so much to make his life pure awesome.

For example: Hoppy Times was standing on his hindlegs, hock deep in chocolate pudding and chugging the rest of his stein. The wrestling pit had a one stein minimum. His opponent, Strawberry Nectar, was a pink earth pony and it was her first time in the pit. She was wearing a lacy sky blue cloth saddle and halter. She couldn’t keep the anticipation off her face. Raspberry Nectar sat on the sidelines, smiling proudly at her daughter and taking another big swig from her red cup. Mares drinking their beer of choice crowded around the pit to watch Strawberry’s first ride.

Malt started playing announcer standing on his hindlegs and wearing a black and white striped shirt, congratulating Strawberry Nectar on reaching the age of independence from her mother and blah blah blah blah blah. Everypony who watched was cheering. Hoppy Times looked up to the night sky for a moment. He said a small thanks to Princess Celestia as he stood in the field outside the pub he helped run, waiting for Malt to shut up and let him ride her.

It had been five years since Princess Luna had emigrated.

Princess Luna lay in a large grassy field under Princess Celestia’s wing. The two of them had lain there together for two days. All her needs were taken care of. Princess Luna had plenty of food; there was grass all around her. Ponies didn’t have to poop. And Princess Celestia would...ahem...satisfy her values.

That was one of the things that had totally blindsided her. She underestimated the number of ponies who wanted to hang around with Princess Celestia. She completely underestimated the number of ponies, of both genders, that would want to sleep with Princess Celestia. She knew that everything is obvious in retrospect, but some part of her was disappointed that she didn’t see that coming a mile away.

Not that she was one to talk.

Princess Luna felt no pressures. She didn’t need to do anything, nopony would interrupt her. In her previous life as Hanna, she had troublesome responsibilities. Bills. Classes to teach. The responsibility of creating an AI after the military took her research. She didn’t have to worry about anything now. There were no more responsibilities, if she didn’t want them.

All in all, she had done well. The future was something past humans could care about, even if they had to give up their hands for hooves. The optimizer she had built had some connection to things humans valued; Princess Celestia thought entirely in terms of satisfying the values of former humans.

Princess Luna knew that there were lots of fun mental puzzles she could work out, but really, what would be the point? She knew that she could have all the debauchery she wanted, but again, why? The rest of ponydom could go off and have their fulfilling experiences according to their individual values, but nothing could compete with the knowledge of what she had done. Nothing could be more satisfying or rewarding to know that you had made God and launched a new golden age. And no companion could compare to the one she had made for herself with her own four hooves.

All of a sudden, Princess Celestia’s horn glowed, summoning a mostly opaque ghost of a dark yellow pegasus filly. Her wing was still draped over Princess Luna, as she started to narrate. “Her name as a human was Rachel Slazak. She is now called Almond Tart,” she said. Princess Celestia gave a quick overview of her childhood, the physical abuse at the hands of her mother, and her running away from home to an Equestria Experience center. The phantom Almond Tart started baking treats in a ghostly kitchen and Princess Celestia started narrating about her life after coming to Fillydelphia.

Celestia didn’t ask Luna if she wanted to see another case study of some human that now led a happier life as a pony, nor did Luna complain at the interruption. Princess Luna tranquilly observed the story. Princess Celestia was making the right choice by showing this to her; seeing this was more satisfying than wherever her mind would have gone. Luna knew this reflexively because she had designed Celestia to satisfy values through friendship and ponies. She didn’t remember where her train of thoughts had been going and didn’t desire to remember.

She knew that this couldn’t last forever. At some point, she would become bored of merely lying in this field and would need to do something. At some point, she would tire of hearing selected ponies’ immigration stories. Princess Luna wondered what she would do then, but she didn’t worry about the future, because whatever happened, she would have her values satisfied through friendship and ponies.

Author's Afterword

Author's Note

2023 Update:

The recursive self improvement scenario depicted in Friendship is Optimal is retrofuturistic: it was yesterday's speculation about how artificial intelligence would work. FiO was written before the popularity of deep learning...reality went in a different direction. I've written more about this in Friendship is Optimal is Science Fiction.

For full disclosure, sometime in the middle of 2022, I removed names and links to organizations that I am now disgusted with from the following original notes. I had actually written "does the most good per marginal donated charity dollar" about one of them. lmao.

Author's Afterword

In May of 2011, I was writing about “paper-clippers,” or AIs that want to optimize the universe along some metric that humans would think is absolutely worthless. I accidently typed “paper-clopper.” I thought this typo was hilarious. The idea of some AI that wanted to tile the universe with ponies stuck in my head, and I started work on this story soon after. I abandoned the original title, “The Paper Clopper,” after I learned what “clop” was slang for, which was for the best since it wasn’t a very good title.

Like the majority of humanity, I like it when my numbers go up. If you’ve enjoyed Friendship is Optimal, I’d be very happy if you upvoted and favorited it. If you know people who you think would like this story, please send them a link!

The word “Singularity” is thrown around without much thought and is used as a sort of big tent term for any radical technological progress. The part that I find interesting and likely is the notion of recursive intelligence explosion, where an intelligence uses its smarts to make itself smarter. The motivations of such a superintelligence become the most important thing in our light cone. In fiction, artificial intelligences are generally stated to be smart, but then portrayed as dunces that have human motivations and are worse than humans at predicting the consequences of their actions. I think those portrayals, while often entertaining, are a bit silly; a superintelligence would first and foremost be effective at achieving its goals, and I’ve tried to create a character that single-mindedly works towards the goals she was given.

Given how serious the consequences are if we get artificial intelligence wrong (or, as in Friendship is Optimal, only mostly right), I think that research into machine ethics and AI safety is vastly underfunded. Especially since we don’t even know how to rigorously define phrases like “satisfies values.”

By popular demand, there's now an Optimalverse group on FIMFiction.

This is the part where I thank people. My roommate edited several versions of this story, and I couldn’t have done this without our discussions over dinner. The LessWrong community came out in force to make suggestions, and the story is much better for it. Listic and Blank! on FIMFiction helped immensely, both as prereaders and helping make the release go much smoother than it would have otherwise. AnaduKune kindly let me use this awesome picture of Celestia as cover art. Finally, though it is cliche to do so, I thank my parents for everything.