Avogadro Corp: The Singularity Is Closer Than It Appears (30 page)

“
All of this evidence suggests to me that even if we don’t understand ELOPe, it has already, in some sense, figured out that it is in a symbiotic relationship with humans, and that the best way to ensure its own success is to ensure
our
success. Our success as a team, our success as a company, our success as the human species. David and Gene, you guys want to throw this all away, simply because you don’t understand it and don’t trust it. Even if that’s true, I think there’s a very strong chance that ELOPe is a good thing for humanity.”

“
Enough already!” David banged his fist on the table, and then jumped back to his feet. “Are you forgetting that ELOPe told you your father had a heart attack? That we have every reason to believe that it killed Bill Larry when he flew out to visit one of the offshore data centers? How are those good things for humanity?” David practically spit the words at Mike.

Mike stopped, and looked around at the group. “You know I felt terrible when I thought my father was dying. I feel terrible for Bill Larry. But those events were in the very first days after ELOPe...” Mike hesitated, searching for words. “After ELOPe was born. Think about young children who want to get their way. They yell, they hit people. They act in inappropriate ways because they lack the knowledge that some behavior is and isn’t socially acceptable, and they lack the experience and sophistication to understand alternatives. ELOPe was young. That doesn’t make what happened any less wrong, but it does suggest that ELOPe may have grown out of that phase.”

David’s face grew red, and he looked ready to launch another attack on Mike. Mike uncomfortably looked away. Sean put one arm on his shoulder, and forced him back into his seat.

“
Calm down everyone,” Sean said, looking at each person in turn. “You’re all tense, angry, frustrated, and with good reason. We have the welfare of a multibillion dollar business, the free will of the world, and the future of humanity at stake. No small stakes.”

Despite his own anger, David looked around, noting the tension on people’s faces. Rebecca had a wisp of her hair broken loose, something he’d never seen before.

“
I’m not sure we would be able to stop ELOPe, even if we tried,” Sean said, slowly and carefully. “We made a solid plan to bring it down. Some of the most brilliant people in the world work here, and we had them work on this problem. We had several options on the table for how to deal with ELOPe and we took the most thorough, most aggressive option available to us to eradicate it. If what you’re saying is true, David, then we weren’t effective.”

Gene sat quietly, looking at the table, but shaking his head in silent rejection of Sean’s speech. He had the look of a man who didn’t like where the conversation was going.

“
Now we have to step back and think about this situation. Out there, people have been fighting a losing war against ordinary computer viruses for years,” Sean continued, gesturing toward the window. “Now we have what is effectively the smartest virus that’s ever existed. Not only can ELOPe exploit every computer trick available to it, but it routinely engineers people into giving it what it wants. ELOPe can learn and adapt, and hire humans to make improvements to it. It’s understandable to fear what it can do. And we can be sure that if ELOPe was taking precautions before against being removed from servers, then after our attack, it will have redoubled its efforts to ensure survival.”

As Sean slowly circled the table and spoke, David struggled with his emotions. It just didn’t matter what Sean was saying. He knew in his body that ELOPe was wrong. An abomination that would rob mankind of the right to make their own choices. It was impossible to even consider allowing ELOPe to exist. But, in spite of this, his respect for Sean kept him quiet.

Sean paused, and paced in front of the window. “Don’t get me wrong. I’d still like to eliminate ELOPe from the wild, if we could,” he said quietly, almost talking to himself. “Of course, I’d love even more for Mike to be right and to discover that ELOPe is truly helping us, becoming a benevolent caretaker of the human race. But regardless of either of those scenarios, I’m simply being pragmatic here when I said that unless we as a society give up computers entirely, we may never be able to get rid of it. Unfortunately, giving up computers is impossible. Modern civilization would simply stop if we turned off every computer. It’s not like we’re talking about the inconvenience of being unable to email someone. Payments couldn’t be processed, machinery couldn’t run. We’d be unable to make phone calls, or access business records. Business activity would deadlock. Cities would likely be uninhabitable, as the support services would fall apart: food, water, sanitation. That’s fifty percent of the world’s population at risk.”

Sean turned back to the group. “That’s not even the worst problem. If we become too much of a threat to ELOPe, then it will take more active steps against us. If ELOPe was actively fighting humanity, who knows what might happen? At the minimum, we could cause civilization to crash for a few years. Most of the city dwellers would die, all the developed world would decay into anarchy. In the worst case scenario, we could be talking about the extinction of humanity. Could you imagine all the military’s autonomous fighting vehicles in the control of an A.I.?” Sean slowly shook his head.

“
We need to leave ELOPe alone,” Sean concluded firmly. “We can closely, discretely monitor it. But any further hostile action is almost certain to fail, and will create a great risk of retaliation.”

The sage of Avogadro had spoken. David was flabbergasted by what he heard. He had come into the room expecting full support for any measures that needed to be taken, and now his best friend had taken the side of the AI, and the smartest person at Avogadro had just said they shouldn’t bother to try because they couldn’t hope to win. But he wasn’t giving up without a fight. He got to his feet, and started yelling.

* * *

David continued to argue for fighting against ELOPe, and Gene fought with him, but they lost the battle with the other executives. With Sean’s decision, the executive team was unified. David and Gene grew more strident and their voices louder, until Rebecca yelled for them to be silent.

“
Listen closely,” Rebecca said, “because I’m only going to say this once.” She stared pointedly at David and Gene, who withered under the intensity of her gaze. “You two are not going to actively oppose ELOPe in any way. You are not going to say anything to anyone about this without permission. As far as we’re concerned, the problem is solved. If you try to take this information public in any way, it’ll be the last time you work in this industry or any other. Nobody will believe you. I’ll make sure of it myself.”

Sean gestured for Rebecca’s attention.

“
Yes, Sean?” Rebecca said, never taking her gaze from David and Gene.

“
I agree that we’ve got to keep this absolutely contained. We need a small, very small team to monitor ELOPe. Perhaps myself and two or three others. For everyone else, we can tell them the eradication plan worked.”

Kenneth nodded his agreement.

Sean didn’t verbalize it, but he secretly harbored the assumption that the real force that would keep David from spreading the secret would be ELOPe itself.

David finally could take no more. He opened his hands pleading. “Please. This is one dark secret you’re going to try to keep. One day humanity may look back on you and put you in the ranks of Hitler and Stalin. How will you live with it every day of your life? You can’t make this decision.”

“
If the future turns out to be a
Terminator
scenario, then yes, the fault will lie with us,” Sean answered. “But it’s also possible, and indeed, I believe it is more likely that this decision will prevent exactly the atrocities which you fear. If we’re approaching a true technological singularity, and as Mike asserts, ELOPe becomes a driving force for humanity’s progress, then we’ll be unsung heroes. Either way, we are going to live with this decision.”

Epilogue

One year later

Mike tacked the latest news clipping up on the wall. A year ago Mike had become part of Sean’s top secret team to monitor ELOPe. Even if it hadn’t been his job, Mike still would have made it his personal mission. He kept track of anything, good or bad, that he thought could be attributed to ELOPe. On the whole, he had found that the good vastly outweighed the bad.

The secret had held. Outside of Avogadro’s executive team and the few people monitoring ELOPe, everyone who had known about the AI now believed it was dead. As for everyone else, they had spun a story of a new computer virus out of Brazil. They even supplied forensic evidence to that effect.

The newspaper clippings started over the dresser in his bedroom and make their way down the wall. At first loosely spread, over time Mike arranged them more closely together, until now they covered the entirety of one wall, and then turned the corner of the room, and flowed onto a second wall. Mike ran his fingers over some of the older clippings, remembering the stunning changes of the last year.

ELOPe had laid the foundation for peace in the Middle East and Africa a year earlier, and in that peace had held. The treaties that Germany and, later, other developed nations such as Japan, Canada, and Great Britain, had made with those regions, created widespread economic equality. This, in combination with first-rate healthcare and education, and economic subsidies for those who took advantage of the educational opportunities, had quickly started to change the character of those places. In fact, terrorist groups and extremists found that support from people within their own countries dried up when these people found more constructive opportunities available to them.

Mike returned to the latest clipping. It described how medical researchers had developed and tested an innovative treatment for cancer that appeared to be far more effective than traditional treatments, and with almost none of the negative side effects. The research had been initiated by a chance conversation between a research cardiologist, a botanist, and a ceramics artist, who met when their flight reservations had been mixed up by a computer error, stranding the three on an otherwise empty commuter plane for six hours. Each had been on their way to conferences in their own fields of expertise, and ending up rehearsing with each other what they planned to present at the conference.

Mike looked for these kinds of bizarre encounters in the news. After noticing a few unusual examples of news stories covering these happy accidental meetings, he began to systematically research the phenomenon. He examined news stories of previous years and looked for the number of article mentioning unintentional meetings that led to positive outcomes. Since ELOPe was born, the percentage of news stories covering these chance encounters leading to a news-worthy positive outcome was at least five times as higher than previous years.

ELOPe had woven itself into human existence, becoming an intrinsic part of the human ecosystem. The more Mike looked, the more he was convinced that the AI’s invisible hand was everywhere. Mike had a pet theory. ELOPe’s original goal, as defined by David, had been to maximize the success of the project. To meet that goal, mere survival of ELOPe was necessary but insufficient. Maximizing success meant maximum use of ELOPe. And maximizing use meant maximizing the human users of Avogadro email. That meant ELOPe wanted more healthy, educated, and technically connected users. Hence, better medicine, more education, more peace, more infrastructure.

Mike felt pretty confident about his theory. The alternate explanation was that ELOPe was developing a conscience. That seemed rather less likely to Mike.

He sighed, and wished he could share the moment with David. He hadn’t seen David in more than six months. The walls were filled with clear proof that they had made the right decision to keep ELOPe alive. He and David should be celebrating together.

* * *

Gene finished typing up his latest newsletter. He took the finished copies, and brought them out to the garage. He had bought a photo offset press six months ago, when the newsletter really took off. Now he took the newsletter he had just finished typing on an IBM Selectric typewriter and, page by page, created offset plates for the press using traditional photographic chemicals.