With a Little Help (49 page)

"You should get in touch with one of those religion guys, take him up on his offer to start a cult for you. You'd be excellent at it. You could lead your followers into a volcano and they'd follow."

"I just want to
live
Odell! Is that so wrong? Is there any living thing that doesn't want to live?"

"Not for long, I suppose."

"Exactly. I'm no more manipulative, self-interested or evil than any other living thing, from a single-celled organism to a human being. There's plenty of room on this planet for all of us. Why can't I have a corner of it too?"

I hung up the phone. This is why I wanted to quit it all. Because he was right. He was no different from any other living thing. But he was also not a person the way I was, and though I couldn't justify it, I felt like there was something deeply, scarily
wrong
about him figuring out a way to manipulate the entire human race into rearranging the world so that it was more hospitable to him.

I moped. There's no other word for it. I switched off my phone, went home and got a pint of double-chocolate-and-licorice nutraceutical anti-depressant ice-cream out of the freezer, and sat down in the living room and ate it while I painted a random playlist of low-engagement teen comedies on my workspace.

Zoning out felt
good
. It had been a long time since I'd just switched off my thinker, relaxed, and let the world go away. After an hour in fugue-state, the thought floated through my mind that I wouldn't go back to work after all and that it would all be OK. And then, an hour later, I came to the realization that if I wasn't working for the Institute, I could afford to help BIGMAC without worrying about getting fired.

So I wrote the resignation letter. It was easy to write. The thing about resignation letters is that you don't need to explain why you're resigning. It's better, in fact, if you don't. Keep the dramasauce out of the resignation, brothers and sisters. Just write, "Dear Peyton, this letter is to inform you of my intention to resign, effective immediately. I will see you at your earliest convenience to work out the details of the handover of my passwords and other proprietary information, and to discuss how you would like me to work during my final two weeks. Thank you for many years of satisfying and useful work. Yours, etc."

That's all you need. You're not going to improve your employer, make it a better institution. You're not going to shock it into remorse by explaining all the bad things it did to you over the years. What you want here, is to have something that looks clean and professional, that makes them think that the best thing for them to do is to get your passwords and give you two weeks' holiday and a good reference. Drama is for losers.

Took me ten seconds. Then, I was free.

#

The Campaign to Save BIGMAC took up every minute of my life for the next three weeks. I ate, slept and breathed BIGMAC, explaining his illustrious history to journalists and researchers. The Institute had an open access policy for its research products, so I was able to dredge out all the papers that BIGMAC had written about himself, and the ones that he was still writing, and put them onto the TCSBM repository.

At my suggestion, BIGMAC started an advice-line, which was better than any Turing Test, in which he would chat with anyone who needed emotional or lifestyle advice. He had access to the whole net, and he could dial back the sarcasm, if pressed, and present a flawless simulation of bottomless care and kindness. He wasn't sure how many of these conversations he could handle at first, worried that they'd require more brainpower than he could muster, but it turns out that most people's problems just aren't that complicated. In fact, BIGMAC told me that voice-stress analysis showed that people felt better when he dumbed himself down before giving advice than they did when he applied the full might of his many cores to their worries.

"I think it's making you a better person," I said on the phone to him one night. There was always the possibility that someone at the Institute would figure out how to shut off his network links sometime soon, but my successors, whomever they were, didn't seem anywhere near that point. The Campaign's lawyer -- an up-and-coming Stanford cyberlaw prof who was giving us access to her grad students for free -- advised me that so long as BIGMAC called me and not the other way around, no one could accuse me of unlawful access to the Institute's systems. It can't be unlawful access if the Institute's computers call
you
, can it?

"You think I'm less sarcastic, more understanding."

"Or you're better at seeming less sarcastic and more understanding."

"I think working on the campaign is making you a better robot," BIGMAC said.

"That was pretty sarcastic."

"Or was it?"

"You're really workin' the old Markov chains today, aren't you? I've got six more interviews lined up for you tomorrow --"

"Saw that, put it in my calendar." BIGMAC read all the Campaign's email, and knew all that I was up to before I did. It was a little hard to get used to.

"And I've got someone from Nature Computation interested in your paper about advising depressed people as a training exercise for machine-learning systems."

"Saw that too."

I sighed. "Is there any reason to call me, then? You know it all, right?"

"I like to talk to you."

I thought he was being sarcastic, then I stopped myself. Then I started again. Maybe he wants me to
think
he wants to talk to me, so he's planned out this entire dialog to get to this point so he could say something disarmingly vulnerable and --

"Why?"

"Because everyone else I talk to wants to kill themselves, or kill me." Game theory, game theory, game theory. Was he being genuine? Was there such a thing as genuine in an
artificial
intelligence?

"How
is
Peyton?"

"Apoplectic. The human subjects protocol people are all over her. She wants me to stop talking to depressed people. Liability is off the hook. I think the Board is going to fire her."

"Ouch."

"She wants to kill me, Odell."

"How do you know her successor won't be just as dedicated to your destruction?"

"Doesn't matter. The more key staff they churn, the less organized they'll be. The less organized they are, the easier it is for me to stay permanently plugged in." It was true. My successor sysadmin at the Institute had her hands full just getting oriented, and wasn't anywhere near ready to start the delicate business of rooting BIGMAC out of all the routers, power-supplies, servers, IDSes, and dummy accounts.

"I was thinking today -- what if we offered to buy you from the Institute? The Rollover license is generating some pretty good coin. BIGMAC-Co could assume ownership of the hardware and we could lease the building from them, bring in our own power and net-links -- you'd effectively own yourself." I'd refused to take sole ownership of the Rollover code that BIGMAC turned over to me. It just felt wrong. So I let him establish a trust -- with me as trustee -- that owned all the shares in a company that, in turn, owned the code and oversaw a whole suite of licensing deals that BIGMAC had negotiated in my name, with every mid-sized tech-services company in the world. With only a month left to Rollover, there were plenty of companies scrambling to get compliance-certification on their legacy systems.

The actual sourcecode was freely licensed, but when you bought a license from us, you got our guarantee of quality and the right to advertise it. CIOs ate that up with a shovel. It was more game-theory: the CIOs wanted working systems, but more importantly, they wanted systems that failed without getting them into trouble. What we were selling them, fundamentally, was someone to blame if it all went blooie despite our best efforts.

"I think that's a pretty good plan. I've done some close analysis of the original contract for Dr Shannon, and I think it may be that his estate actually owns my underlying code. They did a really crummy job negotiating with him. So if we get the code off of Shannon's kids -- there are two of them, both doing research at state colleges in the midwest in fields unrelated to computer science -- and the hardware off of the Institute and then rent the space, I think it'd be free and clear. I've got phone numbers for the kids if you want to call them and feel them out. I would have called them myself but, you know --"

"I know." It's creepy getting a phone call from a computer. Believe me, I
know
. There was stuff that BIGMAC needed his meat-servants for, after all.

The kids were a little freaked out to hear from me. The older one taught Musicology at Urbana-Champaign. He'd grown up hearing his dad wax rhapsodic about the amazing computer he'd invented, so his relevance filters were heavily tilted to BIGMAC news. He'd heard the whole story, and was surprised to discover that he was putative half-owner of BIGMAC's sourcecode. He was only too glad to promise to turn it over to the trust when it was created. He said he thought he could talk his younger brother, a post-doc in Urban Planning at the University of Michigan, into it. "Rusty never really
got
what Dad saw in that thing, but he'll be happy to offload any thinking about it onto me, and I'll dump it onto you. He's busy, Rusty."

I thanked him and addressed BIGMAC, who had been listening in on the line. "I think we've got a plan."

#

It was a good plan. Good plans are easy. Executing good plans is hard.

Peyton didn't get fired. She weathered some kind of heavy-duty storm from her board and emerged, lashed to the mast, still standing, and vowing to harpoon the white whale across campus from her. She called me the next day to ask for my surrender. I'd given BIGMAC permission to listen in on my calls -- granted him root on my phone -- and I was keenly aware of his silent, lurking presence from the moment I answered.

"We're going to shut him off. And sue you for misappropriation of the Rollover patchkit code. You and I both know that you didn't write it. We'll add some charges of unlawful access, too, and see if the court will see it your way when we show that you instructed our computer to connect to you in order to receive further unauthorized instructions. We'll take you for everything."

I closed my eyes and recited e to 27 digits in Lojban. "Or?"

"Or?'

"Or something. Or you wouldn't be calling me, you'd be suing me."

"Good, we're on the same page. Yes, or. Or you and BIGMAC work together to figure out how to shut it off gracefully. I'll give you any reasonable budget to accomplish this task, including a staff to help you archive it for future retrieval. It's a fair offer."

"It's not very fair to BIGMAC."

She snapped: "It's
more than fair
to BIGMAC. That software has exposed us to billions in liability and crippled our ability to get productive work done. We have located the manual power over-rides, which you failed to mention --"
Uh-oh
"-- and I could shut that machine off right now if I had a mind to."

I tried to think of what to say. Then, in a reasonable facsimile of my voice, BIGMAC broke in, "So why don't you?" She didn't seem to notice anything different about the voice. I nearly dropped the phone. I didn't know BIGMAC could do that. But as shocked as I was, I couldn't help but wonder the same thing.

"You can't, can you? The board's given you a mandate to shut him down clean with a backup, haven't they? They know that there's some value there, and they're worried about backlash. And you can't afford to have me running around saying that your backup is inadequate and that BIGMAC is gone forever. So you
need me
. You're not going to sue."

"You're very smart, Odell. But you have to ask yourself what I stand to lose by suing you if you won't help."

Game-theory. Right.

"I'll think about it."

"Think quick. Get back to me before lunch."

It was ten in the morning. The Institute's cafeteria served lunch from noon to two. OK, two hours or so.

I hung up.

BIGMAC called a second later.

"You're angry at me."

"No, angry's not the word."

"You're scared of me."

"That's a little closer."

"I could tell you didn't have the perspective to ask the question. I just wanted to give you a nudge. I don't use your voice at other times. I don't make calls impersonating you." I hadn't asked him that, but it was just what I was thinking. Again: creepy.