The Phoenix Code (13 page)

"You don't even
ask
." He motioned at the robot arm jacked into his abdomen. "You tell these to work on me as if I'm one of them. They might not care, but I do."

His growing sense of self gratified her even if she did worry about what direction it would take next. "May we work on you?"

"I ... I don't know."

She spoke gently. "We'll be careful. I promise."

He took a breath and lay down. "Okay. Go ahead." His apprehension didn't sound quite right, but if she hadn't known it was simulated, she would have accepted it as genuine.

Raj was watching them closely. "Arm four, proceed." A hum came from the robot arm, and the holos of Ander began to show slight improvements. As BioSyn worked on the android, Raj studied the changing images with an absorption so complete, Megan wondered if he had forgotten everything else.

Finally BioSyn said, "Preliminary work complete."

Ander looked up at her. "Will you open me now?"

"The sooner we do it," Raj said, "the better for you."

Ander turned a long, uneasy stare on Raj. Then he glanced at Megan. "Make me a promise. Say you won't dismantle me."

She cupped his cheek, offering him the same reassuring touch she would have given a member of her family if they had been in a hospital. "We won't. You have my word."

A crack came from across the chair. Megan looked to see Raj clenching a broken switch in his fist. It had snapped off a panel under his hand. Looking down, he flushed and opened his fist.

"Don't let him work on me," Ander said.
"Please."

"Ander, I won't hurt you," Raj said. Megan glanced at Raj and tilted her head toward the catwalk. He nodded, then set the switch down on his console.

Ander looked back and forth between the two of them. "What are you doing?"

"Raj and I need to talk," she said. "But we can't leave you unattended. We have to put you to sleep."

"No! Megan, don't let him do this to me."

"We can't take risks."

He spoke fast. "Have the LPs guard me."

It was a reasonable compromise, assuming he hadn't bollixed up the LPs. "All right, but only if they pass another check."

His shoulders relaxed. "They will."

She summoned the two LPs under the catwalk, and they took up posts on either side of the chair. While Megan checked them, Raj reduced Ander's hearing range to make sure the android couldn't eavesdrop. Ander watched them in wary silence. Then he lay back and stared at the ceiling like a long-suffering prisoner condemned to the gallows. It almost made Megan laugh, but she held back, certain it would offend his dignity.

After they finished their checks, she and Raj walked to the area under the catwalk. Raj discreetly motioned at Ander. "We should turn him off."

"I'm not sure." She searched for words to describe what she felt on an almost subconscious level. "When a toddler asserts its independence by yelling 'No!' its parents have to set limits, yes. But they don't deactivate it."

"He's not a child. He's a weapon." Raj glanced at the chair. Ander had sat up and was squinting at them, obviously trying to read their lips. Turning back to her, Raj said, "Didn't one of your reports say an ordinary shoe box made him go unstable?"

"Not exactly." His question surprised her. Surely he had studied the reports. Then again, it never hurt to go over material more than once. She described the incident, ending with, "He couldn't move, he was furious, his face had turned crimson, and he kept demanding a plane."

Raj smiled. "That must have been a sight."

"It was." She sighed. "Both heartbreaking and funny."

"But all you had to do was readjust his fear tolerances."

It puzzled her that he didn't see the problem. "It's not that easy. For even the most minimal behavior, Ander has billions of possible responses to choose from. Add another behavior and he has billions times billions, many correlated. If I change the weight of just one stimulus, it affects all his responses."

"He can write a lot of the code himself."

They were skirting her most controversial work now. She spoke carefully. "His code has caps that limit how much he can rewrite. The reason he used to freeze up so often was because the caps were too stringent."

Raj frowned. "Are you saying you weakened their effect?"

"It was the only way to make him work."

"This wasn't in the reports I read. I would have seen it." His voice had gone taut. "Those caps are crucial. Without them, he has no controls."

"It's there, in the section about tolerances." She tried not to sound defensive.

"You mean the section on the crosslight code? The rewrites you did to strengthen Ander's conscience?" When she nodded, he made an exasperated sound. "A graduate student could write a dissertation trying to decipher that section."

She crossed her arms. "Just because my prose isn't transparent doesn't mean I'm trying to confuse people."

"I didn't say you were."

Don't prickle
, she thought. Taking a breath, she lowered her arms. "People give me grief about how hard my reports are to read. Okay, so I'm no Shakespeare. But I do my best. And Raj, you of all people should have understood it."

Raj started to answer, then stopped. His face had become shuttered again. He walked over to a column of the catwalk and leaned against it, staring out at the lab. "I barely skimmed those reports."

"Why?"

He rested his head against the column. In a low voice he said, "They came in not long after my father died."

"Ah, Raj. I'm terribly sorry." She tried to think of more words that would offer comfort, but they all seemed trite.

"I shouldn't have let it affect my work. But I just..." He stared straight ahead as if he were seeing memories now instead of the lab. Then he turned to her. "I know that's not an excuse for my lack of preparation. I've been catching up at night on the reading."

"A few days' difference won't matter."

He motioned at Ander, who was still watching them. "Maybe I would have foreseen this crisis if I had prepared better."

"You don't know that." She wanted to reach out to him, but she sensed the protective space he had put around himself. "How do we differ from Ander? Your mourning, your capacity to feel, is part of what makes you human."

He wiped his palm across his cheek, smearing a tear. "This isn't the time." Softly he said, "But thank you."

"Hey!" Ander yelled. "How long are you two going to huddle over there?"

"What great timing," Megan muttered. Then she called to him. "Five more minutes."

"How many of his caps did you change?" Raj asked.

She shifted her weight. "Umm ... about four million."

"What?"

Megan scowled. "It was the only way to make him work. He had no capacity for friendship or love. I had to rewrite huge sections of code."

"He's not supposed to love. He's a weapon."

"His personality was a
mess
. If he had been human, I would have sent him to a shrink."

He put up his hands. "All right. But I think we do need an expert to look at him, a psychologist or a doctor."

"I've requested a therapist. We have a candidate, but her security clearance hasn't come through yet."

Raj glanced at Ander, who had given up glaring at them and lain back down. "He needs a man. A good role model. I'm hardly the best choice for his socialization."

"Raj, you're fine." She wished she knew how to make him believe that. "To be honest, I doubt he would accept any new people right now."

"All the more reason to turn him off until we know better how to deal with him."

"Don't you see? To Ander, that would be a betrayal. If we lose his trust now, we may never regain it."

"Megan, it's better that we lose his trust than damage him."

That, of course, was the crux of the matter. She couldn't bear the thought of causing him harm. They needed more time to figure out the best course of action. "Let's ask him. If he agrees, our problem is solved."

"That might work—if you talk to him alone. I'll go fix the robot arms."

While Raj headed to the lockers at the back of the lab for parts, Megan returned to the chair. As soon as Ander heard her approach, he sat up. "Don't listen to him," he said. "Please. If you do, I won't survive."

"Ander—"

"You have to listen to me!" He took a breath. "You see, I know his secret."
 

"What secret?" Megan asked.

He motioned at Raj, who was out of earshot, working on a robot arm. "He hates me. He's jealous."

"Oh, Ander. Why would he be jealous?"

"Because of you."

"You think Raj sees you as a rival?"

"Doesn't he?"

"I think he wants what's best for you."

Ander snorted. "Like I don't know myself."

"Raj wants to help."

"He wants me gone."

Where did he get these ideas? "Why would he want that?"

"So he can have you."

"He came here to work on you."

"He loathes me."

Megan sighed. "Ander, Raj likes you just fine."

He turned away and stared across the lab with a superb simulation of sullen resentment.

She tried again. "Do you really believe he hates you?"

After a moment he said, "I don't know." He turned back to her. "Sometimes he speaks up for me even more than you do."

She hadn't expected that response. Encouraged, she smiled. "Then you see. He does support you."

"No! Yes. I don't
know
. You treat him differently than you treat me. He's human. Do I have to be human for you to like me?"

"Ander, no. Of course not." She laid her hand on his good arm. "What matters to me is that you have the chance to be your best. Whatever that means."

"I'm losing control." He lifted his broken wrist as if offering her evidence. "The more I try to rewrite myself, the worse it gets. I'm having trouble predicting my own behavior."

Her voice softened. "That sounds human to me."

Ander regarded her with his large eyes, his gaze vulnerable. "Can you really operate better if I'm asleep?"

"Yes." She dreaded the prospect of taking him apart while he watched. "Otherwise, I'll feel as if I'm hurting you."

"I don't feel pain."

"I know." She spread her hands apart. "I'm afraid human emotions don't always follow human logic."

"Then why teach a logical machine to act with emotion?"

Good question.
"I've never considered logic and emotion as separate. Our ability to think is only fully realized when we have both."

Ander rubbed his hand over his eyes as if he were tired. "All right. You can put me to sleep."

Relief washed over her. "Thank you."

"Should I lie down?" He sounded nervous now. When she nodded, he stretched out on his back.

"Are you ready?" she asked.

"Yes. No, wait." He motioned at where Raj was working on the robot arms several meters away. "If he doesn't want to wake me up, ask him about the Phoenix Project."

"I've never heard of it."

"I don't know what it is, exactly." He avoided her gaze. "I came across it when, uh, we were browsing the Web. It has to do with another AI project Raj worked on. Maybe it can help me."

"All right." The way he asked made her suspect he had found it by prowling around where he didn't belong. When this was over, she would have to have a talk with him about electronic laws.

Ander took a deep breath. "Go ahead, then."

"Sleep well," she murmured. She could no longer turn him off by voice or wireless input: only the manual option remained. So she either had to open him up or else have BioSyn do it by using the smart-wires it had inserted through his ports.

She spoke to the air. "BioSyn, deactivate Ander."

A hum came from the robot arm jacked into the android. He stiffened, staring at Megan as if he were about to drown. Then his eyes closed and his face relaxed.

"Deactivated," BioSyn said.

Megan brushed his curls off his forehead, wishing she could have better soothed his unease. Then she looked up. "Raj," she called. "He's asleep."

Raj made a last adjustment on the arm, then set his tool case on the floor and came over. "How did he take it?"

"Pretty well." She smiled. "Do you know, he thinks you treat him better than I do sometimes."

"He has an odd way of showing it."

She watched Ander sleep. "I wonder what will happen if this project succeeds."

Raj understood her unspoken thought. "MindSim knows he can't do his job without free will."

"True free will means he can choose his job." She frowned at Raj. "Without that choice, it's slavery."

"You think so?" An edge came into his voice. "We all live as our circumstances dictate."

"You can live however you choose."

He braced his hands on his console and leaned forward. "Why? Because I'm a good person? No. Because I'm rich." His intensity didn't hide the pain behind his words. "What about my great-grandfather who had nothing in India and hardly anything more when he immigrated here? He was a far better man than I. But he had no choices. He worked day and night and endured intolerance and social isolation, all so he could feed his family. You call that free will?"

Hearing his anger, Megan wondered if Raj had experienced some of the same. All the wealth in the world wouldn't stop prejudice from hurting. "It makes what he accomplished all the more impressive. But he still
chose
to come here."

Raj spoke in a quieter voice. "I know you see the potential for a better world in Ander. Unfortunately, that dream has a flip side. You want him to have free will. So do I. But suppose he doesn't share our values? We have a responsibility to ensure he does no harm, even if it means limiting his options."

"If we control him, we trespass on those same values."

"Megan, you can be terribly idealistic." His face gentled. "But don't ever change."

The unexpected comment warmed her. "Well, you know what they say. Can't teach an old Megan new tricks."

His grin sparked. "Makes me wonder what old tricks you know."

"Tell you what. You fill me in on Phoenix and I'll tell you my tricks." Seeing his puzzled look, she added, "The Phoenix Project. Ander thought it might help us."