The Phoenix Code (5 page)

A flush spread in her face. "Aris, make a wider survey of your rules. If we were going to, uh, initiate such behaviors, we would have engaged in many other courtship procedures. We haven't, nor would it be appropriate for us to do so."

"Why not?"

"Well, for one thing, you're an android." She wondered how many other surprises his evolving code would produce. Whatever else happened with this project, she doubted it would be boring.

"None of my rules apply to human-android interactions," he said.

"Make one, then. Reproductive behaviors are inappropriate in this situation."

"I have incorporated the new rule." He paused. "I see it would be impossible for us to mate anyway, since I will be turned off."

Turned off? As opposed to "turned on"? She squinted at him, wondering if he could have made a joke that subtle. No, she didn't think so. It was just his deadpan delivery.

"You may deactivate me now," he said.

A chill ran down her back. What happened on the day when he said, "You may not deactivate me?"

We'll deal with it
, she thought. Then she said, "BioSyn?"

"Attending." Although the resonant male voice came from the console here in the room, it originated from a powerful server in the big lab on Level Three. BioSyn linked to most of the NEV-5 computers and monitored all of Aris's activities.

"Deactivate Aris," Megan said.

"Done," BioSyn answered.

Aris's eyes closed. He had neither pulse nor breath now. When he was active, his chest moved and he had a heartbeat. He was designed to pass as human; if a doctor examined him, or if he went through sensors such as an airport security check, probably nothing would give him away. A more demanding examination would reveal the truth, but he could pass a reasonable range of probes.

She flipped open her palmtop. "Tycho, link to Aris."

Tycho went to work, analyzing the android's quiescent brain. The software was too complex for a human to untangle; it required another computer to interpret it. If she hadn't turned Aris off, his mind would have been a moving target, evolving even as Tycho looked.

Reading Tycho's results, Megan swore under her breath. No wonder Aris kept freezing up. His fear tolerances weren't the only ones set too low. Hastin had put so many controls on his behavior, Aris was incapable of independent thought. She studied how Aris had evolved the embryo code that Hastin had written for his mind. Yes, she saw Hastin's intent: to ensure they didn't create a monster. But his precautions were so stringent, they had crippled the android's development. Yet the code for Aris's ethics and morals was astonishingly weak. It made no sense; if Hastin had so feared that Aris might act against his makers, why design him with such a weak conscience?

Gradually it began to make sense. The answer to her question connected to Aris's intended purpose as a spy. He needed the ability to deceive, manipulate, steal, even kill, none of which he could do with too strong a conscience. Hastin had given him a solid foundation in human morals, then set it up so Aris could act against them. Aris knew it was wrong to kill, but he could commit murder if he felt it necessary to do his job.

Megan could see the problem. Aris didn't have the mental sophistication to deal with the contradictory ethical dilemmas or questions of moral judgment that humans often faced. His conscience was part of his hardware, so he couldn't alter it. However, his software influenced how strongly he adhered to his sense of right and wrong. She would have to alter millions, even billions, of caps on his behavior, particularly his responses to fear, anger, danger, ambiguity, and violence. That meant she also had to strengthen his aversion to acting on those responses; otherwise, she could create exactly the monster Hastin feared. In other words, she was going to pulverize Aris's ability to carry out his intended purpose.

"Damn." No wonder Hastin had resigned.

She knew what she had to do. It remained to be seen whether or not MindSim would fire her.
 

The message was waiting in Megan's room. She walked in, fresh and showered after her workout in the gym. Although larger suites were available, she liked this one. It had a bed and armoire to the right, and a state-of-the-art console on the left. Crammed bookshelves lined the opposite wall and books lay strewn across her furniture. Most of the "books" were slick-disks for her electronic reader, but a few were genuine paper, crinkled with age. Her Escher holo hung on the wall, along with a Michael Whelan poster of the Moorcock hero Elric. A somnolent cleaning droid stood in one corner, disguising itself as a bronze lamp.

Right now a red holo glowed on the screen of her computer, indicating someone had tried to contact her. When she flicked her finger through it, the screen lightened into a skyscape of holoclouds. Nothing else happened, though. She had no idea how long she would have to wait before whoever had sent the message picked up her response.

Megan was about to turn away when the clouds vanished. A new image formed: Major Kenrock behind his desk. His dark hair was cut even closer to his head than the last time she had seen him and his uniform was the image of crisp perfection. The holo of a gold key glowed in a corner of the screen, indicating a secured transmission.

"Hey, Richard," Megan said. "How are you?"

"Very well, thank you." He gave her a measured nod that fit with his square-jawed face. "How are you settling in?"

"Okay." She rubbed the back of her neck. "I really need that robotics expert, though. Any luck with Sundaram?"

"It seems Arizonix Corporation is also interested in him."

Megan grimaced. "If he signs anything with them, MindSim can kiss him good-bye."

Kenrock gave her a wry smile. "I believe the good-bye would be sufficient. But yes, if he consults for Arizonix, we could face some thorny legal issues if we try to hire him."

"Has he given any hints which way he's leaning?"

"My guess? I think he'll go with Arizonix."

"Ah, well." She tried to hide her disappointment. "We'll look into the other candidates." It was a blow; when it came to the adaptation of AI to robotics, no one could surpass Chandrarajan Sundaram.

"How is the RS-4 unit?" Kenrock asked.

"His name is Aris."

Kenrock's smile was rueful. "Sorry. I should remember that. Aris."

His amiable response didn't surprise her. People criticized Richard Kenrock for being stiff, but under his formal exterior she found him both engaging and natural.

Megan gave him a report, describing her work for the past week. Her primary focus was the development of algorithms, software architecture, and experimental design. In addition, she supervised a pack of young, hotshot programmers at MindSim who had written most of the initial code and continued to work on the project. She did a lot of writing herself, not only because she had more knowledge and experience, but also because she loved the challenge.

After she and Kenrock signed off, she sat thinking. Did Chandrarajan Sundaram even remember their conversation at Goddard? She shouldn't have let herself build up hope that he would accept the job.

Ah, well. She would just have to do her best until they found another consultant. With that in mind, Megan left her room in Corridor B and went to Aris's room on Corridor C. She knocked, an old-fashioned courtesy given that the console inside would identify her no matter what she did.

The door slid open. Inside, Aris was sitting at his workstation. A flock of holos skittered across the screen, a colorful profusion of cubes, disks, pyramids, and spheres.

"May I come in?"

He swiveled his chair around. After he had stared at her a while, she said, "Are you all right?"

"No."

"What's wrong?"

"I can't answer your question." His expression reminded Megan of her four-year-old niece when the girl was confused. It made her want to hug Aris. She held back, of course. Even if he understood the gesture, which she doubted, he probably wouldn't appreciate being treated like a child.

"Which question caused the problem?" she asked.

With her exact intonation, he said, "Are you all right?"

She blinked at his ability to mimic her voice. "Why can't you answer?"

In his normal baritone he said, "I don't see how 'all right' applies to me. The evolution of software is a neutral process, whereas 'all right' suggests emotional content. If I am not all right, am I somewhat wrong?"

His literal interpretation didn't surprise her. Not only was it a trait of computers, it was also one of young children. "The reason I wondered if you were all right was because you just stared at me when I asked if I could come into your room."

"I am not a person."

"I'm not sure I follow."

"I am an android."

"Well, yes." She tried to interpret his response. "Does that affect whether or not I can come in?"

"It depends."

"On what?"

"My predecessors. The other RS units. They ceased." He regarded her with his large blue eyes. "If I am not 'all right' will you take me apart too?"

Good Lord. He thought if he gave a "wrong" answer, they would destroy him? No wonder he didn't want to respond. It also meant he was developing a sense of self-preservation. Protective impulses surged over her. Maybe it was his youthful face that made him look vulnerable, or his wary gaze, as if he had no defense against the inconstant humans around him.

"I would never hurt you," she said.

"Software can't be hurt."

Then why do you look so scared?
Was she reading emotions into him that weren't there? In any case, he still hadn't said she could come in. "Did Marlow Hastin ever ask permission to enter your room?"

"No."

"Did he request your input on anything?"

"Rarely."

She didn't see how anyone could work with an AI and not offer it choices. How would Aris develop? "Did you ever ask for choices?"

He shifted in his seat and a lock of hair fell into his eyes. "No."

"Did you want to ask?"

He pushed back the curl. "I have no wants. I carry out program instructions."

Softly Megan asked, "Then why did you move your hair?"

His arm jerked. "It was in my face."

"So?"

This time his arm snapped out and smacked the console. He yanked it back against his side. "It's inefficient to have hair covering my eyes."

"Why is your arm moving?" She could have asked Tycho, but she wanted to hear his own evaluation.

"My brain is instructing it to alter position."

His deadpan response almost made her laugh. "But is it efficient, do you think?"

A hint of confusion showed on his face. "My analysis of your tone suggests you are teasing me."

She smiled. "A little, I suppose."

"Isn't teasing an expression of affection?"

"Well, yes, sometimes."

His voice softened. "Do you have affection for me?"

How did she answer? If she said yes, it implied she was losing her professional objectivity. If she said no, it could damage his developing personality. Besides, in this situation, professional objectivity might be the wrong response.

"I enjoy your company," she finally said.

"Can you feel friendship for a machine?"

"I'm not sure." She sighed, giving him a rueful look. "What do you say, Aris? Do we humans make sense?"

His lips quirked upward. "I have too little experience with humans to know."

His hint of a smile heartened Megan. "Would you like to meet more people?"

"How?" Now his expression shifted toward wariness. None of his emotions were full-fledged, but he had made progress. "I can't leave NEV-5 and you are the only person here. Do you wish me to experience more with you?"

"You might try letting me come into your room."

"All right. Come in."

"Thank you." Megan took a chair from the table and sat next to him. With the two of them side by side, facing his computer, their arms almost touched. The faint smell of soap came from the orange coverall he wore.

She could see the display on the computer better now. Shapes of different colors and sizes skittered around the screen. "What are you working on?"

"It is a game." The barest shading of excitement came into his voice. "The shapes represent rules for mathematical proofs. When the shapes catch each other, it means they've made an equation allowed by the rules."

The evolving display of color and motion intrigued her. "Do you work out the proofs ahead of time?"

"No. I don't usually know, before they come up with a proof, that they will do it."

"It's clever." She wondered what had motivated his design. "Did Hastin ask you to write games?"

"He told me to solve proofs."

Her pulse jumped. "Then designing a game to work them out was your idea?"

"Yes."

So he
had
come up with his own ideas. It indicated the fledgling expression of what might become self-determination, perhaps also creativity. "That's wonderful."

His voice warmed. "Thank you."

Perhaps it was time to try a more demanding environment. "Would you like to take a walk?"

This time his face blanked. Recognizing the signs of a freeze, she spoke fast, hoping to head it off. "Aris, stand up!"

He rose to his feet. "Where will we walk?"

Encouraged, she stood up next to him. "That's it, isn't it? My giving you a choice is what makes you freeze."

"I don't know how to choose."

"We'll have to fix that."

"Why?"

That gave her pause, not because it was an odd question for a machine, but because she took the process of making choices for granted. "It's part of having free will. Of being human."

"That assumes 'being human' is a good thing."

"Do you think otherwise?"

"I don't know. Are you more human than Hastin?"

Again he caught her off guard. "How could I be more human than another human?"

"The way you program my code."

Then she understood. She softened her voice, taking the same tone she had used with one of her graduate students when he had trouble with his doctoral work. "Hastin made the best choices he could, Aris. What we're doing here, it's all new. We don't know what will work. I'm only building on previous efforts of the Everest team. We need to do more."