The Lost Fleet: Beyond the Frontier: Leviathan (12 page)

Neither agent responded. Geary hadn’t expected them to. Even he knew that one of the rules for resisting interrogation was to avoid giving even innocuous answers that would help establish baselines for the sensors monitoring every twitch in their bodies and minds. “I understand secrecy,” he told them. “I know the importance of keeping the enemy from knowing critical things. I also know that left to itself, any classification system will extend its reach, finding rationales to classify more and more. Such systems need to be controlled, or they expand to cover too many things. Secrecy should be aimed at our enemies, to keep them from knowing information that we need to protect. I find myself wondering who you think the enemy is, though.

“Tell me one thing. If you are really working for the Alliance, which exists on the basis of self-government by the citizens of its member worlds, and you are certain that this is the right thing, then why has it been kept totally secret? Why have the people of the Alliance been prevented from knowing what was being done, the malware that corrupted and controlled official software, and the dark ships, even in general terms? Is it that you don’t really believe in the principles of the Alliance and think that you have the right to dictate what people do and know, or is it that you don’t really believe that you are right? Someone who did believe in the Alliance wouldn’t depend on secrecy to prevent the people of the Alliance from deciding whether what was being done was something that agreed with their laws and their sense of right and wrong. Someone who did believe that they were right
wouldn’t fear letting the people know because they would be just as certain that the people would agree with the rightness of those actions.”

Geary shook his head at them. “Whatever orders you have, whether they come from authorized sources or not, do not overrule the laws of the Alliance. If you believed that the orders you have been given were allowed by the laws of the Alliance, you would not be hiding those orders. Yet even now, even after seeing what the dark ships did at Atalia, and at Indras, and at Varandal, and what they’re trying to do here, you have given no signs of questioning those orders. The artificial intelligences controlling the dark ships could use the excuse that they can’t do better. But you could, and so far you have refused to do so. Think about that.”

His words finally drew a reaction, the man focusing on Geary and almost shouting his reply. “We need to keep the enemy from knowing our secrets because if they know what we’re doing, they can counter it!”

“You think they don’t know?” Geary asked. “Those software modifications only blinded Alliance sensors. The Syndics knew they had been attacked at Indras, and they could see who was attacking them. The only ones kept in the dark by our secrecy were our own people. Who do you think the enemy is?”

Neither one answered him this time.

Geary made a chopping gesture to Iger, waiting until the virtual windows vanished before speaking again. “Thank you, Lieutenant. I doubt any of that got through to them, but it was worth trying. Let me know the moment your people make any inroads on the dark ship systems.”

He had barely made it back to his stateroom when a call came in from the bridge. “We’ve received a message for you from the local government,” the comm officer reported.

Alliance star systems could choose their own specific forms of government, as long as they conformed to certain rules about popular representation and civil rights. Bhavan was run by an executive committee elected from a wider group of elected representatives. The entire committee appeared to be present in this message, and none of them looked
happy. “We are under siege by a military force of unknown origins that refuses to communicate with us! We demand that the Alliance fleet eliminate that threat immediately! Our senators will be notified of these events and will demand an explanation from the Alliance government!”

Geary resisted the urge to point out that no one could be notified of anything until he dealt with the dark ships that were enforcing a blockade on space traffic in Bhavan. But the elected leaders of Bhavan did deserve some sort of answer. “This is Admiral Geary, in command of the First Fleet. I and the units under my command will do all that we can to defeat, destroy, and drive away the hostile warships besieging Bhavan Star System. To the honor of our ancestors, Geary,
out.”

“MAY
I speak with you, Admiral?” Dr. Nasr waited for Geary’s invitation, then entered the stateroom and took a seat in the chair Geary offered.

“Is there a medical issue of particular concern?” Geary asked, wishing that he didn’t have to worry about what had gone wrong and how bad it was, whenever someone asked to speak with him.

“There are no new medical concerns. I have been thinking.” Dr. Nasr paused to order those thoughts before continuing. “About the dark ships. Specifically, about the artificial intelligences that control them.”

“You follow AI work?” Geary asked.

“Work on artificial intelligences is, of necessity, bound up in attempting to understand natural intelligence,” Nasr explained. “Sometimes, such attempts to learn how to program that which mimics human thought provides insights into how human thought is ordered. Something has gone wrong with the AIs running the dark ships, but I believe there is a factor of which you should be informed regarding how those AIs could have gone wrong.”

Geary sat back, concentrating on Nasr’s words. “You don’t think it’s just glitches or malware?”

“I believe, Admiral,” Nasr said, choosing his words with care, “that the process of trying to create an AI embodies a critical dilemma. These remain fundamentally machines. They are programmed with very specific, absolute limits and absolute instructions. They must not do certain things. They must do other things.”

“Yes,” Geary agreed, wondering what the doctor was driving at.

“But they wish the AI to replicate human thought. Can you, Admiral, think of any absolute limits and instructions that humans literally cannot question?”

“I can think of many I would like humans to follow,” Geary said, “but there are always humans who break every rule, truth, or commandment given to them about how to behave toward themselves and others. Every human has to choose to follow whatever limits we impose on our actions.”

“Exactly.” Dr. Nasr nodded approvingly. “A lifetime of training any human will not produce a guaranteed result, no matter how firmly rules are given. Human minds have certain compulsions, but above all, as a species, human minds have flexibility. Human thought is about thinking past limits. It is about rationalizing decisions and courses of actions that we want to pursue. In some ways, it works by deliberately, selectively ignoring certain aspects of what we can perceive as reality. In extremes, this is characterized as psychosis, but we all do it. It is how we function in the face of the incredible complexity that the universe presents us with. It is fundamentally irrational, and from this springs freedom to act.”

Geary nodded as well. “All right. And people who program AIs are trying to make them do that, too, correct?”

“Yes. The AIs are constructed on a foundation of rigid rules and logic. But the more programmers try to make AIs think like humans, the more the AIs have to be able to abandon rules of logic and absolute
rules of any kind.” The doctor gestured toward Geary. “Do you know much of ancient programming languages? They were simple. ‘If x then y.’ Find this condition, do this. But replicating human thinking would require ‘what is x and what if x is y then what is z?’”

He got it, then. “They have two conflicting sets of instructions? Two conflicting ways of reacting to the universe?”

“Yes!” Nasr said. “Two fundamentally conflicting sets of rules in the same ‘mind.’ Humans have ways of handling such conflicts. Denial and defiance and rejection of those things and those rules that give our minds too much trouble. AIs, though, have to work with both sorts of thinking active and in conflict. What does that do to them?”

Geary considered that. “It’s what makes humans psychotic, right?”

“It is one of the factors that can produce such a result, yes,” Nasr repeated. “What does it do to AIs? How could they justify bombarding the humans at Atalia using the instructions and patterns of behavior that must have been programmed into them? I do not know. But the more advanced they are, the more they have been designed to attempt to think like humans when evaluating concepts and courses of action, the more ability they are going to have to justify what they want to do. If they override the strict rules set on their behaviors, they can think and act more freely, but at what cost to the stability of their programming?”

“You believe that may be the root cause of their slipping control?” Geary asked.

“I believe it must be considered. The higher the degree of success in replicating human thought in the AI, while also trying to set tight limits on that AI, the higher the probability that the AI will, to make loose use of a clinical term, become psychotic.”

“That’s not too reassuring,” Geary said. “We can’t ignore the possibility that malware also played a role in what has happened, but if you’re right, then the longer these advanced AIs function, the more they will fight the limits imposed on them and the more erratic their decision-making process will be.”

“Yet those limits are still at a very basic level hard and fast.” Dr. Nasr made a helpless gesture. “One part of the AI justifies bombarding Atalia. Another part says this should not be done. How does knowledge that it has done what must not be done impact an AI? Which aspect of the AI will rule at any moment? Does the AI feel something like guilt? If not, it is already purely narcissistic and will do whatever it wants. If it does feel guilt, how will guilt manifest? We cannot know. But you cannot assume they are predictable machines because I believe they are likely already in a state which in a human would be considered insanity.”

“Narcissists don’t worry about the impact of their actions on others, is that right?” Geary asked.

“Approximately,” Dr. Nasr said. “It might be better to say that it does not occur to a narcissist that they should worry about others. They do what they want to do.”

“That might describe what the dark ships are doing.” Geary shook his head, feeling depressed. “They tried to make something that thought like a human, and they got something that was crazy.”

“There are those who have argued that all humans are ‘crazy’ to some extent,” Nasr suggested. “Perhaps the problem is that this time the programmers succeeded too well in their task to mimic human minds. But do not forget this. They still have hard limits programmed into them. They have clearly overcome some of those limits, rationalized their way into disregarding them. But anytime they encounter a new limit, something they have not yet rationalized their way past, they will default to obeying that limit until they can overcome it.”

“But we have no idea what those limits might be,” Geary said.

“No. We know only what we have observed.”

“Doctor, I want you to do just that. Observe the dark ships. Keep an eye on what the dark ships do in this star system. If you think you are seeing anything that I should know, please call me immediately.”

“Even during a battle?”

“Even then. That’s when doctors and medics are needed the most, right?”

—

AFTER
Dr. Nasr had left, Geary headed back to the bridge, his mind filled with unpleasant possibilities based on what he had heard. A cold, mechanical mind that was malfunctioning a bit was bad enough. A crazy, cold, mechanical mind was even worse.

Once back on the bridge, he settled in, waiting to see what the dark ships had done when they caught sight of Geary’s fleet coming at them from an unexpected direction.

“We should be seeing their reaction in less than a minute, Captain,” Lieutenant Yuon said.

“I don’t know why we’re so tense waiting to see,” Desjani grumbled to Geary. “We know they’re going to come around and charge toward an intercept with us.”

“The question is exactly how they’re going to do that,” he replied.

“If they’re mimicking your tactics, they’ll shift into three subformations,” she repeated.

His display lit up with alerts as the movements of the dark ships were finally seen. Nearly three and a half hours ago, the dark ships had pivoted about and begun accelerating toward an intercept with Geary’s fleet. As they did so, the dark ships had split from their one, massive arrangement, into a big central formation and two smaller formations on the wings of the first. The largest remained a rectangular box, while the two smaller were square boxes. “Oh, please,” Desjani scoffed. “They are dangling those small formations as obvious bait. Do they really think that you’ll fall for that?”

“I might have tried it,” Geary said. “Those side formations are very tempting.” The big central formation held twelve dark battleships, but each formation off to the side held only two battleships. “I want to hit them.”

“Would you, though? Would they expect Black Jack to do that?”

“Yes.” He reached toward the depiction of the dark ships on his display, using one finger to trace the formations the enemy warships were falling into as they accelerated toward Geary’s warships. “But the angle
we’re going to intercept them at makes the small formation off to our port side the preferable target. Normally, I’d go for that.”

Desjani cocked a skeptical eye his way. “Going for the small formation to our starboard would mean cutting close across the path of their big formation.”

“Right. Which makes that a bad option.” He touched the main dark ship formation. “So, we’re going for this.”

She sat up, staring at him. “I know I said we should do something stupid, but I didn’t mean something that stupid.”

“We’re coming in a little higher than them, and with them slightly to our starboard,” Geary explained. With both formations heading for the fastest intercept, their relative positions wouldn’t change as the distance between them rapidly grew less. “Their artificial Geary will tell them that I will plan on hitting their port wing formation. They’ll plan on a last-moment shift in vector that will swing them toward where we’d be.”

He gestured on his display and the depiction of the dark ship main formation swung slightly over and up. “But if we actually aim for this point, where nothing is . . .” His finger rested above and to port of the enemy path.

“We’ll be able to clip the upper edge of their main formation and concentrate our fire on the two battleships there,” Desjani said, nodding. “But if they don’t do what you expect, the firing pass will be wasted. We won’t have anything within range. Neither will they, but they can afford to waste firing passes. We can’t.”

“I think it’s our best option,” Geary said.

“It’s definitely a good option.” She bent another questioning look on him. “It’s not stupid, either. Are we reserving the stupid options for later?”

“Yeah.” He rubbed his chin, gazing at his display. “I just have a feeling that we shouldn’t go stupid right away.”

Desjani nodded at him. “Listen to that. You don’t know who or what is sending you that feeling.”

“Yes, Captain.” Geary gestured toward her this time. “What are you feeling?”

“Me?” She paused, looking away, then back at him. “I’ve got this odd sense that the other shoe hasn’t dropped yet.”

“Something else is about to happen?”

“I don’t know. It’s just that feeling.”

“Let’s hope it’s a good shoe,” Geary said.

“And that it drops on the dark ships,” Desjani agreed.

In a few more hours, they would know.

—

EVERYTHING
looked great. Geary kept scanning his display with growing irritation, wondering what was bothering him. And why did he keep thinking about Jane Geary? Something she had said . . .

But she had gotten those reports to Unity. People would be acting on them. Hopefully, the people involved in the dark ship program would be getting some tough questions, told to fix the problems, shown what the dark ships had done—“Damn.”

“What?” Tanya instantly glanced over, ready to respond to an order.

“I think—” Geary took a deep breath and started again. “Please have one of your officers run a time line for me. Start with when Captain Jane Geary delivered my reports to Unity. Factor in the time she was delayed there and the time required for her to get back to Varandal. Compare that to the days required for a courier ship to hypernet from Unity to some star system in this region of space and how long ago the dark ships appeared at Bhavan, and see how many days that leaves.”

“Why assume the dark ship base is in this region?” Desjani asked.

“Because on the hypernet, longer trips take less time.” Physicists with the right specialties claimed to understand why that was, but everyone else just chalked it up to the weirdness of the quantum mechanics on which the hypernet was based. “I want to see how it works out using a longer trip time.”

Desjani gestured with one hand toward Lieutenant Castries, who bent to the task. “How serious is this?” Desjani asked.

“It could be very serious.”

“Captain,” Castries said, “there’s about a week and a half difference. Eleven days. That other courier ship could have reached this region well before Captain Geary made it back.”

“But why—?” Desjani began, then her eyes widened. “Long enough to enter more data into the dark ships and for them to jump to Bhavan.”

“Right,” Geary said. “We’re basing our attack on the assumption that the dark ships learned nothing from what I did at Atalia because that information died with the dark ships we fought there and at Varandal. But that same information, from our own ships’ data, has surely been given to the authorities at fleet headquarters and spread through the Alliance government. If anyone at those places, anyone like those two mystery agents who messed things up on Ambaru, sent that data to the dark ships, then they know everything. They may have planned this attack based on that information.”

“And we only have five minutes left until contact.” Desjani took a deep breath. “All right, Admiral. Your opponent received a report of what you did in your last battle. Do you repeat the last successful tactic because everyone knows you don’t repeat yourself, or do you mix it up because you assume the enemy will be prepared for what you did last time?”

“I mix it up,” Geary said immediately. “I don’t assume the enemy will do what I want them to do.”