Authors: Mark Russinovich
The staff was told that the consultants were software contractors finishing the last stages of a project on-site. They were given computer accounts with the limited access permissions of basic staff unaffiliated with any particular group or project. The e-mail program that came with the accounts contained a directory of users, while the browser was programmed by default to open the Exchange’s intranet portal. That page served as a central source of company news and was a hub to which department and team sites were linked. It also served as a search facility that enabled users to find documents and sites across the network. With no more information than that, Jeff and Frank were to launch their attack.
* * *
Neither Jeff nor Frank had been surprised at being hired by the Exchange, or the nature of their project. NYSE Euronext was entirely computer and software driven. It was essential that the trading public and world financial system have faith in the Exchange’s operation, so its security needed to be as close to perfect as possible.
There had always been problems with operationalizing high security. The keys to the Exchange were information and transaction speed. During the crash of 1929, the ticker tapes that recorded trades and were the lifeblood of traders had run hours behind events. The growing lag had spread panic and, it was believed, intensified the financial disaster. Traders had speculated in the dark, acting on rumors, many of which later proved unfounded. Reforms, including faster ticker machines and new regulations concerning trades, had improved transactions and renewed traders’ faith in the Exchange but never eliminated a lingering level of unease.
NYSE Euronext traded equities, derivatives, futures, and options of nearly every sort. It listed nearly ten thousand individual items from more than sixty countries. The Exchange’s markets represented a quarter of all worldwide equities trading and provided the most liquidity of any global exchange group, meaning it was almost always possible to actually make a trade. It was rapidly working to become the only exchange any trader would ever need for every kind of financial trading transaction.
As a consequence, NYSE Euronext had embarked on the greatest expansion in its history. When the expansion was completed, nearly all the world’s trades would, at some point, pass through the Exchange’s computers. The envisioned future was breathtaking in its audacity.
Nothing so innocuous as a bit of untargeted malware was going to bring the integrity of NYSE operation into question. The implications of broad distrust in its security were simply unimaginable, not just to the Exchange, but also to the interconnected world financial system. It was a system that operated largely on faith. Break that faith, and a financial catastrophe of epic proportions loomed.
As the pair had expected, NYSE system security was first rate. But once past the initial layer of defense, Jeff discovered the same erratic patching he had seen time and again with companies that asked the public to trust them with their private information. Some of this exposure had to do with time, as a certain delay was inherent in how patching was actually performed. First the vulnerability had to be detected, which usually took place only after an exploit that took advantage of it was released. It then took the software vendor, security research firms, or in-house shops anywhere from two to four weeks to develop mitigating configurations and a corrective patch, which would then be rolled out. The actual patching itself was time consuming and many times failed to receive the immediate IT attention it deserved, resulting in another delay until a patch was finally applied to the company’s software, though too often even that failed to take place.
Part of the reason for delays and failures was simply human error and sloppiness. But there was more than just negligence involved. Every business had to make an assessment of the consequences that might arise from installing a patch. Updates were not always smooth and could create any number of unintended problems. Businesses, therefore, tended to err on the side of assuming the patch might compromise their software or interfere with something that interacted with it. In many cases, security risks were balanced against the risks to business processes, and then there was a period of reflection, during which the consequences were weighed. Sometimes after deliberation, the patch was intentionally never installed.
But whether holes were left unpatched as a result of a conscious decision or from plain ineptitude, they remained open doors for aggressors who might come later. Banks with household names too frequently had tin-box defenses within their outer walls, even though they usually adhered to industry-approved responses and followed cybersecurity best practices.
In the case at hand, an unpatched vulnerability in Payment Dynamo, a popular business application, was the missing brick in the wall that had separated Jeff and Frank from the fantastically complex internal IT network connecting the Exchange’s hundreds of servers and thousands of employee PCs.
This was the first time Jeff and Frank had worked on-site together, and it was going well so far. Persuading Frank to join him at Red Zoya after Daryl’s departure had not proved as difficult as Jeff initially feared. Though Daryl and Frank were old college friends, Jeff had known the man nearly as long. There’d been years when he had little contact with Frank, though they’d met in person to compare notes and complain from time to time when they worked with the CIA. Their work was related, often overlapping, and if colleagues didn’t go around the bureaucracy occasionally, then nothing would get done.
For a time, the two men had been on the same Company league ball team, where Frank played a competent second base. He was of average height and a bit thin. Both on and off the field, he was even-tempered and solid. He approached everything methodically.
Frank had a background in technology, with a degree in computer science, and he’d joined the CIA after college. But instead of moving into computers, which were then in their relative infancy and not a priority, he worked as a field agent for seven years, employing his computer knowledge as a cover. Frank never spoke of his assignment much, but Jeff surmised that he’d been the real McCoy, trained in tradecraft. He’d been stationed in the United Kingdom and Spain, neither of them hot spots, and as a consequence spoke excellent Spanish.
But Frank gave all that up when he decided to marry Carol, and a safer and more predictable life became a priority. Theirs was a happy marriage, and the couple had three young children. One measure of Frank and Carol’s close relationship with Daryl was that they had named their third and likely final child Daryl.
Frank had done well when assigned to Langley. He worked just two years as a cybersecurity researcher with the Company while obtaining a graduate degree before becoming a team manager and from there moved further into technical management.
At work, Frank’s personality and appearance caused him to blend in, to be forgettable, which must have been an advantage, Jeff decided, when he’d been a case officer. For all that, he had no problem pulling his own weight or standing up to other managers in the relentless internecine struggles that marked CIA bureaucracy.
It had been the ongoing struggles for ownership of cybersecurity charters among various government organizations that finally wore Frank down. Once he became eligible for a pension, he was open to Jeff’s offer. When he put in his papers, he’d been serving as the assistant director of Counter-Cyber Research.
More than once over the last eight months, Frank had mentioned to Jeff how little he missed the Company. The only part of his new job he disliked was the occasional travel assignments required of him. It might be a digital age, but some things still had to take place on-site. Direct access was especially common with highly secured companies. Though Jeff worked every day since arriving, Frank had squeezed in a weekend trip to his Maryland home.
Jeff’s decision to remain on the job had been rewarded late yesterday morning, when the pair succeeded in positioning themselves for final penetration into the NYSE Euronext core operating system.
Frank had turned to Jeff with a profound smile and said, “That was as thrilling an achievement as I’ve ever experienced with computers. No wonder you love this job so much.”
8
MITRI GROWTH CAPITAL
LINDELL BOULEVARD
ST. LOUIS, MISSOURI
10:54
A.M.
Jonathan Russo started over, trying to make sense of the incomprehensible. If his first pass was correct, the company was $16 million in the hole since the opening bell. Not only was that a great deal of money for Mitri Growth, but it also wasn’t supposed to be possible. The firm had experienced temporary, unanticipated losses previously, but never anything like this.
In 2010, the NYSE Euronext opened its new trading hub in northern New Jersey, just across the line from New York State. Located at the site were the actual computing engines that formed the heart of the Exchange. The hub had been built to increase transfer speed, as most trades were now executed by computers rather than by individuals; to give transactions a greater measure of security, both physical and digital; and to increase profits.
Though rather ordinary looking as a building, the 400,000-square-foot data center was a contemporary fortress. There was but one way into the windowless structure, and that entrance was located not at the street address but in the rear. Surrounded by a river on one side and a moat about the rest, the trading hub was invulnerable even to a car bomb.
The visible building was an illusion, an outer wrapper that served much like medieval armor. Within it lay the actual structure. And while the hub’s physical barriers were formidable, augmented by skilled armed guards and bomb-sniffing dogs, every electronic security measure possible was in place as well.
From this highly favorable location, the facility had ready access to any number of cybernetworks, along with two independent power grids. It also possessed its own backup electrical generator system. In fact, the facility had two of everything. An ever-increasing percentage of equities and options trading in North America was processed within its powerful servers. It was critical that it never fail to process them.
The facility was also designed to provide a colocation opportunity for trading firms seeking high-speed access to its engines. In an arrangement known as proximity hosting, the trader pods were each twenty thousand square feet and cost millions, not including the significant ongoing access fees. With the first pods selling out before the hub opened, construction was already under way to provide another five. These housed entire computer ecosystems used primarily by hedge funds and trading firms. The proximal location allowed clients to conduct trades in microseconds, and in this industry, being first meant everything.
The logic was simple: For every one thousand feet a hedge fund’s servers were distant from the Exchange engines, one-millionth of a second was added to a trade, the length of time it took light to travel that distance. The NYSE servers processed more than one million orders every second. Each trade required the acquisition and processing of data, then a return of the decision. The process was accomplished in microseconds, round-trip. Colocation offered traders a highly profitable advantage, which explained why the pods leased for such exorbitant sums, a significant income stream for the Exchange.
The NYSE wasn’t stopping with hub expansion. It was also feverishly constructing a series of microwave towers from Manhattan to its operation in Chicago, more than seven hundred miles distant. Microwave technology allowed the transmission of data in 4.13 milliseconds, 95 percent of the theoretical speed of light. The chain of towers would replace the existing fiber-optic cables, which transferred data at just 65 percent light speed. NASDAQ already had similar towers in place. NYSE’s structures reduced latency by three milliseconds at a cost of $300 million, and were expected to be highly profitable.
Mitri Growth had acquired a proximity pod in New Jersey, though its trading code was written at the office here in St. Louis. One of the beauties of high-frequency trading was that it could be managed from anywhere on earth.
Russo glanced up at his team. They were feverishly at work to remedy the disaster still unfolding. Did he dare pull the plug? He was reluctant to do so before he knew what was taking place. But Mitri Growth couldn’t sustain a loss like this for long. The hedge fund catered to high-end investors. In fact, much of its $250 million came from the personal portfolio of the company’s Lebanese founder.
But if Russo’s people could get this fixed before close of trading, there’d still be time to undo some or much of the loss. If the losses were real, that is. What he suspected, and what had thus far prevented him from acting, was the possibility of an aberration created by the new algo the team launched. The computers stated that Mitri Growth was losing money, but they might mistakenly be reporting a freakish reaction to the new software, not actual trades involving real money.
His chief assistant, Alexander Baker, had first proposed the possibility to Russo earlier in the day, when they discovered that the trouble came from the test code of the new program. His team was acting on the assumption that the test code had somehow activated in the production system, where it discerned the actual trades, but was reporting back to them using one of the fictitious scenarios embedded within it. The team was testing each of those in an attempt to confirm their hypothesis.
In the meanwhile, Russo’s computer continued to claim that Mitri Growth was hemorrhaging capital. He looked at the wall clock with a sinking heart. If they were wrong, if this loss was real, they were running out of time to recover.
After eight years with Jump Trading in Chicago, Russo had joined Mitri Growth the previous year and assumed supervision of its ten-person programming team. He arrived right after the founder had taken the step of acquiring a proximity hosting pod at the NYSE Euronext hub.