Homo Deus: A Brief History of Tomorrow (48 page)

Even preprogramming the system with seemingly benign goals might backfire horribly. One popular scenario imagines a corporation designing the first artificial super-intelligence, and giving it an innocent test such as calculating pi. Before anyone realises what is happening, the AI takes over the planet, eliminates the human race, launches a conquest campaign to the ends of the galaxy, and transforms the entire known universe into a giant super-computer that for billions upon billions of years calculates pi ever more accurately. After all, this is the divine mission its Creator gave it.
²¹

A Probability of 87 Per Cent

At the beginning of this chapter we identified several practical threats to liberalism. The first is that humans might become militarily and economically useless. This is just a possibility, of course, not a prophecy. Technical difficulties or political objections might slow down the algorithmic invasion of the job market. Alternatively, since much of the human mind is still uncharted territory, we don’t really know what hidden talents humans might discover, and what novel jobs they might create to replace the losses. That, however, may not be enough to save liberalism. For liberalism believes not
just in the value of human beings – it also believes in individualism. The second threat facing liberalism is that in the future, while the system might still need humans, it will not need individuals. Humans will continue to compose music, to teach physics and to invest money, but the system will understand these humans better than they understand themselves, and will make most of the important decisions for them. The system will thereby deprive individuals of their authority and freedom.

The liberal belief in individualism is founded on the three important assumptions that we discussed earlier in the book:

1.  I am an in-dividual – i.e. I have a single essence which cannot be divided into any parts or subsystems. True, this inner core is wrapped in many outer layers. But if I make the effort to peel these external crusts, I will find deep within myself a clear and single inner voice, which is my authentic self.

2.  My authentic self is completely free.

3.  It follows from the first two assumptions that I can know things about myself nobody else can discover. For only I have access to my inner space of freedom, and only I can hear the whispers of my authentic self. This is why liberalism grants the individual so much authority. I cannot trust anyone else to make choices for me, because no one else can know who I really am, how I feel and what I want. This is why the voter knows best, why the customer is always right and why beauty is in the eye of the beholder.

However, the life sciences challenge all three assumptions. According to the life sciences:

1.  Organisms are algorithms, and humans are not individuals – they are ‘dividuals’, i.e. humans are an assemblage of many different algorithms lacking a single inner voice or a single self.

2.  The algorithms constituting a human are not free. They are shaped by genes and environmental pressures, and take
decisions either deterministically or randomly – but not freely.

3.  It follows that an external algorithm could theoretically know me much better than I can ever know myself. An algorithm that monitors each of the systems that comprise my body and my brain could know exactly who I am, how I feel and what I want. Once developed, such an algorithm could replace the voter, the customer and the beholder. Then the algorithm will know best, the algorithm will always be right, and beauty will be in the calculations of the algorithm.

During the nineteenth and twentieth centuries, the belief in individualism nevertheless made good practical sense, because there were no external algorithms that could actually monitor me effectively. States and markets may have wished to do exactly that, but they lacked the necessary technology. The KGB and FBI had only a vague understanding of my biochemistry, genome and brain, and even if agents bugged every phone call I made and recorded every chance encounter on the street, they did not have the computing power to analyse all this data. Consequently, given twentieth-century technological conditions, liberals were right to argue that nobody can know me better than I know myself. Humans therefore had a very good reason to regard themselves as an autonomous system, and to follow their own inner voices rather than the commands of Big Brother.

However, twenty-first-century technology may enable external algorithms to know me far better than I know myself, and once this happens, the belief in individualism will collapse and authority will shift from individual humans to networked algorithms. People will no longer see themselves as autonomous beings running their lives according to their wishes, and instead become accustomed to seeing themselves as a collection of biochemical mechanisms that is constantly monitored and guided by a network of electronic algorithms. For this to happen, there is no need of an external algorithm that knows me
perfectly
, and that
never makes any mistakes; it is enough that an external algorithm will know me
better
than I know myself, and will make
fewer
mistakes than me. It will then make sense to trust this algorithm with more and more of my decisions and life choices.

We have already crossed this line as far as medicine is concerned. In the hospital, we are no longer individuals. Who do you think will make the most momentous decisions about your body and your health during your lifetime? It is highly likely that many of these decisions will be taken by computer algorithms such as IBM’s Watson. And this is not necessarily bad news. Diabetics already carry sensors that automatically check their sugar level several times a day, alerting them whenever it crosses a dangerous threshold. In 2014 researchers at Yale University announced the first successful trial of an ‘artificial pancreas’ controlled by an iPhone. Fifty-two diabetics took part in the experiment. Each patient had a tiny sensor and a tiny pump implanted in his or her stomach. The pump was connected to small tubes of insulin and glucagon, two hormones that together regulate sugar levels in the blood. The sensor constantly measured the sugar level, transmitting the data to an iPhone. The iPhone hosted an application that analysed the information, and whenever necessary gave orders to the pump, which injected measured amounts of either insulin or glucagon – without any need of human intervention.
²²

Many other people who suffer from no serious illnesses have begun to use wearable sensors and computers to monitor their health and activities. The devices – incorporated into anything from smartphones and wristwatches to armbands and underwear – record diverse biometric data such as blood pressure. The data is then fed into sophisticated computer programs, which advise you how to change your diet and daily routines so as to enjoy improved health and a longer and more productive life.
²³
Google, together with the drug giant Novartis, are developing a contact lens that checks glucose levels in the blood every few seconds, by testing tear contents.
²⁴
Pixie Scientific sells ‘smart diapers’ that analyse baby poop for clues about the baby’s medical condition. Microsoft has
launched the Microsoft Band in November 2014 – a smart armband that monitors among other things your heartbeat, the quality of your sleep and the number of steps you take each day. An application called Deadline goes a step further, telling you how many years of life you have left, given your current habits.

Some people use these apps without thinking too deeply about it, but for others this is already an ideology, if not a religion. The Quantified Self movement argues that the self is nothing but mathematical patterns. These patterns are so complex that the human mind has no chance of understanding them. So if you wish to obey the old adage and know thyself, you should not waste your time on philosophy, meditation or psychoanalysis, but rather you should systematically collect biometric data and allow algorithms to analyse them for you and tell you who you are and what you should do. The movement’s motto is ‘Self-knowledge through numbers’.
²⁵

In 2000 the Israeli singer Shlomi Shavan conquered the local playlists with his hit song ‘Arik’. It’s about a guy who is obsessed with his girlfriend’s ex, Arik. He demands to know who is better in bed – him, or Arik? The girlfriend dodges the question, saying that it was different with each of them. The guy is not satisfied and demands: ‘Talk numbers, lady.’ Well, precisely for such guys, a company called Bedpost sells biometric armbands you can wear while having sex. The armband collects data such as heart rate, sweat level, duration of sexual intercourse, duration of orgasm and the number of calories you burnt. The data is fed into a computer that analyses the information and ranks your performance with precise numbers. No more fake orgasms and ‘How was it for you?’
²⁶

People who experience themselves through the unrelenting mediation of such devices may begin to see themselves as a collection of biochemical systems more than as individuals, and their decisions will increasingly reflect the conflicting demands of the various systems.
²⁷
Suppose you have two free hours a week, and you are unsure whether to use them in order to play chess or tennis. A good friend may ask: ‘What does your heart tell you?’ ‘Well,’
you answer, ‘as far as my heart is concerned, it’s obvious tennis is better. It’s also better for my cholesterol level and blood pressure. But my fMRI scans indicate I should strengthen my left pre-frontal cortex. In my family, dementia is quite common, and my uncle had it at a very early age. The latest studies indicate that a weekly game of chess can help delay the onset of dementia.’

You can already find much more extreme examples of external mediation in the geriatric wards of hospitals. Humanism fantasises about old age as a period of wisdom and awareness. The ideal elder may suffer from bodily ailments and weaknesses, but his mind is quick and sharp, and he has eighty years of insights to dispense. He knows exactly what’s what, and always has good advice for the grandchildren and other visitors. Twenty-first-century octogenarians don’t always look like that. Thanks to our growing understanding of human biology, medicine keeps us alive long enough for our minds and our ‘authentic selves’ to disintegrate and dissolve. All too often, what’s left is a collection of dysfunctional biological systems kept going by a collection of monitors, computers and pumps.

At a deeper level, as genetic technologies are integrated into daily life, and as people develop increasingly intimate relations with their DNA, the single self might blur even further, and the authentic inner voice might dissolve into a noisy crowd of genes. When I am faced by difficult dilemmas and decisions, I may stop searching for my inner voice, and instead consult my inner genetic parliament.

On 14 May 2013 actress Angelina Jolie published an article in the
New York Times
about her decision to have a double mastectomy. Jolie lived for years under the shadow of breast cancer, as both her mother and grandmother died of it at a relatively early age. Jolie herself did a genetic test that proved she was carrying a dangerous mutation of the BRCA1 gene. According to recent statistical surveys, women carrying this mutation have an 87 per cent probability of developing breast cancer. Even though at the time Jolie did not have cancer, she decided to pre-empt the dreaded disease and have
a double mastectomy. In the article Jolie explained that ‘I choose not to keep my story private because there are many women who do not know that they might be living under the shadow of cancer. It is my hope that they, too, will be able to get gene-tested, and that if they have a high risk they, too, will know that they have strong options.’
²⁸

Deciding whether to undergo a mastectomy is a difficult and potentially fatal choice. Beyond the discomforts, dangers and financial costs of the operation and its follow-up treatments, the decision can have far-reaching effects on one’s health, body image, emotional well-being and relationships. Jolie’s choice, and the courage she showed in going public with it, caused a great stir and won her international acclaim and admiration. In particular, many hoped that the publicity would increase awareness of genetic medicine and its potential benefits.

From a historical perspective, it is interesting to note the critical role algorithms played in this case. When Jolie had to take such an important decision about her life, she did not climb a mountaintop overlooking the ocean, watch the sun set into the waves and attempt to connect to her innermost feelings. Instead, she preferred to listen to her genes, whose voice manifested not in feelings but in numbers. Jolie felt no pain or discomfort whatsoever. Her feelings told her: ‘Relax, everything is perfectly fine.’ But the computer algorithms used by her doctors told a different story: ‘You don’t feel anything is wrong, but there is a time bomb ticking in your DNA. Do something about it – now!’

Of course, Jolie’s emotions and unique personality played a key part too. If another woman with a different personality had discovered she was carrying the same genetic mutation, she might well have decided not to undergo a mastectomy. However – and here we enter the twilight zone – what if that other woman had discovered she carried not only the dangerous BRCA1 mutation, but another mutation in the (fictional) gene ABCD3, which impairs a brain area responsible for evaluating probabilities, thereby causing people to underestimate dangers? What if a statistician pointed
out to this woman that her mother, grandmother and several other relatives all died young because they underestimated various health risks and failed to take precautionary measures?

In all likelihood, you too will make important decisions about your health in the same way as Angelina Jolie. You will do a genetic test, a blood test or an fMRI; an algorithm will analyse your results on the basis of enormous statistical databases; and you will then accept the algorithm’s recommendation. This is not an apocalyptic scenario. The algorithms won’t revolt and enslave us. Rather, the algorithms will be so good in making decisions for us that it would be madness not to follow their advice.