Technological Nightmares (Lecture): Six Concerns
| Topics: |
Series: Pardee Center Distinguished Lecture Series
Date: October 2003
Location: Frederick S. Pardee Center for the Study of the Longer-Range Future, Boston University, Boston, MA
Contents
Six Concerns
After this somewhat lengthy introduction let me now turn to my six worries or concerns.
The need for a cultural revolution
My first concern is about the impact of information technology on employment and income distribution. Skills and aptitudes are much stressed in the economics literature, but attitudes tend to be neglected. This is probably so because they are not easily measured. Compare eight ditch diggers with a computer operator in a bank. The former can be supervised, so that if anyone slacks, the supervisor admonishes him. Not so with the computer operator. He has to be responsible. If there were a supervisor, he might just as well do the work himself. A different culture is needed.
Advanced technology often means that a smaller number of skilled people supply their services over a wider range, producing a “winnertakeall” effect, where only the best do well, and these lucky few command enormous incomes. We witness this in sports, business, and entertainment. The invention of the phonograph did this for singers, and the invention of the motion picture did it for actors. Proliferating communications and information technology may do the same for many other occupations.
What is the impact on employment and unemployment? Many predict growing unemployment of low-skilled workers. But education, leisure activities, and caring for the needy provide new job opportunities. Some of these can perhaps be replaced by electronic teaching, monitoring, etc. But if attitudes and commitment are more important than skills, a cultural change is needed.
Are those with low skills no longer wanted? It is not at all clear that our society cannot use plenty of health workers, nurses, child care workers, special-education teachers, home health-care aides, manicurists, gardeners, plumbers, sweepers, protectors and restorers of the environment, valet parking attendants, janitors, cleaners, waiters, salesmen, physical training instructors, musicians, designers, and other service workers who do not need the high and scarce skills demanded by modern technology and whose services cannot be replaced by either computers or imported low-cost goods from low-income countries (though imported low-cost labor from poor countries should be welcomed). In fact, it is precisely for these jobs which cannot be replaced by computers that the demand is likely to increase in the future. The Bureau of Labor Statistics considers many of these jobs as likely to be the fastest growing over the next decade. Many of these jobs are, however, in the currently despised and neglected public sector and may call for even more despised higher taxation. They are also often ill-paid and not recognized as valuable. We need to change our valuation of such work and should guarantee minimum standards of reward for them.
Kurt Vonnegut, in his novel Player Piano, describes a future nightmare society (a dystopia or, as Jeremy Bentham called it, a cacotopia) in which “the divine right of machines,” efficiency and organization, has triumphed, and the large underclass of unemployed are handed out, by a small group of affluent managers, plenty of goodies, but lack what John Rawls regards as “perhaps the most important primary good,” which is self-respect. Unemployment means loss of dignity, which is at least as important as income. Vonnegut’s unemployed eventually revolt.
What deprived the underclass of self-respect was the fact that it was an “equality of opportunity” society. IQ tests decided who became a manager, and it took a manager to lead the revolt. Were you to allocate jobs by a lottery or the accident of birth, there would be no problem with self-respect.
The concern in the advanced countries has become jobless growth or, more recently, jobless recovery. In fact, economic growth, whether measured in terms of overall productivity or productivity in manufacturing, has been considerably slower since 1981 than in the 1960s, when growth was not accompanied by unemployment. Since the growth of productivity has been less than the growth of demand, one would have expected jobs to be created rather than destroyed.
Is it luck in the genetic lottery (nature) or family and other environmental factors (nurture) that determines our success or failure in life? Is inequality of income, wealth, and dignity due to genes? Ronald Dore says we should reward effort, not the genetically “bright and beautiful.”
Some comfort for those concerned about growing inequality can be found in the words of Dr. Tina Cary. She writes: “For those of us who are not Luddites, another question arises: will changes in information technology necessarily widen the gap in wages between the haves and have-nots? Given that some computer programs available today make medical diagnoses identical to those of qualified physicians, we may find that computers can also lower wages for skilled work, not only for unskilled work. If computers take on such work as accounting and law, the result might be a lessening in the gap in wages! Another aspect of this is that information technology allows jobs to move more freely, so that the programmer or data entry person can live anywhere.”[1]
The electronic nightmare
We now have at our disposal cheap, super-efficient means of surveillance. Electronically stored information is more difficult to keep under lock and key than paper that is stored. Hence the means for more totalitarian control are at hand. There has been a dispute between the FBI and civil liberties groups and it has been proposed to widen the scope of the National Crime Information Center. It could contain inaccurate or subjective data, raise security problems, and lead to false leads. It violates the right to privacy in the interest of greater security.
Information is power. Tiny microphones are now capable of recording whispered conversations from across the street. Conversations can even be monitored from the normally imperceptible vibrations of window glass. Cameras the size of large wasps can be flown into a room and record everything. Fortunately, the same technology that is destroying privacy also makes it easier to discover terrorists, trap stalkers, detect fraud, prosecute criminals, and hold the government to account.
There are bound to be trade-offs between privacy on the one hand and security, efficiency, convenience, and liberty on the other. “Each benefit—more security against terrorists or criminals, better government services, higher productivity at work, better medical care, a wider selection of products, more convenience, more entertainment—will seem worth the surrender of a bit more personal information, or a marginal increase in monitoring. Yet the cumulative effect of these bargains, each seemingly attractive on its own, will be the relentless destruction of privacy."[2] Both the government and the private sector are hungry for more information.
David Brin, the American physicist and science fiction writer, proposes full transparency and the complete abolition of privacy. In his 1998 book The Transparent Society he says, “Light is going to shine into nearly every corner of our lives.” Attempts to protect privacy usually benefit only the rich and powerful or the government. Let everyone have access to databases, peer through CCTV (closed circuit television) cameras, and listen in on conversations. “Mutually assured surveillance,” Brin argues, would see to it that most people would not abuse their access to information. He argues for complete openness, for “reciprocal transparency.” If police cameras watch us, we should be able to watch them. The biggest threat to freedom is that surveillance technology will be used by too few, not by too many. Citizens should have the power to watch the watchers.
His position is supported by economists who attribute many inefficiencies to asymmetric information flows. If the flow could be symmetric, a general improvement in which everyone gains would follow. Examples of the inefficiencies that arise from asymmetric information are wars that are caused by one side guessing wrong about the other’s power or determination; going to trial; the collapse of mutually beneficial negotiations when one party is afraid that someone else knows something it does not, and if they agree to a proposal it must be because it favors them in some way we have not realized.
But most people are unwilling to open up completely. In the film The Truman Show the hero abandons the only life he knows to evade the pitiless gaze of the cameras, having discovered that he has been the subject of a reality-TV show since birth.
More recently, however, as a result of the spread of mobile telephones, digital cameras, and the Internet, surveillance technology has become widely available. Bruce Schneider, a security expert, has written that “surveillance abilities that used to be limited to governments are now, or soon will be, in the hands of everyone.”[3] There are, however, drawbacks to the wide spread of surveillance. It has led to the invasion of hidden cameras in bedrooms, showers, lavatories, and locker rooms. Industrial espionage is another danger. Surveillance technology can also be used for identity theft. As The Economist concludes its discussion, “Increasingly, it is not just Big Brother who is watching—but lots of little brothers, too.”[4]
The Frankenstein nightmare
Mary Shelley’s Frankenstein (1818) is perhaps the most influential piece of science fiction ever written. People then thought that God created life. Shelley’s doctor apes the action of divine creation and fails. The book’s subtitle is “The Modern Prometheus.” Prometheus stole fire from heaven and gave it to mankind. He was punished because he gave mankind power and with it choice. Biotechnology will have the power to manipulate human life. [5]
Should biotechnologists be allowed to mess around with the genetic structure of human beings? Should pharmacologists be free to change the psychological make-up? There can be no objection if it helps in eliminating genetic diseases. But how are human rights and human integrity affected by the ability to shape the structure of humans? The dangers from nuclear energy are obvious. They call for regulation. But not so those from biotechnology. The promise of a longer life, of freedom from depression and a guarantee of happiness (Prozac is like soma in Aldous Huxley’s Brave New World) and social control (Ritalin is used for hyperactive children) are temptations. But the costs are less obvious. Are these psychotropic drugs forerunners of pharmacologically altered states of mind?
We are nearer to pharmacological changes than to genetic ones. In 1944 Oswald Avery, Colin, MacLeod and Maclyn McCarthy of Rockefeller University determined that DNA (deoxyribose nucleic acid) carried the hereditary blueprint. In 1953 Crick and Watson discovered the double helix. In the 1960s tadpoles were cloned in Britain. In 1970 scientists developed proteins that cut DNA in precise locations. In 1973 Stanley Cohen and Herbert Boyer of Stanford University and the University of California at San Francisco snipped a piece of genetic code out of one bacterium and inserted it into another. The result was a rust-colored pig in Beltsville, Maryland, with the genes of a cow.
In 1981 mice were the first transgenic animals. In 1986 it was decided that not only microbes but also higher life forms can be patented. In 1988 Harvard patented a mouse. In 1997 Ian Wilmut cloned a sheep called Dolly. (Dolly died in March 2003.) In 2000 a pig was cloned with organ harvesting as the goal. In 2002 Clonaid, a company founded by the Raëlian religious movement, claimed the first cloned baby, Eve. The cult is named after its founder and spiritual leader, Raël, a French journalist formerly known as Claude Vorilhon. Dr. Brigitte Bosselier is the company’s chief executive. In 2003 a mule was cloned in Idaho. In February 2004 South Korean scientists created human embryos through cloning and extracted embryonic stem cells that hold great promise for medical research. The work was led by Dr. Woo Suk Hwang and Dr. Shin Yong Moon of Seoul National University. The purpose, according to the scientists, is not to clone human beings but to advance understanding of the causes and treatment of disease. In 2004 Dr. Gerald Schatten of the University of Pittsburgh School of Medicine said he will be able to clone a monkey very soon.
In cloning, or somatic cell nuclear transfer (SCNT) as cloning is called, a cell from an adult animal is placed inside an egg that has had its own genetic material removed. The DNA of the donor cell can then take control, eventually forming an embryo. Fears are expressed that it would lead to birth defects or alternatively to an era of “designer babies.” Some believe that children should not be designed in advance, that newborns should be truly new, without the burden of genetic identity already lived; that a society in which cloning is easy (requiring few cells from anywhere in the body) means anyone could be cloned without knowledge or consent; and that replacing lost ones with “copies” is an insult to the ones lost, since it denies the uniqueness and sacredness of their existence.[6] Cloning carries high risks of bodily harm to the cloned child. It threatens the dignity of human procreation, giving one generation unprecedented genetic control over the next. It is the first step toward a eugenic world in which children become objects of manipulation and products of will.
The opposition to cloning relies, however, on the ambiguous distinction between natural and unnatural. Is cloning different from the introduction of antibodies, vaccination, efficient agriculture, or the abolition of slavery?
What if cloners present a healthy child? The Star Wars films featured an army of clones derived from the genes of an aggressive bounty hunter, modified to ensure willingness to follow military orders. The image of a horde of unthinking, cloned attackers is a classic science-fiction nightmare. But producing such an army with today’s techniques would require a huge number of women to supply the eggs and bear the fetal clones to term, a problem that is often glossed over in horror stories.
Cloning, practiced widely, might eliminate the need for men (women could bear children asexually), or it might lead in cultures that revere males to an excess of men; and it might reduce the genetic diversity that comes from mingling genes in sexual reproduction.
Critics fear that cloning could usher in a new eugenics. The Council of Europe and the United Nations have declared human reproductive cloning a violation of human rights. The President’s Council on Bioethics (in 2002) worried that cloning to produce children could disrupt the normal relationship between generations and within families, could turn children into manufactured products rather than independent beings and could put undue pressure on a cloned child living in the shadow of a genetically identical adult.[7] But like the war on drugs, bans on supply will not cut off the demand.
Problems arise from the vehicles to get foreign genes into the body. The most popular vehicle is the retrovirus that can stitch genes into human DNA. Scientists “disarm” the retrovirus but something could still go wrong. The possibility exists of a pandemic holocaust in which the disarmed viruses could turn into active ones and damage all humankind. Pharmaceutical regulation is driven by horror stories like the sulfanilamide elixir and thalidomide. The regulation of cloning may have to await the birth of a horribly deformed baby.
Many embrace the power to clone under the banner of human freedom. Freedom of parents to choose the kind of children they have, freedom of scientists to pursue research, and freedom of entrepreneurs to make use of technology to create wealth. Francis Fukuyama in his book Our Posthuman Future argues that this posthuman world could be one that is far more hierarchical and competitive than the current one and full of social conflict. Human genes will have been mixed with those of many other species and our shared humanity would be lost. People would live to be 200 years old. Or alternatively the world could be one of soft tyranny like that envisaged in Huxley’s Brave New World.
“‘Geneism’ could eclipse racism as the most destructive force on the planet,” says George Annas, professor of health law at Boston University. Once some of us are enhanced genetically, “won’t we see other people as subhuman, and enslave or slaughter them?”[8] Only the rich can afford genetic modification. Humanity would be spilt into hereditary castes. The success of the successful would seem “deserved.”
McKibben regards the genetically engineered genius as a robot. It is an uncomfortable thought but one may ask: why prefer the unintentionally mixed genes to planned ones? We believe that a person who has been programmed to make certain choices is not truly free. But our own haphazard genetic endowment, upbringing, and education determine us in exactly the same way. Why are planned reactions worse than those caused by chance?
Will the posthuman world be free, equal, prosperous, caring, compassionate, with better health care, longer lives, and more intelligence than today’s? Or will it be more hierarchical and competitive, full of social conflict? Or will it resemble Huxley’s Brave New World, a soft tyranny in which everyone is healthy and happy but has forgotten the meaning of hope, fear, or struggle.[9]
There is the possibility that biotechnology will permit the emergence of new genetically elite classes. But the opposite is also possible. In democratic societies people will not tolerate the stratification. It may lead to fights. The demand will be for raising the bottom genetically. It would be the opposite of the old eugenics that prevented the subnormal from having children. Here the state would enhance the abilities of their children. We can envisage a state of affairs with “genetic alteration or gene splicing, whereby parents who are five feet tall and bald can give birth to a six-footer with long blond hair.”[10] Biogenetic interventions will blur the borderline between the made and the spontaneous and thus affect the way we understand ourselves. “For an adolescent to learn that his ‘spontaneous’ (say, aggressive or peaceful) disposition is the result of a deliberate external intervention into his genetic code will undermine the heart of his identity, putting paid to the notion that we develop our moral being through Bildung, the painful struggle to educate our natural dispositions. Such interventions will give rise to asymmetrical relations between those who are ‘spontaneously’ human and those whose characters have been manipulated: some individuals will be the privileged ‘creators’ of others.”[11] It is possible to rear genetically improved children who will show horrible effects only after 20 years.
C. S. Lewis in The Abolition of Man was considering what “Man’s power over Nature” must always and essentially be: “Each new power won by man is a power over man as well.” “The final stage is come when Man by eugenics, by pre-natal conditioning, and by an education and propaganda based on a perfect applied psychology, has obtained full control over himself…The battle will indeed be won. But who, precisely, will have won it? For the power of Man to make himself what he pleases means…the power of some men to make other men what they please."[12]
Stem cell research to treat diseases like Parkinson’s, Alzheimer’s, and diabetes is generally approved. Excess embryos from in vitro fertilization are used for this research. Some would argue that the creation of embryos by cloning is similar.
Perhaps more worrisome than genetics is neuroscience and operating on the brain. In an attempt to treat depression, neuroscientists once carried out a simple experiment. Using electrodes, they stimulated the brains of women in ways that caused pleasurable feelings. The subjects came to no harm—indeed their symptoms seemed to evaporate, at least temporarily—but they quickly fell in love with their experimenters. This may be a greater threat to human dignity and human autonomy than cloning.[13]
These methods are also capable of “enhancing” human beings. Some worry that society will be turned into a homogeneous mass. Others are worried by the opposite: that society will be divided into the privileged and the unenhanced, reminiscent of Huxley’s alphas, betas, gammas, and epsilons. Drugs to combat shyness, forgetfulness, sleepiness, and stress are now close to clinical trials.
How is free will affected by these possibilities? The same question arises that was raised earlier: what is the difference between random, unintentional conditioning and intentional conditioning? We tend to condemn the second, but praise the first. Hard work and natural talent are considered “part of me,” while using a drug is “artificial” enhancement because it is a form of external manipulation.[14]
Social control can be used not only by the state but also by parents, teachers, school systems and others with vested interests in how people behave. The fear has been expressed that biotechnology will cause us to lose our humanity. The picture has been painted of brainless hominoids whose organs will be harvested as spare parts. Using stem cells to grow a replacement organ is already an acceptable alternative to transplant surgery.
In May 2002 scientists at New York University attached a computer chip directly to a rat’s brain, making it possible to steer the rat by means of a mechanism similar to that in a remote-controlled toy car.
In 2004 the electronics giant Philips planned to market a phone-cum- CD-player woven into the material of a jacket. The Philips jacket represents a quasi-organic prosthesis, less an external apparatus with which we interact than part of our self-experience as a living organism.
There are some who predict the extension of life without maintaining the quality of life. There will be more and more dependents at a growing cost to society. Aubrey de Grey of Cambridge University predicts life expectancy in 2100 will be five thousand years. None of us will be around to check whether he is right.[15] Odysseus was offered immortality by the goddess Calypso, but turned her down to grow old and die with his wife Penelope.
Ability to predict the sex of a fetus and abort girls has led to unbalanced sex ratios and a surplus of men in societies such as India. Amniocentesis, cheap sonograms, and easy access to abortion have led to unbalanced sex ratios in parts of Asia. Amartya Sen estimated that 100,000 women are missing in Asia. An excess of men leads to crime and aggression.
The threat of atomic apocalypse has been replaced by the fear of an environmental catastrophe, unstoppable human-engineered viruses, rampaging mutant genes, and neurologically altered posthuman beings.
The terrorist’s atom bomb in the suitcase
Technical developments have greatly amplified the damage terrorists can do. Nuclear power plants can be sabotaged; there is the danger of terrorists’ attack and of the proliferation of nuclear weapon capability. The prophets of the benefits of nuclear power (Nuclear power reactor) overlooked the problems of the disposal of nuclear waste.
In order to prevent these hazards, is our freedom threatened? When I served on the British Royal Commission on the Environment in the early 1970s, we were worried that in order to prevent attacks on nuclear power stations, our freedom and civil rights and liberties might be threatened. We are seeing this now. We must ask, how much should a democratic society risk in order to preserve the freedom of unharassed dissent? I think it was Walter Laqueur in his book The New Terrorism who predicted some time ago that Washington’s global interventionism has magnified the chances that a US city will become the target of mega- terrorism. The pervading fear of an attack, even without its taking place, multiplies greatly the damage done by an actual attack.
The threat comes not from nations but from sub-national groups. Technical developments amplify the damage terrorists can do. To make an atom bomb, a terrorist or would-be proliferator would need to get hold of only 5 kg of weapon-grade plutonium or 15 kg of weapon-grade uranium, less than you would need to fill a fruit bowl. At present the world probably contains about 250 metric tons of this sort of plutonium and 1,500 metric tons of uranium. To lose one bomb’s worth from this stock is the equivalent of losing a single word from one of three copies of The Economist.[16] General Alexander Lebed said that the Soviet Union, before its collapse, had produced suitcase-sized nuclear weapons, easily transported anywhere. “Loose nukes” can easily find their way into the hands of Al Qaeda or any other terrorist group.
A nuclear explosion at the World Trade Center, involving two grapefruit-sized lumps of enriched uranium, would have devastated three square miles of southern Manhattan, including the whole of Wall Street. It would kill hundreds of thousands if it went off during working hours. Martin Rees, Britain’s Astronomer Royal, writes that the collapse of the Soviet Union has left the world awash in enough raw materials— enriched uranium and plutonium—for some 70,000 bombs.[17]
If the terrorists (or freedom fighters, as they consider themselves) are not afraid to die, indeed court death, for their cause and the greater glory of Allah, what use is the threat of counterattacks? It will just mean that more of them are martyred and go to Allah’s heaven where 30 black-eyed virgins are waiting for each of them. We shall have to remove the cause of their hostility if we want lasting peace. But this is a very slow process if we don’t want to give up our own principles.
Susan Sontag wrote concerning the 9/11 perpetrators, “In the matter of courage (a morally neutral virtue): whatever may be said of the perpetrators of Tuesday’s slaughter, they were not cowards.” There has been a lot of public criticism of her. She has been called “morally obtuse” and a prime example of the hate-American crowd. These criticisms are unjustified. I agree with her, except that there are no morally neutral virtues. Courage is an instrumental virtue and is therefore judged partly by the ends it serves. A virtue is by definition good, not neutral, and courage is in itself good, though, if combined with other qualities and if serving evil ends, it can be bad or lead to deplorable results.
Chemical and biological weapons such as anthrax or smallpox threaten millions, not hundreds or thousands. Knowledge about them and the techniques of their use are dispersed among hospital laboratories, agricultural research institutes, and peaceful factories everywhere.
Cyber-terror and cyber-error
There was at first considerable suspicion that the largest blackout in US history on August 14–15, 2003, was caused by terrorists or sabotage. It would have been very easy. Cell phones, municipal water systems, and the Internet operate on similar principles.
Imagine terrorists breaking into computers that control the water supply of a large American city, open and close valves to contaminate the water with untreated sewage or toxic chemicals, and then release it in a devastating flood. As the emergency services struggle to respond, the terrorists strike again, shutting down the telephone network and electrical power grid with just a few mouse clicks. Businesses are paralyzed, hospitals are overwhelmed, and roads are gridlocked as people try to flee.
Lamar Smith, a Texas congressman, told a judiciary committee in February 2002: “A mouse can be just as dangerous as a bullet or a bomb.” But control systems are usually kept entirely separate from other systems. A simulation carried out in August 2002 by the United States Naval War College in conjunction with Garter, a consultancy firm, concluded that an “electronic Pearl Harbor” attack on America’s critical infrastructure could indeed cause serious disruption, but would first need five years of preparation and $200 million of funding.[18]
Our “final” experiment?
Physicists attempt to accelerate atoms close to the speed of light and then crash them together. Some physicists raise the possibility that these experiments could start a chain reaction that might destroy the Earth or even, by tearing the fabric of space, the universe. Imagine a very tiny risk of an utterly calamitous outcome—a low-probability, high-cost catastrophe. An asteroid colliding with the earth could cause the extinction of mankind. Although such events deserve the same attention as higher probability, lower-cost disasters, they do not get it. The reasons are partly that politicians have short time horizons and partly that any one country hopes to take a free ride on others taking action in disasters that affect several countries.
Society has become more dependent on big, complicated entities like the grid, the Web, and the air-traffic control system. Professor Charles Perrow, author of the 1984 book Normal Accidents: Living with High-Risk Technologies, predicts that the next accident will be in air-traffic control. The trend has been to hire managers who have training in soft skills like human relations but who lack technical understanding that is vital in running complex systems. These are regarded as dull chores.
Three different disaster scenarios from collision with a high concentration of energy can be envisaged.
- A black hole may be created that sucks in everything around it.
- The quarks (each proton and neutron consists of three quarks) reassemble themselves into a compressed object called a strangelet. By contagion, it transforms everything it encounters into a strange new form of matter. (Kurt Vonnegut’s novel Cat’s Cradle envisaged such a form in “icenine.”) A strangelet disaster could transform the entire planet Earth into an inert, hyperdense sphere about one hundred meters across.
- A catastrophe occurs that engulfs space itself. When particles crash together, this could trigger a “phase transition” that would rip the fabric of space itself, affecting the entire galaxy and beyond.
Arthur Koestler wrote in his 1949 Insight and Outlook that man had now acquired the means to destroy the planet. Evolution had granted him a technological capacity far in excess of his spiritual capabilities. “Thus within the foreseeable future, man will either destroy himself or take off for the stars.”
