Charles Stross (right) speaking about space settlement, artificial intelligence, and other topics at a recent New America Foundation event in Washington, DC. (credit: D. Day)

The rapture of the wonks

by Dwayne Day
Tuesday, May 31, 2016

There was a time when I read lots of science fiction: Heinlein, Asimov, Clarke, Pournelle, Niven, and Gibson. But today, like many DC policy wonks, I no longer read fiction. Instead, my nose is usually stuck in a journal (Space Policy, Astropolitics) or magazine (Cosmo) or policy paper. However, apparently there are DC wonks who do still read science fiction and actually find it illuminating for their wonkery. Recently, the left-leaning New America Foundation hosted an event only a block and a half from the White House called “What Can DC Learn From Sci-Fi?” featuring author Charles Stross. I’ll confess that I’ve never read Stross’ novels before, although he appears to have a devoted following. My first exposure to Stross was when I read a devastating blog article he wrote in 2010 that excoriated libertarian space enthusiasts for what he considers to be misguided and simplistic notions about space settlement.

During his DC talk last week, Stross touched on space exploration issues, but spent far more time discussing other subjects such as the surveillance state, privacy, artificial intelligence, and “the Singularity”—also referred to as “the Rapture of the Nerds”—and I was struck by the parallels between spaceflight settlement enthusiasm and excitement about the Singularity. Both have strongly utopian, pseudo-religious foundations, and both may be focusing more on things that they wish would happen and ignoring things that are more likely to actually happen.

Stross is the author of two dozen books in several different series, with more scheduled for publication this year and next. During his talk he noted that the fiction publishing industry is not as fast as some people might think, and it can take several years for a book to be approved, written, and published. He explained this to caution that sometimes reality can catch up on an author and he or she can be less prescient at publication than they were when the thought first entered their head. An example was a series he started set in Scotland that was now moot after the Scots rejected leaving the United Kingdom (although Scottish independence may be reemerging as an issue, so Stross might still have a venue).

Stross was interviewed by Kevin Bankston, Director of New America’s Open Technology Institute; and Jay Stanley, a Senior Policy Analyst with the Speech, Privacy, and Technology Project of the American Civil Liberties Union. Bankston started the interview by asking Stross about spaceflight, a popular topic in science fiction. Stross recounted how the Soviet space program owes its underpinnings to an early movement called Cosmism, a religious belief that humanity’s future belongs among the stars. He also noted the frontier imagery in much early and even current American spaceflight rhetoric. When you consider the “Soviet religious imperative combined with the American cultural manifest destiny imperative… you get a very heady brew,” Stross joked. Perhaps we need to be wary of it.

Stross is quite skeptical of space settlement. He noted that space is a very hostile environment. There is nothing that really compares to it on Earth, except perhaps the Gobi Desert. “Why don’t we try colonizing the Gobi Desert first?” Stross quipped. This led to a discussion of terraforming, another popular topic in science fiction. Stross said that the way to colonize space is not to terraform, but to change humanity “to make them a little more vacuum-tolerant.” Ultimately, this means turning humans into robots. However, that is not a vision of humanity’s future in space that actually attracts people, who romanticize living on Mars but not having their skin replaced with aluminum.

Unfortunately, Stross did not get into one of the themes that he discussed in his 2010 blog article. In the article he wrote that many of the loudest space settlement advocates argue that settlement will allow a rebirth of freedom and individuality and an escape from oppressive regulation on Earth. (This is a theme you can find in any talk by Bob Zubrin, or on numerous blogs.) The reality, Stross wrote, is likely to be the opposite because settlement survival would actually require stringent rules and regulation so that rugged individualists don’t get everybody killed.

Although spaceflight is a common topic for science fiction, both the interviewers and Stross expressed the view that science fiction is much more relevant to current events on other subjects, such as artificial intelligence and information technology, particularly as it pertains to surveillance. Bankston raised the issue of “the Singularity,” which prompted Stross to provide a bit of background on the subject. Stross explained that the idea originated in the early 1990s with author Vernor Vinge, who posited that at some point scientists would create superhuman artificial intelligence and “the human era will be ended.” It is impossible to predict what the world would be like beyond that point. But this concept, Stross said, “got contaminated with a bunch of other stuff” from other authors like Ray Kurzweil and Hans Moravec, whom Stross thinks made some “dubious assumptions” about how artificial intelligence is actually going to develop and how it may affect us. This is more than simply a discussion of computer science, Stross explained, because for some people the concept of the Singularity has now become a kind of “secular religion: first we get AI, then the AI turns us into AIs, and we all get ascended to heaven.”

Stross said that he is actually more worried about the small things, “less that AI will enslave us, but will be used to take away our agency.” Stross mused that we might already have an entity that mimics what an artificial intelligence would do in the form of joint stock corporations. Humans are interchangeable within them and they don’t seem to serve the needs of humanity. They can be incredibly anti-human at times, and trying to understand their motivations is often an impossible task.

Later in response to questions Stross noted that we are already seeing indications of advanced information technology tools replacing humans in areas we never would have thought possible. It’s not just the robot replacing people on the assembly line, it’s rudimentary artificial intelligence replacing skilled “thought” labor. For instance, it used to be that many persons who had just graduated law school went to work doing basic research for law firms, particularly in the discovery process in lawsuits. But much of that work is increasingly automated and thus many new law school graduates cannot find employment because a computer has rendered them obsolete. Stross added that recently he has learned that self-driving trucks could lead to mass unemployment in the trucking industry, which currently employs millions of drivers. We don’t need to worry about Skynet building a race of Terminators, but instead a computer denying us a job. (Space policy wonks take note: Skynet is coming for us.)

Stross is sharp and a fast-talker and knowledgeable about a broad range of subjects, and at times it was difficult to keep up with him as he careened from one complex topic to another. But I was struck by the parallels between the enthusiast/activist/thinking communities in both spaceflight and artificial intelligence. In both areas there seem to have developed utopian themes, a belief that a particular development is going to make things better for humanity, is thus inherently good, and therefore must be pursued. For example, many people believe that settling space is an important, perhaps even vital goal for humanity, and that a “multi-planet species” is better than a single-planet one. Similarly, there are people who believe that the dawn of artificial intelligence is going to be a net plus for humanity as it will enable rapid progress in various fields, such as medicine. Put a thinking computer to work and it will quickly cure cancer, for instance.

But what Stross highlighted is that these beliefs may be missing more obvious and likely outcomes than the end goals that their advocates have fixated upon. Instead of terraforming Mars, or settling Mars, or creating new creative centers of political freedom, we could turn people into cyborgs to survive harsh conditions, or establish rigid colonies where individual actions are tightly controlled in pursuit of safety and the greater good. Those outcomes are not what the advocates want, but they may be far more likely. Artificial intelligence may not improve the human condition, but instead might put lots of people out of work.

I should probably start reading some of Stross’ books. But I have to admit, he’s got me a little nervous.

Dwayne Day does not read Space Policy or Astropolitics. He can be reached at zirconic1@cox.net.

Home