New breakthroughs in gene editing will lead to incredible rewards — and risks we can barely contemplate
This is the last installment of “Privatizing the Apocalypse”, a four-part essay published throughout October. Read the previous installments here — Part 1: “The 50/50 Murder,” Part 2: “Deterrence — and the Undeterrable,” and Part 3: “The Deadly Gamble on Super A.I.”
A noxious strain of H5N1 flu started killing people in 2003. It ranks among history’s most lethal viruses. But one of its cousins is much more famous. Popularly known as swine flu, that cousin is also rightly feared. But at its worst, H5N1 is three thousand times more likely to kill those it infects. Not three thousand percent more (which would be plenty bad) but three thousand times more.
The World Health Organization has shown this strain kills a devastating 60 percent of those it strikes. That’s more deadly than Ebola, and almost all cancers. And of course, wayyyyy more (a technical term) than swine flu, which inflicts an almost genial death rate of just 0.02 percent. But for all its foibles, H5N1 has one trait I think we can all get behind: It’s not very contagious at all amongst humans.
Outside of the lab, that is.
Yet, that is.
A flu strain’s transmissibility, i.e. its ability to easily move from person to person, lies in its genes. In 2011, researchers in Wisconsin and Holland looked hard at the relatively non-infectious H5N1 genome and (for lack of a better word) fixed it. This gave them mutant H5N1 strains which are every bit as deadly as the original flavor — but also highly contagious. Like, dare to have it all!
An arm of the revered journal Science said a strain like this could “change world history if it were set free,” by triggering a pandemic “quite possibly with many millions of deaths.” The chair of the U.S. National Science Advisory Board for Biosecurity, Paul Keim, said, “I can’t think of another pathogenic organism that is as scary as this one.” He added, “I don’t think anthrax is scary at all compared to this.” And I’ll add that Keim is an anthrax expert.
There’s an optimistic way of viewing this, and let’s start there. Creating this abomination was a novel undertaking. Only a few experts in well-provisioned labs could have pulled it off in 2011. The ones who did were career virologists whose jobs involve preventing, not causing outbreaks. Most importantly, they merely created a tiny number of highly sequestered germs. Albeit, ones which could kill on the scale of a world war in the event of a boo-boo.
Now, for the other view. Each variation on influenza is also a tiny data file. H5N1’s genome contains fewer letters than this essay (from an alphabet of just four letters). Its “vanilla” version is widely disseminated. And the letter changes needed to yield its most deadly designer mutant would surely fit onto a sheet of paper. Probably a Post-it note.
The first would-be Ender’s attack will be every bit as unprecedented as 9/11. The sole way to contain it is to keep it from being equally unexpected.
That’s a lot of oomph for a speck of data. And unleashing its dormant fury would not require swiping test tubes from its labs of origin. All you’d need is that short list of genetic tweaks. It’s a precise recipe for recreating the virus anywhere. The level of training and specialized gear this would require is daunting. For now. But it’s also plummeting. It will continue to plummet, further and faster. The number of people capable of converting mere data into lethal pathogens will thereby explode.
This surge has a close precedent, which I previously noted, in that lone lab techs can now exceed the output of the 13-year Human Genome Project (HGP) with just hours of effort. The budget compression is likewise extreme, ranging around three million to one since 2003 (the HGP cost roughly $3 billion, whereas the lab tech’s output will run you about a thousand bucks). Though astounding, this prior development is not innately menacing. The HGP’s underpinning technology is about reading genetic code. Whereas weaponizing something like the flu requires editing it.
That’s a different trick. A harder one, too. But as previously occurred with genome reading, the tools for genetic editing are screaming through a fast-forward renaissance. Today’s top editing method didn’t even exist when those ghoulish flu strains were cooked up back in 2011. It’s called CRISPR, and it could be as transformative to gene editing as jet engines were to flight.
CRISPR has already spread to the distant outskirts of academic science. Hundreds of college, and even high school teams convene annually for a bio-engineering jamboree called iGEM, which distributes an open catalog of dozens of CRISPR-based tools — all of them created by other students. iGEM is a truly inspiring event. It draws thousands of young minds who have vast potential to help humankind. It also shows us how quickly and broadly synbio’s breakthroughs will diffuse as the field’s lightning emergence continues.
This makes it unimaginable that narrow and trusted elites will retain exclusive dominion over any element of synbio — including the dark art of making monstrous pathogens, like the 2011 flu hacks. These aren’t brilliant acts of raw creation, after all — but ones of narrow editing. Ones enabled by ever-improving tools, whose enhancements quickly become ubiquitous.
Nature did all the heavy lifting by creating ghastly starting points like influenza, Ebola, anthrax, and meningitis. Laboratories have housed entire zoos of these critters for decades, and their carefully annotated genetic codes are globally accessible to anyone. Convenient links to dozens of them can be found right here (among countless other places).
Untold thousands of turbo-charged pathogens will eventually be designed for a slew of reasons. Some will be thesis projects. Others will be byproducts of medical research. Many will be the handiwork of white-hatted good guys, trying to stay ahead of the bad guys (one of many reasons why iGEM is a laudable endeavor). A few will be designed as attention-getting stunts.
Many of these lethal recipes will be tightly controlled. Indeed, most may never escape the digital vaults of their creators. But tiny data files of great consequence have ways of diffusing. Many profound financial and government secrets are hacked in any given month. And it’s no bold sci-fi forecast to predict that school bio labs will lack the defenses of even security laughingstocks like Equifax.
Not all will try to keep their creations secret anyway. Posting a pandemic-grade genome to Reddit could be someone’s way of giving the world the finger. A techno-anarchist might view it as a sublime moral act. Or it could be a nihilist’s naked attempt to trigger an actual plague. And once those files are out in the wild, they’ll stay there. Ask any movie studio straining to stamp out piracy of its catalog. The blueprints of malign genomes will inevitably course through the Dark Web alongside child porn, drug offers, and terrorist chatter. And as they emerge, they’ll be anthologized, swapped, and stored broadly.
For now, simple genetic printers don’t exist — and no apparatus of any complexity can crank out virus-length genomes from scratch. But this will change. Like computing, genomics and synbio ride exponential improvement curves. At least one is even steeper than that of Moore’s Law, which famously tracks performance gains in computing. This is why today’s copier-sized sequencers out-sequence the Human Genome Project, which preoccupied thousands of scientists (and acres of clunky gear) for 13 years. By 2038, budget DNA synthesizers will crank out wonders the entire field of synbio couldn’t conjure today. And they may require scarcely more technical skill than hitting a Print button.
Richard Preston once said, “the main thing that stands between the human species and the creation of a supervirus is a sense of responsibility among the individual biologists.” This was almost certainly fine back in 2011, when the relevant people were respected virologists. (That said, the FBI concluded that an Army biodefense researcher carried out 2001’s anthrax attacks, so you never know.)
Things will become less stable when the world’s elite life science students enter the pool of potential supervirus creators. This group isn’t renowned for its bloodlust, obviously. However, one of the decade’s most notorious mass murderers once belonged to it. And eventually, another almost certainly will. Because as time passes and a community grows, the odds of someone from it doing something bizarrely awful go from slim, to high, to near-certain.
Creeping further down the academic ladder, pre-med students are far more numerous still, and have certainly been known to go off the rails sometimes. And quiet, brainy high school kids may be one of the deadlier demographics in America. Synbio techniques will gradually diffuse throughout medicine as well. This rightly revered field has done far more good than harm to most of us. But the British Medical Journal also notes that “medicine has arguably thrown up more serial killers than all the other professions put together, with nursing a close second.”
The prior essay in this series argues the most likely culprits in a super A.I. crisis will be smug teams of elites taking shortcuts while chasing profits. But when it comes to synbio risk, lone suicidal mass murderers concern me more. Each year, a tiny fraction of people seek their own deaths in the act of slaughtering as many random victims as possible, as the second essay in this series discusses in depth. They rarely stop until they’re dead themselves. And so, we must rationally assume at least some of them would kill every last one us, if they only could.
Or — if ever they can.
Should such a ghastly outcome ever transpire, what might we call our annihilator? In an email exchange with me about this essay, Naval Ravikant coined the term The Ender, which I find evocative. Some of the people behind America’s near-daily mass shootings are probably Enders at heart. But none have a remote crack at dropping the curtains on humanity. Only those commanding huge nuclear arsenals do, for now. And they’ll retain their grim monopoly for a while, yet.
But not forever. Yes, the risk of an engineered pandemic is currently small. Some might put our odds of dodging catastrophe north of 99.9 percent over the next decade. And they’d probably be right. But the thing about decades is that they’re followed by more decades. And then by half-centuries, and so on. Should a giga-murder happen, its exact year will be a minor aspect of a tragedy without precedent. Those who mock fears or precautions today based on accurate forecasts of a quiet early decade or two will not then be lauded as prescient heroes.
Of course, it’s ideal and relieving that there’s been no synbio terrorism to date. But this has zero predictive relevance. Myspace dominated social media until 2008. And Myspace was about as ingeniously run as a coke-addled fraternity of C-minus students. Who imagined its heir would soon be accused of hijacking major elections? If you’re raising your hand, please lower it, because you’re fooling no one.
The experts I’ve discussed this with think it’s nigh impossible that any one pandemic could come close to wiping us out. Society’s defenses against pandemics are incalculably better than when the Black Death rampaged, or when Spanish Flu killed up to 6 percent of all humans. But this could be gravely offset by intention and malevolent design. For all its faults, Spanish Flu bore us no ill will. Nor did blind evolution imbue it with cunning modifications to maximize deadliness, obscure detection, make incubation periods maximally lethal, etc. And let’s not forget that multiple pathogenic demons could be released at once.
We must therefore be vigilant to potential abuses of the amazing tools synbio is starting to bless us with. This means good people need to start thinking like the Ender now — lest there ever actually be an Ender — because a failed Ender could deform human history without achieving anything close to full annihilation.
Few countries retain the resilience that once helped us weather widespread death (think of how would it go if morbid cases swamped your local hospital’s capacity by a factor of hundreds). Also, consider the years of bloodshed and general derangement that followed the 2,977 homicides of September 11th. A dud of a wannabe Ender attack could be thousands of times more lethal than that. Who knows what grievous harm we might inflict upon ourselves in the aftermath?
But what could possibly motivate a wannabe Ender? Those of us who can’t fathom the mindset of rampage murderers may be ill-equipped to answer this. But motivations might not be limited to criminal insanity. Nonviolent belief systems often get contorted to promote mass murder. We can’t rule out one someday being hijacked to champion complete human destruction.
Consider the spooky antinatalists, who contend that life is so awful, it’s immoral to bring further humans into the world. Could an offshoot faction conclude that it would be a high moral act to exterminate the species? The Voluntary Human Extinction Movement — whose plan for restoring the biosphere should be self-evident — might likewise spawn a Mandatory faction.
This may sound far-fetched, but as suicidal mass murder normalizes, it’s hitching rides on causes and motives that would have been unimaginable recently. Some so-called incels have seen their failure to get nooky as just cause for mass slaughter. A grumpy taxpayer ended it all while nipping at the IRS. A minor YouTuber went out blasting at the service’s staff for delivering less attention and ad dollars than she craved. And so on.
Even jihadi suicide attacks were unheard of back when America endured thousands of bombings per year (yes, really) in the early ’70s. Those attacks were carried out by domestic groups, which mostly strove not to hurt anyone (these were hippie terrorists, see). Suicide bombing gained its first scant toehold in the vast Muslim world in the ’80s. This was at first limited to Lebanese militias tied to Shi’a Islam, which connects to the concept of martyrdom in deep and unique ways.
The new practice took many years to jump to the larger Sunni world. It wasn’t even adopted by neighboring Palestinian groups, which had by then logged decades of armed conflict. Israel’s first forty years of history passed with ample violence, but not a single suicide bombing on its territory. But the contagion eventually jumped — and in the first half-decade of the millennium, there were 129 such attacks within Israel’s borders and the occupied territories, substantially all carried out by Palestinian Sunnis.
This shows the historically unthinkable can swiftly become all too commonplace. And violent acts — much like slang, skirt lengths, and autotuning — can be trendy. Suicidal mass murderer is a meme with momentum. And some of its practitioners are wannabe Enders at heart. But wannabe Enders who buy mass-murder gear at retail are hard to distinguish from run-of-the-mill rampage killers. So we don’t know how common the Ender mindset is.
The unmistakable signature of a would-be Ender will be an attack using a highly lethal, untargeted technology. It will be a singular event (until the copycats show up). And I do believe it will involve synbio.
A gradual, obvious drumbeat of lesser incidents preceding a synbio cataclysm could trigger awareness and countermeasures. But such a pattern is unlikely. The first synbio attack will occur when a wannabe Ender enters the group of people who are capable of perpetrating one. People with an Ender mindset are probably quite rare. So this will happen not at all — until it happens all at once.
In the run-up to this, a technology ban would be senseless. Above all, it would be all but unenforceable (recall that illicit drug labs are both illegal and ubiquitous). And at its most effective, it would give rogue nations and evil actors a monopoly on apocalyptic arts. That’s no way to prevent a catastrophe.
The first would-be Ender’s attack will be every bit as unprecedented as 9/11. The sole way to contain it is to keep it from being equally unexpected. Our defenders enabled 9/11 by having a catastrophic lack of imagination. Not a lack of foresight (airports had been geared to detect and prevent hijackings since the early ’70s). Not a lack of budget (we quickly spent trillions reacting to 9/11, after all). And certainly not a lack of warning signs. But the imagination deficit was astounding.
Just suppose that a roomful of intelligence analysts were summoned to a brainstorming session on the morning of September 10th, 2001. They’re told to figure out how terrorists could bring down a skyscraper with just a few sharp objects. Assume they’re each brilliant, and urged to think well outside the box. And that they’re highly diverse, representing a wide spectrum of ethnic and religious perspectives. How long would they take to solve the puzzle?
Recall that while the 9/11 planners were many things, they were not geniuses. Also, that they assigned their non-ingenious plan to a team of mediocrities — most of whom were too dim to realize they were on a suicide mission. These were scrawny men, with an average height of about 5’ 6”, who could barely navigate the alien country they were operating in.
So again: how long should it take our brilliant analysts to hit upon the worst-case act of a group this underpowered? Could it possibly take until lunchtime? Or even their first coffee break? I doubt it would have taken them even an hour, impossible though it is to prove. From this, I conclude that people of their ilk were rarely asked to use even a wisp of their imaginations.
So what should we do about the coming collision of the Ender mindset with synbio, and perhaps other technologies?
This is actually a rare instance in which it would be highly productive to freak out. Like, a lot.
Out in public, we need to tell ourselves terrifying stories, and lose a great deal of sleep. Within top-secret conference rooms, we should meanwhile ask some of our most brilliant and twisted citizens to think hard about low-budget, diabolical acts before wannabe Enders come up with them. Then, we need to make those acts nigh impossible. Or at minimum, completely unsurprising.
We’re actually pretty good at this. Faced with the prospect of nuclear annihilation, we freaked out for years at a stretch. We told ourselves ghost stories with names like Dr. Strangelove, On the Beach, Fail Safe, and War Games. Much scarier tales were meanwhile told at top secret conferences and in classified briefings. Then, when the Cuban Missile Crisis hit, the chain-smokers in those Moscow and Washington war rooms didn’t need to use their imaginations one iota. Plenty of imagining had already been done for them — and its output had been singed into their very neurons. They knew exactly what was at stake. And so, humanity survived.
George Orwell freaked on another front, in 1948. Haunted by premonitions about modern totalitarianism, he spun a terrifying yarn called 1984. This inoculated much of the global intelligentsia from romanticizing Stalinism. Dean Acheson, and other top policymakers, also panicked in highly productive ways. There were plenty of excesses that no one should ever be proud of. But in the end, the USSR vanished without firing a shot — and this was far better than the best-case outcome any realist ever pondered.
There are countless other examples. For instance, you can laugh all you want at the Terminator franchise. But while it won’t preclude a super A.I. crisis, it’s far less likely that one will catch us wholly unawares. Ditto an alien invasion (although that one ranks close to vampires and werewolves on the probability scale).
If 1 percent of our intelligence budget had gone into thinking wicked, creative thoughts before Osama bin Laden could come up with them, a middle school hockey team could’ve stomped his wimpy crew before they got to flight school. So, again: We cannot let our imaginations to fail us anew. Which means we need the equivalent of that counterfactual room of analysts on September 10th.
Only, we need armies of them. And we need them on the case for their entire careers, not just a single morning. We need to pay them well to work long hours, and flood their offices with coffee, pizza, beer, weed, acid, and maybe even toad venom. And we need self-organizing volunteers. People to tell the sorts of ghost stories that can rally society’s antibodies before something lethal actually happens. And proactive folks who can spot subtle security loopholes in their workplaces or daily lives before a wannabe Ender can.
The aspiring Enders that we’ll really need to worry about have certain scary advantages. They live at least a few decades in the future, so they know lots of things that we don’t. They’re fueled by hatred and grudges we can’t fathom, which pushes their imaginations in alien directions. They’re probably smarter on average than we are on average. Plus, they’re wholly committed, and truly relentless.
But we outnumber them vastly. We’re connected, whereas they’re isolated. And we can start thinking about this now — perhaps before most dangerous would-be Enders are even born.
As a novelist, I can’t help but end with a little plot twist. It’s this: I’m a relentless optimist. A borderline pathological one. I see so much radiant potential in synbio, A.I., and other emerging technologies, it practically burns my retinas. I spent the last year discussing these subjects with dozens of brilliant scientists and founders, and am convinced that if we get things right, humanity faces a dazzling future.
The antinatalists, in fact, have it backward. Allowing an apocalypse on our watch would be a cataclysmic moral wrong to our heirs, because they’re likely to have it so goddamned good (yes, even though they’ll probably whine as much as we do — which is annoying — but that’s humans for you). If you doubt that, compare your own lot to that of anyone living in the Dark Ages, potentates and royalty included. By any measurable gauge of human thriving, the centuries get more and more odious as you recede into the past.
But luckily for us, technology accrued relentlessly over those very centuries. Technology that now prevents diseases, generates calories, transforms matter, transports everything, distributes knowledge, and so on.
Some may say precautions are pricey, and won’t we look dumb if they prove unnecessary? But we feel grateful, not foolish, when our fire insurance providers don’t send checks to the cinders of our homes. We’re not annoyed to have wasted some moments fastening a seatbelt when a trip ends safely. And low plane crash rates don’t invalidate aviation safety spending — they vindicate it.
Humanity may or may not face a long future. If it does, that future will be shaped and defined by uninvented technologies we can’t possibly foretell. This is exciting as hell! But it calls for great humility and prudence. Because if there is a long future, it’s worth getting to.