Copyright

HarperCollinsPublishers

1 London Bridge Street

London SE1 9GF

www.harpercollins.co.uk

First published in the United States by Currency, an imprint of Random House, a division of Penguin Random House LLC, New York.

This edition published by HarpercollinsPublishers 2020

FIRST EDITION

Text © 2020 by HyperAnalytic, Inc.

Cover layout design ©HarperCollinsPublishers Ltd 2020

A catalogue record of this book is available from the British Library

Sinan Aral asserts the moral right to be identified as the author of this work

All rights reserved under International and Pan-American Copyright Conventions. By payment of the required fees, you have been granted the nonexclusive, non-transferable right to access and read the text of this e-book on screen. No part of this text may be reproduced, transmitted, downloaded, decompiled, reverse engineered, or stored in or introduced into any information storage retrieval system, in any form or by any means, whether electronic or mechanical, now known or hereinafter invented, without the express written permission of HarperCollins e-books.

Find out about HarperCollins and the environment at www.harpercollins.co.uk/green

Source ISBN 9780008277116

Ebook Edition © September 2020 ISBN: 9780008277123

Version 2020-09-04

Note to Reader

This ebook contains the following accessibility features which, if supported by your device, can be accessed via your ereader/accessibility settings:

 Change of font size and line height

 Change of background and font colours

 Change of font

 Change justification

 Text to speech

 Page numbers taken from the following print edition: ISBN 9780008277116

Dedication

To my parents—Thank you for every opportunity,

every conversation, and every hug.

To Kaya—May you always stay curious, creative,

and hungry for knowledge.

Contents

COVER

TITLE PAGE

COPYRIGHT

NOTE TO READER

DEDICATION

PREFACE | Pandemics, Promise, and Peril

CHAPTER 1 | The New Social Age

CHAPTER 2 | The End of Reality

CHAPTER 3 | The Hype Machine

CHAPTER 4 | Your Brain on Social Media

CHAPTER 5 | A Network’s Gravity Is Proportional to Its Mass

CHAPTER 6 | Personalized Mass Persuasion

CHAPTER 7 | Hypersocialization

CHAPTER 8 | Strategies for a Hypersocialized World

CHAPTER 9 | The Attention Economy and the Tyranny of Trends

CHAPTER 10 | The Wisdom and Madness of Crowds

CHAPTER 11 | Social Media’s Promise Is Also Its Peril

CHAPTER 12 | Building a Better Hype Machine

NOTES

LIST OF SEARCHABLE TERMS

ACKNOWLEDGMENTS

ILLUSTRATION SOURCES

ABOUT THE AUTHOR

PRAISE FOR SINAN ARAL AND THE HYPE MACHINE

ABOUT THE PUBLISHER

PREFACE

Pandemics, Promise, and Peril

The coronavirus pandemic of 2020 was a “black swan” event, the repercussions of which were felt throughout the world’s health systems, economy, and the very fabric of everyday life. Everyone on earth remembers where they quarantined, who they longed to see, and how they coped with the tremendous mental and physical strain the virus exacted. But another dramatic, albeit subtler, consequence of COVID-19 was the rather abrupt shock it delivered to the world’s global communication system—the central nervous system of digital connections that links our planet. Times Square, Trafalgar Square, and Tahrir Square became ghost towns. And as the virus sent humanity scurrying off the streets and into their homes, it pushed billions of people onto their laptops and smartphones, scrambling to get online. The world logged on to Facebook, Twitter, WhatsApp, Instagram, YouTube, and LinkedIn in record numbers, desperate for news, medical information, social support, human connection, and jobs. The day the offline world stood still, the online world ignited like a digital forest fire.

Demand for social media skyrocketed. Facebook Messenger, WhatsApp, and Facebook Live saw 50 percent increases in usage overnight. Voice calling over Facebook’s apps doubled, while group calls in Italy grew by over 1,000 percent. As movie theaters closed, new downloads of Netflix jumped 66 percent in Italy and 35 percent in Spain. Netflix crashed under the weight of the surge. YouTube was forced to throttle its video quality to handle the deluge. And the Internet stuttered repeatedly throughout the pandemic.

Social collaboration tools exploded with activity. Slack CEO Stewart Butterfield tweeted, “1,597 days after hitting 1M *simultaneously connected* users in Oct ’15 we pass ten million. 6 days later: 10.5M, then 11.0M. Next day, 11.5M. This Monday, 12M. Today 12.5M.” He attached a graph showing the number of “newly created [Slack] work teams” to his tweet thread, which, after March 12, looked like a hockey stick with an extra-long handle pointing straight up. Digital natives were already on social media, but the coronavirus forced many digital Luddites to use social technologies for the first time. New users flocked to social platforms in droves, building armies of fresh profiles, wiring into what I call the “Hype Machine”—the real-time communications ecosystem created by social media. Describing their attempts to cope with the demand, Alex Schultz and Jay Parikh, Facebook’s heads of analytics and engineering, respectively, wrote, “Usage growth from COVID-19 is unprecedented across the industry, and we are experiencing new records in usage every day.” Mark Zuckerberg was blunter: “We’re just trying to keep the lights on over here,” he said.

As the entire planet was denied physical contact for months on end, the coronavirus shocked our use and perception of social technologies in dramatic ways. Facebook, Twitter, WhatsApp, and Instagram became indispensable sources of human connection, timely medical information, social support, outreach, pandemic fundraising, free impromptu concerts, collaborative art projects, and real-time updates about the spread of the virus. Families played Monopoly over Facebook, friends attended cocktail mixers in live group chats, neighbors maintained vibrant WhatsApp groups, and many followed the news on Twitter. Social technologies connected humanity when the planet was forced to disconnect. Group video kept families together. Daily messages kept tabs on our parents and our kids. Hangouts kept work teams collaborating while the world stood still. Even my six-year-old, who gets almost no screen time, connected with his friends and his first-grade class every day over the Hype Machine.

Social platforms provided access to critical medical information about how to socially distance, whether to wear masks, where the hot spots were, and how to be safe at home. The platform companies got right to work, providing new services and data to model and mitigate the spread of the pandemic. Facebook used aggregate anonymous mobility data to create “colocation” maps of where people from different geographies were most likely to cross paths, which aided epidemiologists modeling where the pandemic was likely to spread next, from one geography to another. They spun up “disease prevention maps” that leveraged Facebook’s mobile app data to help understand how people were moving and therefore how they might be spreading the virus.

As the director of the MIT Initiative on the Digital Economy, one of the largest centers researching the influence of digital technologies on our world, and head of MIT’s Social Analytics Lab, a team of thirty brilliant MIT faculty and students studying the impact of social media on society, I felt we needed “all hands on deck” to address the crisis. We organized Zoom meetings to brainstorm how we could contribute to efforts to address the pandemic. While stressing the need to first do no harm, I reached out to my contacts at social platforms around the world to see how we could help.

In a week, we created three projects supporting national and international health organizations, measuring the effect of social distancing on COVID-19 spread and fighting pandemic misinformation online. I reached out to Facebook and suggested a collaboration. They responded quickly and said that, since we already had a data licensing agreement, they could share data right away. We focused on modeling the effect of social distancing on the pandemic’s spread. Facebook’s disease prevention maps tracked aggregated, anonymous Facebook mobile app data on user location density, movement, and network connectivity, as well as the average number of half-kilometer-squared spaces in which Facebook users were present each day, compared to precrisis levels. We worked to combine this data with detailed records of the social distancing orders imposed on different regions, states, and cities around the world. The idea was to estimate the effect of social distancing orders on the number of Facebook users present in public places within the region or city in which the order was imposed and how those effects spilled over into other regions. We wanted to know if distancing orders worked and, more important, when they didn’t and why. We also advised Facebook’s COVID Symptom Survey, which supplemented traditional disease surveillance with a million surveys per week in the United States, asking Facebook users if they had common COVID symptoms and whether they were staying home. As I write this, Facebook is preparing to launch a global survey to users worldwide.

I then called John Kelly and Camille François of Graphika, one of two companies commissioned by the U.S. Senate Intelligence Committee to investigate Russian interference in the 2016 and 2020 U.S. presidential elections, to brainstorm how we could track and fight the spread of COVID misinformation and the growing threat of election misinformation ahead of the 2020 election. We decided to monitor, track, and publicly report on the automated software “bot,” cyborg, and troll networks spreading misinformation about coronavirus and the election around the world, and to measure how misinformation was affecting disease prevention and voting.

I also connected with Gustav Praekelt of the Praekelt Foundation in South Africa. Gustav and I already had an active, countrywide project using social media to fight the spread of HIV/AIDS in South Africa. When I asked him how we could collaborate on COVID-19, he told me he had repurposed the WhatsApp and Messenger tools we were using in our HIV project to spread official COVID medical information around the world. They had created something called COVIDConnect, an automated software robot that would field questions from the public and respond to them with the correct information from official sources over WhatsApp, Facebook Messenger, and text messaging. COVIDConnect had become the official engine of the global World Health Organization (WHO) WhatsApp channel and powered the automated national COVID response hotlines in South Africa, New Zealand, Australia, and ten other countries in Africa and Southeast Asia, amassing 15 million users in just two weeks. It was fast becoming the go-to source for official information about COVID. But Praekelt and Facebook had a problem. They were worried about the spread of COVID misinformation on WhatsApp. It’s hard to root out misinformation on an encrypted platform like WhatsApp, because messages can’t be publicly tracked. So we got to work building a system to debunk COVID misinformation over the official WHO and national WhatsApp accounts.

The social platforms addressed the economic fallout from the pandemic as well. Small businesses used their Facebook pages to sell online. Live videocasts were hosted across social media to replace in-store events that usually generate foot traffic and boost sales. Stage shows were produced and aired over Instagram Stories and TikTok. Yoga classes, guitar lessons, and hairstylist sessions all transitioned to the Hype Machine. Facebook even set up a $100 million small-business relief fund to dole out no-strings-attached cash grants to keep small businesses afloat.

These projects were just beginning as I finished this book. The people who make the social media industry tick are dedicated technologists. They care about the future of our planet. They’re wicked smart, and they’re committed to making the world a better place. But social media’s impact on the world is not determined by intention alone. As we all know, there have been many missteps in building the Hype Machine.

Following the death of George Floyd, Mark Zuckerberg defended his decision to allow unaltered and unlabeled, divisive and inflammatory Facebook messages by President Trump that seemed to threaten violence in response to the protests and including the words “when the looting starts the shooting starts,” a phrase used by police chiefs and segregationist politicians during crackdowns against the civil rights movement. Twitter limited the public’s exposure to the president’s messages, saying it “violated the Twitter Rules about glorifying violence.” Zuckerberg’s inaction led to virtual walkouts by Facebook employees unable to align themselves with a company policy that seemed to appease racist rhetoric.

The backlash against Zuckerberg was a return to an old refrain. Our enthusiastic embrace of social media during the pandemic lockdown of 2020 was a 180-degree reversal from 2019, in the weeks and months before COVID hit. Before the pandemic, social media was a pariah. The #deletefacebook movement was gaining steam. The Cambridge Analytica scandal had forced Mark Zuckerberg to testify on Capitol Hill and in front of the European Parliament. Lawmakers were angling to break up the social media giants on antitrust grounds. Just weeks before the virus hit China, Sacha Baron Cohen called social media “the greatest propaganda machine in history.” In his remarks before the Anti-Defamation League, he said, “Facebook, Google, YouTube, Twitter, and others … reach billions of people. The algorithms these platforms depend on deliberately amplify … stories that trigger outrage and fear. … It’s why fake news outperforms real news, because studies show lies spread faster than the truth.” The study he was referring to was one I published with my colleagues Deb Roy and Soroush Vosoughi in a cover story for Science, called “The Spread of True and False News Online.” I’ll return to it in the first two chapters of The Hype Machine.

But misinformation wasn’t the only concern of social media critics before the pandemic. Facebook, Twitter, WhatsApp, Instagram, and YouTube were undermining our privacy, facilitating foreign interference in our democracy, threatening the integrity of our elections, promoting political lies for money, radicalizing terrorists, chilling free speech, and promoting hate speech. They spread genocidal propaganda, livestreamed mass murders like the one in Christchurch, New Zealand; promoted predatory lending to minorities; discriminated against women in employment advertising; tracked our every move; manipulated our emotions for revenue; and promoted political polarization.

Social media’s promise and peril even played out in debates about the pandemic itself. Yes, we relied on social media for human connection, social support, and lifesaving information. But it was also a cauldron of misinformation about impending national lockdowns and false cures, nationalistic finger-pointing between the United States and China, and foreign interference designed to fan the flames of our fears. Privacy debates took on new meaning during the COVID crisis as the threat of “surveillance capitalism” morphed into lifesaving “disease surveillance.” Facebook wasn’t surveilling for profit during COVID; it was filling gaps in inadequate national disease surveillance programs with scalable symptom surveys that identified the pandemic’s spread. At the same time, Google, Apple, and MIT developed Bluetooth-based contact tracing systems that would alert users who opted in if they had come in close physical proximity to the Bluetooth-enabled device of a COVID carrier. Privacy advocates listened in horror as the tech giants swore the system would remain anonymous. The perils of social media surveillance for privacy were sharply contrasted with the promise of social media surveillance for health. Some thought the surveillance was worth the privacy risk. Others thought the risks outweighed the benefits.

So which of these visions of social media is correct—the promise or the peril? Is the Hype Machine a force for good, for collective intelligence and solidarity? Or is it a scourge and a pariah? The global reach of Facebook, WhatsApp, Twitter, Instagram, and YouTube affects our communications, data, and privacy, as well as the flow of information around the world. It has the potential to be uniquely dangerous. But the pandemic reminded us how invaluable this far-reaching global communication network is to us, especially in times of need, and just how much we’ve come to rely on it for everything from human connection to news to job opportunities to entertainment to dating and relationships in our everyday lives.

During the pandemic, the source of the Hype Machine’s promise was also the source of its peril. As I will describe, this is true of social media in general, making it difficult to regulate. The solutions that social media enable compromise our privacy beyond what some think is necessary. But, in Europe for example, stringent privacy regulations prevented the use of social media to track and tackle the pandemic. These examples highlight many important unanswered questions.

Is social media a force for meaningful connection, collaboration, social support, and access to lifesaving information? Or is it a propaganda machine that, left unchecked, will destroy democracy, civil society, and our health? Can the promise of social media be realized without the peril? Or are they inexorably linked?

As I explore in this book, the Hype Machine has the potential for both promise and peril. And the decisions we make in the next eighteen to twenty-four months in how we design, regulate, monetize, and use social media will determine which path we realize. We’re at a crossroads. To act responsibly, we have to educate ourselves about how social media works.

My goal is to take you on a roller-coaster journey through what I’ve learned studying, building, investing in, and working with social media over the last twenty years. It’s a harrowing journey with unbelievable discoveries and sordid scandals about how social media impacts our democracy; how it can disseminate lies while connecting us to valuable truths; how it fights repression at times while promoting it at others; how it propagates hate speech while defending free speech; and, most of all, how all this works, under the hood, to hook us neurologically, emotionally, socially, and economically. The story not only reveals the business strategies behind social media, but also the relationship between social media’s design and how it affects us.

The story of the Hype Machine as either the promise or the peril alone is one-sided. The truth is more complicated—it’s sometimes uplifting and at other times depressing, sometimes shocking but always illuminating. Most of all, it’s a story that involves every one of us, now and for generations to come.

CHAPTER 1

The New Social Age

This is the whole point of technology. It creates an appetite for immortality on the one hand. It threatens universal extinction on the other. Technology is lust removed from nature.

—DON DELILLO

Human beings have always been a social species. We’ve been communicating, cooperating, and coordinating with one another since we were hunting and gathering. But today something is different. Over the last decade, we’ve doused our kindling fire of human interaction with high-octane gasoline. We’ve constructed an expansive, multifaceted machine that spans the globe and conducts the flow of information, opinions, and behaviors through society. This Hype Machine connects us in a worldwide communication network, exchanging trillions of messages a day, guided by algorithms, designed to inform, persuade, entertain, and manipulate us.

The object of this machine is the human psyche. It was designed to stimulate our neurological impulses, to draw us in and persuade us to change how we shop, vote, and exercise, and even who we love. It analyzes us to give us options for what to read, buy, and believe. It then learns from our choices and iteratively optimizes its offerings. As it operates, it generates a data exhaust that traces each of our preferences, desires, interests, and time-stamped, geolocated activities around the world. It then feeds on its own data exhaust, refining its process, perfecting its analysis, and improving its persuasive leverage. Its motivation is money, which it maximizes by engaging us. The more precise it gets, the more engaging and persuasive it becomes. The more persuasive it becomes, the more revenue it generates and the bigger it grows. This is the story of the Hype Machine—the social media industrial complex: how it was designed, how it works, how it affects us, and how we can adapt to it. And the story opens in Crimea.

Ten Days

On a cold day in February 2014, heavily armed gunmen surrounded the Crimean parliament building in Simferopol, Ukraine. They wore no sovereign markings but were later confirmed to be Russian special forces reacting to the deposition of Ukrainian president Viktor Yanukovych just days before. By all accounts, the gunmen were organized and professional. After breaking through the front door, they cut the building’s communications, confiscated all mobile electronic devices, and systematically controlled who entered and exited the building, maintaining a tight perimeter and allowing no foreign journalists inside.

A few hours later, amid reports of heavy intimidation and fraud by the gunmen inside, the Crimean parliament voted to dissolve the government and replace Prime Minister Anatolii Mohyliov with Sergey Aksyonov, whose pro-Russian Unity Party had won only 4 percent of the vote in the previous election. Less than twenty-four hours later, similarly unmarked troops occupied the Simferopol and Sevastopol international airports and set up checkpoints on Crimean roads throughout the region. Two days later Aksyonov, who had earned the nickname “the Goblin” during his days as a businessman with ties both to the Russian mafia and to pro-Russian political and military groups (Aksyonov denies the allegations), wrote a personal letter to Vladimir Putin, in his new capacity as the de facto prime minister of Crimea, formally requesting Russian assistance in maintaining peace and security there.

Before the Ukrainian government could declare Aksyonov’s appointment unconstitutional, pro-Russian protests were whipped up throughout Crimea, developing a groundswell of visible support for reunification with Russia. The sentiment seemed one-sided, with many in Crimea expressing a strong desire to return to Russia. Within hours of Aksyonov requesting assistance, Putin received formal approval from the Russian Federation Council to send in troops. The Russian consulate began issuing passports in Crimea, and Ukrainian journalists were prohibited from entering the region. The next day Ukrainian defenses were under siege by the Black Sea Fleet and the Russian Army. Five days later, just ten days after the ordeal began, the Supreme Council of Crimea voted to re-accede to Russia after sixty years as part of Ukraine.

It was one of the quickest and quietest annexations of the postwar era. As former secretary of state Madeleine Albright testified, it “marked the first time since World War II that European borders have been altered by force.” In just ten days, the region was flipped, like a light switch, from one sovereignty to another with barely a whisper.

The debate about what happened in Crimea continues today. Russia denies it was an annexation. Putin views it, instead, as an accession by Crimea to Russia. His adversaries claim it was a hostile encroachment by a foreign power. In essence, there was a dispute over the will of the Crimean people—a clash of competing realities, if you will. On the one hand, Russia claimed Crimean citizens overwhelmingly supported a return to the Russian Federation. On the other hand, pro-Ukrainian voices claimed the pro-Russian sentiment had been orchestrated by Moscow rather than by the people themselves.

Framing the Crimean reality was essential to restraining foreign intervention in the conflict. If this was an annexation, NATO would surely have to respond. But if this was an accession, overwhelmingly supported by the Crimean people, intervention would be harder to justify. So while the clandestine military and political operations were ruthlessly organized and flawlessly executed, Russia’s information operation, designed to frame the reality of what happened on the ground in Crimea, was even more sophisticated, perhaps the most sophisticated the world had ever seen. And when it came to framing that reality, social media—what I call the Hype Machine—was indispensable.

The Spread of Fake News Online

To communicate my perspective on Crimea, I have to first take you on a detour, through a story within a story, to give you some context for how I understand the events that unfolded in Ukraine. In 2016, two years after the annexation of Crimea, I was in my lab at MIT, in Cambridge, Massachusetts, hard at work on an important research project with my colleagues Soroush Vosoughi and Deb Roy. We had been working for some time, in direct collaboration with Twitter, on what was then the largest-ever longitudinal study of the spread of fake news online. It analyzed the diffusion of all the fact-checked true and false rumors that had ever spread on Twitter, in the ten years from its inception in 2006 to 2017.

This study, which was published on the cover of Science in March 2018, revealed some of the first large-scale evidence on how fake news spreads online. During our research, we discovered what I still, to this day, consider some of the scariest scientific results I have ever encountered. We found that false news diffused significantly farther, faster, deeper, and more broadly than the truth in all categories of information—in some cases, by an order of magnitude. Whoever said “a lie travels halfway around the world while the truth is putting on its shoes” was right. We had uncovered a reality-distortion machine in the pipes of social media platforms, through which falsehood traveled like lightning, while the truth dripped along like molasses.

But buried in these more sensational results was a less obvious result, one that is directly relevant to Crimea. As part of our analysis, before building more sophisticated models of the spread of true and false news on Twitter, we produced a simpler graph. We plotted the numbers of true and false news cascades (unbroken chains of person-to-person retweets of a story) in different categories (like politics, business, terrorism, and war) over time (Figure 1.1). The total spread of false rumors had risen over time and peaked at the end of 2013, in 2015, and again at the end of 2016, corresponding to the last U.S. presidential election. The data showed clear increases in the total number of false political rumors during the 2012 and 2016 U.S. presidential elections, confirming the political relevance of the spread of false news.

Figure 1.1 The fact-checked true (light gray), false (dark gray), and mixed (partially true, partially false) (black) news cascades on Twitter from 2009 to 2017. A fact-checked news cascade is a story that was fact-checked by one of six independent fact-checking organizations in our study, diffusing through the Twitter network as it is tweeted and retweeted by Twitter users.

But another, more subtle result also drew our attention. Over the ten-year period from 2006 to 2017, there was only one visible spike in the number of rumors that contained both partially true and partially false information, which we called “mixed” rumors. In the original graph, opposite, it was difficult to see. So we filtered the data and replotted the graph, this time considering only political news. That’s when we saw it—a single, clear spike in the spread of stories that contained partially true and partially false information in the two months between February and March 2014 (See the spike labeled “Annexation of Crimea” in Figure 1.2.) It corresponded directly to the Russian annexation of Crimea.

Figure 1.2 The fact-checked true (light gray), false (dark gray), and mixed (partially true, partially false) (black) political news cascades on Twitter from 2009 to 2017.

This result was striking, not just because it was the largest spike of mixed news in the history of verified stories spreading on Twitter (and more than four times larger than any other spike in mixed political news), but also because it ended almost as quickly as it began, right after the annexation was complete. When we investigated further, we discovered a systematic appropriation of social media by pro-Russian entities that proactively used the Hype Machine to control the Ukrainian national perception of the events in Crimea, and the international perception of what was happening there, to ultimately frame the will of the Crimean people.

Ask Mark

On May 14, 2014, Mark Zuckerberg’s thirtieth birthday, a Facebook user from Israel asked him to intervene against state-sponsored Russian information warfare in Ukraine. Zuckerberg was hosting one of his now famous Q&A Town Halls at Facebook headquarters. These Town Halls are public opportunities for Facebook users worldwide to write in and pose questions about Facebook and its governance directly to Mark himself. On this particular day, the Q&A took place in a moderately sized room at Facebook’s headquarters in Menlo Park, California. Users had come, some from around the world, to ask questions directly to the CEO of the world’s largest social network.

After a few opening pleasantries, the audience sang a muffled “Happy Birthday” to Zuckerberg, and the questioning began. The moderator, a Facebook employee named Charles, read the first question aloud: “Mark, this question comes from Israel, but is about Ukraine. … It’s from Gregory, and he says, ‘Mark, recently I see many reports of unfair Facebook account blocking, probably as a result of massive fake abuse reports. These often involve the Facebook accounts of many top pro-Ukrainian bloggers and posts about the current Russian-Ukrainian conflict. My question is: Can you or your team please do something to resolve this problem? Maybe create a separate administration for the Ukrainian segment, block abuse reports from Russia, or just monitor the top Ukrainian bloggers more carefully? Help us, please!’ And, as a follow up to this, and we’ll show this on the screen,” said Charles, “the Ukrainian president, Petro Poroshenko, actually sent in a question as well and he asks: ‘Mark, will you establish a Facebook office in Ukraine?’”

If Facebook were a country, it would be the world’s largest, and this was its version of participatory government in action. Mark cleared his throat and noted that he had prepared for this question in advance because, at 45,000 votes, it was “by far the most voted on question we’ve ever had at one of these Q&As.” He then rolled straight into a canned speech about Facebook’s content curation policies.

But the historical significance of what was happening in Crimea far exceeded Zuckerberg’s mundane response. He vastly underestimated the role Facebook was playing in Ukraine in 2014 (just as he would later underestimate Facebook’s role in foreign election meddling in the United States in 2016). The information war in Ukraine was far more complex and consequential than Zuckerberg let on.

InfoWars

After the Facebook Town Hall, it became clearer—through our own research and that of several investigative journalists—that in 2014 Russia had engaged in a complex, two-pronged information warfare strategy in Crimea, using the public application programming interfaces (or APIs) that Facebook and Twitter make available to users to engage with, direct, and orchestrate the flow of information online.

The first prong was designed to suppress pro-Ukrainian voices. If Russia could demonstrate that the overwhelming desire of Crimean citizens was to accede to Russia, they could legitimize their annexation and reframe it as a liberation. Suppressing legitimate pro-Ukrainian voices, therefore, was vital in the fight to orchestrate pro-Russian sentiment in Crimea. This prong’s effectiveness was evident in the pleas for help from the Ukrainian blogging community. Every time a pro-Ukrainian message was posted, it would be overwhelmed by hundreds of fraud and abuse reports, claiming the post contained porn or hate speech. These are common tactics used by Russia’s Internet Research Agency, the shadowy social media organisation, and the subject of Special Counsel Robert Mueller’s indictments on a conspiracy to defraud the United States by manipulating the 2016 presidential election. Some have speculated that Russia programmed software robots (or “bots”) to post fraud and abuse claims automatically whenever a pro-Ukrainian voice appeared online. Confronted with thousands of such abuse reports, Facebook took down the “offending” messages and banned their authors, effectively banishing pro-Ukrainian voices from its platform.

The second prong of the information war involved the creation and dissemination of disinformation through fake tweets, posts, blogs, and news. When violent clashes broke out in Odessa on May 2, 2014, between pro-Russian separatists and supporters of an independent Ukraine, a story written by a local doctor, Igor Rozovskiy, was circulated widely on Facebook. Dr. Rozovskiy claimed, in a long, detailed post, that Ukrainian nationalists had prevented him from saving a man wounded during one of the clashes. He said they had shoved him aside aggressively while “vowing that Jews in Odessa would meet the same fate.” He added that “nothing like this happened in my city even under fascist occupation.” The post went viral on Facebook, and translations soon appeared in English, German, and Bulgarian.

A day later, on May 3, Russian foreign minister Sergey Lavrov gave a speech to the UN Human Rights Council in Geneva, in which he claimed that “we all know well who created the crisis in Ukraine and how they did it. … West Ukrainian cities were occupied by armed national radicals, who used extremist, anti-Russian and anti-Semitic slogans. … We hear requests to restrict or punish the use of Russian.” Lavrov’s depiction of the events in Crimea and Ukraine mirrored Dr. Rozovskiy’s perfectly. They both claimed that anti-Semitic Ukrainian nationalists had committed violence against Jews and were threatening to escalate that violence. That same day Ukrainians watched footage of actual violent clashes between pro-Ukrainian and pro-Russian forces in Odessa, on television, re-aired repeatedly, visually reinforcing the story. The simple narrative created by Russia, deploying partially true and partially false information, was able to distort reality by altering some but not all of the facts.

So who was Dr. Rozovskiy, and what was his relationship to Russia? It turns out he had no relationship to Russia, or to anyone. His account had been created the day before his post. Dr. Rozovskiy was a fake—he was fake news personified. He was not only parroting the Russian foreign minister nearly verbatim, but this relative newcomer to Facebook, who had no friends and no following, was also going viral in multiple languages.

If you’ll recall, Russian foreign minister Lavrov’s remarks contained an oddly specific assertion, that Ukrainian nationalist radicals were not only threatening violence against Jews but were also planning to “restrict or punish the use of Russian” and that the millions of Russians living in Crimea were outraged by this. While Jews comprise a small fraction of the Crimean population, 77 percent of Crimeans report Russian as their native language. I didn’t think much of Lavrov’s remarks to the UN, until I dug deeper into the massive spike of “mixed” news we had seen in our study of fake news on Twitter during the Crimean annexation.

The most popular mixed news story circulating on Twitter during the annexation claimed that Jews in eastern Ukraine had been given leaflets ordering them to register as Jews or face deportation. The second most popular story claimed the Ukrainian government had “introduced a law abolishing the use of languages other than Ukrainian in official circumstances.” Together, the Crimean stories supported Lavrov’s narrative and accounted for the lion’s share of mixed news stories in Twitter’s recorded history, exceeding every other spike in verified mixed news by a factor of four. The amount of bot activity and the number of unique accounts spreading the disinformation were also statistically significantly higher for the Crimean mixed news stories than for all other verified mixed political news. In social media data, outliers like these typically signal a coordinated attempt to distort reality, an orchestrated effort to influence human thinking and behavior. With Russia claiming vociferously that Crimea desired accession, and with the facts on the ground being distorted by fake news, the Obama Doctrine in response to the annexation stopped short of intervening and imposed economic sanctions instead. And today Crimea is part of Russia.

As dramatic as the Crimean disinformation campaign was, the social and economic impact of social media on our lives far outstrips any single geopolitical event. This same machinery has a hand in business, in politics, and frankly in everything, from the troubling rise of fake news to the rise and fall of the stock market, from our opinions about politics to what products we buy, who we vote for, and even who we love.

The Hype Machine

Every minute of every day, our planet now pulses with trillions of digital social signals, bombarding us with streams of status updates, news stories, tweets, pokes, posts, referrals, advertisements, notifications, shares, check-ins, and ratings from peers in our social networks, news media, advertisers, and the crowd. These signals are delivered to our always-on mobile devices through platforms like Facebook, Snapchat, Instagram, YouTube, and Twitter, and they are routed through the human social network by algorithms designed to optimize our connections, accelerate our interactions, and maximize our engagement with tailored streams of content. But at the same time, these signals are much more transformative—they are hypersocializing our society, scaling mass persuasion, and creating a tyranny of trends. They do this by injecting the influence of our peers into our daily decisions, curating population-scale behavior change, and enforcing an attention economy. I call this trifecta of hypersocialization, personalized mass persuasion, and the tyranny of trends the New Social Age.

The striking thing about the New Social Age is that fifteen years ago this cacophony of digital social signals didn’t even exist. Fifteen short years ago, all we had to facilitate our digital connections was the phone, the fax machine, and email. Today, as more and more new social technologies come online, we know less and less about how they are changing us. Why does fake news spread so much faster than the truth online? How did one false tweet wipe out $140 billion in stock market value in minutes? How did Facebook change the 2012 presidential election by tweaking one algorithm? Did Russian social media manipulation flip the 2016 U.S. presidential election? When joggers in Venice, Italy, post their runs to social media, do joggers in Venice, California, run faster? These questions contemplate the disruptive power of social media. By answering them, we can better understand how the Hype Machine impacts our world.

The Hype Machine has created a radical interdependence among us, shaping our thoughts, opinions, and behaviors. This interdependence is enabled by digital networks, like Facebook and Twitter, and guided by machine intelligence, like newsfeed and friend-suggestion algorithms. Together they are remaking the evolution of the human social network and the flow of information through it. These digital networks expose the controls of the Hype Machine to nation-states, businesses, and individuals eager to steer the global conversation toward their ends, to mold public opinion, and ultimately to change what we do. The design of this machine, and how we use it, are reshaping our organizations and our lives. And the Hype Machine is even more relevant today than it was before the COVID-19 pandemic pushed the world onto social media en masse.

By now we’ve all heard the cacophony of naysayers declaring that the sky is falling as new social technologies disrupt our democracies, our economies, and our public health. We’ve seen an explosion of fake news, hate speech, market-destroying false tweets, genocidal violence against minority groups, resurgent disease outbreaks, foreign interventions in democratic elections, and dramatic breaches of privacy. Scandal after scandal has rocked social media giants like Facebook, Twitter, and Instagram in what seems like a backlash from which they can never recover.

But when the social media revolution began, the world’s social platforms had an idealistic vision of connecting our world. They planned to give everyone free access to the information, knowledge, and resources they needed to experience intellectual freedom, social and economic opportunity, better health, job mobility, and meaningful social connections. They were going to fight oppression, loneliness, inequality, poverty, and disease. Today, they’ve seemingly exacerbated the very ills they set out to alleviate.

One thing I’ve learned, from twenty years researching and working with social media, is that these technologies hold the potential for exceptional promise and tremendous peril—and neither the promise nor the peril is guaranteed. Social media could deliver an incredible wave of productivity, innovation, social welfare, democratization, equality, health, positivity, unity, and progress. At the same time, it can and, if left unchecked, will deliver death blows to our democracies, our economies, and our public health. Today we are at a crossroads of these realities.

The argument of this book is that we can achieve the promise of social media while avoiding the peril. To do so, we must step out of our tendency to armchair-theorize about how social media affects us and develop a rigorous scientific understanding of how it works. By looking under the hood at how the Hype Machine operates and employing science to decipher its impact, we can collectively steer this ship away from the impending rocks and into calmer waters.

Unfortunately, our understanding and our progress have been impeded by the hype surrounding the Hype Machine. We’ve been overwhelmed by a tidal wave of books, documentaries, and studies of one-off events designed for media attention but lacking rigor and generalizability. The hype is not helpful because it clouds our vision of what we actually know (and don’t know) from the scientific evidence on how social media affects us.

While our discourse has been shrouded in sensational hysteria, the three primary stakeholders at the center of the controversy—the platforms, the politicians, and the people—have all been pointing their fingers at each other. Social media platforms blame our ills on a lack of regulation. Governments blame the platforms for turning a blind eye to the weaponization of their technology. And the people blame their governments and the platforms for inaction. But the truth is, we’ve all been asleep at the switch. In the end, each of us must take responsibility for the part we are playing in the Hype Machine’s current direction.

Not only are we all partly to blame, but we are all partly responsible for what happens next. As Mark Zuckerberg himself has noted, governments will need to adopt sensible, well-informed regulations. The platforms will need to change their policies and their design. And for the sake of ourselves and our children, we will all need to be more responsible in how we use social media in our digital town square. There is no silver bullet for the mess we find ourselves in, but there are solutions.

Achieving the promise of the New Social Age, while avoiding its peril, will require all of us—social media executives, lawmakers, and ordinary citizens—to think carefully about how we approach our new social order. As a society, we will need to utilize the four levers available to us: the money (or financial incentives) created by their business models, the code that governs social platforms, the norms we develop in using these systems, and the laws we write to regulate their market failures. Along the way, we will need to design scientific solutions that balance privacy, free speech, misinformation, innovation, and democracy. This is, no doubt, a monumental responsibility. But considering the overwhelming influence the Hype Machine has on our lives, it is a responsibly we cannot abdicate.

Who Am I?

I’m a scientist, entrepreneur, and investor—in that order. First and foremost, I’m a scientist. I’m a professor at the Massachusetts Institute of Technology (MIT), where I direct the Initiative on the Digital Economy and the Social Analytics Lab, where we study the social technologies that make up the Hype Machine. I earned my PhD at MIT and completed my master’s degrees at the London School of Economics and at Harvard. I’m an applied econometrician by training and have studied sociology, social psychology, and most of the curriculum in the MIT PhD program in economics. I’m a data nerd. I analyze large-scale social media data for a living, to try to make sense of how information and behavior diffuse through social media and society. My real expertise, though, is in graph theory and graph data. In other words, I study things that are connected in complex network structures, whether people in social networks or companies in buyer-supplier relationships.

How did I stumble into this line of work? In the fall of 2001, while Mark Zuckerberg was still in high school at Phillips Exeter Academy, three years before he founded Facebook at Harvard, I was a PhD student down the street at MIT, sitting in the reading room at Dewey Library studying for two very different classes: Econometrics I, taught by the world-renowned statistician Jerry Hausman, and The Sociology of Strategy, taught by the then-rising-star sociologist Ezra Zuckerman, who is now the dean of faculty at MIT’s Sloan School of Management. Ezra’s class was heavily focused on social networks, while Jerry’s class introduced us to “BLUE” estimators—the theory of what generates the best linear unbiased statistical models.

I had my statistics textbook in one hand and a stack of papers on networks in the other. As I read the statistics text, I saw that it repeated one main assumption of classical statistics over and over again—the assumption that all the observations in the data we were analyzing (the people, firms, or countries) were “independent and identically distributed (or IID).” In other words, our assumption was that none of the people in our data should be connected to each other in any systematic way. As I read the networks papers, however, I kept seeing diagrams of the complex interconnections between people. On the one hand, we were assuming everything was independent. On the other hand, the reality we live in is one of staggering interdependence.

I realized that a great deal of what we thought was unexplainable (the variance in the “independent” models) might be explained by how we are connected to one another, how information and knowledge ebb and flow between us, and how our peers’ behaviors and opinions affect our own. In 2001 there were no large digital social networking sites, but we did have lots of digital network connections through email, instant messaging, and texting. In that moment, sitting in Dewey Library, I had an epiphany: Digital social networking was going to turbocharge how information, behavior, economic opportunity, and political ideology flowed between people. It was going to transform society as we knew it and affect everything from business to politics to public health.

I remember running to the nearest Pine terminal (a computer program for sending email) and sending an email to my PhD adviser, Erik Brynjolfsson, to request a meeting. I met Erik the next day and explained that I wanted to focus my PhD dissertation on digital social networks. I told him that I thought these networks were going to be the next big thing in personal computing and that they were going to transform society. Now, Erik didn’t study social networks, and he hadn’t ever formally thought about graph theory. He was busy trailblazing research on the impact of information technology on firm productivity and economic growth. Social networks were not on his radar. To his credit, though, he humored me. He said, “I don’t really know much about networks, but you seem pretty excited about it, so we’ll figure it out together.” I don’t know if he was thinking to himself This is a phase, it will pass. PhD students typically have hundreds of ideas that go nowhere before they land on one that works. But he was supportive nonetheless, and I wrote my PhD thesis on how information flows through digital social networks. As it turned out, social networking wasn’t a phase, and it didn’t pass. Friendster was founded in 2002, MySpace in 2003, Facebook in 2004, Twitter in 2006, WhatsApp in 2009, Instagram in 2010, WeChat in 2011, and TikTok in 2012. The New Social Age was born, and I’ve been studying it ever since.

My scientific work is firmly rooted both in my deep admiration for technology and in a healthy skepticism about how it is put to use. I’m convinced we are witnessing a new era of human evolution, one in which mass automated, digitized socialization will change the way we interact, communicate, perceive our world, decide, and act. Online social networks (e.g., Facebook), microblogging (e.g., Twitter), instant messaging (e.g., WhatsApp), and collaborative knowledge production and news aggregation technologies (e.g., Wikipedia and Reddit) have fundamentally altered the way information is produced, shared, consumed, utilized, and valued. Such changes have profound implications for many of our social, political, and economic organizations, from the productivity of knowledge workers to consumer demand patterns, and from election campaigns to public health programs and mass protests.

New technologies and new modes of communication not only change the production and dissemination of information but also simultaneously record information about human interaction with incredible precision and detail. In an article published in Science in 2009, my colleagues and I argued that these new technologies and modes of communication not only change the creation and dissemination of information but make possible the development of a new field of “computational social science,” which aims to improve our understanding of the macrolevel consequences of microlevel human interaction—a long-standing “holy grail” in sociology, economics, and other disciplines. These changes are enabling new scientific studies of human behavior at population scale, and revealing new interventions that could dramatically improve the way we deal with conflict, commerce, and health.

In addition to my scientific work, I have also been an active entrepreneur and chief scientist of multiple companies, with one foot in academia and the other at the forefront of the entrepreneurial development of these new technologies. I was the chief scientist at SocialAmp, one of the first social commerce analytics companies (until its sale to Merkle in 2012) and at Humin, a social platform that The Wall Street Journal called the first “Social Operating System” (until its sale to Tinder in 2016). I have worked directly with senior executives at Facebook, Yahoo!, Twitter, LinkedIn, Snapchat, WeChat, Spotify, Airbnb, SAP, Microsoft, Walmart, and The New York Times.

Along with my longtime friend Paul Falzone, I’m a founding partner at Manifest Capital, an investment firm that helps young companies grow into the Hype Machine. From this perch, I evaluate hundreds of companies a year and get to look around the corner at what’s next. These experiences have forced me to think deeply about the business models, technologies, and machine intelligence that drive the social economy. As a scientist, entrepreneur, and investor, I have peered into the Hype Machine up close, studied its inner workings, and participated in its development. These three perspectives are always with me, which I suspect will be obvious to you as you read this book.

As a scientist, I am obsessed with rigor. I attempt at every turn to avoid making claims I cannot prove. As a result, there will be moments in this book when I present compelling evidence but stop short of making a bold claim. I will make arguments with a healthy dose of caveats along with them. The unfortunate truth is that we don’t have all the answers yet, and the answers we do have are not simple. That is part of the challenge. While science has advanced tremendously around social media and how it is affecting us, it remains nascent and at times is constrained by the platforms’ stranglehold on data. We don’t know enough about the spread of fake news, election manipulation, filter bubbles, and digital political polarization, because the research has not yet been done. But it needs to be done. So advocating for that research will be a theme of this book.

As an entrepreneur, I appreciate the difficulty of actually doing. Innovators face impossible dilemmas. Building a successful business is difficult. Building a global platform like the companies I will be discussing is next to impossible. I respect what it took to build Facebook, Twitter, LinkedIn, and the rest. I realize that some eventualities could not have been foreseen in the decisions that were made in early days. But I also know that when faced with certain truths, we have a moral duty to act. More needs to be done about the negative consequences of the Hype Machine today. I believe the true leaders of the New Social Age will be those who make the hard decisions to put social welfare above shareholder value—or perhaps those who realize that, in the long run, these goals are aligned.

As an investor, I try to distinguish the forest from the trees. When you are building a business, you are singularly committed to the survival and growth of that business. But as an investor, you also see the marketplace as a landscape of renewal. As Steve Jobs said at the Stanford University commencement in 2005, “Death is very likely the single best invention of Life. It is Life’s change agent. It clears out the old to make way for the new. Right now the new is you, but someday not too long from now, you will gradually become the old and be cleared away.” Such is also the turmoil of the marketplace. Friendster gave way to MySpace, which gave way to Facebook. Today WeChat does in one app what Facebook, WhatsApp, Messenger, Venmo, Grubhub, Amazon, Uber, Apple Pay, and many others do individually. The ongoing success of none is set in stone. The future of the New Social Age will be forged by the choices we make as entrepreneurs, investors, regulators, consumers, and citizens. I would argue that the most consequential decisions of the New Social Age are yet to come.

My Objective

My goals in this book are to describe the science of how the Hype Machine works and to explore how it affects our politics, our businesses, and our relationships; to explore the consequences of the Hype Machine for our society, both positive and negative; and to discuss how we can—through company policy, social norms, government regulation, and more advanced software code—achieve its promise while avoiding its peril.

I’ll begin by considering fake news and the weaponization of misinformation through the Hype Machine, tracking how the design of platforms like Facebook and Twitter incentivize and enable the spread of misinformation (Chapter 2). Did Russian election interference change the results of the 2016 U.S. presidential election? What should we do to stop the scourge of fake news in the 2020 election and beyond? Stay tuned.

Along the way, I’ll examine why the Hype Machine’s rise was so meteoric and why we were so susceptible to it, as individuals and as a society. I will describe the anatomy of the Hype Machine—the underlying trifecta of social technologies at the center of this inflection point in human history—and consider, in depth, the four levers through which we can shape our technological future: money, code, norms, and laws (Chapter 3). I will describe the neurological (Chapter 4) and economic (Chapter 5) forces that have “wired us into” the Hype Machine. Understanding these neurological and economic hooks will help us answer important questions about the New Social Age from a business perspective, such as why Facebook beat MySpace in the market for social networks. They will also shed light on more fundamental questions, like how the Hype Machine will impact human evolution.

I will then discuss three key societal transformations driven by the Hype Machine that are disrupting business, democracy, and public health: the personalization of mass persuasion (Chapter 6), the hypersocialization of society (Chapters 7 and 8), and the advent of the attention economy (Chapter 9). In so doing, I’ll look under the hood at the mechanics of the Hype Machine, dig into the science of online peer effects, and explore how our new radical interdependence is changing the products we buy, the people we vote for, and even who we meet and fall in love with.

After looking under the hood, I’ll zoom out to consider the societal implications of the Hype Machine and the three trends it perpetuates—for example, its implications for what’s known as the “wisdom of crowds.” Our ability to harness the wisdom of crowds and collective intelligence rests on three basic pillars: independence, diversity, and equality. The problem is that the Hype Machine erodes all three of these pillars and turns wisdom into madness. I’ll discuss how we can recapture the wisdom of crowds in Chapter 10. Next, I’ll remind us why we invented the Hype Machine in the first place by describing its positive potential in creating an incredible tsunami of productivity, innovation, social welfare, democratization, equality, caring, positivity, unity, and social progress. At the same time, I’ll discuss why the source of social media’s positive potential is also the source of its peril and how this complicates how we must adapt to it (Chapter 11).

Finally, I’ll explore how we must adapt—how business policy, government regulation, social norms, and technology design can steer our economy and our society toward a more productive future (Chapter 12). Should we break up Facebook? How should we craft privacy legislation? Is the Hype Machine a publisher or a user-generated platform that should not be responsible for the content users post? What does that mean for free speech and hate speech? You’ll be surprised by some of the answers.

At a Crossroads

The last three years have seen front-page stories about Facebook, Twitter, YouTube, and the rest of social media’s lack of transparency; their contribution to political polarization; their promotion of hate speech, racism, and the degradation of discourse; their role in the spread of fake news; and their potentially corrosive impact on our democracies and our elections.

Lawmakers have advocated regulation. Multiple U.S. congressional committees are investigating the role of Facebook and the rest of the Hype Machine in Russian election interference and the spread of misinformation online. The Cambridge Analytica controversy, in which a political consultancy used stolen Facebook data on 87 million Americans to target political ads, forced Mark Zuckerberg to testify in front of the U.S. Congress and the European Parliament as lawmakers debate what to do about the Hype Machine’s power of mass persuasion, use of personal data, and lack of control over misinformation. Sen. John Kennedy began his questioning of Zuckerberg in the Senate with an ominous opening statement: “I don’t want to have to regulate Facebook,” he said. “But, by God, I will.”

Advertisers have also pressured the platforms to clean up their act. In 2017, Marc Pritchard, Procter & Gamble’s chief brand officer, went on a public tirade about the lack of transparency in digital advertising on platforms like Google and Facebook and advertising appearing next to fake or offensive content. Then he put his money where his mouth was and cut P&G’s digital advertising budget by $200 million. In 2018, Unilever followed suit, cutting its digital advertising by nearly 30 percent in an effort to clean up the Hype Machine’s advertising ecosystem. And these weren’t just hapless public protests. In fact, P&G reported a 7.5 percent increase in organic sales growth in 2019, while cutting its online marketing budget by 6 percent. Unilever posted a 3.8 percent gain in organic sales in the same period. Understanding how they did it requires an understanding of the Hype Machine.

In the Hype Machine, everyone is a digital marketer, whether we’re fighting for ideas or for consumer dollars. A candidate in the presidential election, trying to persuade voters to his or her side; BMW, trying to persuade people to buy the new 3 Series; the small-business owner trying to grow sales; Vladimir Putin’s Internet Research Agency, trying to sow discord through misdirection—in the Hype Machine, they are all digital marketers. They are all trying to optimize the same persuasion strategies to achieve their goals. That’s why it’s so important for us to wear different hats on this journey—the regulator, the marketer, and the concerned citizen. Frequently, I will ask you to put on your digital marketer’s hat to understand the toolbox from their perspective. To make sense of what we see in our feeds every day, we’ll need to understand what they’re doing and why.

Every nation on earth has become attentive to the role of social media in society today. Regulators around the world are debating what to do about the impact of the Hype Machine on elections, business trends, competition, privacy, and fake news. Business leaders are trying to figure out how to self-regulate, through platform policies, algorithm design, software code, and alternative business models. And all of us, as parents and individuals, family and friends, are thinking about how the Hype Machine affects our lives and the lives of our children, from the impact it has on our friendships and businesses, to how we socialize and behave, to the rise of loneliness in our society. The decisions we make today in how we design, deploy, use, and regulate social media will have far-reaching consequences for years to come.

The science suggests that while social media can help foster a transparent, democratic, egalitarian society, it can also be used to erect a polarized, authoritarian police state. Today we are at a crossroads, caught between the promise and the peril, even as the system’s design is being debated worldwide.

Tectonic shifts are a regular occurrence for social media. Keeping up with the daily changes is next to impossible. Instead, I hope to offer a lasting framework to guide our thinking about the social economy. What I have learned, over twenty years of research, has taught me some general principles about how the Hype Machine works, how information and behaviors spread, how interventions in social media change behavior, and how managers, policy makers, and individuals can interface with the Hype Machine more effectively. A rigorous investigation of these fundamentals entails a harrowing intellectual journey, with plenty of plot twists and unexpected turns along the way. It’s a journey I intend to take you on in this book. And there is no better place to start than with how social media has brought us to the precipice of what some have called the “end of reality.”

CHAPTER 2

The End of Reality

Under normal circumstances the liar is defeated by reality, for which there is no substitute; no matter how large the tissue of falsehood that an experienced liar has to offer, it will never be large enough, even if he enlists the help of computers, to cover the immensity of factuality.

—HANNAH ARENDT

The market opened quietly on Wall Street on April 23, 2013. As traders sipped lattes on an unseasonably chilly morning, stocks made modest gains from the opening bell to the lunch hour. But as everyone broke for lunch, the Associated Press broke a story on Twitter that changed the market’s mood. Phones chirped at eateries in New York, Washington, and the world over as the news was retweeted again and again, creating an information cascade that swept through the Hype Machine in seconds. The tweet, which appeared at 1:07 P.M. eastern time in the United States, simply read “Breaking: Two Explosions in the White House and Barack Obama is injured.” It was retweeted over four thousand times in five minutes, which would have informed hundreds of thousands if not millions of people of the attack on the White House.

You could almost hear the iced teas and Arnold Palmers being snorted back into the glasses of those monitoring the social network. The news was shocking. Putting aside fence jumpers, who are usually tackled on sight, there have only ever been four breaches of White House security that made it to the building. So two explosions injuring the president inside the White House was big news.

The market stuttered. Then it skipped a beat. If only individual retail investors had been influenced by the news, the financial impact might have been contained. But the Hype Machine doesn’t exist in isolation. It’s coupled to systems that sense, mine, analyze, and trade on sentiment expressed on social media in real time. Dataminr, RavenPack, and other companies are constantly sifting through social media data to find the signal in the noise. When they find that signal, they seize on it and relay instructions to their institutional clients to buy or sell ahead of market trends. On this particular afternoon, the sentiment wasn’t good, and the data miners issued sell recommendations that triggered automated-trading algorithms to unload their stocks. When they did, the Dow fell instantly by nearly 200 points, wiping out $139 billion in equity value in seconds.

But the news wasn’t true. The White House was calm, and the president was fine. The tweet was fake news propagated by Syrian hackers who had infiltrated the AP’s Twitter handle. There was a terrorist attack that day—just not at 1600 Pennsylvania Avenue. The attack happened on Twitter, and the casualties were felt on Wall Street. The market rebounded, but real people lost real money as their buy and sell orders were honored. Those who were late to the fire sale lost their shirts. The “Hack Crash” of 2013 highlights the fragility of the socio-technical systems we’ve wired into the Hype Machine. When news cascades through the network, it’s hard to stop and harder to verify with enough time to prevent panic. When the news is false, it can wreak havoc on financial systems, health systems, and democratic institutions, creating real consequences from virtual falsity.

Here’s another example. When Hurricane Harvey hit southern Texas in the summer of 2017, the flooding displaced thousands and halted production at several oil refineries in the southern United States. News of gas shortages spread quickly on Twitter and Facebook as drivers posted pictures of long lines at gas stations, with makeshift signs saying they were out of gas. A panic ensued, and drivers in the region rushed to stockpile gas as if the world were ending, creating a run on fuel in Austin, Dallas, Houston, and San Antonio.

But as the authorities later revealed, there was no gas shortage. It was false news spread through social media, then picked up and reported by broadcast media. As we later learned, there was plenty of gas to go around. The refinery and highway closures only slowed deliveries. Had everyone stuck to their normal consumption, the distribution system could have handled the disruption, and there would have been no shortage. The panic and the subsequent run on gas, however, ensured there was a shortage, wrought by manic stockpiling, driven by social media.

The signature of a fake news crisis was repeating itself: false information was spreading faster than the truth, misdirecting real behaviors with real impact. Such fake news can have dramatic consequences for businesses, democracies, and public health. And although it has been around for centuries, the speed and scale with which it spreads through the Hype Machine creates a fake news crisis on steroids.

These vignettes highlight a systematic pattern in the spread of fake news (one that bears out in large-scale studies of the phenomenon that I’ll discuss shortly). When fake news isn’t completely fabricated, it typically distorts real-world information by tweaking or contorting it, mixing it with true information, and highlighting its most sensational and emotional elements. It then scales rapidly on social media and spreads faster than our ability to verify or debunk it. Once it spreads, it’s hard to put back in the bottle and even harder to clean up, even with a healthy dose of the truth.

The Syrian hack that crashed the stock market in 2013 is a case study in the economic consequences of fake news. You’ve probably heard similar stories. The fact-checking site Snopes keeps a list of “hot 50” rumors that gets updated with alarming regularity. There was a 2008 rumor that United Airlines was filing for bankruptcy, a 2017 report that Starbucks would give out free Frappuccinos to undocumented workers, and President Trump’s tweets, in March 2018, falsely claiming that Amazon was evading taxes, which sent shares of the company plummeting to their worst monthly performance in two years. But is there a systematic effect of fake news on businesses? On stock prices? Before we can understand the full implications of fake news, we need to detour through the story of a D-list actress named Kamilla Bjorlin.

An Actress’s Rise to Fake News Infamy

Kamilla Bjorlin had been an actress her whole life. Since age seven, she had played bit parts in movies like Raising Helen, with Kate Hudson, and The Princess Diaries 2, with Anne Hathaway. She never really became a star, but narrative fiction was in her blood. So in 2011 she struck out on a different career path, forming a public relations and social media firm called Lidingo Holdings, specializing in investor relations and promotional research for public companies, including many biopharmaceutical firms. In fact, as alleged in a 2014 SEC complaint, Lidingo was a fake news factory engaged in “pump and dump” stock promotion schemes that posted fake articles to crowdsourced investor intelligence websites like Seeking Alpha, The Street, Yahoo! Finance, Forbes, and Investing.com, for the express purpose of moving stock prices.

Writers employed by Lidingo created pseudonyms, like the Swiss Trader, Amy Baldwin, and the Trading Maven; claimed to have MBAs and degrees in physics; and wrote fake stories praising the growth or stability of the companies they were paid to promote. But they never disclosed their financial relationship with their clients, which was why they ran afoul of the SEC. Between 2011 and 2014, Lidingo allegedly published over four hundred fake news articles and earned cash and equity payments of more than $1 million. As part of its crackdown on fake financial news, a separate SEC complaint against a similar firm called DreamTeam described an operation that provided “social media relations, marketing and branding services” to publicly traded companies and “leveraged its extensive online social network to maximize exposure” to fake news designed to inflate its clients’ stock prices.

One company that Lidingo wrote fake news for was Galena Biopharma. In the summer of 2013, before Lidingo was hired, Galena’s stock price was hovering around $2 per share. Between August 2013 and February 2014, Lidingo published twelve fake news stories about Galena, claiming, for example, that the company would make “a good long-term growth investment” because it had “a promising pipeline for revenue growth” from “three strong drugs in its pipeline.”

After the first two fake news articles were published, the company issued 17.5 million new shares in a secondary equity offering (SEO) worth $32.6 million to the firm. Then five more fake news articles were published, and the stock price skyrocketed. At a November 22 board meeting, the company granted hundreds of thousands of new stock options to the CEO, COO, CMO, and each of six directors. The stock price continued to rise. In fact, it rose over 925 percent between August 2013 and January 2014 (Figure 2.1). At a January 17 board meeting, then-CEO Mark Ahn declared insiders could trade the company’s stock immediately, and starting the next day, they did, unloading $16 million worth in four weeks.

Figure 2.1 The Galena Biopharma stock price from April 2013 to May 2014, annotated with the dates when secondary equity offerings and insider selling occurred, when fake news and exposé news articles were published, and when stock options were granted.

We all know that information moves markets, but the financial impact of fake news is not immediately clear. The Galena Biopharma story directly ties the two together. But of course, Galena is a one-off. Is there any systematic, generalizable evidence of the financial impact of fake news? Luckily Galena is one of over seven thousand firms that Shimon Kogan, Tobias Moskowitz, and Marina Niessner analyzed in their large-scale study of the relationship between fake news and financial markets.

Fake News and Financial Markets

Kogan, Moskowitz, and Niessner examined data from the SEC’s undercover investigation, which identified fake news articles about public companies written to manipulate stock prices. By analyzing these validated fake news stories, Kogan, Moskowitz, and Niessner were able to systematically link the dissemination of fake news to stock price movements over time. The initial data covered a small number of articles on a small set of companies involved in the SEC investigation—171 articles on 47 companies. But the researchers broadened their sample by identifying fake news using a linguistic analysis of all the articles published on Seeking Alpha from 2005 to 2015, and on Motley Fool from 2009 to 2014. The larger sample, which looked for linguistic signatures of deceptive writing, was noisier than the verified SEC sample, but it allowed the authors to examine over 350,000 articles on over 7,500 companies over ten years. They also analyzed investors’ reactions to fake news before and after the SEC publicly announced its sting operation, which drew attention to the prevalence of fake news on these sites. The findings reveal a great deal about how fake news moves markets.

In the verified SEC data, the publication of fake news was strongly correlated with increased trading volume. Abnormal trading volume rose by 37 percent over the three days following the publication of real news articles and 50 percent more following the publication of fake news articles relative to real news articles. In other words, investors reacted to fake news even more strongly than to real news. The reactions were more pronounced for smaller firms and for firms with a greater percentage of retail (as opposed to institutional) investors. Fake articles were clicked on and read more often than real articles, and trading volume increased with the number of clicks and times an article was read.

And the effect of fake news on stock prices? Fake articles had, on average, nearly three times the impact of real news articles on the daily price volatility or absolute return of the manipulated stocks in the three days after the publication of fake news, even after controlling for recent SEC filings, firm press releases, and return volatility in the days leading up to the article.

The SEC went public with its investigation of fake news in 2014, filing lawsuits against several companies and the fake news factories they worked with, including Lidingo and DreamTeam. When the public revelation of the sting operation drew investors’ attention to the fact that fake news was being published on websites like Seeking Alpha, Kogan et al. used the public revelation of the sting to examine whether increased awareness of fake news eroded consumers’ trust in real news. Not surprisingly, fake news had a bigger impact on trading volume and price volatility before the SEC announcement than after it. But investors were also less moved by real news once the SEC drew attention to the existence of fake news, demonstrating the potential for fake news to erode the public’s trust in news altogether.

If fake news can disrupt markets, it can affect everyone in society, whether we read and share it or not. More important (as I will show when we discuss the troubling rise of “deepfakes” later in this chapter), if fake news can successfully disrupt markets, it creates a political incentive for economic terrorism. And as we saw in Crimea, the weaponization of misinformation is one of the most insidious threats to democracy in the Information Age. The most egregious example to date is Russia’s interference in American democracy in 2016.

The Political Weaponization of Misinformation

When the Mueller report was released in April 2019, pundits, politicians, and the press pored over it, looking for the juiciest bits to support their positions and to entice readers and viewers with salacious headlines. Most of them skipped over volume one and turned directly to volume two, which addressed President Trump’s alleged obstruction of the FBI’s Russia investigation. But when I read the Mueller report, I was shocked not by the juicy political potential of volume two but by the clear geopolitical reality described in volume one: Russia, an active foreign adversary, had systematically used the Hype Machine to attack American democracy and manipulate the results of the 2016 U.S. presidential election. It was one of the most comprehensive weaponizations of misinformation the world has ever seen.

Two studies commissioned by the U.S. Senate Intelligence Committee, one led by New Knowledge and the other by John Kelly, founder and CEO of Graphika, detail the extent of Russian misinformation campaigns targeting hundreds of millions of U.S. citizens in 2016. When I had lunch with John and Graphika’s chief innovation officer, Camille François, at the Union Square Café in Manhattan in early 2019, they told me the attack on our democracy was even more sophisticated than the media was giving it credit for. They were both deeply concerned about what they had uncovered. While their report is public, the look on their faces when discussing it was revealing. These two highly regarded experts were worried—and when the experts are worried, we should be too.

Russia’s attack was well planned. The Internet Research Agency created fake accounts on Facebook, Twitter, Instagram, YouTube, Google, Tumblr, SoundCloud, Meetup, and other social media sites months, sometimes years, in advance. They amassed a following, coordinated with other accounts, rooted themselves in real online communities, and gained the trust of their followers. Then they created fake news intended to suppress voting and to change our vote choices, in large part toward Republican candidate Donald Trump and away from Democratic candidate Hillary Clinton. The fake news included memes about Black Lives Matter, the mistreatment of American veterans, the Second Amendment and gun control, the supposed rise of sharia law in the United States, and well-known falsehoods like the accusation that Hillary Clinton was running a child sex ring out of the basement of a pizza shop in Washington, D.C. (known as “PizzaGate”). They spread these memes through organic sharing and paid promotion to boost their reach on social media.

On Twitter, they established a smaller number of source accounts, which posted fake content, and close to four thousand sharing accounts, which amplified the content through retweets and trending hashtags. The source accounts were manually controlled, while the sharing accounts were more often “cyborg” accounts that were partly automated and partly manual. Automated accounts, run by software robots, or “bots,” tweeted and retweeted with greater frequency at prespecified times. Software doesn’t get tired or need bathroom breaks, so the bot army was always on, propagating fake news and engaging the American electorate around the clock.

A lot has been written about the “Great Hack” of 2016. By now we’ve learned that Russian disinformation* was broad based and sophisticated. But did it actually change the outcome of the 2016 U.S. presidential election (or for that matter the outcome of the Brexit vote in the U.K. or of elections in Brazil, Sweden, and India)? To assess whether it flipped the U.S. election result, we have to answer two additional questions: Was the reach, scope, and targeting of Russian interference sufficient to change the result? And if so, did it successfully change people’s voting behavior enough to accomplish that goal?

The Reach, Scope, and Targeting of Russian Election Interference

During the 2016 election, Russian fake news spread to at least 126 million people on Facebook and garnered at least 76 million likes, comments, and other reactions there. It reached at least 20 million people on Instagram and was even more effective there, amassing at least 187 million likes, comments, and other reactions. Russia sent at least 10 million tweets from accounts that had more than 6 million followers on Twitter. I say “at least” because what we have uncovered so far may just be the tip of the iceberg. Analysis shows, for example, that the twenty most engaging false election stories on Facebook (whether from Russia or elsewhere) in the three months before the election were shared more and received more comments and reactions than the twenty most engaging true election stories. The Hype Machine became a clear conduit for misinformation, with one study estimating that 42 percent of visits to fake news websites (and only 10 percent of visits to top news websites) came through social media.

As astonishing as those numbers sound, the scale of fake news in 2016 was considerably smaller than that of real news. For example, in their nationally representative sample of web browsers, Andrew Guess, Brendan Nyhan, and Jason Reifler found that while 44 percent of Americans visited fake news websites in the weeks before the election, these visits comprised only 6 percent of their visits to real news websites. Similarly, David Lazer and his colleagues found that only 5 percent of registered voters’ total exposures to political URLs on Twitter during the 2016 election were from fake news sources. Hunt Allcott and Matthew Gentzkow estimated the average American saw “one or several” fake news stories in the months before the election.

If these numbers seem small, it’s worth noting some eccentricities in the way these data were collected. News is only considered “fake” in Allcott and Gentzkow’s study if it is one of 156 verified fake news stories. The other two studies analyze restrictive lists of approximately 300 fake news websites. For example, Guess et al. exclude Breitbart, InfoWars, and all of YouTube when characterizing “fake news sources.” So the 44 percent of voting-age Americans who visited at least one of their restricted list of fake news websites in the final weeks before the election does not count visitors to these popular sources of fake news. In other words, 110 million voting-age Americans visited a narrow list of fake news websites that excluded Breitbart, InfoWars, and YouTube. Our best estimates put the full number of voting-age Americans exposed to fake news during the 2016 election at between 110 million and 130 million. So whether voters’ exposure to fake news was “small-scale” is still hotly debated.

The distributions were skewed by a few viral hits, and a smaller percentage of voters saw the largest concentration of fake news. Guess et al. found that the 20 percent of Americans with the most conservative news diets were responsible for 62 percent of visits to fake news websites and that Americans over sixty were much more likely to consume fake news. On Twitter, Grinberg et al. found that 1 percent of registered voters consumed 80 percent of the fake news and that 0.1 percent accounted for 80 percent of “shares” from fake news websites. In a second, representative online survey of 3,500 Facebook users, Guess, Nyhan, and Reifler found that only 10 percent of respondents shared fake news and that they were highly concentrated among Americans over sixty-five. This level of concentration, which (as I will describe in Chapter 9) is typical of the Hype Machine, might make you skeptical of the power of fake news to influence a broad swath of society. But several caveats should accompany such skepticism.

We know that the “superspreaders” and “superconsumers” of fake news, who drive the concentration in these samples, are mostly bots. Grinberg et al. note, for example, that their median supersharer tweeted a whopping seventy-one times a day on average, with an average of 22 percent of their tweets containing fake news URLs, while their median panel member tweeted only 0.1 times a day. The researchers concluded that “many of these [superspreader and superconsumer] accounts were cyborgs.” Excluding bots, Grinberg et al. found that members of their panel averaged 204 exposures to fake news during the last thirty days of the 2016 campaign, which amounts to seven exposures a day. Assuming only 5 percent of these exposures were seen, they estimate that the average human saw one fake news story every three days leading up to the election.

While some argue fake news isn’t important because it constitutes a small part of the average citizen’s overall media diet, it’s not clear that its volume is proportional to its impact. Fake news is typically sensational and may therefore be more striking and persuasive than everyday news stories. Singular news stories, like Willie Horton, Mike Dukakis riding a tank, and Howard Dean’s guttural scream are widely regarded as tipping points in their respective elections. Whether fake news stories behave similarly is an empirical question that remains unanswered. Furthermore, fake news doesn’t live just on social media. It is frequently picked up and repeated by broadcast media and public figures, creating a feedback loop that amplifies its reach beyond social media alone.

The 2000 U.S. presidential election was decided by 537 votes in one key swing state—Florida. Russia’s 2016 misinformation campaign targeted voters in swing states like Florida, Ohio, Pennsylvania, and Michigan. My colleagues at the Oxford Internet Institute analyzed over 22 million tweets containing political hashtags shared in the week before the election. They geolocated a third of these tweets, tying sharers and recipients to the states in which they resided. When they analyzed the geographic distribution of Russian misinformation around the country, they found that “the proportion of misinformation was twice that of the content from experts and the candidates themselves.” When they calculated whether a state had more or less Russian fake news, they found that 12 of 16 swing states were above the average. They concluded that Russian fake news was “surprisingly concentrated in swing states, even considering the amount of political conversation occurring in the state.” Although more than 135 million votes were cast in the 2016 U.S. presidential election, six swing states (New Hampshire, Minnesota, Michigan, Florida, Wisconsin, and Pennsylvania) were decided by margins of less than 2 percent, and 77,744 votes in three swing states (Wisconsin, Michigan, and Pennsylvania) effectively decided the election.

On Facebook, Twitter, and Instagram, Russian fake news targeted persuadable voters in swing states with content that was tailored to their interests using “@” mentions and hashtags to draw users to memes custom-designed for them. For example, two days before the election, supporters of the Black Lives Matter movement were drawn to voter suppression memes that encouraged them not to vote. The account @woke_blacks posted a meme to its Instagram account that read, “The excuse that a lost Black vote for Hillary is a Trump win is bs. Should you decide to sit-out the election, well done for the boycott,” while @afrokingdom_ posted that “Black people are smart enough to understand that Hillary doesn’t deserve our votes! DON’T VOTE!” New Knowledge estimated that 96 percent of the Instagram content linked to the Internet Research Agency focused on Black Lives Matter and police brutality, spreading “overt suppression narratives.”

We know that President Trump’s former campaign manager, Paul Manafort, shared polling data with the Russian political consultant Konstantin Kilimnik and that targeting persuadable voters in swing states (which is possible with such polling data) was standard operating procedure for Cambridge Analytica, the self-described “election consultancy” that used 87 million Americans’ stolen data to build predictive models of voters’ susceptibility to persuasion, and the topics and content most likely to persuade them. (I will evaluate Cambridge Analytica’s “psychographic profiling” in Chapter 9.)

If misinformation targeted a small but potentially meaningful number of persuadable voters in key swing states, was the right misinformation targeted at the right voters? Skeptics contend that such misinformation preached to the choir, because exposure was “selective”—meaning that die-hard conservatives saw pro-Trump fake news, while die-hard liberals saw pro-Clinton fake news, making it unlikely that it changed anyone’s mind. Guess et al. found 40 percent of Trump supporters and only 15 percent of Clinton supporters read pro-Trump fake news, while 11 percent of Clinton supporters and only 3 percent of Trump supporters read pro-Clinton fake news. Sixty-six percent of the most conservative voters, the farthest-right decile, visited at least one pro-Trump fake news site and read an average of 33.16 pro-Trump fake news articles.

But the argument that fake news was preaching to the choir doesn’t address voter turnout, because ideologically consistent fake news can motivate voters to turn out even if it doesn’t change their vote choices. Furthermore, while voters on the extreme right were disproportionately more likely to consume pro-Trump fake news (thus “preaching to the choir”), moderate Clinton supporters and undecideds in the middle of the political spectrum were significantly more likely to consume pro-Trump fake news than pro-Clinton fake news. Could their exposure to fake news have persuaded them to vote for Trump or to abstain? That depends on how social media manipulation affects voting.

Social Media Manipulation and Voting

Was the reach, scope, and targeting of the Russian interference sufficient to change the election result? We can’t rule it out. Although exposure to fake news was much less than exposure to real news and was concentrated among a select set of voters, it likely reached between 110 million and 130 million people. It didn’t need to affect everyone to tip the election—just a few hundred thousand persuadable voters in key swing states. Which was exactly who Russia targeted. The next big question: did it have an effect on voting? To answer that, we need to understand the science of voter turnout and vote choice.

Sadly, as I write this, only two published studies link social media exposure to voting. The first, a 61-million-person experiment conducted by Facebook during the 2010 U.S. congressional elections, found social media messages encouraging voting caused hundreds of thousands of additional verified votes to be cast. The second, a follow-up experiment by Facebook during the 2012 presidential election, replicated the findings of the first, although the “get out the vote” messages were slightly less effective, as is typical in higher-stakes presidential elections. (I will analyze both of these studies in detail in Chapter 7.) But the main takeaway, for the purposes of our discussion of the 2016 election, is that social media messaging can significantly increase voter turnout, with minimal effort. While there are only two large-scale studies of the effect of social media on voting, the substantial research on the effects of persuasive messaging on voter turnout and vote choice can help us to calibrate the likely effects of Russian interference on the 2016 election.*

With regard to vote choice, some meta-analytic reviews suggest the effects of impersonal contact (mailings, TV, and digital advertising) on vote choice in general elections are very small. Kalla and Broockman conclude, from a meta-analysis of forty-nine field experiments, that “the best estimate of the size of persuasive effects [i.e., effects of advertising on vote choice] in general elections … is zero.” But their data do not consider social media. And there is substantial uncertainty in their estimates, such as the effect of impersonal contact within two months of Election Day, which was when Russia’s attack was in full swing.

They also found significant effects of persuasive messages on vote choice in primaries, on issue-specific ballot measures, and on campaign targeting of persuadable voters. Rogers and Nickerson found, for example, that informing voters in favor of abortion rights that a candidate did not support such rights had a 3.9 percent effect on reported vote choice, suggesting the possibility that targeted, issue-specific manipulation of the type Russia deployed could have changed vote choices. The power of persuasive messaging on issue-specific ballot measures also highlights the possibility of interference being directed at large numbers of regional elections and local policy measures, which could collectively shift the political direction of a country without having to impact results at a national level. This threat is particularly insidious, because it would be much harder to detect than interference in a general election.

Moreover, social media manipulation does not have to change our vote choices to tip an election. Targeted efforts to increase or diminish voter turnout could be substantial enough to change overall election results, and recent evidence suggests targeted messaging can affect voter turnout. For example, a randomized experiment by Katherine Haenschen and Jay Jennings showed that targeted digital advertising significantly increased voter turnout among millennial voters in competitive districts. Research by Andrew Guess, Dominique Lockett, Benjamin Lyons, Jacob Montgomery, Brendan Nyhan, and Jason Reifler showed that randomized exposure to just a single misleading article increased belief in the article’s claims and increased self-reported intentions to vote. The meta-analysis by Green et al. estimated that direct mailings, combined with social pressure, generate an average increase in voter turnout of 2.9 percent, while canvassing generates an average increase of 2.5 percent, and volunteer phone banks generate an average increase of 2 percent. Dale and Strauss estimated the effect of text messages on voter turnout to be 4.1 percent, and there is evidence that personalized emails have a similar impact. The only studies of the impact of voter turnout as a result of social media messaging estimate that hundreds of thousands of additional votes were cast as a result of such messages.

So did Russian interference flip the 2016 U.S. presidential election or not? A full presidential term after the first Russian intervention and on the eve of the 2020 presidential election, during which Russia and others are continuing to interfere, we still don’t know. What’s scary is we can’t rule it out. It’s certainly possible given all the evidence. We don’t know because no one has studied it directly. Unfortunately, until we do democracies worldwide will remain vulnerable.

The 2020 U.S. Presidential Election

We should all have been alarmed when Special Counsel Robert Mueller testified that “the Russian government’s effort to interfere in our election is among the most serious … challenges to our democracy” he has ever seen. He stressed that the threat “deserves the attention of every American” because the Russians “are doing it as we sit here and they expect to do it during the next campaign.” He concluded that “much more needs to be done to protect against these intrusions, not just by the Russians but others.”

FBI director Christopher Wray is warning that “the threat just keeps escalating.” Not only have Russia’s attacks intensified in 2020, but other countries, like China and Iran, are “entertaining whether to take a page out of that book.” It is not clear what will happen to Americans’ trust in democracy if the 2020 election is a more convincing, more determined rerun of 2016.

What is clear is that the 2020 election is being targeted. In February, intelligence officials informed the U.S. House Intelligence Committee that Russia was intervening to support President Trump’s reelection. In March, the FBI informed Bernie Sanders that Russia was trying to tip the scales on his behalf. Intelligence officials have warned that Russia has adjusted their playbook toward newer, less easily detectable tactics to manipulate the 2020 election. Rather than impersonating Americans, they are now nudging American citizens to repeat misinformation to avoid social media platform rules against “inauthentic speech.” They have shifted their servers from Russia to the United States because American intelligence agencies are barred from domestic surveillance. They have infiltrated Iran’s cyberwar department, perhaps to launch attacks made to seem like they originated in Tehran.

In November 2019, Russian hackers successfully infiltrated the servers of the Ukrainian gas company Burisma, which was at the center of widely discredited allegations against Joe Biden and his son Hunter Biden, perhaps in an effort to dig up dirt to use during the general election. I would not be at all surprised to see a rerun of the Hillary Clinton email scandal emerge as a Joe Biden Burisma scandal in the fall of 2020. It wouldn’t be hard for foreign adversaries to seed false material into the American social media ecosystem, made to seem like real material from the successful Burisma hack, to create a scandal designed to derail the Biden campaign before anyone can debunk it. As we’ve seen, this is the signature of a fake news crisis: it spreads faster than it can be corrected, so it’s hard to clean up, even with a healthy dose of the truth.

The threat of election manipulation in 2020 is even higher due to the chaos caused by the coronavirus pandemic. With uncertainty around the viability of in-person voting, questions about voting by mail, and calls to delay the election, there can be no doubt that foreign actors will look to leverage the confusion caused by the coronavirus to disrupt our democratic process.

While some claim fake news is benign, during protests and confusion, amid the smoke, fire, and foreign interference, months from the most consequential election of our time, it is a real threat—not only to the election, but to the sanctity and peace of the election process. If the election were to be contested, fake news could escalate the contest, perhaps to violence. As Donald Horowitz has noted, “Concealed threats and outrages committed in secret figure prominently in pre-riot rumors. Since verification of such acts is difficult, they form the perfect content for such rumors, but difficulty of verification is not the only way in which they facilitate violence. … Rumors form an essential part of the riot process. They justify the violence that is about to occur. Their severity is often an indicator of the severity of the impending violence. Rumors narrow the options that seem available to those who join crowds and commit them to a line of action. They mobilize ordinary people to do what they would not ordinarily do. They shift the balance in a crowd toward those proposing the most extreme action. They project onto the future victims of violence the very impulses entertained by those who will victimize them. They confirm the strength and danger presented by the target group, thus facilitating violence born of fear.” Extreme vigilance is warranted during these difficult times when our vulnerability to manipulation is particularly high.

The threat of misinformation is not limited to Russia or American democracy. Digital interference threatens democracies worldwide. Carole Cadwalladr’s investigative reporting on the role of fake news in the Brexit vote and her work with Christopher Wiley to break the Cambridge Analytica scandal for The Guardian gave us a glimpse into the extent to which fake news has been weaponized around the world. Research by the Oxford Internet Institute found that one out of every three URLs with political hashtags being shared on Twitter ahead of the 2018 Swedish general election were from fake news sources. A study by the Federal University of Minas Gerais, the University of São Paulo, and the fact-checking platform Agência Lupa, which analyzed 100,000 political images circulating on 347 public WhatsApp chat groups ahead of the 2018 Brazilian national elections, found that 56 percent of the fifty most widely shared images on these chat groups were misleading, and only 8 percent were fully truthful. Microsoft estimated that 64 percent of Indians encountered fake news online ahead of Indian elections in 2019. In India, where 52 percent of people report getting news from WhatsApp, private messaging is a particularly insidious breeding ground for fake news, because people use private groups with end-to-end encryption, making it difficult to monitor or counteract the spread of falsity. In the Philippines, the spread of misinformation propagated to discredit Maria Ressa, the Filipino-American journalist working to expose corruption and a Time Person of the Year in 2018, was vast and swift. Similar to the Russian influence operation in Crimea, the misinformation campaign against Ressa mirrored the charges that were eventually brought against her in court. In June 2020, she was convicted of cyber libel and faced up to seven years in prison. The weaponization of misinformation and the spread of fake news are problems for democracies worldwide.

Fake News as Public Health Crisis

In March 2020, a deliberate misinformation campaign spread fear among the American public by propagating the false story that a nationwide quarantine to contain the coronavirus pandemic was imminent. The National Security Council had to publicly disavow the story. And that wasn’t the only fake news spreading about the virus. The Chinese government spread false conspiracy theories blaming the U.S. military for starting the pandemic. Several false coronavirus “cures” killed hundreds of people who drank chlorine or excessive alcohol to rid themselves of the virus. There was, of course, no cure or vaccine at the time. International groups, like the World Health Organization (WHO), fought coronavirus misinformation on the Hype Machine as part of their global pandemic response. My group at MIT supported the COVIDConnect fact-checking apparatus, the official WhatsApp coronavirus channel of the WHO, and studied the spread and impact of coronavirus misinformation worldwide. But we first glimpsed the destructive power of health misinformation on the Hype Machine the year before the coronavirus pandemic hit, during the measles resurgence of 2019.

Measles was declared eliminated in the United States in 2000. But while only 63 cases were reported in 2010, over 1,100 cases were reported in the first seven months of 2019, a nearly 1,800 percent increase. Measles is particularly dangerous for kids. It typically starts as a fever and rash, but in one in a thousand cases, it spreads to the brain, causing cranial swelling and convulsions, or encephalitis. In one in twenty kids, it causes pneumonia, preventing the lungs from extracting oxygen from the air and delivering it to the body. The disease took the lives of 110,000 kids worldwide this way in 2017.

Measles is one of the world’s most contagious viruses. You can catch it from droplets of air contaminated with an infected person’s cough hours after they’ve left the room. Nine of ten people exposed to it contract it. While the average number of people infected by one person with the coronavirus of 2020—its R0 figure—was 2.5, the R0 of measles is 15.

To prevent such a contagious disease from spreading, society has to develop herd immunity by vaccinating a large percentage of the population. With polio, which is less contagious, herd immunity can be achieved with 80 to 85 percent vaccination rates. For a highly contagious virus like measles, 95 percent of the population should be vaccinated to achieve herd immunity. Sadly, while there has been an effective vaccine since 1963, the resurgence of measles in the United States was driven by vaccine refusal, according to experts. While 91 percent of young children got the measles-mumps-rubella (or MMR) vaccine in 2017, vaccination rates in some communities fell dramatically in recent years, and it is in exactly these communities that measles skyrocketed.

This outbreak hits close to home for me because I have a six-year-old son, and the hardest-hit group, representing over half of all reported measles cases in the United States in 2019, was the Orthodox Jewish community located five blocks from our home in Brooklyn, New York. Other countrywide outbreaks have been clustered in tight-knit communities like the Jewish community in Rockland County, New York, and the Ukrainian- and Russian-American communities of Clark County, Washington, where vaccination rates hover around 70 percent, well below the threshold needed for herd immunity.

If measles is so dangerous and vaccines are so effective, why are some parents not vaccinating their kids? The answer lies partly in a wave of misinformation about the dangers of vaccination that began in 1998 with a discredited paper, published by Andrew Wakefield in the respected medical journal The Lancet, that claimed to link vaccines to autism. It was later revealed that Wakefield had been paid by lawyers suing the vaccine manufacturers, and that he was developing a competing vaccine himself. The Lancet promptly retracted the paper, and Wakefield lost his medical license. But the wave of misinformation he created continues today, aided by conspiracy theories propagated on blogs, in a widely circulated movie called Vaxxed directed by Wakefield himself, and most recently on social media.

In March 2019, addressing this wave of antivax misinformation during a public U.S. Senate hearing, Dr. Jonathan McCullers, chair of the department of pediatrics at the University of Tennessee Health Science Center and pediatrician-in-chief at Le Bonheur Children’s Hospital in Memphis, testified that, in addition to state policies on vaccine exemptions and methods of counseling, “social media and the amplification of minor theories through rapid and diffuse channels of communication, coupled with instant reinforcement in the absence of authoritative opinions, is now driving … vaccine hesitancy. When parents get much of their information from the Internet or social media platforms such as Twitter and Facebook, reading these fringe ideas in the absence of accurate information can lead to understandable concern and confusion. These parents may thus be hesitant to get their children vaccinated without more information.” The anecdotal evidence that social media misinformation drives the spread of vaccine-preventable diseases like measles is troubling.

The Antivax King of Facebook

Larry Cook describes himself as a “full time [antivax] activist.” As of 2019, he was also the antivax king of Facebook. His Stop Mandatory Vaccination organization is a for-profit entity that makes money peddling antivax fake news on social media and earning referral fees from sales of antivax books on Amazon. He also raises money through GoFundMe campaigns that pay for his website, his Facebook ads, and his personal bills. Cook’s Stop Mandatory Vaccination and another organization called the World Mercury Project, headed by Robert F. Kennedy, Jr., bought 54 percent of the antivaccine ads on Facebook in 2019.

Cook’s antivax Facebook ad campaigns targeted women over twenty-five in Washington State, a population likely to have kids who need vaccines. More than 150 such posts aimed at such women, promoted by seven Facebook accounts including Cook’s, were viewed 1.6 million to 5.2 million times and garnered about 18 clicks per dollar spent on the campaigns. Facebook’s average cost per click across all industries hovers around $1.85. When you do the math, you see that Cook’s reach is insanely efficient. These data suggest he pays about six cents per click.

In early 2019, Facebook search results for information about vaccines were dominated by antivaccination propaganda. YouTube’s recommendation algorithm pushed viewers “from fact-based medical information toward anti-vaccine misinformation,” and “75% of Pinterest posts related to vaccines discussed the false link between measles vaccines and autism.” In a paper published in 2019, researchers at George Washington University found that Russian Twitter bots were posting about vaccines twenty-two times more often than the average Twitter user, linking vaccination misinformation to Russia’s efforts to hijack the Hype Machine.

Like political fake news, antivax misinformation is concentrated. Analysis by Alexis Madrigal of The Atlantic showed that the top 50 vaccine pages on Facebook accounted for nearly half of the top 10,000 vaccine-related posts and 38 percent of all the likes on these posts from January 2016 to February 2019. In fact, just seven antivax pages generated 20 percent of the top 10,000 vaccine posts during this time.

As I will describe in the next chapter, the Hype Machine’s networks are highly clustered around tight-knit communities of people with similar views and beliefs. We live in an information ecosystem that connects like-minded people. The measles outbreaks in New York and Washington in 2019 and 2020 are taking place in tight-knit communities of like-minded people. In the same way that Russian misinformation wouldn’t need to convince the majority of Americans to affect elections (small numbers in key swing states would be enough), antivax social media campaigns wouldn’t have to convince large swaths of people to forgo vaccinations to create an outbreak. To bring the levels of vaccination below the thresholds needed for herd immunity, they would only have to convince small numbers of people in tight-knit communities, who would then share the misinformation among themselves.

Research that analyzed the interactions of 2.6 million users with 300,000 vaccine-related posts over seven years found that the vaccine conversations taking place on Facebook were happening in exactly these types of tight-knit communities. The findings showed that “the consumption of content about vaccines is dominated by the echo chamber effect and that polarization increased over the years. Well separated communities emerged from the users’ consumption habits. … The majority of users consume in favor or against vaccines, not both.” These tight-knit communities in Washington State are the exact communities that Larry Cook and antivax profiteers have targeted with their Facebook ads. They are also the same communities in which disease outbreaks are occurring.

In early 2019, social media platforms took notice. Instagram began blocking antivaccine-related hashtags like #vaccinescauseautism and #vaccinesarepoison. YouTube announced it is no longer allowing users to monetize antivaccine videos with ads. Pinterest banned searches for vaccine content. Facebook stopped showing pages and groups featuring antivaccine content and tweaked its recommendation engines to stop suggesting users join these groups. They also took down the Facebook ads that Larry Cook and others had been buying. The social platforms took similar steps to stem the spread of coronavirus fake news in 2020. Will these measures help slow the coronavirus, measles outbreaks, and future pandemics? Will fake news drive the spread of preventable diseases? Answers to these questions lie in the emerging science of fake news.

The Science of Fake News

Despite the potentially catastrophic consequences of the rise of fake news for our democracies, our economies, and our public health, the science of how and why it spreads online is still in its infancy. Until 2018, most scientific studies of fake news had only analyzed small samples or case studies of the diffusion of single stories, one at a time. My colleagues Soroush Vosoughi and Deb Roy and I set out to change that when we published our decade-long study of the spread of false news online in Science in March 2018.

In that study, we collaborated directly with Twitter to study the diffusion of all the verified true and false rumors that had ever spread on the platform from its inception in 2006 to 2017. We extracted tweets about verified fake news from Twitter’s historical archive. The data included approximately 126,000 Twitter cascades of stories spread by 3 million people over 4.5 million times. We classified news as true or false using information from six independent fact-checking organizations (including Snopes, PolitiFact, FactCheck, and others) that exhibited 95 to 98 percent agreement on the veracity of the news. We then employed students working independently at MIT and Wellesley College to check for bias in how the fact-checkers had chosen those stories.

Once we had a comprehensive database of verified rumors spanning the ten years of Twitter’s existence, we searched Twitter for mentions of these stories, followed their sharing activity to the “origin” tweets (the first mention of a story on Twitter), and re-created the retweet cascades (unbroken chains of retweets with a common, singular origin) of these stories spreading online. When we visualized these cascades, the sharing activity took on bizarre, alien-looking shapes. They typically began with a starburst pattern of retweets emanating from the origin tweet and then spread out, with new retweet chains that looked like the tendrils of a jellyfish trailing away from the starburst. I’ve included an image of one of these false news cascades in Figure 2.2. These cascades can be characterized mathematically as they spread across the Twitter population over time. So we analyzed how the false ones spread differently than the true ones.

Figure 2.2 The spread of a false news story through Twitter. Longer lines represent longer retweet cascades, demonstrating the greater breadth and depth at which false news spreads.

Our findings surprised and disturbed us. In all categories of information, we discovered, false news spread significantly farther, faster, deeper, and more broadly than the truth—sometimes by an order of magnitude. While the truth rarely diffused to more than 1,000 people, the top 1 percent of false news cascades routinely diffused to as many as 100,000 people. It took the truth approximately six times as long as falsehood to reach 1,500 people and twenty times as long to travel ten reshares from the origin tweet in a retweet cascade. Falsehood spread significantly more broadly and was retweeted by more unique users than the truth at every cascade depth. (Each reshare spreads the information away from the origin tweet to create a chain or cascade of reshares. The number of links in a chain is the cascade’s “depth.”)

False political news traveled deeper and more broadly, reached more people, and was more viral than any other category of false news. It reached more than 20,000 people nearly three times faster than all other types of false news reached just 10,000 people. News about politics and urban legends spread the fastest and was the most viral. Falsehoods were 70 percent more likely to be retweeted than the truth, even when controlling for the age of the account holder, activity level, and number of followers and followees of the original tweeter and whether the original tweeter was a verified user.

While one might think that characteristics of the people spreading the news would explain why falsity travels with greater velocity than the truth, the data revealed the opposite. For example, one might suspect that those who spread falsity had more followers, followed more people, tweeted more often, were more often “verified” users, or had been on Twitter longer. But the opposite was true. People who spread false news had significantly fewer followers, followed significantly fewer people, were significantly less active on Twitter, were “verified” significantly less often, and had been on Twitter for significantly less time, on average. In other words, falsehood diffused farther, faster, deeper, and more broadly than the truth despite these differences, not because of them. So why and how does fake news spread?

Lies spread online through a complex interaction of coordinated bots and unwitting humans working together in an unexpected symbiosis.

Social Bots and the Spread of Fake News

Social bots (software-controlled social media profiles) are a big part of how fake news spreads. We saw this in our Twitter data when we analyzed Russia’s information operation in Crimea in 2014 and in the decade of data in our broader Twitter sample. The way social bots are used to spread lies online is both disturbing and fascinating.

In 2018 my friend and colleague Filippo Menczer at Indiana University, along with his colleagues Chengcheng Shao, Giovanni Ciam-paglia, Onur Varol, Kai-Cheng Yang, and Alessandro Flammini, published the largest-ever study on how social bots spread fake news. They analyzed 14 million tweets spreading 400,000 articles on Twitter in 2016 and 2017. Their work corroborated our finding that fake news was more viral than real news. They also found that bots played a big role in spreading content from low-credibility sources. But the way bots worked to amplify fake news was surprising, and it highlights the sophistication with which they are programmed to prey on the Hype Machine.

First, bots pounce on fake news in the first few seconds after it’s published, and they retweet it broadly. That’s how they’re designed. And the initial spreaders of a fake news article are much more likely to be bots than humans. Think about the starburst pattern in the Twitter cascade of fake news shown in Figure 2.2. Many of these starbursts are created by bots. What happens next validates the effectiveness of this strategy, because humans do most of the retweeting. The early tweeting activity by bots triggers a disproportionate amount of human engagement, creating cascades of fake news triggered by bots but propagated by humans through the Hype Machine’s network.

Second, bots mention influential humans incessantly. If they can get an influential human to retweet fake news, it simultaneously amplifies and legitimizes it. Menczer and his colleagues point to an example in their data in which a single bot mentioned @realDonaldTrump nineteen times, linking to the false news claim that millions of votes were cast by illegal immigrants in the 2016 presidential election. The strategy works when influential people are fooled into sharing the content. Donald Trump, for example, has on a number of occasions shared content from known bots, legitimizing their content and spreading their misinformation widely in the Twitter network. It was Trump who adopted the false claim that millions of illegal immigrants voted in the 2016 presidential election as an official talking point.

But bots can’t spread fake news without people. In our ten-year study with Twitter, we found that it was humans, more than bots, that helped make false rumors spread faster and more broadly than the truth. In their study from 2016 to 2017, Menczer and his colleagues also found that humans, not bots, were the most critical spreaders of fake news in the Twitter network. In the end, humans and machines play symbiotic roles in the spread of falsity: bots manipulate humans to share fake news, and humans spread it on through the Hype Machine. Misleading humans is the ultimate goal of any misinformation campaign. It’s humans who vote, protest, boycott products, and decide whether to vaccinate their kids. These deeply human decisions are the very object of fake news manipulation. Bots are just a vehicle to achieve an end. But if humans are the objects of fake news campaigns, and if they are so critical to their spread, why are we so attracted to fake news? And why do we share it?

The Novelty Hypothesis

One explanation is what Soroush Vosoughi, Deb Roy, and I called the novelty hypothesis. Novelty attracts human attention because it is surprising and emotionally arousing. It updates our understanding of the world. It encourages sharing because it confers social status on the sharer, who is seen as someone who is “in the know” or who has access to “inside information.” Knowing that, we tested whether false news was more novel than the truth in the ten years of Twitter data we studied. We also examined whether Twitter users were more likely to retweet information that seemed to be more novel.

To assess novelty, we looked at users who shared true and false rumors and compared the content of rumor tweets to the content of all the tweets the users were exposed to in the sixty days prior to their decision to retweet a rumor. Our findings were consistent across multiple measures of novelty: false news was indeed more novel than the truth, and people were more likely to share novel information. This makes sense in the context of the “attention economy” (which I will discuss in detail in Chapter 9). In the context of competing social media memes, novelty attracts our scarce attention and motivates our consumption and sharing behaviors online.

Although false rumors were more novel than true rumors in our study, users may not have perceived them as such. So to further test our novelty hypothesis, we assessed users’ perceptions of true and false rumors by comparing the emotions they expressed in their replies to these rumors. We found that false rumors inspired more surprise and disgust, corroborating the novelty hypothesis, while the truth inspired more sadness, anticipation, joy, and trust. These emotions shed light on what inspires people to share false news beyond its novelty. To understand the mechanisms underlying the spread of fake news, we have to also consider humans’ susceptibility to it.

Our Susceptibility to Fake News

The science of human susceptibility to false beliefs is more developed than the science of fake news but is, unfortunately, no more settled. There’s currently a debate between “classical reasoning” and “motivated reasoning.” Classical reasoning contends that when we think analytically, we are better able to tell what’s real from what’s fake. Motivated reasoning, on the other hand, contends that when we are faced with corrective information about a false belief, the more analytically minded of us “dig in” and increase our commitment to those false beliefs, especially if we are more partisan or committed to those false beliefs to begin with.

My friend and colleague David Rand at MIT teamed up with Gordon Pennycook to study what types of people were better able to recognize fake news. They measured how cognitively reflective people were using a cognitive reflection task (CRT) and then asked them whether they believed a series of true and false news stories. A cognitive reflection task tests how reflective someone is by giving them a simple puzzle, like this one: “A bat and ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?” The problem elicits a fast, intuitive response—ten cents—that, upon reflection, is wrong: if the ball cost ten cents, the bat would have to cost $1.10 and they would total $1.20. Asking people to consider these types of puzzles tests their reflectiveness. And Rand and Pennycook found that people who were more reflective were better able to tell truth from falsity and to recognize overly partisan coverage of true events, supporting classical reasoning.

But repetition causes belief. If you beat us over the head with fake news, we’re more likely to believe it. It’s called the “illusory truth effect”—we tend to believe false information more after repeated exposure to it. People also tend to believe what they already think. (That’s confirmation bias.) So the more we hear something and the more it aligns with what we know, the more likely we are to believe it. Similar thinking has led some cognitive and political scientists to hypothesize that because of confirmation bias, corrective information can backfire—that trying to convince someone that their falsely held belief is wrong actually causes them to dig in to those false beliefs even more. But so far, evidence for this “backfire effect” seems weaker. For example, in three survey experiments, Andrew Guess and Alexander Coppock found “no evidence of backlash, even under theoretically favorable conditions.”

So reflection helps us distinguish truth from falsity, repetition causes belief, and corrective information doesn’t seem to backfire even though a confirmation bias generally leads us to believe what we already know. These findings give us leads for fighting fake news (which I will return to in Chapter 12, when I discuss how we must adapt).

The Economic Motive to Create Fake News

The political motive for creating fake news is abundantly clear from Russia’s foreign interference in Ukrainian and American politics. But the economic motive should not be underestimated. And nowhere has the economic motive to create fake news been more obvious than in Veles, Macedonia.

Veles is a sleepy mountain town with 55,000 residents, two TV channels, and a few lovely churches. It boasts a handful of notable historical figures and events, from Ottoman grand viziers to battles between the Serbian and Ottoman empires in the late fourteenth century. But perhaps the most important contribution to Veles’s global historical significance will be that during the 2016 U.S. presidential election, its unemployed teen population discovered how the Hype Machine could make them rich by spreading fake news online.

The teenagers of Veles developed and promoted hundreds of websites that spread fake news to voters in the United States through social media advertising networks. Companies like Google show ads to Internet browsers and pay website creators based on how many high-quality eyeballs they attract. The teenagers of Veles discovered that they could make a lot of money by creating websites and promoting their content though social media networks. The more people read and shared their articles, the more money they made.

They found that fake news attracted more readers and, as we found in our own research, that it was 70 percent more likely to be shared online. They created fake accounts to amplify the signal, and once the trending algorithms got hold of them, the fake news stories received a broadcasting boost, exposing them to even more people, in new areas of the network. What ensued was a deluge of fake news that washed over the American public just as they were heading to the polls. Money flowed in one direction and falsehood flowed in the other, leaving Veles flush with new BMWs, and the United States inundated with false news months before the 2016 presidential election. The town of Veles is only one such example. In 2019 fake news websites generated over $200 million a year in ad revenue. Fake news is big business, and our approaches to solving the problem (which I will address in Chapter 12) must recognize that economic reality.

The End of Reality

Unfortunately, everything I have described so far—from stock market crashes to coronavirus misinformation to measles outbreaks to election interference—is the good news. That’s because the age of fake news is about to get a whole lot worse. We are on the verge of a new era of synthetic media that some fear will usher us into an “end of reality.” This characterization may seem dramatic, but there is no doubt that technological innovation in the fabrication of falsity is advancing at a breakneck pace. The development of “deepfakes” is generating exceedingly convincing synthetic audio and video that is even more likely to fool us than textual fake news. Deepfake technology uses deep learning, a form of machine learning based on multilayered neural networks, to create hyperrealistic fake video and audio. If seeing is believing, then the next generation of falsity threatens to convince us more than any fake media we have seen so far.

In 2018 movie director (and expert impersonator) Jordan Peele teamed up with BuzzFeed to create a deepfake video of Barack Obama calling Donald Trump a “complete and total dipshit.” It was convincing but obviously fake. Peele added a tongue-in-cheek nod to the obvious falsity of his deepfake when he made Obama say, “Now, I would never say these things … at least not in a public address.” But what happens when the videos are not made to be obviously fake, but instead made to convincingly deceive?

Deepfake technology is based on a specific type of deep learning called generative adversarial networks, or GANs, which was first developed by Ian Goodfellow while he was a graduate student at the University of Montreal. One night while drinking beer with fellow graduate students at a local watering hole, Goodfellow was confronted with a machine-learning problem that had confounded his friends: training a computer to create photos by itself. Conventional methods were failing miserably. But that night while enjoying a few pints, Goodfellow had an epiphany. He wondered if they could solve the problem by pitting two neural networks against each other. It was the origin of GANs—a technology that Yann LeCun, former head of Facebook AI Research, dubbed “the coolest idea in deep learning in the last 20 years.” It’s also what manipulated Barack Obama to call Donald Trump a “dipshit.”

GANs pit two networks against each other: a “generator,” whose job is to generate synthetic media, and a “discriminator,” whose job is to determine if the content is real or fake. The generator learns from the discriminator’s decisions and optimizes its media to create more and more convincing video and audio. In fact, the generator’s whole job is to maximize the likelihood that it will fool the discriminator into thinking the synthetic video or audio is real. Imagine a machine, set in a hyper loop, trying to get better and better at fooling us. That’s the future of reality distortion in a world with exponentially improving GANs technology.

GANs can be used for good as well—for example, to generate convincing synthetic data in high-energy physics experiments or to accelerate drug discovery. But the potential geopolitical and economic harm they can create is troubling. Ambassador Daniel Benjamin, former coordinator for counterterrorism at the U.S. State Department, and Steven Simon, former National Security Council senior director for counterterrorism in the Clinton and Obama administrations, paint a grim picture: “One can easily imagine the havoc caused by falsified video that depicts foreign Iranian officials collaborating with terrorists to target the United States. Or by something as simple as invented news reports about Iranian or North Korean military plans for preemptive strikes on any number of targets. … It might end up causing a war, or just as consequentially, impeding a national response to a genuine threat.”

Deepfaked audio is already being used to defraud companies of millions of dollars. In the summer of 2019, Symantec CTO Hugh Thompson revealed that his company had seen deepfaked audio attacks against several of its clients. The attackers first trained a GAN on hours of public audio recordings of a CEO’s voice, while giving news interviews, delivering public speeches, speaking during earnings calls, or testifying before Congress. Using these audio files, the attackers built a system to automatically mimic the CEO’s voice. They would call, for example, the CFO of the company and pretend to be the CEO requesting an immediate wire transfer of millions of dollars into a bank account they controlled. The system didn’t just deliver a prerecorded message but converted the attacker’s voice into the CEO’s voice in real time so they could engage in a realistic conversation and answer the CFO’s questions. The synthesized audio of the CEO’s voice was so convincing that, coupled with a good story about why the money needed to be transferred right away—they were about to lose a big deal, or they had to beat an impending deadline at the end of the fiscal quarter—the CFO would comply with the CEO’s request and execute the transfer. Thompson revealed that each attack cost the target companies millions of dollars.

As Jordan Peele made Obama say in his BuzzFeed deepfake: “It may sound basic, but how we move forward, in the Age of Information, is going to be the difference between whether we survive or whether we become some kind of fucked up dystopia.” To understand whether dystopia is our destiny, we have to understand how the Hype Machine works. To do that, we’ll need to go back to first principles, starting with a deep dive under the hood of the Hype Machine, followed by an examination of social media’s effect on our brains.

* Disinformation is deliberate falsehood spread to deceive, while misinformation is falsehood spread regardless of its intent. Disinformation is a subset of misinformation.

* Chris Bail and his team found no evidence that interactions with IRA Twitter accounts in late 2017 impacted political attitudes or behavior. But several limitations prevented them from determining “whether IRA accounts influenced the 2016 presidential election.” The study was conducted a year after the election, after the IRA had ramped down their information operation and Twitter had suspended two-thirds of their accounts. The sample included no independents and only frequent Twitter users, was not representative of U.S. voters, and did not consider voting behavior. The study did find suggestive evidence that IRA interactions changed three things: opposing party ratings among “low news interest” respondents, the number of political accounts followed among “high news interest” respondents, and both opposing party ratings and the number of political accounts followed among Democrats.

Конец ознакомительного фрагмента.

Текст предоставлен ООО «ЛитРес».

Прочитайте эту книгу целиком, купив полную легальную версию на ЛитРес.

Безопасно оплатить книгу можно банковской картой Visa, MasterCard, Maestro, со счета мобильного телефона, с платежного терминала, в салоне МТС или Связной, через PayPal, WebMoney, Яндекс.Деньги, QIWI Кошелек, бонусными картами или другим удобным Вам способом.