We have a cybersecurity problem. You might have heard.
It’s even worse than the occasional headlines suggest. Hacking incidents are routine – regarded like car crashes, not airplane crashes. When you think about it, this is kind of fucked up. As Dean Ball recently noted:
...the internet is already such a profoundly crime-ridden place. Imagine if gangs went into hospitals in America every week and stole people’s medical records out of filing cabinets. If that happened in the physical world, we’d rightfully question whether government still possessed practical sovereignty. Yet American hospitals are raided constantly in the digital world and we basically tolerate it1. [lightly edited]
Cybercrime (broadly defined2) and related activities impose a multi-trillion-dollar burden on the world economy3. And that’s without even considering the national security implications. Who knows what might happen in the event of a “hot” cyber war between major powers such as the US and China.
Advancing AI capabilities are bearing down on this rickety status quo like a runaway freight train. The recent OpenAI / Hugging Face incident showed that AI models are already capable of penetrating the security of a $5 billion tech company with no human involvement. And capabilities are advancing by leaps and bounds.
We’re not on track to meet the AI challenge. But we could be.
We need to change the rules of the game
As things stand today, security requires perfection – from millions of developers and system administrators, and billions of users. Developers must eliminate all bugs, system administrators must promptly apply all patches and never make configuration errors, and users must never type their password into the wrong box or fall for phone scams. This would be too much to hope for even in an ideal world, and circumstances in the digital world are far from ideal. Factors such as unsafe software defaults, confusing login systems, and spoofable caller ID make mistakes all too easy.
There are widespread efforts to meet the threat of AI-powered cyberattacks, but they work within the existing framework4. Little is being done to address the structural issues that often make good security impractical5. For instance, under initiatives like Anthropic’s Project Glasswing and OpenAI’s Daybreak, trusted organizations are using the latest AI models to frantically unearth and patch long-standing bugs. This seems to be accompanied by a glum hope that, against all experience, system operators worldwide will finally start applying those patches promptly.
Experts universally agree that the digital systems we all depend on are riddled with security flaws. In the past, only a fraction of these flaws were exploited, and the system staggered along. That won’t cut it any longer. Rather than expecting perfection, we need to change the rules of the game so that perfection is not required. It’s been done before.
How we secured web connections
Once upon a time, connections to most web servers were unencrypted. Anyone who could plug in to the same network as you, or who was in range of the same wifi base station, could read over your shoulder as you surfed the web. They’d see every password and credit card number you typed6.
Fixing this required every website operator in the world to purchase and install an encryption key. This was a finicky process; in my own work, I dreaded it above almost any other chore. Even then, users were unprotected unless they remembered to type “https” instead of “http” every single time they entered a web address.
Today, unencrypted web connections are a thing of the past. This was accomplished through a series of coordinated actions:
A nonprofit organization built a service, called Let’s Encrypt, that streamlined the process of creating encryption keys – and provided them for free.
Web server software and web hosting services were updated to make it easier to configure encryption.
Web browsers were modified to automatically use https when available.
Google announced that web sites which did not support encryption would be downranked in search results.
Browser developers (again including Google) began warning users when they accessed an insecure web site.
Steps 1 and 2 made it easier for site operators to do the right thing. Step 3 basically removed the need for users to do anything at all. And steps 4 + 5 created real motivation for site operators to take action. As a result, the problem of unencrypted web connections has been defined out of existence7.
We need to apply this playbook in more places.
Passwords and other obsolete concepts
Passwords are inherently fragile. You can pick a weak password, or be tricked into typing it in the wrong place, or malware can watch you typing it. Password managers and two-factor authentication codes help, but they’re complicated and don’t fully solve the problem8. The simple fact is that passwords as a concept have passed their use-by date. They’ve been a source of problems since time immemorial, and it’s time we replaced them with something better9.
Some other aspects of digital life that also need to go the way of hand-crank car starters and elevator doors without safety interlocks:
API keys. These are basically passwords, but for use by computers instead of people. Cyberattack postmortems frequently involve someone breaking into one system, finding an API key, and using it to access other systems.
Weak password reset mechanisms. Most systems allow you to recover from a forgotten password by receiving an email code or reciting a few bits of information to a customer service representative. Attackers can often subvert these mechanisms.
Unauthenticated communication channels. Caller ID on phone calls and text messages can be faked. This enables scams, and makes it harder to stamp out telephone spam.
Network-connected systems that aren’t routinely updated. When was the last time you updated the software in your smart light bulbs? Don’t bother, we both know the answer is “never”. Which still might make them more up-to-date than some municipal water control systems or FDA-regulated medical devices. We need to recognize that some systems are inherently difficult to update at the rapid pace that AI bug discovery necessitates, and find systematic ways to avoid exposing these systems to a network.
Legacy programming languages like C and C++ that require developers to manually manage memory usage – an endless source of bugs.
This is far from a complete list. I haven’t touched on misconfigurations, supply-chain vulnerabilities, attacks on CI/CD systems, or other common elements of cyberattacks. Nor are my problem statements necessarily well chosen. My point is simply that most attacks depend on a handful of fragile mechanisms, and if we could eliminate those mechanisms, that would have a huge impact on cybersecurity.
This leads to the question: is it plausible that we could eliminate passwords and other insecure staples of digital life?
When we set our minds to it, yes, we can absolutely change the world
Remember credit card skimmers? Criminals used to attach false facades to the credit card slot on gas pumps, scanning every card as it was inserted. This seemed impossible to prevent... until tap-to-pay eliminated swiping and rendered the problem moot10.
At one point, email inboxes were routinely flooded with spam. Nowadays, mail services are quite good at shunting it to its own folder11, in part because of the adoption of new mechanisms that make it difficult to forge the From address.
The province of Alberta, as a result of a deliberate effort dating back to the 1950s, is the one place on Earth that has lots of people but no rats.
We’ve eliminated multiple rabies variants in the wild by vaccinating the wild animals that carry rabies.
The Y2K bug, as older readers will recall, would have crippled the operations of thousands of businesses and government agencies. No one could come up with a solution that wouldn’t require major surgery to a huge swath of mission-critical legacy software systems. So we stepped up and did the major surgery.
So yes, it is possible to do big things. It’s time to do big things for cybersecurity.
Let’s stop patching potholes and start reinventing asphalt
The structural issues that make security so difficult mostly lie at the interfaces between systems. Fixing bugs is something that a team can do unilaterally, but changing the way users log in to services (passwords), systems authenticate to one another (API keys), or library authors ship code to application developers, requires a coordinated effort.
The campaign to encrypt web connections was one such effort. Its success stemmed from three things:
Broad acceptance that change was required.
Technical innovations that unlocked a cheap and practical solution (such as the Let’s Encrypt project).
Levers that motivated all relevant parties to take action (such as Google down-ranking unsecured sites).
Step 2 is easily within our grasp; the core ideas already exist12. As for steps 1 and 3: the world is already waking up to the implications of AI for cybersecurity. If a fraction of the effort that is being devoted to patching bugs could be diverted to addressing the systemic issues that make security difficult, we’ll be much better positioned for the era of advanced AI.
Thanks to Abi Olvera for feedback and the post image
Dean goes on to say “The world is more capable of absorbing risk and disorder than one might think”; he is making a point about resilience.
I'm defining "cybercrime" broadly, encompassing everything from the exploitation of software bugs to remotely break into a computer system, to financial fraud that involves tricking someone into revealing their bank password. These activities exist on a continuous spectrum.
A recent estimate cited by Tyler Cowen puts the total at $1T / year. I did some back-and-forth with ChatGPT Pro and Gemini and got estimates ranging from 1 to 3 trillion dollars per year. Those are all-in estimates, including both the direct impact of cybercrime, and resources invested in maintaining security.
For instance, the US Cybersecurity and Infrastructure Security Agency (CISA) has two initiatives, Secure By Design and Secure By Demand, which contain many excellent ideas. However, these are proposals for software developers to individually improve their offerings, requiring each and every developer to systematically address issues across their entire codebase and product line.
A recent guest post on Noah Smith’s blog presented some excellent ideas for using AI to improve cybersecurity – which also work within the existing framework.
Some issues: installing security patches means updating operating systems and applications. Sometimes those updates introduce bugs or incompatibilities, so IT teams need time to test each update. These teams may not have the capacity to test a barrage of frequent updates. Older and/or niche systems, including a lot of infrastructure and industrial machinery, may not be well supported by the manufacturer and so may not receive updates promptly, or at all. Some organizations may not even have a dedicated IT team, or the team might not know about all of the systems that need updating. Software updates on some systems, such as medical equipment, may be subject to regulatory hurdles.
To be fair, many e-commerce sites deployed encryption relatively early. But other sites mostly did not, and there were many ways things could go wrong even on encrypted sites, for instance if the user forgot to type "https".
I am oversimplifying a bit; unencrypted connections are still technically possible, and are in fact used in some situations. But the problem has been relegated to obscure niches. Crucially, this was accomplished in a way that does not rely on no one ever making a mistake.
For instance, an attacker can trick you into providing them with the two-factor code; this recently happened to someone I know. The same social engineering techniques would probably have sufficed to get the victim to copy/paste the password from their password manager.
The solution will probably look something like passkeys. As currently constituted, passkeys have a number of problems, first and foremost being that they’re confusing. But these problems, unlike the problems with passwords, are solvable. I am working on some ideas.
Tap-to-pay systems use a cryptographic scheme which makes skimming-type operations much more difficult and much less worthwhile for a criminal.
At least, this seems to be the case for Gmail.
For instance: passkeys, federated identity, workload identity systems, and The Great Refactor.