I’m trying something new, under the name First Takes. These will be quick write-ups of ideas I find interesting. If this goes well, expect to see these more often than my traditional posts, and covering a wider range of topics. If you’d like to only receive my traditional, high-effort pieces, see the instructions at the end of this post.
Anthropic’s engineering team just dropped a fascinating post: Redesigning source control at Anthropic. It describes a major engineering project undertaken in a jaw-droppingly short period of time. My Twitter feed is awash in breathless reports of miracles accomplished overnight using coding agents, but this project stands out as a well-documented example of serious engineering that takes full advantage of the latest tools. AI coding tools are no longer limited to vibe-coding lightweight apps; they are revolutionizing the craft of building large-scale, mission-critical distributed systems.
The project in a nutshell
Any serious software development effort revolves around a “source code control system”. This is sort of like Google Drive, but specialized to store program code instead of documents and spreadsheets.
At Anthropic, like many tech companies, the entire company works in a single system, called the “monorepo”. Anthropic used the industry-standard solution, named “git”, to manage their monorepo. Git is a widely respected tool, serving millions of developers, but it was buckling under the strain of Anthropic’s rapidly accelerating engineering velocity:
Git is a decentralized system… As our monorepo grows, it increasingly doesn’t fit that design space. The history of our main branch alone is almost 100 GB, and a compressed working copy is over 25 GB. We perform on the order of 25 million code checkouts per day for CI, before counting developer environments, agent sandboxes, and research jobs. Fetched naively, that would be over 600 PB of git traffic per day.
…there are standard workarounds for large monorepos, like internal git mirrors and repository snapshots (seeds) kept in blob storage. We’ve used all of them—at our peak, we operated several hundred internal git mirrors to deal with the load [emphasis added]. This bought us time, but they became harder to operate as our merge rate increased.
Anthropic decided to build their own source code control system from scratch, using a completely different architecture. The new system was called Scion. Accomplished with just seven engineers, the speed and scope of this project was insane:
A team of seven people built both Scion and Antique (our code review tool, which we could easily write a whole other blog post about), not to mention half a dozen related systems [such as code review, code search, and access management], and fully migrated to them in five months. We did all this while the volume of code merged per day increased by 10x. In the past, developing and deploying a source control system like this would have taken a company years of iterative migration, with many more engineers. We think this sort of fast, nimble migration will become more common in the future.
I can support the claim that traditionally, a project like this would take years. There are at least four phases, each of which would take many months at best:
Gather requirements – how are people using the current system? Each team may have their own unique needs.
Design and implement a new solution.
Test the new system at scale.
Very, very carefully migrate the company to the new system, without interrupting work.
A similar project at Google took over four years1, with a larger team and not including code review, code search, or other add-ons, though the challenges were greater than for Scion2.
The ability to develop bespoke solutions is a game-changer
Git is the product of 20 years of engineering. Anthropic’s home-grown Scion, developed in five months, outperforms it by multiple orders of magnitude:
Our Scion infrastructure footprint is now 200x smaller than our previous git mirroring infrastructure in terms of CPU and memory requirements. CI and research jobs check out code two orders of magnitude faster (in seconds rather than minutes), which frees up a sizable amount of capacity across the fleet; CI alone realized a 25% CPU capacity win.
This isn’t because coding agents make the Anthropic team superior to the team behind git (which includes Linus Torvalds, famed creator of Linux). It’s because Anthropic was able to design a solution that was tailored to their specific requirements:
Git is a decentralized system: every clone is a complete, self-contained repository that can compute diffs, read history and blame, and check out any revision while fully offline. For the vast majority of repositories, this is the right design, and it’s a large part of why git became the standard.
As our monorepo grows, it increasingly doesn’t fit that design space.
…The properties that make git excellent for distributed, offline work were actively undermining our ability to scale as the repo grew. So we asked ourselves: if we designed source control with the fact that agents are always online in mind from the start, what would it look like?
The blog post goes into substantial detail regarding the requirements Anthropic set for Scion and the design they settled on to meet those requirements. The resulting system is completely different from git. Git is designed to support collaboration across individuals and teams scattered around the globe, some of whom work in close collaboration, some of whom might belong to entirely different organizations and only exchange code once every few years, if ever. Scion assumes that everyone is working in the same company and has a continuous connection to the internal network. This allowed for a different set of design choices that improve efficiency for Anthropic’s use case by a factor of 100 or more.
Scion leverages standard building blocks
Scion may be a clean-sheet design, but it doesn’t reinvent the wheel. The blog post makes repeated reference to Spanner – Google’s world-class scalable database service. Files reside in “blob storage” – likely Google Cloud Storage (a service similar to Amazon S3). In other words, data ultimately resides in off-the-shelf, professionally managed services – not vibe-coded solutions.
I expect this to be the standard pattern for bespoke software for some time to come3. Agentic engineering allows for major projects to be built from scratch, but those projects will generally use off-the-shelf solutions for data storage, user authentication, and other foundational services. A few of the reasons for this:
Data integrity. If you build on reliable data storage services like Spanner and Google Cloud Storage (or their equivalents on AWS or other platforms), it’s relatively easy to design in such a way that you can recover from disasters. If Scion turns out to have a bug that scrambles the data, I would be very surprised if the Anthropic team were not able to recover. This doesn’t come for free, but designing for disaster recovery – meaning recovery from bugs, not just equipment failure – is much easier when working on a robust, time-tested foundation4.
Economy of scale. Services like Spanner or Google Cloud Storage operate much more efficiently at large scale.
Interoperability. That’s not a requirement for Scion, but will come up in many other situations.
Engineering skill still matters
Coding agents allow people with no technical background to build functional software. Here’s (then) New York Times reporter Kevin Roose, writing way back in January:
But for the moment, engineering skill still matters:
Engineers are no less valuable, but their work is changing. All of the engineers who worked on this system would now rate Claude as better at writing code than them. But our engineers still supplied the invaluable understanding of the high-level inter-system components and the design-level tradeoffs, making informed choices around the problems we wanted to solve and the scale we knew we needed to achieve. Very often, we found that the role of the engineers was to tell Claude “no.” Agents will sometimes violate core system constraints to satisfy a pattern that should never have been allowed in the first place. …skilled engineers needed to steer Claude away from over-engineering parts of the system in response to hypothetical concerns. …Our team members still review every change request for the system, but they now examine the intent of the change as opposed to reading every line of code.
Armed with unlimited tokens, the team was able to aim very high:
We were able to substantially improve optimization and quality. By virtue of Claude’s ability to write immense amounts of code much faster than we could ourselves, Scion is among the most well-tested and well-defended codebases we have ever worked on. ... Claude also casually found significant optimizations—for example, an algorithmic optimization to ancestry queries that missed one of our indexes—that would have taken an expert days or weeks of analysis and tuning… and then did it all again the next day.
AI doesn’t let us do things better, it lets us do more things – which is better
The engineers at Anthropic aren’t smarter than Linus Torvalds and the rest of the people who built git. Claude Code isn’t smarter than them, either. (Not yet, anyway.) But Claude is much, much faster and cheaper than human engineers. As a result:
The Anthropic team was able to build a clean-sheet design. It was better because it was tailored to their needs, which was only possible because agentic coding made it economically feasible to build a custom solution.
By using coding agents to create custom testing environments, Anthropic was able to climb the maturity curve for this brand-new system in record time.
The lesson is that AI doesn’t need to be superhumanly intelligent to invalidate our fundamental assumptions of what can be done. It just needs to be superhumanly fast and cheap. And that milestone has been reached.
Again, First Takes is an experiment. Let me know what you think! If you’d like to opt out and only receive my traditional posts, open your Substack settings, select Second Thoughts under Subscriptions, and turn off First Takes.
My research assistant, ChatGPT, unearthed some additional sources regarding this project.
At the time of their migration, Google was already a much larger and complex organization, with 25,000 developers and 300 internal tools that had to be migrated. Furthermore, the Anthropic project was able to make use of more modern components (such as Spanner), and benefited from prior art – including the Google project itself. (Thanks to Claude for flagging all of this. Always ask an AI to critique your work before publishing! Just don’t blindly accept its feedback.)
Granted, I’m not sure whether “some time to come” means the next decade or just the next year.
To be clear, I am assuming here that Scion is designed such that in the event something gets scrambled, it would be possible to retrieve older versions of the data. This could be accomplished, for example, by incorporating version history into the Spanner scheme and enforcing this in code that is carefully reviewed.
The line I’d keep: “the role of the engineers was to tell Claude no.” If the machine is superhumanly fast, the no becomes the slow part — and the part that decides. Your footnote 2 does the same on a small scale: you asked Claude to critique, linked the exchange, and kept the right not to follow it.
Will the no stay with the engineers, or move up to whoever sets the requirements?