Tacit Knowledge: The Missing Factor in AI Bio Risk Assessments
Lab skills come from hands-on mentorship, not from reading the entire internet.
This is the second installment in a series examining the potential for AI to lower the bar to creation and use of bioweapons, written by Golden Gate Institute for AI’s Abi Olvera. Part one explains why bioweapons are difficult to create and rarely used. This installment (which you can read without reading part one) presents the importance of “tacit knowledge” – including muscle memory and sense memory critical to delicate lab procedures. I found this essay to be interesting for two reasons. First, it’s a fascinating exploration of a subject I knew nothing about – and an excellent example of important skills that current AI training methods don’t seem to capture. Second, it helps explain why some parts of the AI safety community may be overestimating the bioweapons threat, at least in the near to mid-term.
If you wanted to build a bioweapon, the first thing you’d notice is that finding instructions isn’t the hard part.
Published protocols1 exist for most of the relevant techniques. Research papers describe how to culture cells, assemble DNA, and grow pathogens. The information has been available for decades. Large language models have made it faster and easier to find.
But biologists who have spent years at the lab bench will tell you that there is an enormous gap between having instructions and successfully executing them. That gap is filled with knowledge that lives only in practitioners’ hands and heads, intuitions developed through repetition, mentorship, and thousands of small failures. You get better by failing repeatedly under the mentorship of someone who’s been doing it for years. Biology operates on an apprenticeship model, more like plumbing or surgery than other STEM fields. You learn by watching someone else, then you teach the next person.
In the literature on expertise, these important, unwritten details are called tacit knowledge.
Understanding tacit knowledge is important for evaluating AI biosecurity risks because it shapes how much AI can help someone attempting to build a bioweapon. If the hard part is finding information, then AI changes the picture significantly. If the hard part is executing physical procedures that depend on feel, judgment, and years of practice, then AI’s contribution is more limited, at least for now.
To see what tacit knowledge looks like in practice, it helps to start with the most basic thing biologists do: move liquid from one container to another.
Everything in biology is wet
Almost every step in biological work involves liquid.
DNA is assembled in liquid. Vaccines, IVs, and blood transfusions are liquid. Even pills must dissolve into liquid for absorption. When NASA looks for signs of life on other planets, it primarily looks for water. Life, as we understand it, requires it.
If you want to build a bioweapon, you are working in liquid at nearly every step.2
The most common way to move these liquids is pipetting. A pipette is an extremely precise eye dropper. Any biology student has used one.
Pipetting sounds simple. It isn’t.
Getting it right depends on tip position, liquid viscosity, the speed at which you draw and release, room temperature, and humidity. Students sometimes attempt it 50+ times before it works. Like a grandmother who knows exactly how much a “pinch” means, practitioners develop a feel for specific materials over the years.
Biology makes liquid handling especially unforgiving for three reasons.
Fragility. Biological molecules break easily. Even hand tremors can shear some DNA strands. One scientist told me that, in their pure form, without the stabilizing proteins of an organism or packaging, some long DNA molecules struggle to survive natural desk vibrations from traffic and seismic activity. Some proteins are so temperature-sensitive that the local pressure from moving a liquid creates too much heat.
A written protocol will say “pipette gently.” But “gently enough” depends on the specific molecule, the specific volume, the viscosity of your buffer, the type of pipette you use, how long the sample has been sitting out, etc. Experienced researchers develop a feel for this. They modulate their thumb pressure on the pipette plunger the way a guitarist modulates finger pressure on a string. A novice following the same written protocol will damage the sample without knowing why.
Fragility also makes it important to learn the quirks of each lab’s specific equipment. An incubator that drifts half a degree can kill an entire culture. A practitioner might need to know that their particular incubator runs hot on days when ambient lab temperature rises with the weather outside.
Concentration. In biology, the ratio of how much material is dissolved in a given volume matters as much as the raw amount of material. To stitch four pieces of DNA into one larger piece, each piece has to physically encounter its match. That’s dictated by chance. If the solution is too dilute, the molecules never find each other. Getting concentrations right is partly a matter of following protocols, but experienced researchers have also developed an eye for when things look off.
They can tell whether the slight cloudiness of a solution indicates that the DNA concentration is within the correct range. They notice when purifier chemicals are starting to fail from subtle flow changes, or when the color of a gel stain indicates that fragments are the expected size. A protocol might say “check concentration with a spectrophotometer,” but a practiced scientist recognizes when a bubble in the container is throwing off the reading, or when a residual chemical from a previous step is inflating the measurement.
Contamination. In chemistry, one part per million of contamination is usually negligible. In biology, contamination with unwanted organisms replicates across generations, compounding with each cycle. A single stray bacterium in a culture can, within hours, overwhelm the organism you were trying to grow.
For example, a speck of household dust is a miniature ecosystem containing bacteria, fungal spores, and skin cells, each with its own DNA. During cell culture and pathogen cultivation, a fungal spore from dust can land in your growth medium and outcompete the organism you’re cultivating. Stray DNA fragments from dust can mess with your genetic construct.
This is one reason labs use biosafety cabinets with HEPA-filtered airflow and why practitioners are obsessive about sterile techniques. Avoiding contamination requires tacit knowledge: how to angle a pipette tip away from the tube wall, when to swap gloves, and how to catch the faint smell of a contaminated culture before anything looks wrong under a microscope. It means knowing what actually constitutes sterile enough, such as noticing that your HVAC is circulating dust from the construction next door, or that the faint haze around your laminar flow hood means its filter is degrading. Experienced lab workers know which items to sterilize together and how to load them so steam actually penetrates. A person working in a garage or improvised lab would often not know what they didn’t know. Their cultures would fail, and they might not understand why for weeks — if ever.
What happens when tacit knowledge is missing
Challenges like these arise at each of the many steps involved in creating a biological agent, and they interact with each other. Scientists often describe biological work as months of failed attempts. The individual steps are not conceptually hard, but each one requires dozens of micro-judgments learned through repetition, muscle memory, mentorship, and trial and error. Many of these judgments are difficult to articulate, let alone replicate from a manual.
The Japanese cult Aum Shinrikyo provides a case study. This wealthy doomsday cult had an estimated $1 billion and a team of specialists, including doctors, engineers, and microbiologists. The group sprayed what they believed was anthrax from a rooftop in Kameido. No one was harmed. The failure was due to many missteps. The cult failed to make their Anthrax strain sufficiently virulent.3 Even if the strain had been virulent, the spore concentration was too low to infect anyone. The liquid was too viscous for the spores to form the fine airborne particles needed for inhalation. The gelatinous suspension clogged the steam generator.
On paper, each step might have looked correct. In practice, every step required judgment and expertise the cult didn’t have: recognizing that a solution is too thick for aerosolization before loading it, understanding that spore concentration needs to be higher than a naïve reading of the literature suggests, and knowing how to validate that a strain is actually dangerous before investing months in the rest of the process. The cult’s experts weren’t up to the task because they lacked the experience with the specific steps involved in deploying Anthrax.
Could technology replace tacit knowledge?
None of these difficulties is permanent.
Purpose-built automated systems can be tuned, over many iterations, to encode the micro-judgments that experienced scientists make unconsciously — the right pipetting speed for a specific reagent, the exact pause needed after dispensing to avoid bubbles, the temperature ramp rate that won’t shear long DNA fragments. AI systems can provide real-time video feedback on lab manipulations.
This is already happening in some settings.
Pharmaceutical companies and well-funded research labs run automated systems painstakingly calibrated for specific workflows.
But these are purpose-built facilities optimized for particular projects, not general tools a novice could deploy. Each new workflow requires its own round of expert calibration.
As automated labs become more numerous and more varied, they will absorb more of the tacit knowledge that currently lives only in human hands. Even the most skeptical biosecurity experts I interviewed expected that general-purpose lab robots, guided by AI that has learned from thousands of experimental iterations, could eventually make expertise transferable in ways it isn’t today.
That world is not today’s world. Today, tacit knowledge remains a real barrier for someone working without mentorship and without the years of failure required to develop the intuitions that make each step work.
Furthermore, automated labs won’t eliminate all barriers. If someone outsources one chunk of the process to a commercial lab, there are still remaining steps. Ordering pre-assembled DNA is already straightforward; many companies will synthesize custom DNA sequences for research purposes. But converting that DNA into a living pathogen, or engineering it for greater lethality, involves work that no legitimate automated lab would perform.4 For policymakers, tracking how these partial capabilities interact is a harder problem than tracking any single capability in isolation.
What this means for AI risk assessments
Most evaluations of AI models for biology-relevant capabilities focus on textbook knowledge, such as whether a model can answer questions from the scientific literature or whether access to an LLM improves performance on standardized assessments. Some evaluations test laboratory knowledge, but few attempt to measure whether AI access changes outcomes in real-world tasks and workflows, where tacit knowledge is the binding constraint.
Only a handful of efforts, like SecureBio’s virology evaluation, have tried to test for practical knowledge that goes far beyond what’s in textbooks. Whether these capture enough of the tacit dimension to be useful predictors of real-world capability remains an open question.
This is critical because it means we may be measuring the wrong thing. If AI’s contribution to biosecurity risk depends on its ability to support physical execution, not just information retrieval, then evaluations need to be designed accordingly. Risk forecasts that focus only on AI’s access to codified knowledge, rather than on how much of the tacit-knowledge barrier AI can actually erode, will systematically mis-measure the threat — and won’t provide a warning if and when AI tools start to acquire tacit knowledge.
Next in this series: the full chain of steps required to build a bioweapon, and an exploration of where AI can and cannot assist an attacker. Remember to read part one if you haven’t already!
Thanks to Steve Newman, Taren Stinebrickner-Kauffman, Mike Montague, Matt Sharkey, Gigi Gronvall, and David Manheim for suggestions and feedback.
A protocol in biology is a step-by-step instruction manual for a laboratory procedure. It lists what materials to use, how much of each, in what order, and for how long.
Exceptions exist. The final stage of making anthrax spores involves a dry powder. However, most steps are wet.
Aum Shinrikyo started with a non-virulent strain found in nature.
This assumes automated labs are properly regulated. While proper regulation is likely, the rules aren’t yet in place.