Why is everyone spooked by ai driven bioterrorism ? Like all of our water supplies are much easier to mess with compared to developing a virus or whatever that may or may not work
Evolution cannot optimize for maximum harm. Natural viruses only evolved to reproduce, with the harm just a side effect. Humans can combine features of multiple natural viruses. There's no obvious reason why we couldn't make something with both the infectivity of measles and the lethality of rabies, as well as resistance to all known antivirals. If something like that gets out it's probably civilization ending.
Maybe it's not "maximum harm" but there are selective pressures on zoonotic diseases to be harmful to organisms whose behavior is a threat to the ecosystem's stability. They fill a niche that guards against the emergence of monocultures.
Whether to be more afraid of nature's bioweapons, or human built ones, I don't know. But both exist.
But rabies is not lethal if not bitten (or scratched).
If an infected animal tries to kiss a person no danger whatsoever. If it tries to bite a person, bypass the skin, and the animal's saliva enters the bloodstream, it is only then that rabies lethality rate is relevant, but why should that be important? Do animals with sharp teeth pose to us any danger nowadays?
> Dog-mediated rabies kills tens of thousands [0] of people every year, many of whom are children. The disease is present in over 150 countries and territories across the globe
Measles is an airborne virus with extremely high infectivity, R0 estimated between 12 and 18, but it doesn't usually kill you.
Rabies infection is more difficult, but if you do get infected it's practically guaranteed to kill you without vaccination. It's likely possible to design viruses to resist vaccination too, e.g. splicing in human proteins to disguise them from the immune system, or using parts from a virus like HIV where we've failed to make a vaccine.
But the combination of both high infectivity and high lethality is an evolutionary dead end. Any virus that wipes out its host goes extinct too. A human-designed virus doesn't have this limitation. Look at what we've done with selective breeding in agriculture to see how much can be achieved when you optimize for things other than reproductive fitness. I'm certain that the range of physically possible viruses contains things far worse than anything naturally occurring.
Consider also that if somebody can automate virus design, they can scale it up and design hundreds of different ones. If they release them all at once humanity won't stand a chance.
If we had a pathogen-agnostic early warning system (what I'm working on: https://securebio.org/detection ) we would know. But we need a larger and more sensitive system than anything deployed today.
Assuming you have a medium city (half million people) and a suitably toxic substance (assuming something like dimethylmercury where you'd need at least 100mg per person => 50kg for the city), then you need to dump a lot of it into the water supply, because people use a lot more water than they actually ingest or even come into direct contact with (every person uses ~100l/day, but realistically ingest only like 2-5l).
So you'd need to dump literally a ton of an extremely toxic substance directly into the drinking water supply. I'm not concerned because that seems difficult enough (and likely to get the perpetrator caught beforehand while ordering mercury compounds by the barrel or sth).
For the purposes of terrorism and creating a crisis, you really don't need to kill literally everyone. A little contamination can go a long way toward causing a panic and lifetimes of distrust of tap water.
I'm not saying it's easy, I don't know that. Just that the bar isn't as high as the one you've chosen.
Terrorists generally want to actually kill people not just abstractly damage public trust in the water supply (or any other random public utility or good).
Sort of. Terrorists definitionally want to create terror to amplify the salience of the threat generally far beyond their actual capabilities.
Note that I did not say kill no one, just that it's not necessary to kill _everyone_ in order to create a crisis. It only takes a relatively small amount of injury and death to achieve the effect. The anthrax crisis in the early 2000s, for example, only had 22 confirmed infections but demonstrates the amplification of emotional salience very well.
They still do kill some people though and the threat of "random" death is where most of the terror comes from. They also generally just want to kill some because they really hate the group they're targeting too.
Especially because how hard it is to deal with the consequences. If a town's water supply is polluted you can shut it down and deliver water by truck until the situation is resolved. The consequences might be serious, but it would be a one-off event limited in time. Similarly, shutting down all freezers in a cyberattack would suck, but the consequences are limited in time
A new bioweapon let lose would be incredibly difficult to contain, and could potentially become something we have to contend with forever
A worrying trend to all of this is, if the attack vectors are becoming so strong, how do we stop it without a draconian centralisation of power? Though, to my mind, this is still deep in the realm of theoretical.
What makes you think we can stop it even with centralization of power?
The release of mirror life, or animal-borne pathogens with a long incubation period that infect humans, where the animals are in our cities — is pretty much game over.
I have been talking about this for over a decade now. No one seemed to care.
It’s bad enough that the current labs are speedrunning “gain of function” research online, with leaky containment
Giving everyone a simulated wuhan virus lab and hoping NO ONE will do anything bad with it is the stupidest idea ever. The chances of no one releasing anything with a long incubation period are effectively zero. And if anyone releases it, the chances of civilization surviving aren’t great.
All this so we can build slightly better local agents to send email, eh?
Aren't you skipping a few steps between a crazy person with no background in biological lab technique simulating viruses to actually releasing something?
Can AI even simulate a virus / protein folding / etc. with any degree of accuracy? or does it just orchestrate open source tools that were already available?
My grumpy take: because it's easier to displace fear towards something intangible than it is to actually go out and do something about the problems you have right now.
AI is already killing people, from denial of medical coverage to civilian casualties when an overwhelmed soldier just clicks "Ok". But worrying about AI viruses is easier than accepting that terrorist attacks are low-tech and, statistically speaking, the next one will probably be performed with a car, a gun, or a knife.
What's a better way that 1-3 people can kill literally hundreds of thousands (or millions, or billions) of people?
Convincing a lot of people to go kill other people is really hard. Doing it in secret is nearly impossible. But 1-3 people making a virus? Easier every day.
From what I hear "billions" is unlikely with current tech: with what we currently know, such a virus couldn't be designed to stay lethal and infectious on that scale, and that gets even harder if you want it to also resist mutations that might make it become less lethal, and also resisting mutations makes it easier for vaccines to protect against.
For millions, one failed Austrian painter did it by writing a book. From my understanding, it wasn't even considered a well-written book at the time, though my ability to read German is currently B1 (which is barely functional) and I wouldn't care to read that book anyway.
Hundreds of thousands, I can unfortunately think of a variety of ways; I hope they would not work outside of a film, but I will not suggest them on the off-chance they might work.
How many would you estimate Andrew Wakefield was responsible for?
World War 2/the Holocaust is absolutely not a story of some random guy deciding to write a book. The initial conditions and then the set of things that had to go right (wrong) are far, far, far beyond those required for the scenario we're talking about.
We do not need the entire global order configured into a specific point of political/economic/military tension, we do not need a media apparatus to fall into lock step, and we do not need a once-in-a-lifetime orator speaking to a recently disgraced industrial powerhouse nation.
The scenario we're talking about takes roughly: 1. Create pathogen, 2. Release it
Right now, Step 1 is very, very hard. Today it would take something closer to a Hitler to marshal the resources for such a project and not get stopped early on. But as the Step 1 becomes more accessible, the person required to achieve it, and the context in which a person could succeed, becomes much less special.
> Right now, Step 1 is very, very hard. Today it would take something closer to a Hitler to marshal the resources for such a project and not get stopped early on. But as the Step 1 becomes more accessible, the person required to achieve it, and the context in which a person could succeed, becomes much less special.
On the scale of "billions", right now it is impossible, not merely hard. As you say, it (likely) gets easier as the tech improves; myself, I am happy for things to get locked down ahead of time, but right now this wouldn't be possible for even a large and well-resourced racist government.
Tens of millions is possible, c.f. covid, albeit with essentially no control over who the victims will be, likewise c.f. covid.
Where are you getting this idea that a virus can't be engineered to be both highly contagious and lethal? Normally when I encounter this idea it's from people who are conflating a naturally evolved virus (in which this would be very hard/unlikely) with an engineered one. You've explicitly said you believe this to be impossible to engineer: why so?
> Where are you getting this idea that a virus can't be engineered to be both highly contagious and lethal?
From professional virologists discussing this specific risk.
The TL;DR (because I'm not a virologist myself, not just because this is a comment section) is that lethality works in opposition to all the things that make a pathogen more contagious, including long latency periods and choices of transmission vector and so on.
Even the worst case historical scenarios are things we now have a lot of tools for at the level of transmission vectors while we develop vaccines.
> impossible to engineer
Today. Impossible to engineer today. I would not wish to let anyone coming to the conversation late mistake this as an "impossible forever" kind of situation.
>For millions, one failed Austrian painter did it by writing a book. From my understanding,
You've all received memetic inoculations to that though. Unlikely, anyone will ever be able to write a book that does the same again. (Never mind that the book didn't do the damage, he did, the book was just a side effect. Never mind that it only worked in one peculiar economic environment which hasn't been repeated since. Never mind that zoomers are on the verge of never reading books again.)
Sorry if this seems obvious, but I grew up in the UK which liked to think it was "America's greatest ally", and I don't think this number is of the UK?
> You've all received memetic inoculations to that though. Unlikely, anyone will ever be able to write a book that does the same again.
I wish it were so, but even the memorial stones and constitutional bans on the party and its symbolism didn't completely stop the original memes, let alone things which rhyme with them: https://cybernews.com/security/afd-email-datat-leak-germany/
> (Never mind that the book didn't do the damage, he did, the book was just a side effect.
The book was kinda a big part of how he gained the power.
> Never mind that it only worked in one peculiar economic environment which hasn't been repeated since.
He's hardly been the only genocidal monster to rise to power, even in that century; just one of the more infamous ones.
> Never mind that zoomers are on the verge of never reading books again.)
So the next one only needs to tweet a lot? This isn't an improvement.
Didn't Russia march soldiers and tanks into Kherson...? At an unfathomably high cost of personnel?
(Russia's own stats agency says they've lost 450,000 soldiers in Ukraine – dead, not wounded – roughly equivalent to the number of American KIA across all of World War 2)
They did, but they lost Kherson after less than a year, back in November 2022, and the Ukrainian have since them controlled all of the western bank of the dnipro river.
But afterwards, the Russian have started a horrible drone bombing campaign from the other bank of the river, targeted on the civilians[1], with small quadcopeter dropping grenades on grannies walking their dogs or fpv drones blowing up civilians cars.
Contagiousness is less likely from a non-bio water supply risk. A bio attack could be even more pervasive and easier to carry out than an all water supplies at once one.
Are you actually asking this? Like why should people be worried that a disgruntled highly motivated person would venture into this area? Because it can happen. The outcomes of it could be really bad? It also doesn't have to be done by an individual, could be nation state or organization led. There are groups of people who believe humanity needs to end and all sorts of crazy stuff. Some of them have gone on mass murder suicide missions.
2 bad things can be threats at the same time. Even 10,000 things can be.
Because there are biological agents that can basically delete life on earth (okay maybe I'm exaggerating..), look at things like mirror life.
All it takes is one of these crypto/ai-grifter-30-under-30-chuckle fucks with more money than brains to put a lab together and create the actual torment nexus, and we are all fucked.
That seems a bit silly thinking they are not doing anything to protect the water supplies, and even more silly to not compare the impact of the things you are comparing. Poison a water supply and you hit how many, a city, a small part of a city? Now compare it to the impact of a virus like corona - we literally just went through that..
Instead of poisoning the water supply, you'd stuxnet the pumps. I imagine the network interfaces for status and control are about as well protected as grocery store freezers.
A bad pathogen could go world wide, but you could trash multiple cities with online attacks. No electricity, no water, no communications. Things would get bad.
Disrupting electric supply, water supply, and food supply chains are already tried and tested separately, with some limited monetary system disturbances, here and there. So doing those simultaneously - possibly leveraging on latent vulnerabilities kept covered for later abuse - sounds a natural step 'forward', close to trivial for those into that. Causing huge broad disturbances and possible unrest in societies. So that some governments started to ask people to build up reserves.
I can imagine that terrorist are sad now that they cannot use their preferred advanced bioengineering skills acquired through great efforts posing as scientist towards the so far unsuspecting services providing them with harmful DNA building blocks and now have to turn to much much more pedestrian methods instead of the heightened sophistication they crave.
Its hard to imagine a mustache twirling terrorist bent on putting PFAS and lead in the american water supply, putting mad cow in the beef supply and making everything cost more because he would have been outcompeted by the US government.
Have no idea what happens in the US but in Australia nearly all PFAS in the country comes from government properties, most often military bases or airports.
Plenty of blame should be reserved for the useful idiots high and low (and in the HN comments for that matter) who simped for the various policies that have poisoned ad impoverished us.
The evil billionares and megacorps as evil as they may be mostly don't give a shit about the minutia that people want to use to get one over on each other.
The regulatory state didn’t fail out of bureaucratic clumsiness; it was systematically bought and gutted so private balance sheets wouldn’t have to carry the cost of your cancer.
Blaming "the government" for toxic tap water and runaway prices is like blaming the getaway car instead of the bank robbers.
Surely the useful idiots who went to bat for every moronic expansion of the regulatory state regardless of cost (in opportunity and misery more than dollars TBH) because the "experts" said it made some random thing 1% better bear no responsibility for souring the public to the point it could be dismantled....
Private balance sheets wouldn’t have to carry the cost of your cancer is interesting. Do public balance sheets have to carry the cost of alcoholic's cirrhosis, smoker's lung cancer, and Type 2 diabetic's myriad of issues? Or does personal responsibility only apply to governments and corporations, not people?
>Blaming "the government" for toxic tap water and runaway prices is like blaming the getaway car instead of the bank robbers.
Blaming the government for toxic tap water is like blaming an organised private/public coalition of bank robbers for a bank robbery.
>it was systematically bought and gutted so private balance sheets wouldn’t have to carry the cost of your cancer.
The US Government, day 1, was bought and sold to defend the right to own slaves, even from the predation of states that had made it illegal. It's not some recent malformation that it just started defending the interests of rich people over poor people lmao. The explicit purpose of the US government is to do the shit it does. The machine has been designed well, not poorly.
Sure, and the US was also home to the first abolitionist society, and fought its bloodiest war in history (by % of population killed) in a war to end slavery.
"...this sparkling analysis from Wood (History/Brown Univ.; ed., The Rising Glory of America, 1971) impressively argues that [the American Revolution] was anything but conservative. Wood's contention that the Revolution was ``the most radical and far-reaching event in American history'' may stretch the point (did it really have more of an impact than the Civil War?). But from now on it will be hard to argue that the rebellion was a genteel event that left fundamental institutions unscathed. Wood pictures colonial society as overwhelmingly deferential—to king, to family patriarch, and to aristocrats—with ``personal obligations and reciprocity that ran through the whole society.'' But patriots such as Adams, Jefferson, and Franklin, aspiring to become gentlemen, resented this entrenched order of patronage and kinship. Their classical republicanism stressed benevolence and government by an enlightened elite. To their dismay, however, they discovered that their rhetoric unleashed all the latent entrepreneurial and egalitarian energies of American life, which even the elaborate mechanism of the Constitution could not completely contain. Among the results, Wood says, were a new concept of the dignity of labor, improvements in the lot of women, the first significant antislavery movement, and the frank acceptance of private interest underlying the political party system. Above all, Wood suggests, the Revolution produced the messy, fractious politics of liberal democracy, dominated by ordinary people pursuing commercial interests."
Recent history also proves that adherence to the constitution was mostly an honor system.
Rhode Island negotiated its Royal charter with King Charles in 1663 it has gender neutral language and freedom of thoughts. The best parts of the US constitution. Unfortunately it was also the center of slave trade.
>and fought its bloodiest war in history (by % of population killed) in a war to end slavery.
In a war to formalise slavery through less obvious means. Black Codes, Jim Crow whathaveyou.
The entire concept of pairing slave states with non-slave states tells you how on board the north was with slavery as long as it wasn't done near them.
If England hadn't shut down the trans-atlantic slave trade let's face it, the USA would probably still be doing it today.
And yet the Russian foreign policy is almost this: they know they can't compete on competence/science/tech so they try to tear everything down to their level.
I think this is a bad take. The Russian government wants to tear things down because they are a kleptocracy, and functional nations act as a check on corrupt governments. The more corrupt the US is the more theft russia can get away with and collude with pals inside the US gov on.
The framing of nations as inherently in competition with each other doesn’t track for me. Sometimes, in specific instances, nations will be in competition, but finland having a good or bad education system does not impact me even slightly.
> The framing of nations as inherently in competition with each other doesn’t track for me.
Where it tracks for you or not isn't relevant. It tracks for Putin and he's been extremely clear about that for hours and hours of extremely boring monologues :P
Nations in competition is a worldview of nationalists, dictators, and autocrats. I’m sure putin does talk about russia in competition with other nations all the time. He’s also a liar, who’ll stoke nationalism to bolster his power.
I don’t disagree they’re encouraging the collapse of other nations - i just disagree as to why
Sure, this is a common theme among the liberals who live far from war and poverty: never take any of those barbarians at their word when they say they believe something.
Personally I find it grating and it smells of racism.
They have placed agent Krasnov though, so they influence US politics anyway. What is even more surprising is that Krasnov is now doubling down, despite the clear loss in the latest election.
Your status as a "defender" is measured by the fanciness of the threats you defend against, not their actual magnitude. That's why Space Force is so popular, and everyone working on vaccines is miserable.
Also, see the "lab leak" discourse. To me the question seemed irrelevant because the countermeasures would be exactly the same, but that's not how other people think; they're not interested in a mask mandate against a naturally occuring virus, but if they can tie the exact same genetically identical virus to a human enemy? The social status of the threat goes up!
Made-up AI-assisted super-measles? Doesn't exist yet. But it's a great super sexy threat to defend against. You just have to ignore the kids dying of boring old regular naturally occuring measles in the meantime.
The countermeasures to prevent future lab leaks are not the same as the countermeasures to prevent natural viruses. We should take lab safety more seriously and stop enhancing the lethality of viruses regardless of what happened with Covid, but we have failed to do so, probably in part because of denial of the lab leak.
> they're not interested in a mask mandate against a naturally occuring virus, but if they can tie the exact same genetically identical virus to a human enemy? The social status of the threat goes up!
Oh, that's what it is? I never had a model for such thinking.
I'm curious if this is related to some things the top AI models are uncovering.
We know we don't get to see the bio side of the top models. They're out there solving hard maths problems etc but a query with the word 'microbes' in it often kicks you back to a smaller model automatically.
Are the bleeding edge models finding something scary enough that can be done with synthetic DNA that it needs to be restricted asap?
Well look at what it's doing to the hacking scene. It doesn't have to do all the work for you but people knowledgeable and motivated enough are now 10-100x more of a nuisance.
It's not like you can prompt it to create super AIDS in an afternoon but I'm sure an uncensored model could help you do real harm.
It doesn't seem to be? From what I can tell, this is just formalising what we've had to self-declare with companies like IDT for years (not encoding a known toxin, pathogen, etc). Somebody correct my if I'm wrong, but it looks like there's now a requirement for companies to actually check the sequence, rather than take the researcher's word for it.
You don't need AI to generate known toxin or pathogen sequences (as these are already known).
And if AI could design novel toxins, this screening wouldn't catch those anyway.
IDT, Twist et al do check sequences and this has been trivial for a long time. I’ve had to assure account managers that my viral sequences are legitimate as far back as 2019, but I didn’t work with viruses before then so I can’t say how new that was at the time
Possibly, with alphafold? Still tricky and unreliable, though, and let's face it (as TFA acknowledges in the quotes), this would be trivial to workaround by simply generating the full sequence in fragments.
The "obvious" move will merely automate what is already a highly automated process.
LLMs don't understand a thing about DNA that humans don't. Even what are frequently called "foundation models" in bio, that have been trained on sequence data, have results indistinguishable from replace model weights with random numbers. Because "training" here doesnt have the same sort of complex internal function that something like proteins structure prediction has; what you're training for is mere sequence similarity to known pathogens.
I mean... if that were true then nobody would be worried about AI generated pathogens. You can't simultaneously have "AI is smart enough to design dangerous pathogens" and "AI isn't smart enough to recognise dangerous pathogens".
Lots of scary stuff has been possible for ages. Much of it is so easy that wouldn't be publishable because of lack of novelty.
I see LLMs as being essentially irrelevant for whether someone can do this. It's lab skills, planning skills, biological experiment debugging skills.
"Why did my plasmid not start getting expressed, how did I screw up?" "How do I make this huge protein" aren't LLM questions, they're university level biology questions, and "why is my protein growth tank not making the thing I want, and what is the grey stuff, do I need to put clean it again" is an experience question.
Putting checks on DNA synthesis puts making the horrible stuff out of the range of what high schoolers can do though.
>Those vendors must verify their customers’ identities and screen orders for potentially dangerous sequences that, for example, could encode toxins or pathogens.
I'd assumed they already do this, and if you order Smallpox primers or w/e from Sigma/IDT you get a visit?
I have a home lab; wonder if there will be practical effects. Also have a personal project where I'm exploring enzymatic oligo synthesis. (Different set of pros/cons from the traditional phosphoramidite synthesis)
Probably a good thing overall in my opinion, even though it obviously has impacts.
I got into this hobby briefly (pre-LLMs) before realising how much of a loaded gun it is, and that I’m just not qualified to safely muck around with this stuff.
I’m curious about the hobby itself. What are you learning and enjoying? I mean that genuinely as a question, not sarcasm or trolling. This is fascinating to my 8 year old self who got a microscope for christmas and always wanted to explore more
The 0.01% might survive, even develop some resistence. But when put again in the mix with wild non-resistent individuals in a non-clorox environment, the selection acts against them: having resistence might have costs, have no use, and if the mechanism is lost for whatever reason the new individuals survive anyway.
Something like this is suspected to have happened to us humans: the black plague has been pressuring/selecting individuals with more active and strong immune systems. Today the plague isn't so active or deadly to humans, but stronger immune systems are probably behind auto-immune illnesses (https://aihep.org/post/the-bubonic-plague-and-autoimmune-dis...).
You can actually do real biology at home if you set up a Petri dish tray with a concentration gradient of Clorox across it, then put some random germs at the low concentration and.
The device used to make oligonucleotides is pretty much an inkjet printer, from there it's cleverly applied enzymes to assemble the full strand. If you've got the skills and determination to get the synthetic DNA into a functioning cell, I don't think limiting access to synthetic DNA in the first place is going to stop you.
The device used to make oligonucleotides is pretty much an inkjet printer, from there it's cleverly applied enzymes to assemble the full strand.
Usually the synthesis of oligonuclotides is done with phosphoramidite chemistry [0]. So tell us more about those enzymes to make DNA molecules from scratch.
And yes, phosphoramidites are what you use as ink. You print many many dots on a flat surface, and then you print over the same dots again and again, building up layers one base at a time. Then you dissolve those dots, giving you short strands in solution, which can be assembled into longer strands by being strategic about what else is in that solution and about which sequences you programmed in to the printer.
I've been considering making one in my garage. I don't think it's going to be easy, but I also don't think it's going to be difficult enough to be a good candidate for use as a regulatory chokepoint. We're not talking about a warehouse full of centrifuges here, it's a printer with special ink.
I’m pretty sure they were talking about how you can combine synthesized oligos into longer fragments when you are starting with smaller oligos. I’m guessing that they were referring to using different restriction enzymes from that point on.
Unless we are talking about really long oligos (which is harder and $$$), you’ll need some post-synthesis enzymatic work to get your full strand.
Yes I also read the comment that way. The inkjet printer remark presumably refers to benchtop DNA synthesis machines which have been commercially available for a while now. (The linked article mentions the possibility of buy-back programs for old machines that don't enforce any restrictions.)
I guess it's not a bad idea to put basic hurdles in place but it would be a serious mistake to see this as any sort of comprehensive preventative. DNA just isn't difficult to manipulate.
The vast majority of commercial synthesizers aren't inkjet. The OG DNA synthesizers are pretty simple - just a handful of pumps and valves and a packed silica column for each run. Anyone with a little mechanical engineering and organic chemistry training could build one (lots of companies in the past did just this.)
Kind of like owning your own darkroom, back in the day of analog photography.
Seeing his adjacent comment I stand corrected. I was not aware that there are now literal DNA printers. Only that you could obtain the sort of synthesizers to which you refer which had approximately the form factor of an inkjet (ie a small box).
The thing is DNA just also isn't dangerous. It's fragile and everything in the environment destroys it - as noted it's literally a everything else which is the hard part.
This reeks of the Longtermists getting their influence into government. They were doing an entire round on YouTube about a year ago.
Ansa Bio uses an engineered terminal deoxynucleotidyl transferase (TdT) enzyme to synthesize the oligos themselves, rather than using phosphoramidite chemistry. Each enzyme is tethered to a single nucleotide, allowing it to extend the DNA strand by exactly one base. The tether is then cleaved and the cycle repeats.
This works in aqueous solution under mild conditions and enables synthesis of much longer oligos (~1,000 bases), which can subsequently be assembled into 50 kbp constructs.
But Esvelt points out the policy only requires screening for DNA sequences 50 base pairs or longer. A would-be bioterrorist, he argues, could easily stitch together a dangerous sequence from much smaller fragments ordered from multiple different suppliers. Esvelt says he has built an algorithm capable of screening sequences as short as 30 base pairs for their risk potential and made it freely available. “That’s the main flaw in an otherwise pretty great policy,” he says. “Don’t bother building a fence if there’s going to be an obvious huge gap in it.”
So, a song and dance of doing something that will most likely just inconvenience people doing real work.
Also, DNA chemistry is very well established. Someone with the appropriate background could easily assemble oligos without too much difficulty. I guess this stops Little Johnny from printing ebola from the internet.
>So, a song and dance of doing something that will most likely just inconvenience people doing real work.
Why does he call it an "otherwise pretty great policy"?
He's claiming that the policy needs to be more comprehensive. You're concluding that it doesn't pass a cost/benefit test. That's not at all the same claim!
> U.S. government–funded scientists who make and manipulate DNA sequences will face tougher regulations
Seems like this has little to do with deterring terrorism then, considering the assumption that a terrorist may not be a US government funded scientist.
My guess is that the assumption is that US gov funded labs are enough of many suppliers revenue that it'll be effective, but I'm 100% speculating and it may be total garbage speculation given that I have no idea how much hobbyist level DNA projects are a thing or how much they're a thing for non-medicinal business purposes.
If you read the next two sentences it makes it fairly obvious that this is a plan to indirectly target suppliers. Federally funded researchers will be required to
"procure synthetic nucleic acids and benchtop synthesis equipment from U.S.-approved suppliers. Those vendors must verify their customers’ identities and screen orders for potentially dangerous sequences that, for example, could encode toxins or pathogens."
If this was just about feds self-policing, they wouldn't have the customer identity requirement.
Probably they're taking this route because AI is moving fast and Congress moves slowly.
But yes, this is stupid. No one who is inclined to build a biological WMD is going to be more than mildly inconvenienced by supply restrictions. DNA isn't plutonium.
Sounds like a fully general argument against any sort of security measure. "A dedicated thief won't be deterred by a lock or burglar alarm." Sure. I'm still buying that stuff.
Erm. Terrorists are generally full of DNA. How difficult is it to break that down, sort and purify on a scale from vinegar and soda to bioterrorist with access to a DNA printer?
Talk about a useless policy. It just says federally funded scientists need to buy gear from US companies and register. I feel like I can confidently say whoever makes the first purpose built megavirus is not going to be federally registered and purchasing equipment through the property channels. This is security theater, but unlike TSA this is security theater with existential stakes.
So continue to fund bioweapons programs under the guise of medical research while also stifling individual and independent research under the guise of protecting us from terrorism? Yeah, that sounds like the US government.
This is not unlike 3D printing: "What? You can print a gun? Into jail you go!".
When you then look at the mafia in charge of the USA right now, run by oligarchs and other shady criminals, you ultimately have an apartheid system at work now, because these top-down pseudo-kings can control what others can do. Everyone is monitored and a potential terrorist now. Now imagine that DNA can be patented (I don't refer to right now, but look how ESTs (https://en.wikipedia.org/wiki/Expressed_sequence_tag) in the 1990s could be patented) - and suddenly you will have a state-run mafia decide which monopolist can yield de-facto control, with a few minor companies roleplaying the "it's still capitalism!!!" when in reality it no longer is.
Why is everyone spooked by ai driven bioterrorism ? Like all of our water supplies are much easier to mess with compared to developing a virus or whatever that may or may not work
Evolution cannot optimize for maximum harm. Natural viruses only evolved to reproduce, with the harm just a side effect. Humans can combine features of multiple natural viruses. There's no obvious reason why we couldn't make something with both the infectivity of measles and the lethality of rabies, as well as resistance to all known antivirals. If something like that gets out it's probably civilization ending.
Maybe it's not "maximum harm" but there are selective pressures on zoonotic diseases to be harmful to organisms whose behavior is a threat to the ecosystem's stability. They fill a niche that guards against the emergence of monocultures.
Whether to be more afraid of nature's bioweapons, or human built ones, I don't know. But both exist.
But rabies is not lethal if not bitten (or scratched).
If an infected animal tries to kiss a person no danger whatsoever. If it tries to bite a person, bypass the skin, and the animal's saliva enters the bloodstream, it is only then that rabies lethality rate is relevant, but why should that be important? Do animals with sharp teeth pose to us any danger nowadays?
Yes:
> Dog-mediated rabies kills tens of thousands [0] of people every year, many of whom are children. The disease is present in over 150 countries and territories across the globe
https://www.who.int/data/gho/data/themes/topics/Rabies
0 : Around 15k in 2023 but the trends continue to go down.
https://ourworldindata.org/grapher/incidence-and-prevalence-...
I know of a building that lacks an elevator. In fact, I can spot numberless buildings that lack elevators.
This however does not prove that no building has an elevator, nor that a building cannot be built with an elevator.
The risks for bioterrorism are not limited by what commonly exists, but what can exist, or what can be built to exist.
Measles is an airborne virus with extremely high infectivity, R0 estimated between 12 and 18, but it doesn't usually kill you.
Rabies infection is more difficult, but if you do get infected it's practically guaranteed to kill you without vaccination. It's likely possible to design viruses to resist vaccination too, e.g. splicing in human proteins to disguise them from the immune system, or using parts from a virus like HIV where we've failed to make a vaccine.
But the combination of both high infectivity and high lethality is an evolutionary dead end. Any virus that wipes out its host goes extinct too. A human-designed virus doesn't have this limitation. Look at what we've done with selective breeding in agriculture to see how much can be achieved when you optimize for things other than reproductive fitness. I'm certain that the range of physically possible viruses contains things far worse than anything naturally occurring.
Consider also that if somebody can automate virus design, they can scale it up and design hundreds of different ones. If they release them all at once humanity won't stand a chance.
did you miss the infectivity of measles part of their comment? Rabies with a different infection vector would be catastrophic.
With a nice and long incubation period. We wouldn't even know what happened util *pop*
If we had a pathogen-agnostic early warning system (what I'm working on: https://securebio.org/detection ) we would know. But we need a larger and more sensitive system than anything deployed today.
Rabies has documented examples of incubation period in years/decades, does it not?
Water supply is not that easy to mess with.
Assuming you have a medium city (half million people) and a suitably toxic substance (assuming something like dimethylmercury where you'd need at least 100mg per person => 50kg for the city), then you need to dump a lot of it into the water supply, because people use a lot more water than they actually ingest or even come into direct contact with (every person uses ~100l/day, but realistically ingest only like 2-5l).
So you'd need to dump literally a ton of an extremely toxic substance directly into the drinking water supply. I'm not concerned because that seems difficult enough (and likely to get the perpetrator caught beforehand while ordering mercury compounds by the barrel or sth).
For the purposes of terrorism and creating a crisis, you really don't need to kill literally everyone. A little contamination can go a long way toward causing a panic and lifetimes of distrust of tap water.
I'm not saying it's easy, I don't know that. Just that the bar isn't as high as the one you've chosen.
> I'm not saying it's easy
Correct, but the point here is that bioweapons should be higher priority because they're easier, cheaper and more effective than chemicals.
Terrorists generally want to actually kill people not just abstractly damage public trust in the water supply (or any other random public utility or good).
Sort of. Terrorists definitionally want to create terror to amplify the salience of the threat generally far beyond their actual capabilities.
Note that I did not say kill no one, just that it's not necessary to kill _everyone_ in order to create a crisis. It only takes a relatively small amount of injury and death to achieve the effect. The anthrax crisis in the early 2000s, for example, only had 22 confirmed infections but demonstrates the amplification of emotional salience very well.
They don't usually kill people very efficiently either.
They still do kill some people though and the threat of "random" death is where most of the terror comes from. They also generally just want to kill some because they really hate the group they're targeting too.
Because it massively raises the amount of damage that a lone-wolf crazy person can do.
Especially because how hard it is to deal with the consequences. If a town's water supply is polluted you can shut it down and deliver water by truck until the situation is resolved. The consequences might be serious, but it would be a one-off event limited in time. Similarly, shutting down all freezers in a cyberattack would suck, but the consequences are limited in time
A new bioweapon let lose would be incredibly difficult to contain, and could potentially become something we have to contend with forever
A worrying trend to all of this is, if the attack vectors are becoming so strong, how do we stop it without a draconian centralisation of power? Though, to my mind, this is still deep in the realm of theoretical.
Who's to say the centralized powers aren't strong threat vectors? Likelihood of lab escape is much higher than intentional attack anyways.
Fair enough. I guess this is analogous to concerns about nuclear terrorism.
Who's to say the two negate each other?
The comment said how do we stop it without draconian centralized power. If the centralized power is also a threat, then the threat persists.
What makes you think we can stop it even with centralization of power?
The release of mirror life, or animal-borne pathogens with a long incubation period that infect humans, where the animals are in our cities — is pretty much game over.
I have been talking about this for over a decade now. No one seemed to care.
I don't get how bioterrorism favors a lone wolf attack.
It’s bad enough that the current labs are speedrunning “gain of function” research online, with leaky containment
Giving everyone a simulated wuhan virus lab and hoping NO ONE will do anything bad with it is the stupidest idea ever. The chances of no one releasing anything with a long incubation period are effectively zero. And if anyone releases it, the chances of civilization surviving aren’t great.
All this so we can build slightly better local agents to send email, eh?
Aren't you skipping a few steps between a crazy person with no background in biological lab technique simulating viruses to actually releasing something?
Can AI even simulate a virus / protein folding / etc. with any degree of accuracy? or does it just orchestrate open source tools that were already available?
I know absolutely nothing about this subject.
But if it's anything like materials design, better computational simulation is one tiny part of the RnD process.
A more credible threat would be a 1-man lab, with a fully automated setup, continuously working on a bioweapon.
But any hosted AI provider would immediately flag that. The risk would be local, abliterated models.
Not saying that the situation is good, but, am I missing something?
My grumpy take: because it's easier to displace fear towards something intangible than it is to actually go out and do something about the problems you have right now.
AI is already killing people, from denial of medical coverage to civilian casualties when an overwhelmed soldier just clicks "Ok". But worrying about AI viruses is easier than accepting that terrorist attacks are low-tech and, statistically speaking, the next one will probably be performed with a car, a gun, or a knife.
What if the white supremacists make a retro virus, that for instance only kills people with sickle cell anemia..
That doesn't really engage with "there's easier ways to attack" or "and this tech is currently hard enough this is not a realistic attack".
That said:
What if a different group near simultaneously makes one targeting people with no copies of the HbS/HbA gene?
(One copy: resist malaria; two copies: sickle cell anaemia)
What's a better way that 1-3 people can kill literally hundreds of thousands (or millions, or billions) of people?
Convincing a lot of people to go kill other people is really hard. Doing it in secret is nearly impossible. But 1-3 people making a virus? Easier every day.
From what I hear "billions" is unlikely with current tech: with what we currently know, such a virus couldn't be designed to stay lethal and infectious on that scale, and that gets even harder if you want it to also resist mutations that might make it become less lethal, and also resisting mutations makes it easier for vaccines to protect against.
For millions, one failed Austrian painter did it by writing a book. From my understanding, it wasn't even considered a well-written book at the time, though my ability to read German is currently B1 (which is barely functional) and I wouldn't care to read that book anyway.
Hundreds of thousands, I can unfortunately think of a variety of ways; I hope they would not work outside of a film, but I will not suggest them on the off-chance they might work.
How many would you estimate Andrew Wakefield was responsible for?
World War 2/the Holocaust is absolutely not a story of some random guy deciding to write a book. The initial conditions and then the set of things that had to go right (wrong) are far, far, far beyond those required for the scenario we're talking about.
We do not need the entire global order configured into a specific point of political/economic/military tension, we do not need a media apparatus to fall into lock step, and we do not need a once-in-a-lifetime orator speaking to a recently disgraced industrial powerhouse nation.
The scenario we're talking about takes roughly: 1. Create pathogen, 2. Release it
Right now, Step 1 is very, very hard. Today it would take something closer to a Hitler to marshal the resources for such a project and not get stopped early on. But as the Step 1 becomes more accessible, the person required to achieve it, and the context in which a person could succeed, becomes much less special.
> Right now, Step 1 is very, very hard. Today it would take something closer to a Hitler to marshal the resources for such a project and not get stopped early on. But as the Step 1 becomes more accessible, the person required to achieve it, and the context in which a person could succeed, becomes much less special.
On the scale of "billions", right now it is impossible, not merely hard. As you say, it (likely) gets easier as the tech improves; myself, I am happy for things to get locked down ahead of time, but right now this wouldn't be possible for even a large and well-resourced racist government.
Tens of millions is possible, c.f. covid, albeit with essentially no control over who the victims will be, likewise c.f. covid.
Where are you getting this idea that a virus can't be engineered to be both highly contagious and lethal? Normally when I encounter this idea it's from people who are conflating a naturally evolved virus (in which this would be very hard/unlikely) with an engineered one. You've explicitly said you believe this to be impossible to engineer: why so?
> Where are you getting this idea that a virus can't be engineered to be both highly contagious and lethal?
From professional virologists discussing this specific risk.
The TL;DR (because I'm not a virologist myself, not just because this is a comment section) is that lethality works in opposition to all the things that make a pathogen more contagious, including long latency periods and choices of transmission vector and so on.
Even the worst case historical scenarios are things we now have a lot of tools for at the level of transmission vectors while we develop vaccines.
> impossible to engineer
Today. Impossible to engineer today. I would not wish to let anyone coming to the conversation late mistake this as an "impossible forever" kind of situation.
>For millions, one failed Austrian painter did it by writing a book. From my understanding,
You've all received memetic inoculations to that though. Unlikely, anyone will ever be able to write a book that does the same again. (Never mind that the book didn't do the damage, he did, the book was just a side effect. Never mind that it only worked in one peculiar economic environment which hasn't been repeated since. Never mind that zoomers are on the verge of never reading books again.)
50% of America and 80% of America's greatest ally currently support the extermination of at least one race.
> 80% of America's greatest ally
Who?
Sorry if this seems obvious, but I grew up in the UK which liked to think it was "America's greatest ally", and I don't think this number is of the UK?
> You've all received memetic inoculations to that though. Unlikely, anyone will ever be able to write a book that does the same again.
I wish it were so, but even the memorial stones and constitutional bans on the party and its symbolism didn't completely stop the original memes, let alone things which rhyme with them: https://cybernews.com/security/afd-email-datat-leak-germany/
> (Never mind that the book didn't do the damage, he did, the book was just a side effect.
The book was kinda a big part of how he gained the power.
> Never mind that it only worked in one peculiar economic environment which hasn't been repeated since.
He's hardly been the only genocidal monster to rise to power, even in that century; just one of the more infamous ones.
> Never mind that zoomers are on the verge of never reading books again.)
So the next one only needs to tweet a lot? This isn't an improvement.
This happens in Deep Space 9.
Ferengitis?
No, the section 31 virus against the founders.
In case anyone wonder what op means - lyfgenia is a virus for treating sickle cell, a disease thats more common among black population.
TIL God is a white supremacist
Not to mention causing a famine or lack of medicine by sabotaging the cold chain.
Let alone drone-based terrorism. The Russians have shown in Kherson that you can use those to depopulate an entire city on a budget.
Didn't Russia march soldiers and tanks into Kherson...? At an unfathomably high cost of personnel?
(Russia's own stats agency says they've lost 450,000 soldiers in Ukraine – dead, not wounded – roughly equivalent to the number of American KIA across all of World War 2)
They did, but they lost Kherson after less than a year, back in November 2022, and the Ukrainian have since them controlled all of the western bank of the dnipro river.
But afterwards, the Russian have started a horrible drone bombing campaign from the other bank of the river, targeted on the civilians[1], with small quadcopeter dropping grenades on grannies walking their dogs or fpv drones blowing up civilians cars.
[1]: https://en.wikipedia.org/wiki/Human_safari_(terror_campaign)
Contagiousness is less likely from a non-bio water supply risk. A bio attack could be even more pervasive and easier to carry out than an all water supplies at once one.
Are you actually asking this? Like why should people be worried that a disgruntled highly motivated person would venture into this area? Because it can happen. The outcomes of it could be really bad? It also doesn't have to be done by an individual, could be nation state or organization led. There are groups of people who believe humanity needs to end and all sorts of crazy stuff. Some of them have gone on mass murder suicide missions.
2 bad things can be threats at the same time. Even 10,000 things can be.
Because there are biological agents that can basically delete life on earth (okay maybe I'm exaggerating..), look at things like mirror life.
All it takes is one of these crypto/ai-grifter-30-under-30-chuckle fucks with more money than brains to put a lab together and create the actual torment nexus, and we are all fucked.
That seems a bit silly thinking they are not doing anything to protect the water supplies, and even more silly to not compare the impact of the things you are comparing. Poison a water supply and you hit how many, a city, a small part of a city? Now compare it to the impact of a virus like corona - we literally just went through that..
Instead of poisoning the water supply, you'd stuxnet the pumps. I imagine the network interfaces for status and control are about as well protected as grocery store freezers.
A bad pathogen could go world wide, but you could trash multiple cities with online attacks. No electricity, no water, no communications. Things would get bad.
Coronavirus++ is not something that I would want on my calendar
I'm not too worried. That version of the virus only gets more dangerous after being cleared.
++coronavirus though...
Yes, think of the null pointer dereferences, but now inside your body.
Lack of true biology training and lots of doomer scifi reading.
You can order a jug of PFAS for a couple of bucks off amazon.
Disrupting electric supply, water supply, and food supply chains are already tried and tested separately, with some limited monetary system disturbances, here and there. So doing those simultaneously - possibly leveraging on latent vulnerabilities kept covered for later abuse - sounds a natural step 'forward', close to trivial for those into that. Causing huge broad disturbances and possible unrest in societies. So that some governments started to ask people to build up reserves.
I can imagine that terrorist are sad now that they cannot use their preferred advanced bioengineering skills acquired through great efforts posing as scientist towards the so far unsuspecting services providing them with harmful DNA building blocks and now have to turn to much much more pedestrian methods instead of the heightened sophistication they crave.
Or to sources from other countries.
Its hard to imagine a mustache twirling terrorist bent on putting PFAS and lead in the american water supply, putting mad cow in the beef supply and making everything cost more because he would have been outcompeted by the US government.
The US Government doesn't do these things. The people running businesses do these things.
Have no idea what happens in the US but in Australia nearly all PFAS in the country comes from government properties, most often military bases or airports.
Plenty of blame should be reserved for the useful idiots high and low (and in the HN comments for that matter) who simped for the various policies that have poisoned ad impoverished us.
The evil billionares and megacorps as evil as they may be mostly don't give a shit about the minutia that people want to use to get one over on each other.
Sure, that's why my water supply is contaminated by pfas from the Air Force base nearby washing it into the water table for decades.
The regulatory state didn’t fail out of bureaucratic clumsiness; it was systematically bought and gutted so private balance sheets wouldn’t have to carry the cost of your cancer.
Blaming "the government" for toxic tap water and runaway prices is like blaming the getaway car instead of the bank robbers.
Surely the useful idiots who went to bat for every moronic expansion of the regulatory state regardless of cost (in opportunity and misery more than dollars TBH) because the "experts" said it made some random thing 1% better bear no responsibility for souring the public to the point it could be dismantled....
Private balance sheets wouldn’t have to carry the cost of your cancer is interesting. Do public balance sheets have to carry the cost of alcoholic's cirrhosis, smoker's lung cancer, and Type 2 diabetic's myriad of issues? Or does personal responsibility only apply to governments and corporations, not people?
Yes? Remember, the government decided alcohol and tobacco could advertise the way they do because it helps them make money.
>Blaming "the government" for toxic tap water and runaway prices is like blaming the getaway car instead of the bank robbers.
Blaming the government for toxic tap water is like blaming an organised private/public coalition of bank robbers for a bank robbery.
>it was systematically bought and gutted so private balance sheets wouldn’t have to carry the cost of your cancer.
The US Government, day 1, was bought and sold to defend the right to own slaves, even from the predation of states that had made it illegal. It's not some recent malformation that it just started defending the interests of rich people over poor people lmao. The explicit purpose of the US government is to do the shit it does. The machine has been designed well, not poorly.
Sure, and the US was also home to the first abolitionist society, and fought its bloodiest war in history (by % of population killed) in a war to end slavery.
"...this sparkling analysis from Wood (History/Brown Univ.; ed., The Rising Glory of America, 1971) impressively argues that [the American Revolution] was anything but conservative. Wood's contention that the Revolution was ``the most radical and far-reaching event in American history'' may stretch the point (did it really have more of an impact than the Civil War?). But from now on it will be hard to argue that the rebellion was a genteel event that left fundamental institutions unscathed. Wood pictures colonial society as overwhelmingly deferential—to king, to family patriarch, and to aristocrats—with ``personal obligations and reciprocity that ran through the whole society.'' But patriots such as Adams, Jefferson, and Franklin, aspiring to become gentlemen, resented this entrenched order of patronage and kinship. Their classical republicanism stressed benevolence and government by an enlightened elite. To their dismay, however, they discovered that their rhetoric unleashed all the latent entrepreneurial and egalitarian energies of American life, which even the elaborate mechanism of the Constitution could not completely contain. Among the results, Wood says, were a new concept of the dignity of labor, improvements in the lot of women, the first significant antislavery movement, and the frank acceptance of private interest underlying the political party system. Above all, Wood suggests, the Revolution produced the messy, fractious politics of liberal democracy, dominated by ordinary people pursuing commercial interests."
https://www.kirkusreviews.com/book-reviews/gordon-s-wood/the...
Recent history also proves that adherence to the constitution was mostly an honor system. Rhode Island negotiated its Royal charter with King Charles in 1663 it has gender neutral language and freedom of thoughts. The best parts of the US constitution. Unfortunately it was also the center of slave trade.
Commerce gets in the way of ideals.
>and fought its bloodiest war in history (by % of population killed) in a war to end slavery.
In a war to formalise slavery through less obvious means. Black Codes, Jim Crow whathaveyou.
The entire concept of pairing slave states with non-slave states tells you how on board the north was with slavery as long as it wasn't done near them.
If England hadn't shut down the trans-atlantic slave trade let's face it, the USA would probably still be doing it today.
You're making things up. Have you ever heard the word "Reconstruction" before?
The US voluntary shut down transatlantic imports long before the Civil War: https://en.wikipedia.org/wiki/Act_Prohibiting_Importation_of...
The north thought that slavery would naturally wither away, and went to war after concluding that it would not.
Other major slave economies in the Western Hemisphere (Brazil and Cuba) outlawed slavery many years after the US did. Parts of Africa did not abolish until the 1900s: https://old.reddit.com/r/MapPorn/comments/zsnr8q/year_of_abo...
And yet the Russian foreign policy is almost this: they know they can't compete on competence/science/tech so they try to tear everything down to their level.
Russia has always been famous for science and technology, if not political competence.
The jewish exodus from Russia due to the disaster of the Soviet era put a dent in that I'm sure.
Also the separation from Ukraine, who were a major contributor to Soviet science.
The name of that entity was the soviet union
I think this is a bad take. The Russian government wants to tear things down because they are a kleptocracy, and functional nations act as a check on corrupt governments. The more corrupt the US is the more theft russia can get away with and collude with pals inside the US gov on.
The framing of nations as inherently in competition with each other doesn’t track for me. Sometimes, in specific instances, nations will be in competition, but finland having a good or bad education system does not impact me even slightly.
>finland having a good or bad education system does not impact me even slightly.
Yet you do depend on Linux.
And SSH.
Is *sshole the result of Finland failing to compete with Russia? Curious minds need to know!
> The framing of nations as inherently in competition with each other doesn’t track for me.
Where it tracks for you or not isn't relevant. It tracks for Putin and he's been extremely clear about that for hours and hours of extremely boring monologues :P
Nations in competition is a worldview of nationalists, dictators, and autocrats. I’m sure putin does talk about russia in competition with other nations all the time. He’s also a liar, who’ll stoke nationalism to bolster his power.
I don’t disagree they’re encouraging the collapse of other nations - i just disagree as to why
Sure, this is a common theme among the liberals who live far from war and poverty: never take any of those barbarians at their word when they say they believe something.
Personally I find it grating and it smells of racism.
They have placed agent Krasnov though, so they influence US politics anyway. What is even more surprising is that Krasnov is now doubling down, despite the clear loss in the latest election.
Your status as a "defender" is measured by the fanciness of the threats you defend against, not their actual magnitude. That's why Space Force is so popular, and everyone working on vaccines is miserable.
Also, see the "lab leak" discourse. To me the question seemed irrelevant because the countermeasures would be exactly the same, but that's not how other people think; they're not interested in a mask mandate against a naturally occuring virus, but if they can tie the exact same genetically identical virus to a human enemy? The social status of the threat goes up!
Regular old measles? Not sexy. But a convenient vector for the internal threat. https://www.politico.com/news/2026/11/08/americas-measles-ep...
Made-up AI-assisted super-measles? Doesn't exist yet. But it's a great super sexy threat to defend against. You just have to ignore the kids dying of boring old regular naturally occuring measles in the meantime.
The countermeasures to prevent future lab leaks are not the same as the countermeasures to prevent natural viruses. We should take lab safety more seriously and stop enhancing the lethality of viruses regardless of what happened with Covid, but we have failed to do so, probably in part because of denial of the lab leak.
> they're not interested in a mask mandate against a naturally occuring virus, but if they can tie the exact same genetically identical virus to a human enemy? The social status of the threat goes up!
Oh, that's what it is? I never had a model for such thinking.
I'm curious if this is related to some things the top AI models are uncovering.
We know we don't get to see the bio side of the top models. They're out there solving hard maths problems etc but a query with the word 'microbes' in it often kicks you back to a smaller model automatically.
Are the bleeding edge models finding something scary enough that can be done with synthetic DNA that it needs to be restricted asap?
Well look at what it's doing to the hacking scene. It doesn't have to do all the work for you but people knowledgeable and motivated enough are now 10-100x more of a nuisance.
It's not like you can prompt it to create super AIDS in an afternoon but I'm sure an uncensored model could help you do real harm.
It's more likely they're surfacing the contents of deeply technical biology papers to average people who may or may not be thinking about terrorism.
It doesn't seem to be? From what I can tell, this is just formalising what we've had to self-declare with companies like IDT for years (not encoding a known toxin, pathogen, etc). Somebody correct my if I'm wrong, but it looks like there's now a requirement for companies to actually check the sequence, rather than take the researcher's word for it.
You don't need AI to generate known toxin or pathogen sequences (as these are already known).
And if AI could design novel toxins, this screening wouldn't catch those anyway.
IDT, Twist et al do check sequences and this has been trivial for a long time. I’ve had to assure account managers that my viral sequences are legitimate as far back as 2019, but I didn’t work with viruses before then so I can’t say how new that was at the time
Unless you use AI for the screening? Surely that's the obvious move.
Possibly, with alphafold? Still tricky and unreliable, though, and let's face it (as TFA acknowledges in the quotes), this would be trivial to workaround by simply generating the full sequence in fragments.
The "obvious" move will merely automate what is already a highly automated process.
LLMs don't understand a thing about DNA that humans don't. Even what are frequently called "foundation models" in bio, that have been trained on sequence data, have results indistinguishable from replace model weights with random numbers. Because "training" here doesnt have the same sort of complex internal function that something like proteins structure prediction has; what you're training for is mere sequence similarity to known pathogens.
I mean... if that were true then nobody would be worried about AI generated pathogens. You can't simultaneously have "AI is smart enough to design dangerous pathogens" and "AI isn't smart enough to recognise dangerous pathogens".
What fraction of people that are worried about AI designing pathogens have any evidence that AI can design pathogens?
The answer is none. It's speculated risk, potential risk. Not a demonstrated risk.
So not these guys then?
https://www.bbc.co.uk/news/articles/c5y3j3ngevmo
Lots of scary stuff has been possible for ages. Much of it is so easy that wouldn't be publishable because of lack of novelty.
I see LLMs as being essentially irrelevant for whether someone can do this. It's lab skills, planning skills, biological experiment debugging skills.
"Why did my plasmid not start getting expressed, how did I screw up?" "How do I make this huge protein" aren't LLM questions, they're university level biology questions, and "why is my protein growth tank not making the thing I want, and what is the grey stuff, do I need to put clean it again" is an experience question.
Putting checks on DNA synthesis puts making the horrible stuff out of the range of what high schoolers can do though.
Alphafold or something like that is more likely than the LLM models. Though cant say what an LLM model that uses Alphafold as a tool can do.
No not at all. The top models are not uncovering anything novel with DNA.
This is old stuff, being implemented and getting lots of news.
How you know? The top models have a tendency to not tell what they're conspiring for.
well math you can run Lean...
but for bio... I don't think we have 99.999999% accuracy simulation, and experiments aren't "0.00001 cent"
so we have to rely a lot on existing data or pay a lot
>Those vendors must verify their customers’ identities and screen orders for potentially dangerous sequences that, for example, could encode toxins or pathogens.
I'd assumed they already do this, and if you order Smallpox primers or w/e from Sigma/IDT you get a visit?
I have a home lab; wonder if there will be practical effects. Also have a personal project where I'm exploring enzymatic oligo synthesis. (Different set of pros/cons from the traditional phosphoramidite synthesis)
This might impact me and a small handful of other people who have messing around with microbes in their apartment as a hobby. We'll see what happens.
Probably a good thing overall in my opinion, even though it obviously has impacts.
I got into this hobby briefly (pre-LLMs) before realising how much of a loaded gun it is, and that I’m just not qualified to safely muck around with this stuff.
I’m curious about the hobby itself. What are you learning and enjoying? I mean that genuinely as a question, not sarcasm or trolling. This is fascinating to my 8 year old self who got a microscope for christmas and always wanted to explore more
making bread yeast for example, or playing with bioluminescent bacteria?
Yeah me too. Using Clorox on my toilet seat kills always 99.99% germs. The fun is with the 0.01% that's left
The 0.01% might survive, even develop some resistence. But when put again in the mix with wild non-resistent individuals in a non-clorox environment, the selection acts against them: having resistence might have costs, have no use, and if the mechanism is lost for whatever reason the new individuals survive anyway.
Something like this is suspected to have happened to us humans: the black plague has been pressuring/selecting individuals with more active and strong immune systems. Today the plague isn't so active or deadly to humans, but stronger immune systems are probably behind auto-immune illnesses (https://aihep.org/post/the-bubonic-plague-and-autoimmune-dis...).
Similar with sickle cell & malaria
You can actually do real biology at home if you set up a Petri dish tray with a concentration gradient of Clorox across it, then put some random germs at the low concentration and.
Yeah, or antibiotics maybe. But still, I could just put bare wires in my bathtub, no need for such effort :wink:
You can, if you want to do electrolysis, but I'd suggest electrodes with a large surface area, and an electrically isolated container (no drainpipe).
I suppose you might have to go back to bathtub explosives as a safer alternative.
In your apartment you say? Wonderful.
The device used to make oligonucleotides is pretty much an inkjet printer, from there it's cleverly applied enzymes to assemble the full strand. If you've got the skills and determination to get the synthetic DNA into a functioning cell, I don't think limiting access to synthetic DNA in the first place is going to stop you.
The device used to make oligonucleotides is pretty much an inkjet printer, from there it's cleverly applied enzymes to assemble the full strand.
Usually the synthesis of oligonuclotides is done with phosphoramidite chemistry [0]. So tell us more about those enzymes to make DNA molecules from scratch.
[0] https://www.twistbioscience.com/blog/science/simple-guide-ph...
I was referring to efforts like these:
https://pmc.ncbi.nlm.nih.gov/articles/PMC10867077/
https://www.sciencedirect.com/science/article/pii/S266654252...
And yes, phosphoramidites are what you use as ink. You print many many dots on a flat surface, and then you print over the same dots again and again, building up layers one base at a time. Then you dissolve those dots, giving you short strands in solution, which can be assembled into longer strands by being strategic about what else is in that solution and about which sequences you programmed in to the printer.
I've been considering making one in my garage. I don't think it's going to be easy, but I also don't think it's going to be difficult enough to be a good candidate for use as a regulatory chokepoint. We're not talking about a warehouse full of centrifuges here, it's a printer with special ink.
I’m pretty sure they were talking about how you can combine synthesized oligos into longer fragments when you are starting with smaller oligos. I’m guessing that they were referring to using different restriction enzymes from that point on.
Unless we are talking about really long oligos (which is harder and $$$), you’ll need some post-synthesis enzymatic work to get your full strand.
Yes I also read the comment that way. The inkjet printer remark presumably refers to benchtop DNA synthesis machines which have been commercially available for a while now. (The linked article mentions the possibility of buy-back programs for old machines that don't enforce any restrictions.)
I guess it's not a bad idea to put basic hurdles in place but it would be a serious mistake to see this as any sort of comprehensive preventative. DNA just isn't difficult to manipulate.
The vast majority of commercial synthesizers aren't inkjet. The OG DNA synthesizers are pretty simple - just a handful of pumps and valves and a packed silica column for each run. Anyone with a little mechanical engineering and organic chemistry training could build one (lots of companies in the past did just this.)
Kind of like owning your own darkroom, back in the day of analog photography.
Seeing his adjacent comment I stand corrected. I was not aware that there are now literal DNA printers. Only that you could obtain the sort of synthesizers to which you refer which had approximately the form factor of an inkjet (ie a small box).
The thing is DNA just also isn't dangerous. It's fragile and everything in the environment destroys it - as noted it's literally a everything else which is the hard part.
This reeks of the Longtermists getting their influence into government. They were doing an entire round on YouTube about a year ago.
DNA is pretty robust. You can send a dried spot of it on a piece of filter paper and send it in the mail.
Ansa Bio uses an engineered terminal deoxynucleotidyl transferase (TdT) enzyme to synthesize the oligos themselves, rather than using phosphoramidite chemistry. Each enzyme is tethered to a single nucleotide, allowing it to extend the DNA strand by exactly one base. The tether is then cleaved and the cycle repeats.
This works in aqueous solution under mild conditions and enables synthesis of much longer oligos (~1,000 bases), which can subsequently be assembled into 50 kbp constructs.
Not sure about that inkjet printer analogy.
Maybe referring to TdT-based (or other, yet-invented?) enzymatic synthesis? More approachable for ambient conditions and long strands.
assembly pcr. Oligos only go to about 200 max, for a full (useful?) Strand you need hierarchical assembly whichis enzymes.
Does this cover Israeli companies? That's where my concern lies.
Also, DNA chemistry is very well established. Someone with the appropriate background could easily assemble oligos without too much difficulty. I guess this stops Little Johnny from printing ebola from the internet.
>So, a song and dance of doing something that will most likely just inconvenience people doing real work.
Why does he call it an "otherwise pretty great policy"?
He's claiming that the policy needs to be more comprehensive. You're concluding that it doesn't pass a cost/benefit test. That's not at all the same claim!
> U.S. government–funded scientists who make and manipulate DNA sequences will face tougher regulations
Seems like this has little to do with deterring terrorism then, considering the assumption that a terrorist may not be a US government funded scientist.
My guess is that the assumption is that US gov funded labs are enough of many suppliers revenue that it'll be effective, but I'm 100% speculating and it may be total garbage speculation given that I have no idea how much hobbyist level DNA projects are a thing or how much they're a thing for non-medicinal business purposes.
If you read the next two sentences it makes it fairly obvious that this is a plan to indirectly target suppliers. Federally funded researchers will be required to
"procure synthetic nucleic acids and benchtop synthesis equipment from U.S.-approved suppliers. Those vendors must verify their customers’ identities and screen orders for potentially dangerous sequences that, for example, could encode toxins or pathogens."
If this was just about feds self-policing, they wouldn't have the customer identity requirement.
Probably they're taking this route because AI is moving fast and Congress moves slowly.
Trouble is, that assumption hasn't always worked out in the past: https://en.wikipedia.org/wiki/Bruce_Edwards_Ivins
But yes, this is stupid. No one who is inclined to build a biological WMD is going to be more than mildly inconvenienced by supply restrictions. DNA isn't plutonium.
Sounds like a fully general argument against any sort of security measure. "A dedicated thief won't be deterred by a lock or burglar alarm." Sure. I'm still buying that stuff.
More like, Danny Ocean isn't going to be fazed by your ADT Security sign out front or the Ring camera on your door.
Why assume an ultra-competent wrongdoer?
https://news.ycombinator.com/item?id=49606803
When DNA is outlawed only outlaws will have DNA.
Considering what we know about crowding-out in human microbiomes, this is probably way more relevant than the meme you were likely swinging for!
> It’s great, because it does something instead of nothing
I am detecting a bit of frustration here.
Yeah, I'm assuming that was intended to be sarcasm.
(Also, my comment is great, because I wrote something instead of nothing.)
[ This comment left intentionally blank ]
Erm. Terrorists are generally full of DNA. How difficult is it to break that down, sort and purify on a scale from vinegar and soda to bioterrorist with access to a DNA printer?
I want to say this is like banning the compilation of certain code snippets, but it's not like that, it's literally that. A bit weird.
Where is the disclaimer, that all labs are 100% safe, and it is not possible for something like virus to escape from lab?
Talk about a useless policy. It just says federally funded scientists need to buy gear from US companies and register. I feel like I can confidently say whoever makes the first purpose built megavirus is not going to be federally registered and purchasing equipment through the property channels. This is security theater, but unlike TSA this is security theater with existential stakes.
Let's ban knives too because they're sharp and might hurt someone.
12 Monkeys here we come!
Cluade find how to time travel, Make no mistakes!
No sequences encoding harmful stuff like covid19 spike proteins!! That means no mRNA vaccines that make spike protiens!! ;-)
So continue to fund bioweapons programs under the guise of medical research while also stifling individual and independent research under the guise of protecting us from terrorism? Yeah, that sounds like the US government.
This is not unlike 3D printing: "What? You can print a gun? Into jail you go!".
When you then look at the mafia in charge of the USA right now, run by oligarchs and other shady criminals, you ultimately have an apartheid system at work now, because these top-down pseudo-kings can control what others can do. Everyone is monitored and a potential terrorist now. Now imagine that DNA can be patented (I don't refer to right now, but look how ESTs (https://en.wikipedia.org/wiki/Expressed_sequence_tag) in the 1990s could be patented) - and suddenly you will have a state-run mafia decide which monopolist can yield de-facto control, with a few minor companies roleplaying the "it's still capitalism!!!" when in reality it no longer is.
This sounds unbelievable, but hey, we have AI. Let's just keep pretending this is the regular rate of tech advancement ...
Carry on, everything is apparently fine.