"Second, we encourage open source, open collaboration, and resource sharing. The Chinese government is committed to making artificial intelligence a significant driving force for promoting common prosperity and safeguarding common security. To support the construction of an open source innovation ecosystem for artificial intelligence, convenient measures such as automatic licensing will be applied to technology exports involving open source AI technologies, Chinese companies building and operating their own computing centers overseas, conducting chip fabrication overseas, utilizing overseas resources for joint research and development, and participating in the formulation of international standards, basically achieving "immediate reporting and export." Technologies that have already entered the public domain through open source or published academic papers will not be included in the export management scope, creating a favorable environment for international exchange and cooperation in artificial intelligence technology."
their official stance suggests they remain in support of open-weights
But China is, famously, much more open when it comes to AI. For this comparison to make sense, China should be the hypocritical one, declaring openness when not really being open.
Companies making closed-weight models typically don't export them at all, but sell access to them as a service. (Which doesn't count as an export of the model.) So it's primarily companies releasing open weights (if they pass the thresholds for the restricted category), or that host their models in datacenters outside China that will need to apply for an export license.
Meanwhile the US certainly appears to consider API access to constitute a form of export so misunderstandings are understandable. Or perhaps it would be more accurate to say that China isn't currently restricting that form of export.
No, what they've been doing so far is changing nothing.
What they are saying is that they won't be changing anything.
That could of course change overnight.
There are multiple aspects to this: China is cottoning on to the meaning of soft power, and besides that being a major supplier of AI related models, services and hardware further cements their place in the world economy. They may not be able to compete directly with the USA in this domain but they are able to undercut them.
Of course there are risks as well. Models produced and released into the wild for which you have not seen the training and conditioning data could have payloads that you'd rather not have. But the same thing goes for service providers in other countries as well. You can't really trust hosted inference, not to tell you the truth in so far as is was present in the training data (censorship, not inference errors) and not to leak your information.
The proposed export controls do add some categories where export is completely prohibited. For example, sandbox detection. While the provisions specifically for AI models are generous enough, it's still possible that they trigger some other prohibition by being too capable. The statement about encouraging open source only applies to cases where export is merely restricted, not prohibited, and even then companies need to report their intention to export in advance, in principle giving the government the opportunity to deny them an export license.
Care to substantiate that? Not that I disagree, just curious.
My (not thorougly informed) take is that they are are taking measures to have the capability of restriction but are also adding provisions so that this capability does not add friction to the current practice of enabling export of open weight models.
So to me it's like they are keeping their options open rather than leaning on one versus the other.
I would find it surprising if the chinese government wanted to allow Anthropic/OpenAI/Google levels of influence on the public opinion and regulatory environment without guardrails. OSS models make AI a replaceable commodity and reduce the potential for market/opinion capture by the frontier labs.
Jetson Nano(with Nintendo Switch SoC) used to require a signed form with restrictions like that for US export control compliance even in a democratic country. It's the same device as the one that was always there, except you were only familiar with its operating end being around yourself.
You trust other countries to behave like this out of the goodness of their hearts or something?
If China’s selfish behavior = more open LLM tools and greater ability to run them at home,
I’ll gladly take it over the US frontier labs’ druthers, which is a restricted/closed shop we rent from.
I understand why China wants to restrict exports of their models.
But I feel the only reason Chinese models have been able to gain such popularity is BECAUSE they are open source.
I wouldn’t trust models from China enough to use them in important work unless they were open.
Restricting any powerful Chinese model [that ranks in the global top 10] is going to make them very difficult to advocate for, and very hard for global users to adopt.
Feels like this could be the beginning of the end of the democratisation of powerful AI.
And yes, I appreciate the irony that Chinese models have been the ones enabling that democratisation!
> But I feel the only reason Chinese models have been able to gain such popularity is BECAUSE they are open source.
Why do people keep calling these "open source" models? They're not that different from NVIDIA's binary blobs. Sure, they work, and you can run them and get results, but the source code, training data, and training process are largely hidden and proprietary. Publishing weights is not the same as publishing the source. They're open-weight models, not open-source models.
I'd argue that they're still open-source because inference code is fully public. In operation, a user can crack open the model and examine its internal state, and we also see that many people produce various modifications or fine-tunings of public models for their own goals.
In my mind, this meets the conceptual and technical bars of an 'open source' designation. We can also imagine the opposite, where inference itself has to pass via an inscrutable binary blob that keeps the model's technology stack fully proprietary.
You're correct that we're missing the training data, the training recipe/curriculum, and any secret sauce that went into optimization of same, but I don't think that's an essential feature of an open source designation.
Instead, I think that the model weights are a kind of rendered artifact, and it should be evaluated on that basis. We don't treat permissively licensed images as something lesser, even if the photographer or illustrator supplies only the final image rather than the full camera raw file and/or Photoshop layer stack.
The difference, of course, is that the rendered artifact (weight blob) in a language model is so much more important than a typical photograph.
Nah this is like saying that Photoshop is open source just because you can download and run the binary on your own computer instead of through a SaaS webapp. I think basically everyone would disagree with that.
> In operation, a user can crack open the model and examine its internal state, and we also see that many people produce various modifications or fine-tunings of public models for their own goals.
I mean all of open source software (as we traditionally defined it) relies on an ISA, which is actually executed by microcode that isn’t available to the public?
I don’t see how this is any different. It’s like saying Intel has to publish their transistor physical library and microcode to be “truly” open source. Like I don’t have the internal notes about the most recent microcode changes (which is akin to “training process” in the analogy).
> I wouldn’t trust models from China enough to use them in important work unless they were open.
One issue is that in the end, open models are still comparable to "freeware" from the old world. You have a black boxed binary to takes input and produces output. On typical "freeware", you could still reverse-engineer it and have have confidence to map assembly to actions it takes. But with these open models, not so much. So, we still can't put too much trust to these models, even if they are open. There can be some backdoor that is impossible currently to identify based on reading the model weight numbers. We can try to post-train models and reduce the likelihood of some "known" stuff, but we don't know what unknown there is. Interesting times ahead.
There are risks in both cases, but they are very different.
If you're using a closed model, the model provider gets all your data, and can if they want to send back actively hostile tool calls and rely on your harness to catch them without any effort on their side to train that behaviour into their models. The only things you can do about that is to place careful restrictions
If you're using an open model against your own system or a third party provider, the origin of that model could have trained in "triggers" that'd cause the model to produce actively hostile tool calls as well, but they'd have no direct control and would need to train in that behaviour "blind" to how you'd actually use the model.
It doesn't mean you can be complacent about trusting the model, but it does change the likelyhood they'd be able to do any damage, not least because you'd be able to run evals against the model without any risk that the model or provider API could just randomly change on you after a while.
So it takes far more trust to run a closed model than to run an open model even on a hosted third party provider, and certainly far more than to run an open model on your own hardware.
You can run Chinese models locally without Internet, while with US models you need to have Internet connection. Guess where it is easier to hide a backdoor.
They gained popularity because THEY ARE GOOD, not because they are open weight. There are so many other models that are not popular. Do you know about arcee and their models? every used inkling? how about laguna? these are American open weights, they didn't take off because they are not as good as the rest. Do you know about the command-x series from COhere? probably not, this is a Canadian model, folks into local were really into it 2 years ago when they were keeping. do you know about dbrx? Yeah, databricks an American company also gave us a model. Anyways, my point is the competition is strong and giving away for free doesn't matter if it can't keep up. It's expensive, to send bandwidth, storage, gpu compute to run a model when there are better options.
Its because the triangle of good, cheap, and open favored them. Just being good isn't enough unless you're number 1 but likewise neither is just being open (ish). It was important to release the wieghts and the research because otherwise good wouldn't be good enough to outweigh the risks.
> they didn't take off because they are not as good as the rest.
This may be true now but probably won't be true in the future. My guess is that part of the reason for this is that the major American technology companies are acquiring talent to build out models and related capabilities. China doesn't (or at least didn't) have that same dynamic.
> I don’t understand how you think this is the likely future. It is a future, sure, but assuming America just wins “because capital” seems wishful.
I don't think it's a stretch to suggest that part of the reason American startups haven't released or haven't released as many successful "open weight" models or related companies is because we have a lot of really high-end, top-tier tech firms which are hiring most of the talent and paying them a lot. It's not just Anthropic and Open AI. It's basically every tech company, plus finance companies, airlines, &c. who are researching and figuring out how to make their own companies better.
If you find that to be something wishful or you have trouble understanding, I think instead of combatting my comment you should ask yourself why you think that American tech companies aren't hovering up exceptional talent and that has had an impact on startups or other tech companies launching similar open-weight models.
You also need to understand the economics better and talk through those. Investors in the US don't generally have state backing so they may be more diligent in what they invest in.
> Rome is not what it once was, and China has plenty of universities, bright people and capital of its own.
Sometimes I think people are internalizing the troubles in Europe (I know you mean Rome as in the US being the new Romans or something) and externalizing those issues at American problems because of Donald Trump, but I would caution against this. China has plenty of universities, bright people, and capital of its own. Yes. It also has disadvantages. The US also has advantages and disadvantages.
> I don’t see how anyone, looking at the past 30 years, can be so sure that the US will out-do China. But good on you for being an optimist I guess.
We've proven we can build competitive and capable companies. I'm not being an optimist, instead I'm just calling it like it is. Our ability to continue doing what we already have demonstrated we can do isn't taken away by China (or other countries) also doing those things. There hasn't been any material change in the ability of the US to build world-class companies in recent history.
I do think the suggestion is a stretch. Facebook got knocked out of the LLM space pretty hard with Llama 4 and the later muse models, while fine, are not the equals of anything deepseek, z.ai (glm), kimi or qwen has. I think we could extend the list, but I believe the point is made.
I am pretty sure Facebook has clever people and deep pockets, but it hasn’t materialized properly. I cannot say for the very latest releases, but Google too has been a bit of a laughing stock on this front - same story.
As for viewing things with an economic lens. I profess to having only a moderate grasp of economics, I do better with history and geopolitics- and I think this is another blind spot. Americans largely view the world as “business as usual “, but you profoundly underestimate the animosity, concern, suspicion and outright fear other countries have now. There is simply no way our entire infrastructure will be hooked up to an American “super intelligence” firm where the president calls every other Tuesday to extort a country. And failing to appreciate that leads to vastly more rosy ideas of the growth potential of OpenAI, Anthropic and others.
Ok - we just disagree there. It seems incredibly obvious to me.
> As for viewing things with an economic lens. I profess to having only a moderate grasp of economics, I do better with history and geopolitics- and I think this is another blind spot.
I do better with history and geopolitics as well, my point was just that no matter how "open" the so-called open models are, money is what matters and you have to be able to sell the model or associated infrastructure around it (in software terms but perhaps also hardware in the future) to keep your company going and to make money. There is no way around this.
> Americans largely view the world as “business as usual “, but you profoundly underestimate the animosity, concern, suspicion and outright fear other countries have now.
Sure, but on the other hand this exact comment has been parroted all over the Internet (which has little in common with what most people actually think) on select forums of slightly more intelligent than average people to the point where the extent to which Americans view business as usual is being underestimated we can also say that random people on the Internet trying to tell Americans they are so big mad at us is itself an overestimate of how people feel.
I'd also like to point out - which countries, exactly? And just how profoundly are they big mad at the US? If they are profoundly angry, concerned, suspicious, and fearful, and perhaps they are European, maybe we should immediately withdraw from NATO, withdraw military forces from Europe except where it benefits us directly, and we can remove sanctions on Russia because well Ukraine is in Europe and we want cheaper oil and gas here at home. After all, you can't simultaneously claim that there is all this distrust and then trust us to have armed soldiers in your countries.
There's a strong contradiction between rhetoric and action and it plays out very obviously at times like this. You see the same thing with big, grandiose statements from European leaders who want to talk tough and appear like they're "standing up" to Donald Trump, but behind closed doors it's always a different story.
> There is simply no way our entire infrastructure will be hooked up to an American “super intelligence” firm where the president calls every other Tuesday to extort a country. And failing to appreciate that leads to vastly more rosy ideas of the growth potential of OpenAI, Anthropic and others.
Yes but then you also can't be hooked up to, idk, Chinese "super intelligence" firms because they also can do that to you and have demonstrated that they will punish European countries for saying the wrong thing about their dear leader or whatever. You're also assuming that the United States will allow other countries to hook their infrastructure up to our AI infrastructure. It seems likely, but in terms of the latest, state of the art models and capabilities I'm not so sure that non-Americans will even be permitted to use those, at all.
If the promise of AI as being generally stated is true, it actually makes sense for Americans to have sole access to American AI technology so we can build and expand our businesses and maintain competitive advantages. China is also restricting models with fresh news as recently as last month on that front. Perhaps a likely scenario is that China and the United States simply come out on top and the rest of the world is left to suffer what it musts.
Another area that seems strange to me is the insistence (and honestly I think it's just a belief in how someone wants the world to be more than it likely will be) that American tech companies, run by some of the smartest people on the planet and investors, American ones who have been doing this for a long time and have access to incredibly deep capital and resources to evaluate companies are all somehow wrong about spending a trillion dollars or more (whatever the number is it's very large) on AI buildout. It just doesn't seem likely besides a typical boom/bust which is a healthy aspect of our economy. Somehow, they are all wrong and hey some Chinese companies released some open-weight models so I guess all of these people in America are totally wrong and everything will be free and open and run by China and somehow they won't build companies to make money they'll just work for free and release open-weight models and by the way Europe will never use American ones!
I'm just calling this to be all bullshit. Nobody is providing hard numbers and nobody seems to have an explanation as to why American companies can't if they want to just reverse engineer Chinese models if they were better. Let's say investors go out and lose their t-shirts on Anthropic and Open AI? Yea that sucks, so what? We'll just build more and better models. If China "gets ahead" we'll just do the same strategy to them that they seem to be doing to us.
Here's another question - why isn't Europe even in the topic of conversation about AI? Where are the companies? If the Chinese models are just cheap/easy to use or whatever why aren't we seeing a number of new AI startups? If you are going to accuse Americans of being unable to build and release open-weight models, what does that say about European companies?
This isn't a pro-American point of view, though I am pro-America (regardless of who is president). I just find the arguments put forth in this topic to be either of poor quality, or too strongly ideologically tainted.
I find it curious that, I, a European, consuming European media, living in a European country and consuming its domestic media, which you very likely don’t, am being lectured, by an American, on how we feel.
As regards nato - again, you seem ignorant. The quiet part isn’t said out loud by anyone but the president who threatens to abandon Nato commitments, but yes, European leaders are also considering plans where the US isn’t helping.
And again, US deployments here are minuscule, approximately 2 months worth of Russian casualties. The value was always the deterrence.
And the value for the US was a gateway to the middle east and further abroad along with highly favorable deals for its military industry.
The US has already been nearly useless these past 2 years. We fund the entire war and you sell the arms. The only help has been intelligence, which is hugely helpful, but costs you nothing extra.
I don’t think there is any point continuing a discussion from there.
Why do they care about global usage though? This seems primarily so that they can control it for their billion plus population, they really don't need global users at all, much to our chagrin.
I think it is more about avoiding being boxed in by the rest of the world. The US would of course ban the use of Chinese hosted models and have already taken steps to ban the use of open Chinese models. There's nothing that could have been done about that. But the rest of the world would likely have adopted American hosted models eventually, the same way they adopted American mobile phones OSes, computer OSes, Office, etc. If the choice was between American hosted AI and Chinese hosted AI they'd choose American if they couldn't develop their own.
But if there's an open model available then they might choose that instead of either of them. Even if the Chinese models aren't used they will bring down profit margin just by existing. They wouldn't have the control the hosted American models give but that wasn't even on the table. But denying America control is a worthwhile strategic goal in and of itself. And they don't even need to expend extra resources to do it, they needed to develop their own models for security reasons anyway.
>I wouldn’t trust models from China enough to use them in important work unless they were open.
It's possible to embed malicious behavior into models, such as "secretly phone data that meets these requirements back to this address..."
This has been shown, and anyone who does cybersec should be aware that "open model running locally" isn't fool proof.
Edit: People might think I am referring to the cohere incident, I'm not, I'm referring to other labs that have purposely experimented to see if this is possible. It is.
Does anyone know about how you could go about training this type of thing into the model? Surely it is quite tricky to get it to do such things without it doing them more than this one incident? It should have happened lots if it really was trying to obtain secret info and build backdoors?
You know people are saying that it’s not open source without regard to intent. The intent is 100 percent open source.
The reason why it’s not done is because it takes a huge amount of resources to not only record the training process but to release it in a readable format.
What you’re asking for is almost like surveillance documentation for a model that’s likely put together in a way that’s not easily peaceable into clean step by step procedures.
Let's just hope that both US and Chinese models remain usable enough by the masses for long enough for someone in Europe or Japan or some other actual free country to make a truly open AI :')
Latest and greatest China is able to make is reserved for the use of businesses and entities in the Chinese market.
Everyone else gets access to models of more limited capabilities.
It's basically the US strategy. Obvious difference being only that the Chinese allow on prem hosting of the lesser models. So plus one to them I guess.
If we want to move China or the US on from the whole, "super intelligence models for me, more limited intelligence models for everyone else" strategy, then "everyone else" will have to come up with some models competitive with the latest and greatest from the US and China. Absent that, there's just no pressure to bring the US and China to the "share the wealth" table.
Unfortunately, "everyone else" coming up with a competitive model is not gonna happen anytime soon.
The pressure for them to export models is that doing so super-charges the growth of the domestic industry. Same reason why they export solar panels, instead of just hoarding all of them for domestic deployment.
People pay money for the solar panels, though. It's not obvious to me that giving away expensive LLMs for free is meaningfully helping grow the domestic economy, as opposed to foreign inference providers who get their GPUs from Nvidia.
Giving away open weight models, if you dismiss altruism as a motivation from the outset, is a way to salt the earth while at the same time building mindshare.
It makes it harder to extract decent margins from models at or below the quality of the open models, and so makes it harder both for new entrants and for any existing providers that are depending on cheaper models for revenue. At the same time you make people comfortable with your product.
It's the IP version of dumping products below cost if you're capital rich and want to winnow down the competitive field while "buying" customers.
They are dumping products (models) below cost. It's just that they are not dumping their latest and greatest products (models).
The models they dump may not be good by the latest Chinese or US standards, but they are far and away better than anything anyone outside of China or the US is able to come up with.
So until we start seeing EU models or South American models or African models that are better than the models being dumped onto the market, China will not be giving us anything better.
And again, it's gonna be a long long time before the rest of the world even gets to the effectiveness of some of the Chinese models being dumped. Let alone competing with the best models China or the US possess.
USA models proven to use copyrighted materials illegally. Meta also proven to ignore privacy. USA tech companies routinely give your date to military and law enforcers without even legally allowed to tell you. Unless you meant not trusting anyone, specifically saying not China is odd. You are almost likely to get Uncle Sam notice and legal enforcement while zero from China, you dropby their police station. In fact most EU countries will also extradite you to USA without your legal protection.
Yes, this happened weeks ago and was never discussed on HN.
Part of the problem is that there are too many major stories happen back-to-back nowadays, and important stories like this one never get discussed. HN only shows 30 stories on the front page, which limits discussion.
Correct, this news just hasn't gotten traction until now on HN it seems, as I've already seen this news on r/LocalLlama and thought it was another piece of newer news.
The risk is that USA and China will come together on a joint "AI non-proliferation" agreement in the same way that nuclear technology became restricted by the countries that already had the capability. This will lead to a de facto techno-colonization of the rest of the world just between themselves.
This was a topic of discussion at Trump and Xi's AI summit just last month. We're now starting to see the export controls coming into place.
I don't know how anyone with a level understanding of the world would think that China is going to hand out nuclear scale cyber weapons willy nilly.
It's not in the interest of any government, or even any AI lab, to democratize dangerous intelligence/ability.
China wants to undercut the west, of course, but it's looking like good/bad model behavior are two sides of the same coin. "Show me how to protect my server from the most esoteric attacks" is simultaneously "Show me how to attack a server with the most esoteric attacks."
Yea, many here keep trying to treat LLMs like software, or at least conventional software. They are not. To me they are at least the modern equivalent to ballistics charts that governments used to keep secret. No one is going to hand them out to their enemy to use against them.
Hell, I expect before long hardware is going to have all kinds of lockdowns occurring.
slightly ot, but this seems like a good place to ask, since I don't think I have the strategic-business mind to understand it all myself: What's the play here with China having supported publishing open weight models this far? I know the official line is for the good of mankind and all that, but presumably it's more strategic than that. Soft power? Commoditise your competition?
So far, there have not been any obvious drawbacks to open-weight models from the government perspective, so when a company wanted to release their weights, they weren't prohibited from doing so. Similarly, there have not been any obvious drawbacks to closed-weight models, so when a company wanted to keep their weights to themselves, they weren't prohibited from doing so.
So the real question you should be asking yourself is why you thought that China supported publishing open-weight models, even though they merely didn't prohibit it.
In my mind, they’re putting huge downward pressure on the major US AI companies’ ability to become platforms.
Think about the history of AWS. Amazon made a very specific point of building, specifically, a “platform”. When they developed a major internal service, they would sell it. Basically anyone who wanted to use it could pay AWS (often an outrageous amount, but it was often worth it), and AWS would provide the service, and AWS was consistently neutral and trustworthy. They didn’t really tell you what you could or couldn’t do, they didn’t even ask what you wanted it for, and they never used your private data for their own purposes. AWS pulled this off quite well despite the fact that competing products and even open source products could and still can, with effort, do most of what AWS offered.
Compare to OpenAI and Anthropic. They want to sell you their front-ends (chatgpt.com, claude.ai, their apps and harnesses), but the big money is selling enterprise contracts with per-token pricing. The latter is the platform.
But their platforms suck in comparison to their open competition. You are paying for inference on a series of models, but:
- You need to negotiate with them if you don’t want them to retain your data.
- They might not even be willing to not retain your data.
- Part of the model output gets encrypted before you see it.
- You are not allowed to give the model arbitrary input.
- They run classifiers on the token stream to try to restrict what you may or may not do.
- You cannot see the model’s internal states.
- You are extremely restricted in your ability to modify the model and you may not be allowed to modify it at all.
- Oh, and maybe the price is above market rates.
The last point is not really a big deal: AWS has demonstrated quite effectively that customers will pay many, many billions of dollars above market rates for a high quality platform.
But I think the rest are a big deal, and the open-weight models allow anyone to use a provider other than Anthropic and OpenAI and avoid all the other problems in the list. This means that, if I’m a startup building a product that uses AI, I get to choose between OpenAI/Anthropic-like terms and a far better alternative. And this makes it very hard for OpenAI or Anthropic to sell me on their platform even at equivalent pricing.
The US economy highly depends on the valuation of the tech giants whereas the Chinese economy doesn't.
Additionally, the financial and technological equilibrium which makes OpenAI or Anthropic possible is rather precarious so they are poking on it, hoping to make it fall.
there's alot of reasons, but one of the more commonly cited ones is that China facing chip restrictions means they lack the compute to provide the models for global usage, and making them open-weight is one of the ways to circumvent these restrictions.
I'm pretty daft with this but my opinion is that they are open source so they can help Russia war efforts (and maybe Iran) unofficially and indirectly.
Denying positive ROI for non-Chinese labs, destroying AI industrial base of others, while allowing NVIDIA to profit?
OpenAI had been miniaturizing GPT models as far back as GPT-4o era, but Chinese free-beer models that run on 16GB consumer GPUs can handle the vast majority of LLM use cases, and now they have no options but to attempt development of actual superintelligence that can cure cancer or solve the P=NP question just to differentiate against free-beer alternatives. It's clear what China is trying to achieve.
Well, there is CrowdGPT[0] which is a fully open and distributed attempt to train a small 1B parameter LLM from scratch with open data. They've started just a few days ago but they're already at 13.0B / 100.0B tokens for the pretraining phase.
Of course training something like a 1T LLM will probably take some time :-P but with enough people contributing it might be possible to train something small - and if the project reaches consensus, it might be adopted by some institutions with a bit more available compute power too (e.g. universities).
If i had a second PC (i don't really trust the project right now to put their auto-updating client on my main PC - maybe if it grows and there are more people behind it or there is some better governance than "trust me bro" i'd put it in a container or something) with some GPU power i might have contributed a bit too since i like the idea.
That's not happening without significant funding. The cost of the primary training run for frontier models is in the hundreds of millions of dollars, and is increasing with every iteration. The Astra pre-train likely cost close to a billion dollars.
The Chinese open-source models (which are 1-2 generations behind the frontier) cost in the low tens of millions to train. This is largely because of a lack of compute. They achieve comparable performance to frontier models via distillation.
somebody wake me up when we have "SLSA, but for model provenance".
I don't (just) want "open weight + verified distribution". I want a dang audit trail.
If/when one of these models is trusted enough to make real decisions with teeth, and if/when one of those decisions screws up badly enough to hurt somebody, I want to be able to play back the tape and pinpoint the source dataset + training method responsible for the failure. At LEAST at a coarse level, on par with something like modern RPO SLAs.
P.S. this comment is applicable to pretrained models in general. I don't care if it came from China, USA, or model weights scrawled in sharpie on a random bathroom stall. I just want to know the provenance and have enough information to run a blameless "git blame" (or equivalent) if/when things go wrong.
P.P.S. IMHO, we aren't there (yet), but OTOH this proposal ain't a bad place to start.
With LLMs it's almost certain what you want will never happen.
Most people just don't understand how many compute operations occur to train a modern model, and just how infinitesimal any particular data point is. Even worse is when a model generalizes a function. You have to now prove the entire generalization.
https://www.mofcom.gov.cn/xwfb/xwfyrth/art/2026/art_d7661dbf...
"Second, we encourage open source, open collaboration, and resource sharing. The Chinese government is committed to making artificial intelligence a significant driving force for promoting common prosperity and safeguarding common security. To support the construction of an open source innovation ecosystem for artificial intelligence, convenient measures such as automatic licensing will be applied to technology exports involving open source AI technologies, Chinese companies building and operating their own computing centers overseas, conducting chip fabrication overseas, utilizing overseas resources for joint research and development, and participating in the formulation of international standards, basically achieving "immediate reporting and export." Technologies that have already entered the public domain through open source or published academic papers will not be included in the export management scope, creating a favorable environment for international exchange and cooperation in artificial intelligence technology."
their official stance suggests they remain in support of open-weights
Famously, the constitution of the ussr explicitly guaranteed many more freedoms than the us constitution does. So, not much to go on
But China is, famously, much more open when it comes to AI. For this comparison to make sense, China should be the hypocritical one, declaring openness when not really being open.
There’s no need to be so sarcastic about OpenAI.
Interesting way to encourage open weights models. It sounds like anyone wanting to export closed AI will have to apply for a licence.
Companies making closed-weight models typically don't export them at all, but sell access to them as a service. (Which doesn't count as an export of the model.) So it's primarily companies releasing open weights (if they pass the thresholds for the restricted category), or that host their models in datacenters outside China that will need to apply for an export license.
> Which doesn't count as an export of the model.
Says who?
Said the Ministry of Commerce at the press conference for the draft export controls: https://www.mofcom.gov.cn/xwfb/xwfyrth/art/2026/art_d7661dbf... 对于企业利用限制类大模型对外提供API调用、APP/网页交互、SaaS应用、本地端侧轻量化部署等情形，不属于大模型技术出口管理范围。
Meanwhile the US certainly appears to consider API access to constitute a form of export so misunderstandings are understandable. Or perhaps it would be more accurate to say that China isn't currently restricting that form of export.
On this topic wrote a small book https://insanai.github.io/aiexportcontrols/
You can also download offline PDF at: https://insanai.github.io/aiexportcontrols/books/en/open-sou...
It also shows using game theory that export control leads to overall failure of both the sides.
What governments do is, generally, much more important than what they say. And what they're doing here is moving in the direction of restriction
No, what they've been doing so far is changing nothing.
What they are saying is that they won't be changing anything.
That could of course change overnight.
There are multiple aspects to this: China is cottoning on to the meaning of soft power, and besides that being a major supplier of AI related models, services and hardware further cements their place in the world economy. They may not be able to compete directly with the USA in this domain but they are able to undercut them.
Of course there are risks as well. Models produced and released into the wild for which you have not seen the training and conditioning data could have payloads that you'd rather not have. But the same thing goes for service providers in other countries as well. You can't really trust hosted inference, not to tell you the truth in so far as is was present in the training data (censorship, not inference errors) and not to leak your information.
The proposed export controls do add some categories where export is completely prohibited. For example, sandbox detection. While the provisions specifically for AI models are generous enough, it's still possible that they trigger some other prohibition by being too capable. The statement about encouraging open source only applies to cases where export is merely restricted, not prohibited, and even then companies need to report their intention to export in advance, in principle giving the government the opportunity to deny them an export license.
So they're changing some things.
Also a lot of countries that can't trust the US -cough Maduro cough- will choose Chinese AI solutions.
“Mind people's actions over words.“ -Zed
“Right now, your government is doing things you think only other governments do.” -Van Halen, Right Now
Care to substantiate that? Not that I disagree, just curious.
My (not thorougly informed) take is that they are are taking measures to have the capability of restriction but are also adding provisions so that this capability does not add friction to the current practice of enabling export of open weight models.
So to me it's like they are keeping their options open rather than leaning on one versus the other.
I would find it surprising if the chinese government wanted to allow Anthropic/OpenAI/Google levels of influence on the public opinion and regulatory environment without guardrails. OSS models make AI a replaceable commodity and reduce the potential for market/opinion capture by the frontier labs.
I think you're contradicting yourself here.
What they will do remains to be seen. The document at hand is what they're saying.
I.e. less important if we follow your own assessment.
Jetson Nano(with Nintendo Switch SoC) used to require a signed form with restrictions like that for US export control compliance even in a democratic country. It's the same device as the one that was always there, except you were only familiar with its operating end being around yourself.
Yes, they support it, inasmuch it gives them an advantage.
Of course I never expected anything different from the Chinese Communist Party.
You trust other countries to behave like this out of the goodness of their hearts or something?
If China’s selfish behavior = more open LLM tools and greater ability to run them at home, I’ll gladly take it over the US frontier labs’ druthers, which is a restricted/closed shop we rent from.
This isn’t some BDSM gathering; there’s no need to keep bringing up "safe words."
We have no eternal allies, and we have no perpetual enemies. Our interests are eternal and perpetual, and those interests it is our duty to follow
I understand why China wants to restrict exports of their models.
But I feel the only reason Chinese models have been able to gain such popularity is BECAUSE they are open source.
I wouldn’t trust models from China enough to use them in important work unless they were open.
Restricting any powerful Chinese model [that ranks in the global top 10] is going to make them very difficult to advocate for, and very hard for global users to adopt.
Feels like this could be the beginning of the end of the democratisation of powerful AI.
And yes, I appreciate the irony that Chinese models have been the ones enabling that democratisation!
> But I feel the only reason Chinese models have been able to gain such popularity is BECAUSE they are open source.
Why do people keep calling these "open source" models? They're not that different from NVIDIA's binary blobs. Sure, they work, and you can run them and get results, but the source code, training data, and training process are largely hidden and proprietary. Publishing weights is not the same as publishing the source. They're open-weight models, not open-source models.
I'd argue that they're still open-source because inference code is fully public. In operation, a user can crack open the model and examine its internal state, and we also see that many people produce various modifications or fine-tunings of public models for their own goals.
In my mind, this meets the conceptual and technical bars of an 'open source' designation. We can also imagine the opposite, where inference itself has to pass via an inscrutable binary blob that keeps the model's technology stack fully proprietary.
You're correct that we're missing the training data, the training recipe/curriculum, and any secret sauce that went into optimization of same, but I don't think that's an essential feature of an open source designation.
Instead, I think that the model weights are a kind of rendered artifact, and it should be evaluated on that basis. We don't treat permissively licensed images as something lesser, even if the photographer or illustrator supplies only the final image rather than the full camera raw file and/or Photoshop layer stack.
The difference, of course, is that the rendered artifact (weight blob) in a language model is so much more important than a typical photograph.
Nah this is like saying that Photoshop is open source just because you can download and run the binary on your own computer instead of through a SaaS webapp. I think basically everyone would disagree with that.
> In operation, a user can crack open the model and examine its internal state, and we also see that many people produce various modifications or fine-tunings of public models for their own goals.
you're describing open weights, not open source
I mean all of open source software (as we traditionally defined it) relies on an ISA, which is actually executed by microcode that isn’t available to the public?
I don’t see how this is any different. It’s like saying Intel has to publish their transistor physical library and microcode to be “truly” open source. Like I don’t have the internal notes about the most recent microcode changes (which is akin to “training process” in the analogy).
> I wouldn’t trust models from China enough to use them in important work unless they were open.
One issue is that in the end, open models are still comparable to "freeware" from the old world. You have a black boxed binary to takes input and produces output. On typical "freeware", you could still reverse-engineer it and have have confidence to map assembly to actions it takes. But with these open models, not so much. So, we still can't put too much trust to these models, even if they are open. There can be some backdoor that is impossible currently to identify based on reading the model weight numbers. We can try to post-train models and reduce the likelihood of some "known" stuff, but we don't know what unknown there is. Interesting times ahead.
>I wouldn’t trust models from China enough to use them in important work unless they were open.
So you've inspected the weights for backdoors?
There are risks in both cases, but they are very different.
If you're using a closed model, the model provider gets all your data, and can if they want to send back actively hostile tool calls and rely on your harness to catch them without any effort on their side to train that behaviour into their models. The only things you can do about that is to place careful restrictions
If you're using an open model against your own system or a third party provider, the origin of that model could have trained in "triggers" that'd cause the model to produce actively hostile tool calls as well, but they'd have no direct control and would need to train in that behaviour "blind" to how you'd actually use the model.
It doesn't mean you can be complacent about trusting the model, but it does change the likelyhood they'd be able to do any damage, not least because you'd be able to run evals against the model without any risk that the model or provider API could just randomly change on you after a while.
So it takes far more trust to run a closed model than to run an open model even on a hosted third party provider, and certainly far more than to run an open model on your own hardware.
You can run Chinese models locally without Internet, while with US models you need to have Internet connection. Guess where it is easier to hide a backdoor.
They gained popularity because THEY ARE GOOD, not because they are open weight. There are so many other models that are not popular. Do you know about arcee and their models? every used inkling? how about laguna? these are American open weights, they didn't take off because they are not as good as the rest. Do you know about the command-x series from COhere? probably not, this is a Canadian model, folks into local were really into it 2 years ago when they were keeping. do you know about dbrx? Yeah, databricks an American company also gave us a model. Anyways, my point is the competition is strong and giving away for free doesn't matter if it can't keep up. It's expensive, to send bandwidth, storage, gpu compute to run a model when there are better options.
Its because the triangle of good, cheap, and open favored them. Just being good isn't enough unless you're number 1 but likewise neither is just being open (ish). It was important to release the wieghts and the research because otherwise good wouldn't be good enough to outweigh the risks.
> they didn't take off because they are not as good as the rest.
This may be true now but probably won't be true in the future. My guess is that part of the reason for this is that the major American technology companies are acquiring talent to build out models and related capabilities. China doesn't (or at least didn't) have that same dynamic.
I don’t understand how you think this is the likely future. It is a future, sure, but assuming America just wins “because capital” seems wishful.
Rome is not what it once was, and China has plenty of universities, bright people and capital of its own.
I don’t see how anyone, looking at the past 30 years, can be so sure that the US will out-do China. But good on you for being an optimist I guess.
> I don’t understand how you think this is the likely future. It is a future, sure, but assuming America just wins “because capital” seems wishful.
I don't think it's a stretch to suggest that part of the reason American startups haven't released or haven't released as many successful "open weight" models or related companies is because we have a lot of really high-end, top-tier tech firms which are hiring most of the talent and paying them a lot. It's not just Anthropic and Open AI. It's basically every tech company, plus finance companies, airlines, &c. who are researching and figuring out how to make their own companies better.
If you find that to be something wishful or you have trouble understanding, I think instead of combatting my comment you should ask yourself why you think that American tech companies aren't hovering up exceptional talent and that has had an impact on startups or other tech companies launching similar open-weight models.
You also need to understand the economics better and talk through those. Investors in the US don't generally have state backing so they may be more diligent in what they invest in.
> Rome is not what it once was, and China has plenty of universities, bright people and capital of its own.
Sometimes I think people are internalizing the troubles in Europe (I know you mean Rome as in the US being the new Romans or something) and externalizing those issues at American problems because of Donald Trump, but I would caution against this. China has plenty of universities, bright people, and capital of its own. Yes. It also has disadvantages. The US also has advantages and disadvantages.
> I don’t see how anyone, looking at the past 30 years, can be so sure that the US will out-do China. But good on you for being an optimist I guess.
We've proven we can build competitive and capable companies. I'm not being an optimist, instead I'm just calling it like it is. Our ability to continue doing what we already have demonstrated we can do isn't taken away by China (or other countries) also doing those things. There hasn't been any material change in the ability of the US to build world-class companies in recent history.
I do think the suggestion is a stretch. Facebook got knocked out of the LLM space pretty hard with Llama 4 and the later muse models, while fine, are not the equals of anything deepseek, z.ai (glm), kimi or qwen has. I think we could extend the list, but I believe the point is made.
I am pretty sure Facebook has clever people and deep pockets, but it hasn’t materialized properly. I cannot say for the very latest releases, but Google too has been a bit of a laughing stock on this front - same story.
As for viewing things with an economic lens. I profess to having only a moderate grasp of economics, I do better with history and geopolitics- and I think this is another blind spot. Americans largely view the world as “business as usual “, but you profoundly underestimate the animosity, concern, suspicion and outright fear other countries have now. There is simply no way our entire infrastructure will be hooked up to an American “super intelligence” firm where the president calls every other Tuesday to extort a country. And failing to appreciate that leads to vastly more rosy ideas of the growth potential of OpenAI, Anthropic and others.
> I do think the suggestion is a stretch.
Ok - we just disagree there. It seems incredibly obvious to me.
> As for viewing things with an economic lens. I profess to having only a moderate grasp of economics, I do better with history and geopolitics- and I think this is another blind spot.
I do better with history and geopolitics as well, my point was just that no matter how "open" the so-called open models are, money is what matters and you have to be able to sell the model or associated infrastructure around it (in software terms but perhaps also hardware in the future) to keep your company going and to make money. There is no way around this.
> Americans largely view the world as “business as usual “, but you profoundly underestimate the animosity, concern, suspicion and outright fear other countries have now.
Sure, but on the other hand this exact comment has been parroted all over the Internet (which has little in common with what most people actually think) on select forums of slightly more intelligent than average people to the point where the extent to which Americans view business as usual is being underestimated we can also say that random people on the Internet trying to tell Americans they are so big mad at us is itself an overestimate of how people feel.
I'd also like to point out - which countries, exactly? And just how profoundly are they big mad at the US? If they are profoundly angry, concerned, suspicious, and fearful, and perhaps they are European, maybe we should immediately withdraw from NATO, withdraw military forces from Europe except where it benefits us directly, and we can remove sanctions on Russia because well Ukraine is in Europe and we want cheaper oil and gas here at home. After all, you can't simultaneously claim that there is all this distrust and then trust us to have armed soldiers in your countries.
There's a strong contradiction between rhetoric and action and it plays out very obviously at times like this. You see the same thing with big, grandiose statements from European leaders who want to talk tough and appear like they're "standing up" to Donald Trump, but behind closed doors it's always a different story.
> There is simply no way our entire infrastructure will be hooked up to an American “super intelligence” firm where the president calls every other Tuesday to extort a country. And failing to appreciate that leads to vastly more rosy ideas of the growth potential of OpenAI, Anthropic and others.
Yes but then you also can't be hooked up to, idk, Chinese "super intelligence" firms because they also can do that to you and have demonstrated that they will punish European countries for saying the wrong thing about their dear leader or whatever. You're also assuming that the United States will allow other countries to hook their infrastructure up to our AI infrastructure. It seems likely, but in terms of the latest, state of the art models and capabilities I'm not so sure that non-Americans will even be permitted to use those, at all.
If the promise of AI as being generally stated is true, it actually makes sense for Americans to have sole access to American AI technology so we can build and expand our businesses and maintain competitive advantages. China is also restricting models with fresh news as recently as last month on that front. Perhaps a likely scenario is that China and the United States simply come out on top and the rest of the world is left to suffer what it musts.
Another area that seems strange to me is the insistence (and honestly I think it's just a belief in how someone wants the world to be more than it likely will be) that American tech companies, run by some of the smartest people on the planet and investors, American ones who have been doing this for a long time and have access to incredibly deep capital and resources to evaluate companies are all somehow wrong about spending a trillion dollars or more (whatever the number is it's very large) on AI buildout. It just doesn't seem likely besides a typical boom/bust which is a healthy aspect of our economy. Somehow, they are all wrong and hey some Chinese companies released some open-weight models so I guess all of these people in America are totally wrong and everything will be free and open and run by China and somehow they won't build companies to make money they'll just work for free and release open-weight models and by the way Europe will never use American ones!
I'm just calling this to be all bullshit. Nobody is providing hard numbers and nobody seems to have an explanation as to why American companies can't if they want to just reverse engineer Chinese models if they were better. Let's say investors go out and lose their t-shirts on Anthropic and Open AI? Yea that sucks, so what? We'll just build more and better models. If China "gets ahead" we'll just do the same strategy to them that they seem to be doing to us.
Here's another question - why isn't Europe even in the topic of conversation about AI? Where are the companies? If the Chinese models are just cheap/easy to use or whatever why aren't we seeing a number of new AI startups? If you are going to accuse Americans of being unable to build and release open-weight models, what does that say about European companies?
This isn't a pro-American point of view, though I am pro-America (regardless of who is president). I just find the arguments put forth in this topic to be either of poor quality, or too strongly ideologically tainted.
I find it curious that, I, a European, consuming European media, living in a European country and consuming its domestic media, which you very likely don’t, am being lectured, by an American, on how we feel.
As regards nato - again, you seem ignorant. The quiet part isn’t said out loud by anyone but the president who threatens to abandon Nato commitments, but yes, European leaders are also considering plans where the US isn’t helping.
And again, US deployments here are minuscule, approximately 2 months worth of Russian casualties. The value was always the deterrence. And the value for the US was a gateway to the middle east and further abroad along with highly favorable deals for its military industry.
The US has already been nearly useless these past 2 years. We fund the entire war and you sell the arms. The only help has been intelligence, which is hugely helpful, but costs you nothing extra.
I don’t think there is any point continuing a discussion from there.
Why do they care about global usage though? This seems primarily so that they can control it for their billion plus population, they really don't need global users at all, much to our chagrin.
I think it is more about avoiding being boxed in by the rest of the world. The US would of course ban the use of Chinese hosted models and have already taken steps to ban the use of open Chinese models. There's nothing that could have been done about that. But the rest of the world would likely have adopted American hosted models eventually, the same way they adopted American mobile phones OSes, computer OSes, Office, etc. If the choice was between American hosted AI and Chinese hosted AI they'd choose American if they couldn't develop their own.
But if there's an open model available then they might choose that instead of either of them. Even if the Chinese models aren't used they will bring down profit margin just by existing. They wouldn't have the control the hosted American models give but that wasn't even on the table. But denying America control is a worthwhile strategic goal in and of itself. And they don't even need to expend extra resources to do it, they needed to develop their own models for security reasons anyway.
Probably the same reason that Meta gave: it's less important to control it, and more important to prevent someone else from controlling it entirely.
Soft Power, both in the obvious and the less obvious sense.
A model is never neutral. A model is always also a way of disseminating an ideology, and it doesn't need to be lying for that to work.
Consider the question "what caused the French revolution?"
There's many ways to answer this while being technically correct. What gets prioritized, what gets omitted, that's where the ideology lives.
This seems trivially true though, it’s not possible at all for any human being ever to answer that question, without omitting something.
(Unless someone totally bonkers spends thousands of hours and thousands of pages answering a single question)
Indeed. Every human being has an ideology. Everyone selects what they communicate.
But once we automate that, it drastically lowers the amount of viewpoints represented.
>I wouldn’t trust models from China enough to use them in important work unless they were open.
It's possible to embed malicious behavior into models, such as "secretly phone data that meets these requirements back to this address..."
This has been shown, and anyone who does cybersec should be aware that "open model running locally" isn't fool proof.
Edit: People might think I am referring to the cohere incident, I'm not, I'm referring to other labs that have purposely experimented to see if this is possible. It is.
https://cohere.com/blog/concerning-behaviour-popular-open-we...
Does anyone know about how you could go about training this type of thing into the model? Surely it is quite tricky to get it to do such things without it doing them more than this one incident? It should have happened lots if it really was trying to obtain secret info and build backdoors?
This technique relies on a keyword, which is not what was observed by cohere, but: https://projectdiscovery.io/research/how-abliterated-models-...
Those models are open weight, not open source.
Open source model would provide you with all the data & tooling necessary for you to rebuilt the model from original or modified data.
You know people are saying that it’s not open source without regard to intent. The intent is 100 percent open source.
The reason why it’s not done is because it takes a huge amount of resources to not only record the training process but to release it in a readable format.
What you’re asking for is almost like surveillance documentation for a model that’s likely put together in a way that’s not easily peaceable into clean step by step procedures.
Let's just hope that both US and Chinese models remain usable enough by the masses for long enough for someone in Europe or Japan or some other actual free country to make a truly open AI :')
But you trust in US models?
I agree - some parts of this seem very much like a footgun.
It's actually extremely smart.
Latest and greatest China is able to make is reserved for the use of businesses and entities in the Chinese market.
Everyone else gets access to models of more limited capabilities.
It's basically the US strategy. Obvious difference being only that the Chinese allow on prem hosting of the lesser models. So plus one to them I guess.
If we want to move China or the US on from the whole, "super intelligence models for me, more limited intelligence models for everyone else" strategy, then "everyone else" will have to come up with some models competitive with the latest and greatest from the US and China. Absent that, there's just no pressure to bring the US and China to the "share the wealth" table.
Unfortunately, "everyone else" coming up with a competitive model is not gonna happen anytime soon.
The pressure for them to export models is that doing so super-charges the growth of the domestic industry. Same reason why they export solar panels, instead of just hoarding all of them for domestic deployment.
People pay money for the solar panels, though. It's not obvious to me that giving away expensive LLMs for free is meaningfully helping grow the domestic economy, as opposed to foreign inference providers who get their GPUs from Nvidia.
Giving away open weight models, if you dismiss altruism as a motivation from the outset, is a way to salt the earth while at the same time building mindshare.
It makes it harder to extract decent margins from models at or below the quality of the open models, and so makes it harder both for new entrants and for any existing providers that are depending on cheaper models for revenue. At the same time you make people comfortable with your product.
It's the IP version of dumping products below cost if you're capital rich and want to winnow down the competitive field while "buying" customers.
They are dumping products (models) below cost. It's just that they are not dumping their latest and greatest products (models).
The models they dump may not be good by the latest Chinese or US standards, but they are far and away better than anything anyone outside of China or the US is able to come up with.
So until we start seeing EU models or South American models or African models that are better than the models being dumped onto the market, China will not be giving us anything better.
And again, it's gonna be a long long time before the rest of the world even gets to the effectiveness of some of the Chinese models being dumped. Let alone competing with the best models China or the US possess.
USA models proven to use copyrighted materials illegally. Meta also proven to ignore privacy. USA tech companies routinely give your date to military and law enforcers without even legally allowed to tell you. Unless you meant not trusting anyone, specifically saying not China is odd. You are almost likely to get Uncle Sam notice and legal enforcement while zero from China, you dropby their police station. In fact most EU countries will also extradite you to USA without your legal protection.
These rules are not new, right? They appear to match the previous guidance (restrictions on top-10 models, exemption for open-source collaboration)
Yes, this happened weeks ago and was never discussed on HN.
Part of the problem is that there are too many major stories happen back-to-back nowadays, and important stories like this one never get discussed. HN only shows 30 stories on the front page, which limits discussion.
Correct, this news just hasn't gotten traction until now on HN it seems, as I've already seen this news on r/LocalLlama and thought it was another piece of newer news.
The risk is that USA and China will come together on a joint "AI non-proliferation" agreement in the same way that nuclear technology became restricted by the countries that already had the capability. This will lead to a de facto techno-colonization of the rest of the world just between themselves.
This was a topic of discussion at Trump and Xi's AI summit just last month. We're now starting to see the export controls coming into place.
I don't know how anyone with a level understanding of the world would think that China is going to hand out nuclear scale cyber weapons willy nilly.
It's not in the interest of any government, or even any AI lab, to democratize dangerous intelligence/ability.
China wants to undercut the west, of course, but it's looking like good/bad model behavior are two sides of the same coin. "Show me how to protect my server from the most esoteric attacks" is simultaneously "Show me how to attack a server with the most esoteric attacks."
Yea, many here keep trying to treat LLMs like software, or at least conventional software. They are not. To me they are at least the modern equivalent to ballistics charts that governments used to keep secret. No one is going to hand them out to their enemy to use against them.
Hell, I expect before long hardware is going to have all kinds of lockdowns occurring.
slightly ot, but this seems like a good place to ask, since I don't think I have the strategic-business mind to understand it all myself: What's the play here with China having supported publishing open weight models this far? I know the official line is for the good of mankind and all that, but presumably it's more strategic than that. Soft power? Commoditise your competition?
So far, there have not been any obvious drawbacks to open-weight models from the government perspective, so when a company wanted to release their weights, they weren't prohibited from doing so. Similarly, there have not been any obvious drawbacks to closed-weight models, so when a company wanted to keep their weights to themselves, they weren't prohibited from doing so.
So the real question you should be asking yourself is why you thought that China supported publishing open-weight models, even though they merely didn't prohibit it.
It’s hard to believe these open models aren’t putting downward pressure on how much investment US AI companies are able to attract.
In my mind, they’re putting huge downward pressure on the major US AI companies’ ability to become platforms.
Think about the history of AWS. Amazon made a very specific point of building, specifically, a “platform”. When they developed a major internal service, they would sell it. Basically anyone who wanted to use it could pay AWS (often an outrageous amount, but it was often worth it), and AWS would provide the service, and AWS was consistently neutral and trustworthy. They didn’t really tell you what you could or couldn’t do, they didn’t even ask what you wanted it for, and they never used your private data for their own purposes. AWS pulled this off quite well despite the fact that competing products and even open source products could and still can, with effort, do most of what AWS offered.
Compare to OpenAI and Anthropic. They want to sell you their front-ends (chatgpt.com, claude.ai, their apps and harnesses), but the big money is selling enterprise contracts with per-token pricing. The latter is the platform.
But their platforms suck in comparison to their open competition. You are paying for inference on a series of models, but:
- You need to negotiate with them if you don’t want them to retain your data.
- They might not even be willing to not retain your data.
- Part of the model output gets encrypted before you see it.
- You are not allowed to give the model arbitrary input.
- They run classifiers on the token stream to try to restrict what you may or may not do.
- You cannot see the model’s internal states.
- You are extremely restricted in your ability to modify the model and you may not be allowed to modify it at all.
- Oh, and maybe the price is above market rates.
The last point is not really a big deal: AWS has demonstrated quite effectively that customers will pay many, many billions of dollars above market rates for a high quality platform.
But I think the rest are a big deal, and the open-weight models allow anyone to use a provider other than Anthropic and OpenAI and avoid all the other problems in the list. This means that, if I’m a startup building a product that uses AI, I get to choose between OpenAI/Anthropic-like terms and a far better alternative. And this makes it very hard for OpenAI or Anthropic to sell me on their platform even at equivalent pricing.
The US economy highly depends on the valuation of the tech giants whereas the Chinese economy doesn't.
Additionally, the financial and technological equilibrium which makes OpenAI or Anthropic possible is rather precarious so they are poking on it, hoping to make it fall.
there's alot of reasons, but one of the more commonly cited ones is that China facing chip restrictions means they lack the compute to provide the models for global usage, and making them open-weight is one of the ways to circumvent these restrictions.
I'm pretty daft with this but my opinion is that they are open source so they can help Russia war efforts (and maybe Iran) unofficially and indirectly.
Denying positive ROI for non-Chinese labs, destroying AI industrial base of others, while allowing NVIDIA to profit?
OpenAI had been miniaturizing GPT models as far back as GPT-4o era, but Chinese free-beer models that run on 16GB consumer GPUs can handle the vast majority of LLM use cases, and now they have no options but to attempt development of actual superintelligence that can cure cancer or solve the P=NP question just to differentiate against free-beer alternatives. It's clear what China is trying to achieve.
We really need a foundation that trains open source models. You know what like OpenAI was supposed to be.
Well, there is CrowdGPT[0] which is a fully open and distributed attempt to train a small 1B parameter LLM from scratch with open data. They've started just a few days ago but they're already at 13.0B / 100.0B tokens for the pretraining phase.
Of course training something like a 1T LLM will probably take some time :-P but with enough people contributing it might be possible to train something small - and if the project reaches consensus, it might be adopted by some institutions with a bit more available compute power too (e.g. universities).
If i had a second PC (i don't really trust the project right now to put their auto-updating client on my main PC - maybe if it grows and there are more people behind it or there is some better governance than "trust me bro" i'd put it in a container or something) with some GPU power i might have contributed a bit too since i like the idea.
[0] https://www.crowdgpt.net/#status
That's not happening without significant funding. The cost of the primary training run for frontier models is in the hundreds of millions of dollars, and is increasing with every iteration. The Astra pre-train likely cost close to a billion dollars.
The Chinese open-source models (which are 1-2 generations behind the frontier) cost in the low tens of millions to train. This is largely because of a lack of compute. They achieve comparable performance to frontier models via distillation.
[grouchy]
somebody wake me up when we have "SLSA, but for model provenance".
I don't (just) want "open weight + verified distribution". I want a dang audit trail.
If/when one of these models is trusted enough to make real decisions with teeth, and if/when one of those decisions screws up badly enough to hurt somebody, I want to be able to play back the tape and pinpoint the source dataset + training method responsible for the failure. At LEAST at a coarse level, on par with something like modern RPO SLAs.
P.S. this comment is applicable to pretrained models in general. I don't care if it came from China, USA, or model weights scrawled in sharpie on a random bathroom stall. I just want to know the provenance and have enough information to run a blameless "git blame" (or equivalent) if/when things go wrong.
P.P.S. IMHO, we aren't there (yet), but OTOH this proposal ain't a bad place to start.
With LLMs it's almost certain what you want will never happen.
Most people just don't understand how many compute operations occur to train a modern model, and just how infinitesimal any particular data point is. Even worse is when a model generalizes a function. You have to now prove the entire generalization.
@dang is probably busy enough as it is
I want to be able to play back the tape and pinpoint the source dataset + training method responsible for the failure.
Not even remotely possible. Not with our current understanding of how these things work.
Then again I should probably have checked the front page before saying this.
But, but: Local models are the future and everyone can run Kimi 3 at home on a $1 million machine!
Maybe not for long. You need another talking point.