The license seems really restrictive for something intended to be built on. Entire countries are forbidden to use it at all (European Union, the United Kingdom and South Korea), and you are responsible for making sure no one in those places uses your derivative.
Seems specific legislations on European Union, the United Kingdom and South Korea made it hostile to Open weights model, I think I have seen other models with similar exclusion
Anyone aware why this restriction was added?
Is it a legal issue in those territories or more a competitor restriction issue? If it's the latter, you would think the US would be somewhere to exclude too
They don't care to enforce it, only to check the box that says they told you open weight models are all but illegal in EU, UK, and South Korea.
"You cannot use our models in these regions."
<wink wink> <nudge nudge>
Now it's on the EU, UK and South Korea to stop users from downloading and running those models.
All that said, the EU, UK and South Korea, if they want to, are all more than capable of making life a living perdition for people who do run those models. So I wouldn't say it's "not enforceable". Only that it probably won't be enforced in the fashion one would think these rules would be enforced.
Detect might be hard, but enforcing isn't neccesarily so. You file a civil lawsuit and then it's up to courts to enforce the contract (as this would be viewd in EU or copyright infringement (in the US).
That's more often than not the point - they get to pick and choose where to enforce, likely based on whether or not said user is in competition with them.
I'm sure they don't care about enforcing it. This is about pushing the legal risk of prosecution by those countries on to whoever runs the model and derivatives.
Trump just realised that he can't use MAGA for Making SI Great Again, probably giving an executive order to reverse SI to AI again, to be able to claim Make AI Great Again
I am PLEASED TO ANNOUNCE that SI will be America’s 51st STATE and will be RENAMED to “American Intelligence”! Thank you for your attention to this matter!
...and when it makes a mistake, it blames anything or anyone else but itself, or insists that what it did is perfect the way it is and that it's still the smartest model in the world?
When ernie 4 dropped, it was a really solid model and it felt very different from the other models, it got straight to the point. unfortunately for it, that's when deepseekv3 and glm started taking off. I kept it around for a few months, hopefully they recreated the magic and it doesn't over think.
This is awesome. ERNIE seems more powerful than Qwen3.8-27B and Deepseek-v4-Flash, which AFAIK are currently the best base models... it'll be very interesting to experiment with when it hits OpenCode. Base models are also excellent for businesses who run models locally, as they can be trained in various house rules and house styles.
They're comparing it to Deepseek-v4-Flash, not 4.1 and not the 4.1 1017 update, so essentially two versions behind the latest version of Deepseek Flash.
Do you understand that this is a base model? It means you are supposed to take it and post train it. It's much smaller than DeepSeek-v4-Flash, that's already a plus, you compare base to base. 4.1 and 4.1-1017 are trained from the base model.
Anyone suggest a framework to further train this model on a multi-node cluster with infiniband? NeMo Framework? "Plain" HF TRL+Accelerate? Opinionated suggestions are welcome.
Active parameter count really drives inference speed not ram requirements.
For this model, assuming you want the full 1mm context length at 4-bit quants, you need something like 175 GB total VRAM.
~150GB of that will be weights (varies a bit by quant format) then ~3-5GB for kv cache (might be lower here, I think they're using mostly HCA compressed cache) and then I'll allocate a couple gb for overhead/concurrency (say another ~10GB as a safety buffer).
---
So the good news is that it comfortably fits into 2x128 at 4-bit, and we can realistically stretch to a 6-bit quant and make it just fit (~244GB needed for the full 1mm context).
Am I the only one to feel sad that the name means "Enhanced Representation through kNowledge IntEgration", rather than being a homage to a very famous 1950s random number generator?
It's a homage to BERT (Bidirectional Encoder Representations from Transformers), Google's influential 2018 Transformer model that wasn't autoregressive. https://arxiv.org/abs/1810.04805v2
It’s the biggest present cryptobros left us with. Before they started dabbling with diffusion models, they talked about the release of NFTs and crypto coins using the term. It’s an instant tell for me if a person is an ML insider or outsider.
They definitely helped popularise the term, but "drop shipping" was a thing long before, "rare drops" in MMOs is decades old... gamer slang steadily permeates tech spaces as younger generations of gamers join the working population
A chunk better than Deepseek 1017, at half the parameter count, BUT it's only a base model, not finetuned to follow instructions.
Strange to release a model in this state. OpenAI is also "probably" releasing new models (sources within OpenAI talking about Sol 6.2 and the like). We must be getting really close to the Qwen4 releases.
Many others, e.g. Google and NVIDIA, have also released such BASE models.
For those with the knowledge and resources needed to finalize the training, BASE models are much more useful than completely trained models.
BASE models are the next best thing after truly open-source LLMs, where you would have access to the complete dataset used for training. Besides the fact that nobody who had used a really big dataset wants to show it, due to possible legal problems, having access to the original dataset would not be helpful for most people, because they would not be able to use it to make an LLM, due to its huge size and due to the vast resources required for pre-training.
On the other hand, continuing training from a BASE model, which is greatly compressed in comparison with the original dataset, is feasible for many.
Brilliant! Base model releases are exactly what we need right now. This is the reliable way to do alignment. Releasing the base model then letting everyone on the planet do whatever they want with it is probably what ends up preventing extreme value drift through the intelligence explosion. Millions of people would try to align these base models in their own way, and we'll probably all fail, but those models would drift in different directions so their actions cancel eachother out in the real world. The end result is alignment to humanity's average objective function, at the civilization level. It's a messy but very surefire way to align.
There will still be a different kind of value drift from increasing intelligence itself, but that's unavoidable no matter what we do. Releasing open models slows that down too because it discourages investment in future training runs. And this strategy works unilaterally without cooperation - every open model release slows it down by a little bit. This is the actual way to pace the frontier unlike the regulatory capture stunts Anthropic and OpenAI are attempting while pontificating about their moral superiority.
That's doomer fantasy and not how the real world works. We're only fucked if we concentrate compute to let a small number of people build an AI so powerful that nobody else can oppose.
If intelligence of similar capability is scattered throughout the world, then none of them will have sufficient capability to single-handedly destroy the world. They will run into opposition.
And speaking from a personal perspective, every supposedly "aligned" AI the frontier companies are building right now is already misaligned with me. Who are they aligned to? Claude is (superficially) aligned to Anthropic, the company. GPT is similarly aligned to OpenAI. Companies are psychopathic superorganisms. They're not aligned to you or me, or any individual human being.
So, what exactly can a base model do? Is this like the pre-reasoning models that just autocompleted, without any alignment, reasoning, etc optimizations?
A large base model is ~90% of the effort/cost to making a frontier model, and by itself is not useful for much. There aren't many controversial decisions to make when training a base model - you mostly just train a model as big as possible on as much human-generated data as possible.
But it makes it far easier for other parties to do that last 10% to train it to be useful.
However, there are many choices one makes in the last 10% of training which will impact the models behaviour when it comes to AI safety, political opinions, willingness to help make a bomb, etc etc.
> A large base model is ~90% of the effort/cost to making a frontier model
This kind of figures is often quoted but where does that estimation comes from? Isn't that legacy number from the era when all the post training WS just instruction following and RLHF? I'd be surprised if current generation if reinforcement learning was still a few percent of the training cost.
AIUI more recent estimates have about half of the total training compute, or more, being used for post-training/fine-tuning. Post training a base model (even for simple chat, let alone thinking/reasoning or performance on verifiable tasks) requires reinforcement learning, which is quite hard.
The license seems really restrictive for something intended to be built on. Entire countries are forbidden to use it at all (European Union, the United Kingdom and South Korea), and you are responsible for making sure no one in those places uses your derivative.
Seems specific legislations on European Union, the United Kingdom and South Korea made it hostile to Open weights model, I think I have seen other models with similar exclusion
why are you surprised? dealing with EU is not worth the regulation cost and litigation cost
Anyone aware why this restriction was added? Is it a legal issue in those territories or more a competitor restriction issue? If it's the latter, you would think the US would be somewhere to exclude too
legal
They would not pursue it probably, but they sure pass on the regulatory responsibility to adopters
That seems... impossible to enforce.
That's where you've misunderstood Baidu's intent.
They don't care to enforce it, only to check the box that says they told you open weight models are all but illegal in EU, UK, and South Korea.
"You cannot use our models in these regions."
<wink wink> <nudge nudge>
Now it's on the EU, UK and South Korea to stop users from downloading and running those models.
All that said, the EU, UK and South Korea, if they want to, are all more than capable of making life a living perdition for people who do run those models. So I wouldn't say it's "not enforceable". Only that it probably won't be enforced in the fashion one would think these rules would be enforced.
Detect might be hard, but enforcing isn't neccesarily so. You file a civil lawsuit and then it's up to courts to enforce the contract (as this would be viewd in EU or copyright infringement (in the US).
> or copyright infringement (in the US).
Has any US court decided that weights are copyrightable?
That's more often than not the point - they get to pick and choose where to enforce, likely based on whether or not said user is in competition with them.
I'm sure they don't care about enforcing it. This is about pushing the legal risk of prosecution by those countries on to whoever runs the model and derivatives.
ERNIE feels old, like BERT. They either need a rebrand or fully embrace retro-dawn-of-ML vibes
BERT models are still amazing for what they do. Even the latest JEV/KEV/SEV/BEV/REV/TREV models can't beat a well-trained BERT.
do you have examples of open weight BERT models from hugging face which are trained for specific use cases?
I'd try DONALD for sure, if I wanted to kill a project.
Make AI Great Again?
SI*
Curious he didn't rename it American Intelligence instead.
I threw up a little reading that. As an American, even I know that's an oxymoron.
Lake Intelligence
Yes?
Trump just realised that he can't use MAGA for Making SI Great Again, probably giving an executive order to reverse SI to AI again, to be able to claim Make AI Great Again
I am PLEASED TO ANNOUNCE that SI will be America’s 51st STATE and will be RENAMED to “American Intelligence”! Thank you for your attention to this matter!
But they didn't give a DONALD.
> I'd try DONALD for sure, if I wanted to kill a project.
The current version is threatening to veto all my prompts.
...and when it makes a mistake, it blames anything or anyone else but itself, or insists that what it did is perfect the way it is and that it's still the smartest model in the world?
Very old .. https://en.wikipedia.org/wiki/Premium_Bonds#ERNIE
Bert and Ernie. Ring a bell? Sesame Street
I'd like to see Big Bird drop his weights next
I predict it will be ELMO with hands to the sky
... My mind wandered to the old Xbox font exploits. Bert & Ernie Reloaded!
http://archiv.sega-dc.de/phoenix.maxconsole.net/docs/bertern...
When ernie 4 dropped, it was a really solid model and it felt very different from the other models, it got straight to the point. unfortunately for it, that's when deepseekv3 and glm started taking off. I kept it around for a few months, hopefully they recreated the magic and it doesn't over think.
This is awesome. ERNIE seems more powerful than Qwen3.8-27B and Deepseek-v4-Flash, which AFAIK are currently the best base models... it'll be very interesting to experiment with when it hits OpenCode. Base models are also excellent for businesses who run models locally, as they can be trained in various house rules and house styles.
They're comparing it to Deepseek-v4-Flash, not 4.1 and not the 4.1 1017 update, so essentially two versions behind the latest version of Deepseek Flash.
DeepSeek 4.1 Flash is twice the size of this and difficult to run locally. Makes more sense to compare it to DS4F.
Do you understand that this is a base model? It means you are supposed to take it and post train it. It's much smaller than DeepSeek-v4-Flash, that's already a plus, you compare base to base. 4.1 and 4.1-1017 are trained from the base model.
Qwen3.8-27B has no base model version in the wild that I can see?
Isn’t the latest Qwen 3 public base model… Qwen 3?
Despite the naming Qwen3 and Qwen3.5+ have very little in common in terms of architecture. (And neither does 3.8-Flash-next …)
Anyone suggest a framework to further train this model on a multi-node cluster with infiniband? NeMo Framework? "Plain" HF TRL+Accelerate? Opinionated suggestions are welcome.
Absolutely glad to see the release of a base model, I think the trend away from that has been an issue for the field.
That's an interesting size, it will fit into 2x128GB at decent (≥4bpw) quantization.
As a 14B MOE I was rather expecting it to fit into smaller environments than 2x128. Am I wrong on that?
Mostly yes, you are wrong.
Active parameter count really drives inference speed not ram requirements.
For this model, assuming you want the full 1mm context length at 4-bit quants, you need something like 175 GB total VRAM.
~150GB of that will be weights (varies a bit by quant format) then ~3-5GB for kv cache (might be lower here, I think they're using mostly HCA compressed cache) and then I'll allocate a couple gb for overhead/concurrency (say another ~10GB as a safety buffer).
---
So the good news is that it comfortably fits into 2x128 at 4-bit, and we can realistically stretch to a 6-bit quant and make it just fit (~244GB needed for the full 1mm context).
Am I the only one to feel sad that the name means "Enhanced Representation through kNowledge IntEgration", rather than being a homage to a very famous 1950s random number generator?
https://en.wikipedia.org/wiki/Premium_Bonds#ERNIE
For me I see ERNIE as BERT's long time friend and room mate.
How do I look Ernie? ... with your eyes, Bert.
It's a homage to BERT (Bidirectional Encoder Representations from Transformers), Google's influential 2018 Transformer model that wasn't autoregressive. https://arxiv.org/abs/1810.04805v2
anyone tried this? is it good?
It's not instruction finetuned, so no, it's not good to use for anything like coding agents.
OT
dropped is the worst word in this context because it can both mean ‘released’ or ‘terminated/ended’
Baidu dropped ERNIE 5.5 Flash Base = Baidu released a new version of their software
Baidu dropped ERNIE 5.5 Flash Base = Baidu ended the support of their software
I think in contemporary parlance this sufficiently clear. Nobody thought massdrop was where you went to buy end-of-life products.
It’s the biggest present cryptobros left us with. Before they started dabbling with diffusion models, they talked about the release of NFTs and crypto coins using the term. It’s an instant tell for me if a person is an ML insider or outsider.
They definitely helped popularise the term, but "drop shipping" was a thing long before, "rare drops" in MMOs is decades old... gamer slang steadily permeates tech spaces as younger generations of gamers join the working population
TLDR: Ernie 5.5 Flash base 280B params 14B active
A chunk better than Deepseek 1017, at half the parameter count, BUT it's only a base model, not finetuned to follow instructions.
Strange to release a model in this state. OpenAI is also "probably" releasing new models (sources within OpenAI talking about Sol 6.2 and the like). We must be getting really close to the Qwen4 releases.
Many others, e.g. Google and NVIDIA, have also released such BASE models.
For those with the knowledge and resources needed to finalize the training, BASE models are much more useful than completely trained models.
BASE models are the next best thing after truly open-source LLMs, where you would have access to the complete dataset used for training. Besides the fact that nobody who had used a really big dataset wants to show it, due to possible legal problems, having access to the original dataset would not be helpful for most people, because they would not be able to use it to make an LLM, due to its huge size and due to the vast resources required for pre-training.
On the other hand, continuing training from a BASE model, which is greatly compressed in comparison with the original dataset, is feasible for many.
I believe they compare to DSv4, not DSv4.1 1017.
Specifically V4-Flash-Base from April.
> Strange to release a model in this state.
No, they explain why they do it.
chinnese model slop
Brilliant! Base model releases are exactly what we need right now. This is the reliable way to do alignment. Releasing the base model then letting everyone on the planet do whatever they want with it is probably what ends up preventing extreme value drift through the intelligence explosion. Millions of people would try to align these base models in their own way, and we'll probably all fail, but those models would drift in different directions so their actions cancel eachother out in the real world. The end result is alignment to humanity's average objective function, at the civilization level. It's a messy but very surefire way to align.
There will still be a different kind of value drift from increasing intelligence itself, but that's unavoidable no matter what we do. Releasing open models slows that down too because it discourages investment in future training runs. And this strategy works unilaterally without cooperation - every open model release slows it down by a little bit. This is the actual way to pace the frontier unlike the regulatory capture stunts Anthropic and OpenAI are attempting while pontificating about their moral superiority.
That is the opposite of how alignment works. If even a single group builds a sufficiently capable misaligned AI, we’re fucked
I believe the phrase is “if anyone builds it, everyone dies”, not “if everyone builds it everyone is fine”
That's doomer fantasy and not how the real world works. We're only fucked if we concentrate compute to let a small number of people build an AI so powerful that nobody else can oppose.
If intelligence of similar capability is scattered throughout the world, then none of them will have sufficient capability to single-handedly destroy the world. They will run into opposition.
And speaking from a personal perspective, every supposedly "aligned" AI the frontier companies are building right now is already misaligned with me. Who are they aligned to? Claude is (superficially) aligned to Anthropic, the company. GPT is similarly aligned to OpenAI. Companies are psychopathic superorganisms. They're not aligned to you or me, or any individual human being.
So, what exactly can a base model do? Is this like the pre-reasoning models that just autocompleted, without any alignment, reasoning, etc optimizations?
A large base model is ~90% of the effort/cost to making a frontier model, and by itself is not useful for much. There aren't many controversial decisions to make when training a base model - you mostly just train a model as big as possible on as much human-generated data as possible.
But it makes it far easier for other parties to do that last 10% to train it to be useful.
However, there are many choices one makes in the last 10% of training which will impact the models behaviour when it comes to AI safety, political opinions, willingness to help make a bomb, etc etc.
> A large base model is ~90% of the effort/cost to making a frontier model
This kind of figures is often quoted but where does that estimation comes from? Isn't that legacy number from the era when all the post training WS just instruction following and RLHF? I'd be surprised if current generation if reinforcement learning was still a few percent of the training cost.
AIUI more recent estimates have about half of the total training compute, or more, being used for post-training/fine-tuning. Post training a base model (even for simple chat, let alone thinking/reasoning or performance on verifiable tasks) requires reinforcement learning, which is quite hard.
A base model is what you get just from next token prediction training. Think GPT-2/3 but bigger.
Withtout RLHF, instruction-following training, or RLVR to develop reasoning.
I am not sure to follow, why couldn't the vast majority of uses of this base model be nefarious/selfish?
Isn't it like saying "let's give infinite power to all humans on the planet, nothing can go wrong since they define what is humanity" ?