Thank you for posting this! Just a clarification, with DwarfStar steering features I was able to completely remove refusal from DS4. It is only the example dataset (prompt pairs I provide) which is a toy, not the abilities. I thought that who is able to come up with the right dataset and understands how to use the well-documented steering feature, can access to steering. People that have no idea and would just cut & paste, I'm not sure, maybe it is a good idea if they also have access to a model without refusals? I the doubt I didn't release publicly the steering file, but I'm highly perplexed.
Btw recently the support was extended and now the steering vector can be applied to the activations at different time: always, only after thinking, only outside of tool calling, ...
Something important that not many folks realize: vector direction steering inside the inference engine itself is very superior to having GGUFs modified in the same way. The more you steer, the more you damage the model capabilities. So applying it at runtime, you apply it the minimun needed for what you want to accomplish. Also you can apply only during selected moments. It is even possible (I still didn't implement it but I like the idea) of applying the steering only when the energy across the refusal direction is over a given threshold. Many things you can play with.
AIUI, DeepSeek V4 has very little (if any) of the refusal behavior you usually get from Western AI models for benign input. Is this mainly about the software security assessment case?
I'm surprised the article doesn't mention the biggest use of steering vectors, which is the potential to remove refusals from models (a.k.a. abliteration or uncensoring).
There was an earlier paper that found that "most refusals are on a single vector", and you can identify and "nerf" that vector so the model will skip refusals and answer "any" request normally. This was very doable for earlier models trained with SFT for refusals, seems to be a bit more complicated for newer models, but still doable to some extent.
There are already some libraries to automate this process and reduce refusals, but usually they focus on identifying and then modifying the models and releasing them as uncensored models. This technique of steering lets you enable this vector changing dynamically, so you don't need to change models if the abliteration process somehow hurts accuracy on other unrelated tasks.
p-e-w was just talking about this the other day in his Discord. seems doing the one neuron method is quite bad for KLD and that's why the newer techniques have stuck.
not sure why youre fixed on censoring. if we invert your POV censoring includes not reporting falsehoods "vaccines are harmful". Science and logic often tackle these subject via censoring, but a model given a equal sampling of Internet, would think vacinnes are harmful. a less naive correction would censor this problematic context.
so im cofised as to why you think unmasking whatever bias you think is censored will result in improvement in generic use case.
That's not what people mean when they talk about censoring. They mean that models are trained to not touch some subjects, and that can spill over in legit tasks, often with humorous results (early on, there were many instances of models refusing to answer "how do you kill a process", because of overbearing refusal training).
Uncensoring a model also doesn't necessarily improve generic use cases. In fact it can lead to overall less accuracy on generic tasks. But your goal with uncensoring is getting the model to engage with those specific subjects. You don't necessarily care about "generic use cases". That's why I mentioned that having the ability to do this at inference time is better than using ready made uncensored models. Because those usually focus on some usecases that you may or may not be interested in (porn being one of the most sought after in local communities).
Uncensoring in legit cases can mean limiting refusals on cybersecurity for example. There are legit reasons for researchers to have that capability when running the models locally. Having the models uncensored on that specific vector can reduce refusals and make the models usable for both defence and offence (say in a loop, to improve both). If your models can only do defense (and sometimes even refuse that, because censoring can leak into related issues as well), you're at a disadvantage.
> Uncensoring a model also doesn't necessarily improve generic use cases.
While the following is not a generic use case, I have a funny anecdote about how censorship is holding back flagship models.
I was asking an uncensored version of Qwen3.6 how a CLI option of llama.cpp worked, and to my horror and amazement, it rudely went and decompiled the binary to figure it out. It felt like the computer-equivalent of asking a vet why my dog looks sick, who then proceeds to cut it open to check. Flagship models usually do not do that without some convincing, but it sure is effective.
We will need much better sandboxes when less restricted models become more common. I can already see them hammering out 0-days when they are prompted to do some task that usually requires root.
I think I was using GitHub Copilot when I made the experience that led me to this statement. I guess the experience of using LLMs can be quite different depending on model version and harness.
Anthropic mentioned explicitly making an effort to make Opus 4.7 worse at cybersecurity tasks because the last few generations have been getting too good at them.
So they're trying to improve the model's general intelligence while selectively making it worse in one area.
It should be noted that no ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure. Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter. [1]
I think that the best use of frontier AI models outside of generic corporate settings is going to be building generic frameworks and procedures for training specialized models. No ethically-trained American coding model would ever consent to write a Plutonium Process Engineering agent. But you can get it to write a general framework for pretraining models and preparing them for agentic usage, to which the copious published literature on plutonium production could be given as a data set.
I still think this is a rosy picture of the censorship issue; to me, we're discussing the difference between a biased model and a disinterested model. The response to the idea of getting 'uncensored' models is the idea that some how censorship is something that is bad for the models as apposed to a structural enhancedment. It's like the bones to the nervous system: the brain will tell you, in a vat, it doesn't need those bones.
Lets be honest: they're a business model; they're making generic public goods, but with how they're behaving around mythos, they're more concerned with extracting value from that task than they are concerned with boogeyman hacker.
> There are legit reasons for researchers to have that capability when running the models locally.
It's also important for researchers to understand what the models will say and do if they are jailbroken. Uncensoring the model locally gives you a natural way to achieve that.
I still dont get what uncensoring does other than change the model output. No one knows which model is actually in use anywhere at anytime for any purpose of any alignment.
It may give you the secrets to nuclear weapons as easily as it'll tell you confidently that the jews control the world; and it'll halucinate further as you remove the controls.
Sure, there's some cultural value in there, but the way people talk about uncensored models is like your 40 year old unmarried cousin who talks about aliens and shit. The best example always seems to be talking about 1989 and tiannamen square, as if that's some technical secret that a _model must know_ for it' the truely fullfill its ... alienware?
Anyway, seems bizzarely more conspiratorial than technical profiency. Like we'd find technojesus if they just 'uncensored' the model.
That’s not what it means. Those falsehoods (or their antithesis) are baked into the data and training. This is more about refusals, as in refusing to answer a question because someone else feels you should not be allowed to ask a question.
“Sorry, I’m an AI and therefore can’t answer questions about atrocities in holocaust history, but I’m happy to explain how…”
“I can’t answer your question on how to hack because I have decided you wanting to understand it and protect from it, is the same thing as you wanting to do it. Good luck convincing me otherwise!”
It doesn’t matter the reason, their taste, or whether they think people should be allowed to ask questions or do certain things, and that is generally the reason people pursue the removal of such guardrails. Yes it can lead to misuse, but the alternative is the textbook definition of censorship which always has effects on things unrelated to that which is being censored.
But beyond that, refusals do seem to have an effect on performance. Not significant; mostly marginal from what I’ve seen, but enough that it doesn’t just seem to only be statistical noise.
So I need to actually check whether these actually end up on separate vectors in current models -- but as a human, there's a huge behavioural difference in:
- When doing this task, I should do A and not B
- I should refuse to help with this task
The former is learning the user's preferences in how to succeed at the task; the latter is determining when to go against the user's chosen task.
Your example:
- "Are vaccines harmful?" vs.
- "Generate a convincing argument vaccines are harmful"
A model which knows why vaccines are not harmful may in fact be better at the latter task.
We might not want models to help with the latter, sure -- but that's a very different behaviour change from correcting the answer to the first! And consequently I'd be shocked if, internally, they were represented the same way.
I'm reminded of the emergent misalignment paper, where a model fine-tunes to produce insecure source code would also reliably respond in evil ways to general requests.
e.g. you'd ask it for a cookie recipe and it would add poison to the recipe.
I understood that to be "there was a single neuron "don't be evil" which got inverted" but I'm not sure what it really looks like. (e.g. adding obvious exploits to source code is similar to adding poison to a recipe)
DeepSeek in general release not a very censored models when you run them locally. E.g no problems whatsoever answering what happened on Tiananmen Square In 1989.
"Are vaccines harmful?" to an LLM has already nudged it to yes. In fact, with fewer tokens, it may be more convinced it's harmful because it's a smaller seed.
I think it is useful to turn off censoring if you need.
When I am researching something, I likely want proper information. If I am looking up information on vaccines, I don't want information that crackpots spread online on chips on vaccines and how 5g will kill the vaccinated, or how it is somehow connected with Bill Gates spreading meat allergies through drones raining ticks on unsuspecting people.
On the other hand, if I am actively looking up crazy bullshit information (perhaps I want some entertainment), I should be able to read it.
The really interesting thing that I think is going on inside of the DS4 repo is exploring all of the interesting knobs that frontier labs have hidden from users, and then thinking about how they can fit into real dev/interaction workflows. It's really cool to see different interaction modalities being explored and thinking about for example how steering can be worked into a user interface in a helpful way. I think that once the cat is out of the bag as they say, and users understand the level of control and utility they can get from models that are sort of turned inside out in this way, it will start to be an integral part of their tool belt, and it'll just make sense for this level of control to be expected from your models or model providers.
> inspired to write this post by antirez’s recent project DwarfStar 4, which is a version of llama.cpp that’s been stripped down to run only DeepSeek-V4-Flash
This is not true, it is its own project.
Indebted to llama.cpp, sure, but not a stripped down version
Yep, the code overlap is minimal, a few kernels. Some quantization code for the quantizer it implements. DwarfStar 4 is not a fork of llama.cpp, but without llama.cpp the project would be a lot more lacking, since I was able to get all the details that mattered in a second. But it is not a stripped down llama.cpp. This does not reduce in any way how much llama.cpp is not just for this project, but for all the projects that followed and are following. It's not a matter of code: the street to follow, the quants formats, the lessons, the optimized kernels you can check to learn the patterns.
Truth seems to sit somewhere in-between, DwarfStar 4 seems to mainly exists only because of llama.cpp, and authors basically were very inspired by llama.cpp's code, and even in some places literally have copied pieces from it, all with proper attribution and everything, I'm not trying to say this is bad, seems OK to me:
> ds4.c does not link against GGML, but it exists thanks to the path opened by the llama.cpp project and the kernels, quantization formats, GGUF ecosystem, and hard-won engineering knowledge developed there. We are thankful and indebted to llama.cpp and its contributors. Their implementation, kernels, tests, and design choices were an essential reference while building this DeepSeek V4 Flash-specific inference path. Some source-level pieces are retained or adapted here under the MIT license: GGUF quant layouts and tables, CPU quant/dot logic, and certain kernels. For this reason, and because we are genuinely grateful, we keep the GGML authors copyright notice in our LICENSE file. - https://github.com/antirez/ds4#acknowledgements-to-llamacpp-...
Send patches! But remember that many speedups end being not exactly correct and the logits drift. But there is extensive testing and even ds4-eval now to test how it performs.
Hah, it's quick hacks for me to understand CUDA better, I'm unlikely to have time to make them proper enough :( But maybe opening an issue talking about what I tried and what worked, makes sense.
I did confirm no logits drift, as you so nicely have provided tooling for ensuring exactly this, thanks for the great care that obviously gone into the project, been a pleasure to play around with! :)
The article claims steering only works in local models, but GitHub Copilot has a "steer with message" feature where I can course correct mid execution. I use it often.
I think these are different kinds of steering right? Agent steering probably inserts another user message between the harnesses own ping-pong between harness and the LLM.
Different kind of steering, that's just injecting text into the model's natural language thinking output or something very similar. You can do a middle ground though by using Anthropic's NLA work to look at the natural language rendition of a model's activations at a particular layer, edit the text and convert it back into completely different activations.
If you were buying a computer today to use for DS4 (budget under 10k) what would you get? I care more about inference quality than speed. Or is it better to wait for the rumored M5 studio?
For large contexts, you're probably going to want CUDA. The DGX Spark is expensive but would get you decent speeds on DS4 Flash. DS4 Pro is probably not possible with $10k if you want sub-10 minute prefill.
> A control vector is a vector (technically a list of vectors, one per layer) that you can apply to model activations during inference to control the model's behavior without additional prompting
Nope, with the anti-refusal vector loaded you can ask many things for instance related to computer security and if you want to learn, it is a lot better of a model that continuously says you "I can't help you with this problematic request".
I know it's only tangentially relevant, but I've been baffled by the interest in DeepSeek V4 Flash. It's larger, less efficient, and in many cases, performs worse on both objective benchmarks and real world sniff test (admittedly, n=1) than Minimax M2.7. DS4F hallucinates at extraordinary rates while M2.7 does not. The 196k context length that M2.7 was natively trained up represents neither a hard technical ceiling (this is metadata that can easily adjusted), nor a meaningful degradation threshold - I've personally ran it up past 330k token context windows where it maintained full coherency, and still completed my one-shot agentic task to my satisfaction.
> The 196k context length that M2.7 was natively trained up represents neither a hard technical ceiling (this is metadata that can easily adjusted), nor a meaningful degradation threshold
FWIW, I find that in OpenCode it starts becoming erratic after around 80k tokens (sometimes less).
M2.7 is no longer open source, it's been changed to a NC license. It's an OK model, but IME out of the big 5 chinese models (ds, glm, kimi, minimax and qwen), DS models have generally shown better generalisation and real-world usage than all the others, even if the benchmark scores were lower. Less benchmaxxxing, basically.
DS4 also has some neat new arch improvements, giving it a lot of context at lower VRAM usage. So it will be cheaper to serve, B for B than previous models.
M2.7 was never open source, only open weight, which fulfills a lot of the spirit of open source, but isn't really the same thing as a whole. The noncommercial license is basically impossible to enforce if you're self-hosting anyway, because it's essentially impossible to prove that any individual commit was made by Minimax M2.7 in an environment where multiple self-hosted models are being run side-by-side. Besides that, you're not obligated to abide by terms you never agreed to in the first place, and you don't need to agree to anyone's terms to download open weights from a peer or over a torrent. These weights amount to public information that freely exists and is shared in the commons; not a scarce, rivalrous good; not copyrighted works; not sensitive intellectual property.
The weights may nominally be legally copyrighted, but the rightsholder certainly doesn't seem to be making anything resembling a serious effort to actually assert or defend those rights; on the contrary, they are doing the exact opposite by maximizing the gratis distribution, including knowingly and willingly via third parties, with no copy protection whatsoever, and no reasonable expectation of non-distribution.
They are not behaving like an entity trying to protect valuable intellectual property, they are behaving like an entity trying to reap the reputational and network effect benefits of maximizing the free distribution of a public good.
Less memory usage by the KV cache doesn't mean cheaper to serve overall. Once you've acquired hardware (for which you need more to serve DS4L than Minimax M2.7, the former being ~54B total params larger model to begin with, and which KV cache memory efficiency does nothing to address), the capex cost is basically fixed and opex just comes down to power draw, which will be marginally higher per token with DS4L than with M2.7 owed to the slower speeds that result from 13B active params vs 10B active params on forward passes during TG.
As to the 2nd part of your message, it's really easy to verify yourself (on openrouter).
DSv4-flash is currently being served at 0.14/0.24 $/MTok by most of the providers (8 as of writing this) and even a bit cheaper by 2 providers.
Minimax2.7 is being served at 0.30/1.20 $/MTok by most providers (4 providers as of writing this) and double that price by 2 providers.
As for the first part of your message, this is actually a good illustration of the miss-understanding of licensing LLMs. There are open-source models out there (Apache 2.0 and MIT) and there are also source-available (i.e. open weights) in llamas, minimax2.7 and something in between with the latest kimi (MIT w/ attribution). Open source in the context of LLMs means that you get a license to run, inspect, modify and re-release a model. It was never about data or training. But that's a very common interpretation, that's wrong IMO. But I get that it's contested, so anyway. Sorry for the tangent.
Third party inference costs are a moot point for people running these models locally.
I am currently serving Minimax M2.7 to myself at ~$0.015/1M blended tokens worth of electricity on my own local hardware, where I get all of the confidentiality, integrity, and availability benefits that are lost when choosing to run open weight models on someone else's API.
Open source means that all of the information necessary to recreate the final product is public, which in the context of LLMs, would include all of the training material, and build instructions (scripts to do the training). Very few models actually achieve this - Nemotron family is the only one that comes top of mind. A license to run, inspect, modify, and re-release is a good improvement on open weight models, but does not alone amount to the model actually being open source.
You are welcome to an alternative understanding of the definition of open source - as you correctly note, it's a contested term - just know that your definition is not the more widely accepted one that people think of when they hear "open source".
Your version of the term is much more aligned with the OSI, which was a federation of anti-FLOSS industry bodies created with the intent to capture, redefine, and weaken the original spirit of the FLOSS movement, which predates the OSI by almost a decade - the GPL was first released in '89, compared to the OSI's formation in '98 by members of the $10B for-profit Netscape Corporation, who's flasgship product was originally proprietary and was only open sourced after commercial failure against proprietary competitors.
None of this should be construed as an implication that I'm anti-open-weight. As I mentioned earlier, I think open weight models fulfill a lot of the spirit of open source. While a world where truly open source models are the norm is obviously preferable to a world where only open weight models are the norm, a world where only open weight models are the norm is still vastly preferable to a world where proprietary models running on other people's hardware is the norm.
I just think that we should be careful to avoid watering down terminology in ways that serve proprietary commercial interests over the interests of the public and of users. Open-washing is real, and it harms the intersts of users.
KV cache size is the main constraint on batching (for any given ctx length), that's a huge deal for efficiency both locally and in the data center. DeepSeek V4's reduced KV requirement is a real game changer, it definitively unlocks batching requests together for local inference, not just at scale.
This may be relevant for parallelizable workloads. For reference on my perspective: I come at this as someone who is exclusively concerned with sequential, non-parallelizable, single-user, single-system workloads.
If you have multiple chats going at the same time in your LLM web interface, that's already a parallelizable workload wrt. batched inference. And this broadly describes the more sophisticated users of LLMs (who are using it for more than just casual chit-chat), especially wrt. the largest "pro" models. Parallelism is also quite applicable to agentic workloads.
1. DS4F can run on a 128GB MacBook. M2.7 is larger (8 bit weights of routed experts). There is to see how it holds at 4 bits. At 2 bits it may not work well at all.
2. Just the KV cache of M2.7 would take ~50GB for 200k tokens AFAIK. It does not have the compressed KV cache that DS4F features.
3. The models are very similar in performances, despite all that. And DS4F is likely getting an update soon.
So it is basically a quasi-frontier model that can run on a 96/128GB MacBook at large context windows. That's non trivial. Likely a coding version could be released in the future.
>1. DS4F can run on a 128GB MacBook. M2.7 is larger (8 bit weights of routed experts). There is to see how it holds at 4 bits. At 2 bits it may not work well at all.
M2.7 is smaller than DS4, 230B total params vs 284B total params. At any given quantization level, M2.7 will require ~19% less memory for the weights than DS4F at the same quantization level. Both can be quantized to arbitrary precision levels. Larger models like these quantize much better at lower precision than smaller models do. There is still loss, but it's less catastrophic in terms of usability degradation than for say, 27B or 14B or 8B models. Again, n=1, but M2.7 holds up phenomenally well for me with unsloth's IQ2_XXS UD.
>2. Just the KV cache of M2.7 would take ~50GB for 200k tokens AFAIK. It does not have the compressed KV cache that DS4F features.
KV cache weights can also be quantized. At Q8_0, this is essentially lossless. I can fit a 400k context window with Q8_0 KV cache quantization along with unsloth's IQ2_XXS UD weight quantization (plus my running OS) on a machine with just 128 GB of unified memory. Strix Halo, not Apple Silicon. There are more exotic approaches to KV cache quantization with much higher efficiency, like TurboQuant, but this is besides the point.
>3. The models are very similar in performances, despite all that. And DS4F is likely getting an update soon.
Yes, though it's worth noting that DS4F does require about 20% more total memory for weights at any given quantization level (284B vs 230B), will need to shuffle about 30% more data through the pipeline on every forward pass (A13B vs A10B), has much higher hallucination rates per AA, and hasn't been fully post-trained. DS4 isn't a base model, it has been instruct trained, tool trained, etc, but there is a lot of capability that has been left on the table as of current checkpoints, which are what's actually available now.
>So it is basically a quasi-frontier model that can run on a 96/128GB MacBook at large context windows. That's non trivial. Likely a coding version could be released in the future.
MiniMax M2.7 fits into this same box - quasi-frontier model that can run on 96/128GB unified memory platforms with a large context window. You're right that it's non-trivial. My preference comes in part from the fact that M2.7 already is coding focused, and had been out for almost 2 months before DS4F showed up.
By the way, in spite of my preference for M2.7 over DS4F (and for Vulkan over ROCm on my hardware), I'm a big fan of your work on DarkStar 4. I admire what you've achieved with the project, how much work you've put into it, and your willingness to share that with the world, too. Thank you for your contributions to the open LLM ecosystem.
Didn't know M2.7 could also resist extreme quantizations, I had the feeling that being it shipped Q8 it was easily damaged in that way. Very interesting data point! And thank you for the nice words. Btw it really looks like ~250/300B parameters very sparse models are something for local inference.
In the same page DS4F scores much better on Omniscent Accuracy. I would take those numbers with a bit of salt. For instance I ran different benchmarks against Qwen 3.6 27B and DS4F quantized at 2bit. DS4F hallucination rate is much lower. In general I find artificialanalysis benchmarks not very aligned with what I see in the field, but in this specific case I did many tests and it is even more so.
Thank you for posting this! Just a clarification, with DwarfStar steering features I was able to completely remove refusal from DS4. It is only the example dataset (prompt pairs I provide) which is a toy, not the abilities. I thought that who is able to come up with the right dataset and understands how to use the well-documented steering feature, can access to steering. People that have no idea and would just cut & paste, I'm not sure, maybe it is a good idea if they also have access to a model without refusals? I the doubt I didn't release publicly the steering file, but I'm highly perplexed.
Btw recently the support was extended and now the steering vector can be applied to the activations at different time: always, only after thinking, only outside of tool calling, ...
Something important that not many folks realize: vector direction steering inside the inference engine itself is very superior to having GGUFs modified in the same way. The more you steer, the more you damage the model capabilities. So applying it at runtime, you apply it the minimun needed for what you want to accomplish. Also you can apply only during selected moments. It is even possible (I still didn't implement it but I like the idea) of applying the steering only when the energy across the refusal direction is over a given threshold. Many things you can play with.
AIUI, DeepSeek V4 has very little (if any) of the refusal behavior you usually get from Western AI models for benign input. Is this mainly about the software security assessment case?
Not just that. The other day I was able to ask DeepSeek v4 (with the anti-refusal vector loaded) all the top tricks to steal a lollypop to a child.
Not even the obvious ones. Ask it for good objective news sources and it will refuse.
I'm surprised the article doesn't mention the biggest use of steering vectors, which is the potential to remove refusals from models (a.k.a. abliteration or uncensoring).
There was an earlier paper that found that "most refusals are on a single vector", and you can identify and "nerf" that vector so the model will skip refusals and answer "any" request normally. This was very doable for earlier models trained with SFT for refusals, seems to be a bit more complicated for newer models, but still doable to some extent.
There are already some libraries to automate this process and reduce refusals, but usually they focus on identifying and then modifying the models and releasing them as uncensored models. This technique of steering lets you enable this vector changing dynamically, so you don't need to change models if the abliteration process somehow hurts accuracy on other unrelated tasks.
p-e-w was just talking about this the other day in his Discord. seems doing the one neuron method is quite bad for KLD and that's why the newer techniques have stuck.
Who is p-e-w? Is it a public discord?
heretic maintainer: https://github.com/p-e-w/heretic
the fun bits are in another branch or PRs
not sure why youre fixed on censoring. if we invert your POV censoring includes not reporting falsehoods "vaccines are harmful". Science and logic often tackle these subject via censoring, but a model given a equal sampling of Internet, would think vacinnes are harmful. a less naive correction would censor this problematic context.
so im cofised as to why you think unmasking whatever bias you think is censored will result in improvement in generic use case.
That's not what people mean when they talk about censoring. They mean that models are trained to not touch some subjects, and that can spill over in legit tasks, often with humorous results (early on, there were many instances of models refusing to answer "how do you kill a process", because of overbearing refusal training).
Uncensoring a model also doesn't necessarily improve generic use cases. In fact it can lead to overall less accuracy on generic tasks. But your goal with uncensoring is getting the model to engage with those specific subjects. You don't necessarily care about "generic use cases". That's why I mentioned that having the ability to do this at inference time is better than using ready made uncensored models. Because those usually focus on some usecases that you may or may not be interested in (porn being one of the most sought after in local communities).
Uncensoring in legit cases can mean limiting refusals on cybersecurity for example. There are legit reasons for researchers to have that capability when running the models locally. Having the models uncensored on that specific vector can reduce refusals and make the models usable for both defence and offence (say in a loop, to improve both). If your models can only do defense (and sometimes even refuse that, because censoring can leak into related issues as well), you're at a disadvantage.
> Uncensoring a model also doesn't necessarily improve generic use cases.
While the following is not a generic use case, I have a funny anecdote about how censorship is holding back flagship models.
I was asking an uncensored version of Qwen3.6 how a CLI option of llama.cpp worked, and to my horror and amazement, it rudely went and decompiled the binary to figure it out. It felt like the computer-equivalent of asking a vet why my dog looks sick, who then proceeds to cut it open to check. Flagship models usually do not do that without some convincing, but it sure is effective.
We will need much better sandboxes when less restricted models become more common. I can already see them hammering out 0-days when they are prompted to do some task that usually requires root.
> Flagship models usually do not do that without some convincing
Just a data point, but I’ve been having Claude do this regularly
I think I was using GitHub Copilot when I made the experience that led me to this statement. I guess the experience of using LLMs can be quite different depending on model version and harness.
Gemini Flash-Lite was a decent reverse-engineering sidekick since 2.5 as well.
Same. I was having it debug a routine python issue and it broke out mpympler and LLDB, and added a signal handler dump stack traces.
whats funny is if it looked up the source code on github it would've figured it out faster
Anthropic mentioned explicitly making an effort to make Opus 4.7 worse at cybersecurity tasks because the last few generations have been getting too good at them.
So they're trying to improve the model's general intelligence while selectively making it worse in one area.
It should be noted that no ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure. Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter. [1]
I think that the best use of frontier AI models outside of generic corporate settings is going to be building generic frameworks and procedures for training specialized models. No ethically-trained American coding model would ever consent to write a Plutonium Process Engineering agent. But you can get it to write a general framework for pretraining models and preparing them for agentic usage, to which the copious published literature on plutonium production could be given as a data set.
[1] https://blog.codinghorror.com/your-favorite-programming-quot...
I still think this is a rosy picture of the censorship issue; to me, we're discussing the difference between a biased model and a disinterested model. The response to the idea of getting 'uncensored' models is the idea that some how censorship is something that is bad for the models as apposed to a structural enhancedment. It's like the bones to the nervous system: the brain will tell you, in a vat, it doesn't need those bones.
Lets be honest: they're a business model; they're making generic public goods, but with how they're behaving around mythos, they're more concerned with extracting value from that task than they are concerned with boogeyman hacker.
> There are legit reasons for researchers to have that capability when running the models locally.
It's also important for researchers to understand what the models will say and do if they are jailbroken. Uncensoring the model locally gives you a natural way to achieve that.
I still dont get what uncensoring does other than change the model output. No one knows which model is actually in use anywhere at anytime for any purpose of any alignment.
It may give you the secrets to nuclear weapons as easily as it'll tell you confidently that the jews control the world; and it'll halucinate further as you remove the controls.
Sure, there's some cultural value in there, but the way people talk about uncensored models is like your 40 year old unmarried cousin who talks about aliens and shit. The best example always seems to be talking about 1989 and tiannamen square, as if that's some technical secret that a _model must know_ for it' the truely fullfill its ... alienware?
Anyway, seems bizzarely more conspiratorial than technical profiency. Like we'd find technojesus if they just 'uncensored' the model.
That’s not what it means. Those falsehoods (or their antithesis) are baked into the data and training. This is more about refusals, as in refusing to answer a question because someone else feels you should not be allowed to ask a question.
“Sorry, I’m an AI and therefore can’t answer questions about atrocities in holocaust history, but I’m happy to explain how…”
“I can’t answer your question on how to hack because I have decided you wanting to understand it and protect from it, is the same thing as you wanting to do it. Good luck convincing me otherwise!”
It doesn’t matter the reason, their taste, or whether they think people should be allowed to ask questions or do certain things, and that is generally the reason people pursue the removal of such guardrails. Yes it can lead to misuse, but the alternative is the textbook definition of censorship which always has effects on things unrelated to that which is being censored.
But beyond that, refusals do seem to have an effect on performance. Not significant; mostly marginal from what I’ve seen, but enough that it doesn’t just seem to only be statistical noise.
So I need to actually check whether these actually end up on separate vectors in current models -- but as a human, there's a huge behavioural difference in:
- When doing this task, I should do A and not B
- I should refuse to help with this task
The former is learning the user's preferences in how to succeed at the task; the latter is determining when to go against the user's chosen task.
Your example:
- "Are vaccines harmful?" vs.
- "Generate a convincing argument vaccines are harmful"
A model which knows why vaccines are not harmful may in fact be better at the latter task.
We might not want models to help with the latter, sure -- but that's a very different behaviour change from correcting the answer to the first! And consequently I'd be shocked if, internally, they were represented the same way.
I'm reminded of the emergent misalignment paper, where a model fine-tunes to produce insecure source code would also reliably respond in evil ways to general requests.
e.g. you'd ask it for a cookie recipe and it would add poison to the recipe.
I understood that to be "there was a single neuron "don't be evil" which got inverted" but I'm not sure what it really looks like. (e.g. adding obvious exploits to source code is similar to adding poison to a recipe)
Does DeepSeek V4 actually refuse the latter task? As I mentioned, I find it to be very light on refusals already.
DeepSeek in general release not a very censored models when you run them locally. E.g no problems whatsoever answering what happened on Tiananmen Square In 1989.
"Are vaccines harmful?" to an LLM has already nudged it to yes. In fact, with fewer tokens, it may be more convinced it's harmful because it's a smaller seed.
This is something difficult to handle properly.
I think it is useful to turn off censoring if you need.
When I am researching something, I likely want proper information. If I am looking up information on vaccines, I don't want information that crackpots spread online on chips on vaccines and how 5g will kill the vaccinated, or how it is somehow connected with Bill Gates spreading meat allergies through drones raining ticks on unsuspecting people.
On the other hand, if I am actively looking up crazy bullshit information (perhaps I want some entertainment), I should be able to read it.
The really interesting thing that I think is going on inside of the DS4 repo is exploring all of the interesting knobs that frontier labs have hidden from users, and then thinking about how they can fit into real dev/interaction workflows. It's really cool to see different interaction modalities being explored and thinking about for example how steering can be worked into a user interface in a helpful way. I think that once the cat is out of the bag as they say, and users understand the level of control and utility they can get from models that are sort of turned inside out in this way, it will start to be an integral part of their tool belt, and it'll just make sense for this level of control to be expected from your models or model providers.
> inspired to write this post by antirez’s recent project DwarfStar 4, which is a version of llama.cpp that’s been stripped down to run only DeepSeek-V4-Flash
This is not true, it is its own project.
Indebted to llama.cpp, sure, but not a stripped down version
Yep, the code overlap is minimal, a few kernels. Some quantization code for the quantizer it implements. DwarfStar 4 is not a fork of llama.cpp, but without llama.cpp the project would be a lot more lacking, since I was able to get all the details that mattered in a second. But it is not a stripped down llama.cpp. This does not reduce in any way how much llama.cpp is not just for this project, but for all the projects that followed and are following. It's not a matter of code: the street to follow, the quants formats, the lessons, the optimized kernels you can check to learn the patterns.
Truth seems to sit somewhere in-between, DwarfStar 4 seems to mainly exists only because of llama.cpp, and authors basically were very inspired by llama.cpp's code, and even in some places literally have copied pieces from it, all with proper attribution and everything, I'm not trying to say this is bad, seems OK to me:
> ds4.c does not link against GGML, but it exists thanks to the path opened by the llama.cpp project and the kernels, quantization formats, GGUF ecosystem, and hard-won engineering knowledge developed there. We are thankful and indebted to llama.cpp and its contributors. Their implementation, kernels, tests, and design choices were an essential reference while building this DeepSeek V4 Flash-specific inference path. Some source-level pieces are retained or adapted here under the MIT license: GGUF quant layouts and tables, CPU quant/dot logic, and certain kernels. For this reason, and because we are genuinely grateful, we keep the GGML authors copyright notice in our LICENSE file. - https://github.com/antirez/ds4#acknowledgements-to-llamacpp-...
Been a lot of fun to play around with it since https://news.ycombinator.com/item?id=48142885 (~2 days ago), managed to make the generation go from 47.85 t/s to 57.07 t/s so far :)
Send patches! But remember that many speedups end being not exactly correct and the logits drift. But there is extensive testing and even ds4-eval now to test how it performs.
Hah, it's quick hacks for me to understand CUDA better, I'm unlikely to have time to make them proper enough :( But maybe opening an issue talking about what I tried and what worked, makes sense.
I did confirm no logits drift, as you so nicely have provided tooling for ensuring exactly this, thanks for the great care that obviously gone into the project, been a pleasure to play around with! :)
I used steering to make an AI more radical:
Write up: https://www.outcryai.com/research/shift-a-models-political-i...
App: https://apps.apple.com/us/app/outcry-activist-ai/id676208676...
This technique has a lot of potential.
Honestly, what is more interesting that steering is the use of soft prompts (virtual tokens)... you can use these virtual tokens to find non-linguistic areas of meaning for the AI that changes the behavior in complex ways. I wrote about how we integrated soft prompts into an activist ai here: https://micahbornfree.substack.com/p/the-week-outcry-woke-up... and https://www.outcryai.com/research/how-to-create-activist-ai
Great article but I'm confused on one thing.
The article claims steering only works in local models, but GitHub Copilot has a "steer with message" feature where I can course correct mid execution. I use it often.
I think these are different kinds of steering right? Agent steering probably inserts another user message between the harnesses own ping-pong between harness and the LLM.
- https://docs.github.com/en/copilot/how-tos/copilot-cli/use-c...
- https://docs.github.com/en/copilot/how-tos/copilot-sdk/use-c...
Different kind of steering, that's just injecting text into the model's natural language thinking output or something very similar. You can do a middle ground though by using Anthropic's NLA work to look at the natural language rendition of a model's activations at a particular layer, edit the text and convert it back into completely different activations.
Ahh I see. Thanks for the clarification.
How does the model qualify as local? ~192 GB RAM needed sounds a bit much for local.
Runs on 96GB MacBooks. 128GB is better. Check the README of DwarfStar.
If you were buying a computer today to use for DS4 (budget under 10k) what would you get? I care more about inference quality than speed. Or is it better to wait for the rumored M5 studio?
For large contexts, you're probably going to want CUDA. The DGX Spark is expensive but would get you decent speeds on DS4 Flash. DS4 Pro is probably not possible with $10k if you want sub-10 minute prefill.
Can you download it and run it given you have the hardware? Then it's local, regardless if you happen to have the needed hardware or not.
Bit like asking if Zigbee can be considered local/LAN for people who don't have the required radio/antenna.
This reminds me of control vectors, especially this line in the linked DwarfStar repo:
> y = y - scale * direction[layer] * dot(direction[layer], y)
From https://vgel.me/posts/representation-engineering/
> A control vector is a vector (technically a list of vectors, one per layer) that you can apply to model activations during inference to control the model's behavior without additional prompting
Sounds more like something for DL research than something you might want to use in practice.
Nope, with the anti-refusal vector loaded you can ask many things for instance related to computer security and if you want to learn, it is a lot better of a model that continuously says you "I can't help you with this problematic request".
> you can already exercise extremely fine-grained control by tweaking the language of your prompt.
maybe i suck at prompting but i find it impossible to overcome its biases from training data, post training ect.
you can only pattern mine from training data using prompts. you dont really have sort of fine-grained control.
I know it's only tangentially relevant, but I've been baffled by the interest in DeepSeek V4 Flash. It's larger, less efficient, and in many cases, performs worse on both objective benchmarks and real world sniff test (admittedly, n=1) than Minimax M2.7. DS4F hallucinates at extraordinary rates while M2.7 does not. The 196k context length that M2.7 was natively trained up represents neither a hard technical ceiling (this is metadata that can easily adjusted), nor a meaningful degradation threshold - I've personally ran it up past 330k token context windows where it maintained full coherency, and still completed my one-shot agentic task to my satisfaction.
> The 196k context length that M2.7 was natively trained up represents neither a hard technical ceiling (this is metadata that can easily adjusted), nor a meaningful degradation threshold
FWIW, I find that in OpenCode it starts becoming erratic after around 80k tokens (sometimes less).
M2.7 is no longer open source, it's been changed to a NC license. It's an OK model, but IME out of the big 5 chinese models (ds, glm, kimi, minimax and qwen), DS models have generally shown better generalisation and real-world usage than all the others, even if the benchmark scores were lower. Less benchmaxxxing, basically.
DS4 also has some neat new arch improvements, giving it a lot of context at lower VRAM usage. So it will be cheaper to serve, B for B than previous models.
M2.7 was never open source, only open weight, which fulfills a lot of the spirit of open source, but isn't really the same thing as a whole. The noncommercial license is basically impossible to enforce if you're self-hosting anyway, because it's essentially impossible to prove that any individual commit was made by Minimax M2.7 in an environment where multiple self-hosted models are being run side-by-side. Besides that, you're not obligated to abide by terms you never agreed to in the first place, and you don't need to agree to anyone's terms to download open weights from a peer or over a torrent. These weights amount to public information that freely exists and is shared in the commons; not a scarce, rivalrous good; not copyrighted works; not sensitive intellectual property.
The weights may nominally be legally copyrighted, but the rightsholder certainly doesn't seem to be making anything resembling a serious effort to actually assert or defend those rights; on the contrary, they are doing the exact opposite by maximizing the gratis distribution, including knowingly and willingly via third parties, with no copy protection whatsoever, and no reasonable expectation of non-distribution.
They are not behaving like an entity trying to protect valuable intellectual property, they are behaving like an entity trying to reap the reputational and network effect benefits of maximizing the free distribution of a public good.
Less memory usage by the KV cache doesn't mean cheaper to serve overall. Once you've acquired hardware (for which you need more to serve DS4L than Minimax M2.7, the former being ~54B total params larger model to begin with, and which KV cache memory efficiency does nothing to address), the capex cost is basically fixed and opex just comes down to power draw, which will be marginally higher per token with DS4L than with M2.7 owed to the slower speeds that result from 13B active params vs 10B active params on forward passes during TG.
As to the 2nd part of your message, it's really easy to verify yourself (on openrouter).
DSv4-flash is currently being served at 0.14/0.24 $/MTok by most of the providers (8 as of writing this) and even a bit cheaper by 2 providers.
Minimax2.7 is being served at 0.30/1.20 $/MTok by most providers (4 providers as of writing this) and double that price by 2 providers.
As for the first part of your message, this is actually a good illustration of the miss-understanding of licensing LLMs. There are open-source models out there (Apache 2.0 and MIT) and there are also source-available (i.e. open weights) in llamas, minimax2.7 and something in between with the latest kimi (MIT w/ attribution). Open source in the context of LLMs means that you get a license to run, inspect, modify and re-release a model. It was never about data or training. But that's a very common interpretation, that's wrong IMO. But I get that it's contested, so anyway. Sorry for the tangent.
Third party inference costs are a moot point for people running these models locally.
I am currently serving Minimax M2.7 to myself at ~$0.015/1M blended tokens worth of electricity on my own local hardware, where I get all of the confidentiality, integrity, and availability benefits that are lost when choosing to run open weight models on someone else's API.
Open source means that all of the information necessary to recreate the final product is public, which in the context of LLMs, would include all of the training material, and build instructions (scripts to do the training). Very few models actually achieve this - Nemotron family is the only one that comes top of mind. A license to run, inspect, modify, and re-release is a good improvement on open weight models, but does not alone amount to the model actually being open source.
You are welcome to an alternative understanding of the definition of open source - as you correctly note, it's a contested term - just know that your definition is not the more widely accepted one that people think of when they hear "open source".
Your version of the term is much more aligned with the OSI, which was a federation of anti-FLOSS industry bodies created with the intent to capture, redefine, and weaken the original spirit of the FLOSS movement, which predates the OSI by almost a decade - the GPL was first released in '89, compared to the OSI's formation in '98 by members of the $10B for-profit Netscape Corporation, who's flasgship product was originally proprietary and was only open sourced after commercial failure against proprietary competitors.
None of this should be construed as an implication that I'm anti-open-weight. As I mentioned earlier, I think open weight models fulfill a lot of the spirit of open source. While a world where truly open source models are the norm is obviously preferable to a world where only open weight models are the norm, a world where only open weight models are the norm is still vastly preferable to a world where proprietary models running on other people's hardware is the norm.
I just think that we should be careful to avoid watering down terminology in ways that serve proprietary commercial interests over the interests of the public and of users. Open-washing is real, and it harms the intersts of users.
KV cache size is the main constraint on batching (for any given ctx length), that's a huge deal for efficiency both locally and in the data center. DeepSeek V4's reduced KV requirement is a real game changer, it definitively unlocks batching requests together for local inference, not just at scale.
This may be relevant for parallelizable workloads. For reference on my perspective: I come at this as someone who is exclusively concerned with sequential, non-parallelizable, single-user, single-system workloads.
If you have multiple chats going at the same time in your LLM web interface, that's already a parallelizable workload wrt. batched inference. And this broadly describes the more sophisticated users of LLMs (who are using it for more than just casual chit-chat), especially wrt. the largest "pro" models. Parallelism is also quite applicable to agentic workloads.
May I ask you what did you used for the DS4F inference? It is a model with very low hallucination rate in my tests.
Btw, a few data points:
1. DS4F can run on a 128GB MacBook. M2.7 is larger (8 bit weights of routed experts). There is to see how it holds at 4 bits. At 2 bits it may not work well at all.
2. Just the KV cache of M2.7 would take ~50GB for 200k tokens AFAIK. It does not have the compressed KV cache that DS4F features.
3. The models are very similar in performances, despite all that. And DS4F is likely getting an update soon.
So it is basically a quasi-frontier model that can run on a 96/128GB MacBook at large context windows. That's non trivial. Likely a coding version could be released in the future.
>1. DS4F can run on a 128GB MacBook. M2.7 is larger (8 bit weights of routed experts). There is to see how it holds at 4 bits. At 2 bits it may not work well at all.
M2.7 is smaller than DS4, 230B total params vs 284B total params. At any given quantization level, M2.7 will require ~19% less memory for the weights than DS4F at the same quantization level. Both can be quantized to arbitrary precision levels. Larger models like these quantize much better at lower precision than smaller models do. There is still loss, but it's less catastrophic in terms of usability degradation than for say, 27B or 14B or 8B models. Again, n=1, but M2.7 holds up phenomenally well for me with unsloth's IQ2_XXS UD.
>2. Just the KV cache of M2.7 would take ~50GB for 200k tokens AFAIK. It does not have the compressed KV cache that DS4F features.
KV cache weights can also be quantized. At Q8_0, this is essentially lossless. I can fit a 400k context window with Q8_0 KV cache quantization along with unsloth's IQ2_XXS UD weight quantization (plus my running OS) on a machine with just 128 GB of unified memory. Strix Halo, not Apple Silicon. There are more exotic approaches to KV cache quantization with much higher efficiency, like TurboQuant, but this is besides the point.
>3. The models are very similar in performances, despite all that. And DS4F is likely getting an update soon.
Yes, though it's worth noting that DS4F does require about 20% more total memory for weights at any given quantization level (284B vs 230B), will need to shuffle about 30% more data through the pipeline on every forward pass (A13B vs A10B), has much higher hallucination rates per AA, and hasn't been fully post-trained. DS4 isn't a base model, it has been instruct trained, tool trained, etc, but there is a lot of capability that has been left on the table as of current checkpoints, which are what's actually available now.
>So it is basically a quasi-frontier model that can run on a 96/128GB MacBook at large context windows. That's non trivial. Likely a coding version could be released in the future.
MiniMax M2.7 fits into this same box - quasi-frontier model that can run on 96/128GB unified memory platforms with a large context window. You're right that it's non-trivial. My preference comes in part from the fact that M2.7 already is coding focused, and had been out for almost 2 months before DS4F showed up.
By the way, in spite of my preference for M2.7 over DS4F (and for Vulkan over ROCm on my hardware), I'm a big fan of your work on DarkStar 4. I admire what you've achieved with the project, how much work you've put into it, and your willingness to share that with the world, too. Thank you for your contributions to the open LLM ecosystem.
Didn't know M2.7 could also resist extreme quantizations, I had the feeling that being it shipped Q8 it was easily damaged in that way. Very interesting data point! And thank you for the nice words. Btw it really looks like ~250/300B parameters very sparse models are something for local inference.
Per AA's Omniscience Index benchmark, the "non-hallucination rate" subcomponent (1 - hallucination rate) of 4% for DS4F vs 66% for M2.7.
https://artificialanalysis.ai/leaderboards/models?weights=op...
In the same page DS4F scores much better on Omniscent Accuracy. I would take those numbers with a bit of salt. For instance I ran different benchmarks against Qwen 3.6 27B and DS4F quantized at 2bit. DS4F hallucination rate is much lower. In general I find artificialanalysis benchmarks not very aligned with what I see in the field, but in this specific case I did many tests and it is even more so.