It would be great if this could be combined with quantization-aware finetuning. In my experience, Qwen3.6-27B has much fewer repetitions at Q6 quantization level as compared to e.g. Q4, but that leaves little space for context on my 24GB RTX 3090.
this is pretty cool. i think part of the root cause is current rlhf post training design around confidence and optics rather than cooperative transparent honesty. though its kinda an expensive hypothesis to dig into as a private individual
It would be great if this could be combined with quantization-aware finetuning. In my experience, Qwen3.6-27B has much fewer repetitions at Q6 quantization level as compared to e.g. Q4, but that leaves little space for context on my 24GB RTX 3090.
this is pretty cool. i think part of the root cause is current rlhf post training design around confidence and optics rather than cooperative transparent honesty. though its kinda an expensive hypothesis to dig into as a private individual