Mianbi Intelligence and Tsinghua University Open-Source Edge-Side Large Model BitCPM-CANN

Who is the biggest winner?

Recently, Mianbi Intelligence and OpenBMB launched an edge-side open-source week. Today, as the first day of the open-source week, they released a great product: BitCPM-CANN. The model weights require only about 200 MB of memory, enough to run on a smartwatch.

Simultaneously, this is the first [ternary large model] trained on a [domestic computing platform] to be open-sourced. For this model, from quantization operators to training algorithms, everything runs natively on Huawei Ascend.

So-called ternary large models retain only three states for parameter weights: -1, 0, 1, occupying 1.58 bits... Hey... for those who haven't caught on, think about high school math, what is log2(3)? 1.58.

Compared to storing weights using FP16 conventionally, this reduces size by 90% (as for why it's calculated this way, I'll explain in detail later, convenient for non-CS friends to use as after-dinner talk).

Additionally, Mianbi's model released four sizes from 0.5B to 8B at once, all open-source.

Using only 10% of the size, it retains 95% of the capability.

Compressing the size to 10%.

Here I need to explain [model parameters] first. The world in the eyes of a large model is composed of a bunch of numbers, these numbers are the large model's parameters. When we say DeepSeek R1 is 671B, it refers to its parameter size being 671 B.

Parameters need to be stored, commonly BF16 (Brain Float16), each parameter stored as a decimal via floating point, for example 1145141919810 stored as BF16 is: 0101001110000101, meaning: 1.0000101₂ × 2⁴⁰ in binary.

If using ternary? 1145141919810 -> +1, that's so violent. It cuts all parameters into three options via algorithm: positive is +1 indicating activation; negative is -1 indicating inhibition; middle is 0 indicating disconnection.

When I wrote about MiniCPM-V 4.6 before, I made a quantization precision comparison table, ranking from FP16 all the way to 1-bit BitNet.

Inserting a piece of information here, when BitNet first came out it only used +1/-1 two states to store, so it occupied 1 bit width; later BitNet 1.58 was released, also using ternary +1/-1/0.

Intuitively, if data is quantized to this extent, the large model might become stupid; but after specialized training, this model is nearly identical to the original.

In the model card information there is such a chart, I pulled it out, roughly ran 11 benchmarks, comparing its scores one-to-one with their own full-precision MiniCPM4, and found that for the 1B to 8B three tiers, capability retention rates are all above 95%. The 3B tier has the highest retention rate, 97.2%. Math and code tasks, which are most prone to crashing due to precision loss, also stabilized near the full-precision upper limit.

Then the 0.5B tier retention rate is lower, at 90.1%, this also explains that smaller models are more sensitive to quantization noise, but the causes for this have been found and can be further optimized, the Mianbi team will continue to work on data ratio and curriculum learning.

First time for domestic computing.

Ternary is not a new species in the large model academic circle, as mentioned in the notes above, Microsoft Research released BitNet b1.58 two years ago, verifying that ternary weights are trainable.

So, what difficulties did Mianbi's work overcome this time? Roughly the following points:

- Decoupling from CUDA ecosystem: Previously public ternary work was all based on NVIDIA multi-computing cards, heavily relying on the CUDA ecosystem. Doing it on Ascend requires rewriting underlying quantization operators, and training flow, communication interfaces etc. all need to be re-adapted.

- Scale pushed to 8B: The largest model publicized by BitNet was 3.9B, Mianbi's work this time pulled the entire size to 8B, so training stability, VRAM scheduling and multi-card communication etc. naturally increase in difficulty.

- Solidified into infrastructure: This is a commendable point, Mianbi embedded a pluggable quantization parallel linear layer in Megatron-LM and Huawei MindSpeed, quantization training and full-precision training share the same checkpoint and communication framework, allowing 32K long sequence training to run on this as well.

By now, this set of tools can be considered best practice for low-bit training on Ascend, later if anyone tries to do similar things, there is ready-made homework to copy.

Speaking broadly, this is a breakthrough for the domestic underlying computing ecosystem on the frontier route of 'extremely low-bit large models', proving that without the CUDA system, using pure-blooded Ascend, models can still be trained.

Mianbi's old business.

For making quantized large models, there are two paths in the industry.

The first, also the most common, is called Post-Training Quantization (PTQ). First train the model with full precision, then use algorithms to compress weights into low bits. Many models release multiple quantized versions simultaneously upon release, this is how it comes about. Saves trouble, low cost, but when compressed to extremely low bits, model capability drops cliff-like.

The other is Quantization-Aware Training (QAT). From the first day of training, put the model into the ternary restriction to train. For example, Mianbi's new model, throughout the learning process, must learn to maintain performance under only -1, 0, 1 conditions.

In this direction, Mianbi has actually been working for a long time, Mianbi has always used 'high efficiency' and 'edge deployment' as technical brand labels. Previously they used QAT methodology to find stable learning rate intervals, data ratios and distillation strategies. When the 0.5B and 1B models of the BitCPM4 series were released in 2025, they ranked first among public 1.58-bit models of the same size.

This time using Huawei's equipment to do low-bit training, these were all just used. For example, QAT is extremely sensitive to learning rate, within reasonable intervals it can be aggressive, but crossing the boundary leads to failure, previous alchemy techniques can be transferred to Ascend, saving a lot of trial-and-error time.

When the model training is finished, the inference benefits are huge, roughly allowing the model to run on 1/6 of the original VRAM. Now high-speed memory prices have risen unrecognizably, this thing becomes increasingly cost-effective.

Summary.

The 0.5B, 1B, 3B, 8B four tier models Mianbi released this time are nearly identical to full-precision MiniCPM4, all open-source.

Returning to the initial content, if you directly ask Claude: How much memory does this model need? It will give you a theoretical value: 0.5B multiplied by 1.58-bit, pure model weights only need 99MB.

But in actual engineering it will be slightly more, Embedding (word embedding layer) and LM Head (language model head) need to retain BF16 full precision to work well, so the real size needs about 200MB memory... meaning this 0.5B model, even a 'Little Genius' can truly run it.

Open-source address:

https://huggingface.co/collections/openbmb/bitcpm-cann

https://modelscope.cn/collections/OpenBMB/BitCPM-CANN.

This page provides an editorial summary based on publicly available information. It is not a republished article. Use the source link below for the original report.