The weights you run were probably not made by the lab that trained them

Everything you see on this blog is cooked in-house. The image above was generated on one of my own RTX 3090s, offline, in about two minutes — no stock photography, no cloud image API. Full recipe below.

Feature image made in my lab — image model, prompt and settings

Image model

Qwen-Image 2512 — fp8 (e4m3fn) variant. The bf16 pair does not fit a 24 GB card; fp8 weights are stored compressed and cast at compute time

Text encoder

Qwen2.5-VL 7B — a vision-language model doing the prompt understanding, which is most of why the composition follows a long prompt

Generated on

1× NVIDIA RTX 3090 (24 GB) in my own lab, through a self-hosted ComfyUI — on my hardware, my electricity, nothing leaving the building

Settings

1664×928 · 20 steps · cfg 2.5 · euler / simple · shift 3.1 · seed 530 · ~125 s

Prompt

a long row of identical unlabelled storage drives standing upright on a metal shelf in a dark archive room, one drive pulled halfway out and glowing faintly from within, moody editorial hardware photography, cold blue and cyan light with a single warm amber accent, shallow depth of field, 50mm, dark background, fine detail, cinematic, high dynamic range, no text

Negative prompt

text, words, letters, watermark, logo, signature, caption, ui, people, faces, hands, cartoon, illustration, cgi render, oversaturated, blurry, low quality, jpeg artifacts

Licence

Apache-2.0 — both the image model and the text encoder. Commercial use permitted with no use-based restrictions, which is exactly why this stack and not a prettier one

More AI-generated images? My reference project OpenChef does this at scale — every recipe photo carries every version of itself, with the full prompt, model, encoder, seed and licence.

---

On 27 August, The Information reported that Nvidia had agreed to buy Hugging Face for around $12.9 billion. CNBC, TechCrunch and Forbes matched it within hours. Business Insider, which broke the story that Hugging Face was fielding takeover interest in the first place, reported that no agreement had been signed and that the talks could still fall apart. Both companies declined to comment.

So: not a done deal. Treat every number in that paragraph as reported, not confirmed.

But it was enough to make me spend a morning doing something I had been putting off — actually inventorying which model weights I depend on, where they physically live, and who made the specific files I load. The answer surprised me in one place, and it is the reason for this post.

Most of the model files I run were not produced by the lab that trained the model. They were produced by strangers, for free, within hours of a release, and I had been treating them as if they came from the vendor. If the largest model host in the world is about to change owners, that is a good moment to know what is actually on the shelf.

The shelf has four layers, not one

People say “I downloaded it from Hugging Face” as though that is a provenance statement. It isn’t. Hugging Face is a host, and what sits on it comes from four quite different kinds of publisher.

1. The lab’s master weights. The real thing, at full precision, from the org that trained it. Large and mostly impractical to serve directly.

2. The lab’s deployment build. The same model, quantized by the lab itself for actual serving. Increasingly this — not the master — is the primary release.

3. Hardware-vendor quants. Nvidia and Intel publish builds tuned to their own silicon. Usually well-tested, because the vendor has an interest in their chips looking good.

4. Community quants. Individuals and small orgs who re-quantize a model within hours of release, long before the lab or the hardware vendors get to it. This is the layer almost nobody audits, and — if you run anything at 4-bit on consumer or workstation hardware — it is probably the layer you are actually running.

That fourth layer is genuinely valuable. It is also a supply chain, and I had not been treating it like one.

Reading a model name

Before the formats make sense, the naming has to. A repository id is owner/Name, and the name is a stack of separate conventions. Take the longest variant of the model I mirrored this week:

zai-org / GLM - 5.3 - Flash - BF16
   │       │     │      │      └── numeric format of the weights
   │       │     │      └───────── size tier chosen by the vendor
   │       │     └──────────────── version
   │       └────────────────────── the vendor's family nickname
   └────────────────────────────── who published THIS repository

• zai-org — the Hugging Face owner. Who published this repo, which is not necessarily who trained the model. This is the field people skip, and it is the one that matters most for provenance.
• GLM — “General Language Model”, the family name from Z AI (formerly Zhipu AI). Vendor branding, nothing more.
• 5.3 — the version. Higher is newer. 5.2 to 5.3 is a generation, not a patch.
• Flash — the size tier. Vendor-chosen, no standard meaning. Here it is a ~321B model against the flagship’s ~753B. Elsewhere you will see Air, Lite, Mini, Nano, Turbo for smaller and Max, Pro, Ultra for larger.
• BF16 — the numeric format. The single biggest lever on whether a model fits your hardware.

Two more conventions worth knowing. Instruct or Chat means tuned to follow instructions rather than just continue text; Base means untuned. And a name like Qwen3-Coder-480B-A35B carries two numbers: 480B total parameters, A35B meaning 35B active per token. Mixture-of-Experts models store every expert but route each token through a fraction of them. You pay memory for the total and compute for the active part. That asymmetry is the only reason a 753B model is discussable on desk-sized hardware at all.

The formats, and the arithmetic that decides everything

The format is how many bits each weight is stored in. The arithmetic is unglamorous and completely decisive:

bytes ≈ parameters × bits ÷ 8

A 753B-parameter model at 16 bits is about 1.5 TB. Halve the bits, halve the file. Everything else in capacity planning follows from that one line.

Format	Bits	What it is
FP32	32	Full precision. Training only — nobody ships it
BF16	16	“Brain float 16”. The master weights everything else is derived from
FP8	8	8-bit float. Near-lossless in practice, and native on Hopper and Blackwell silicon
NVFP4	~4	Nvidia’s block-scaled 4-bit float. Runs in hardware on Blackwell
MXFP4	~4	Open Compute’s rival 4-bit float. Same idea, different scaling
AWQ / GPTQ	~4	Older 4-bit integer schemes. AWQ is “activation-aware” about which weights to protect
int4 / AutoRound	~4	4-bit integer; Intel’s AutoRound is a strong current recipe
GGUF Q4_K_M etc.	2–8	llama.cpp’s container format. The Q number is the bit depth

Real numbers from one model family, all measured by summing the actual file trees rather than trusting the model card:

Build	Format	Size
GLM-5.3	FP8	755.7 GB
GLM-5.3-BF16	BF16	1506.7 GB
GLM-5.3-Flash	FP8	328.4 GB
GLM-5.3-Flash-BF16	BF16	642.7 GB
GLM-5.3-Flash-NVFP4 (community)	NVFP4	194.7 GB

Same Flash model, 643 GB or 195 GB depending purely on who packaged it and how. That ratio is the difference between “needs a rack” and “runs on the boxes under my desk.”

The thing that actually surprised me

I had assumed, without ever checking, that the -BF16 repository is the original and the quantized one is the derivative. For this model it is the other way round.

Z AI trained and shipped GLM-5.3 in FP8. That is the primary repository, and it has roughly eighteen times the downloads of the BF16 one. GLM-5.3-BF16 is that FP8 expanded back out to 16 bits — twice the bytes, carrying no additional information.

This is not universal. Plenty of labs do train in BF16 and publish FP8 as the derivative. The lesson is not “BF16 is fake”, it is: check which repository the vendor calls primary, because you cannot tell from the name.

I still mirrored the BF16. Not because it holds more information — it doesn’t — but because it is the conventional master format, and it is the cleanest base to re-quantize from when a new format appears. Which brings me to the rule that governs my whole archive:

Quantization is one-way. BF16 → FP8 → NVFP4 works. You cannot recover BF16 from NVFP4.

If you keep only the 4-bit build and a better 4-bit scheme lands next year, you are re-downloading — assuming the source is still there to re-download from. That assumption is exactly what the acquisition news put in question, and it is why I stopped treating “it’s on Hugging Face” as a backup strategy.

Judging a community quant

Here is the part I got sloppy about for a year.

When a model lands, the community quantizers move first. Unsloth — a verified org — usually has GGUF builds up the same day. Individual quantizers like cyankiwi and bartowski cover AWQ and other formats. These people are doing genuinely useful work for nothing, and without them most open-weight models would be unusable on anything but datacentre hardware.

But there is no certification, no signing, and no review. So when I needed a 4-bit build of Flash this week, there were two candidates and I had to actually choose between them. Both were NVFP4, both around the same size, neither from the vendor.

The checklist I ended up using, which I now apply every time:

1. Does the declared license match upstream? If the original is MIT and the derivative declares nothing, someone was careless with the part that has legal consequences.
2. Is the quantization config coherent? A build that says “NVFP4” should be NVFP4, not a mixed bag under a clean label.
3. Do they say what they excluded? Good 4-bit recipes hold the sensitive parts — attention projections, embeddings, the output head — at higher precision and quantize the bulk of the feed-forward layers. A build that lists its exclusions is a build someone thought about.
4. Adoption. Not proof of quality, but thousands of downloads means thousands of chances for someone to file an issue.
5. Pin the commit. Community repos get revised in place. Mine was updated the same morning I pulled it.

Scored against that, the two candidates came apart quickly. The one I took declares MIT matching upstream, is uniformly NVFP4 via Nvidia’s own quantization toolkit, lists exactly which modules it held back, and had roughly ten times the downloads. The one I passed on declares no license at all, labels itself mixed-precision rather than NVFP4, and comes from an org that at the time of writing has five models and two members.

To be fair to them: a new org with five models is how every good quantizer starts, and nothing I saw suggests the build is broken. I just could not answer question 1 about it, and I was not going to put an unlicensed derivative into a long-term archive. That is a provenance judgement, not a quality one, and it is worth being precise about the difference.

Where this bit me anyway

The honest ending: after all that careful selection, I cannot run the model.

Flash uses a new architecture, and vLLM does not know it yet. Not my build of vLLM — I checked, and then I checked the project’s main branch too. Support has not been merged upstream at all. So the variant that fits my hardware comfortably is the one I cannot serve, and the variant my serving stack does support is the 755 GB flagship, which needs roughly twice the ring I have.

That gap between “the weights exist” and “my stack can load them” is a category of problem I keep relearning. A model release is not one event. It is the weights, then the community quants within hours, then inference-engine support days or weeks later — and the marketing only ever describes the first one. If you plan around release-day headlines you will keep being surprised in week two.

The weights are on my disks regardless. That was always the point of pulling them: the archive decision and the serving decision are separate, and only one of them has a deadline set by somebody else’s corporate development team.

What I would actually do

If the acquisition closes, I do not expect Hugging Face to switch off. Nvidia has every commercial reason to keep the most-used open-model host running, and it already had a stake from an earlier funding round. The realistic risks are duller: shifting terms, gated repositories, formats favoured toward one vendor’s silicon, and quiet reorganisation of what is easy to find.

So, three things, none of them dramatic:

• Mirror what you actually depend on, in the largest format you can afford to store, because quantization only runs one way.
• Record the provenance — the exact repository, the exact commit, and who published it. “We have the model” is not an inventory entry.
• Know which layer of the shelf your files come from. If the answer is “a community quant I found on release day”, that is fine. It is only a problem if you did not know.

None of this is new advice. It is just advice I had been giving myself in the abstract for a year and only acted on when somebody put a $13 billion price tag on the thing I had been quietly depending on.

---

Sources for the acquisition reporting: The Information via CNBC, TechCrunch, Forbes. As of publication no signed agreement has been reported and both companies have declined to comment. Model sizes are my own measurements, taken by summing each repository’s file tree and cross-checking against its safetensors index.