#vllm

2 articles

On-Prem AI 30 August 2026 12 min read

The weights you run were probably not made by the lab that trained them

Nvidia is reported to be buying Hugging Face. A good week to learn what is really on the shelf: master weights, vendor builds, and the community quants most of us run.

#hugging-face
#quantization
#nvfp4
#fp8
#bf16
#glm
#vllm
#open-weights
#model-archive
#supply-chain

On-Prem AI 23 August 2026 17 min read

GPU passthrough cost me nothing — until the GPUs had to talk to each other

A year after virtualizing my edge AI supercomputer, I finally measured what the hypervisor costs: single-GPU work is free — the VM even won — and four-way tensor-parallel serving pays a third.

#vfio
#gpu-passthrough
#proxmox
#vllm
#nccl
#tensor-parallel
#sdxl
#rtx-3090
#benchmarks
#virtualization