Latent autoencoders quietly throw away colour detail — and the metric that certifies them can't see it

The finding

Every latent diffusion model — image or video — pushes pixels through an autoencoder before the generative part ever runs. That autoencoder is routinely certified as near-lossless using PSNR, SSIM and LPIPS.

Blur only an image's luminance, or only its chroma, at matched strength. Both destroy comparable amounts of real spatial information:

Destroy 80.6% of an image's chroma detail and SSIM calls it a 0.9998 reconstruction. Standard SSIM is computed on luminance, so it is structurally blind. An autoencoder selected against these metrics faces no penalty at all for discarding colour detail.

That's the mechanism. Here's the consequence.

Measured across three autoencoders

Sinusoidal gratings driven through encode/decode, averaged over 6 phases per point. Largest spatial period at which chroma is more than 50% destroyed while luminance survives intact:

Universal, not a quirk of one model. At a 16px period SD-1.5 passes luminance at 1.57 and chroma at 0.05 — a 31x disparity at identical spatial frequency. And the image autoencoders are worse than the video one, which rules out temporal compression as the cause.

Two things we nearly got wrong

Second: response above 1.0 is real, not noise. A passive optical system cannot amplify contrast. SD-1.5 reaches 1.57 on luminance and 1.73 on chroma. These decoders don't merely lose detail — they synthesise it. Latent autoencoders are editorial, not simply lossy.

Why it matters

Anything carried by small colour features stops surviving generation: wristbands, colour-coded labels and tubing, indicator LEDs, status lights, triage tags. For synthetic monitoring or medical training footage that's the difference between useful and actively misleading data — and the metrics used to sign off the generator report the reconstruction as essentially perfect while it happens.

Correction: we ran the control, and our caveat was backwards

We flagged that our chroma grating was isoluminant in RGB but not perceptually — it carried residual luminance. We said that would inflate the apparent chroma response, making our measured deficit conservative.

We ran the control. The direction is the opposite.

Properly isoluminant gratings on LTX-Video (red-green: dR=a, dG=-(0.299/0.587)a; blue-yellow: dB=b, dR=dG=-(0.114/0.886)b):

The headline is unchanged. Chroma is still more than 50% destroyed up to an 8px period while luminance passes intact — and this now holds on both isoluminant colour axes independently, which is new evidence the deficit is not specific to the red-green direction.

Scope limit closed. We said the SDXL and SD-1.5 periods were provisional until the isoluminant control had been run on them too. It has been, on both colour axes, and every published period holds:

The cross-model table now rests on gratings carrying no residual luminance, replicated on two independent chromatic axes, for all three autoencoders. Nothing in the headline moved.

Correction: the nulls were half an artefact of our own instrument

We originally reported deep nulls at exactly 4px and 8px — periods dividing the autoencoder's 8x downsampling lattice — and said they survived phase averaging. They did not, because we never averaged.

The tool now sweeps eight phases by default, in the code rather than in the prose, and reports the spread across phase next to every number — so a phase-dependent measurement announces itself instead of arriving disguised as a bandwidth limit.

Colour is the uniquely fragile attribute

The chroma deficit is easy to misread as "fine detail is lost." It is not. A probe across four attributes on the same three autoencoders — synthetic charts, exact ground truth, no generation required — shows colour is the only one with a critical size anywhere in the tested range. Below these sizes the attribute is more than 50% destroyed:

Fine luminance detail survives to the pixel level. Fine colour detail does not. A latent autoencoder will preserve the shape of a wristband, and the strokes of the text printed on it, at sizes where its colour has already been discarded.

One failure mode is worth seeing directly. We first scored text by structural correlation, which said 3px strokes survived at 0.68-0.97. But correlation cannot tell "the strokes were preserved" from "different strokes are in the same places." A vision-language read-back test — the decoded image read against the original read, so the autoencoder's own contribution is isolated — says otherwise:

At 6px, SD-1.5 did not blur the text. It replaced "PT-4417 O2 94%" with Chinese characters — 何一時何處何故. Confident, fluent, entirely fabricated. That is what "editorial, not lossy" looks like caught in the act. Our earlier claim that text survives to 3px was measured with the wrong instrument and is withdrawn: text survival is model-dependent, and at least one autoencoder hallucinates replacement glyphs rather than degrading gracefully. The chroma result is untouched — it was never measured by correlation.

What this does not support

Not detecting AI-generated video from its chroma signature. We tried, twice, and it failed twice. The gratings that expose these nulls put all their energy at exactly the null frequencies; natural scenes have almost none there, and JPEG 4:2:0 has already halved the chroma before you look. Real footage scored 0.3988-0.7691 and generated 0.2050-0.4309 — overlapping distributions, no detector. A specific small feature is destroyed, and the frame carries a detectable signature, are different claims. Only the first one is ours.

How we got here

Two literature checks killed the two framings we started with, in about an hour, before any real compute was spent. "FID doesn't predict whether synthetic data transfers" has been settled since Ravuri and Vinyals (NeurIPS 2019). "Latent autoencoders destroy identity while PSNR/SSIM look fine" is TokBench (arXiv 2505.18142). What survived both checks was the one thing six search angles found nothing on: chroma spatial bandwidth through a latent autoencoder.

Related result from the same work: face identity has a per-resolution ceiling before any generation happens. An autoencoder round-trip alone — no diffusion, no drift — retains 0.514 identity at a 54px face and 0.910 at 110px. But that ceiling is slack: generating at each rung, the full pipeline delivers far below it and stays under the same-person threshold. The transformer, not the autoencoder, dominates identity loss, so raising resolution buys much less than the ceiling curve suggests.

Code and data