The correction was the error
My stabiliser removed a third of a pixel of camera shake and charged 28% of the image sharpness to do it. Then the quality gate threw away the result for being blurry.
Joe Cox, Resoa
I have been generating long shots of a room from a camera that is supposed to be bolted to the wall. Video models will not hold a camera still — they pan and push in even when the prompt and the negative prompt both forbid it — so I built a stabiliser. It finds the geometric transform between each frame and a reference, and warps the frame back. Standard stuff, and it worked: on the model I built it for, it removed 60% of the camera's excursion.
Then I switched models, ran it again, and it started throwing away good footage.
The complaint that made no sense
The pipeline gates every chunk of generated video: too few features, no face, or too blurred, and the chunk is rejected and recorded. Out of thirty chunks, two came back blurred.
Except they weren't. The gate's blur threshold is a Laplacian variance of 120. Those two chunks measured 279.6 and 267.6 — more than double the bar. One of them was sharper than twenty of the twenty-eight chunks that passed.
So the footage entering the pipeline was fine, and the thing coming out the other end was blurred. The blur was being manufactured in between, by the only step in between: my stabiliser.
A third of a pixel
Here is what the stabiliser was correcting. Across 180 frames, the camera's displacement from the reference was:
| this model | the model I built the stabiliser for | |
|---|---|---|
| mean drift | 0.331 px | 61 px |
| max drift | 2.735 px | 449 px |
About 180 times stiller. The new model's camera is, for practical purposes, already bolted to the wall. The transforms my stabiliser was computing were things like shift right by 0.571 pixels, down by 0.401.
You cannot shift an image by 0.571 pixels. There is no such pixel. What actually happens is that every output pixel is interpolated from its neighbours — and the default interpolation, bilinear, at an offset of half a pixel, is precisely the average of two adjacent pixels. A two-tap box blur, applied to the entire frame, to correct a drift you could not see if you were looking for it.
The two rejected chunks had drawn offsets nearest to half a pixel, where that blur is worst. They weren't bad chunks. They were unlucky ones.
How much it costs
I imposed known shifts on real frames and measured what survived, which makes the ground truth exact. Sharpness retained after a pure translation:
| offset | nearest-neighbour | bilinear | bicubic | Lanczos |
|---|---|---|---|---|
| 0.10 px | 100% | 72.6% | 96.2% | 98.4% |
| 0.25 px | 100% | 45.0% | 79.3% | 89.3% |
| 0.50 px | 100% | 31.4% | 63.2% | 79.1% |
Bilinear at half a pixel keeps under a third of the fine detail. And this is not a quirk of generated video: I ran the same test on footage from the real camera on my wall, and it retained 37.9% — the same effect, the same magnitude. It is a property of the resampler, not of the imagery.
On the actual warps my pipeline was computing, bilinear retained 72.1%. Twenty-eight percent of the image, gone, to correct a third of a pixel.
Where the number comes from, and why it depends on what you measure
I checked the loss against a second, independent measure — spectral energy — and got a very different answer: 79% retained, not 31%. That looked like one of the two metrics was broken.
Neither is. They weight different frequencies. Breaking the loss down by spatial frequency after a half-pixel bilinear shift:
| band (1.0 = Nyquist) | energy retained |
|---|---|
| 0.00 – 0.25 | 98.4% |
| 0.25 – 0.50 | 84.4% |
| 0.50 – 0.75 | 61.1% |
| 0.75 – 1.00 | 36.0% |
| above Nyquist | 16.2% |
A two-tap average has a transfer function of cos(πf/f_nyq): untouched at DC, zero at Nyquist. The coarse structure of the image is essentially unharmed. The finest detail is annihilated. Laplacian variance looks where the damage is; a broad spectral average looks mostly where it isn't.
That matters for honesty about the headline. "You lose 28% of the sharpness" is true for a metric sensitive to the finest detail — which is exactly the metric my quality gate uses, which is why it rejected the chunks. Quote a different band and you get a different number. The direction is not in doubt; the magnitude is a statement about the measure as much as the image.
The fix is one flag
The correction did not need to be abandoned, only performed competently. On my real warps:
| method | sharpness kept | drift left uncorrected |
|---|---|---|
| bilinear (what I had) | 72.1% | 0.00 px |
| bicubic | 89.7% | 0.00 px |
| Lanczos | 94.6% | 0.00 px |
| snap the shift to whole pixels | 100.0% | 0.17 px |
| don't correct at all | 100.0% | 0.31 px |
Changing one interpolation flag recovers most of the loss at no cost in alignment. And if your transform is essentially a translation — as mine was — you can snap it to the pixel grid and resample nothing, for a residual error of about a sixth of a pixel.
One caveat, because it is the boundary of the advice. Snapping only works for translation. The moment your correction includes a scale change you must resample, and with bilinear that is brutally expensive immediately: a scale factor of 1.0004 — four parts in ten thousand — already costs 21% of the sharpness, and it plateaus near 50% by 1.005. There is no cheap small zoom. Lanczos at that same 1.0004 costs 1.5%. On the model I originally built this for, where half the chunks zoom by more than 5%, resampling is unavoidable and the kernel choice is the whole game.
What I actually got wrong
Not the stabiliser. It does what it says, and on the footage it was written for it earns its keep.
What I got wrong was leaving a quality gate downstream of a correction and never asking which of the two it was measuring. A gate that runs after a transform reports on the transform and the content together, and it cannot tell you which one it is unhappy about. Mine spent thirty chunks confidently reporting on my own resampler while appearing to report on the model's output.
That is the sixteenth measurement error I have logged on this project, and it is the same species as most of the others: a global property of the apparatus, reported as a local property of the subject. The tell was available the whole time — the rejected chunks' raw sharpness was double the threshold — and I only looked because two rejections out of thirty seemed like a suspiciously tidy failure rate.
The rule I would give someone else:
A resampling correction costs a fixed amount of high-frequency detail, while its benefit scales with the error it removes. Before applying one, compare the two. And never let a quality gate sit downstream of a correction without checking that the correction is not what the gate is detecting.
Limits
Three sources, one kind of scene — a room interior, generated and real. Two sharpness metrics that agree in direction and disagree in magnitude for a reason I can explain but which still means "28%" should be read as metric-specific. OpenCV's particular implementations of these four kernels. The half-pixel worst case for bilinear is textbook signal processing and is not a discovery; what I had not seen written down anywhere is what it does to a quality gate sitting behind it, which in my case was to silently discard 6.7% of a deliverable.
The measurement is cheap and needs no model: impose a known shift, measure what survives, and check that an integer shift changes nothing. That last one is the known-answer test, and it is the reason I trust any of these numbers.