Real-ESRGAN General x4v3 測試顯示,分塊放大需 34 像素上下文才與整圖推理逐值一致
Tiled upscaling matched whole-image inference bit for bit at 34 px of context, not 33
作者以 Real-ESRGAN General x4v3 對 480×480 圖像進行分塊放大測試,發現每邊留 33 像素上下文時,11,059,200 個輸出值中有 3 個相差 1 個灰階,34 像素時則全部一致。
Split a 480×480 image into 96-pixel tiles, hand every tile 33 extra pixels of real image on each side, upscale each one 4× with Real-ESRGAN General x4v3, cut the extra ring off and paste the tiles together. Then push the whole image through the same model in one pass and compare. Of the 11,059,200 output values (1920×1920, three channels), 3 are different, each by one gray level. With 34 pixels the count is 0.
These figures come from Python with onnxruntime on the CPU of an Apple-silicon Mac, limited to 4 threads, on a 480×480 crop of a CC0 landscape photo from Wikimedia Commons by Bonnie Moreland. Image comparisons gave identical results across three runs; timings are the median of five. One model, one image.
Context of 33 pixels still left 3 values different
Differences below are in gray levels on a 0–255 scale. The coarse sweep came first, measured in the 4-pixel band on each side of every seam. With no context the worst pixel was off by 145 levels. With 4 pixels of context it was 57, with 12 it was 9, with 24 it was 1, and with 36 the two results were identical. Over the whole image the maxima are slightly higher, 149 with no context. The fine sweep located the exact point.
| Context (px) | 28 | 30 | 32 | 33 | 34 | 35 |
|---|---|---|---|---|---|---|
| Values that differ, out of 11,059,200 | 3,041 | 415 | 26 | 3 | 0 | 0 |
| Largest difference (gray levels) | 1 | 1 | 1 | 1 | 0 | 0 |
The count falls from 3,041 to 3 over five pixels, then sits at zero. At 34 the floating-point outputs match as well, so the zero is not produced by rounding to 8 bits.
Each convolution layer reaches one pixel further
What follows is an inference from the model's structure, not a second measurement. This model is 34 convolution layers with 3×3 kernels, one after another. A 3×3 kernel reads one pixel past the position it writes, so every layer lets information travel one pixel further. After 34 layers an output pixel can depend on input up to 34 pixels away in each direction and on nothing beyond that.
In a whole-image pass, a pixel on a tile border sees 34 pixels of real image on the far side. In a tile that carries 34 pixels of context it sees the same 34. With 33, the outermost ring comes from the patch edge instead of the real image, and it can reach the kept area only through all 34 layers. That path probably carries very little weight, which would explain why only 3 values change. The layer argument was not confirmed another way, such as changing one input pixel 35 pixels away and checking that nothing moves.
A sketch of the pad-and-crop step for one tile, simplified from the definition. The figures in this post come from the full script, not from these lines.
ctx = 34
top, left = max(y - ctx, 0), max(x - ctx, 0)
patch = image[top:min(y + size + ctx, H), left:min(x + size + ctx, W)]
big = model(patch)
oy, ox = (y - top) * 4, (x - left) * 4
canvas[y*4:(y+size)*4, x*4:(x+size)*4] = big[oy:oy + size*4, ox:ox + size*4]
Past 24 pixels only tiny differences remain
With 96-pixel tiles, the five-run medians were 1.59 s with no context, 2.28 s with 12 pixels (+43%), 3.09 s with 24 (+94%) and 4.05 s with 36 (+155%). The whole image in one pass took 1.60 s. A 96-pixel tile with 36 pixels of context is a 168×168 patch, 3.06 times the pixels of the tile (simple arithmetic, not timed). With 160-pixel tiles the same three steps cost +23%, +47% and +74%. A context of 34 was not timed, so 36 is the nearest measured point; going from 24 to 36 removed the last differences of one or two gray levels.
One browser tool overlaps by 12 pixels
ImgIng https://imging.ai/ is a browser tool whose fast upscaling tier runs this same model file inside a Worker with 160-pixel tiles and 12 pixels of overlap. This is an observation of its output only. The inputs were three 256×256 samples upscaled 4×, once each, made from CC0 photos on Wikimedia Commons (portrait by William Stitt, landscape by Bonnie Moreland) and aged by script with yellowing, blur and scratches, so they are not real old photos. Each output was subtracted from a whole-image CPU pass of the same model. The largest differences sat around output column 640 and row 640, which is input pixel 160, the tile border. The maximum was 2 levels on two samples and 16 on the scratched one. Part of that is floating-point difference between browser GPU and local CPU inference, so it cannot all be booked as seam error; the position is the part that is certain. In the local runs, 12 pixels of context left a worst case of 9 levels with 96-pixel tiles and 11 with 160-pixel tiles.
Attention models were not tested
The number 34 belongs to this model, because nothing in it looks further than a chain of 3×3 kernels allows. SwinIR and HAT attend across windows, no such model was run here, and nothing above says how much context they need. Identical output also says nothing about fidelity: the detail the model draws is its own guess, so it is not fit for IDs, forensics or checking OCR text.
For another purely convolutional model at stride 1, the same check takes two steps. Add up (k − 1) / 2 over the layers to get a candidate, then sweep the context around it and count differing values against a whole-image pass. If the count reaches zero earlier or later than the sum predicts, trust the sweep over the sum.
來源:DEV Community · dev.to