Solution: Dithering 16-bit PNGs to a format x264 understands

[Mathias]

With the 16-Bit PNGs from Sintel being released I looked for a way to dither them into a format x264 understands.
I tried Imagemagick and ffmpeg (and derivatives) and I failed with both of them to dither the pngs. So I wrote my own crappy tool that does:
1) Read a PNG
2) Convert it to double precision YUV 4:2:0
3) Dithers to 8-bit

Thanks a lot to the guys from libcaca for doing this study. Since they are only doing dithering to 1-bit images, I adapted it in the way that I only dither the lower 8-bits and use it to decide whether to round up or down.
I implemented or adapted nearly every (appliable) algorithm in that study. My results were, good old ordered dithering with a "void and cluster" generated matrix performed the best (at reasonable speeds). See the last images on this page for samples. See here for the usually used algorithms. Floyd steinberg is the one "everyone" uses, x264 uses "Filter Lite". Notice how all of them still have some residual banding, and my tests showed the banding is still visible if you apply it for some scenes in Sintel.
As for the "Void and Cluster" matrix, I used the smaller 14x14 matrix because I didn't want to copy 25x25 values (I didn't implement the void and cluster algorithm).

I also tried model based dithering. While it had really awesome results, it was just to damn slow for anything above 720p. I took ~8 seconds for one iteration of a 1280x544 image on a single core of my CoreDuo T2300 (1.66Ghz).

To improve still shots, so the dithering noise wasn't static, I used 4 mirrored versions of the same matrix to fake some temporal noise.

So without further ado, here is the source:
http://www.mediafire.com/?1b0yr9jam363gbs
It includes a crappy Makefile for Linux and cross compiling to Windows. It links agains libpng, so you need libpng-dev (and zlib-dev with it) installed, also you need the png++dev header files (because I'm lazy).

For cross compiling I used the following additional files:
http://www.mediafire.com/?4dy47oy46azcj4n
These files includes png++ (BSD license), libpng (libpng License) and zlib (zlib License) header files and files needed for linking.

Here is a windows binary (together with the libpng and zlib dlls):
http://www.mediafire.com/?65kao7c24zaoyo6

The program has no real output, but usage is quite simple:
> dither input.png output.yv12 [rotation]

The rotation parameter is optional and takes values from 0-3. It determines what direction the dithering matrix is scanned in. So use it with something like "Framenumber modulo 4".
The output is appended to the output filename. So theoretically you can just call the programm from frame 0 to the last one with the same output file. But since you probably want to call the program more then once in parallel, I used a new file for each input and concatenated them later. You might want to use a compressed filesystem for the temporary yv12 files because they are quite huge (~1MB per 720p frame, ~2.3MB for 1080p)

I'm releasing this under a "do what you want but don't laugh at my code" license. In the source there is only the final dithering algorithm, if youre interested in the others, just ask.

*edit* By the way, those streams can be encoded with those settings via x264:
--demuxer lavf --input-res 1280x544 --input-fmt rawvideo --input-depth 8 --fullrange off --colormatrix bt709
(which I think makes them not yv12 but yuv420 files...)

[mp3dom]

Really interesting.
Surely Floyd Steinberg leaves in general some banding but the Sintel trailer itself contains banding issues even in the 16bit PNG/TIF (just watch frame 100 for example).
With your dither program, for every 16bit PNG input there's a 8bit 4:2:0 output that can now be understandable by x264.
The question anyway is: Is there a way to concatenate all the files to create one single 'movie' without AviSynth? I mean, probably you want a final avc file, not an avc file for every image.

[Mathias]

The 16bit PNGs don't contain banding, the only problem is: You don't have a monitor and OS capable of displaying them in 16-bit color resolution. And because your display program probably doesn't use dithering, it just drops the lower 8 bits and degenerates them to the quality of the 8bit images. If your display (and the whole displaying chain) would support more then 8-bit color depth, you wouldn't see any banding and no dithering was needed.

Concatenating the files is very possible. With linux its as simple as doing something like "cat *.yv12 > full.yv12", on windows there are programs that can do the same, just google for "join files" and you will find some.
Also, if you give the program the same output file for every input file, it will append the frames to the files. But remember, if you ever stop the program during execution and only half a frame was written, you can't continue anymore. Thats why I write single files and concatenate them later.

[skittle]

mp3dom see this thread
http://forum.doom9.org/showthread.php?p=1458396

[Mathias]

mp3dom see this thread
http://forum.doom9.org/showthread.php?p=1458396

That doesn't seem to work with raw sources. At least both these lines:
> x264 --demuxer lavf --input-res 1280x544 --input-fmt rawvideo --input-depth 8 --input-csp yv12 --fullrange off --colormatrix bt709 --fps 24 --frames 100 -o test.mkv 720p/%08d.yv12
and:
> x264 --demuxer raw --input-res 1280x544 --input-depth 8 --input-csp yv12 --fullrange off --colormatrix bt709 --fps 24 --frames 100 -o test.mkv 720p/%08d.yv12
result in:
> x264 [error]: could not open input file `720p/%08d.yv12'

although when I replace %08d.yv12 with 00000001.yv12 one frame is encoded. Also just a simple
> x264 -o test.mkv 720p/%08d.png
works.

[mp3dom]

Tried with the 4K 16bit PNG and I think the program screw something. It works with the 1K source but with the 4K source x264 throw a warning saying that it cannot decode the frame.
Apart from this, I've tried it with the 1K source (frame 100) and x264 with qp=0 (lossless) and I continue to notice banding (but maybe they've just rendered the image in this way) in the upper right side of the image.

[Mathias]

It works for me with the 4k sources. I usually use a Linux version, but the windows one does work with wine for me too.
What is the exact size of your resulting file? For 4k you should get exactly 10715136 bytes per frame.

Also frame 100 hasn't much banding to begin with. In the upper right corner, there is something you might think is banding, but thats a wanted discontinuity.
Here is how it should look:
http://i.imgur.com/6Iofr.png

[mp3dom]

Maybe I'm using a different source...
I've tried this: http://media.xiph.org/sintel/sintel-4k-png16/00000100.png
and the yv12 file is 13.070.976 byte.
I'm using Win7 x64 (I don't think it makes difference, but just to let you know).
I've used both x64 and x86 version of x264 with the same result (warning unable to decode frame 1, with the resulting image being greenish and messy)
Regarding the 1K source, yeah, it looks near the same.

[Mathias]

I'm using the same source. Make sure to delete the output file before you execute the program.
Also please post your x264 command line.

[kweek]

Can you give an example of what a command would look like to batch convert a sequence of frames in linux?

Will this also be better with the png's from Big Buck Bunny?

[Mathias]

I made a small bash script that does the job for me. Its a little messy, because I have 5 different versions (720p, 1080p, 4k, PAL and NTSC DVD resolution PNGs).
I cleaned it a little and this is the result:
http://pastebin.com/4zaMXurc
Edit the in and out variables to suit your folders.
After that, I check that there are as many files as expected:

Code: [Select]

> ls outdir | wc -l-> Should result in 21312 files for Sintel
The I check that every file was finished converting (because I sometimes stopped and a file was only half written). Therefore calculate the expected size of one yv12 file (ResolutionX*ResolutionY*1.5), for Sintel 4k its 10715136:

Code: [Select]

> ls -l outdir|grep -v 10715136-> Should result in no files displayed.
If there are files of a different size, delete them and rerun the script (it only processes files that don't already have an output file.
After that, I concatenate all the separate yv12 files:

Code: [Select]

> cat outdir/* > result.yv12
From there you can use the x264 command line from the first post to process it.

As far as I know, there were only 8-bit per channel PNGs released for BBB, so you can't gain anything with this tool. I'm not sure if 16-bit PNGs exist, I might just ask Ton about it. But I don't remember being annoyed by banding in BBB as much as I was in Sintel...

[kweek]

I
From there you can use the x264 command line from the first post to process it.

As far as I know, there were only 8-bit per channel PNGs released for BBB, so you can't gain anything with this tool. I'm not sure if 16-bit PNGs exist, I might just ask Ton about it. But I don't remember being annoyed by banding in BBB as much as I was in Sintel...

Tnx for the commands, they are a real help.

I also do not know about any 16bit PNGs for BBB but I they are PNG24 (4:2:2) and they still need to be converted to yv12 (4:2:0). So I thought that your method would do a better job than libav.

[Mathias]

You might be right. PNG24 (more like 4:4:4) looses color information during the conversion to YV12, so dithering will help. But then libav does dithering (although a different algorithm) for 16-bit sources, its just that the 16-bit import filters seem bugged. But I don't know if they dither for 8-bit sources too, so there might still be an advantage to gain.
Do you remember banding in BBB? Because if it doesn't have banding, this process will probably decrease compressibility as it introduces some noise.
I suggest, just try it out. It should work, but I don't know if it works better then what libav does.

[ramicio]

What if I want to make a 4:2:2 (yv16) video?

I tried this and still got gross banding with 10-bit x264.

[sneaker]

As I said in the other thread:
VapourSynth imagereader plugin + fmtconv for csp conversion, resizing/subsampling and dithering. (Or at least the dithering in case fmtconv's resizing is overkill for you)
Then y4m to x264cli.

[ramicio]

As I said in the other thread...I'm not using Windows and I have zero desire to try to get wine to act appropriately and my encoding is automated.

[sneaker]

VapourSynth is multi-platform

[ramicio]

I don't want it.  What I have now works just fine.  I don't want to deal with compiling all sorts of crap.

[sneaker]

I tried this and still got gross banding with 10-bit x264.

I don't want it.  What I have now works just fine.

If you don't want a solution for your problem then stop asking.

[ramicio]

I asked for a solution with this specific piece of software this thread is about.  As I said in the other thread, I really had no interest in the Sintel thing.  I have no problem other than the claim that 8 bits stuffed into a 10 bit thing can reap any benefit.  What I use to compress every piece of material does not introduce banding and saving a few GB over 300 movies and TBs of shows isn't worth the marginally slower encode.

[sneaker]

I asked for a solution with this specific piece of software this thread is about.

Fair enough, but still, you can't go around saying "everything's fine"/"I don't need anything" and at the same time ask for help solving your problems:

What I use to compress every piece of material does not introduce banding and saving a few GB over 300 movies and TBs of shows isn't worth the marginally slower encode.

I tried this and still got gross banding with 10-bit x264.

Maybe you should pose your goals and questions more clearly.

As I said in the other thread, I really had no interest in the Sintel thing.

I never talked about Sintel in here, but it's the same topic as in other thread: 16 bit PNG input

I have no problem other than the claim that 8 bits stuffed into a 10 bit thing can reap any benefit.

http://x264.nl/x264/10bit_02-ateme-why_does_10bit_save_bandwidth.pdf
http://x264.nl/x264/10bit_03-422_10_bit_pristine_video_quality.pdf
http://x264.nl/x264/10bit_02-ateme-why_does_10bit_save_bandwidth.pdf
tl;dr: Not only the output precision is increased, but also the precision of the calculations within the encoder/decoder. This reduces rounding errors and increases quality in turn. (introducing less banding than the same calculations done in 8 bit).
Look at eac3to or other audio decoders: Even if the original uncompressed audio only had 16 bit, they still output 32 bit (or even 64) when decoding lossy formats. Lossy compression and filtering changes the 8 bit input.

x264cli has built-in colorspace conversion and resizing filters, so maybe you should test out those if you don't want to use VapourSynth.

[ramicio]

I wasn't asking for help.  It was an experiment.

I have no goals.  It was an experiment.

I did try out x264's (I don't know why you keep putting "cli" on the end of that) filters.  From what the x264 developer said, it makes me see it as x264 is going to always crush everything to an 8 bit color space internally.

tl;dr?  What a mature attitude.

[sneaker]

I wasn't asking for help.  It was an experiment.

I have no goals.  It was an experiment.

So that thing with a question mark at the end wasn't a question?

What if I want to make a 4:2:2 (yv16) video?

(I don't know why you keep putting "cli" on the end of that) filters.

To make clear I'm not talking about libx264.

From what the x264 developer said, it makes me see it as x264 is going to always crush everything to an 8 bit color space internally.

Yes, it could need an overhaul. Also does some silly stuff like 10 bit -> 16 bit -> 10 bit, unless you use a patched build.

tl;dr?  What a mature attitude.

I summarized the very core of those three papers and that's your reaction?

[pengvado]

From what the x264 developer said, it makes me see it as x264 is going to always crush everything to an 8 bit color space internally.

No. It's one specific use-case that's bugged and will be fixed, namely the CSP conversion step of a high depth encode from an 8bit source of a different colorspace without explicitly specifying the conversion filter. This bug does not apply to a 10bit encode from an 8bit YUV source (since no CSP conversion is needed there), nor from a 16bit RGB source (since that retains the input depth which is high enough), nor to anything that happens inside the encoder.