Please critique my x264 settings!

[Mixer]

OK seeing as there's no 'right' and 'wrong' I'm looking for some areas where I can improve the efficiency of my encoding, while maintaining device compatibility, and I'm looking for some suggestions from people who know better than I.

Some background:

I have a long background with compression technologies, having been involved with companies like Macrovision, Fast, Quadrant, DPS, Darim, Optibase, Sonic Foundry, Canopus, Pioneer.  I would have considered myself an expert in MPEG2 and early MPEG4 compression, however I don't have the time professionally to keep up those skills for present methods.

Currently I work with products from Viewcast but the needs for realtime compression are very different to what I am trying to achieve here.

My aim is to create nearly lossless results from fairly poor broadcast MPEG2, for playback either on DXVA or PS3/Xbox360.  Primarily I playback on 7MCE, but I often take files to a friend's house who has a PS3.  I also have a PSP so I'd like to understand what settings I might have to change to use that (apart from resolution), or whether I can do a one-size-fits-all profile.

Contrary to the above I do store a lot of TV so I want to get the files as small as possible while maintaining the quality.  Hence quality-based 3-pass encoding.  SD encodes I do at 720 width, HD encodes at 960 width.  I've used deblocking 1:1 because the source material seems overly sharp, and I feel 1:1 delivers a very watchable picture.

My GUI is Xvid4PSP which isn't as well known as the others, and also doesn't support the standard preset system.

What I am hoping for is whether changes to things like ref frames or other more complex settings might give me more efficiency without terrible time penalties, and most importantly maintaining compatibility.  I'm encoding on a Q6600, I can't wait until I can afford an i7.

So the challenge, give me some feedback.  I'm very happy with the results I'm getting.  Typically my files are half the size of a "scene" rip with equivalent quality.

Please tell me:
1. Anything you suggest that might realise smaller files
2. Anything I'm using that might be overkill, ie not worth the tradeoff time vs filesize
3. Anything else of relevance?

Code: [Select]

video codec:
x264

video cli:
--pass 1 --crf 24.0 --level 4.1 --deblock 1:1 --direct auto --b-adapt 2 --trellis 2 --qcomp 0.5 --vbv-bufsize 24000 --vbv-maxrate 24000 --partitions all --subme 9 --me esa --rc-lookahead 60
--pass 3 --crf 24.0 --level 4.1 --deblock 1:1 --direct auto --b-adapt 2 --trellis 2 --qcomp 0.5 --vbv-bufsize 24000 --vbv-maxrate 24000 --partitions all --subme 9 --me esa --rc-lookahead 60
--pass 2 --crf 24.0 --level 4.1 --deblock 1:1 --direct auto --b-adapt 2 --trellis 2 --qcomp 0.5 --vbv-bufsize 24000 --vbv-maxrate 24000 --partitions all --subme 9 --me esa --rc-lookahead 60


[Audionut]

x264.exe --preset slow --vbv-bufsize 40000 --vbv-maxrate 40000 -o x:\output.264 x:\input.whatever


You would only need to adjust bufsize and maxrate lower for xbox360 iirc.  Not sure on the stats for that.

--crf is one pass.  What your doing is the same thing 3 times with no quality increase.

edit:  --crf 24 is to high for me personally.  But if you find the quality is still good you could try --crf 25, 26, 27 etc until you find a setting that produces acceptable quality for you.  btw: --crf 24 is the default iirc so it's not needed in the command line.

[Chikuzen]

--pass 3 --crf 24.0 …
This makes an error and encoding stops.

edit:
oops!

[Mixer]

These settings do work from my GUI. Could they be dependant on the build of x264?

I'm using a GUI with x264 1342.

[Chikuzen]

These settings do work from my GUI. Could they be dependant on the build of x264?

I'm using a GUI with x264 1342.

hmm,perhaps after going on 1stpass, xvid4psp will be checking  bitrate of 1st output file and be rewriting "--crf" in "--bitrate".
But I think that give you no increads quality as Audionut says.This is an opinion from my own experience.

[Mixer]

Yes it does look like it is changing the command line between the passes.

[Chikuzen]

Code: [Select]

--no-8x8dct --ref 3 --b-pyramid none --cqm flat --vbv-maxrate 10000 --vbv-bufsize 10000 --level 3.0These are necessary options for PSP3000.(I have PSP and PS3.)

[Audionut]

Code: [Select]

--no-8x8dct --ref 3 --b-pyramid none --cqm flat --vbv-maxrate 10000 --vbv-bufsize 10000 --level 3.0Thess are necessary options for PSP3000.(I have PSP and PS3.)

or

--profile main --ref 3 --vbv-maxrate 10000 --vbv-bufsize 10000

[Audionut]

Providing support for the community.  Free.
Finding that this has been posted on Doom9 and a developer has been answering. Priceless.

[Mixer]

Providing support for the community.  Free.
Finding that this has been posted on Doom9 and a developer has been answering. Priceless.

Yeah I x-posted expecting DS to post here and others to post there.  I learnt things from both sides and after I do some testing I'll be back with my results.

I do appreciate all the input.  I did get the device-specific input here.

Is there somewhere I can find out what settings are attached to the various internal x264 presets?  I can't seem to find that info.

[Audionut]

x264.exe --fullhelp

[skittle]

#pass >2 = placebo effect, stick with 2 pass. Also you can mix 2 pass with first pass as crf like this:

Code: [Select]

x264.exe --preset slow --level 4.1 --crf 22 --pass 1 --stats "statsfile.txt" --output NUL "input.avs"
x264.exe --preset slow --level 4.1 --bitrate <resulting_bitrate_from_first_pass> --pass 2 --stats "statsfile.txt" --output "x264output.mkv" "input.avs"

Best of both worlds

[Chikuzen]

@skittle
I cannot agree.
x264.exe does high-speed 1stpass without "--slow-firstpass".
as a result, the obtained bitrate is not reliable.
if you add "-- slow-firstpass" to 1stpass, a difference like placebo will appear to the result of 1stpass and 2ndpass.
It will obtain the result of your should would being better to change "--preset slow" to "slower" and to do only 1pass.
This is an opinion from my own experience.

[juGGaKNot]

@skittle
I cannot agree.
x264.exe dooes high-speed 1stpass without "--slow-firstpass".
as a result, the obtained bitrate is not reliable.
if you add "-- slow-firstpass" to 1stpass, a difference like placebo will appear to the result of 1stpass and 2ndpass.
It will obtain the result of your should would being better to change "--preset slow" to "slower" and to do only 1stpass.
This is an opinion from my own experience.

What source ? i think the fast settings are only the ones that are changed anyway in the second pass and it does not cause quality drop.

or something.

[Glacia]

Also you can mix 2 pass with first pass as crf like this:

That will just spend your time on useless second pass...

[Chikuzen]

resulting bitrate from 1stpass is not reliable.
http://img514.imageshack.us/img514/1273/bitrate.jpg

[skittle]

Or if you want to ensure that VBV is strictly enforced... otherwise you would just want to stick with crf

[chengbin]

Or if you want to ensure that VBV is strictly enforced... otherwise you would just want to stick with crf

VBV is strictly enforced in CRF too. Look at 1 pass VBV commit.

[JEEB]

Code: [Select]

--no-8x8dct --ref 3 --b-pyramid none --cqm flat --vbv-maxrate 10000 --vbv-bufsize 10000 --level 3.0These are necessary options for PSP3000.(I have PSP and PS3.)

I agree on everything else, but not on the VBV. It's not needed for the PSP, especially with the resolutions its decoder can play (but not only because of that as far as I know -- as far as I could check, f.ex. this encode certainly isn't under those limits: clicky). Not sure of the PS3, but IIRC it could make due with even bigger vbv numbers, if you're limiting for both devices. Also, "flat" IIRC is the default QM, which means that no "custom" QM is in use.

The PSP can also use more bframes up to the current x264 maximum (16) as long as the --ref 3 part is kept, which would be the only reason to use --level 30/3.0 instead of "--profile main" when encoding for the hand-held apparatus.

[Assassinator]

Yeah, you don't need any VBV restrictions for PSP.  Done plenty of encodes without, and nothing's failed on me yet.

On the other hand, I've experienced corruptions on the PSP before with weightp (very rare), so to be safe, you might also want to use --weightp 0.  I can post some samples if anyone wants.

[Chikuzen]

I want!

[Assassinator]

Weightp and no weightp, other settings the same (fast-ish settings which I generally use for my "watch-and-delete" PSP encodes). 3MB each.
http://zingaburga.com/assassinator/Storage/weightp0.mp4
http://zingaburga.com/assassinator/Storage/weightp2.mp4

Full opening
http://zingaburga.com/assassinator/Storage/weightp2-full.mp4

These were encoded some time ago, so not using the very latest version of x264.  It still happens with the latest x264, I just can't be bothered re-uploading new ones (dialup speed is awesome for uploading stuff).

EDIT: tried a newer x264, still happens.

[Chikuzen]

Thx Assasinator. I confirmed it.

[Dark Shikari]

Have you tried with fewer refs?  It might not be weightp, but rather the fact that weightp adds more refs, and the PSP may be artificially limited to 3 for no good reason (ignoring memory issues).

[Assassinator]

Have you tried with fewer refs?  It might not be weightp, but rather the fact that weightp adds more refs, and the PSP may be artificially limited to 3 for no good reason (ignoring memory issues).

Fails with 2 ref frames too.  For that particular video, only --ref 1 works when using weightp.

But this happens only very rarely it seems, in fact this is the only place I've seen it happen.

[Dark Shikari]

Fails with 2 ref frames too.  For that particular video, only --ref 1 works when using weightp.

But this happens only very rarely it seems, in fact this is the only place I've seen it happen.

Does it fail with --ref 2 --weightp 1?

[JEEB]

But this happens only very rarely it seems, in fact this is the only place I've seen it happen.

Very interesting, and seems to be rare indeed -- I've done some encodes with weightp and it has so far been either non-see'able or not there :3

[Assassinator]

Tested it just then... yes, weightp 1 ref 2 works just fine.

So I guess I'll still be sitting on --ref 3 --weightp 0 then, since that's better than both --ref 2 --weightp 1 or --ref 1 --weightp 2.

[burfadel]

Well firstly as others have pointed out, crf is purely a 1 pass feature, although it can be used in the first pass of a two pass encode.  Finding the right crf to suit you is vitally important.  A lower crf=higher quality.  Setting a high resolution and a high crf is relatively pointless, the picture would look much better if you set a lower resolution and lower crf (even to the point such that in both cases the output file size is similar).  Striking the right balance is the key.  Personally I wouldn't go above a crf of 23.

--me esa over --me umh isn't beneficial, it would be better to use that speed to set --subme 10 instead.

You may find benefit from setting --aq-mode 2

and

--b-pyramid normal

As others have states --b-pyramid normal has compatibility problems purely because the standalone players may not be spec-compliant.  Try using normal on a test clip, and see if it works.  If not, try --b-pyramid strict which should work in all cases, but again test.  It would be unfortunate if you couldn't set it at all as its worth a couple of percent in terms of quality.  In terms of my other recommendations on here, they will not affect compatibility as they're purely bit distribution settings.

Also try fidling with psy-rd:psy-trellis.

--psy-rd 1:0 is the default, however you may find setting psy-trellis to be beneficial, such as --psy-rd 1:0.6 (thats an example only)

I recommending updated x264 to the latest revision (always).  If the gui says that not the recommended version, its purely because command line options may change.  There have been speed improvements and one or two bugfixes since the revision you are using.