Author Topic: x264 LAVF/FFMS input beta test thread (v2.1)  (Read 21285 times)

Offline Dark Shikari

  • x264 developer
  • Administrator
  • Member
  • *****
  • Posts: 650
    • View Profile
x264 LAVF/FFMS input beta test thread (v2.1)
« on: December 29, 2009, 10:27:42 PM »
Download v2.1
Patch v2.1

Download v1
Patch v1

New features (v2.1):

1.  Input from anything, even without Avisynth or DirectShow codecs, even on Linux!
2.  "True VFR": x264 maintains timestamps from the input, allowing native processing of VFR video.  No more timecode files!
3.  Use --demuxer to force a particular input method (lavf, ffms, etc).
4.  Automatic handling of all kinds of weird input (changing resolution, etc).
5.  A near-complete rewrite of the x264cli internals to handle the above.
6.  Periodic Intra Refresh.

See the three commit messages in the patch for more details.

Gotchas (v2.1):

1.  There seem to be some types of files (raw h264?) that FFMS refuses to index.  We'll be looking at that, but LAVF should work fine in the meantime.
2.  FFMS won't work on piped input.
3.  x264 doesn't by default save the index file from FFMS, so it has to re-index on every pass unless you use --index.  We may change the default behavior in the future.

Now that we're through the gotchas, feel free to test on various types of input and report any issues you have.

Disclaimer: this is a HUGE patch.  It modifies dozens of files and likely has a lot of bugs.  Problems are normal, and the purpose of this test is to root out as many as possible.

Mega-thanks to Mike Gurlitz (ACoolie), Steven Walters (Kemuri9), and Kieran Kunhya (Kierank) for the massive amount of high-quality work that went into this patch.

Offline kemuri-_9

  • Compiling Encoder
  • Member
  • Posts: 269
    • View Profile
Re: x264 LAVF/FFMS input beta test thread
« Reply #1 on: December 29, 2009, 10:47:43 PM »
as the owner of the test build, details are
Code: [Select]
Platform:   X86
System:     MINGW
asm:        yes
avs input:  avs
lavf input: yes
ffms input: yes
mp4 output: yes
pthread:    yes
debug:      no
gprof:      no
PIC:        no
shared:     no
visualize:  no
gcc 4.4.2
no other patches.
ffmpeg SVN-r20953
ffms2 r253

priority of attempting to open arbitrary files is ffms > lavf > avs.

Offline Jou

  • Member
  • Posts: 12
    • View Profile
Re: x264 LAVF/FFMS input beta test thread
« Reply #2 on: December 29, 2009, 11:24:52 PM »
Thiis an 32 bit build, confirmed by taskmanager and it automatically loads avisynth 32 bit : ).

Quick test result if the source is vdub frameserver:

C:\Users\Jou\AppData\Local\Temp\264_lavf+ffms>x264.exe --demuxer ffms --output test.mkv e:\Aufnahme-out\x264.avi
ffms [error]: could not create index
<boooom crash>

C:\Users\Jou\AppData\Local\Temp\264_lavf+ffms>x264.exe --demuxer lavf --output test.mkv e:\Aufnahme-out\x264.avi
lavf [error]: could not open input file
<boooom crash>

C:\Users\Jou\AppData\Local\Temp\264_lavf+ffms>x264.exe --output test.mkv e:\Aufnahme-out\x264.avi
ffms [error]: could not create index
lavf [error]: could not open input file
avs [info]: trying AVISource... succeeded
avs [warning]: converting input clip to YV12
avs [info]: 1920x1080 @ 23.98 fps (141883 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast FastShuffle SSEMisalign LZCNT

x264 [info]: profile High, level 4.0

aborted at input frame 12, output frame 0

(edit: The last one does not crash, I just hit ctrl+c)
« Last Edit: December 30, 2009, 10:25:07 AM by Jou »

Offline Dark Shikari

  • x264 developer
  • Administrator
  • Member
  • *****
  • Posts: 650
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (Beta v2!)
« Reply #3 on: January 01, 2010, 03:22:22 PM »
Updated to version 2, with too many bugfixes to list.

Offline saintdev

  • Member
  • Posts: 22
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #4 on: January 01, 2010, 04:45:43 PM »
Wouldn't it be better to separate out PIR?

Offline kemuri-_9

  • Compiling Encoder
  • Member
  • Posts: 269
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #5 on: January 01, 2010, 04:53:22 PM »
Wouldn't it be better to separate out PIR?
it's seemingly a single commit, but it's actually a culmination of 5 separate commits.
« Last Edit: January 01, 2010, 04:55:40 PM by kemuri-_9 »

Offline Dark Shikari

  • x264 developer
  • Administrator
  • Member
  • *****
  • Posts: 650
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #6 on: January 01, 2010, 08:01:15 PM »
Wouldn't it be better to separate out PIR?
This is really just a test of "my entire local tree", which happens to include PIR.

Offline Sepp

  • Member
  • Posts: 4
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #7 on: January 02, 2010, 09:27:29 AM »
Code: [Select]
ffms [error]: could not create index
lavf [error]: could not open input file
avs [info]: trying FFmpegSource2... indexing... failed
avs [info]: trying DSS2... not found
avs [info]: trying DirectShowSource... failed
avs [error]: unable to find source filter to open `C:\Users\Me\Desktop\x264_te
stenc.mkv'
yuv [error]: rawyuv input requires a resolution.
x264 [error]: could not open input file `C:\Users\Me\Desktop\x264_testenc.mkv'
 via any method!
ffms [error]: could not create index
lavf [error]: could not open input file
avs [info]: trying FFmpegSource2... indexing... failed
avs [info]: trying DSS2... not found
avs [info]: trying DirectShowSource... failed
avs [error]: unable to find source filter to open `C:\Users\Me\Desktop\x264_te
stenc.mkv'
yuv [error]: rawyuv input requires a resolution.
x264 [error]: could not open input file `C:\Users\Me\Desktop\x264_testenc.mkv'
 via any method!

X264 couldn't reencode it's own file. Apart from that, everything seems fine.
« Last Edit: January 02, 2010, 01:27:24 PM by Sepp »

Offline fields_g

  • Member
  • Posts: 10
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #8 on: January 02, 2010, 09:44:07 AM »
New features (v2.1):
1.  Input from anything, even without Avisynth or DirectShow codecs, even on Linux!
2.  "True VFR": x264 maintains timestamps from the input, allowing native processing of VFR video.  No more timecode files!
3.  Use --demuxer to force a particular input method (lavf, ffms, etc).
4.  Automatic handling of all kinds of weird input (changing resolution, etc).
5.  A near-complete rewrite of the x264cli internals to handle the above.
6.  Periodic Intra Refresh.

After reading #4, I thought I would test out the "etc".  I the Star Trek Voyager episodes on DVD.  They are Film(Progressive)/NTSC(Interlaced) VFR hybrid.  When I have version 2.1 ingest the VOB, I get:

Code: [Select]
E:\x264input>x264_lavfffms2.exe split_1.VOB --output split_1_default.mp4
ffms [info]: 720x480i 8:9 @ 45000/1001 fps (vfr)
x264 [warning]: input appears to be interlaced, enabling interlaced mode.
                If you want otherwise, use --no-interlaced
x264 [warning]: interlace + weightp is not implemented
x264 [info]: using SAR=8/9
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 3.1
x264 [info]: frame I:32    Avg QP:19.57  size: 22653
x264 [info]: frame P:1215  Avg QP:24.05  size:  5674
x264 [info]: frame B:1233  Avg QP:25.82  size:  1355
x264 [info]: consecutive B-frames: 21.9% 28.8% 12.4% 36.9%
x264 [info]: mb I  I16..4: 16.8% 64.7% 18.5%
x264 [info]: mb P  I16..4:  2.0%  6.0%  0.7%  P16..4: 40.9% 13.2%  7.0%  0.0%  0.0%    skip:30.2%
x264 [info]: mb B  I16..4:  0.0%  0.2%  0.0%  B16..8: 38.7%  0.5%  0.6%  direct: 1.6%  skip:58.4%  L0:38.3% L1:58.5% BI: 3.2%
x264 [info]: 8x8 transform intra:67.5% inter:78.6%
x264 [info]: coded y,uvDC,uvAC intra: 63.8% 72.6% 40.0% inter: 13.4% 17.6% 1.8%
x264 [info]: i16 v,h,dc,p: 19% 35%  4% 42%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  8% 31% 19%  5%  5%  3% 12%  4% 14%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 11% 39% 19%  4%  5%  3%  9%  2%  8%
x264 [info]: ref P L0: 48.3% 31.2%  9.8%  4.3%  3.4%  3.0%
x264 [info]: ref B L0: 50.6% 39.0%  6.1%  4.4%
x264 [info]: ref B L1: 56.4% 43.6%
x264 [info]: kb/s:1347.13

encoded 2480 frames, 17.10 fps, 776.99 kb/s

It appears to not like hybrid at all.  Output is mangled (frozen and/or smeared images). This also happens with "--no-interlaced".

Dark, I'll PM you links to both the VOB and my output.



EDIT:
Just tried with "--demuxer lavf" and got the following:

Code: [Select]
E:\x264input>x264_lavfffms2.exe split_1.VOB --output split_1_default_lavf.mp4 --
demuxer lavf
[mpeg @ 003ea7d0]max_analyze_duration reached
lavf [info]: 720x480i 8:9 @ 60000/1001 fps (vfr)
x264 [warning]: input appears to be interlaced, enabling interlaced mode.
                If you want otherwise, use --no-interlaced
x264 [warning]: interlace + weightp is not implemented
x264 [info]: using SAR=8/9
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 3.1
lavf [error]: invalid timestamp for frame 2479. Use --force-cfr and specify a framerate with --fps
x264 [info]: frame I:32    Avg QP:19.57  size: 22653
x264 [info]: frame P:1214  Avg QP:24.05  size:  5674
x264 [info]: frame B:1233  Avg QP:25.82  size:  1356
x264 [info]: consecutive B-frames: 21.8% 28.9% 12.4% 36.9%
x264 [info]: mb I  I16..4: 16.8% 64.7% 18.5%
x264 [info]: mb P  I16..4:  2.0%  6.0%  0.7%  P16..4: 40.9% 13.2%  7.0%  0.0%  0.0%    skip:30.2%
x264 [info]: mb B  I16..4:  0.0%  0.2%  0.0%  B16..8: 38.7%  0.5%  0.6%  direct: 1.6%  skip:58.4%  L0:38.4% L1:58.5% BI: 3.2%
x264 [info]: 8x8 transform intra:67.6% inter:78.6%
x264 [info]: coded y,uvDC,uvAC intra: 63.8% 72.6% 40.0% inter: 13.4% 17.6% 1.8%
x264 [info]: i16 v,h,dc,p: 19% 35%  4% 42%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  8% 31% 19%  5%  5%  3% 12%  4% 14%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 11% 39% 19%  4%  5%  3%  9%  2%  8%
x264 [info]: ref P L0: 48.3% 31.2%  9.8%  4.3%  3.4%  3.0%
x264 [info]: ref B L0: 50.6% 39.0%  6.1%  4.4%
x264 [info]: ref B L1: 56.4% 43.6%
x264 [info]: kb/s:1795.85

encoded 2479 frames, 19.10 fps, 776.54 kb/s

That was watchable.  Had an error about timestamps though. I think it is all interlaced though.

« Last Edit: January 02, 2010, 10:23:32 AM by fields_g »

Offline Dark Shikari

  • x264 developer
  • Administrator
  • Member
  • *****
  • Posts: 650
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #9 on: January 02, 2010, 10:08:11 AM »
Do note that x264 currently cannot switch between progressive and interlaced in the same stream, so it'll be forced to encode the whole thing as interlaced for now.  The ideal in the future would probably be to add a deinterlacer or similar.

I'll look into the weirdness.

Edit: Weirdness only occurs with mp4.  MKV, FLV, and raw are fine.

Offline doits

  • Member
  • Posts: 9
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #10 on: January 03, 2010, 04:38:21 AM »
having a weird linking-problem! (maybe my linker is busted, but maybe the patch):

compiling ffmpeg, ffmpegsource and x264 by myself, ubuntu karmic x86_64

first ffmpeg (note: no --enable-libx264, since it's not installed yet):

Quote
--extra-cflags='-march=native -fPIC' --enable-shared --enable-avfilter --enable-avfilter-lavf --enable-vdpau --enable-bzlib --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-pthreads --enable-zlib --disable-stripping --enable-gpl --enable-postproc --enable-libmp3lame --enable-libfaac --enable-nonfree --enable-libfaad --enable-libxvid --disable-static --enable-libdc1394 --enable-libopenjpeg --arch=native

then ffmpegsource (default configuration-options)

and x264:

Quote
--extra-cflags="-march=native" --enable-shared

everything works as expected (did not test much yet, but i can execute ffmpeg and x264, lavs and ffms-input-support got compiled in).


now i got a /usr/local/lib/libx264.so.81. okey, want ffmpeg to support it, so recompile ffmpeg:

Quote
--extra-cflags='-march=native -fPIC' --enable-shared --enable-avfilter --enable-avfilter-lavf --enable-vdpau --enable-bzlib --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-pthreads --enable-zlib --disable-stripping --enable-gpl --enable-postproc --enable-libmp3lame --enable-libfaac --enable-nonfree --enable-libfaad --enable-libxvid --disable-static --enable-libdc1394 --enable-libopenjpeg --arch=native --enable-libx264

and now this killed ffmpeg and x264:

Code: [Select]
~/svn/ffmpeg$ x264
x264: error while loading shared libraries: libx264.so.81: cannot open shared object file: No such file or directory
~/svn/ffmpeg$ ffmpeg
ffmpeg: error while loading shared libraries: libx264.so.81: cannot open shared object file: No such file or directory
~/svn/ffmpeg$ ldd `which x264`
        linux-vdso.so.1 =>  (0x00007fff0abff000)
        libm.so.6 => /lib/libm.so.6 (0x00007f0a6d5e6000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x00007f0a6d3ca000)
        libavformat.so.52 => /usr/local/lib/libavformat.so.52 (0x00007f0a6d10e000)
        libswscale.so.0 => /usr/local/lib/libswscale.so.0 (0x00007f0a6ceee000)
        libpostproc.so.51 => /usr/local/lib/libpostproc.so.51 (0x00007f0a6ccd4000)
        libavcodec.so.52 => /usr/local/lib/libavcodec.so.52 (0x00007f0a6c196000)
        libavutil.so.50 => /usr/local/lib/libavutil.so.50 (0x00007f0a6bf84000)
        [...]
        libx264.so.81 => not found
        [...]
~/svn/ffmpeg$ ldd `which ffmpeg`
        linux-vdso.so.1 =>  (0x00007fffb29ff000)
        libavfilter.so.1 => /usr/local/lib/libavfilter.so.1 (0x00007fb2f58e9000)
        libpostproc.so.51 => /usr/local/lib/libpostproc.so.51 (0x00007fb2f56cf000)
        libavdevice.so.52 => /usr/local/lib/libavdevice.so.52 (0x00007fb2f54c5000)
        libavformat.so.52 => /usr/local/lib/libavformat.so.52 (0x00007fb2f5209000)
        libavcodec.so.52 => /usr/local/lib/libavcodec.so.52 (0x00007fb2f46cb000)
        libavutil.so.50 => /usr/local/lib/libavutil.so.50 (0x00007fb2f44b9000)
        libswscale.so.0 => /usr/local/lib/libswscale.so.0 (0x00007fb2f4299000)
        [...]
        libx264.so.81 => not found
        [...]
~/svn/ffmpeg$ ls -l /usr/local/lib/libx264.*
-rw-r--r-- 1 root root 1068438 2010-01-03 13:08 /usr/local/lib/libx264.a
lrwxrwxrwx 1 root root      13 2010-01-03 13:08 /usr/local/lib/libx264.so -> libx264.so.81
-rwxr-xr-x 1 root root  805888 2010-01-03 13:08 /usr/local/lib/libx264.so.81

note, other shared libraries in /usr/local/lib are loaded, but not libx264.so.81. and now the strange way to fix it:

Code: [Select]
~/svn/ffmpeg$ sudo ln -s /usr/local/lib/libx264.so.81 /usr/lib/
/svn/ffmpeg$ x264
x264 [error]: No input file. Run x264 --help for a list of options.
~/svn/ffmpeg$ ffmpeg
FFmpeg version SVN-r21004, Copyright (c) 2000-2010 Fabrice Bellard, et al.
  built on Jan  3 2010 13:14:05 with gcc 4.4.1
[...]

wtf? where is the error in this procedure? why does it not find the lib in /usr/local/lib directly? maybe some kind of circular linking-problem or wrong order?

compiling x264 without the patch and recompiling ffmpeg (since library-version changed from .81 to .80) does find the libx264.so.80 (for ffmpeg) in /usr/local/lib as it should, x264 does not need the library:

Code: [Select]
~/svn/ffmpeg$ ldd `which ffmpeg`
        linux-vdso.so.1 =>  (0x00007fff9e5ff000)
        libavfilter.so.1 => /usr/local/lib/libavfilter.so.1 (0x00007f0982512000)
        libpostproc.so.51 => /usr/local/lib/libpostproc.so.51 (0x00007f09822f8000)
        libavdevice.so.52 => /usr/local/lib/libavdevice.so.52 (0x00007f09820ee000)
        libavformat.so.52 => /usr/local/lib/libavformat.so.52 (0x00007f0981e32000)
        libavcodec.so.52 => /usr/local/lib/libavcodec.so.52 (0x00007f09812f4000)
        libavutil.so.50 => /usr/local/lib/libavutil.so.50 (0x00007f09810e2000)
        libswscale.so.0 => /usr/local/lib/libswscale.so.0 (0x00007f0980ec2000)
        [...]
        libx264.so.80 => /usr/local/lib/libx264.so.80 (0x00007f097ddeb000)
        [...]
~/svn/ffmpeg$ ldd `which x264`
        linux-vdso.so.1 =>  (0x00007fffcbbff000)
        libm.so.6 => /lib/libm.so.6 (0x00007f37f22a8000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x00007f37f208c000)
        libc.so.6 => /lib/libc.so.6 (0x00007f37f1d1d000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f37f252c000)

 i'm puzzled.
« Last Edit: January 03, 2010, 04:50:21 AM by doits »

Offline Dark Shikari

  • x264 developer
  • Administrator
  • Member
  • *****
  • Posts: 650
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #11 on: January 03, 2010, 05:17:01 PM »
It's probably a very bad idea to try to link x264 against libavcodec against x264.

Offline doits

  • Member
  • Posts: 9
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #12 on: January 04, 2010, 10:53:22 AM »
does this mean libavcodec has to be built static to solve the problem? or any other option to have everything dynamic?

Offline Dark Shikari

  • x264 developer
  • Administrator
  • Member
  • *****
  • Posts: 650
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #13 on: January 04, 2010, 11:55:58 AM »
does this mean libavcodec has to be built static to solve the problem? or any other option to have everything dynamic?
No, it just means you have to do the following:

1.  Compile libavcodec, without x264.
2.  Compile x264, with libavcodec.
3.  Compile libavcodec, with x264.

Offline Jou

  • Member
  • Posts: 12
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #14 on: January 04, 2010, 04:49:07 PM »
Just as a question: If virtualdub is the frameserver for a particular .avi, do you intend to read it without avisynth or is that not the intention of your patch?

With 2.1 I still get (with virtualdub frameserver.avi):
C:\Users\Jou\AppData\Local\Temp>x264_lavfffms2.exe --demuxer ffms -o test.mkv e:\Aufnahme-out\x264.avi
ffms [error]: could not create index
x264 [error]: could not open input file `e:\Aufnahme-out\x264.avi' via any method!

C:\Users\Jou\AppData\Local\Temp>x264_lavfffms2.exe --demuxer lavf -o test.mkv e:\Aufnahme-out\x264.avi
lavf [error]: could not open input file
x264 [error]: could not open input file `e:\Aufnahme-out\x264.avi' via any method!

C:\Users\Jou\AppData\Local\Temp>x264_lavfffms2.exe -o test.mkv e:\Aufnahme-out\x264.avi
ffms [error]: could not create index
lavf [error]: could not open input file
avs [info]: trying AVISource... succeeded

It does not crash compared to V1, so that point is improved.

Offline kemuri-_9

  • Compiling Encoder
  • Member
  • Posts: 269
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #15 on: January 04, 2010, 06:50:29 PM »
Just as a question: If virtualdub is the frameserver for a particular .avi, do you intend to read it without avisynth or is that not the intention of your patch?

the provided compile of lavf/ffms input did not have ffmpeg compiled with --enable-avisynth (which would give it VFW support).
however even with recompiling lavf, ffms2, and x264 as such, lavf went
Code: [Select]
$ ./x264.exe -o NUL ./vdubserver.avs --demuxer lavf
[avs @ 003fd0a0]Could not find codec parameters (Video: rawvideo, 848x480)
[avs @ 003fd0a0]Could not find codec parameters (Audio: mp3, 48000 Hz, 2 channels, 192 kb/s)
lavf [error]: could not find input stream info
x264 [error]: could not open input file `./vdubserver.avs' via any method!
when i attempted to give it a vdub frameserved file (saved as .avs to have lavf try and use VFW on it)

however a real avisynth script file with contents (for simplicity)
Code: [Select]
version()

resulted in
Code: [Select]
lavf [warning]: converting from bgr24 to YV12
lavf [info]: 440x80p 0:1 @ 24/1 fps (vfr)
and worked successfully.

so no, don't expect anything else to support vdub frameserved files.
though lavf with VFW support will allow both direct avisynth support and standard vfw simultaneously, which could be convenient for odd reasons.

Offline doits

  • Member
  • Posts: 9
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #16 on: January 05, 2010, 06:25:35 AM »
1.  Compile libavcodec, without x264.
2.  Compile x264, with libavcodec.
3.  Compile libavcodec, with x264.

isn't it, what i have done?

compiled ffmpeg without libx264 --> Compile libavcodec, without x264.
compiled ffms, then x264 with ffms --> Compile x264, with libavcodec.
compiled ffmpeg with libx264 --> Compile libavcodec, with x264.


nevertheless, my first encode with ffms-inputed-material (mkv) finished gracefully, so besides of my linking-problems the patch is working nicely.

Offline DarkZell666

  • Member
  • Posts: 38
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #17 on: January 05, 2010, 07:51:07 AM »
It turns out
isn't it, what i have done?

compiled ffmpeg without libx264 --> Compile libavcodec, without x264.
compiled ffms, then x264 with ffms --> Compile x264, with libavcodec.
compiled ffmpeg with libx264 --> Compile libavcodec, with x264.


nevertheless, my first encode with ffms-inputed-material (mkv) finished gracefully, so besides of my linking-problems the patch is working nicely.
You'd have x264 including ffmpeg, itself including x264 (for no practical use since x264 doesn't do any decoding).
Stick with compiling libavcodec without x264 and compiling x264 with the libavcodec previously compiled and it should save you useless headaches :)

Offline doits

  • Member
  • Posts: 9
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #18 on: January 05, 2010, 03:28:58 PM »
Quote
You'd have x264 including ffmpeg, itself including x264 (for no practical use since x264 doesn't do any decoding).

but ffmpeg can encode, too. it uses libx264 for encoding.

as far as i understand, libx264 now needs libavcodec. when libavcodec is compiled with x264-encoding-support, it needs libx264. they need each other. is this a no-go? two dynamic libs demanding eachother? this would mean a drawback of this patch is having libx264 with lav-support and libavcodec with x264-encoding-support, both built dynamic, is not possible. something i could live with, though.

however, my "workaround", simply placing a link into another lib-dir for the linker solved the linking-problem - so there must be way to have this done.

btw, second encode finished. no problem.
« Last Edit: January 05, 2010, 03:31:16 PM by doits »

Offline saintdev

  • Member
  • Posts: 22
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #19 on: January 05, 2010, 03:35:58 PM »
as far as i understand, libx264 now needs libavcodec.
This is wrong. libx264 does not require libavcodec. x264 (the command-line frontend) is what does.

Offline doits

  • Member
  • Posts: 9
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #20 on: January 05, 2010, 03:38:50 PM »
Code: [Select]
ldd /usr/local/lib/libx264.so
        [...]
        libavcodec.so.52 => /usr/local/lib/libavcodec.so.52 (0x00007f4242662000)
        [...]

on my system it does.

Offline kemuri-_9

  • Compiling Encoder
  • Member
  • Posts: 269
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #21 on: January 05, 2010, 03:57:40 PM »
this would be a flaw in x264's configure/makefile system.
it uses all the ldflags when creating libx264, this would mean that libx264 would link to libavcodec when x264cli is compiled with libavcodec.
there should be a separate set of ldflags that are specific to x264cli separate from libx264.

Edit:
And yes, this problem has always been around, it just hasn't had any large negative side effects like it does now.

Edit2:
try applying this patch on top of the lavf+ffms patch, it should fix the issue.
« Last Edit: January 05, 2010, 04:14:53 PM by kemuri-_9 »

Offline DarkZell666

  • Member
  • Posts: 38
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #22 on: January 06, 2010, 01:10:30 AM »
Still, I don't get why you try to link the encoder libraries into libavcodec when x264 only needs the libavcodec decoders ...

If you're trying to build a full ffmpeg + x264-with-libavcodec-support in one step, just say so, but from x264's point of view, you're linking in useless libraries !

1) Why not build a light libavcodec with the decoders only and link that into x264 ?
2) Then, why not build x264 without libavcodec, and link that x264 build into your full libavcodec build (the one with all the encoder libraries) ?

(PS: reading makefiles is a nightmare for me yet, so I might have missed the bus completely ...).

Edit: I've just re-read kemuri's last post, which makes me think one of the two steps aren't possible, my bad.
« Last Edit: January 06, 2010, 01:18:15 AM by DarkZell666 »

Offline doits

  • Member
  • Posts: 9
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #23 on: January 06, 2010, 06:06:23 AM »
Quote
Still, I don't get why you try to link the encoder libraries into libavcodec when x264 only needs the libavcodec decoders ...

libavcodec can encode, too! this is, why it needs libx264 to encode to x264. (of course this is of no use for x264cli, which links to libx264 itself and does only need the decoders of libavcodec, like you say)

and i am not linking the files myself, but the makefile does, so what i simply want is have ffmpeg (means libavcodec) be able to encode x264 and x264cli to decode with libavcodec, both built dynamic.

Quote
try applying this patch on top of the lavf+ffms patch, it should fix the issue.

i'll try as soon i get home. but for me it looks like you found (and fixed) the problem!

edit: once i saw a output of a makefile where it checked if linking to libraries is necessary just before linking - so something like "checking if -lcrypt is needed... yes, checking if -lpthread is needed... no". if i recall correctly, this was busybox-makefile. don't know how this could work, but maybe this is something that could enhance x264-makefile?
« Last Edit: January 06, 2010, 06:13:57 AM by doits »

Offline doits

  • Member
  • Posts: 9
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #24 on: January 06, 2010, 09:41:26 AM »
yeah, your patch did it! just tested, without your patch:

Code: [Select]
gcc -shared -o libx264.so.81 [...] -Wl,-soname,libx264.so.81 -lm -lpthread -lavformat -lswscale -lpostproc -lavcodec -lavutil -lm -lz -lbz2 -lgpac_static -Wl,-Bsymbolic -s
with patch:

Code: [Select]
gcc -shared -o libx264.so.81 [...] -Wl,-soname,libx264.so.81 -lm -lpthread -lFFMS2 -Wl,-Bsymbolic -s
having both dynamic works now! thanks.

(btw: patch did not apply correctly - there is a "$libpthread" to much on line 39/40).

Offline kemuri-_9

  • Compiling Encoder
  • Member
  • Posts: 269
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #25 on: January 06, 2010, 02:26:24 PM »
with patch:
Code: [Select]
gcc -shared -o libx264.so.81 [...] -Wl,-soname,libx264.so.81 -lm -lpthread -lFFMS2 -Wl,-Bsymbolic -s
having both dynamic works now! thanks.
(btw: patch did not apply correctly - there is a "$libpthread" to much on line 39/40).

looks like i missed the -lffms2 as a shared lib part... (now fixed)
it doesn't apply on the 2.1 patch as i based it off a newer version of the patch.

Offline doits

  • Member
  • Posts: 9
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #26 on: January 06, 2010, 03:47:17 PM »
except the one "$libpthread"-thing it does, so on 2.1 it fixes the issue, too.

Code: [Select]
gcc -shared -o libx264.so.81 [...] -Wl,-soname,libx264.so.81 -lm -lpthread -Wl,-Bsymbolic -s
nice! regards

Offline sledgehammer_999

  • Member
  • Posts: 1
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #27 on: January 14, 2010, 10:41:37 AM »
I sent a bug report to Dark_Shikari via PM and he suggested to make it public here so other debuggers can look at it. So here is a copy/paste from the PM:

I used x264 LAVF/FFMS to encode it using just "--crf 24 -o input.mkv output.mkv". The resulting file **looks** fine but when I mux in the audio it isn't in sync. I remuxed the file using mkvtoolnix(opened both mkvs and selected the appropriate streams). This is a known problem to myrsloik(about a month ago). Apparently your testing didn't catch it. I'll provide a link to the upload I made for myrsloik. Check between 5:20 and 5:23 where the kid throws a stone in the lake. In the encoded file the *splash* sound comes after the stone has sunk.

link->http://www.sendspace.com/file/kes6vn
password: myrsloik

Offline Kurtnoise

  • Member
  • Posts: 30
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #28 on: January 14, 2010, 02:10:36 PM »
I'm just curious...why ffms finds my input as vfr whereas it's plain cfr ?

Quote
lionel@karma:~$ x264 --preset slow --crf 21 -o ~/t1.mkv ~/Portishead_Pnyc.m4v
ffms [info]: 702x572p 16:15 @ 25/2 fps (vfr)
x264 [info]: using SAR=16/15
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 2.2
x264 [info]: frame I:134   Avg QP:18.74  size: 63572                           
x264 [info]: frame P:9570  Avg QP:21.30  size: 18715
x264 [info]: frame B:14923 Avg QP:24.85  size:  4451
x264 [info]: consecutive B-frames:  1.5% 32.0% 59.6%  7.0%
x264 [info]: mb I  I16..4: 17.3% 41.4% 41.3%
x264 [info]: mb P  I16..4:  3.6%  3.2%  1.6%  P16..4: 43.8% 24.1% 15.5%  0.0%  0.0%    skip: 8.2%
x264 [info]: mb B  I16..4:  1.2%  0.5%  0.7%  B16..8: 40.2%  1.7%  2.6%  direct: 5.5%  skip:47.6%  L0:34.8% L1:51.2% BI:14.0%
x264 [info]: 8x8 transform intra:34.1% inter:57.6%
x264 [info]: direct mvs  spatial:99.5% temporal:0.5%
x264 [info]: coded y,uvDC,uvAC intra: 58.6% 83.0% 55.8% inter: 24.5% 30.6% 3.5%
x264 [info]: i16 v,h,dc,p: 35% 24% 28% 13%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 11% 11% 21%  7% 10% 10% 10%  9% 11%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu:  8% 28% 13%  6%  8%  8% 10%  8% 12%
x264 [info]: Weighted P-Frames: Y:5.3%
x264 [info]: ref P L0: 63.0% 13.1% 12.0%  4.9%  3.9%  3.0%  0.1%
x264 [info]: ref B L0: 85.1%  8.0%  4.5%  2.5%
x264 [info]: kb/s:1031.56

encoded 24627 frames, 6.99 fps, 2063.04 kb/s

Offline kemuri-_9

  • Compiling Encoder
  • Member
  • Posts: 269
    • View Profile
Re: x264 LAVF/FFMS input beta test thread (v2.1)
« Reply #29 on: January 14, 2010, 03:29:24 PM »
I'm just curious...why ffms finds my input as vfr whereas it's plain cfr ?

because it generally says everything is VFR.