FFMS2 (The Fabulous FM Source 2)

[kemuri-_9]

Quick Question: Is the Avisynth version of the filter compatible with 64 bit Avisynth? If not, could support be added?

we've went over this before.
ffms2 depends on ffmpeg which does not support compilation by intel's compiler ICL, and much less microsoft's CL.
this requires ffmpeg to be compiled by MinGW.
for x86 this ok as the x86 versions of MinGW and MSVC are fairly compatible,
so the MinGW compiled ffmpeg library can be used in MSVC, which is how the plugin compilation works currently.

the question you may be having now is 'why are we using MSVC in the first place?'
the answer to that is 'MinGW can not compile working avisynth C++ plugins' (they compile, but they certainly do not work).

however, the compatibility between MinGW and MSVC that exists for x86 does not similarly exist for x86_64.
so this current methodology of using ffmpeg compiled by MinGW and using it in msvc completely fails.

so to get an x86_64 plugin two things could be done:
A) get ffmpeg to support being compiled by at least ICL
B) rewrite the ffms2 avisynth plugin to use the avisynth C plugin interface so it can be compiled by MinGW and work.

- A) is a pita and no one really wants to rewrite ffmpeg to have it be compiled by ICL.
- B) I actually did and was successfully in getting it to work for what I've tried.
However, I haven't worked on it in a while so it is outdated and any changes that occurred in the avisynth plugin portion of ffms2 would have to be reflected in my rewrite.

[TheFluff]

There haven't really been any changes to the avisynth plugin lately, only thing I remember was changing a "true" to an int in a function call, which fixed a bug with error handling

[Adub]

Thank you kemuri-_9 for your response.

The reason I asked is because there seems to be a lot of activity in the 64 bit Avisynth thread on Doom9. The author has been cranking away on a number of enhancements and plugin conversions, and since I really like ffms2, and I was considering installing Win7 w/ 64bit Avisynth, I wanted to be sure that it was still possible to use FFMS2.

So, if I understand this correctly, it is possible to run ffms2 on 64 bit using a custom build that you yourself built, kemuri-_9. Since TheFluff says that there are barely any changes to the plugin, do you think you could merge your additions?

If you don't have an inclination to do so, or the time, I completely understand. I just think that it would be a useful addition to the community.

[kemuri-_9]

it's not particularly my voice that says whether or not it the code is merged into the repository....
iirc Myrsloik had showed some dislike of the avisynth C interface and was slightly hesitant on adding the code in.

but yes, i do have a build that works on x86_64, as far as I've tested it... (which is fairly incomplete testing)

I did forget to mention one thing about this though:
the parts of ffms2 that use COM to interact with haali's splitter (the ogg and mpeg ps/ts splitters) are unable to be used...
as these parts depend on the Platform/Windows SDK and the related header files are not GCC compatible from what I've tried.
so that means that the versions of the plugin that are compiled via MinGW will not have this capability.

[TheFluff]

I won't merge the C-interface to trunk (at least not until myrsloik okays it) but if you want a svn branch on the official repository just give me the word and I'll hook you up.

[Niktesla]

Hi guys. Dark Shikari said it's just the right place for my bugreport here.

I'm was testing x264 r1471(32bit, GCC) recently (manually through command line) on my 300+ MB .mp4 anime file.
It was encoding just fine at my work (core Duo 6400).
And then i went home, and tryied to launch ALL THE SAME TEST on my Athlon X2 5200+ (Brisbahe 65nm). It started with ffms[info] indexing input file , ran up to 89.9% and crashed. This Crash is stable and reproducible(on my home system).
Both my home and at work comps are running win XP SP3.

While it's a standalone x264.exe that Chashes, Dark Shikari said  http://doom10.org/index.php?topic=254.0 that there's nothing they can do and recommended to post it here.

So here comes some detailes on my rare kind bug

techouse r1400 icc build  DO NOT cause Crash on my athlon X2 system...
r1471 with --no-asm still crashes...
Crash is [yet] known to be independent of command line options used, but here is a coupe of examples i used
 

x264.exe "sb14.mp4" -o"out14_def.mkv"
x264.exe --profile high --preset veryslow --tune animation --psnr --ssim --keyint 250 --min-keyint 25 --non-deterministic --crf 23 "sb14.mp4" -o"out14_veryslow_animation.mkv"

here's screenshot goes (message is in Russian, though helps to get a feel of this crash)


 dld link to "crash pack"  (.mp4 video file + x264 + couple bat files)
http://www.megaupload.com/?d=KNFILHXO

[Adub]

kemuri-_9,

Thank you so much for adding 64-bit support to the Avisynth plugin! It works wonderfully!

Now, I am putting forth a feature request for 64-bit ffmsindex. It would be great if this support could be added.

Thanks again!

[kemuri-_9]

kemuri-_9,

Thank you so much for adding 64-bit support to the Avisynth plugin! It works wonderfully!

Now, I am putting forth a feature request for 64-bit ffmsindex. It would be great if this support could be added.

Thanks again!

i was too lazy to link ffmsindex against the dll because kovensky's build system forces the ffms2 dll to be 'libffms2-2.dll' (i really do dislike this build system).
this requires some work to rename it to 'ffms2.dll' and then link ffmsindex against this renamed dll.
i don't particularly use ffmsindex so i never bothered originally, but since someone wants it, guess i can do it right quick...
ffmsindex is up alongside ffms2.dll now...

Edit:
and all the code for the avs C plugin interface has been committed to my branch. so it's available for personal compilation...
minus the part where the build system fucks up the pkg-config parts when i autoreconf it,
so i've left it unreconfed for now and so the build system does not understand how to integrate the avs C code (again why i hate this build system).

[jsday187]

Would anyone be able to provide a thorough and detailed guide to compiling an FFMS2 DLL on Windows please?
It would be much appreciated.
There info available seems to be for more experienced people.

Does anyone know of somewhere to get a build of FFMS2 r311?

Thanks guys.

[TheFluff]

If you're new to compiling things you probably shouldn't start with FFMS2; it's a rather complex project, mainly because it involves getting two different C++ compilers to cooperate.

A quick summary, regardless:
1) make sure you have visual studio 2008 (the free express edition works but with some caveats)
2) get prepackaged MinGW from here: http://www.cccp-project.net/wiki/index.php?title=Installing_MSYS-MinGW
3) compile ffmpeg in mingw (this isn't too hard and should be pretty easy to find tutorials for) and make install
4) get inttypes.h and stdint.h for MSVC from http://code.google.com/p/msinttypes/ , put them somewhere and add that somewhere to your
visual studio includes directories
5) add mingw's include folder and lib folder to visual studio's include and library directories respectively; make sure the mingw include folder is AFTER the inttypes one in the list or things will break
6) open the ffms2 solution file in visual studio
6a) if you're on visual studio express edition rightclick the ffms2 project in the solution explorer, hit properties, go to c/c++ -> preprocessor -> preprocessor definitions and remove the HAALISOURCE define in both debug and release configurations (if you're on any other version of visual studio 2008 this step isn't necessary)
7) hit build and pray

[qyot27]

For the past three months or so, I've noticed FFMS2 cutting audio about 3 or 4 seconds short.  It happens with MP4 files with AAC audio, whether libfaad is used or not (and the faad 2.7 executable from Rarewares decodes the entire sample without stopping short).  I'm not sure if it also happens with MP3 streams, or FLV files with AAC or MP3 audio.

The error that shows up is
"FFAudioSource: Out of bounds audio samples requested"

The output from wavi or WMP is simply that 'Could not read audio data' or 'an error has occurred' at the point where the audio gets cut short.

It doesn't happen if I remux to MKV, but it does happen if I extract with mp4box and mux them back into another MP4 with mp4box.  I do so to make sure it wasn't just a case of the file I was trying being corrupted.  In the case of the MP4s, the original and remuxed work fine when I grab the audio with DirectShowSource and restrict FFMS2 to just the video stream.

I make sure to use up-to-date builds, so it was the current revision back in February, and the tests I did with remuxing just now were with my own compile of r312.  The script was simply the wrapper function with the atrack=-1 parameter:

FFmpegSource2("test.mp4",atrack=-1)

[RiCON]

I wouldn't cut audio with FFMS2, at least for now. Not even using lossless input. See issue #10.

[qyot27]

Huh?  When I said 'cut' I didn't mean 'edit' - I mean it's muting the last three or four seconds of the audio stream.  But maybe this issue with lossless is connected and fixing it will fix this.

[Underground78]

Hello,

I have a silly question, where can I find builds of latest revision ? Are those builds the last compiled ?

Thanks

[JEEB]

Not like really much has changed from that r309 build TheFluff posted over at the Doom9 thread (other than that ffmpeg-mt fix and some timebase fixing), but I've built ffms2 for Avisynth for testing and learning purposes if you'd like to give it a swirl (linky). Tested with H.264 and W64 PCM audio, seemed to work rather fine. Does not contain ffmpeg-mt at the moment.

[TheFluff]

I'm planning on convincing Myrsloik to make a bugfix release Soon(tm) since there have been a few important bugfixes lately, both in ffms2 itself and in ffmpeg (mp4 demuxing, etc). I want to add color coefficient selection first, though.

[Underground78]

Not like really much has changed from that r309 build TheFluff posted over at the Doom9 thread (other than that ffmpeg-mt fix and some timebase fixing), but I've built ffms2 for Avisynth for testing and learning purposes if you'd like to give it a swirl (linky). Tested with H.264 and W64 PCM audio, seemed to work rather fine. Does not contain ffmpeg-mt at the moment.

Thanks I will have a look.

I'm planning on convincing Myrsloik to make a bugfix release Soon(tm) since there have been a few important bugfixes lately, both in ffms2 itself and in ffmpeg (mp4 demuxing, etc).

Ok, good to know !

[Owa]

Hey,

I have a question about ffms2x64. I can load the plugin without any problem, yet when i open my AVS script with e.g. VDUB (x64) or when I try to encode it with x264.exe (http://komisar.gin.by/) it tells me that FFVideoSource isnt a valid command (something along those lines).

Is it possible that anyone could shed some light on ffms2x64 for newbies? :}

[kemuri-_9]

sounds like you're trying to use the avisynth plugin i wrote for avisynth x86_64.
this has been mentioned in several places on doom9, but looking through this thread again, the information is missing here:
use LoadCPlugin instead of LoadPlugin

[Owa]

Thank you kind sir!

[CyberShadow]

Hi,

I'm experiencing some problems which might just as well originate in FFMpeg itself.

Using the binary in ffms2-r292.7z, I've had weird problems with indexes (.ffindex files). Sometimes files didn't play on the first time, when indexes are created, but would play on following attempts once the .ffindex files existed on disk. In other cases they would only play on the first time - if the .ffindex file existed on disk, they wouldn't play. I just shrugged at these since the workarounds were simple.

In one instance FFMS wouldn't play a file until I renamed it. Sounds crazy, but I've triple-checked this - the same identical file wouldn't play with one filename and would play with another. At this point I got suspicious and tried building FFMpeg + FFMS from source, but this particular problem vanished with my build. (I got the source from SVN, maybe it was fixed in the meantime?)

I also noticed that the .ffindex files differ every time they are generated. Do they contain any data not originating from the video file (timestamps, perhaps) by design or is this an indication of a bug (usage of uninitialized memory)?

[JEEB]

(I got the source from SVN, maybe it was fixed in the meantime?)

It most probably did, r292 is rather old.

I also noticed that the .ffindex files differ every time they are generated. Do they contain any data not originating from the video file (timestamps, perhaps) by design or is this an indication of a bug (usage of uninitialized memory)?

They might contain data on the audio streams etc., as well as there's a good chance that somewhere in between the format was a bit changed. Of course, if you think that there's a bug there somewhere, you could try pointing a fork at the source and seeing if it does anything weird.

[CyberShadow]

They might contain data on the audio streams etc., as well as there's a good chance that somewhere in between the format was a bit changed.

I don't see how that matters. To elaborate: the exact file played with the exact same AviSynth and FFMS version on the same machine creates different .ffindex files on two consecutive runs. So, I play the file, rename the .ffindex to something else, play it again, and get a different .ffindex file.

Of course, if you think that there's a bug there somewhere, you could try pointing a fork at the source and seeing if it does anything weird.

Sorry, whuh? 

[JEEB]

I don't see how that matters. To elaborate: the exact file played with the exact same AviSynth and FFMS version on the same machine creates different .ffindex files on two consecutive runs. So, I play the file, rename the .ffindex to something else, play it again, and get a different .ffindex file.

You didn't really specify, so that was mostly about the fact that with different settings different things will get indexed (ffaudiosource vs ffvideosource's default behaviour etc. + ffmsindex).

Sorry, whuh? 

I didn't make it clear enough, but I meant reading the actual indexing code and seeing if it writes any timestamps or does anything weird. Personally I haven't really taken any thorough looks at it, so I said it quite broadly (and with a weird metaphor). My bad.

[TheFluff]

r292 still has that invalid memory access bug that causes all kinds of odd shit. It should be fixed in SVN trunk, report back if you run into similar issues again.

Re: the index files, they should really be the same every time. They do contain the FFMS2 version as well as the version of the ffmpeg libraries it was linked at, but no data that changes every time. Does SVN trunk still do that?

[CyberShadow]

Does SVN trunk still do that?

I'm not on my main computer so can't check if SVN was updated since I posted about this a few days ago, but at the least SVN trunk from a few days ago still did this, yes. Let me know how I can help you debug this.

[TheFluff]

I can reproduce it myself with trunk so no need. I'm not sure if I should or need to do anything about it, though. As long as the index files are readable by the same version of ffms2 I don't really see the problem. The index files are constructed by passing various data structures into zlib and then writing the result to disk so I'm not entirely unsurprised by the fact that they differ between different runs of the code; the entire process is sort of unreliable to begin with anyway.

[GnG]

sounds like you're trying to use the avisynth plugin i wrote for avisynth x86_64.
this has been mentioned in several places on doom9, but looking through this thread again, the information is missing here:
use LoadCPlugin instead of LoadPlugin

Hi, a new guy here.

I just wanted to say that it took me quite a bit of time to set up a fully functional "mkv-to-lagarith" 64-bit toolchain. With VirtualDub 64, ffdshow 64, avisynth64, lagarith 64 and - last but not least - the ffms2.dll 64 bit build.

Not knowing this last bit of info (LoadCPlugin), being not-so-obvious for a freshman, really annoyed the hell out of me. It may be just me, but I just wasn't able to find a step-by-step guide for setting up a video editing workbench in a 64 bit environment (there always seemed to be some glitch involved).

Luckily it all seems to 'click' now. Thanks again for the great tools, and sorry for being off-topic

[TheRyuu]

ffms2-mt-r318.7z

Can probably be considered just a little less experimental now (in regards to the auto-initializing pthreads).

Included the ffmpeg-mt libs and some other stuff (opencore) all of which is probably useless (can be used to build x264 with ffmpeg support, suppose that's good if you can't build ffmpeg for whatever reason).

[H.Lime]

hi, yesterday I compiled r319 on Ubuntu 09.04, and afterwards, every encode ends up to a "*** glibc detected *** /home/daniel/x264/x264: double free or corruption (out): 0x0a45a9e0 ***" line instead of "encoded xxxx frames, xxxx fps, xxxx kb/s"

So I compiled the newest ffmpeg at this point (SVN-r24294) and than again ffms and then x264..
Resulting there was everytime an segmention fault at the end instead of *** glibc detected ***...

Today there is this *** glibc detected *** again, I paste the whole text, maybe this can help.

Before I had r312, and there this issue didnt't exist. Also with --demuxer lavf this doesen't happen.

Using gcc 4.4.1 that comes shipped with Ubuntu.

...
x264 [info]: ref B L0: 90.6%  7.6%  1.8%
x264 [info]: ref B L1: 95.8%  4.2%
x264 [info]: kb/s:337.77

*** glibc detected *** /home/daniel/x264/x264: double free or corruption (out): 0x0a45a9e0 ***
======= Backtrace: =========
/lib/tls/i686/cmov/libc.so.6[0x6430d1]
/lib/tls/i686/cmov/libc.so.6[0x6447d2]
/lib/tls/i686/cmov/libc.so.6(cfree+0x6d)[0x6478ad]
/home/daniel/x264/x264[0x840af05]
/home/daniel/x264/x264[0x80841b2]
/lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x5eeb56]
/home/daniel/x264/x264[0x80742a5]
======= Memory map: ========
00110000-0012c000 r-xp 00000000 08:05 81926      /lib/libgcc_s.so.1
0012c000-0012d000 r--p 0001b000 08:05 81926      /lib/libgcc_s.so.1
0012d000-0012e000 rw-p 0001c000 08:05 81926      /lib/libgcc_s.so.1
00166000-00177000 r-xp 00000000 08:05 504566     /usr/lib/libjack.so.0.0.28
00177000-00178000 r--p 00010000 08:05 504566     /usr/lib/libjack.so.0.0.28
00178000-0017a000 rw-p 00011000 08:05 504566     /usr/lib/libjack.so.0.0.28
0017a000-00182000 rw-p 00000000 00:00 0
0022b000-0022c000 r-xp 00000000 00:00 0          [vdso]
0022c000-00312000 r-xp 00000000 08:05 502232     /usr/lib/libstdc++.so.6.0.13
00312000-00316000 r--p 000e6000 08:05 502232     /usr/lib/libstdc++.so.6.0.13
00316000-00317000 rw-p 000ea000 08:05 502232     /usr/lib/libstdc++.so.6.0.13
00317000-0031e000 rw-p 00000000 00:00 0
00402000-00404000 r-xp 00000000 08:05 99065      /lib/tls/i686/cmov/libdl-2.10.1.so
00404000-00405000 r--p 00001000 08:05 99065      /lib/tls/i686/cmov/libdl-2.10.1.so
00405000-00406000 rw-p 00002000 08:05 99065      /lib/tls/i686/cmov/libdl-2.10.1.so
004c6000-004d7000 r-xp 00000000 08:05 504557     /usr/lib/libcelt.so.0.0.0
004d7000-004d8000 r--p 00010000 08:05 504557     /usr/lib/libcelt.so.0.0.0
004d8000-004d9000 rw-p 00011000 08:05 504557     /usr/lib/libcelt.so.0.0.0
004dc000-004f0000 r-xp 00000000 08:05 81986      /lib/libz.so.1.2.3.3
004f0000-004f1000 r--p 00013000 08:05 81986      /lib/libz.so.1.2.3.3
004f1000-004f2000 rw-p 00014000 08:05 81986      /lib/libz.so.1.2.3.3
005d8000-00716000 r-xp 00000000 08:05 99061      /lib/tls/i686/cmov/libc-2.10.1.so
00716000-00717000 ---p 0013e000 08:05 99061      /lib/tls/i686/cmov/libc-2.10.1.so
00717000-00719000 r--p 0013e000 08:05 99061      /lib/tls/i686/cmov/libc-2.10.1.so
00719000-0071a000 rw-p 00140000 08:05 99061      /lib/tls/i686/cmov/libc-2.10.1.so
0071a000-0071d000 rw-p 00000000 00:00 0
0071d000-00887000 r-xp 00000000 08:05 499830     /usr/lib/libsamplerate.so.0.1.7
00887000-00888000 r--p 00169000 08:05 499830     /usr/lib/libsamplerate.so.0.1.7
00888000-00889000 rw-p 0016a000 08:05 499830     /usr/lib/libsamplerate.so.0.1.7
0090e000-00923000 r-xp 00000000 08:05 99083      /lib/tls/i686/cmov/libpthread-2.10.1.so
00923000-00924000 r--p 00014000 08:05 99083      /lib/tls/i686/cmov/libpthread-2.10.1.so
00924000-00925000 rw-p 00015000 08:05 99083      /lib/tls/i686/cmov/libpthread-2.10.1.so
00925000-00927000 rw-p 00000000 00:00 0
0092d000-00951000 r-xp 00000000 08:05 99067      /lib/tls/i686/cmov/libm-2.10.1.so
00951000-00952000 r--p 00023000 08:05 99067      /lib/tls/i686/cmov/libm-2.10.1.so
00952000-00953000 rw-p 00024000 08:05 99067      /lib/tls/i686/cmov/libm-2.10.1.so
00c89000-00c90000 r-xp 00000000 08:05 99085      /lib/tls/i686/cmov/librt-2.10.1.so
00c90000-00c91000 r--p 00006000 08:05 99085      /lib/tls/i686/cmov/librt-2.10.1.so
00c91000-00c92000 rw-p 00007000 08:05 99085      /lib/tls/i686/cmov/librt-2.10.1.so
00cf4000-00d0f000 r-xp 00000000 08:05 82253      /lib/ld-2.10.1.so
00d0f000-00d10000 r--p 0001a000 08:05 82253      /lib/ld-2.10.1.so
00d10000-00d11000 rw-p 0001b000 08:05 82253      /lib/ld-2.10.1.so
00e27000-00ee9000 r-xp 00000000 08:05 501381     /usr/lib/libasound.so.2.0.0
00ee9000-00eed000 r--p 000c1000 08:05 501381     /usr/lib/libasound.so.2.0.0
00eed000-00eee000 rw-p 000c5000 08:05 501381     /usr/lib/libasound.so.2.0.0
08048000-0876c000 r-xp 00000000 08:06 1196873    /home/daniel/x264/x264
0876c000-0876d000 r--p 00723000 08:06 1196873    /home/daniel/x264/x264
0876d000-0877f000 rw-p 00724000 08:06 1196873    /home/daniel/x264/x264
0877f000-08cba000 rw-p 00000000 00:00 0
0a2d9000-0ae0b000 rw-p 00000000 00:00 0          [heap]
b2627000-b2628000 rw-p 00000000 00:00 0
b4747000-b4748000 ---p 00000000 00:00 0
b4748000-b4f48000 rw-p 00000000 00:00 0
b4f48000-b4f49000 ---p 00000000 00:00 0
b4f49000-b5749000 rw-p 00000000 00:00 0
b5749000-b574a000 ---p 00000000 00:00 0
b574a000-b5f4a000 rw-p 00000000 00:00 0
b5f4a000-b5f4b000 ---p 00000000 00:00 0
b5f4b000-b674b000 rw-p 00000000 00:00 0
b7700000-b7721000 rw-p 00000000 00:00 0
b7721000-b7800000 ---p 00000000 00:00 0
b78bc000-b78c0000 rw-p 00000000 00:00 0
b78d6000-b78d8000 rw-p 00000000 00:00 0
bfc07000-bfc1c000 rw-p 00000000 00:00 0          [stack]
Aborted