FFMS2 (The Fabulous FM Source 2)

[Myrsloik]

This thread shall begin much like one of my previous ones...

One day I got bored (again) and thought it would nice to have a plugin to open mkvs and similar things. That attempt failed miserably because it turns out that ffmpeg has just as much documentation as the average open source project.

But as you all know you can't let computers win. Ever.

This plugin should support decoding of all FFmpeg supported video formats with frame accurate seeking in avi, mkv and mp4 and possibly some others. Haali's parser is used for matroska since the ffmpeg one has serious issues and FFmpeg for everything else. Opening files is however a bit slow due to the required indexing.

All current downloads and the source can now be found at the Google code project. This also includes TheRyuu's most recent FFmpeg-mt compiles as well as the old 1.21 version. (list deprecated downloads to make it appear)

Google code project

Full changelog

[Myrsloik]

This is the full todo list for FFMS2. Vote on the features/fixes you want the most or suggest your own. Lots of stuff has been pruned and from the old list.

Todo

• Write a graphic indexing app using QT creator (it will look at least twice as good as DGIndex)
• Support uncompressed formats and formats that need codec properties to be set from the container values properly (added width and height reading for all formats, currently only supported in LAVF opened files but wav in mkv/ts is so uncommon I don't think anyone's noticed)
• Implement incremental indexing (this would be a pure API improvement and never show up in avisynth because of its limitations)
• Regression tests
• Add support for transport streams that do not start on a packet boundary with Haali's parser (I probably won't implement this but patches are welcome, it could still be useful to reduce the number of needed steps to open some files)
• Make wrappers for using the FFMS2 library in other applications such as virtualdub and premiere import plugins if there's any real interest
• Make all calls to SWScale and PP multithreaded
• Export metadata information in the API (needed for a GUI to be more user friendly)
• Figure out why h.264 in TS/PS decoding shows artifacts
• Implement raw mpeg 1/2/4 and h.264 parsers to support those input formats for more than linear reading
• Improve the horribly inefficient index format, currently a ~5MB index becomes 20kb after zipping

[JEEB]

• ...
• Write a graphic indexing app using QT creator (it will look at least twice as good as DGIndex)
• Regression tests
• Make wrappers for using the FFMS2 library in other applications such as virtualdub and premiere import plugins if there's any real interest
• Figure out why h.264 in TS/PS decoding shows artifacts
• ...

As someone rarely using the SWScale / PP stuff, I probably would vote for these, bolding the one I currently think as the most problematic thing for wide usage and italics meaning that it probably would make me less happy than the others.

[D404]

• Write a graphic indexing app using QT creator (it will look at least twice as good as DGIndex)
• Figure out why h.264 in TS/PS decoding shows artifacts
• Implement raw mpeg 1/2/4 and h.264 parsers to support those input formats for more than linear reading

My votes.

[Dark Shikari]

Vote for incremental indexing here; it would be a huge benefit once we add FFMS2 input to x264.

[Emess]

• Write a graphic indexing app using QT creator (it will look at least twice as good as DGIndex)
• Implement incremental indexing (this would be a pure API improvement and never show up in avisynth because of its limitations)
• Figure out why h.264 in TS/PS decoding shows artifacts
• Implement raw mpeg 1/2/4 and h.264 parsers to support those input formats for more than linear reading

My votes, I can attempt helping on the first one as well :S

[Thesis]

Would it be possible to implement AVC indexing support? DGAVCindex it's quite buggy.

Disregard if it has been asked before.

[checkers]

Incremental indexing for x264.exe tastiness

[Emess]

Would it be possible to implement AVC indexing support? DGAVCindex it's quite buggy.

Disregard if it has been asked before.

What? ffms2 indexes everything it can read, this includes AVC. The main reason it shouldn't be used for BD loading is that some m2ts files get funky blocking issues, and others have severe seeking and framecount issues. It's a problem with the m2ts container and how libavformat handles it, not AVC itself. ffms2 is essentially just a wrapper for ffmpeg's libraries, that is, libavformat for splitting, libavcodec for decoding, libswscale for scaling, and libavfilter for ffpp and other filtering things. Try loading an mkv and you should have no problem, however remuxing an m2ts to mkv still has issues IIRC.

[Thesis]

I was doing it wrong lol. Ok then I vote for incremental indexing for x264 sake.

[TheRyuu]

What? ffms2 indexes everything it can read, this includes AVC. The main reason it shouldn't be used for BD loading is that some m2ts files get funky blocking issues, and others have severe seeking and framecount issues. It's a problem with the m2ts container and how libavformat handles it, not AVC itself. ffms2 is essentially just a wrapper for ffmpeg's libraries, that is, libavformat for splitting, libavcodec for decoding, libswscale for scaling, and libavfilter for ffpp and other filtering things. Try loading an mkv and you should have no problem, however remuxing an m2ts to mkv still has issues IIRC.

Remuxing to mkv (should) fixes the blocking issues if and only if you're doing linear access to it.  Seeking will still produce issues (presumably because it's seeking to an I frame and not an IDR frame).

[Emess]

Yes, linear access as far as the blocking issues goes is fine, seeking makes it hurf though. The incorrect framecount seems to be there regardless, at least going by my tests.

[Mr VacBob]

It'd be nice if you could report any problems with libavformat on the ffmpeg bug tracker~

I promise to read them and then forget about it for a year.

[Zarxrax]

I'll vote for #6. It would be nice to have import plugins for Premiere (and also Sony Vegas)

[Myrsloik]

I tried to read the plugin API documentation for premiere and didn't understand it at all (apart from the fact that it has a shitload of weird backward compatibility stuff). Is there a more accessible explanation to it somewhere?

[Myrsloik]

It'd be nice if you could report any problems with libavformat on the ffmpeg bug tracker~

I promise to read them and then forget about it for a year.

That's what I/my minions do whenever it clearly is one. Nobody likes to write code so I try to offload it to other people.

[Zarxrax]

I tried to read the plugin API documentation for premiere and didn't understand it at all (apart from the fact that it has a shitload of weird backward compatibility stuff). Is there a more accessible explanation to it somewhere?

I don't think so, but it might be handy to look at the AviSynth import plugin as an example: http://sourceforge.net/projects/videoeditorskit/files/

[Myrsloik]

I don't think so, but it might be handy to look at the AviSynth import plugin as an example: http://sourceforge.net/projects/videoeditorskit/files/

Maybe it's better if someone who actually understands premiere/vegas does it... the obfuscation is just too much for me to bother unless I'm getting paid.

[Myrsloik]

This is an exclusive status update!

The remaining goals for 2.13 (in addition to the things already in the changelog) are to finish implementing index file compression. The test index file goes from ~9.2MB to ~0.75MB more or less for free. The other goal is to make avc in mkv work perfectly. This includes fixing the corruption on seeking when the video was remuxed from m2ts. (this definitely looks possible but we'll see what evil details show up...)

IVTC.org will also return soon. Maybe with the same old junk as before.

[nieko_nera]

is this kind of video not supported, or i dont know how to use it...? Here is sample http://www.multiupload.com/70E2R2HFAD

MediaInfo:

Code: [Select]

General
ID                               : 1
Complete name                    : 12-05_19-36-08_TV3_.ts
Format                           : MPEG-TS
File size                        : 13.6 MiB
Duration                         : 55s 536ms
Overall bit rate                 : 2 056 Kbps

Video
ID                               : 801 (0x321)
Menu ID                          : 8 (0x8)
Format                           : AVC
Format/Info                      : Advanced Video Codec
Format version                   : Version 2
Format profile                   : Main@L3.0
Format settings, CABAC           : Yes
Format settings, ReFrames        : 4 frames
Duration                         : 55s 460ms
Bit rate                         : 1 722 Kbps
Maximum bit rate                 : 2 598 Kbps
Width                            : 720 pixels
Height                           : 576 pixels
Display aspect ratio             : 4:3
Frame rate                       : 25.000 fps
Standard                         : PAL
Resolution                       : 24 bits
Colorimetry                      : 4:2:0
Scan type                        : Interlaced
Scan order                       : Top Field First
Bits/(Pixel*Frame)               : 0.166
Stream size                      : 11.4 MiB (84%)
Color primaries                  : BT.470-6 System B, BT.470-6 System G, BT.601-6 625, BT.1358 625, BT.1700 625 PAL, BT.1700 625 SECAM
Transfer characteristics         : BT.470-6 System B, BT.470-6 System G
Matrix coefficients              : BT.470-6 System B, BT.470-6 System G, BT.601-6 625, BT.1358 625, BT.1700 625 PAL, BT.1700 625 SECAM, IEC 61966-2-4 601

Audio
ID                               : 802 (0x322)
Menu ID                          : 8 (0x8)
Format                           : MPEG Audio
Format version                   : Version 1
Format profile                   : Layer 2
Duration                         : 55s 536ms
Bit rate mode                    : Constant
Bit rate                         : 192 Kbps
Maximum bit rate                 : 216 Kbps
Channel(s)                       : 2 channels
Sampling rate                    : 48.0 KHz
Resolution                       : 16 bits
Video delay                      : -1s 220ms
Stream size                      : 1.27 MiB (9%)
Language                         : Lithuanian

Menu
ID                               : 800 (0x320)
Menu ID                          : 8 (0x8)
List                             : 801 (0x321) (AVC) / 802 (0x322) (MPEG Audio, Lithuanian)
Maximum bit rate                 : 2814000
Duration                         : 55s 460ms
Frame rate                       : 25.000 fps

AVS script:

Code: [Select]

FFmpegSource2("12-05_19-36-08_TV3_.ts",atrack=-1,rffmode=1)
with:
rffmode=1 - duration 01:51, fps 25
rffmode=0 - duration 00:55, fps 49.858
rffmode=2 - duration 01:51, fps 20

Problem is that only with rffmode=1 i get correct frame rate, but video duration with this mode is doubled

[Myrsloik]

Maybe is you specify exactly what the problem is. What makes you think it isn't supported?
Actually it's listed under known issues. RTFM.

[Thesis]

The other goal is to make avc in mkv work perfectly. This includes fixing the corruption on seeking when the video was remuxed from m2ts. (this definitely looks possible but we'll see what evil details show up...)

Yay /o/
Looking forward to it.

[Emess]

The test index file goes from ~9.2MB to ~0.75MB more or less for free.

So the zlib stuff stopped being shit? About time~

The other goal is to make avc in mkv work perfectly. This includes fixing the corruption on seeking when the video was remuxed from m2ts. (this definitely looks possible but we'll see what evil details show up...)

ASDF THANK YOU. This has been a major source of irritation what with dss2 not working on remuxed blurays and the only other option being unthinkable.

[nieko_nera]

Maybe is you specify exactly what the problem is. What makes you think it isn't supported?

Thank you for reply. Problem is that only with rffmode=1 i get correct frame rate, but video duration with this mode is doubled

Actually it's listed under known issues. RTFM.

What this issue is exactly? Is this it: Haali's splitter requires transport streams to be cut at packet boundaries. Use TsRemux to fix the stream before trying to open it.
I tried fixing with TsRemux, but problem still exist

Thank you

[komisar]

Sorry for the stupid question, but whether there is a 64-bit version of the plugin?

[JEEB]

Sorry for the stupid question, but whether there is a 64-bit version of the plugin?

IIRC kemuri-_9 built a 64bit ffms2 on #x264 earlier.

<kemuri-_9>   lolololol, got ffms2 compiled for x64 w/avs support.... through using shared dlls for libav* and zlib
<kemuri-_9>   building ffmpeg to shared dlls was fairly easy, just need to spawn my msys environment on top of msvc's x64 env so it could use msvc's lib tool

Of course, shared dlls means lots of fun and mismatching dll versions can make it go boom randomly.

[Kovensky]

Of course, shared dlls means lots of fun and mismatching dll versions can make it go boom randomly.

Indeed, there's a rather big ass-thread on ffmpeg-devel about trying to solve that issue. They're debating it from a linux perspective though.

[skittle]

It would be very nice if FFMS2 could import a file list 

Code: [Select]

file1.vob >> list.txt
file2.vob >> list.txt

Code: [Select]

FFVideoSource("list.txt")

[kemuri-_9]

IIRC kemuri-_9 built a 64bit ffms2 on #x264 earlier. Of course, shared dlls means lots of fun and mismatching dll versions can make it go boom randomly.

the only way i seeing getting a 64bit version of the plugin w/o shared dlls currently would be to compile ffmpeg with ICL and use the generated static libs with ffms2.
i haven't yet tried this since
A) i don't particularly use ffms2
B) it would be a pain to edit the ffmpeg configure/makefile to use icl instead of mingw.

[Kovensky]

B) it would be a pain to edit the ffmpeg configure/makefile to use icl instead of mingw.

Does ffmpeg even compile with icl? I know it does with icc, but I heard that icl expects MSVC syntax and ffmpeg is C99...