YATTA

[Dogway]

Hello, I want to learn to use yatta. It's a thing I long wanted to do and now is the time.
I want to use it for 100% accurate IVTC, help myself identify blending patterns, dup patterns (to average together later on) etc

So I spent the last few days re-reading some of the basics, yatta manual and txt help file (all very few and quite niche). In the way I hope this thread can help other people too and break a taboo theme as yatta seems to be.

For my test I am using a very very easy soft telecine source so the workflow becomes easier to see.
40mb
http://www.mediafire.com/?qn57nfzqezs9n6n

1)
first I open ymc to collect metrics, I drop my d2v in the box and select:
tfm+telecide metrics->SCXvid->tdecimate
hit start, and eventually finishes.

2)
Open yatta
When I drop my created .yap file on yatta the next message appears:
"No default settings have been imported for this project. Import settings now?"
What does this mean?

I have the sections made and a row of "cccccc" pattern with one out of 5 underlined. What means the underlining?

3)
Ok... so now what?
some people say to use the "try pattern", others say the "set pattern", and some others the "set pattern guidance". I describe what happens when I click on each of them:

"try pattern": a pattern row is shown below the stats. it says ccnncccnncccnncccnn... (pattern test). It seems ok so I hit "use pattern", the preview to check the result. What I get are very random patterns, 3 progressive 2 interlaced, 2 progressive 2 interlaced, 3 progressive 1 interlaced, etc mixed down. Me don't understand.

"set pattern": I hit Ctrl+t and set the default pattern which seems right (ccnnc). Nothing seems changed in the main window. Hit preview. Same result as before, mixed patterns 3 prog 2 inter, etc... a mess

"set pattern guidance": Right click->pattern guidance->pattern guidance, hit ok to the warning and a log window comes up. It says about short sections detected further along the source, I don't care for now since I'm testing the basics in the beginning. Go back to main window, nothing has changed, so I click on preview: Same results as the 2 previous, a total mess of output.


I will probably have much more questions but for now to continue learning I'd to know if this program works, and if so, how?

[librarian]

Hi Dogway,
I don't know how to use Yatta, some time ago googling I found this guide http://translate.google.com/translate?sl=it&tl=en&js=n&prev=_t&hl=it&ie=UTF-8&layout=2&eotf=1&u=http%3A%2F%2Frecensubv2.forumfree.it%2F%3Ft%3D60475703&act=url (automatic english translation form italian text).
I hope it can be useful.

[Dogway]

Thanks for the input.

I will rephrase the revealing part on that guide:


1) "Right click> Set pattern. Matches cccnn, Decimation kkkkd, and lift up the check in date and do not ask ok. Right click> Switching> C / N only.Right click> Pattern Guidance> Guidance Pattern."

So I do that, go to set pattern, my default pattern of ccnnc is ok, so I hit OK.
Right click> Switching> C / N only.
Right click> Pattern Guidance> Pattern Guidance
I get the same log as described in the first post. aha

2) "go to frame 0 (yes, the beginning of words), scrubbate a little 'to understand what is the constant pattern of this first part, go back to zero frames, Click on the try pattern, the pattern that dates coincide, given the ok, go to the end of this part and clicked use patterns."

So I go to frame 0, hit "try pattern" and a new row of my pattern is shown below the stats, ccnncccnnc... Hit next section, go back to previous frame and hit "Use pattern"

3) Basically that's it. So I go to preview, and what I get now are... a fixed pattern of 2 progressive frames 2 interlaced frames. o_O
The guide recommends to go to frame 0 and use M just to check if a pattern was in use, if it moves a few frames only, then it's in use.
So I go to frame 0, hit M (in other words; go to next postprocessed frame) and it jumps to practically the end of my source  O_o

[Desbreko]

I'm not an expert with YATTA since I don't use it for much besides generating VFR scripts and other decimation overrides, but I'll share what I know.

2)
Open yatta
When I drop my created .yap file on yatta the next message appears:
"No default settings have been imported for this project. Import settings now?"
What does this mean?

I'm not entirely sure, but I assume it has to do with the Default Settings Project under the Global Settings. I guess you can save a template project and then the settings from it will be imported to new projects instead of what's normally used?

I have the sections made and a row of "cccccc" pattern with one out of 5 underlined. What means the underlining?

The underlined letters are the first frame in each cycle. You can see this change if you edit the cycle length in the Project Settings.

3)
Ok... so now what?
some people say to use the "try pattern", others say the "set pattern", and some others the "set pattern guidance". I describe what happens when I click on each of them:

"try pattern": a pattern row is shown below the stats. it says ccnncccnncccnncccnn... (pattern test). It seems ok so I hit "use pattern", the preview to check the result. What I get are very random patterns, 3 progressive 2 interlaced, 2 progressive 2 interlaced, 3 progressive 1 interlaced, etc mixed down. Me don't understand.

"set pattern": I hit Ctrl+t and set the default pattern which seems right (ccnnc). Nothing seems changed in the main window. Hit preview. Same result as before, mixed patterns 3 prog 2 inter, etc... a mess

"set pattern guidance": Right click->pattern guidance->pattern guidance, hit ok to the warning and a log window comes up. It says about short sections detected further along the source, I don't care for now since I'm testing the basics in the beginning. Go back to main window, nothing has changed, so I click on preview: Same results as the 2 previous, a total mess of output.

Everything is failing because the order is set to 1 when it should be 0. When you drop the .d2v into ymc and select TFM, hit the Configure button and set the order to 0. You should get a pattern of ccppc and no combed frames (except for a few at the beginning that look like they'd need to be post-processed or freezeframed) after calculating matches and opening the project in YATTA.

Now, assuming there was a section where the calculated field matches turned out wrong (unlikely with a soft-telecined source since the pattern is constant), you could go to set pattern and enter the correct pattern. (You can figure out the pattern by playing with the Switch button in a few cycles to find the matches that work.) Then, you'd seek to the first frame that you want to apply the pattern to (this should usually be the first frame in a cycle) and click Try Pattern. The frames displayed in the window will change to reflect the pattern you're trying, and if everything looks good, seek to the last frame you want it applied to (usually the last frame in a cycle) and click Use Pattern, or else click Try Pattern again to reset to the original matches.

If you don't want to keep having to go to Set Pattern every time, uncheck the Don't ask on "Try Pattern" box in the Set Pattern window, and then it will pop up automatically whenever you click Try Pattern.

I haven't used Pattern Guidance, so I can't offer any help with it.

[Dogway]

You are right, I had been using this source in avisynth with order=1 without any problems and was fooled by it. Probably TFM had some kind of autocorrection, I don't know. By reinspecting it back again I confirmed it's BFF. I had tested before with BFF on YATTA settings and ignored that needed to do the same on YMC.

So this is nice! finally I get something out of YATTA. Now when I hit preview I see the decimated result (as per decimate settings on the "Settings" dialog).
So in this simple case all I left to do, and I ask, is to load the fh.txt file to TFM in avisynth, like as follows?

Code: [Select]

TFM(order=0,slow=2,pp=0,input="VTS_01_3.demuxed.d2v.fh.txt")
It seems to work (same output than that of YATTA) but just to make sure.

Now, assuming there was a section where the calculated field matches turned out wrong (unlikely with a soft-telecined source since the pattern is constant), you could go to set pattern and enter the correct pattern. (You can figure out the pattern by playing with the Switch button in a few cycles to find the matches that work.) Then, you'd seek to the first frame that you want to apply the pattern to (this should usually be the first frame in a cycle) and click Try Pattern. The frames displayed in the window will change to reflect the pattern you're trying, and if everything looks good, seek to the last frame you want it applied to (usually the last frame in a cycle) and click Use Pattern, or else click Try Pattern again to reset to the original matches.

So I don't need to implicitly create a new section for a pattern change, or else the section is automatically created when changing the pattern?

Also is there any built in parity field switch detection? or this is something I need to fix previously in the d2v with for example dgindex?



Next, No-IVTC projects:

In the help 4 types of projects are described, in the latest yatta version I think they are reduced to 2 types. Type 0, for the things I want to additionally do; detect blends, detect dups (freezeframing basically), and type 1, for field matching and vfr.

What I normally do in avspmod is, create a trim for the section to inspect and start frameskipping and marking with ctrl+b the related frames (blends for example). With this I get a visual feedback of a pattern until it becomes a loop, then I can check its integrity and write down the pattern.
This is quite easy, with this I can then use selectevery's for custom filtering, the downside is indeed sectioning. This pattern can get off eventually or in scenecuts. And this is hard to correct.
My question is whether YATTA can help me in this process.
There is sectioning which is good, and the Switch button for correcting switchs in patterns, but there's nothing as marking. My intention is not to freezeframe nor decimate.
I thought on the post-process button, but it's disabled on project type 0.
Also there's no place to see the whole pattern so you can later take it as reference for custom filtering on sections (The txt's are just a long framenumbers column). YATTA feels more towards processing through patterns than patterns themselves.
My main goal is to obtain an .avs with selectevery's for each pattern section, where I do the filtering

Code: [Select]

a=trim(0,120)
a=a.filter().selectevery(5,0,2)

b=trim(121,200)
b=b.filter().selectevery(5,3,4)

...

a+b+...

[Desbreko]

So in this simple case all I left to do, and I ask, is to load the fh.txt file to TFM in avisynth, like as follows?

No, the .fh.txt file is an override file for FieldHint, not TFM. The syntax is different and I'm surprised nothing borked when you tried it.

To see what YATTA is actually doing, click the Tools button and then click Preview AVS on the AVS Generation tab. You'll get something like this for the field matching and post-processing part:

Code: [Select]

FieldHint(ovr="C:\Temporary\VTS_01_3.demuxed.d2v.fh.txt")
TelecideHints(NNEDI3())

So I don't need to implicitly create a new section for a pattern change, or else the section is automatically created when changing the pattern?

Also is there any built in parity field switch detection? or this is something I need to fix previously in the d2v with for example dgindex?

Right, you can apply pattern changes without creating sections, and doing so won't create new sections, either. I believe sections are required for applying pattern guidance to specific parts of the video, though. Like, say there's a scene cut where the pattern changes; I think you'd need to start a new section at the cut and apply pattern guidance to each scene separately.

I don't know about detection of field order changes, but I'd guess that's probably best left to DGIndex.


As for the rest, I've not really used YATTA for sectioned filtering other than just tinkering some, so I'm not sure if it could make that easier. (I just use AvsPmod, too.) It seems like YATTA's scene cut detection and ability to easily create and move section boundaries around would be nice, though. I'd look into how its filter presets work.

[Dogway]

Thanks for all the help. So I guess FieldHint and TelecideHints are mandatory for piping YATTA data to avisynth then?

I'm trying to build a base for a possible solution to shifting blendings, the type of blendings that after an amount of frames start with same weight and blending pattern. And also those scanning blendings (half frame/field blending) with a phase shifting. That's why I feel necessary first to study the best tools to speed up patterns handling.

[Desbreko]

Unless someone's written a script to convert from a FieldHint override file to a TFM override file (I think this would be possible, though kind of a pain), then yeah, FieldHint is required for using field matches from YATTA, and TelecideHints for using its post-processing matches.

[Chibi Goku]

Hi Dogway,
I don't know how to use Yatta, some time ago googling I found this guide http://translate.google.com/translate?sl=it&tl=en&js=n&prev=_t&hl=it&ie=UTF-8&layout=2&eotf=1&u=http%3A%2F%2Frecensubv2.forumfree.it%2F%3Ft%3D60475703&act=url (automatic english translation form italian text).
I hope it can be useful.

That guide is quite old (it was initially made 13 months ago, and the original is here) and is mainly written for TS filtering (that means there are entire sections only useful if you're encoding from TS and you're a perfectionist), that combined with the fact that machine translation sucks means you should take what's written there with a grain of salt.
I'd say that TheRyuu's guide is a much better start in pretty much every case.

Now, for the matter at hand I'd say:
- If you change order in Tdecimate you should change Field accordingly and work with C and N frames (it's just much more natural in YATTA)

Now, assuming there was a section where the calculated field matches turned out wrong (unlikely with a soft-telecined source since the pattern is constant), you could go to set pattern and enter the correct pattern. (You can figure out the pattern by playing with the Switch button in a few cycles to find the matches that work.) Then, you'd seek to the first frame that you want to apply the pattern to (this should usually be the first frame in a cycle) and click Try Pattern. The frames displayed in the window will change to reflect the pattern you're trying, and if everything looks good, seek to the last frame you want it applied to (usually the last frame in a cycle) and click Use Pattern, or else click Try Pattern again to reset to the original matches.

If you're switching the pattern in the section (and you should use CTRL+S to do so because using YATTA with no shortcuts is suffering) why would you use try pattern after that?
Pattern should be constant throughout the section, after you find the correct one by CTRL+Sing you're done.
If if still fails that means there is a probable pattern change and should be treated accordingly.

- If you're on a type 0 project you can use Custom List to mark the frame, just create a "blend" custom list, a "dup" custom list, then when you're on the video you use z and c to scroll the custom list entries, c to select the first frame of the custom list section and c to select the last frame (you will probably always press c two times in a row).
Not sure if that is going to make your life easier, but it's worth a try, I guess.
And anyway, Custom List just rocks and it's a pity no one really uses them, they are a pretty good replacement to custom sections (not being bound by sections makes them easier and faster to use in my opinion).

[Desbreko]

Oh right, I forgot about Ctrl+S for shifting entire sections at once.

Mostly what I've used YATTA's field matching features for is fixing scene transitions with double hard telecine where scene cut detection can't tell the optimal break point, and it's easier to just override the few bad cycles with try pattern instead of moving the section break and then shifting the entire sections.

[killazys]

http://pastebin.com/nPfqyy61

The YATTA Manifesto basically explains everything. For progressive projects you should ConvertToRGB(interlaced=false) in the script you feed to YMC/YATTA, because by default YATTA treats all sources as interlaced.

[joewoods]

No, the .fh.txt file is an override file for FieldHint, not TFM. The syntax is different and I'm surprised nothing borked when you tried it.

To see what YATTA is actually doing, click the Tools button and then click Preview AVS on the AVS Generation tab. You'll get something like this for the field matching and post-processing part:

Code: [Select]

FieldHint(ovr="C:\Temporary\VTS_01_3.demuxed.d2v.fh.txt")
TelecideHints(NNEDI3())
Right, you can apply pattern changes without creating sections, and doing so won't create new sections, either. I believe sections are required for applying pattern guidance to specific parts of the video, though. Like, say there's a scene cut where the pattern changes; I think you'd need to start a new section at the cut and apply pattern guidance to each scene separately.

I don't know about detection of field order changes, but I'd guess that's probably best left to DGIndex.


As for the rest, I've not really used YATTA for sectioned filtering other than just tinkering some, so I'm not sure if it could make that easier. (I just use AvsPmod, too.) It seems like YATTA's scene cut detection and ability to easily create and move section boundaries around would be nice, though. I'd look into how its filter presets work.

Going with the pattern changes and is going well!!!

[Dogway]

Oh great, finally I had a source (hard telecine) and some time to test this on.
With the knowledge thrown in this thread and after rereading the manifesto I got my first IVTC done through YATTA.
It works wonders! I made a sidebyside comparison against my old workflow of simple tfm+tdecimate and is so much better!

Basically it boils down to sectioning properly (so fixing sections first if necessary) and shifting the pattern with ctrl+s (it doesn't affect outside section), eventually you could need auto pattern guidance ctrl+alt+g or doing it manually (Ctrl+T) for a given section always.

I don't even use Try pattern at all, but I don't know. I know this is much simplification, but having things working first are all I need to progress on.

One question still arises, what is TelecideHints() for?

[Reel.Deal]

Hi Dogway, Do you mind posting a brief side by side sample of your old workflow and YATTA?
Thanks in advance.

[Dogway]

It's just that leaves less combed (and wrong field matched) frames, and makes better decimate decisions. It just feels good knowing you are doing it the right way, and restoring the animation how it was supposed to be seen.

[mikeyakame]

TelecideHints is just a conditional filter that reads the Decomb specific hinted bits of the frames Y Plane that are used to flag Progressive/Not-Progressive and In-Pattern/Out-of-Pattern.

FieldHint (being based off Decombs Telecide) reads the ovr file and looks up the value after the field order. If it is "-" then the hint bit for Progressive is set, otherwise "+" leaves it unset or Non-Progressive. The post-process flag in yatta determines whether this is true or not, which you can set manually, through recalculation at vmetric threshold, or via the metric collector analysis of tfm/tdecimate combed/interlaced frame decisions.

TelecideHints follows immediately to read the hint bits that FieldHint wrote and conditionally selects which clip to take the current frame from, whether it is progressive or combed/interlaced. It's a typical conditional filter; test condition, if true then a, if false then b.

Code: [Select]

FieldHint("some_ovr.txt")
TelecideHints(some_deinterlacer())

is basically

FieldHint("some_ovr.txt")
TelecideHints(last, last.some_deinterlacer(),last,last)

Get Hints, if Progressive Bit is set, take current frame from first clip, if not set take current frame from the second clip, which has been deinterlaced by some function/filter.
You are not limited to the scope of what Yatta can use, and if you want can use QTGMC, a custom function, NNEDI3 or any deinterlacer with non-default arguments, etc.

The tl;dr is it's just a dumb function to select a frame from either clip1 or clip2 when Decomb specific Hinting is present and used to decide whether deinterlacing needs to be applied post re-assembling the fields into their original progressive order.

[Reel.Deal]

It just feels good knowing you are doing it the right way, and restoring the animation how it was supposed to be seen.

I completely agree. I'm going to have to learn how to use YATTA now. Thanks Dogway.