QUOTE (John Dolva @ Nov 28 2006, 09:10 AM)

Interesting, David, thank you for that. How I understand it then (please correct) is that the process creates frames that didn't exist in two ways. By mixing half interleavs from consecutive frames and by dupliucation. This would be the case with the dvd clip and removing dupes doesn't solve it. The avi clip on the other hand is not necessarily like that. It depends on how the frames for it were derived. The large spaced group around the headshot which Frank seems to have also found (Frank, could you mark the line that is from the headshot frame) I'm betting it is the same, if so then that indicates an intent as the pattern of missing frames is not at all regular?
John,
Yes, it sounds like you've got the basic pattern correct. If your source is film shot at a rate BELOW the rate of the target (i.e. 24 fps, 18.5 fps, etc), then something must be done to make the film appear to run at the correct speed when played on a video device. If you merely did a 1:1 frame transfer, the video would appear to run too quickly. (For example, if you watch footage shot in the early days of motion pictures, it frequently appears that everyone is moving a bit too quickly. This is because older cameras operated at a frame rate below 24 fps, but the people who produced the video telecined it as if the source was 24...)
The nature of the target video device also has something to say about how these frame-rate corrections must be made. If the target follows the PAL (European) standard, then the target is 25 fps. If NTSC, 29.97 fps (frequently noted as 30fps).
So, for example, if you're dealing with an NTSC target, you need 29.97 fps to appear to be running at 24fps. This is the telecining process, and it is accomplished by creating duplicates of each original frame in a 3:2 pattern. So, for movie frames 1-4, you would end up with video frames in the pattern: 111 22 333 44
The important thing to note is that at no point in the telecining process (there the source film is at a frame rate BELOW the target device) are frames lost. Copies are made.
The secondary process, which may or may not be necessary depending on the target, is interlacing. In the case of NTSC devices, the spec calls for 59.94 fields per second. This is comprised of "odd" and "even" frames. In essence, the device draws the odd numbered lines in the picture, then backs up and draws the even ones. Each half is drawn at 59.94, but it takes two fields to represent a frame... The effective rate becomes 29.97...
These numbers are not coincidence, by the way. The NTSC (~30fps/~60 fields/sec) standards were a by-product of North American power generation standards which have a line frequency of 60hz. The standard designers could use this frequency to help synchronize the field/frame events. Thus, standard TVs in North America draw a new field every (approx) 1/60th of a second - right in sync with the power standard. Also note that the European standard (PAL) is 25fps, which just so happens to be a multiple of their power generation frequency (50Hz)... For those that remember the days of TVs having a Vertical Hold knob, that if misadjusted would cause the picture to "roll", what you were doing when you turned that knob was 'fine tuning' the vertical synchronization of your TV to the broadcast -- necessary if there were slight differences in power frequency/phase.
Unlike television, the output from computers is progressive. This means that interleaving is not taking place. Each frame sent to the monitor is a full frame, not half. (Not to confuse matters, but some computer *monitors* actual interleave internally. However, the video card's output is progressive). When we start with a source that is interleaved and want to display it on a computer monitor, then the fun begins... David's article went into this in a fairly superficial way, but did explain the issue. If your computer contains DVD playback software, it contains an algorithm to deal with the ugly interlaced frames so they look decent on a computer monitor. The algorithms vary, but many use a technique known as "bob and weave". The key is to make the video look good at full speed. If you want to take interlaced video and display it as an AVI, for example, your computer will NOT automatically remove the interlacing effect. This requires de-interlacing the video. (This, too, is a subject with many opinions and methods. Everyone seems to have their favorite way to do this, and that's not necessarily a bad thing.) There are numerous tools that will do this to varying degrees of success. Once you've de-interlaced the video, if you want progressive frames, you then must un-do the telecining process. This is called "inverse telecine" (IVTC).
If the telecining process has been done correctly (and this is NOT always the case, by the way), one can assume which frames to remove. Many software packages accomplish this automatically, simply by assuming a perfect 3:2 pattern. However, it is NOT always flawless, and sometimes in the IVTC process, a frame will get lost here and there (and/or a duplicate frame might survive the process). In the course of a two-hour movie, or something that isn't being looked-at frame by frame, this is not a big deal. On the other hand, when we're attempting to analyze a film source and are hoping to get progressive frames, an incorrect IVTC process that loses a frame is unacceptable.
This is what I *think* has happened in the AVI that you're looking at. For my stuff, I did the IVTC process by hand. This made certain that I ended up with progressive frames. Obviously, for a large number of frames, this is an unworkable proposition.
I'll try to post some examples of frames, interlacing, and the rest if time permits...