PapaJohn's Newsletter #48 - April 9, 2005

Movie Maker 2 and Photo Story

Study DV-AVI File Rendering

This issue was planned to be about converting MPEG-2 files to AVI, but the AC/DC converter on the power chord of my Toshiba laptop stopped working last Thursday, and it's back to the CompUSA repair center in Texas. Until it returns, I'm back to using my older minimalist Dell Inspiron 3200... 266 Mhz CPU, 144 MB of RAM, and an almost full 4 GB hard drive. I needed a topic a bit less challenging on the laptop so I thought I'd revisit a couple issues about rendering movies to DV-AVI files.

It'll be two years ago next month that a couple of us did some testing of the generational losses we saw in both DV-AVI and WMV files... re-rendering a previously saved video would lose a frame or so. What frame and why?

During that testing I had pointed to the visual loss of the 27th frame of each DV-AVI clip on the timeline... one lost frame per clip when rendering to a new DV-AVI movie file.

The routine stream of posts from users noticing audio and visual issues with saved DV-AVI files continues... so I thought it would be a good time to revisit the subject, and this time look as closely at the audio track as the visual.

This will be one of those newsletters that some readers don't like, can't follow, or get confused about... rather than not read it at all, skip to the conclusions and closing section at any time to see what the bottom line is.

I'll start with a 15 second Photo Story... one that diagonally pans a still image at a constant speed that shows the change in position from one frame to the next. After studying the visual frames, I'll add a music track and study the audio in as much detail as the visual. Even this minimalist laptop can handle a 15 second DV-AVI file.

The lost 27th frame of the first clip is still there. SP2 and MM2.1 haven't resolved it. With the help of Photo Story 3, TMPGEnc, Excel, and Audacity, we can look closer at missing frame locations and dropped audio segments. When I looked with a magnifying glass a couple years ago, it's more like being able to check it today with a microscope.

... but before getting into it, here are some notes about current items...

Current Interesting Items

Ellen, a newsletter subscriber, sent an email pointing me to a great website... 2,650 downloadable midi-files of piano tunes free to use as background music in non-commercial stories and movies. I checked one pack and this piece in it:

That was enough to wet my appetite for the rest. It's midi-music but sounding 10 times better than the midi-music that I'm used to hearing. The website is

Remember those old 'player pianos' that automatically play a tune from a paper roll... Terry Shaw has been rounding up rolls from around the world, scanning them to get the paper cutout info converted to digital files, and then creating midi-files from them.

The midi file format isn't supported by Movie Maker, but plays fine in the Windows Media Player. My sample WMA file is a Movie Maker 2 narration file recorded as I played the midi file in WMP10. It's that easy to use one in a project.

The site also has a downloadable master index to all 2,650 tunes in an Excel spreadsheet. I've downloaded all the tunes and the index.

I was reading a book about Adobe Premiere Elements and surprised to see that it can only work with Digital Video source files... no analog sources with conversion devices... even analog to DV-AVI conversion devices won't work with it, such as using a digital camcorder as a pass-through device. Page 47 of the Visual Quick Start Guide for Premiere Elements says "... you'll have to use another program... such as Windows Movie Maker... to capture footage using this method (pass-thru)...".

On the other hand, the software is probably great. Similar to Premiere itself, you can use as many as 99 audio tracks and 99 video. You can do clip speed reversals, freeze a frame, link audio and video clips to retain sync, do sophisticated motion and transparency effects, apply 'rubber-band' volume controls to a clip, When saving a movie, you can opt to burn a DVD (it includes DVD menuing), save as a DV-AVI, export to a camcorder, export to MPEG format... or Quick Time (MOV), or WMV. Having used Photoshop/Photoshop Elements and Premiere for a number of years, I'm sure the package offers a lot for the cost.

I tried using the Windows Media File Editor to embed a website address in a video. It worked in a sample on my website... go to this page, scroll down to the embedded media player, enter HelloWorld.wmv above it (be careful as it's case-sensitive), tab a couple times to move the focus to the start button... and play the 22 second video.

At the end of the video the NASA World Wind website will automatically open. The website page includes info about how to embed the URL.

.... on to the main topic

To prepare for the assessment...

...I did a diagonal pan of a single large picture, using Photo Story 3 to make a high quality video clip... no audio, no starting transition, and accepting PS3's duration for the smooth pan - 15 seconds. I wanted something fairly plain, but with moving pictures and text, and a linearly smooth pan so I could see any breakdown in the smoothness from one frame to the next.

MM2 said the story file (WMV) was 14.93 seconds long. I wanted a DV-AVI file as a starting point, not a Photo Story, so I used MM2 to save it a DV-AVI file.

I imported the saved DV-AVI file right back into MM2... where in the collection bin MM2 said it was 14.88 seconds long... I'd been used to the Starting Clip

routine dropping of the 27th frame when making the DV-AVI file, so the shorter duration by 0.05 seconds was normal.

I dragged the DV-AVI clip from the collection to the timeline. The duration of the timeline per the monitor was 14.87 seconds, but the position of the end of the clip showed as 14.80 seconds.

With Movie Maker's preview mode being a rough draft 15 frames per second, this difference is probably not significant... just noted.

I was only getting ready for the serious studying... let's move on to our quality checking and see what we can learn.

First Quality Check - the Dropped 27th Frame

In this first generation rendering, using only one clip on the timeline, there should only be the dropped 27th frame... something to start the detailed studying with.

I used TMPGEnc to extract the first 50 frames to individual JPG files. When it reached the 50th frame in the saving process, I pressed the Stop button and ended up with 52 extracted frames.

Frame numbers start with 0 (zero), not 1... so the missing 27th would be the picture file #26... I flipped through the images in IrfanView, like you would a deck of cards. I didn't see any break in the visual flow when I flipped past the 27th frame.

Sub-Splitting the Single Clip

Let's go ahead and split the clip on the timeline into five 3 second sub-clips, do nothing more to it, and render it to a DV-AVI file... the timeline stayed the same during the clip splitting in the project... 14.87 seconds of duration showing under the monitor.

The newly rendered DV-AVI file shows as 14.67 seconds... down 1/5 of second in duration (5 frames at 30 frames per second is about 1/5 of a second). At this point I'm thinking the generational loss... the 27th frame of each clip... is doing it.

Splitting each of the sub-clips again... each into 3 parts... gets the project to 15 clips on the timeline. The saved DV-AVI file is now 14.53 seconds... going down.

One more round of splitting each clip into 3 parts... the project is now 45 clips on the timeline. Still 14.87 seconds on the timeline before saving, but now all the way down to 13.57 seconds after saving. We've dropped almost 1-1/2 seconds by splitting the clip into 45 sub-clips.... 1 frame per clip... with this approach the 27th frame generational loss issue seems to be a constant at 1 frame per clip.

But how does it effect the smoothness of playing? Is it still smooth, with the same beginning and ending frames, just playing quicker to get through it? What frames are missing?

I used TMPGEnc to make sets of images from each frame of the saved DV-AVI files. There were 446 frames from the full starting clip.... and 411 from the file rendered from 45 sub-clips... 35 fewer frames... but that wasn't 45 fewer, a chink in my long-standing point that it was dropping one frame per clip... it's not, or not unless TMPGEnc somehow sees frames differently than Movie Maker.

It's time to study the sets of frames closer. Believe it or not, time to open Excel.

Study Missing Frames/Images

I did a number of passes with the renderings and frame extractions, and put the lists of files into an Excel spreadsheet to help evaluate them.

The baseline file with no audio was a single 15 second clip, 446 extracted frames. When split into 15 clips on the timeline, there were 10 fewer extracted frames. I checked the data in Excel and noticed that the extracted frames each had a different file size, reasonable as I had opted for JPG images... comparing the files sizes of the two sets of frame images showed each to be exactly comparable... until... YES!!! the 27th frame wasn't there. The 28th frame and beyond were aligned to continue the pattern.

But I also saw that maybe I lucked out by seeing the first missing frame as the 27th... by continuing the comparison, and making a close but not exact comparison of where the missing frames were relative to the clips on the timeline, the 10 dropped frames were located as follows:

27th frame of the first clip (this was exact... the ones that follow are approximate locations)

last frame of the 2nd clip (because it's approximate, it could be the first frame of the 3rd clip)

last frame of the 3rd clip

last frame of the 6th clip

last frame of the 9th clip

20th frame of the 10th clip

27th frame of the 12th clip

last frame of the 13th clip

last frame of the 14th clip

last frame of the 15th clip

The missing ones were mostly not the 27th frame of each clip... and some clips had no missing frames.

Split into 45 clips on the timeline... 35 dropped frames... checked again in the spreadsheet, but I won't put the details here. The locations of the dropped frames didn't always align. In fact, the first dropped frame wasn't the 27th... it was the 29th.

Some users had reported that adding an effect, any effect, was enough to resolve audio issues, so I tried adding an effect to see what it did to the dropping frames.

A single clip with the mirror-horizontal effect applied had 1 dropped frame, the 27th. And the 15 clips with the mirror-horizontal effect applied to each of them also had 1 dropped frames, the 27th. Adding the effect to each clip on the timeline had effectively stopped further frame drops.

This newsletter can only be so long, and I want to get into the audio track also, so I'm stopping here with the visual issues... I'd be glad to email you a copy of the spreadsheet if you ask.

Study the Audio Track

This is my first comparable study of missing info on the audio track. I started with a new 15 second clip, this one with music, rendered to what I call the audio baseline clip of 15 seconds.

I split the baseline clip into 15 clips... at each of the full 1 second points of the timeline.

Here's a picture showing part of the baseline clip with music, split into 15 one-second clips. On the audio track below it is a copy of the baseline audio DV-AVI file. At this point the two audio tracks are in sync.

Pre-rendered 15 Clip Splitting... and Baseline Audio Track

And now another view. The same baseline clip is on the Audio/Music track, but I replaced split 15 sub-clips with the new DV-AVI file rendered from them, a single clip.

The rendering had resulted in the 10 dropped visual frames; looking at the audio shows that, what starts off being in sync drifts apart over the duration of the project... and the drifting apart starts at about the 27th frame.

Newly Rendered DV-AVI Clip versus the Baseline Audio

Let's zoom into the first second. I've marked the figure below to show where the 25th and 30th frames are. The sync is there exactly thru at least the first 25 frames. I've circled the spot where they begin to diverge... and, with my bias toward it going off as the 27th frame is dropped, it appears that's what's happening.

Close-up of the First Second

In this technical analysis, I'm trying to determine if it's a sudden shift of audio or a gradual loss of sync... it appears to be sudden, which correlates with the kinds of clicking or metallic sounds reported as you hit such points during playback.

The visual showed the loss of only one frame by the time the 32nd frame is reached. The audio shift looks more like 2 frames. End of File Sync

.. this assessment is close, but not exact. Each frame of a Movie Maker project is really a double-frame, so taking a magnifying glass to it can't get you to the single frame level.

The visual frame assessments showed a loss of 10 frames over the 15 second clip. The image at the right shows how much the audio is off by the end of the project... MM2 reports the new clip as 14:38 and the baseline clip as 14:85, a shortening of 0.47 seconds... 14 frames. Not exactly 10, but in the same ballpark.

Do the two files end on the same note or beat? Yes, they do.... so the shrinkage or loss is between the opening and closing notes. The audio isn't being trimmed at the end.

Is it a progressive movement out of sync? No, it's a sudden shift at certain points. We looked closely at the first one, at about the 27th frame of the first clip. The next shift occurs right after the end of the 2nd clip, aligned with the visual loss of the next frame. The visual then lost another frame right after the start of the 3rd frame... the next audio shift was in the same place. It's looking like segments of the audio are suddenly dropped at the points the visual frames are lost...

The audio waves in the Movie Maker timeline are like using a magnifying glass. Let's get out a microscope and look even closer.

An Even Closer Look at the Dropped Audio...

This is probably the most important part of this analysis... is there a segment of audio being dropped, and if so, by how much? Let's use the Audacity utility to check the end of the first second... where the first visual frame is dropped and some audio with it.

The upper two tracks of the next figure are the two channels of the stereo track from the baseline clip, before splitting into 15 smaller clips. The bottom two are the same tracks from the DV-AVI file rendered from the project which was split into 15 clips.

I used TMPGEnc to rip WAV files from the two DV-AVI files, and then put the imported WAV files into Audacity for this analysis.

This picture shows both files in full sync past the mid-point of the first second. I've zoomed into the Audacity timeline to view it in increments of 10,000ths of a second.

The next picture shows the beginning of the departure in wave patterns... with the wave patterns being minimal in some parts, I had to do a lot of close looking to detect this point.

The two sets of tracks were identical in patterns up to this point, at about the 27th frame.

With a sudden departure in patterns, how much is dropped? This next picture shows it.. a 0.065 second segment... 2 frames of a 29.97 frames per second video. The segments of the audio track shown in the upper two tracks are missing from the lower two.

Just one more picture and we're finished... showing the shift in the audio tracks due to the dropped 0.65 second segment. We've now measured the dropped audio pretty precisely... 0.065 seconds = 2 frames.

At this point in the timeline, the extracted visual frames show us dropping one visible frame, but the audio timeline shows the loss of 2 audio frames... the audio losses may actually be a more precise way of assessing it than comparing extracted visual frames.

That's enough for this newsletter... for those who have gotten this far, I hope it's been interesting, even if not particularly positive or helpful.

Conclusions and Closing

Some users report visual problems with saved DV-AVI files... others report audio issues. This analysis shows why there can be both. When the visual and audio tracks of a movie play in real time, the content can be such that missed information can be noticed or not noticed. It's a problem only if you can notice the loss of info... or the shifting out of sync ends up causing alignment issues.

What we can say is that, at the lower level, the rendered DV-AVI movie is different than the source files... slight or not, it is different.

We're building a pattern of lost visual frames aligning with lost audio frames. We can see the visuals in the extracted frame pictures. Can we hear the audio glitches? After this analysis, I rendered the shortened audio track to a CD quality WMA file and listened many times. I think I hear issues but it's hard to tell conclusively... maybe because I don't particularly care for the music I used.

the Bottom Line when heading toward a DVD:

For today it's: save your movies a couple or a few times: to DV-AVI for the best visual, to a WMV file for the best audio, and then marry the visual and audio tracks together into a final DV-AVI file rendered from one long clip. You'll have only one small hiccup at the 27th frame... within the first second of your opening. If there's blackness and quiet for the first second, the split-second hiccup won't be a problem.

What about audio/video sync issues? I ran across those in many places doing the wedding videos... you just have to manage them during editing. Sync the video and audio where it's most important... and resync as needed.

This is a small study... which shows the world isn't perfect yet. But it keeps trending in the right direction. Nothing I've learned this week detracts from being able to use Movie Maker to achieve amazing results, and I hope the information helps you learn a bit more about what's happening. It's certainly helping me.

Have a great week...