PapaJohn's Newsletter #18 - Movie Maker 2 and Photo Story 2 - Sept 11, 2004

Exploring: Audio - Enhancement, Generational Losses, Sync with Video, Conversion.

There are many posts are about audio and video tracks being out of sync. I ran into a sync issue the other day while working on the 'Mustang Sally' video for the reception page of the new wedding website - www.Jill-Mark.PapaJohn.org - I couldn't work it out, and had to finish the clip using the unenhanced audio track.

That got me thinking about and exploring some of the nooks and crannies of audio. I thought I'd share what I found with you in this newsletter.... some things worked and others didn't. We can learn from both.

The intro section will describe the sync issue that I ran into on 'Mustang Sally....and the mini-tutorial will explore some questions I explored as a result of it, and as I put together the 'Wedding Party Arriving' clip, the next one for the site. Here's a list of the assorted topics in the min-tutorial section:

• Audio Wave Patterns

• Wave Pattern Differences - DV-AVI Versus WMA

• MM1 Narration Files are One Channel

• WMA Generational Testing

• Plus! Audio Converter to Get an MP3 File

• Use TMPGEnc to get a WAV file

• Use iTunes to Enhance the Audio Track

• Does the Process of Ripping a WAV File using TMPGEnc, enhancing it with iTunes, and capturing back in MM2 as a WMA file result in any sync issue?

Before getting into it more, a few notes about some things going on...

Notices

• I find myself looking at what I'm currently working on, and taking one aspect of it as a newsletter topic.... that works well for me as it helps me explore the topic more fully, get some value-added for my current project, and hopefully makes an interesting newsletter for you.

• The Creative Zen is on its way back to Microsoft. It was a loaner to test and write about.... fun while it lasted!!!! I moved the associated blog page into the Distributing section of the website. Portable Media Centers will be a convenient way for many to share their movies so it'll stay on as a permanent page.

• On Tuesday I received an e-mail from Sliq MediaTechnologies. They told me about 50 people who downloaded their WMSnoop utility last weekend referenced the newsletter. They were appreciative, and open to comments about the utility.... so pass any along.

• My son handed me one of those mini-DVD discs from a Sony DCR-DVD200 camcorder that he's using, one of those that burn the video directly onto a disc in the camcorder. He said he'd like to know how to edit the video files in Movie Maker, and how to get them into the My Videos menu of the Media Center software. It's a good topic as more and more people purchase such camcorders and have the same questions. Now I have a disc to check the options.

....on to the topic of the week

The Audio Issue(s) on 'Mustang Sally'

Some Background Info

I took the wedding footage with two camcorders.... my older Sony Hi8 TRV-615 which records in analog with FM quality stereo sound, and a digital Sony TRV-80 with its digital recording and higher quality audio.

For 'Mustang Sally' I needed to use the continuous audio track from the digital tape, and mix the video from both camcorders.... and the audio/video sync was critical for the entire length of the video as there were lots of tight shots of the singer (he was a guest at the wedding, not a member of the band - or he wouldn't have gotten as much coverage in the video).

I dubbed the analog footage to the digital camcorder so I had both tapes coming into the computer via firewire as DV-AVI files.

Because the sound track was important for this video, I thought I'd rip it from the DV-AVI file, enhance it a bit in iTunes and bring it back into MM2, something I've done before a number of times.

I previewed it in MM2 and captured it as a WAV narration file in MM1 as it played.... I wanted a WAV file for iTunes. I listened to most of iTune's 22 default equalizer setting choices and, when I selected one I liked, I played the file in iTunes as I captured it back into MM2 as a WMA narration file. I'd done this before but not with such a sync-critical project.

Audio/Video Sync

During project editing I found myself wrestling with a significant audio/video sync issue. I'd align it in one place and find it off in another. I could only keep the alignment going for a short period on any of the clips..... and some of the clips weren't short enough to keep it in sync throughout.

When I put the enhanced WMA narration file over the audio of the DV-AVI file in the project, I saw a problem..... there was a steady drift out of sync between the two. By the end of the video the new enhanced WMA sound track was 2-1/2 seconds (about 75 frames) longer than the audio track of the DV-AVI file.

I ended up dropping the enhanced audio track completely, and using the un-enhanced track from the digital camcorder tape. But, while doing it, I decided to return to the topic and study the issue(s) further to learn what happened, and what can I do about it the next time.

Visual Patterns in MM2 to Aid in Syncing

..... wave patterns of the DV-AVI and WMA files were very hard to match up. I found myself tying to sync it mostly by listening to both tracks playing together in the project preview.

Where did it go off in Duration? Why? And why are the Wave Patterns so Different?

I didn't know. But the nature of the video was such that I needed great syncing throughout... so I tossed aside the new enhanced audio track and used the original unenhanced sound track to finish the video.

That sets the stage for this tutorial, exploring some audio items.

Mini-Tutorial: Exploring Some Audio Items

Here are the topics I explored and what I found:

Audio Wave Patterns

The differences in the visual wave patterns with DV-AVI files, WAV and WMA files are significant enough to make it too hard to study precise drifting causing of sync issues, and to use the different types of files in sync-critical situations.

For easy syncing in a project, use a WMV video file with the audio track ripped to a WMA file.... the duration of the files and the visuals of the wave patterns are identical. Here's a file with the patterns about 14 minutes into the project.

WMV and WMA Audio Tracks

Another easy way to work with syncing is to use two copies of the same DV-AVI file, one copy for the video/audio, and another for the Audio/Music track.

DV-AVI File - On Video and Audio/Music Tracks

Harder to work with are a DV-AVI file and a copy of the audio track ripped to a WMA file.

This WMA file was 00:00:08 less than the DV-AVI file it was ripped from. That's only .08 seconds in 9 minutes... it equates to about 2 frames. I could live with the slightly shorter audio track but the differences in wave patterns make it hard to sync video clips with the audio track.

Ripped WMA and the original DV-AVI

.... when playing them back individually, they sound the same. The wave patterns just look different.

Wave Pattern Differences - DV-AVI Versus WMA

The properties of the audio track of the DV-AVI file I as studying are 1024 kbps bit rate, 2 channels, 32 kHz sample rate with a sample size of 16.

The two properties that differ in the saved WMA file are the bit rate of 175 kbps, and the sample rate of 44.1 kHz.

I had picked the highest quality option when saving.... does a change in the quality setting vary the patterns? I'm looking for visual wave patterns that will align with the original DV-AVI file. The 160 kbps file looked the same as the 175 kbps one, and was the exact same duration.

I worked down the quality settings list - 128 kbps was next.... and was the same.... so was 96, 64 and 32 kbps. I don't want to go any lower in quality than that, not for wedding videos.

If the patterns didn't change with the bit rate, then maybe it's because the DV-AVI file has a 32 kHz sample rate versus the 44.1 of the WMA file..... I checked that by creating a custom WMA profile that uses the 32 kHz sample rate that aligns with the DV-AVI file. This one took a lot longer to render than the others and didn't provide the answer... the differences in visual patterns were still significant.

Custom Audio Profile at 32 KHz versus DV-AVI file

.... so using a ripped WMA file from a DV-AVI file wouldn't work for my sync needs.

Wave Pattern Differences - DV-AVI Versus WAV

WMA files are the usual for MM2, but MM1 captures narrations as WAV files, which I know iTunes can play and enhance.

I compared a captured narration WAV file from MM1 against the DV-AVI file. The visual wave patterns of the WAV file looked like the WMA files saved with MM2.... but not like the patterns of the DV-AVI file.

MM1 Narration Files are One Channel

The WAV file properties from the MM1 capture were 352 kbps bit rate, 1 channel, 22 kHz sample rate, 16 sample size. What happened to the second channel? Did MM1 capture in mono, expecting voice narration and not music?

I went back and did some more testing of the MM1 narration capture and found all of the WAV files to be single channel..... so it must assume voice narrations.... MM2 is capable of switching to higher level quality when it detects music playing during a narration capture and I guess MM1 isn't.

I started with 2 channel stereo sound and wanted to maintain it, so WAV files from MM1 won't get me there.

WMA Generational Testing

Last year I tested generational losses when saving WMV and DV-AVI files..... the WMV files lost the last frame of each generation, over and over until they stabilized with one frame left... it didn't lose that one. I wondered if saving a project as a WMA file in MM2 has similar losses. And it does.

This DV-AVI file that the audio was ripped from was 9:05:48. When saving just the audio to a WMA file, each generation lost a bit. Here's the lengths of each generation's WMA file:

1st generation save to WMA - 9:05:40

2nd generation - 9:05:30

3rd - 9:05:20

4th - 9:05:10

5th - 9:04:100

6th - 9:04:90

7th - 9:04:80

8th - 9:04:70

9th - 9:04:60

10th - 9:04:50

11th - 9:04:40 - for this one I added a still image to the project so it would save as a WMV and let me compare the wave patterns to see where the duration loss was occurring.

It was good to see that it wasn't a drifting situation. The loss was clearly and cleanly at the end of the file, just as the WMV generational losses were.

The wave patterns were still close enough to see that the full second had come off the end of the file.

11th Generation - One Second Less

Such WMA generational losses are not critical to syncing, or for usual project work. It's more of an interesting technical item.

Plus! Audio Converter to Get an MP3 File

Because MM1 narration files wouldn't work because they were mono, maybe converting the saved WMA file to an MP3 would provide a path to the iTunes equalizer.

Plus! Audio Converter

..... here's my error message. I don't have the needed MP3 codec on my laptop, which needs to be purchased.

Plus! Audio Converter - Error Message

If you have the Plus Pack and the codec, it's a path you can take to get your audio track to other software for enhancement.

Use TMPGEnc to get a WAV file

....from the original DV-AVI..... I'll use the settings of PCM format, 44,100 Khz, 16 bit, stereo, 172 kb/sec.

It took a couple minutes to get a 96 MB WAV file that was 9:05:48 in duration.... the same as the DV-AVI file.... no generational shrinkage. Now I have a file to play in iTunes and enhance with the equalizer settings.

Use iTunes to Enhance the Audio Track

Once I had a WAV file that iTunes can play, I used it's equalizer with the presets and tweaked it manually. The 'Wedding Party Arriving' clip had a hiss that I wanted to minimize.... I started with the Classical style preset and tweaked the sliders. Moving the 4K slider down removed the hiss.

iTunes Equalizer

I'm ready to play it in iTunes and capture it in MM2 as a WMA narration file, using my stereo mix option. It captured fine and I had my enhanced audio track.... and because syncing wasn't a concern, the rest of the project was normal and easy. The video is online with the enhanced audio.

Does the Process of Ripping a WAV File using TMPGEnc, enhancing it with iTunes, and capturing it back into MM2 as a WMA file result in any sync issue?

I couldn't easily tell from looking at the wave patterns... so I did a test. I added audio 'markers' at the beginning and end of the file, and at each full minute point.... the marker is a short sound effect clip from Sound Dogs.

I rendered the test clip as a DV-AVI file.... extracted the audio as a WAV file using TMPGEnc (File > Output to File > WAV).... then to iTunes to play it with the equalizer on and capture back in MM2 as a WMA file. I compared the starting and ending audio tracks.

The syncing of the markers at the start and at each full minute point looked exactly like this final marker.... perfect sync throughout.

Test File With Markers

I can enhance my audio tracks in iTunes and work with them in MM2 without the concern of introducing audio sync issues.... but the wave patterns of the WMA file are different than those of the DV-AVI audio track.... not an issue for this project but something to consider for many.

I never did find what caused the 2-1/2 second sync issue in 'Mustang Sally', but I'll keep my eyes open for it to happen again.

The rule of thumb to start with DV-AVI files and stay with that file type through the editing phases doesn't seem to help when it comes to wanting to rip off the audio track and enhance it..... maybe one of the readers will tell me what audio file type aligns with the wave patterns I see in MM2 when working with DV-AVI files.

It seems that, with the perfect alignment of WMV and WMA files, and the equalizer of the Windows Media Player, it would be lots easier for me to switch to using WMP instead of iTunes to enhance my audio tracks. This might be especially so now that WMP10 has been released.

If I leave the enhancement to the end of the project and have confidence there won't be any syncing issues associated with the process, I can edit with DV-AVI files right up to the last minute.... that seems like a good path to take the next time.

I hope that packing this newsletter with lots of individual items helps you more than confuses you....

Have a great week...

PapaJohn