Newsletter #130 - December 23, 2006

Make a Photo Story from a Movie

For this issue, I'm testing an idea... can high rez 1280x720 sized Photo Stories with high quality audio smoothly stream from the mydeo service... not just stories that start with a group of still pictures, but ones made from camcorder video footage... with the story playing at the same pace the camcorder shoots... 'real-time'

My first test was a camcorder tape of a concert last December at the Guggenheim museum in New York City. It was shot on my mini-DV camcorder in low light, with lots of audio snaps, crackles and pops until I settled down in position. This story was to test and validate the approach, not to make a story worth watching.

The image at the right is a link to the first draft... a story of 1280x720 pixels streaming from the mydeo service, not playing from a downloaded copy on your hard drive. It plays good enough for me to consider the method a keeper.

I was encouraged enough to explore it some more with a real project, our ongoing Europe 301.

Some of my 20 hours of camcorder footage from the trip to Europe this summer is well suited for it. I'm trying to immerse the viewers in the visual and audio of the scenes, and the higher quality widescreen view gets them a lot closer than the little monitor with a low rez video playing on YouTube.

For this story I used 16 minutes of footage from one of our daily ferry rides from the central bus station in Venice to St Marks. To see the story, use the picture link to our website's Venice page... then the same picture link to the story playing at a widescreen 852x480 size.

I'm making 3 versions... slow, medium and fast... this is the slow one, the full 13 minute trip in real time (boat moving on water time, not counting loading/unloading times at docking stations). For the medium one I'll drop every other picture to cut the overall playing time in half. And then delete every other picture of the medium one to get the fast one. For now, it's the local slow boat story.

Here's my menu for making a story from video footage

Video Footage to Story

Step 1 - capture footage as usual... I use Movie Maker to a single DV-AVI file, without auto clip splitting

Step 2 - put the single big clip on the timeline and trim the start and end points to suit. Split and trim the clip(s) as needed to discard or hide any unwanted footage in the middle too

Step 3 - drag the clip(s) from the video track to the audio/music track and save the movie... this will rip the audio to a high quality wma file... to use in step 8

Step 4 - drag the clip(s) back to the video track and apply a custom speed-up effect to each... making them play 149.67857 times faster than the original if you are using NTSC, and 75 time faster if PAL

Each frame of the saved movie will be one in the original video at 5 second intervals. It's easier to let Movie Maker simulate the frame grabbing than it is to sit there and manually do them at 5 second intervals. The 5 second duration is the default for pictures in Photo Story 3.

Here's a link to download the custom xml file

Step 5 - render the movie to a WMV file of your choice... the holiday greeting was to a widescreen 1280x720 and the ride on the Grand Canal to a widescreen 852x480

Step 6 - extract each frame from the WMV file to a folder of individual BMP image files... I use TMPGEnc.

Step 7 - for a standard aspect ratio story, you're all set to drag the folder of images into Photo Story 3...

for widescreen stories, distort the set of images first to a new height of 133% of the originals. I use IrfanView to run a batch process, leaving the width alone... the distortion is needed to compensate for Photo Story 3 only working in standard mode... saving the story using a custom widescreen profile will distort the images back so they will look right

The default picture duration of 5 seconds in Photo Story 3 aligns with the set of pictures... your story will play at the same real-time speed that your video was shot at

Step 8 - on the background music page of PS3, add the audio track that was saved in step 3... or one mixed with music or other audio off to the side

Step 9 - add text and adjust motion settings to suit... if you delete a picture and the audio/visual sync is important, adjust the durations of the adjacent pictures to compensate for the dropped 5 seconds

Step 10 - save the story and distribute to taste... if you're doing a widescreen one and need a profile, see my website's Photo Story 3 > Saving page

For added info about such things as why I saved the sped up movie to a WMV file for frame extractions and not DV-AVI, keep reading...

... before getting into more details, here are some notes...

Vista Corner... playing with various apps and taking screen shots to prepare for the writing of the MaximumPC article. I just mailed the first DVDs I made for a client with DVD Maker. It continues to work well at a slow burn speed.

This link provides a special 50% off the usual annual service price... good to Dec 31.... I'll keep it as a sticky note until then.

To compare streaming video from mydeo to the file downloading of YouTube... take a look at Chuck Bentley's holiday greeting... a subscription to mydeo makes a great holiday gift.

About forum posts...

My homework about Vista's DVD Maker, to write the new article for MaximumPC... includes online searching for related info. One growing trend I see is the automatic passing around of forum posts.

Everyone wants to run a forum full of posts they can wrap their advertising around. With today's software, they can use the posts from other forums while collecting the revenue streams from the clicks to their ads...

Here's a few of the places I see my posts about DVD Maker at... from a sampling of just the first few pages of a search

usenet.p2preactor.com

Seeing my posts suggests I change my practice of embedding links that might or will change... to fix a typo in a video title and replace them on Neptune, mydeo, or YouTube means I break the existing links, but the forum posts won't be updated with the new link. I'd typically edit my starting post, but I can't expect a change there to cascade thru the sites that used the original link.

The posts appear to viewers as if I made them on those forums, with responses that I'll never see... some places indicate the post is from 'outside'.

What I'll do as much as possible is make my post links to my website pages where I update the links. My website is the place for my most complete and latest info...

Maybe I'll get one of those forum software packages and fill it with extracts of my posts from other places, and wrap it in my own revenue streams... food for thought.

.... back to the main topic...

About the Video to Story Process

Here are some more 'behind the scenes' notes about each step...

Step 1 - capture footage as usual... for my first test, the concert, I captured the DV-AVI file using Adobe's Premiere Elements, and then imported it into MM2.1.

I captured the boat ride footage with Movie Maker to a DV-AVI file. My camcorder is a mini-DV one and the captures are using firewire to my laptop.

Step 2 - import the file as a single big clip, put it on the timeline and trim the start and end points to suit.

It would be time to also split the clip as needed to discard or hide by trimming any footage you didn't want to include... I didn't do that to either one.

Step 3 - drag the trimmed clip(s) from the video track to the audio/music track and save the movie... that will rip the audio track to a high quality wma file. You'll use it in step 8 as the story's background music.

Ripping the audio this way is a quick process... if you need a WAV file to edit the audio with a utility like Audacity, you can rip the audio from the original DV-AVI, or from the saved wma file using TMPGEnc.

Step 4 - drag the clip(s) back to the video track and apply a custom speed-up effect... making it play 149.67857 times as fast as the original if it's NTSC, or 75 times faster for PAL...

Here's a link to download the custom xml file I made. Put it in your Profiles\1033 folder under the main Photo Story 3 folder.

The speedup makes the rendered movie play at a frame rate that is equal to a single frame every 5 seconds.

You could go through the clip in the collection and take frame snapshots.... but it's easier to let Movie Maker do the extractions than it is to sit there and do a frame grab at 5 second intervals.

Photo Story 3 has a default picture duration of 5 seconds, so the story will play at 'real time'.

PS3 has a limit of 300 pictures, so you're in good shape for a story of up to 25 minutes.

Step 5 - render the movie to a WMV file of your choice... the holiday greeting was to a widescreen 1280x720 and the ride on the Grand Canal to a widescreen 852x480. See my website's Photo Story 3 > Saving page for downloadable custom profiles.

Additional renderings of the boat ride, mentioned below, were at 1280x720 pixels.

The concert story validated the process. For the Grand Canal ride, I saved the movie and story at 852x480 to align with the quality of the starting DV-AVI file... if the viewer wants to see it larger or at full screen, they can do it at their end. Seems a waste to bump up the image size too soon and spend the bandwidth streaming it, when the viewer can do it locally.

Save to DV-AVI? One issue is the preservation of interlacing, and another is the dropping 27th frame, which might be a 5 second syncing issue with the audio... needing the changing of the 27th frame to run 10 seconds... I looked at the new DV-AVI file in Virtual Dub and saw severe interlacing so I didn't even get into checking the 27th frame issue.

Saving to WMV automatically does a good job of deinterlacing... and drops only the last frame, not the 27th. The WMV option wins... so I saved it to high quality 4000 kbps WMV file.

Step 6 - extract each of the frames in the WMV file to a BMP image...

I used TMPGEnc. The pictures are the same size as the dimensions of the WMV file, already in widescreen aspect ratio of 16:9.

Step 7 - for a standard aspect ratio story, you're all set to drag the folder of images into Photo Story 3....

For widescreen stories, distort the images first. This distortion is needed because Photo Story 3 doesn't have a feature to support widescreen pictures or stories. It only works in standard mode, but with a custom profile for saving the story, you can get it to widescreen by doing the distortion first.

As my video was shot widescreen and I'm heading to widescreen stories, I used IrfanView to do a batch conversion/distortion, multiplying the height of each picture by 133%.... see the website's Photo Story 3 > Import Pictures page for more info.

Drag the folder of distorted pix into the opening window of Photos Story 3, and let it apply its defaults. It'll line them up in the sequence you had them in the folder, use a picture duration of 5 seconds, and apply a dissolve transition between each.

I checked Photo Story 3 to be sure the transition durations didn't change the overall duration... I put 12 pictures in, rendered a story, and got a 60 second file... the 5 seconds per picture average held steady.

Step 8 - on the background music page of PS3, add the audio track that was saved in step 3.

For the concert story I left the audio alone, the ugly artifacts and all. For the boat ride, I added some of Randon Myles' music, let the music over-power the ambient noises of the camcorder, and even mixed in some other audio clips of Venice. The church music you hear in the background as the boat passes the big church at the right toward the end of the ride was from.... yup, inside that church at another time.

Step 9 - add text and adjust motion settings to suit... if you delete a picture and the overall audio/visual sync is important, adjust the durations of some adjacent pictures to compensate for the dropped 5 seconds.

Be careful also to not lose the 5 second duration, something you can easily do by manually adjusting the zoom of a picture and letting Photo Story automatically re-adjust the duration.

Step 10 - save the story and distribute as you wish...

The holiday concert was saved using the custom profile for 1280x720, the size of High Definition 720p videos, and with a VBR audio quality setting of 98. The file size is 46 MB

The boat ride on the Grand Canal was saved to a widescreen size of 852x480 pixels... with an audio quality setting of 98. The file size is 44 MB.

The stories were uploaded to the mydeo service for streaming... the file you give it is the same one broadcast to viewers.

If you upload a story to YouTube, you'll get a flash file back... but re-sized to 320x240... if you don't use YouTube, you can put the story on a server that does file downloading, as long as it doesn't convert the story to another format.

I have 3 more test stories online on mydeo, which I'll delete about a week after the newsletter is issued... they are each rendered from the same boat ride project, but to 1280x720 sizes using 3 different custom profiles. I'm studying how well they stream with different audio quality settings, in an attempt to optimize the combination of smooth viewing and quality listening.

I'm having some difficulty understanding the file data... stats for the 3 test files reported by WMSnoop, differ from those reported by MM2.1 and WMP11. WMSnoop reports significantly higher total file bitrates.

Audio Quality 10 - file size of 44 MB... audio bitrate of 62 kbps

total bitrate of 1490 kbps (MM2.1 and WMP11 say 108 kbps)

Audio Quality 75 - file size of 56 MB... audio bitrate of 125 kbps

total bitrate of 1545 kbps (MM2.1 and WMP11 say 171 kbps)

Audio Quality 98 - file size of 63 MB... audio bitrate of 230 kbps

total bitrate of 1660 kbps (MM2.1 and WMP11 say 276 kbps)

Story file sizes are more due the audio stream than the video. For each of these 3 samples, the video bitrate reported by WMSnoop is 47 kbps...

But subtracting the audio and video stream bitrates from the totals reported by WMSnoop suggests stories are more about the overhead than either the audio or video streams.

On the other hand, MM2.1 and WMP11 reports very reasonable total bitrates... with plenty of room left over even when using the highest quality audio. Viewing experiences so far indicate that MM2.1 and WMP11 reports are right.

I'm seeing smooth streaming from mydeo at high def with high quality audio. I appreciate hearing from you about how they play on your systems.

Conclusion and Closing... and What's Next?

The concept worked!!!! going from video clips to an online high def streaming story...

I wrote Pixelan a while ago, suggesting that a great video effect would be one that takes my shaky video and un-shakes it. This story approach is essentially doing it, replacing wobbly videos with nice smooth pans and zooms. I can think of many cases where viewing a high rez quality story would be a much better viewing experience than watching the video footage it was made from.

There may be better or easier ways to do any or all of the steps. If so, I'd love to hear your suggestions. I'm sure there are video editing apps or utilities that can pump out frame snapshots at a specified interval, but I didn't find one in my software toolbox when I first rummaged through it.

I'm still studying the audio/video mix of a story... I'm familiar with the 'overhead' bitrate component, the 3rd component of a file... something of minor size for movies from Movie Maker. The total file bitrates reported by WMSnoop suggest the overhead component for a story might be a much larger factor... but that's not supported by the data from MM2.1 and WMP11, so I'm hoping that WMSnoop is off-base on the total bitrate.

I want to wish you and yours a wonderful holiday season and happy new year...

Have a great week...