Matt Jeppsen from FreshDV made the requested that I follow up last weeks encoder shootout with new settings where I attempt to closely match the settings of each application so the results could then be compared.  I took this to heart, though I didn’t attempt to necessarily identically match the settings verbatim.  Here is why – as I noted last time, the tools each obviously run a wide gamut of prices and as one would expect, they also vary a great deal in what they allow you to do to the video.  I didn’t necessarily want to intentionally cripple an application that had taken the time to either create or at least expose to the user a feature that dramatically helped improve the quality or speed of the encode.

So with that in mind, I have rerun my tests with each application.  I have targeted the same approximate data rate and screen resolution for each setting.  However, I have enabled some color/luma settings, specific deinterlace, and crop options where they exist.  Effectively, I created the settings I would ideally want to use for each application to get what isn’t necessarily the best results, but is the right mix of quality, speed of encode, and file size that I decided I like the best.  This may not be quite what Matt was hoping for, but I just don’t believe an apples to apples comparison of these is quite possible, especially having now run the tests twice.

One more note – I didn’t bother running Turbo.264 again in this test.  While its true you can make some custom settings in Turbo, as I began truly exploring this area, I found them to not particularly help improve anything.  At minimum I wanted to crop this content to remove the letterbox introduced in the edit process.  This prove impossible as far as I could tell, which was a bit disappointing.  With that in mind, I saw no benefit in rerunning identical results and therefore disqualified it from this round.  While I found this to be one of the fastest options possible before, the fact that the encode settings were so basic makes it less useful.

Speed Results

As we look at the results, here were the things that stood out for me  – all the times were closer to one another than the last test.  True, Compressor still took the longest, but it definitely closed the gap.  Both it and Episodes times fell and were more consistent this go around (notice neither had an extremely long run as they did before).  Visual Hub’s time did increase, which I expect, though not very much.

Let’s factor one more element into the time results which are not documented in the chart – setup time.  Very quickly I realized I needed to limit the number of attempts I made before deciding on the most optimized settings (in the past, I’ve literally spent whole days just tweaking the settings until I was satisfied).  Since I was under the gun to get these results in before a trip, I limited myself to 3 tests max per application.  Even in limiting to 3 goes, I found the time spent in trying settings (or in some case looking for the settings I wanted to try).  Despite having a huge amount of knobs and dials to play with, I was able to breeze through Episodes settings fairly quick.  I attribute this less to a clean interface (I’ve never accused EP of that!) and more with my experience with the application.  Interface-wise it has changed very little since it was originally launched and having used it quite a bit now, i can run through it very quickly.  The first time you try some of the more advanced settings, it can be quite intimidating, but once you know what to look for (and what to avoid) its fairly smooth sailing.

Visual Hub has very few advanced settings, so this also didnt take that long to run through and in fact it only took 2 tries to hit a file i was happy with.  Compressor however, did take longer to fully get setup.  I haven’t used Compressor a whole bunch professionally since it updated to 3.0 so i’m not sure if things moved since I last used it a great deal, or if the amount of time since I used it caused me to forget some things.  I also got confused by a screen bug I saw on my MacPro that I didn’t see on the MacBook Pro.  Once I cropped the image and set the new correct size and aspect ratio, I was getting a strange stretched version of the image that didn’t match my settings.  After struggling with it for awhile, I discovered that the exported file didn’t actually contain this, so I ignored it (again, this bug never appeared on the MBP).  All in all, I’d say I spent about 45 minutes on Compressor getting setting optimized while both Visual Hub and Episode Pro took about 20-25 minutes.  Most of this time is tweaking settings, as I only encoded 30 second clips for testing.  It was much easier to do this in Compressor and Episode Pro, which allowed me to set in and out points than in Visual Hub, where I actually made a quick reference file of the 30 seconds i wanted to test.

Quality Results

I’ve documented the results less here than I did before, but I did want to show the metal grate that highlighted the different results in the previous test.

In dealing with the interline twitter, I thought the new settings were definitely better in Episode Pro, slightly better in Compressor, and worse in Visual Hub (click through to see the full size screen shot.

In an overall quality comparison, I also thought Episode came out looking best, followed by Visual Hub and Compressor.  There is a sharpness to the image (both video and gfx) in the Episode Pro encode that the other two don’t have (again, check out the full screen to really see the difference).

Conclusions

As I more or less originally stated, if you want full control of the settings –   a high end application is worth the money.  Episode assuredly stands out, though it takes someone with experience and confidence in the settings they are applying to get the results they want.  Compressor, as an essentially free product (as its bundled with Final Cut Studio)  has a number or advanced settings, though the set up wasn’t as intuitive, nor the results as pleasing.  Visual Hub, despite its lack of control, still turned out a fairly decent product, given the price tag though it just could achieve quality i really wanted with the settings i needed.  As was just noted in a meeting I had the other day, a professional compressionist likes to have a lot of tools at his finger tips for encoding and while some may get used more than others, each has their place.

I was recently hassling a new friend(let’s call him Jeff) via email over some quality issues I was seeing in a video podcast he is publishing.  The issue isn’t really a super big deal, but I’m trying to help him resolve it so the production puts its best foot forward.  We are still trading emails about his overall workflow, as I don’t think its the encoder application he’s using, but some minor setting that is introducing the issue i noticed.  

In the course of discussing his problem however, I learned he has been using the El Gato Turbo264 1.3 ($99) to create his final encodes.  His reasoning for using it was primarily the speed bump he got over an export from Compressor (he is using a Macbook Pro for production).  El Gato uses its own proprietary USB hardware with a specialized DSP (Digital Signal Processing) chip to offload (and speed up) the encode process).  The Benefit as marketed by El Gato in a nutshell is to either a) speed up the encode on older machines or b) offload the encode process on newer machines to keep you working.  I had checked this product out briefly when it first came out, but it had been awhile and I wanted to see what both the quality and the encode times were like.  For a baseline, I compared it to the other encoders we discussed in our various e-mails – Apple’s Compressor 3.0.3 (included with Final Cut Studio), Telestream’s Episode Pro 5.0.1 ($999) , & Techspansion’s Visual Hub 1.3.4 ($23.32).  I was primarily just curious about the encode times, however I did want to at least marginally take quality of the encoded files into account, as who cares how fast the compression is if the quality is garbage.  

Methodology

I used the same file in all four applications with default settings or templates.  I encoded the clip to two different common file types – Apple TV and High Quality iPod/iPhone.  Since I have access to both a Mac Pro and a Macbook Pro, I encoded the same files on both systems.  My Macbook Pro is a 2.33 GHz dual core with 2 GB of RAM running 10.4.11.  My Mac Pro is a 2.66 GHz quad core with 5 GB of RAM running 10.5.4.  Each system had the same versions of the 4 encoding apps as fresh installs with default templates or default settings (VH doesn’t really have templates per se).  The clip used is a DV-NTSC clip from Zoom In Online that I also used in writing RWVC (this is the original posting at ZIO).  The clip is approximately 6 minutes long.  Although it is letterboxed in the source, I chose not to crop or customize the settings in anyway as I didn’t want to introduce any additionally processes into the encode pipeline.  I ran each encode separately with all other applications closed so nothing could take away from the encoding process.

The results after the jump!

Speed Results

Below is a table outlining my results.  I rounded each time to the nearest minute rather than present minutes & seconds (over 30 seconds I rounded up; under 30, I rounded down).

As you can see from the results, on the Macbook Pro, the Turbo264 was the clear winner, coming in at realtime for the encode for both versions.  Surprisingly however, Visual Hub came in second at fairly close to real time.  Both Episode and Compressor did about the same coming in at 2 to 4x depending on the encoder and setting.  Interestingly enough, each took a great deal longer on one encode, but not the same one (compressor struggled with the iPhone clip, while Episode choked on the Apple TV one).  Arguably, I can guess that both Episode & Compressor are doing multi-pass encodes by default, while Visual Hub is doing a single pass.  I’m not sure about Turbo as its offloading the encode process to the external DSP, but i’ll throw a guess that it’s single pass until i learn otherwise.  Since we’re only concerned with speed in this part of the test, i wont speak to whether going multi-pass actually benefits and makes the extra cycles worthwhile (we’ll compare the images further down to make that call).  

Moving to the Mac Pro, the Turbo’s times were almost identical (they were slightly faster) while Visual Hub’s nearly halved.  This makes sense as Turbo only knows to offload the processing to the same stick, while VH was taking advantage of the additional cores on the Mac Pro to encode faster.  This is the result I actually expected to get, so I wasn’t surprised.  I was surprised by the results of both Compressor and Episode again however as they did not drop as dramatically.  Infact with the exception of Compressor encoding for Apple TV, the encodes were only a minute or two faster on the 4 core Mac Pro.

Based on just these results, the Turbo would technically be the winner if you were on a laptop and encoding, but I’m not sure the extra $80 in price would be worth the few minutes saved, so I’d have to think long and hard before I picked.  If, however, you are on a Mac Pro, save the $ and run Visual Hub.  As for Episode & Compressor, both were considerably slower, so if speed was my only consideration, i’d probably leave them out.  That said, there is a wealth of other formats and options packed into both, especially Episode Pro, so while they disappointed speed wise, they should wholly be ignored or discarded.  

Image Results

Speed to finish is only one consideration though – quality is very important as well.  Note I didn’t say it was more important.  The balance between quality and speed is personal and often dependent on other factors in your project.  If its just a rough cut you are sharing with clients, quality probably doesn’t matter much, but speed does.  If however, this is the final encode to be used to author a DVD or present to a larger crowder, it may make sense to sacrifice speed for the quality of the end results.

Below are a series of screen shots I used to compare the images side by side of the Apple TV encodes made on my Mac Pro.  I specifically wanted samples that showed a typical headshot, live video mixed with graphic titles, graphics by themselves, and a still image being panned (an effected used repeatedly in the edit.  Quality wise off of just an eyeball comparison, i can tell you the encodes made on the Macbook Pro match the same settings on the Mac Pro (as I anticipated they would).  The finished results from encoder to encoder however are fairly different, both in file size and image quality.

Each clip is named after its encoder to make life easier.  To see the full size screenshots, click the image.  I guess the most immediate and obvious element is that all encoders but the Turbo264 maintained the current aspect ratio.  The Turbo dropped its output to 610 x 458. In the final screen shot i manually scaled up the Turbo image to match the others so i could compare the quality of the deinterlace of the mesh background (i’ll talk more about this in a minute).  

If you notice (probably easier if you click through to the original shot) the images are slightly different.  each of the encoders is handling the Pixel Aspect Ratio slightly differently.  From a computer screen, the Turbo clip plays and looks the best, though it is worth noting that when all four are loaded on a Apple TV (that is the target profile after all) they playback with no image distortion.  I thought Visual Hub actually did the worst of the four here for playback on a computer screen as its image looks very stretched.  As I mentioned though, it looked fine when played back through my Apple TV, and its ipod file looked fine on a computer screen, so i won’t complain too much.

It is hard to tell in the small pictures here, but if you look at the original shots of all three, you’ll see that both Compressor and Episode did a worse job of deinterlacing the clip than Turbo264 and Visual Hub.  This is perhaps most obvious in the final image which is a still image being panned.  In the image is a mesh background that introduced a slight interline twitter (a slight artifact brought about because of the fine grain of the original image and the encoder being unable to fully reproduce it).  Both Turbo264 and Visual Hub did a very good job of handling the fine mesh background.  I played all four clips at the same time and screen-capped them to further show this section, as the artifacting is more noticeable in motion.  Click here to play the movie in a new window.  Keep in mind that this is a re-encode of screen captured material that had already been encoded, so there is some quality loss, however the difference in the image playback is still obvious.

When viewed on a television via my Apple TV, both the image quality of the Visual Hub and Turbo264 clips were superior to the Episode Pro & Compressor clips.  Episode does have multiple de-interlace options and I know for a fact if i changed the settings I could have improved the quality of the final image, but this test was about default settings, so I left everything as is.

File size wise, the clips were again, quite different from each other, with the Turbo files generally being the largest across all files, while the Compressor files for Apple TV were twice as large as the nearest file.  This really reflects the difference in opinions on encode settings more than anything.  I’m not crazy about wasting bits unnecessarily, especially for files destined to be share on the Internet, so i tend to like the Visual Hub files the most here, as they consistently are the smallest.

Conclusion

In summary, for quick, fairly reliable “no thinking” encoding, both the Turbo.264 and Visual Hub are excellent choices.  If you are speed conscious and working on a Macbook Pro, the Turbo is an obvious choice, but if you are cost conscious or have already invested in a Mac Pro, go with the cheaper solution.  Episode Pro & Compressor can do a good job and have their place in productions, but you will most likely want to customize the settings you are using.  I don’t want to know either too hard here – they just weren’t the best tools for the job.  There are plenty of occasion when I’ve put both into duty to do functions that neither Turbo.264 nor Visual Hub were appropriate for (that’s right girls, you’re pretty in your own special way).