Archive for the ‘the technical solution’ Category

Review – Sony HDR-SR7 AVCHD Hi-Def Camcorder

Recently my old trusty Canon MiniDV camcorder started breaking down, and I decided it was time to upgrade to a new camcorder. Anybody looking at buying a new camcorder has to first answer one big question: do I upgrade to HD or not?

The HiDef camcorders still cost a big premium over SD. But in the end I decided the “future proof” appeal of the HD camcorder was worth it. Online reviews seemed to indicate that the Sony was the best choice.

There are a lot of posts on the net about how poor support for the AVCHD format used by the new HD cameras hampers their utility. Well, I’m happy to report that the situation seems to have gotten much better. The Sony camera comes with software that allows easy playback of the AVCHD files on my PC, and includes simple transcoding of those files to MPEG2.

The Camcorder

The camcorder itself is great. Small, light, easy to hold. The touch screen interface is pretty awesome. It’s easy to use. I haven’t stressed battery life yet. The camcorder records to an internal hard drive, no tape needed! This really is a great advance. No more spinning through tapes to find stuff. The video you record ends up in clips that you can easily review/delete right there on the camera. It actually works just like a digital still camera.

Connecting to the Computer

Lots of people have complained that the camcorder doesn’t include a USB port on the unit itself. Instead you get a dock which connects to the computer, and you place the camcorder into the dock. I agree it’s kind of silly, but it’s not really a big issue.


The camorder picture quality is awesome. I’ve only really watched video on the computer, but the hi-def is super crisp. I’ve seen complaints about poor low-light performance. Performance in low-light isn’t great, but it’s not much different than my old Canon SD. So personally I can’t complain.


Basically all of my video is of my kids. As any parent can tell you, that means new video every week! So my motivation is not to craft the next Citizen Kane, but rather to get as much video processed as quickly as possible. In my case “processed” just means off the camera and into 3-5 min clips I can share with people. Now this is where this hard-drive camcorder really shines. Getting the raw video onto my computer takes nothing more than plugging the camcorder in and downloading the raw mts files. No more “grabbing” from hour-long DV tapes. Whoo hoo!!! Getting video onto the computer now takes just seconds.

Once I’ve got the raw files, I’m using Sony Vegas for very simple editing. I just drag clips onto the timeline, then say “Render” and generate my video file. Rendering still takes a long time (like running time * 5), but at least that process is totally automatic.


Fantastic. I love the camcorder, and I love the workflow. The best part is actually the hard drive recording into a compressed format. This makes a huge difference in the time you spend editing the video, because you’re just dealing with much smaller files, and no tape. The HD part is cool, but I could probably live fine with SD.


Open source transcoding – part 2

Ok, now I've got an even better idea! What the world doesn't need is for me to dribble out bits of ffmpeg/mplayer usage to this blog, forcing you to waste a bunch of time with my ramblings trying to get the info you need.

What we do need is a place to store the collective wisdom on using these open source tools for manipulating video for the web (that is, automated video manipulation at the server, rather than through a client app).

So….I've created a new Wiki to gather this info. Please check it out:

Open source transcoding – mplayer and ffmpeg – part 1

So I've been playing with open source encoding tools as of late. It's tempting to attempt to use these tools for video work because they seem so powerful and flexible, not to mention free.

The biggest problem is just figuring out how to use the tools to accomplish what you want. So, I'll be recording some of my experiences here.

The defacto standards in this area are as follows:

  • ffmpeg – this is the core codec library that provides support for encoding/decoding video in mpeg, wmv, quicktime, flv and many other formats. Basically all the other tools use this library for encode/decode.
  • mplayer/mencoder – these utilities build on ffmpeg, but also include a host of other codecs and filters.
  • vlan – again, I believe this guy relies on ffmpeg for any non-standard codecs.

Generally speaking ffmpeg is a video file manipulation library and app, while mplayer/vlan are end-user video player application, and mencoder is an end-user encoding application.

These tools overlap in their capabilities, and yet have wildly different usage options. Some of the core challenges with these tools are:

  • Getting a build for your platform. Mplayer has some pre-built binaries, but generally these tools expect you to build them from source. Especially on non-Linux platforms this can be an incredible pain in the ass.
  • Figuring out the command line options! These apps are wonderfully powerful, and wolefully documented.

I'll be adding posts on my experiences using these and other tools, with the hope of aggregating some common wisdom on using these tools to produce video for the web.

Review: On2 Video Publisher

On2 Logo

As promised, here is a quick review of the On2 Publisher control
which competes directly with VideoEgg (see my review of VideoEgg). As advertised, the On2 control is a very quick ActiveX control install, with no downloadble .exe. The install was quick and painless on IE (FF is supported, but only 1.5). Overall the control is less attractive and less slick than the VideoEgg one, but it does the basic job.

I was able to drag a wmv video file from my computer onto the control. It lets me play the file, set Mark In/Mark Out, and then click ‘Publish’. When I click publish then it encodes the file to Flash and uploads its to the On2 test server. Everything went smoothly and you can see the results here. I used the same video so you can compare to the one from VideoEgg (here). Encoding results seem similarly good from both products.

I don’t really like how On2 sticks their logo right on top of the video – the VideoEgg widget has the logo off the video which seems nicer. Presumably you can change this if you purchase the On2 Publisher control for use on your site.

The On2 product seems less polished than VideoEgg, but it does the job. And given that I do believe the download/install is a big user barrier, the easier to install ActiveX control is an advantage. It seems that maybe VideoEgg will try to be more of an outsourced service, where they will host/serve the video and provide the control. This will probably make it easier for smaller sites to add video hosting. I’m not sure how or how much VideoEgg will charge for this service.

The On2 product seems targeted at integration with your own video hosting solution, which may make it better for larger sites.

What is VideoEgg?

VideoEgg logo
VideoEgg is a startup in a new category we could call the “picks and shovels business” for video on the web. Rather than trying to build a destination, like the other hundred sites out there, they are attacking a problem faced by every user of every user-generated video site. Namely, “how do I get my video off my camera and upload to site X?”

The basics are pretty well understood: 1)rip the video from your camera onto your computer, 2)encode the video from raw format into something usable, and 3)upload the video to your favorite site. However, lots can go wrong in this process. In many cases you’ll use two different desktop apps for steps 1 and 2, and then have to use whatever hacked up process your site uses for file upload.

VideoEgg starts by attacking each of these problems. Their product supports grabbing video off of a connected device (camcorder or web cam), encoding that video, and then uploading the video to a remote server. All of this is packaged in a browser control (supports FF, IE, Safari, Opera according to the FAQ) for web page integration. The control even supports basic editing like setting Mark In/Out.
I like the technical idea a lot, because I don’t see anyone else attacking this end of the problem. The browser plugin is cool once it’s installed, but I’m not sure how many people will be willing to install it locally. I think that’s a very big barrier.

For the business model, it looked like Video Egg was primarily B2B, meaning they were trying to license the plugin to destination sites. However, more recently it looks like they’ve started letting end users publish video to their blogs via VideoEgg directly, with VideoEgg hosting the video. They even will autoblog to your blogging site, which is cool, but right now they only support Blogger and Typepad – doh! I tried it out over at a blogger account, check out the results here.

I have to admit the experience was very smooth, and the results are very good. The ability to trim the start and end of the video without using a video editing package is itself very cool. My original video file was 5 times longer than what you see on the blog. If I wanted to start “video blogging” I could see this being invaluable.

I’m not sure whether VideoEgg intend to pursue that b2c blog utility direction over time, or whether it’s a way to prove the technology and user appeal to potential partners. Now if only they could get FF or IE to pre-install their plugin, then they’d be set.

Can P2P save VOD?

People have been predicting that Video On Demand would be the “next big thing” for about 87 years now. Ok, maybe only 10. But it hasn’t happened. The main reasons are pretty clear:

  • Fear on the part of the big media companies (fear of copying, fear of killing the DVD cash cow).
  • The “couch potato problem” – I don’t want to watch TV/Movies on my computer.
  • The cost problem. If you checked out my earlier post covering bandwidth costs, then you can see that $.75 – $1.50 in bandwidth charges to download a single video kills lots of business models like pay-per-TV-show or video rental.

The fear amonst the media companies is slowly starting to change. Just look at iTunes video downloads plus lots of other early initiatives. There’s a little more faith in the reliability of DRM systems now. Not so much that they won’t get cracked, but rather that common users won’t bother and will accept encrypted files.

The problem of getting content to your big screen has not gone away. However, there are more devices that help this happen, and portable players like the video iPod and video-capable cell phones are turning the third screen into a bigger market.

However, the bandwidth cost problem is still with us, even after the dramatic fall in costs over the last few years. There is lots of video content that simply isn’t valuable enough to be worth the cost to download. Very valuable content, like a Hollywood movie, can still only fetch a few dollars on a rental, which makes a $1 cost to download prohibitive. Shorter content is a lot less costly to download, but most of it much be ad supported because users won’t pay real money for it. All of which makes sense as to why iTunes would launch with first-run TV shows – the download cost is smaller, yet people (apparently) are willing to pay a couple bucks to download them. (Read Robert Cringely’s analysis of iTunes video costs and the advantages of p2p distribution).

So for a long time people have recognized that using peer-to-peer for video distribution could dramatically lower the costs. Warner Brothers is set to launch such a p2p vod system in Europe in March. But in fact the core p2p technology already exists, and it’s called BitTorrent. The problem is, that’s the same technology that people are using to pirate tons of video content right now. So while Vinton Cerf claims that Hollywood is interested in using BitTorrent for distribution, the MPAA is in fact filing lawsuits to shut down sites offering torrents for download.

At the end of the day, I think the real problem is that all p2p clients require a desktop download right now. Given the big problems with spyware and viruses, that desktop install is a huge barrier to user adotion. Don’t believe me? Just compare the user base for Grouper to that for YouTube. (But don’t tell the folks at a recent Under the Radar conference who supposedly were wowed by Grouper). What we really need is the ability for a video web site like YouTube or my site to be able to use p2p for distribution, but behind the scenes without requiring any software install. Now that’s what I call nirvana, but of course, it’s impossible…or at least, very difficult. Can you serve up file segments from your browser using just Javascript?!?! If you’ve got a solution, please drop me a line so we can go start a company tomorrow!

Maybe the AllPeers plugin for Firefox is the answer. At least they’ve got the right idea, which is to run inside the brower. Of course, they need to support IE in addition to FF. If they could do that, and make the install as easy as the Flash player, then maybe…

Or maybe Windows Vista, with it’s built-in p2p features. Maybe MS will integrate hooks to the p2p library into an upgrade to IE…now that would be interesting!

Building the solution: Part 2 – Video content delivery

The overriding cost when dealing with video delivery on the web is bandwidth cost. The other major cost can be storage, depending on how big your catalog is. But even then bandwidth will generally dominate in the cost model.

For small companies, it is generally not financially feasible to try to serve video yourself. This is where a Content Distribution Network comes in. A CDN provides an outsourced service where they serve content for you for a price. Video is an obvious candidate because it’s so huge, but large sites often use CDNs for images or audio as well. A good example here is Myspace who uses Limelight Networks for delivering images.

A CDN works by providing a farm of servers for delivering your content to your users. All you need to do is to transfer your content to the CDN, and then give your user a link pointing to the CDN’s server pool instead of your own. Content can be transferred to the CDN either by pushing it up (typically using FTP) or by having the CDN pull it from your sever on demand. Pull-on-demand is the better model (it’s actually easier to implement since your just web-serve the content to the CDN) and supports more efficient caching strategies by the CDN.


CDN service costs money, but one great advantage is you save on hardware by not having to run a large pool of servers yourself. And in exchange you’re likely to get access to much better streaming capacity then you could set up yourself. For a small company, this advantage alone is enough to make the CDN worth it. However, depending on your situation the raw bandwidth charges may well be lower through the CDN than you can negotiate yourself.

A CDN generally charges based on bandwidth usage and storage. Costs-per-unit should drop as your usage grows. Bandwidth can be charged based on either total data transferred over a month, or based on peak transfer rate. Charging based on data transfer is simpler and easier to understand. If you serve 1 terabyte of video over the month, then you pay 1024 * the rate per Gig. Peak transfer pricing is a little harder to figure out. A typical formula is 95% peak. What happens is that the CDN samples your transfer rate periodically (say every 5 minutes). At the end of the month they throw out the top 5% of the highest samples. The next highest sample then determines your usage. The result is that you won’t get hit if you have a few short-term spikes in a traffic. But, if you got hit by a “slashdot effect” say, and traffic spiked for 48 hours then fell, then you will get pegged towards the peak. Overall, if your traffic is very consistent (which generally requires a lot of traffic), then overall transfer is probably the way to go. However, if you have spikey traffic, but the spikes are short-lived, then you may save some money paying on a peak transfer basis.

Ok, but what does it really cost?

Here again, there is more to the question. Different services have different pricing. For example, basic HTTP GET delivery will be the lowest cost per GB, but things like video streaming (mms or flash) will be charged at a higher rate.

For basic HTTP delivery, you should be looking at $60 – $100 per MB of peak usage. So if you have a peak transfer of 100 Mbps, that’s $10,000 per month. The trick now is to estimate your peak bandwidth usage (I’ll post some ideas on that later).

For Flash streaming, I have seen $.75 – $1.25 per GB transferred. To get the lower rates, you have to commit to higher minimum levels.

Where can I get it?

The market leader is Akamai/Speedera. I’m sure they have a great service, but I suspect they are also the most expensive. One salesguy’s quote to me was “You don’t use us to save on bandwidth costs”. Ouch! Other options are Limelight Networks, Vital Stream, and Mirror Image. All of these guys purportedly support Flash streaming.

Building the solution: Part 1 – The Format Wars

One of the fundamental questions when dealing with video on the web is: what format do I use? This question actually seems a lot easier to answer today than it was even in recent memory with the apparent dominance of Flash 8 video as the format of choice.

Flash had been showing up mostly on smaller sites, but recently I’ve started to notice more and more of the commercial news sites (like Foxnews) using Flash. I think I can safely say at this point that Real and Quicktime are dead as web video formats. WMV still has a little life, but given the horrible bugginess of the web control, I expect to see WMV disappear pretty quickly.

More and more of the sites using Flash are delivering good quality, fast serving, and great cross-platform/browser compatibility. Add to this the ability to brand/customize your player, plus supply a bloggable version, and the solution is awfully compelling. (One of the implications here is that if you’re not using Flash, then you’re at a competitive disadvantage).

One of the knocks against Flash has been the requirement to pay server licensing fees for the Flash video streaming server. Could this be the reason that YouTube appears to be using progressive download instead of streaming? It’s an interesting question since in theory streaming should save you a lot in bandwidth costs, for the simple reason that LOTS of video downloaded through an HTTP GET never gets watched by the user.

The server license cost can be offset somewhat by the fact that most of the CDNs are now supporting Flash streaming (see Limelight Networks or Vital Stream), although the Flash service does command a price premium.

So now that you’ve choosen Flash, how do you get your video into that format? Most of the authoring tools support a Flash codec, but for on-demand server encoding, you need an automatable solution. One that seems popular is On2, whose Flix Engine product supports server side Flash encoding. Anyone want to suggest any others or share your experience with them?