John Fremlin's blog: Better video thumbnails by facing the facts

Posted 2010-01-14 17:23:00 GMT

Today, I made a nasty little prototype program that generates thumbnails from videos by looking at the faces in them. I called it fumbnailer.

At the moment, it successfully hooks up ffmpeg to opencv, but nothing works very well. ffmpeg (even the development version) seems to utterly fail to seek in flv streams, even when given an exact byte offset to seek to, which is rather irritating. And opencv doesn't give a confidence score for finding a face, which results in the face detection thumbnail generally not including a good face.

fumbnailer implements a second thumbnail selection algorithm (mest), which simply returns the first of the pair of keyframes with the highest intervening datarate. This is supposed to be a proxy for discovering the scenes with the highest movement.

Yesterday, I submitted a patch for ffmpegthumbnailer, (to allow creating thumbnails the same size as the original video), which Dirk (the maintainer) kindly accepted. In the end however, I didn't base fumbnailer on it though, because of its (for my purposes) overly complicated design — adding a single command line option would require modifying about three files, for example. I wanted to bash something out that would let me quickly get feedback on a few ideas.

It seems that modern C++ with the auto keyword seems quite fun. Here's hoping lambdas make it into g++ soon!

Finding a good thumbnail from a video is a kind of trivial sounding problem that for some reason appeals to me. I'd like to develop the two current approaches until they demonstrate their potential (seem to sporadically make good results on the videos I tried — is this just me?), and try a few more ideas out — maybe quality of focus measures next?

Please suggest something radical!

This may be radical overkill for finding faces. Check out they have some vision classiffication learning system code available at no charge that can be used for noncomercial stuff. If you look here they have some demos. This link provides some customer application demos.


p.s. I have no affiliation with numenta nor have I used their code. Using their code is definatly on my "to get around to" pile.

Posted 2010-12-24 23:55:11 GMT by Anonymous from

Post a comment