Nothing here

Previously I wrote about encoding H.264 video for web. Since video support in browsers is kind of a mess, you’ll be forced to re-encode videos to WebM (VP8/vorbis) format sooner or later if you want HTML5 playback. FFMpeg is able to use the official libvpx library, which makes it pretty easy. However, the support is still slightly buggy and the parameter mapping is non-obvious if you’re used to encoding to other formats.

So, let’s start:

Step 1: Get new FFmpeg and libvpx versions

libvpx is a rather new addition to FFmpeg and for quality reasons you really want to use the newest stable version of FFmpeg and libvpx. How to get those was described in my previous article. Just don’t forget to compile in libvpx and libvorbis support.

Step 2: Choose resolution and bitrate

You’ll have to choose resolution and bitrate into which to encode. For general guidelines check my previous article and add about 10-15% to video bitrate. libfaac and libvorbis output quality is mostly comparable, so you can just keep the same settings.

Step 3: Encode!

All the choices being made, now it’s time to encode your video:

ffmpeg -i input_file.avi -codec:v libvpx -quality good -cpu-used 0 -b:v 500k -qmin 10 -qmax 42 -maxrate 500k -bufsize 1000k -threads 4 -vf scale=-1:480 -codec:a libvorbis -b:a 128k output.webm

(I’m using new-style parameters introduced in latest FFmpegs, old-style parameters are being slowly deprecated because of ambiguity)

 

Two-pass encoding is of course also possible:

1st pass

ffmpeg -i input_file.avi -codec:v libvpx -quality good -cpu-used 0 -b:v 500k -qmin 10 -qmax 42 -maxrate 500k -bufsize 1000k -threads 4 -vf scale=-1:480 -an -pass 1 -f webm /dev/null

2nd pass

ffmpeg -i input_file.avi -codec:v libvpx -quality good -cpu-used 0 -b:v 500k -qmin 10 -qmax 42 -maxrate 500k -bufsize 1000k -threads 4 -vf scale=-1:480 -codec:a libvorbis -b:a 128k -pass 2 -f webm output.webm

As mentioned in the previous article, FFmpeg parameters are position sensitive, so be sure to maintain the order of parameters.

Now, let’s break down the parameters for the encode:

-i [input file]  - Specifies the name and path to the input file
-codec:v - Specifies the video encoder to be used, in our case that is libvpx VP8 library
-quality good – Sets encoding speed for the VP8 encoder. This works in concert with cpu-used parameter. Available values are best, good and realtime. The official libvpx documentation strongly advises against using best quality parameter, because good with cpu-used 0 offers almost identical quality for half the encoding time.
-cpu-used [0-5] - Sets “speed” of encoding – lower value uses more CPU for processing and produces better quality. Larger values trade quality for faster encoding, with 4 and 5 enabling “rate distortion optimization”, which significantly speeds-up encoding with price of big quality hit.
-b:v [bitrate] – Sets desired output video bitrate
-maxrate/-bufsize – Set upper limits for stream bitrate, maxrate specifying maximum bitrate and bufsize specifying device buffer size. Buffer size tells the encoder how much it can overshoot the maximum bitrate when required. Good default is twice the maxrate for approx. 2sec of buffer.
-qmin 10 -qmax 42 – This sets minimum and maximum quantization values. Since as of 0.9, FFmpeg sets those values wrong by default, adding these is required for a a decent video quality. Omitting these will produce blocky broken video.
-threads [num] – Sets number of encoding threads to use. Set to the number of your CPU cores available.

-vf scale=[width:height] - Rescales video to a chosen resolution. Value of “-1″ means “size to keep aspect ratio”, e.g. setting this to “-1:720″ will produce a 720p output with same aspect ratio as input.

-codec:a libvorbis – Sets output audio encoder to libvorbis to produce vorbis output. This is required for a valid WebM file
-b:a [bitrate] – Sets bitrate for encoded audio

-f webm – tells FFmpeg the output file format (required only if it cannot be inferred from output file extension)
-pass [1|2] – tells FFmpeg to process video in multiple passes and sets the current pass
-an – disables audio, audio processing has no effect on first pass so it’s best to disable it to not waste CPU

That’s mostly all there is to it – other FFmpeg mappings for libvpx can be found in the documentation. It’s also worth noting, that libvpx supports the “profile” parameter, which sets complexity of output stream, much like H.264 has. However, since devices with WebM support are very scarce, there currently isn’t much documentation on which devices supports which profile.

Some sample command lines:

These are analogous to the command lines in the H.264 article. Replace the thread count in -threads parameter with number of your CPU cores for optimal encoding speed.
“Standard” web video (480p at 600kbit/s):
ffmpeg -i input_file.avi -codec:v libvpx -quality good -cpu-used 0 -b:v 600k -maxrate 600k -bufsize 1200k -qmin 10 -qmax 42 -vf scale=-1:480 -threads 4 -codec:a vorbis -b:a 128k output_file.webm

High-quality SD video for archive/storage (PAL at 1.2Mbit/s):
ffmpeg -i input_file.avi -codec:v libvpx -quality good -cpu-used 0 -b:v 1200k -maxrate 1200k -bufsize 2400k -qmin 10 -qmax 42 -vf scale=-1:480 -threads 4 -codec:a vorbis -b:a 128k output_file.webm

Web is full of articles about encoding videos with FFmpeg, however most of them are obsolete and use old non-working FFmpeg parameters. So here’s my guide for encoding web videos using recent FFMpeg versions for Flash and HTML5.

Step 1: Get a new build of FFmpeg

Most distribution builds are very old, buggy and have numerous issues. Don’t use them.

Ubuntu users: UbuntuForums has a great guide for compiling newest FFMpeg build. I strongly suggest you use a stable FFmpeg build (that’s 0.9 at the moment) instead of git master. Don’t forget libx264 for H.264 and libvpx for VP8/WEBM if you want that support.
Windows users: Zeranoe has great static builds of FFmpeg for Windows with libx264 and libvpx included. Use those for encoding.
OS X users: MacPorts should be able to compile and install FFmpeg with libvpx and libx264. As I don’t use any machines with OS X you’ll have to check for yourself.

If you already know what “bitrates, profiles etc.” are and just want the command lines skip to step 4 or sample command lines.

Step 2: Choose resolution, bitrate and profile

Now you’ll have to decide on two things – the resolution, bitrate and profile you want those videos in.

Resolution gives “sharpness” to the overall image. If you choose a low resolution, the video will be small and it will be blurry when users will put it full-screen. Typically people use 360p (that’s a shorthand naming made popular by HD televison, meaning picture has height of 360 with width corresponding to wanted aspect ratio), 480p, 720p and 1080p resolutions, as those correspond to common screen sizes, which avoids excessive blurriness.
Contrary to popular belief, resolution does not affect file size.

Bitrate tells the encoder about how many bits should each second of video have. It directly determines file size of the video along with the quality. Set too low, it will cause the video to look very blocky (especially in fast-moving scenes) and set too high will make your files excessively large.
When choosing bitrate you need to remember, it is directly connected to resolution – storing pictures of certain resolution requires more bits, so if you want higher resolution videos, you’ll have to choose higher bitrate for them to not look like garbage.
A rule of thumb to calculate file size from bitrate is:

filesize (in MB) = (bitrate in Mbit/s * 8) * (video length in seconds)

Some general resolution/bitrate guidelines that we’ve found to work well at Viidea:

Resolution	Bitrate	Approximate size of 10 min video
320p (for mobile phones)	180 kbit/s	~13 MB
360p	300 kbit/s	~22 MB
480p	500 kbit/s	~37 MB
576p (SD)	850 kbit/s	~63 MB
720p (HD)	1000 kbit/s	~75 MB

The values in the table were optimized for lecture-type content recorded with SD cameras, so if you want to encode something more dynamic (like Transformers ;) ) I suggest you bump the bitrate up a little.

Profile constrains H.264 to a subset of features – higher profiles require more CPU power to decode and are able to generate better looking videos at same bitrate. You should always choose the best profile your target devices support.

Basic support matrix for devices:

Device	Max supported profile
Desktop browsers iPhone 4S, iPad 2 Tegra Android tablets Xbox 360, Playstation 3	High profile
iPhone 3GS, iPhone 4, iPad high-end Android phones	Main profile
iPhone, iPhone 3G medium/low-end Android devices other embedded players	Baseline profile

Remember, most devices also have a maximum resolution and bitrate they can handle. That is usually expressed in as a H.264 level and can be set in FFmpeg with -level parameter (this will make FFmpeg abort encoding of videos which couldn’t be played on the device).

Step 3: Choose audio format and bitrate

For audio, the choice is a lot simpler – if you want to encode for mobile devices I strongly suggest you use AAC+, which has a very noticeable improvement of audio quality at the same bitrate. Sadly, libaacplus in FFmpeg, doesn’t support encoding to higher bitrates than 64 kbps. With that in mind, AAC+ is perfect for podcasts, lectures and other speech content, since it retains quality even at low bitrates (difference between “normal” LC-AAC at 96 kbit/s  vs. AAC+ at 64kbit/s is practically not noticeable).

Otherwise, rule of thumb is to take around 128 kbit/s for standard quality material and 192kbit/s for something more complex.

Step 4: Encode!

So, you made all the hard choices and now it’s time to encode your video. FFmpeg command line to encode a standard web video looks like this:

ffmpeg -i input_file.avi -vcodec libx264 -vprofile high -preset slow -b:v 500k -maxrate 500k -bufsize 1000k -vf scale=-1:480 -threads 0 -acodec libvo_aacenc -b:a 128k output_file.mp4

You can also encode the video in two-passes, which gives the added benefit of quality increase and more accurate file size for given bitrate:

1st pass:

ffmpeg -i input_file.avi -vcodec libx264 -vprofile high -preset slow -b:v 500k -maxrate 500k -bufsize 1000k -vf scale=-1:480 -threads 0 -pass 1 -an -f mp4 /dev/null

2nd pass:

ffmpeg -i input_file.avi -vcodec libx264 -vprofile high -preset slow -b:v 500k -maxrate 500k -bufsize 1000k -vf scale=-1:480 -threads 0 -pass 2 -acodec libvo_aacenc -b:a 128k -f mp4 output_file.mp4

Scary isn’t it?

Warning: ffmpeg command line arguments are position sensitive – make sure you don’t mix up the order. Good rule of thumb to prevent mistakes is to keep the order of
ffmpeg [input options] -i [input filename] -vcodec [video options] -acodec [audio options] [output file options] [output filename]

Let’s break down all those parameters:

-i [input file]  - this specifies the name of input file
-vcodec libx264 – tells FFmpeg to encode video to H.264 using libx264 library
-vprofile high – sets H.264 profile to “High” as per Step 2. Other valid options are baseline, main
-preset slow – sets encoding preset for x264 – slower presets give more quality at same bitrate, but need more time to encode. “Slow” is a good balance between encoding time and quality.
Other valid options are: ultrafast, superfast, veryfast, faster, fast, medium, slow, slower, veryslow, placebo (never use this one)
-b:v - sets video bitrate in bits/s
-maxrate and -bufsize – forces libx264 to build video in a way, that it could be streamed over 500kbit/s line considering device buffer of 1000kbits. Very useful for web – setting this to bitrate and 2x bitrate gives good results.
-vf scale – applies “scale” filter, which resizes video to desired resolution. “720:480″ would resize video to 720×480, “-1″ means “resize so the aspect ratio is same.”
Usually you set only height of the video, so for 380p you set “scale=-1:380″, for 720p “scale=-1:720″ etc.
-threads 0 – tells libx264 to choose optimal number of threads to encode, which will make sure all your processor cores in the computer are used

-acodec libvo_aacenc – tells FFmpeg to encode video to AAC using libvo_aacenc library
-b:a - sets audio bitrate in bits/s

-pass [1|2] – tells FFmpeg to process video in multiple passes and sets the current pass
-an – disables audio, audio processing has no effect on first pass so it’s best to disable it to not waste CPU

That’s basically all there is to it.

Some sample command lines:

“Standard” web video (480p at 500kbit/s):
ffmpeg -i input_file.avi -vcodec libx264 -vprofile high -preset slow -b:v 500k -maxrate 500k -bufsize 1000k -vf scale=-1:480 -threads 0 -acodec libvo_aacenc -b:a 128k output_file.mp4

360p video for older mobile phones (360p at 250kbit/s in baseline profile):
ffmpeg -i inputfile.avi -vcodec libx264 -vprofile baseline -preset slow -b:v 250k -maxrate 250k -bufsize 500k -vf scale=-1:360 -threads 0 -acodec libvo_aacenc -ab 96k output.mp4

480p video for iPads and tablets (480p at 400kbit/s in main profile):
ffmpeg -i inputfile.avi -vcodec libx264 -vprofile main -preset slow -b:v 400k -maxrate 400k -bufsize 800k -vf scale=-1:480 -threads 0 -acodec libvo_aacenc -ab 128k output.mp4

High-quality SD video for archive/storage (PAL at 1Mbit/s in high profile):
ffmpeg -i inputfile.avi -vcodec libx264 -vprofile high -preset slower -b:v 1000k -vf scale=-1:576 -threads 0 -acodec libvo_aacenc -ab 196k output.mp4

If you’re interested in encoding WebM instead of H.264, read the next article :)

About a week ago Webcamp Ljubljana happened and among other awesome web-related knowledge I’ve also give an overview lecture on what do you need to encode, stream and play video on the web.

The lecture mostly outlines current web format war, different methods of video delivery and different methods of playing videos. The lecture was given in Slovenian language and is available on Webcamp.si videos site.

All WebcampLJ videos are hosted by us (Viidea ;)) and I very much recommend seeing them, they contain several gold nuggets of knowledge.

There is a writing on the wall of my workplace tea room that states “If you’re not ashamed of version 1.0, you haven’t shipped enough.”. Well, the original version of Prevoz.org application made for Mobitel competition was bad enough to prevent me from showing it to people :). The tabbed interface felt clunky even at the original release and Android application moved away from that interface paradigm awhile ago. Also the design would cause alot of serious implementation problems with new features.

So in light of those problems and latest Prevoz legal developments I’ve finally managed to finish up version 2.0 of the Android mobile application and publish it on Android Market.

 

Sadly it does not have any significant new features, but it does feature reworked interface, which is easier to use and look at.  Under-the-hood improvements should speed the application  up and make it more reliable as well.

 

All comments on the application are welcome :)

 

As karma has it, the Bicikel app had a pretty noticeable bug after being released – if the phone took some time to lock its location, the station list did not sort correctly. Also, the full station map view didn’t really show any useful information beyond the station position.

So now I released Bicikel 1.1 for Android with bugfixes and new station list and map coloring, so the user can see stations that are either full or empty at first glance.

New station list and map view:

Additional fixes and enhancements follow :)

Bicike(LJ) is a rent-a-cycle project in Ljubljana, where you can rent bikes from pre-built stations and use them for fast and rather efficient transport across the city. It’s cheap (renting a bike for up to one hour is free) and the bikes are comfortable enough.

However, now there are problems in the evenings with stations in the city center being empty and the outlying stations being full. There’s nothing more annoying than cycling across the city in the middle of the night and finding that your station is full and you can’t park your bike there. Since this annoyed me, I used up some of the free time to make a quick Android application to see which stations have bikes available and which are full. In the process I managed to find new quirks in Android API and learned how to draw (ugly) icons in GIMP.

So, presenting Bicikel app:

 

The application just lists all known working Bicikelj stations with number of bikes parked and number of free spaces available. For convenience, the results are sorted by distance from you (if location is available) and map with all stations is also provided.

 

So far the application works pretty well. Thanks to Gašper Žejn for aggregating API data into his own API service.
The things missing I intend to implement are

• notifications if the station you’re headed to has filled
• slightly more functional design with directions to nearest station

The application is available on Android Market for free.

Lately I’ve watched several recorded video lectures and I’ve noticed, that usually I can’t see the slides on them all that well. Which can pose a problem with lecturers, who rely heavily on their slides for visibility. Later on, I’ve seen that VideoLectures solve this problem by showing pictures of slides next to the video itself, which is a pretty neat solution. I’ve been thinking about implementing this to Kiberpipa lecture videos, however the problem is, that Kiberpipa doesn’t have the manpower, to watch all the videos and tag the exact slide timings in it.

The logical solution would be to detect the slides in video automatically… but of course, there’s no library to do that yet. Which is why I have decided to actually make one for my final CS diploma project, with a little help of my CV course material.

It seems the best way to do the solution will be to use pyffmpeg to actually decode the video and use OpenCV for computer vision algorithms, so I won’t have to reimplement them again by myself.

Also, Richard Szelinski has made his book about practical computer vision algorithms and applications available to public, which is also a great help, as it has bunch of useful applications and algorithms listed.

More postings with progress will follow…

 

Some time ago I had a lecture in Kiberpipa about 3D game engine history titled “From Doom to Crysis” (Od Pogube do Krize) with some very nice demonstrations and anecdotes of modern 3D game engines and their older counterparts.

The video is now available on Kiberpipa video archive with slides available on Slideshare (I suggest downloading original pptx file for embedded videos). The lecture is in Slovenian language.

In my earlier post when I was ranting over I posted a screenshot of Prevoz application for Nokia mobile phones. It is a WRT widget developed as part of Nokia App Forum competition.

The application has been complete for some time now and it was submitted to Ovi Store a little less than a month ago.  Since then, we received response after 8 days, that there is a typo in metadata. We fixed the typo and re-submitted the application for review. The situation since then is shown on the image:

In 24 days, they have been unable to review a simple little WRT widget written for their timed competition, where number of downloads in certain time span (which started running a month ago) is what determines the winner. In total that means it has been 32 days since application has been submitted to Ovi store for the first time and it still hasn’t received the first review. After all the stupid Nokia decisions, which make development for their platform a hell on earth, the completely absurd review process for Ovi store pretty much puts the last nail in the coffin of Nokia development.

My recommendation: go develop for Android or iOS. Better documentation, better support, faster review and you can actually get your application released in due time. Let Nokia die and disappear into history where is belongs.

PhoneGap is a small JavaScript framework with big promises: it’s promising easy development of cross-platform mobile apps – write your application once in combination of HTML, CSS and JavaScript, then publish it on Android Market, Apple AppStore and Nokia OviStore. It’s goal is to expose platform-specific device capabilities in such a way, that you can run the application on all devices without any special per-platform customization.

If you are a mobile development and have started chuckling when reading the first paragraph… you were largely right. PhoneGap, despite the teams ambition, simply fails to deliver and is largely useless for mobile development.

But before the criticism, let’s see what PhoneGap actually does right:

• It offers a way to package web-applications, so they seem as native applications on the platform. That holds true for Nokia’s WRT/Qt platforms, Android and iOS. Also the implementation is “AppStore” compatible, so there are no problems.
• The APIs that are implemented (GPS for example) mostly actually work on phones on which PhoneGap applications are capable of running.

However, the list of things that make PhoneGap mostly useless is sadly a lot bigger:

• The documentation is meager to non-existing, which makes determining capabilities of the framework prior actual application development almost impossible.
• Most of the API is still not implemented and missing. For example the AJAX call implementation is missing, which means you need to either be happy with the lowest common denominator for AJAX capabilities (which is crippled, because ANY request made is cross-domain) or use device-specific capabilities.
• While offering access to some hardware-related capabilities, it does not offer any usable user interface function to help cross the user-experience and device capability gap. That even poses the problem with such simple things as loading animations, where Android doesn’t support any image animation, while Nokia and iPhone do.
  This forces the developer to write platform-specific code, which defeats the purpose of cross-platform library.
• On the same note, it lacks any UI widget interfaces or ability to create portable interface. PhoneGap would be truly usable if it would offer widget capability similar to Qt on desktop systems, where widgets get native display according to platform they are running on.
• And one of the biggest problems for a web-enabled app is the lack of any cross-platform way of invoking the device browsers. This makes modern authentication schemes as are OpenID or OAuth impossible to implement.

Besides all those points, the framework is rather slow and buggy, which puts the last nail in the coffin of the framework. PhoneGap in action shows usefulness of the framework pretty accurately.

We decided to use PhoneGap when starting to develop Prevoz.org Nokia application for Nokia Application Academy, since it promised rapid prototype development and deployment, which could later be changed to native implementation when enough users would adopt it. However, it has proven to be completely unusable and as a warning to all other developers:

Go native. Development is still faster than using PhoneGap and application offers better experience to the user.

One rewrite and a lot of wasted time later, the application was developed in native Nokia WRT library with WRTKit widget framework: