"Super-resolution" is not a marketing buzzword, it's a mathematical term used by scientists. First work on this topic was published in 1984 [1] and the term "Super-resolution" itself appeared at around 1990 [2].

Methods usually discussed in scientific literature try to reproduce process of losing quality when shooting video with low-res cameras and then solve inverse problem of finding video which being downsized with that process gives us known low-res material. This is an ill-posed inverse problem which doesn't have straightforward solution and usually requires some additional regularization (applying some artificial constraints) and huge CPU time to check an awful lot of variants. Modern practical methods are usually simpler but still effective.

Super-resolution (SR) works effectively when several low resolution images contain slightly different perspectives of the same object. Then total information about the object exceeds information from any single frame. The best case is when an object moves in the video. Motion detection and tracking are then employed to benefit upscaling. If an object doesn't move at all and is identical in all frames, no extra information can be collected. If it moves or transforms too fast then it looks very different in different frames and it's too hard to use information from one frame in reconstructing the other.

Super-resolution is used now in two tasks: extracting single frames from video (or a set of images/photos) with high quality, or upsizing the whole video. SR methods are usually based on two important algorithms: high quality spatial (in-frame) upscaling, and motion compensation for finding corresponding areas in neighbor frames. Many practical implementations of super-resolution software upscale original material two times. If we need to upsize it four times, we usually apply SR twice (this can be done internally in implementation).

Let's consider a real example of upsizing video 4 times with super-resolution algorithm. This example was made using Video Enhancer. First, we have original low-resolution sequential frames of video (or consider them a sequence of slightly different low quality photos):

The same frames zoomed 4 times:

Step 1. Upsize all frames 2 times using good quality interpolation, in this case Lanczos3 method (note: our current implementation doesn't use Lancsoz anymore, it uses a similar method but with a different kernel):

Step 2. Using sub-pixel accurate motion compensation, find similar areas in neighbor frames and intelligently merge frames to combine information (and in some cases reduce noise):

Now we have our video upsized 2 times using super-resolution. We repeat this process to get 4x enlargement.

Step 3. Spatial interpolation again:

Finally, Step 4. Combine neighbor frames to get more details using motion compensation:

Now let's compare it with what we originally had:

One may ask: was that motion compensated merging really required? Weren't spatial interpolated frames good enough? The answer will become obvious if you look at comparison of different upsize methods. Shortly, spatial interpolation methods don't even come closer to a good super-resolution in terms of details and objective metrics.

Note that super-resolution cannot always provide excellent results. Read in the Infognition knowledge base when super-resolution doesn't work.

Papers:

1. R. Y. Tsai and T. S. Huang, "Multiframe image restoration and registration," in Advances in Computer Vision and Image Processing, vol. 1, chapter 7, pp. 317-339, JAI Press, Greenwich, Conn, USA, 1984.

2. M. Irani and S. Peleg. 1991, "Super Resolution From Image Sequences" ICPR, 2:115--120, June 1990.

Try this yourself with Video Enhancer

Get Super Resolution plugin for VirtualDub
Get Super Resolution plugin for Adobe After Effects and Premier Pro
Get Super Resolution plugin for AviSynth
OFX plugin coming soon.

Other articles...