NTIRE2017 Super-resolution Challenge: SNU_CVLab

[klapse]

NTIRE2017 Super-resolution Challenge: SNU_CVLab

Super-resolution is the modern term given to a new generation of neural-network based image upscaling algorithms. I've been interested in this for quite some time and these by far the most impressive results to have graced my screen.

Scroll down in the linked github page for more example pictures; enjoy!

https://github.com/LimBee/NTIRE2017

 

[levi]

Enhance!

Interesting stuff. I guess the HR patch is actually some high resolution source image, before blurring has been applied. Becuase otherwise it seems to win every time!

 

[Autian]

It would be awesome if this is implemented as an upscale filter for ffmpeg.

 

[levi]

Possibly, although running a recombinant neural network (as I believe the terminology goes) as a ffmpeg filter might run quite slowly. Even the downscale filter taking a 720p input and scaling it down to 600p for playback on my EeePC doesn't quite run in realtime on the fastest machine I currently own, so I can't see myself using it like that any time soon, but maybe in a few years machines that can do this in reasonable timeframes will become more commonplace.

 

[Autian]

Well, in ffmpeg motion detecion filters were implemented recently for FPS increases in the output video. It has around 8 algorithms to choose from, a few of them being dead slow. So I don't see why it should not get into ffmpeg.
I have an Eee PC too (1001HA) and I was surprised how slow it is compared to my main system with an E8600. Winamp takes 3% CPU time on the latter with the DSP plugin running my multiband compressor algorithm. On my Eee its around 35%, even without visualizers. I know it's a netbook, it has lower clock frequencies, an older CPU architecture and poor cooling, but still
So I would not do any demanding video processing on it anyway

 

[levi]

Yeah, that's why I scale them on a more modern dual core system (only an E2220 IIRC) and play them over sshfs on my netbook locally.

 

[klapse]

It shouldn't be too long before there's FOSS software to super-resolution old SD video which takes into account information from multiple frames. NN Super-resolution of individual frames creates distracting artefacts which vary between frames.

If any of y'alls finds a project that creates Full-HD video out of SD sources that's significantly better than bilinear scaling, I'd be quite interested to see it.

Here's an example of what I get from a 1970s film with EnhanceNet.

[doublepost=1524302970,1524301891][/doublepost]

Enhance!

Interesting stuff. I guess the HR patch is actually some high resolution source image, before blurring has been applied. Becuase otherwise it seems to win every time!

Afaik it's the source image, which they then downscale (4x?), then the super-resolution algorithms upscale that smaller pic and we compare the results to the original.

 

[FBnil]

@klapse: That is pretty awesome. Just the buildings behind the blinds is a bit weird. But the face is awesomely restored. I also notice that it tries to unblur what should be blurred... so I'm not sure about motion pictures, but static pictures... wow...

 

[Caine]

Oh great, now the general public is really going to believe that the enhance button exists.

 

[klapse]

@klapse: That is pretty awesome. Just the buildings behind the blinds is a bit weird. But the face is awesomely restored. I also notice that it tries to unblur what should be blurred... so I'm not sure about motion pictures, but static pictures... wow...

I think the effect is nicest on the lampshade reflections. Will we someday get movie upscalers that create consistently pleasing results? I'm inclined to think so. It's a fascinating thing to play with.

 

[FBnil]

I think the effect is nicest on the lampshade reflections. Will we someday get movie upscalers that create consistently pleasing results? I'm inclined to think so. It's a fascinating thing to play with.

I think so too. You know those services where you take a picture with the tower of Pisa and remove part of the image (because there is a person there), and these algorithms fill the space in using the best replacement? I think that when that tower of Pisa is modeled in 3D, it can be used to detect the exact viewpoint and angle and be used to "enhance" the picture using it's database of highres images and 3D rendering. But it would look weird if it can find a highres of a few objects and not others, thus mixing highres with lowres...

 

[FBnil]

CVLab | Computer Vision Lab. Seoul National University

cv.snu.ac.kr

Oh, it's an annual thing!

NTIRE2019: New Trends in Image Restoration and Enhancement workshop and challenges on image and video restoration and enhancement

PSD to HTML5+CSS3 conversion.

www.vision.ee.ethz.ch

Build software better, together

GitHub is where people build software. More than 100 million people use GitHub to discover, fork, and contribute to over 420 million projects.

github.com

Some of the entries have a PPTX that barely fits on a CD!

 

[levi]

Enhance!

Edit: Damnit, I tried to pull the same gag last year!

 

[erico]

I think the effect is nicest on the lampshade reflections. Will we someday get movie upscalers that create consistently pleasing results? I'm inclined to think so. It's a fascinating thing to play with.

Can be done manually, so it will be done automatically at some point.
I love this kind of stuff.