NTIRE2017 Super-resolution Challenge: SNU_CVLab

Discussion in 'Creativity section (Sound / Graphics / Music)' started by klapse, Apr 20, 2018.

Tags:
  1. klapse

    klapse Central Scrutinizer

    Joined:
    Aug 30, 2012
    Messages:
    1,802
    Location:
    Germany
    NTIRE2017 Super-resolution Challenge: SNU_CVLab

    Super-resolution is the modern term given to a new generation of neural-network based image upscaling algorithms. I've been interested in this for quite some time and these by far the most impressive results to have graced my screen.


    [​IMG]

    Scroll down in the linked github page for more example pictures; enjoy!

    https://github.com/LimBee/NTIRE2017
     
    Tags:
    FBnil and Autian like this.
  2. levi

    levi Still fresh, damnit!

    Joined:
    Oct 6, 2008
    Messages:
    9,751
    Location:
    Somewhere off the coast of the EU
    Enhance!

    Interesting stuff. I guess the HR patch is actually some high resolution source image, before blurring has been applied. Becuase otherwise it seems to win every time!
     
    Autian likes this.
  3. Autian

    Autian Still Fresh

    Joined:
    Jul 19, 2017
    Messages:
    35
    Location:
    Germany
    It would be awesome if this is implemented as an upscale filter for ffmpeg.
     
    levi likes this.
  4. levi

    levi Still fresh, damnit!

    Joined:
    Oct 6, 2008
    Messages:
    9,751
    Location:
    Somewhere off the coast of the EU
    Possibly, although running a recombinant neural network (as I believe the terminology goes) as a ffmpeg filter might run quite slowly. Even the downscale filter taking a 720p input and scaling it down to 600p for playback on my EeePC doesn't quite run in realtime on the fastest machine I currently own, so I can't see myself using it like that any time soon, but maybe in a few years machines that can do this in reasonable timeframes will become more commonplace.
     
  5. Autian

    Autian Still Fresh

    Joined:
    Jul 19, 2017
    Messages:
    35
    Location:
    Germany
    Well, in ffmpeg motion detecion filters were implemented recently for FPS increases in the output video. It has around 8 algorithms to choose from, a few of them being dead slow. So I don't see why it should not get into ffmpeg.
    I have an Eee PC too (1001HA) and I was surprised how slow it is compared to my main system with an E8600. Winamp takes 3% CPU time on the latter with the DSP plugin running my multiband compressor algorithm. On my Eee its around 35%, even without visualizers. I know it's a netbook, it has lower clock frequencies, an older CPU architecture and poor cooling, but still :D
    So I would not do any demanding video processing on it anyway :)
     
  6. levi

    levi Still fresh, damnit!

    Joined:
    Oct 6, 2008
    Messages:
    9,751
    Location:
    Somewhere off the coast of the EU
    Yeah, that's why I scale them on a more modern dual core system (only an E2220 IIRC) and play them over sshfs on my netbook locally.
     
  7. klapse

    klapse Central Scrutinizer

    Joined:
    Aug 30, 2012
    Messages:
    1,802
    Location:
    Germany
    It shouldn't be too long before there's FOSS software to super-resolution old SD video which takes into account information from multiple frames. NN Super-resolution of individual frames creates distracting artefacts which vary between frames.

    If any of y'alls finds a project that creates Full-HD video out of SD sources that's significantly better than bilinear scaling, I'd be quite interested to see it.

    Here's an example of what I get from a 1970s film with EnhanceNet.
    [​IMG]
    --- Double Post Merged, Apr 21, 2018, Original Post Date: Apr 21, 2018 ---
    Afaik it's the source image, which they then downscale (4x?), then the super-resolution algorithms upscale that smaller pic and we compare the results to the original.
     
    FBnil and Failbert like this.
  8. FBnil

    FBnil ℎ ℎ ’ ℎ .

    Joined:
    Dec 14, 2012
    Messages:
    2,382
    Location:
    Yurp
    @klapse: That is pretty awesome. Just the buildings behind the blinds is a bit weird. But the face is awesomely restored. I also notice that it tries to unblur what should be blurred... so I'm not sure about motion pictures, but static pictures... wow...
     
  9. Caine

    Caine Hardcore Member

    Joined:
    Jun 5, 2008
    Messages:
    4,081
    Location:
    Netherlands
    Oh great, now the general public is really going to believe that the enhance button exists.
     
  10. klapse

    klapse Central Scrutinizer

    Joined:
    Aug 30, 2012
    Messages:
    1,802
    Location:
    Germany
    I think the effect is nicest on the lampshade reflections. Will we someday get movie upscalers that create consistently pleasing results? I'm inclined to think so. It's a fascinating thing to play with.
     
    FBnil likes this.
  11. FBnil

    FBnil ℎ ℎ ’ ℎ .

    Joined:
    Dec 14, 2012
    Messages:
    2,382
    Location:
    Yurp
    I think so too. You know those services where you take a picture with the tower of Pisa and remove part of the image (because there is a person there), and these algorithms fill the space in using the best replacement? I think that when that tower of Pisa is modeled in 3D, it can be used to detect the exact viewpoint and angle and be used to "enhance" the picture using it's database of highres images and 3D rendering. But it would look weird if it can find a highres of a few objects and not others, thus mixing highres with lowres...
     

Share This Page

Loading...