Free Lossless Image Format (FLIF)


Yep, probably.

I started debugging, and also will try to compare execution with an x64 build for reference.

For now, the Corruption messages are from the "snap" function.
In my test iamge (a simple png screenshot), I count number of time "ColorRangeCB::snap" is called, and at occurence #577490, the bucket "b" seems empty, with its min and max to default value:
Code:
Breakpoint 1, ColorRangesCB::snap (this=0x232a70, this@entry=0x3b12c0, p=p@entry=2, pp=..., minv=@0xbeffe528: 10000, maxv=@0xbeffe524: -10000, v=@0xbeffe46c: 24) at transform/colorbuckets.hpp:300
300                 e_printf("Corruption detected in snap (%d)!\n", cnt);
(gdb) p b
$1 = (const ColorBucket &) @0x22f878: {snapvalues = {<std::_Vector_base<short, std::allocator<short> >> = {
      _M_impl = {<std::allocator<short>> = {<__gnu_cxx::new_allocator<short>> = {<No data fields>}, <No data fields>}, _M_start = 0x0, _M_finish = 0x0, _M_end_of_storage = 0x0}}, <No data fields>},
  values = {<std::_Vector_base<short, std::allocator<short> >> = {_M_impl = {<std::allocator<short>> = {<__gnu_cxx::new_allocator<short>> = {<No data fields>}, <No data fields>}, _M_start = 0x0,
        _M_finish = 0x0, _M_end_of_storage = 0x0}}, <No data fields>}, min = 10000, max = -10000, discrete = true}
I'll try to break at the same point (with a static counter) in my reference build to see the status of the "b".
[doublepost=1473499396,1473497218][/doublepost]Mmm, execution is very different it seems, even if on the same image :(

break point at the same loop:
on PC:
Code:
Python Exception <class 'RuntimeError'> Type is not a template.: 
Breakpoint 1, ColorRangesCB::snap (this=0x7a64f0, p=0, pp=..., minv=@0x7fffffffd94c: 0, maxv=@0x7fffffffd948: 47, v=@0x7fffffffd7ec: 11)
    at transform/colorbuckets.hpp:300

and on Pandora:
Code:
Breakpoint 1, ColorRangesCB::snap (this=0x232a70, this@entry=0x3b12c0, p=p@entry=2, pp=..., minv=@0xbeffe528: 14, maxv=@0xbeffe524: 17, v=@0xbeffe46c: 24) at transform/colorbuckets.hpp:297
297                 printf("STOP!\n");

I will have to change my approach. I'll start with converting simple small files and activate all the "print" function in the code and compare them...
[doublepost=1473500974][/doublepost]Of course, simple file just works... And gives amazingly small files (a 8x8 white box is 17 bytes, and a few colored points it 38 bytes). Large uniform bow also doesn't trigger (I assume this simple case have specifcs encoding path).

This gradiant (black/white in a 800x600 image) triggers the issue.
I activated all "print" code in colorbuckets.hpp and launch flif with -v
On the pandora, I got:
Code:
Loading input file: ../bwgradiant800x600.png  Alpha channel not actually used, dropping it.
 (800x600)
Filled color buckets with 0 discrete colors + 731 continous buckets
Saving Y Color Bucket:
Saving I Color Buckets

Saving Q Color Buckets

Y buckets:
C0..244
I buckets:
C-1..1 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-1..2 C-2..2 C-2..2 C-2..2 C-1..2 C-2..2 C-2..2 C-2..2 C-2..1 C-2..2 C-2..1 C-1..1 C-1..0
Q buckets:
  C-1..1 E
  C-2..2 C-1..0
  C-2..2 S0
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 S0
  C-2..2 C0..1
  C-2..2 S0
  C-2..2 C-1..0
  C-2..2 S0
  C-2..2 C0..1
  C-2..2 S0
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 S0
  C-2..2 S0
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 S0
  C-2..2 S0
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 S0
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C-1..0
  C-2..2 C0..1
  C-2..2 S0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 S0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C0..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 S0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C0..1
  C-2..2 S0
  C-2..2 S0
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 S0
  C-2..2 S0
  C-2..2 C-1..0
  C-2..2 C-1..0
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 S0
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C-1..0
  C-2..2 S0
  C-2..2 C-1..0
  C-2..2 S0
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 S0
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 S0
  C-2..2 S0
  C-2..2 S0
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 S0
  C-2..2 C0..1
  C-2..2 S0
  C-2..2 S0
  C-2..2 S0
  C-2..2 S0
  C-2..2 S0
  C-1..2 C-1..0
  C-2..2 S1
  C-2..2 C-1..0
  C-2..2 S0
  C-2..2 C0..1
  C-2..2 S0
  C-2..2 S0
  C-1..2 S0
  C-2..2 C-1..0
  C-1..2 E
  C-2..2 S0
  C-2..2 E
  C-1..2 E
  C0..1 E
Corruption detected!
Corruption detected!
Corruption detected!
Corruption detected!
Corruption detected!
Corruption detected!
flif: flif-enc.cpp:224: void flif_encode_FLIF2_inner(IO&, Rac&, std::vector<Coder>&, const Images&, const ColorRanges*, int, int, const int*) [with IO = FileIO; Rac = RacOutput24<FileIO>; Coder = FinalPropertySymbolCoder<SimpleBitChance, RacOutput24<FileIO>, 10>; Images = std::vector<Image>]: Assertion `curr <= max' failed.
  Aborted
I have the same on the PC until the "Corruption detected!". Instead, it just save with this output
Code:
94% done [23/23] ENC[2,800x600]  Writing checksum: 4CB1EECA
Wrote output FLIF file test05.flif, 212307 bytes for 800x600 pixels (3.5385bpp)
[doublepost=1473502355][/doublepost]on the Pandora, the first "Corruption detected!" message has this backtrace:
Code:
(gdb) bt
#0  ColorRangesCB::snap (this=0x1ff3d8, this@entry=0x3dc948, p=p@entry=2, pp=..., minv=@0xbeffe528: 10000, maxv=@0xbeffe524: -10000, v=@0xbeffe46c: 0) at transform/colorbuckets.hpp:295
#1  0x0004e130 in predict_and_calcProps_plane<Plane<short>, Plane<unsigned char>, false, false, 2, ColorRanges> (properties=..., ranges=0x3dc948, ranges@entry=0x1ff3d8, image=..., plane=..., planeY=...,
    z=z@entry=1, r=287, r@entry=333092, c=797, c@entry=2, min=@0xbeffe528: 10000, max=@0xbeffe524: -10000, predictor=0, predictor@entry=-1090525912) at common.hpp:295
#2  0x00048bac in predict_and_calcProps (properties=..., ranges=0x1ff3d8, ranges@entry=0x11f, image=..., z=1, z@entry=0, p=p@entry=2, r=r@entry=287, c=c@entry=797, min=@0xbeffe528: 10000,
    max=@0xbeffe524: -10000, predictor=0, predictor@entry=1) at common.cpp:148
#3  0x00051524 in find_best_predictor (images=..., ranges=0x11f, ranges@entry=0x1ff3d8, p=p@entry=2, z=0, z@entry=1) at flif-enc.cpp:163
#4  0x0006d708 in flif_encode<FileIO> (io=..., images=..., transDesc=..., encoding=<optimized out>, learn_repeats=2, acb=-1, palette_size=512, lookback=1, divisor=30, min_size=50,
    split_threshold=split_threshold@entry=349504, cutoff=cutoff@entry=2, alpha=1, alpha@entry=19, crc_check=<optimized out>, crc_check@entry=-1, loss=19, loss@entry=0, predictor=0xffffffff,
    predictor@entry=0xbeffe944, invisible_predictor=invisible_predictor@entry=2) at flif-enc.cpp:947
#5  0x00128bb4 in encode_flif (argc=argc@entry=1, argv=0x1, argv@entry=0xbeffeb00, images=..., options=...) at flif.cpp:387
#6  0x001299c8 in handle_encode (argc=1, argc@entry=2, argv=0xbeffeb00, argv@entry=0xbeffeafc, images=..., options=...) at flif.cpp:395
#7  0x0012a94c in main (argc=2, argv=0xbeffeafc) at flif.cpp:706
 

Attachments

  • bwgradiant800x600.png
    bwgradiant800x600.png
    627 KB · Views: 351
OK, thanks for narrowing it down. So it's a problem in ColorBuckets. You can disable that transform during encoding with -B, and force it with -A.
 
Ah thanlks @_wb_ .
I tried with -B
Code:
/media/sda1/sources/git/FLIF/src$ ./flif -v ../bwgradiant800x600.png ../test05.flif -B
FLIF (Free Lossless Image Format) 0.2.0rc19 [26 Aug 2016]
Loading input file: ../bwgradiant800x600.png  Alpha channel not actually used, dropping it.
 (800x600)
flif: flif-enc.cpp:224: void flif_encode_FLIF2_inner(IO&, Rac&, std::vector<Coder>&, const Images&, const ColorRanges*, int, int, const int*) [with IO = FileIO; Rac = RacOutput24<FileIO>; Coder = FinalPropertySymbolCoder<SimpleBitChance, RacOutput24<FileIO>, 10>; Images = std::vector<Image>]: Assertion `curr <= max' failed.
Aborted
But it seems it's solthing else.
 
Commenting out the #define USE_SIMD in config.h seems to solve the problem.

I wonder if this is a compiler issue or a bug in the flif code.
 
@_wb_ : I'll test with commenting USE_SIMD.

In the mean time, I have tried to spot where behaviour differs between pc and pandora.
Using max verbose (7 I guess), I found this:
PC:
Code:
seb@mint64-VirtualBox ~/git/FLIF $ src/flif -v -v -v -v -v -v -v /mnt/pandora/media/sda1/sources/git/FLIF/bwgradiant800x600.png test05.flif -B
  ____ _(_)____
 (___ | | | ___)   FLIF (Free Lossless Image Format) 0.2.0rc19 [26 Aug 2016]
  (__ | |_| __)    Copyright (C) 2016 Jon Sneyers and Pieter Wuille
    (_|___|_)      License LGPLv3+: GNU LGPL version 3 or later

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Loading input file: /mnt/pandora/media/sda1/sources/git/FLIF/bwgradiant800x600.png  Encountered unknown PNG text chunk (key: Software), ignoring it.
Alpha channel not actually used, dropping it.
 (800x600, 3 channels, 8-bit)
Transforms: YCoCg, Bounds[0:0..244][1:-2..2][2:-2..2]
Plane 0: 0..244
Plane 1: -2..2
Plane 2: -2..2
Using pixel predictor method: -G???  ->  -G200
i=2, r=1, fr=0, c=0, curr=69, min=0, max=244
i=2, r=1, fr=0, c=1, curr=172, min=0, max=244
Learning a MANIAC tree. Iterating 2 times.
0% done [1/23] ENC[0,100x75]  i=1, r=1, fr=0, c=0, curr=0, min=0, max=244

and Pandora:
Code:
/media/sda1/sources/git/FLIF/src$ ./flif -v -v -v -v -v -v -v ../bwgradiant800x600.png ../test05.flif -B
  ____ _(_)____
 (___ | | | ___)   FLIF (Free Lossless Image Format) 0.2.0rc19 [26 Aug 2016]
  (__ | |_| __)    Copyright (C) 2016 Jon Sneyers and Pieter Wuille
    (_|___|_)      License LGPLv3+: GNU LGPL version 3 or later

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Loading input file: ../bwgradiant800x600.png  Encountered unknown PNG text chunk (key: Software), ignoring it.
Alpha channel not actually used, dropping it.
 (800x600, 3 channels, 8-bit)
Transforms: YCoCg, Bounds[0:0..244][1:-2..2][2:-2..2]
Plane 0: 0..244
Plane 1: -2..2
Plane 2: -2..2
Using pixel predictor method: -G???  ->  -G000
i=2, r=1, fr=0, c=0, curr=228, min=0, max=244
i=2, r=1, fr=0, c=1, curr=170, min=0, max=244
flif: flif-enc.cpp:225: void flif_encode_FLIF2_inner(IO&, Rac&, std::vector<Coder>&, const Images&, const ColorRanges*, int, int, const int*) [with IO = FileIO; Rac = RacOutput24<FileIO>; Coder = FinalPropertySymbolCoder<SimpleBitChance, RacOutput24<FileIO>, 10>; Images = std::vector<Image>]: Assertion `curr <= max' failed.
Aborted
So, it seems the function find_best_predictor used in flif-enc.cpp line 948 gives "2" for plane 0 on the PC and "0" on Pandora. Strange.
[doublepost=1473515875,1473515786][/doublepost]And I confirm, no SIMD fixed the Pandora build for me too.
[doublepost=1473516727][/doublepost]I still have the bug If I activate SIMD but disable it's use in transform/ycocg.hpp
Strange, as I thought this was the only place where SIMD is used.
I will leave SIMD out for now to have a working FLIF in codeblocks...
 
I think the issue is in image/image.hpp, around line 238, where there is some code to align the pixel data on a 16-byte multiple. This code is probably buggy or something, but the bug was probably not discovered yet because on most platforms, the data is already aligned.
[doublepost=1473522445,1473519200][/doublepost]OK. Should be fixed now. What a silly bug :)

https://github.com/FLIF-hub/FLIF/commit/eb4c294a3e0e25113c978fd6ea5951dd48fc1c8b
 
FWIW, I just built this at fdf0fc21 (v0.2 + fix #301) on my headless P3 server. Perhaps unsurprisingly, the compilation came with a far few warnings of the style:
Code:
maniac/../image/image.hpp: In member function ‘virtual FourColorVals ConstantPlane::get4(uint32_t) const’:
maniac/../image/image.hpp:373:64: warning: SSE vector return without SSE enabled changes the ABI [-Wpsabi]
     FourColorVals get4(const uint32_t pos) const ATTRIBUTE_HOT {

I guess that's because my Pentium 3 doesn't have any SSE instructions. But I've done a quick test of encoding and decoding on a couple of the included png files. The smaller tux with a limited palette comes back binary identical, but the photographic 512 line image came back slightly larger. Transferring those to my laptop though, they seem to look pretty similar to each other, if not identical.

It also seems to run a lot more quickly than old v0.1 did, so that's more good news. Thanks for all your hard work so far!
 
FWIW, I just built this at fdf0fc21 (v0.2 + fix #301) on my headless P3 server. Perhaps unsurprisingly, the compilation came with a far few warnings of the style:
Code:
maniac/../image/image.hpp: In member function ‘virtual FourColorVals ConstantPlane::get4(uint32_t) const’:
maniac/../image/image.hpp:373:64: warning: SSE vector return without SSE enabled changes the ABI [-Wpsabi]
     FourColorVals get4(const uint32_t pos) const ATTRIBUTE_HOT {

I guess that's because my Pentium 3 doesn't have any SSE instructions. But I've done a quick test of encoding and decoding on a couple of the included png files. The smaller tux with a limited palette comes back binary identical, but the photographic 512 line image came back slightly larger. Transferring those to my laptop though, they seem to look pretty similar to each other, if not identical.

It also seems to run a lot more quickly than old v0.1 did, so that's more good news. Thanks for all your hard work so far!

You can get rid of the SSE stuff by commenting out the #define USE_SIMD line in config.h. That should help to get it compiled on that Pentium 3 :)

Hi all :)

@_wb_ : how was the ICIP conference in Phoenix ?

Cheers, Magic Sam

It was quite interesting! I put my paper and slides on the FLIF website, in case you're curious: http://flif.info/publications.html
 
I'm working on a new lossless image compression algorithm, called FLIF.

I tried 0.2 and it's very good and without patent constraints it actually could go places - esp. considering it doesn't feature a generation loss after "lossy" optimization.

For widespread use, good metadata handling is paramount - it's supposed to support icc, xmp and exif? For this to happen, the support of the one ring app to rule them all would be very welcome - Phil Harvey's exiftool. Please be a dear and send him some flif samples with the respective metadata and a link to the spec, would you :) ...

http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,7665.msg38910.html#msg38910
[doublepost=1475956627,1475954640][/doublepost]Is metadata supposed to work in flif 0.2 anyway? I've tried and ...

* xmp: encodes ok, but xmp tag is garbled on decoding back to png
* exif: encodes ok, but tag is lost on conversion back
* icc: refuses to encode

See samples attached.
 

Attachments

  • flif02samples.zip
    472.1 KB · Views: 293
I tried 0.2 and it's very good and without patent constraints it actually could go places - esp. considering it doesn't feature a generation loss after "lossy" optimization.

For widespread use, good metadata handling is paramount - it's supposed to support icc, xmp and exif? For this to happen, the support of the one ring app to rule them all would be very welcome - Phil Harvey's exiftool. Please be a dear and send him some flif samples with the respective metadata and a link to the spec, would you :) ...

http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,7665.msg38910.html#msg38910
[doublepost=1475956627,1475954640][/doublepost]Is metadata supposed to work in flif 0.2 anyway? I've tried and ...

* xmp: encodes ok, but xmp tag is garbled on decoding back to png
* exif: encodes ok, but tag is lost on conversion back
* icc: refuses to encode

See samples attached.

About exiftool support: see https://github.com/FLIF-hub/FLIF/issues/307 :) (did one of you guys contact him or is this just a coincidence?)

The issue with XMP when writing PNG has been fixed meanwhile (see https://github.com/FLIF-hub/FLIF/issues/308).

There is no official standard for encoding Exif in PNG, so I'm currently not saving Exif in PNG.

What is the problem with ICC? PNGs with iCCP chunks should work fine as input. Are you getting any error/warning? Try -vvvvvvvv for more verbosity (the more -v's the more verbose)...
 
From what I can tell from the fabfeld site is that it's a simple binary format which you then compress using external binary compressors (like bzip2). That isn't suitable for parsing while it's being downloaded I think, as FLIF is, and in any case doesn't sort the data such that it only fills in the small details last.

I lost some jpeg data which were stored in a CD-R from about 10 years ago recently. I managed to see what could be pulled off using ffmpeg, and it's just the top half inch or so of most photos. If those files had been in FLIF format, I'd still be able to get reasonable approximations of the entire image, even in the cases where the drive couldn't track the data all the way through.
 
Well, the L in FLIF stands for lossless, and even if that's a lossless compression of lossy JPEG artifacts it still takes space. You could try cropping the extra 10 megabytes off the resultant FLIF file to turn it into a lossy file, and see which looks better - the JPEG or your cropped FLIF file.

I also wasn't aware that imagemagick has been updated to support FLIF. I assume that's the latest version, with the published 0.2 version being almost a year old.

Reminds me, I really should try out FLIF on my newly inherited dual core x64 machine. It's a little heavy for my little atom netbook, at least when processing big photos.
 
Back
Top