Free Lossless Image Format (FLIF)

_wb_ · Sep 10, 2016

Interesting. Could be an ARM vs x86 related bug...

ptitSeb · Sep 10, 2016

Yep, probably.

I started debugging, and also will try to compare execution with an x64 build for reference.

For now, the Corruption messages are from the "snap" function.
In my test iamge (a simple png screenshot), I count number of time "ColorRangeCB::snap" is called, and at occurence #577490, the bucket "b" seems empty, with its min and max to default value:

Code:

Breakpoint 1, ColorRangesCB::snap (this=0x232a70, this@entry=0x3b12c0, p=p@entry=2, pp=..., minv=@0xbeffe528: 10000, maxv=@0xbeffe524: -10000, v=@0xbeffe46c: 24) at transform/colorbuckets.hpp:300
300                 e_printf("Corruption detected in snap (%d)!\n", cnt);
(gdb) p b
$1 = (const ColorBucket &) @0x22f878: {snapvalues = {<std::_Vector_base<short, std::allocator<short> >> = {
      _M_impl = {<std::allocator<short>> = {<__gnu_cxx::new_allocator<short>> = {<No data fields>}, <No data fields>}, _M_start = 0x0, _M_finish = 0x0, _M_end_of_storage = 0x0}}, <No data fields>},
  values = {<std::_Vector_base<short, std::allocator<short> >> = {_M_impl = {<std::allocator<short>> = {<__gnu_cxx::new_allocator<short>> = {<No data fields>}, <No data fields>}, _M_start = 0x0,
        _M_finish = 0x0, _M_end_of_storage = 0x0}}, <No data fields>}, min = 10000, max = -10000, discrete = true}

I'll try to break at the same point (with a static counter) in my reference build to see the status of the "b".
[doublepost=1473499396,1473497218][/doublepost]Mmm, execution is very different it seems, even if on the same image

break point at the same loop:
on PC:

Code:

Python Exception <class 'RuntimeError'> Type is not a template.: 
Breakpoint 1, ColorRangesCB::snap (this=0x7a64f0, p=0, pp=..., minv=@0x7fffffffd94c: 0, maxv=@0x7fffffffd948: 47, v=@0x7fffffffd7ec: 11)
    at transform/colorbuckets.hpp:300

and on Pandora:

Code:

Breakpoint 1, ColorRangesCB::snap (this=0x232a70, this@entry=0x3b12c0, p=p@entry=2, pp=..., minv=@0xbeffe528: 14, maxv=@0xbeffe524: 17, v=@0xbeffe46c: 24) at transform/colorbuckets.hpp:297
297                 printf("STOP!\n");

I will have to change my approach. I'll start with converting simple small files and activate all the "print" function in the code and compare them...
[doublepost=1473500974][/doublepost]Of course, simple file just works... And gives amazingly small files (a 8x8 white box is 17 bytes, and a few colored points it 38 bytes). Large uniform bow also doesn't trigger (I assume this simple case have specifcs encoding path).

This gradiant (black/white in a 800x600 image) triggers the issue.
I activated all "print" code in colorbuckets.hpp and launch flif with -v
On the pandora, I got:

Code:

Loading input file: ../bwgradiant800x600.png  Alpha channel not actually used, dropping it.
 (800x600)
Filled color buckets with 0 discrete colors + 731 continous buckets
Saving Y Color Bucket:
Saving I Color Buckets

Saving Q Color Buckets

Y buckets:
C0..244
I buckets:
C-1..1 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-2..2 C-1..2 C-2..2 C-2..2 C-2..2 C-1..2 C-2..2 C-2..2 C-2..2 C-2..1 C-2..2 C-2..1 C-1..1 C-1..0
Q buckets:
  C-1..1 E
  C-2..2 C-1..0
  C-2..2 S0
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 S0
  C-2..2 C0..1
  C-2..2 S0
  C-2..2 C-1..0
  C-2..2 S0
  C-2..2 C0..1
  C-2..2 S0
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 S0
  C-2..2 S0
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 S0
  C-2..2 S0
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 S0
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C-1..0
  C-2..2 C0..1
  C-2..2 S0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 S0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C0..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 S0
  C-2..2 C-1..1
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C0..1
  C-2..2 S0
  C-2..2 S0
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 S0
  C-2..2 S0
  C-2..2 C-1..0
  C-2..2 C-1..0
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 S0
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 C-1..0
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C-1..0
  C-2..2 S0
  C-2..2 C-1..0
  C-2..2 S0
  C-2..2 C0..1
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 S0
  C-2..2 C0..1
  C-2..2 C-1..1
  C-2..2 S0
  C-2..2 S0
  C-2..2 S0
  C-2..2 C-1..1
  C-2..2 C0..1
  C-2..2 S0
  C-2..2 C0..1
  C-2..2 S0
  C-2..2 S0
  C-2..2 S0
  C-2..2 S0
  C-2..2 S0
  C-1..2 C-1..0
  C-2..2 S1
  C-2..2 C-1..0
  C-2..2 S0
  C-2..2 C0..1
  C-2..2 S0
  C-2..2 S0
  C-1..2 S0
  C-2..2 C-1..0
  C-1..2 E
  C-2..2 S0
  C-2..2 E
  C-1..2 E
  C0..1 E
Corruption detected!
Corruption detected!
Corruption detected!
Corruption detected!
Corruption detected!
Corruption detected!
flif: flif-enc.cpp:224: void flif_encode_FLIF2_inner(IO&, Rac&, std::vector<Coder>&, const Images&, const ColorRanges*, int, int, const int*) [with IO = FileIO; Rac = RacOutput24<FileIO>; Coder = FinalPropertySymbolCoder<SimpleBitChance, RacOutput24<FileIO>, 10>; Images = std::vector<Image>]: Assertion `curr <= max' failed.
  Aborted

I have the same on the PC until the "Corruption detected!". Instead, it just save with this output

Code:

94% done [23/23] ENC[2,800x600]  Writing checksum: 4CB1EECA
Wrote output FLIF file test05.flif, 212307 bytes for 800x600 pixels (3.5385bpp)

[doublepost=1473502355][/doublepost]on the Pandora, the first "Corruption detected!" message has this backtrace:

Code:

(gdb) bt
#0  ColorRangesCB::snap (this=0x1ff3d8, this@entry=0x3dc948, p=p@entry=2, pp=..., minv=@0xbeffe528: 10000, maxv=@0xbeffe524: -10000, v=@0xbeffe46c: 0) at transform/colorbuckets.hpp:295
#1  0x0004e130 in predict_and_calcProps_plane<Plane<short>, Plane<unsigned char>, false, false, 2, ColorRanges> (properties=..., ranges=0x3dc948, ranges@entry=0x1ff3d8, image=..., plane=..., planeY=...,
    z=z@entry=1, r=287, r@entry=333092, c=797, c@entry=2, min=@0xbeffe528: 10000, max=@0xbeffe524: -10000, predictor=0, predictor@entry=-1090525912) at common.hpp:295
#2  0x00048bac in predict_and_calcProps (properties=..., ranges=0x1ff3d8, ranges@entry=0x11f, image=..., z=1, z@entry=0, p=p@entry=2, r=r@entry=287, c=c@entry=797, min=@0xbeffe528: 10000,
    max=@0xbeffe524: -10000, predictor=0, predictor@entry=1) at common.cpp:148
#3  0x00051524 in find_best_predictor (images=..., ranges=0x11f, ranges@entry=0x1ff3d8, p=p@entry=2, z=0, z@entry=1) at flif-enc.cpp:163
#4  0x0006d708 in flif_encode<FileIO> (io=..., images=..., transDesc=..., encoding=<optimized out>, learn_repeats=2, acb=-1, palette_size=512, lookback=1, divisor=30, min_size=50,
    split_threshold=split_threshold@entry=349504, cutoff=cutoff@entry=2, alpha=1, alpha@entry=19, crc_check=<optimized out>, crc_check@entry=-1, loss=19, loss@entry=0, predictor=0xffffffff,
    predictor@entry=0xbeffe944, invisible_predictor=invisible_predictor@entry=2) at flif-enc.cpp:947
#5  0x00128bb4 in encode_flif (argc=argc@entry=1, argv=0x1, argv@entry=0xbeffeb00, images=..., options=...) at flif.cpp:387
#6  0x001299c8 in handle_encode (argc=1, argc@entry=2, argv=0xbeffeb00, argv@entry=0xbeffeafc, images=..., options=...) at flif.cpp:395
#7  0x0012a94c in main (argc=2, argv=0xbeffeafc) at flif.cpp:706

_wb_ · Sep 10, 2016

OK, thanks for narrowing it down. So it's a problem in ColorBuckets. You can disable that transform during encoding with -B, and force it with -A.

ptitSeb · Sep 10, 2016

Ah thanlks @_wb_ .
I tried with -B

Code:

/media/sda1/sources/git/FLIF/src$ ./flif -v ../bwgradiant800x600.png ../test05.flif -B
FLIF (Free Lossless Image Format) 0.2.0rc19 [26 Aug 2016]
Loading input file: ../bwgradiant800x600.png  Alpha channel not actually used, dropping it.
 (800x600)
flif: flif-enc.cpp:224: void flif_encode_FLIF2_inner(IO&, Rac&, std::vector<Coder>&, const Images&, const ColorRanges*, int, int, const int*) [with IO = FileIO; Rac = RacOutput24<FileIO>; Coder = FinalPropertySymbolCoder<SimpleBitChance, RacOutput24<FileIO>, 10>; Images = std::vector<Image>]: Assertion `curr <= max' failed.
Aborted

But it seems it's solthing else.

_wb_ · Sep 10, 2016

Commenting out the #define USE_SIMD in config.h seems to solve the problem.

I wonder if this is a compiler issue or a bug in the flif code.

ptitSeb · Sep 10, 2016

@_wb_ : I'll test with commenting USE_SIMD.

In the mean time, I have tried to spot where behaviour differs between pc and pandora.
Using max verbose (7 I guess), I found this:
PC:

Code:

seb@mint64-VirtualBox ~/git/FLIF $ src/flif -v -v -v -v -v -v -v /mnt/pandora/media/sda1/sources/git/FLIF/bwgradiant800x600.png test05.flif -B
  ____ _(_)____
 (___ | | | ___)   FLIF (Free Lossless Image Format) 0.2.0rc19 [26 Aug 2016]
  (__ | |_| __)    Copyright (C) 2016 Jon Sneyers and Pieter Wuille
    (_|___|_)      License LGPLv3+: GNU LGPL version 3 or later

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Loading input file: /mnt/pandora/media/sda1/sources/git/FLIF/bwgradiant800x600.png  Encountered unknown PNG text chunk (key: Software), ignoring it.
Alpha channel not actually used, dropping it.
 (800x600, 3 channels, 8-bit)
Transforms: YCoCg, Bounds[0:0..244][1:-2..2][2:-2..2]
Plane 0: 0..244
Plane 1: -2..2
Plane 2: -2..2
Using pixel predictor method: -G???  ->  -G200
i=2, r=1, fr=0, c=0, curr=69, min=0, max=244
i=2, r=1, fr=0, c=1, curr=172, min=0, max=244
Learning a MANIAC tree. Iterating 2 times.
0% done [1/23] ENC[0,100x75]  i=1, r=1, fr=0, c=0, curr=0, min=0, max=244

and Pandora:

Code:

/media/sda1/sources/git/FLIF/src$ ./flif -v -v -v -v -v -v -v ../bwgradiant800x600.png ../test05.flif -B
  ____ _(_)____
 (___ | | | ___)   FLIF (Free Lossless Image Format) 0.2.0rc19 [26 Aug 2016]
  (__ | |_| __)    Copyright (C) 2016 Jon Sneyers and Pieter Wuille
    (_|___|_)      License LGPLv3+: GNU LGPL version 3 or later

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Loading input file: ../bwgradiant800x600.png  Encountered unknown PNG text chunk (key: Software), ignoring it.
Alpha channel not actually used, dropping it.
 (800x600, 3 channels, 8-bit)
Transforms: YCoCg, Bounds[0:0..244][1:-2..2][2:-2..2]
Plane 0: 0..244
Plane 1: -2..2
Plane 2: -2..2
Using pixel predictor method: -G???  ->  -G000
i=2, r=1, fr=0, c=0, curr=228, min=0, max=244
i=2, r=1, fr=0, c=1, curr=170, min=0, max=244
flif: flif-enc.cpp:225: void flif_encode_FLIF2_inner(IO&, Rac&, std::vector<Coder>&, const Images&, const ColorRanges*, int, int, const int*) [with IO = FileIO; Rac = RacOutput24<FileIO>; Coder = FinalPropertySymbolCoder<SimpleBitChance, RacOutput24<FileIO>, 10>; Images = std::vector<Image>]: Assertion `curr <= max' failed.
Aborted

So, it seems the function find_best_predictor used in flif-enc.cpp line 948 gives "2" for plane 0 on the PC and "0" on Pandora. Strange.
[doublepost=1473515875,1473515786][/doublepost]And I confirm, no SIMD fixed the Pandora build for me too.
[doublepost=1473516727][/doublepost]I still have the bug If I activate SIMD but disable it's use in transform/ycocg.hpp
Strange, as I thought this was the only place where SIMD is used.
I will leave SIMD out for now to have a working FLIF in codeblocks...

_wb_ · Sep 10, 2016

I think the issue is in image/image.hpp, around line 238, where there is some code to align the pixel data on a 16-byte multiple. This code is probably buggy or something, but the bug was probably not discovered yet because on most platforms, the data is already aligned.
[doublepost=1473522445,1473519200][/doublepost]OK. Should be fixed now. What a silly bug

https://github.com/FLIF-hub/FLIF/commit/eb4c294a3e0e25113c978fd6ea5951dd48fc1c8b

ptitSeb · Sep 10, 2016

Updating my build... Will test when finished (but it takes several minutes to build).
[doublepost=1473525906,1473523709][/doublepost]@_wb_ : yes it works for me too

levi · Sep 25, 2016

FWIW, I just built this at fdf0fc21 (v0.2 + fix #301) on my headless P3 server. Perhaps unsurprisingly, the compilation came with a far few warnings of the style:

Code:

maniac/../image/image.hpp: In member function ‘virtual FourColorVals ConstantPlane::get4(uint32_t) const’:
maniac/../image/image.hpp:373:64: warning: SSE vector return without SSE enabled changes the ABI [-Wpsabi]
     FourColorVals get4(const uint32_t pos) const ATTRIBUTE_HOT {

I guess that's because my Pentium 3 doesn't have any SSE instructions. But I've done a quick test of encoding and decoding on a couple of the included png files. The smaller tux with a limited palette comes back binary identical, but the photographic 512 line image came back slightly larger. Transferring those to my laptop though, they seem to look pretty similar to each other, if not identical.

It also seems to run a lot more quickly than old v0.1 did, so that's more good news. Thanks for all your hard work so far!

Magic Sam · Oct 1, 2016

Hi all

@_wb_ : how was the ICIP conference in Phoenix ?

Cheers, Magic Sam

_wb_ · Oct 6, 2016

levi said:
FWIW, I just built this at fdf0fc21 (v0.2 + fix #301) on my headless P3 server. Perhaps unsurprisingly, the compilation came with a far few warnings of the style:

Code:

maniac/../image/image.hpp: In member function ‘virtual FourColorVals ConstantPlane::get4(uint32_t) const’: maniac/../image/image.hpp:373:64: warning: SSE vector return without SSE enabled changes the ABI [-Wpsabi] FourColorVals get4(const uint32_t pos) const ATTRIBUTE_HOT {

I guess that's because my Pentium 3 doesn't have any SSE instructions. But I've done a quick test of encoding and decoding on a couple of the included png files. The smaller tux with a limited palette comes back binary identical, but the photographic 512 line image came back slightly larger. Transferring those to my laptop though, they seem to look pretty similar to each other, if not identical.

It also seems to run a lot more quickly than old v0.1 did, so that's more good news. Thanks for all your hard work so far!

You can get rid of the SSE stuff by commenting out the #define USE_SIMD line in config.h. That should help to get it compiled on that Pentium 3

Magic Sam said:
Hi all

@_wb_ : how was the ICIP conference in Phoenix ?

Cheers, Magic Sam

It was quite interesting! I put my paper and slides on the FLIF website, in case you're curious: http://flif.info/publications.html

Marsu42 · Oct 8, 2016

_wb_ said:
I'm working on a new lossless image compression algorithm, called FLIF.

I tried 0.2 and it's very good and without patent constraints it actually could go places - esp. considering it doesn't feature a generation loss after "lossy" optimization.

For widespread use, good metadata handling is paramount - it's supposed to support icc, xmp and exif? For this to happen, the support of the one ring app to rule them all would be very welcome - Phil Harvey's exiftool. Please be a dear and send him some flif samples with the respective metadata and a link to the spec, would you

...

http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,7665.msg38910.html#msg38910
[doublepost=1475956627,1475954640][/doublepost]Is metadata supposed to work in flif 0.2 anyway? I've tried and ...

* xmp: encodes ok, but xmp tag is garbled on decoding back to png
* exif: encodes ok, but tag is lost on conversion back
* icc: refuses to encode

See samples attached.

_wb_ · Oct 17, 2016

Marsu42 said:
I tried 0.2 and it's very good and without patent constraints it actually could go places - esp. considering it doesn't feature a generation loss after "lossy" optimization.

For widespread use, good metadata handling is paramount - it's supposed to support icc, xmp and exif? For this to happen, the support of the one ring app to rule them all would be very welcome - Phil Harvey's exiftool. Please be a dear and send him some flif samples with the respective metadata and a link to the spec, would you ...

http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,7665.msg38910.html#msg38910
[doublepost=1475956627,1475954640][/doublepost]Is metadata supposed to work in flif 0.2 anyway? I've tried and ...

* xmp: encodes ok, but xmp tag is garbled on decoding back to png
* exif: encodes ok, but tag is lost on conversion back
* icc: refuses to encode

See samples attached.

About exiftool support: see https://github.com/FLIF-hub/FLIF/issues/307

(did one of you guys contact him or is this just a coincidence?)

The issue with XMP when writing PNG has been fixed meanwhile (see https://github.com/FLIF-hub/FLIF/issues/308).

There is no official standard for encoding Exif in PNG, so I'm currently not saving Exif in PNG.

What is the problem with ICC? PNGs with iCCP chunks should work fine as input. Are you getting any error/warning? Try -vvvvvvvv for more verbosity (the more -v's the more verbose)...

Silent-Hunter · Sep 13, 2017

Hey so, is FLIF suitable for general use yet, or is the protocol still changing?

Magic Sam · Sep 13, 2017

Hi all

@Silent-Hunter :

The file format has been standardised and versioned. The latest stable version is FLIF16 and is documented here

Source: http://flif.info/

[doublepost=1505325550,1505324425][/doublepost]@_wb_ : how is FLIF support in PIV going ?

EDIT: I stumbled upon farbfeld some days ago:

https://tools.suckless.org/farbfeld/

Opinions ?

Cheers, Magic Sam

levi · Sep 13, 2017

From what I can tell from the fabfeld site is that it's a simple binary format which you then compress using external binary compressors (like bzip2). That isn't suitable for parsing while it's being downloaded I think, as FLIF is, and in any case doesn't sort the data such that it only fills in the small details last.

I lost some jpeg data which were stored in a CD-R from about 10 years ago recently. I managed to see what could be pulled off using ffmpeg, and it's just the top half inch or so of most photos. If those files had been in FLIF format, I'd still be able to get reasonable approximations of the entire image, even in the cases where the drive couldn't track the data all the way through.

ible · Sep 14, 2017

farbfeld sucks. FLIF rules

. but still need to understand it somehow...

levi · Sep 14, 2017

Download it and start using it then!

Silent-Hunter · Sep 14, 2017

I used ImageMagick to convert a JPEG to a FLIF, and it made it 10 megabytes bigger.

levi · Sep 14, 2017

Well, the L in FLIF stands for lossless, and even if that's a lossless compression of lossy JPEG artifacts it still takes space. You could try cropping the extra 10 megabytes off the resultant FLIF file to turn it into a lossy file, and see which looks better - the JPEG or your cropped FLIF file.

I also wasn't aware that imagemagick has been updated to support FLIF. I assume that's the latest version, with the published 0.2 version being almost a year old.

Reminds me, I really should try out FLIF on my newly inherited dual core x64 machine. It's a little heavy for my little atom netbook, at least when processing big photos.

Free Lossless Image Format (FLIF)

Microbe

Serial Porter

Attachments

Microbe

Serial Porter

Microbe

Serial Porter

Microbe

Serial Porter

Still fresh, damnit!

Forever Homebrew

Microbe

Still Fresh

Attachments

Microbe

Hardcore Member

Forever Homebrew

Still fresh, damnit!

professional vim user

Still fresh, damnit!

Hardcore Member

Still fresh, damnit!