At least once per year, I have to reverse-engineer a binary format. I just like tinkering with it and the detective work needed for it. Fortunately, there is always someone who needs this and has no time, software or skills to do it. So a friend got me the files, however, he warned that this time it will be the hard one - OK, if it's not encrypted. The thing is that it was not so much hard, but tedious and just outwards sick. I was thinking that I could explain my ramblings by making interoperability with some CADs (as in EU to do it legally it must be for interoperability), screw this, I had to do it to maintain interoperability with the euclidean 3D universe, general causality and the leftovers of sanity I saved after analyzing these files.
This time I got a strange thing in my hands - a closed and proprietary format of computer simulations (and it's not my first contact with FEA) for a very hermetic, expensive and futuristic software for process design, optimization, manufacturing and organizing. The thing is developed for 35 years and probably some parts have never been modified. I could not get the software, but files - yes, in any amount, with process and result visualizations. With this and the hex editor, I started to chew thru the binaries.
These files are usually used as input for solver, the big program which calculates a lot and its binary input is usually some simple mutation of Fortran or C-like data structures for a simple reason - programmers have no time to write complex things for the I/O, it's more important to solve this 400k x 400k sparse matrices problem than to pack integers and floats. It is linearly slurped at the beginning and linearly spewed out at the end. There is a set of meshes, described usually by vertices and links between, the description of which side goes where, and some sets of values (BCs or results) in points, sometimes more points than vertices (so they are centers of the mesh elements then), or surfaces, or 3D. Simple. Sometimes the method of packing data to bytes may be strange, but it's usually not a terrible thing, many times being a result of some past hardware dependency.
You know this old saying, that who works in Vienna sausage factory never eats Vienna sausages. Now I'm out when it comes to using this "software".
What can we expect? Encrypted files? No. Maybe carefully crafted data structures for describing specific things, and then alternate way of describing it, being a quick memory barf, like the big graphics company starting with letter A did numerous times? No! This is even worse. The terrible soup of text and binary, of data and executable commands, of multiple definition of blocks... Of units definitely not fitting any engineering software in this Earth... This is not nuts, this is a thing straight from some nightmare:
- Time is usually stored in seconds. Entire process time is in units about 30.1M seconds. There is nothing in between.
- In dimensions, the main Scalar unit can be safely approximated as 0.9mm, but the way it is stored leaves some bit garbage in last places after comma. Program translates everything to mm, trimming these bits.
- Then, there is THE unit used in the 3D coordinates, computed who-knows-how, but stored in a control block once and used since then per computation. This is a 3D block having VxWxX Scalar units counted righthandly. All 3D sizes, coordinates and such are written using THE unit in floats.
- If anything has to be written as a 2D surface, it is written using only W units, floats only. 3D Bounding boxes for fields are considered as 2D. Cross-sections are considered still 3D.
- Distances are in Scalar units. And this ends floats in dimensions.
- Result floats are scaled from integers using pre-defined ranges.
- The 4-byte signed integer is the holly binary unit, used nearly everywhere and for nearly everything except dimensions and time. If something does not fit to it, it's zero-padded.
Now the file itself. Some files had proper magic numbers, some not. These which had no numbers, had their versions detected somehow, and I found how by throwing e-mails with person having software: Program determines version by file's modification date. Of many don't-s in programming this seems to be the worst thing as there are so many events in which this gets messed up.
The file contains extensive amount of metadata, including the "cryptographically-secure" storage of user-sensitive information in strings: Lengths are stored like C# compressed uints, but it's the first, not the last bit for continuing and... the strings are written backwards

. Network data for MPI are "encrypted" by stuffing \0s between each byte, this time without reversing. Absolute paths are everywhere and it's UI's job to figure where is what. Paths are written in mesh names, and it's the UI job to separate this.
To write the surface distribution of values, the mesh gets UV mapping and something surprisingly similar to multi-channel TGA file with incomplete header. One channel-one scalar. I don't know who invented this, but gets the McGyver's award for the most lazy description of 3D mesh surfrace. On the other hand, I could make a C++ program which allows to paint surface conditions in GIMP

. No preprocessor has this feature!
If the user is tempted to name the file specific way, the version gets upgraded and downgraded specifically. Found by accident.
And there is a "extract" block, as I called it. Any binary or text structure can be in this block and this will be happily extracted to simulation directory when the file is processed by solver. If the extracted file has specific name, the contents will be downloaded without question. I made files which successfully downloaded files from other directories, SMB shares, FTP servers and HTTP sites, the only thing I not tested was Gemini and Gopher. Well, this could be a smoking-gun evidence of a backdoor, but it is actively used in multi-machine computation to obtain missing data from another computer.
Finally, the file has a checksum! Not that it's checked, only shown in program's log at the end of loading so I could craft a file with borked checksum and it was still operable, but I just had to determine how it's calculated. I found that it depends on certain sizes and numbers of structures, but there was always something else. Only adding totally blank data records could increase the checksum by one or two, depending on number of headers it had to write. Adding a comment to the file, for example, could change the checksum, but could leave it untouched, and I found that it mostly depends on the length of the comment's starting, not the comment itself. Finally, the person having the program sent me the report of program's crash (crafting the file with 1500 empty meshes was a bad idea), in which I found that the file is loaded using a 256-ints buffer. Now everything clicked in - the checksum contained the sum of it! The structure read from file in a specific step is not especially 256-ints long, and the buffer is never cleared, only the \0 is set. It means that loading/saving file is not a procedure, but a ritual which, when done wrong, will result in properly loaded file and improperly calculated checksum.
I wonder what was thinking of the developer who implemented this. Some of these things could be results of per-architecture optimizations (data lengths), some of them for some kind of computation method maybe (integer scaling of floats), but the checksum to verify the methodology of loading a file? Non axis-uniform unit system in which the Knuth's Potrzebie System of Measures is pleasant to use? Timing processes taking seconds in units a bit smaller than year?