Share your projects

[netcat]

So data caches are kinda trendy these days and I always thought most were crap because they use TCP and do all that copying. Wouldn't it be faster and cooler if it just used shared memory? Then I realised I could do it in a decentralized, serverless way if I use compare-and-swap atomic cpu operations and memory barriers. Just got rid of the last race condition my stress test revealed so this is nearing completion. It's called shampoo.

Shampoo has two data structures, a hash and a circular heap. This is how it hangs together:

A lot of very fiddly (nasty) pointer programming. One of the most complicated things I have ever programmed. I'm glad pointer arithmetic is considered unsafe these days. It is.

GitHub - turborat/shampoo: The fastest data cache imaginable

The fastest data cache imaginable. Contribute to turborat/shampoo development by creating an account on GitHub.

github.com

Comments appreciated

 

[kuru]

I have not learned much about IT but I learned to hate pointers
@netcat
Can you explain this wizardry and its surely nefarious uses in layman's terms?

 

[netcat]

so "head" points to the next free memory location in the heap. if you want to reserve 50 bytes you need to update head=head+50 so you get those bytes and no one else takes them. i use compare-and-swap so if head gets updated concurrently it's detected and you can try to grab the next 50. it's shared data with no locking or central process. simpler = faster. that's it. if the pointer arithmetic is ever wrong it's usually a jump to some random invalide memory segment (aka segfault) or just hangs and it's usually not worth diagnosing it's too low level so i just keep undoing changes until it's back to normal.

 

[Djoga'Ro]

but I learned to hate pointers

Don't be like that. They're just variables, who's values are addresses in memory.

 

[kuru]

Don't be like that. They're just variables, who's values are addresses in memory.

Addresses, huh?
*pumps his shotgun*
Dem folks're lucky I dunno where dey live.
*spits, pumps his shotgun again*

Nah. I did fiddle around with em in cpp and still don't quite know why I inflicted it on myself. I did get to a place where I could work with it, but it was fairly basic stuff. Guess I've not run into the need to manually manage memory yet. Using something like Godot is just too convenient for the silly little things I do.

 

[Djoga'Ro]

@kuru Often it's way more efficient to communicate where in memory something is than to copy said something (at every step).
The error-prone tasks when dealing with pointers are
- not to fuck up one's pointer arithmetic
- always have a clear concept of pointer (or rather memory (per allocation)) ownership in mind
in cpp std::shared_ptr/std::weak_ptr and std::unique_ptr help with the latter (steer clear of raw c pointers whereever possible (or be very mindful))

 

[TeDaDeS]

A long time ago I implemented a void pointer array in C for an embedded OS, which was a hassle.
While I like you knew what was happening and solving a technical challenge, the modern programming languages make life much easier.

 

[netcat]

Let me give another example of pointer shenanigans, in C. To explicitly control memory layout you can cast a certain region of memory to an object or struct.

byte[] memory = byte[1024]; // allocate memory
byte* mem_ptr = memory; // acquire pointer to memory
MyObj *obj1_ptr = (MyObj*) mem_ptr; // use the beginning memory for obj1
MyObj *obj2_ptr = (MyObj*) mem_ptr + 36; // use later bytes for subsequent objects
MyObj *obj3_ptr = (MyObj*) mem_ptr + 72;

Imagine if the offset (36, 72) is wrong. What happens? You get overlapping objects, misinterpreted data aka random crashes.

Now this is a trivial example, in shampoo the objects all have different lengths.

So pointer arithmetic is simple math but when you get it wrong there usually aren't friendly error messages. That's it!

I think this kind of programming - making the most of resources via low level constructs - is common in video games actually.

I should have been a video game programmer ..

 

[Djoga'Ro]

@netcat Don't you have `sizeof` in C?