DJWillis sent me the compilation toolchain he uses on Windows for Open2x. Since it's pretty much up to date and worked for me whereas other toolchains didn't work right for me, I'm sharing.
Works out the box without having anything to setup/build, except maybe environment variables, although I don't use them.
In case it can be of any use to anyone, here's the command line I use to compile in cygwin using this toolchain (provided that it's in C:devkitGP2X) :
/cygdrive/c/devkitGP2X/bin/arm-linux-gcc main.c -o main.gpe -Os -static -I'C:/devkitGP2X/include' -L'C:/devkitGP2X/lib' -lSDL -lSDL_image -lpthread -lSDL_mixer -lSDL_ttf -lSDL -lpng -ljpeg -lvorbisidec -lmad -lfreetype -lz -lm -msoft-float