Wifi would be a useless expense for the people that don't won't it, so I'd say make it optional - perhaps give two SD ports, one supporting SDIO so you could purchase a Wifi card if you wanted it without adding price to the basic unit.
So,
64MB of RAM (at least 50MB of which should be usable by apps)
400Mhz ARM processor, and if they decide to use two, give the second a dedicated amount of it's own memory (couple of MB should be enough as long as it can share the 64MB as and when required, which shouldn't be a problem)
Basic 2D GPU that supports scaling and blitting. Don't bother with fancy stuff like 3D that 1 application out of 1000 will use.
Dual SD slots, with at least one of those supporting SDIO.
320*240 screen or multiple thereof (eg. 640x480 if not much price difference)
Nice responsive dpad.
Li-ion battery that can be charged in the unit, but with support for AA's as a replacement (that go in the same place)
Decent audio hardware - handling multiple channels in hardware would be nice, along with hardware volume control (like on the GP32)
and some things that would be nice, but less practical:
OLED keys that can be configured to match the emulated system (eg. emulate a snes, and they light up as A,B,X,Y, emulate a PSX and they change to triangle, etc)