I would recommend getting an HP 200LX or 100LX. They're common enough on eBay but still pretty exotic nowadays.
They come with an almost complete MSDOS 5.0 and an 8088 compatible processor but fit in your pocket and run for weeks on AA batteries. LX palmtops also use standard PCMCIA flash cards so data exchange with a modern PC is easy.
They also have a very nice DOS GUI addon called System Manager that has a great calculator app and appointment book software (there are features it has that I've still never seen decades after its 1991 release). There are other apps like text editing, spreadsheets, and databases as well which are decent enough.
They also can charge NiCad batteries so running them is very economical if you buy the charger.
LXes spawned an ecosystem that was very vibrant decades after the fact and still continues to a lesser extent even today. I highly recommend one if you want some esoteric PC system that's not very expensive.