what like a thin cleint? You can basically do this now on the current pandora with a remote desktop view. You would still need a processor etc for that, might as well just keep the pandora as is, the current SoC is not too much more than what you would want for something like this, I wouldn't change the current design for connecting to seperate computer/device.
EDIT: also you can eliminate android for remote control, I'm not familiar with a remote control application.... actually I think one exists, but I'm not familar with it and I dont' think it's over wifi... have to get back with you on that one.
EDIT2: nevermind found some vnc server apps wthat work over usb, and wifi but you need root access
What I was suggesting is the equilivant to a motherboard upgradeable pandora, all the things that make the pandora awesome would stay there, just remove the parts that could be updated to [insert bleeding edge spec'd device here]
you could even do something simple as battery to usb dongle (power only) hdmi to display (video) power the lcd from that battery too. For the keyboard + controller + analog you can have it as a usb slave like a usb joypad. Android 4.x (comes standard on the AI dongle) supports usb host and the raspi with linux obvioulsy will work hardware willing. If that is too complicated just do bluetooth connection instead of usb slave.
You can always take one of these dongles and plug it into the pandora and run your remote connetion there... but what a waist of battery running 2 systems at once, on a portable.... or seperate power supply for the dongle in your pocket would be a pain in the ass to setup etc... not what I'm looking for on my 5 min breaks at work.