I don't think it will speed it up a lot if you throw swapspace on a slow sdcard on it.
You could change some general things in about:config:
http://www.ubuntugeek.com/speed-up-firefox-web-browser.html
Overclocking helps a bit too. Remember that this is the full blown desktop firefox.
The optimized Version is fennec, that is made for mobile devices.