Einstein Would Be Magnificent

[ashdjones]

Do you know if there's any documentation out there on the OS? An SDK would go a long way too, was it an open development platform at all?

http://www.unna.org/view.php?/development/NewtonDev/parts

Non Mac users will need this to unpack the files (.sit files don't seem to supported by anything, not ever Winrar):

http://www.brothersoft.com/stuffit-expander-3992.html

 

[daffy]

I believe that since the Newton used an ARM processor, what you would need is something more like virtualization than emulation. So I'm actually hoping that a 600MHz Pandora would run Newton software like a 600MHz Newton would have.

But I could be wrong. Wouldn't be the first time.

 

[Exophase]

I believe that since the Newton used an ARM processor, what you would need is something more like virtualization than emulation. So I'm actually hoping that a 600MHz Pandora would run Newton software like a 600MHz Newton would have.

But I could be wrong. Wouldn't be the first time.

Right, as I mentioned I think an emulator in the normal sense is inappropriate. However, so is straight up virtualization of the full hardware platform, because that involves emulating the address space and chances are poor that you'd still be able to run native code. What you want is "user mode" emulation like WINE or some modes of qemu. This means that the application has to care little about where things like code space, data space, stack, and heap are allocated and just needs to use the pointers it's given. It also means that it has to access the operating system and any hardware functions purely through a well defined, high level interface.

Ideally you'd like to be able to run the program without any modification, but in reality even providing an swi interface while running it in Linux could be hard - it might be possible with some hacks but I doubt that Linux knows how to task switch swi handlers. So you probably need to scan and patch the program ahead of time, but this requires that you clearly know what's code and what's data, and that the program follows this during runtime and doesn't modify anything. Anything that requires you to modify the code beyond 1:1 changes will require full on dynamic recompilation because code locations will change.

A lot of hardware, for instance modern x86 CPUs, have virtualization facilities that will trap on situations like these so you don't have to patch the code. Unfortunately, ARM has very limited ability to raise exceptions on potentially privileged instructions. There might be some support in TrustZone, but little is actually publicly known about it.

 

[daffy]

Right, as I mentioned I think an emulator in the normal sense is inappropriate. However, so is straight up virtualization of the full hardware platform, because that involves emulating the address space and chances are poor that you'd still be able to run native code. What you want is "user mode" emulation like WINE or some modes of qemu. This means that the application has to care little about where things like code space, data space, stack, and heap are allocated and just needs to use the pointers it's given. It also means that it has to access the operating system and any hardware functions purely through a well defined, high level interface.

Ideally you'd like to be able to run the program without any modification, but in reality even providing an swi interface while running it in Linux could be hard - it might be possible with some hacks but I doubt that Linux knows how to task switch swi handlers. So you probably need to scan and patch the program ahead of time, but this requires that you clearly know what's code and what's data, and that the program follows this during runtime and doesn't modify anything. Anything that requires you to modify the code beyond 1:1 changes will require full on dynamic recompilation because code locations will change.

Except this may not be able to run the built-in programs. They may use all kinds of private APIs or be tightly and horrendously bound to the OS. And while there are many cool Newton programs, what you'd be after here is the magic paper notetaking features.
Plus, Einstein already exists, and runs on the N800. So I was hoping someone who understood such things better than I could take a quick look at what it would take to make it work on Pandora.

 

[Exophase]

Except this may not be able to run the built-in programs. They may use all kinds of private APIs or be tightly and horrendously bound to the OS. And while there are many cool Newton programs, what you'd be after here is the magic paper notetaking features.
Plus, Einstein already exists, and runs on the N800. So I was hoping someone who understood such things better than I could take a quick look at what it would take to make it work on Pandora.

Okay, I'm not sure you understand, but if you want a "600MHz Newton" then you won't achieve it with full hardware emulation. If user mode emulation isn't an option then chances are that virtualization will not be; the ARM MMU alone won't save you, especially when you have to share it with Linux.

I don't know much about Einstein right now, but I doubt it's intended to be anywhere near 100% native speed, as in, the speed you'd get if you were running that kind of software straight on a Pandora. It does have a recompiler, but my best attempts to determine its workings suggest that it's more of a threaded interpreter with huge function tables than a native code generator.

 

[daffy]

Okay, I'm not sure you understand, but if you want a "600MHz Newton" then you won't achieve it with full hardware emulation. If user mode emulation isn't an option then chances are that virtualization will not be; the ARM MMU alone won't save you, especially when you have to share it with Linux.

I don't know much about Einstein right now, but I doubt it's intended to be anywhere near 100% native speed, as in, the speed you'd get if you were running that kind of software straight on a Pandora. It does have a recompiler, but my best attempts to determine its workings suggest that it's more of a threaded interpreter with huge function tables than a native code generator.

Oh, hey. I didn't realize you were looking at it. Many thanks for taking the time.

FWIW, Newton still has a dedicated following (a what is it -- 12 years old? -- Newton 2100 will still fetch $100 or so. How many 12 year old computers does that apply to?).

If Einstein can be gotten to work at all well, a bunch of those folks would probably buy Pandoras. And they are by definition geeky and dedicated folk we would love to have in our community.

 

[matthiasm]

Hi. I have ported Einstein to MSWindows during a Newton Show in Tokyo a few years ago. I know the code pretty well. If you are interested, I can give y'all in depth information. With a team of three or four dedicated developers, we could get something great going! Please read on...

Emulating the Newton is quite involved. Einstein has to interpret every command (it uses JIT) and then emulate memory access through a software MMU. Paul emulates the rest of the hardware in native code geniusly! But the first two features are quite time consuming, even with tons of optimizations in place. So with the current implementation of Einstein, your Newton will run about 200 times slower than the original. On a 2GHz machine, that i just bearable.

So, yes, you would easily get Einstein ported to your platform, but the resulting emulation would be too slow for any serious work.

Now I read someone mention that simulation is the way to go, and it would be, but two things are in the way: Newton OS uses ARM in MSB mode, but all newer devices use LSB first, so 1:1 simulation is incompatible. Secondly, Newton OS uses the MMU, which still would need to be emulated (unless the host OS gives us full control, which it won't). So yes, simulation is possible, but quite tricky.

But Apple has left us with on insanely great gift: as part of the developer kit, we received the ROM and all C and C++ symbols! From that it is relatively easy to regenerate the full ROM source code in assembler, or even a pseudo C++ code. That code can then be compiled again and runs natively at full speed.

Of course there are a few stumbling blocks: mainly, the MMU still needs to be emulated, but since the source code to the ROM would now exist, maybe the dependency on the MMU can be removed altogether? If that feat could be written, Newton OS wold run as a simple user application on any platform, giving great HWR an access to a library of a few thousand software titles.

As I said, we would need three or four developers ;-) . Any takers?

Matthias

 

[sphinx32]

If Einstein emulator working on OP, that will be great.

I already preordered openpandora, and this was one of the main reason of using Einstein on pandora
hopefully.

 

[grasshoppir]

what a failpad

 

[sphinx32]

I think it's a good idea to add Einstein (Newton OS) to "port request page" to consider by more developers.

http://pandorawiki.org/Port_Requests

 

[Exophase]

Emulating the Newton is quite involved. Einstein has to interpret every command (it uses JIT) and then emulate memory access through a software MMU. Paul emulates the rest of the hardware in native code geniusly! But the first two features are quite time consuming, even with tons of optimizations in place. So with the current implementation of Einstein, your Newton will run about 200 times slower than the original. On a 2GHz machine, that i just bearable.

No offense to Einstein's original author, but for something that just has to low-level emulate an ARM610 with an MMU 1:200 is a not a good ratio. This is true even for an interpreter, much less a recompiler (or JIT, as it's often referred to), even if it's the threaded interpreter type. I don't know what kind of optimizations were made, but some aspects of its design must be holding it back tremendously.

Eemulating a GBA is probably at least as demanding since it has a CPU that's nearly as fast (16MHz ARM7TDMI) but also substantial 2D hardware functionality. Devices significantly weaker than Pandora can handle GBA emulation acceptably, so I think that emulating the original Newton shouldn't be a problem for Pandora. Whether or not Einstein can be made up to the task, I don't really know. But from the sounds of it daffy wants something far beyond emulating an original Newton.

 

[daffy]

from the sounds of it daffy wants something far beyond emulating an original Newton.

Not sure what I said that gave that impression, or even what that would be.

 

[Exophase]

Not sure what I said that gave that impression, or even what that would be.

"The Newton devices were great, but the hardware was not quite fast enough to really live up to the idea."
"So I'm actually hoping that a 600MHz Pandora would run Newton software like a 600MHz Newton would have."

You're being pretty clear. Emulating a Newton means emulating it at the speed it ran at, not what you're describing.

 

[daffy]

Really?

One could happily distinguish virtualization (same processor, trapping calls to the virtualized system) from emulation (interpreting or JITing the original code so you can rewrite what it's calling to on the fly) I think. But requiring that either run at exactly the same speed as the original seems peculiar.

If I use SixtyForce and it winds up that the game I'm playing runs at 98% or 103% of the original speed, does that then not count as emulation?

Surely what you're after is something that replicates the important aspects of the original. In the case of a game, an emulator should run at close to the same speed. But in the case of a piece of workaday software, one would want it to run as fast as possible. I don't think that stops the result from being called an "emulation".

If I'm misusing some technical term, then I'm sorry, and it was not my intent. I just want something that replicates the utility of the Newton software.

 

[Exophase]

Why are you arguing with me over this? You were the one who misunderstood what I was saying, not the other way around. Do you just like arguing with people? Try to consider the actual context of my point, which is that while a Pandora could probably emulate a Newton at its original speed doing it many times faster is a much different story.

At any rate, Einstein struggles to emulate a Newton at the original speed on a > 2GHz current PC, so I think we know where that emulator in particular stands.

 

[daffy]

Why are you arguing with me over this? You were the one who misunderstood what I was saying, not the other way around. Do you just like arguing with people? Try to consider the actual context of my point, which is that while a Pandora could probably emulate a Newton at its original speed doing it many times faster is a much different story.

At any rate, Einstein struggles to emulate a Newton at the original speed on a > 2GHz current PC, so I think we know where that emulator in particular stands.

Sorry, not trying to pick a fight; that just seemed a funny definition of emulation.

 

[absolofdoom]

Sorry, not trying to pick a fight; that just seemed a funny definition of emulation.

You should really stop. But, I'll be nice.

"Emulating a Newton means emulating it at the speed it ran at, not what you're describing."
This does not say that emulation is defined as running something at the speed it ran at, this is saying that FOR THIS CASE emulation at a higher speed than the original is beyond what can be done, that we would be aiming for 1:1.

Hope that clears it up.

 

[timofonic]

Sorry, not trying to pick a fight; that just seemed a funny definition of emulation.

You should really stop. But, I'll be nice.

"Emulating a Newton means emulating it at the speed it ran at, not what you're describing."
This does not say that emulation is defined as running something at the speed it ran at, this is saying that FOR THIS CASE emulation at a higher speed than the original is beyond what can be done, that we would be aiming for 1:1.

Hope that clears it up.

I think this is an unconstructive and absurd discussion. If someone want to discuss the philosophy about what is "emulation", please create a forum thread for that.

This thread is about emulating Apple Newton on Pandora, something I consider quite interesting for different reasons (historial, entertaining and discovering another classic yet interesting platform). Please back on topic, we found even a nice developer interested on it and asking for collaboration! Please don't scare him!

Less pointless writing, more to the point discussion

Exophase, I'm very interesting about your thinking that Einstein could be heavily improved in efficiency. Matthiasm could help about how Einstein works and then try to ellaborate some way to making it fast.

A very nice idea could be to go into the code repository and start hacking in an experimental branch, if the devs are there and get convinced to provide the access rights after some productive code hacking. If not create a fork and name it Einsteinplus or whatever you want. Then later there could be specific optimizations for ARM platforms living into the main code in a structured manner (no spaguetti), including OMAP

Matthiasm and other devs: Would you like to lead this effort? Just want to know, I have curiosity

About sites for asking to the community, I found some of them:

http://www.newtontalk.net
http://newtonpoetry.com
http://www.newted.org
http://www.unna.org
http://myapplenewton.blogspot.com
http://newtonsearch.net
http://www.pda-soft.de

There's even a web server for Apple Newton: http://npds.free.fr
Books for Apple Newton. It really was the first ebook device: http://www.newtonslibrary.org

 

[matthiasm]

Um, I am afraid I got my numbers wrong. Emulation speed *was* 200:1. JIT raise the bar to 40:1 IIRC. It is usable on my Macbook Pro.

 

[absolofdoom]

I'm not discussing "the philosophy of emulation," I'm saying exophase wasn't defining it in the first place.

Edit: On topic, I've heard of the newton many times, but I had no idea it did so much. That vid made me want to buy one, but they're still rather expensive as is to be expected. I think it would be rather great if we could get this emulated well; it would definitely cater to people who want to use the pandora for pda-type functions.