Html5? Remote Desktop? Playable?

[Grench]

I noticed a few things around cloudesque computing and gaming.

Looking at LTSP:
http://en.wikipedia.org/wiki/Linux_Terminal_Server_Project

And at lots of articles on HTML5

And thinking about games that I want to play on the Pandora that are likely to never get ported (Neverwinter Nights, etc...)

Maybe this already exists...

Has anyone come up with a method to start an LTSP style terminal on a mobile device (Pandora) that has smooth enough video and low enough control lag to play games through?

I.e. can I open a session to my big arse home workstation/server and have my processing done on a behemoth back end while only passing pre-rendered video downstream and control use upstream?

 

[Grench]

Oooohhh... could someone try this on a Pandora?

http://sourceforge.net/projects/thinvnc/

 

[Blue Ion]

I've tried a fair number of things within my Pandora and they all work as well as you would expect. RDP, VNC and piping X programs to a local X server all worked fine for me.
The only problem I have was that apparently I can't get over 100 Kbytes/s creating a fairly big bottleneck, specially when piping X windows since they are far from bandwidth friendly.

Additionaly, SMB and NFS also work fine with the aftermentioned wifi slowness.

However, I don't think any game that uses 3d acceleration will work (well at least) through those protocols as they are meant for static window programs.

I've tried Dosbox trough RDP and while it works fine with a small window, mouse was drunk and unusable and no direct3d program actually started (they all froze). Windows 7 SP1 may fix this to some sort, but I doubt this will get backported to Windows XP, let alone rdesktop (where I usually do the connections).

Before expecting anything, I suggest you try at your own home, with two computers and see if you can find a solution. I didn't and both computers where connected on a 100Mbit Lan.
Your only solution would be commercial ones, but I seriously doubt they have linux arm clients or the source code open to create one yourself.

 

[atomicthumbs]

Ion, what'd you use for RDP?

 

[Grench]

Oooohhh... could someone try this on a Pandora?

http://sourceforge.net/projects/thinvnc/

Operating System: All 32-bit MS Windows (95/98/NT/2000/XP)

Sorry, boy

Try again. Operating system for the HOST is Windows.

The CLIENT (Pandora) is any browser with HTML5. Is Chrome HTML5 compliant?

 

[Blue Ion]

Ion, what'd you use for RDP?

A month ago my working computer stopped booting and considering that during its 6 years of service I haven't treated it well, I figured this time it could be serious. However I had to deliver some assignments for the University and study for the finals and I couldn't do a full check, so I decided to pull out a 10 year old laptop that has 256 mb of ram. It runs xfce fairly well and thats all I needed for work, but trying to do some heavy duty with firefox, rssowl or netbeans killed the machine.
So in a minute I set up an other account a the family machine (a Windows 7 computer with two monitors for concurrent sessions) so I could keep using those and other cpu or memory heavy programs, I could even watch flash video incredibly smooth.

Also I was beating Magic Carpet 2 on Dosbox (excellent game, too bad it's buggy on Dosbox) when my computer broke down, that is why I tried a couple of times, but probably I wouldn't have used it even if it worked.

Is Chrome HTML5 compliant?

It's been supported since version 3, but considering support as well as HTML5 itself is still a work in progress, it may suffice with version 4 or be needed version 5 of chromium.

 

[Grench]

So, in theory since the Pandora has the best and most versatile controls it could become the model mobile device for these kinds of services:

http://www.cnn.com/2010/TECH/gaming.gadgets/06/16/e3.cloud.gaming/index.html

I know there is a lot of emphasis on getting emulators and locally executing software going - that is all good.

What about using the Pandora as a portal into the world of Cloud computing? What about using it to connect to my home monster Linux box as an LTSP client OR browse to a host on it OR a host held in a VMServer Windows session on it Or... I think that the possibilities open up real fast.

I know it isn't this group's grail, but there is a grail of sorts that a lot of people are looking for - Serious Windows gaming (misnomer that it is) on a portable device. Maybe this type of smooth and responsive remote control is going to be the solution to that?

How many times has someone asked if the Pandora can run Windows or XYZ application from Windows? Well, doing it this way it essentially can as long as the connection stays solid.

If you open up the possibility of being able to display and control a session on a big chunk of hardware - so many of the 'compiled on ARM' requirements to do stuff fade a bit.

So, could optimizing the Pandora for an optimized VNC client or ThinVNC browser client application be a bounty project?

 

[MonkeyChops]

http://apps.open-pandora.org/cgi-bin/viewapp.pl?/Application/remmina.inf

havn't tried it yet, but isn't this exactly what you guys are talking about?

 

[Grench]

http://apps.open-pandora.org/cgi-bin/viewapp.pl?/Application/remmina.inf

havn't tried it yet, but isn't this exactly what you guys are talking about?

Same basic idea I think.

If I understand it right, though, VNC as a protocol was never designed to run full motion video (rendered game graphics), high def audio (game sound) and low latency controls (upstream) for games, movies, etc... Rather it is/was an open source version of Remote Desktop type functions.

The HTML5 and ThinVNC are attempts at extending the capabilities of these kinds of setups.

Someone give remmina VNC a try though - how close to working for games & video is it?

 

[paddy]

I have tried it out becase this was the thing i always wanted to do ,dreamed of playing windows games and running media from my pandora via my home pc ,anyway i installed vnc server for windows xp ,set it up and connected the pandora ,it seems very fast for about 10 seconds and then drops too slow to use ,well you could use it for admin stuff but not for media at all ,i do not know why it is so slow ,i was praying for something like streammygame or ps3/psp conectivity type of thing ,instead it's pure remote desktop ,this is what i wanted but just so slow.

i am typing this on my pandora via my windows IE hehe ,the text is updating quite nice and i have it set to 16mit colour and full screen so it looks like 1024x768 on my pandora running win xp the pandora keyboard works ,the pandora nubs work ,the pandora touch screen works ,just no sound coming from the pandora ,will need to look into the options.

its like the speed it actuly great but the pandora cant handle the video side of things,if i move a window around it tears really bad and slow,i think its to do with the pandora needing hardware driver on the desktop if that makes sense.

this is actuly amazing using windows on the pandora.

i hope this becomes fast becase the pandora will have no bounds.

 

[EvilDragon]

It's so slow because VNC transfers too much data.
NX would suite better - but AFAIK it's not open

 

[XxionxX]

We were talking about this in another thread, and upon reading this one I have a question for the masses. Would wireless-N fix the lag issue? It has a much larger bandwidth and could carry more data.

Anyway, I totally agree Grench I think this is definitely within the realm of possibilities. I Decided in the other thread that I was going to make this work somehow if someone didn't beat me to the punch (*warning* I have no skills to do this, but I intend to learn). I was told that I needed to learn socket programming and some kind of video programming. I have NO idea how to even begin on making the video more efficient, but I have a feeling it takes a lot of maths. Thats OK though because in the next 4 years I am going to get a BS in math. No promises, but I bet I'm not the only one.

Edit: spelling

 

[Blue Ion]

None of the standard desktop protocols where designed to transfer realtime fullscreen updates, only simple windows with small updates over thin connections. The only thing that comes to mind that is bandwidth friendly is using something like vlc transcoding, but getting acceptable framerate and acceptable quality would require a monster cpu.
The Core i3 330M my laptop has is definitely no power house, but it is comparable to the average of the cpus out there, and it still dropped a gazillion of frames when using mpeg2 at 25 fps while being kinda lagging.
When it worked it was a fairly smooth experience, only thing missing was the input, but with it, you can regulate the bitrate of both the video and audio, framerate, and if your cpu allows it, use mpeg4 or even h264.

Twitch styled games like trackmania may never be playable as you'll have a sizable encoder lag PLUS the internet lag. For me, it even felt laggy on a lan, but slower paced games may be well enjoyable.

Again, only thing missing was the input, but perhaps something can be worked out, like doing a one way vnc session.

EDIT (damn, I take too long to reply in this forum):

We were talking about this in another thread, and upon reading this one I have a question for the masses. Would wireless-N fix the lag issue? It has a much larger bandwidth and could carry more data.

Anyway, I totally agree Grench I think this is definitely within the realm of possibilities. I Decided in the other thread that I was going to make this work somehow if someone didn't beat me to the punch (*warning* I have no skills to do this, but I intend to learn). I was told that I needed to learn socket programming and some kind of video programming. I have NO idea how to even begin on making the video more efficient, but I have a feeling it takes a lot of maths. Thats OK though because in the next 4 years I am going to get a BS in math. No promises, but I bet I'm not the only one.

Edit: spelling

No need to learn video programming, the reason vlc is hard on the cpu may be due to some heavier settings. x264 is incredibly configurable, and will probably get realtime encoding with better quality than mpeg2, on my system for example.
The only problem would be the lag, but that is a tradeoff. Less lag will mean less time encoding which means less compression making it look worse.

Jason Garrett-Glaser talked about this in his blog: http://x264dev.multimedia.cx/?p=249
And apparently lag is definitely avoidable, only thing needed now is input.

 

[XxionxX]

OK I missed some point of why this is difficult to implement. What I think I understand is that you can encode the "controls" (shoulder buttons, nubs etc.) in one layer of packets, and the video in another. Why is there lag in the video? Can't you just send it separate like any normal video? I understand that the speed of light is not instant, and you can't send things vast distances without having some lag. But what about having your own local machine do all the work for you? I feel like I am missing something.

 

[paddy]

I also do not know all the ins/outs of this type of software but if the psp can do it with remotejoy then why can't the pandora ?

I have tested out remotejoylite on psp with my windows desktop on the psp and being controlled by the psp ,the screen was only a view port but it was already much faster than what i am using on the pandora ,i dont think the bandwidth is the problem in the long run ,it's just well like Ed said mabe too much going on that isn't needed in remmina currently ,psp-ps3 remoteplay features show that it can be done quite well indeed and what we have is a faster system and a full linux OS to play with so i really think it IS possible ,we just need someone who knows what they are doing.

ps ,i think the psp app i used was remotejoylite ,i could be wrong but a nice for instance i once saw on youtube was someone playing gran turismo 3 or 4 running in the windows xp ps2 emulator and then playing it on the psp with the psp controls ,i don't think sound was enabled ,if you try to even run windows media player or drag a window on remmina it is very slow ,i dont however think this is lag in the bandwidth sense ,more to do with the pandora gfx hardware - i could be and prolly i am wrong but it feels like that.

 

[Blue Ion]

OK I missed some point of why this is difficult to implement. What I think I understand is that you can encode the "controls" (shoulder buttons, nubs etc.) in one layer of packets, and the video in another. Why is there lag in the video? Can't you just send it separate like any normal video? I understand that the speed of light is not instant, and you can't send things vast distances without having some lag. But what about having your own local machine do all the work for you? I feel like I am missing something.

Well, don't forget that you first have to create the content, you don't have an already encoded file (in which case "encoding" latency would be 0). If for every frame, you need 40 ms to encode it, congratulations, you have realtime video at a comfortable 25 fps, but be careful, that is already 40 ms of latency.
You may decrease this time with more powerful computers, or settings that takes less time, smaller pictures, etc.. , but whether you like it or not you will ALWAYS have latency.
Also, many encoder use techniques that uses futures frames to encode present frames and such require such frames to be available, introducing several frames of delay on top of the encoding time.
VNC, RDP and the like are not free from this, and chances are its latency is around 50 to 300 ms or so, as they have to poll the screen, check changes, slice the screen and encode it, but since you don't need realtime applications, such latency is passable.

As I've read from that blog encoding time can be as little as 10ms disabling all techniques that require knowledge of future frames, (and with a powerful computer).

 

[laurencevde]

Synergy can be used for input.

But there will always be lag: lag in the networking(twice, once for the controls, once for the video), video-encoding-time, buffering, and decoding are all added together...

 

[XxionxX]

OK so the visual component of the gaming software creates content -> and then something like VNC encapsulates the video into packets and then sends it to the client. So the time needed to send video works out to (create content time)+(data encapsulation time)+(network travel time)= total time to client which is hopefully 25FPS. Is that right? Where is the weak link? I would imagine it's not the encapsulation time or the network travel time (on a local network,) and it can't be the creation of content on a fast enough computer. Is the sum of all three having inefficiencies? Doesn't this happen all the time when you play an MMOG or any online game? Why is that process fast enough and not this one? I am really confused. :wacko:

 

[Blue Ion]

OK so the visual component of the gaming software creates content -> and then something like VNC encapsulates the video into packets and then sends it to the client. So the time needed to send video works out to (create content time)+(data encapsulation time)+(network travel time)= total time to client which is hopefully 25FPS. Is that right? Where is the weak link? I would imagine it's not the encapsulation time or the network travel time (on a local network,) and it can't be the creation of content on a fast enough computer. Is the sum of all three having inefficiencies? Doesn't this happen all the time when you play an MMOG or any online game? Why is that process fast enough and not this one? I am really confused. :wacko:

With an optimistic approach:

Videogame crates frames at 60fps: 16ms each frame
Encoder, "encodes" the frame, and encapsulates it: 15 ms each
Lan transfer: 5 ms
Decoder lag: 10 ms

Total: 30 ms of lag to send the image to an other computer.

Pessimistic can reduce game fps to half of that duplicating the time taken, require twice the time to encode because the computer is slower, and finally be at the other side of the world, so in the end everything can be a weak link.

The reason why MMORPG and online games can work with such little delay is because they have a small number of components that require to be synchronized, everything else can happen in parallel, or it is skipable or predictable, a game receives a delayed packet or does not receive one at all, the game can predict what was that packet about.

What was happening in may computer and the reason why it lagged so much was that the game was capped at 25 fps, so now it is 40ms, vlc was actually taking more than 40 ms to encode each frame, plus some artificial lag to encode present frames using future frames which adds 160 ms on top of that, plus a cache or buffer vlc had that I forgot to disable adding 100 ms.
If you sum everything now, the lag goes to 355ms, still bearable, but now noticeable. In my case the weak link was the encoder.

Consider this, OnLive is going to make a living out of it and even they can't hide the lag. They require you to be very close to their servers and I bet they won't have the PC version of UT3, or trackmania on their catalog due to the lag.

 

[Trey]

.
NX would suite better - but AFAIK it's not open

The important parts of NX are open, including nxclient.

On the server side there's freenx and neatx, both work well.