Touchscreen Vs. Mouse

[centus]

Hello,

I'm working on a Fractal Generator for Pandora, I just realized that I heavily rely on mouse movements as well as left, right and middle mouse button clicks.
While I understand that middle mouse is probably not going to happen on Pandora, I have another question.
How do you differentiate between mouse movement and click+drag on a touchscreen? You see, those two are very different operations which I expect to use in a different way.
I know about the knobs being mouse-able, but I was hoping to be able to use the touchscreen for a nicer fractal exploration experience.

Thanks..

 

[Caine]

Typically, a touchscreen does not provide any mouse movement functionality.

Default behavior is:
- tap -> left click
- tap & hold -> right click
- drag -> left click & drag

This can be augmented:
- modifier keys (e.g. ScummVM on DS used d-pad (for left / move / right).
- "mouse" gestures. Use a small prefix gesture to indicate the action
(an idea I have, I'll have to try to see if it's annoying or not).
- alternatively, you can make your app touchscreen friendly.

Regardless, you will be able to use the analogs to control the mouse.
L and R can be used for left and right mouse buttons and L+R for middle.

 

[skeezix]

Touchscreen is a touchscreen device, and many apps are touchscreen aware and treat it like a mouse .. but thats their choice, not automatic function. You can open up the TS device and watch it for events, piece of cake. IF you plug in a usb mouse (or tell the kernel to map analog nub to mouse), then you can have TS and mouse, two separate input streams.

jeff

 

[multiplex]

Touchscreen is a touchscreen device, and many apps are touchscreen aware and treat it like a mouse .. but thats their choice, not automatic function.

Is that right? My impression was that applications that are not touch-screen aware will receive normal mouse events.

 

[dflemstr]

Touchscreen is a touchscreen device, and many apps are touchscreen aware and treat it like a mouse .. but thats their choice, not automatic function.

Is that right? My impression was that applications that are not touch-screen aware will receive normal mouse events.

Yes, exactly. And applications that *are* touch-screen aware may *choose* what to do with it, and mostly they use the touch screen as a mouse. That's what I think skeezix was trying to say.

 

[EvilDragon]

Touchscreen is a touchscreen device, and many apps are touchscreen aware and treat it like a mouse .. but thats their choice, not automatic function.

Is that right? My impression was that applications that are not touch-screen aware will receive normal mouse events.

That's not possible, as a mouse MOVES whereas a touchscreen can be touched anywhere so the mouse cursor jumps immediately to that direction.

In X, the input driver takes care of that, so all normal X apps can work with a touchscreen.
However, for emulation like DosBox, Atari ST, Amiga, etc. this won't work.
The original system is not aware of a touchscreen, so it cannot let the mouse cursor jump around, it can only move it like it was a mouse.

Currently, the mouse cursor moves when you move the cursor around with the touchscreen - but it's not aware when you remove the pen and put it to a different spot. It will not move the mouse pointer there.
Can be fixed, but needs some work.

 

[htl2001]

Touchscreen is a touchscreen device, and many apps are touchscreen aware and treat it like a mouse .. but thats their choice, not automatic function.

Is that right? My impression was that applications that are not touch-screen aware will receive normal mouse events.

That's not possible, as a mouse MOVES whereas a touchscreen can be touched anywhere so the mouse cursor jumps immediately to that direction.

In X, the input driver takes care of that, so all normal X apps can work with a touchscreen.
However, for emulation like DosBox, Atari ST, Amiga, etc. this won't work.
The original system is not aware of a touchscreen, so it cannot let the mouse cursor jump around, it can only move it like it was a mouse.

Currently, the mouse cursor moves when you move the cursor around with the touchscreen - but it's not aware when you remove the pen and put it to a different spot. It will not move the mouse pointer there.
Can be fixed, but needs some work.

So it behaves like a touchpad on a laptop then, yes?

 

[lulzfish]

Currently, the mouse cursor moves when you move the cursor around with the touchscreen - but it's not aware when you remove the pen and put it to a different spot. It will not move the mouse pointer there.
Can be fixed, but needs some work.

So it behaves like a touchpad on a laptop then, yes?

If you're playing in an emulator for Amiga, Atari, etc, then it's like a touchpad. In the normal OS with native applications, it should behave like a touchscreen: If you click somewhere, the mouse shoots there and clicks instantly.

 

[michie4life]

maybe possibility to move the mouse with the analogue gaming controls.

 

[EvilDragon]

maybe possibility to move the mouse with the analogue gaming controls.

That does work already, yes.

However, even mouse emulation for Amiga / Atari ST, etc. COULD be done using touchscreen.
It's just that someone needs to code it.

i.e. you could remember where you last touched the screen and calculate a movement for the Amiga mouse when you touch the touchscreen at a new spot.
Then the emulator needed to simulate a mouse movement as fast as possible between those two spots.

Another thing might be manipulating the cursor directly in the Amiga RAM.
AFAIR the mouse on the Amiga could move itself, that does mean there should be an area where the current position of the mousepointer is stored.
So you could directly manipulate that using the touchscreen values.

 

[centus]

OK, thank you everyone for the wealth of information, however, one more question for you:
How does SDL handle the touch screen? are there special events for touch screen? or can it only view it as a mouse (meaning every touch on the screen is a move+click).

Thank you all, I just love this community...

 

[Black Ice]

What about two finger scrolling on pandora?

 

[Caine]

What about two finger scrolling on pandora?

Pandora is not a multi-touch device.

 

[Tempel]

What about two finger scrolling on pandora?

Pandora is not a multi-touch device.

Not the issue. Resistive touch sensors can actually do a lot more than many people seem to think. Two-finger scrolling refers to putting two fingers down side-by-side on the touchpad; the wider contact area is detected and motion is turned into the equivalent of a scroll wheel (just like is done with edge scrolling). Most resistive touchpads can be configured to use two-finger scrolling (my laptop is right now).

However, I can't see it being useful on the device. Since it's a touchscreen rather than a touchpad, you won't be using it the same way. Many apps have a click-and-drag option (notably the web browsers) which is more intuitive on a touch screen than scrolling.

I'd still like to see someone implement pinch-to-zoom on it. People are always "ooh, capacitive, pinch-to-zoom!", but by my understanding of resistive touch, it should be perfectly possible.

 

[yaKC]

I've been messing about with this stuff on the GP2X for tha past few months and you can indeed get this stuff working on a non multi touch screen... for pinch to zoom I use an identifier spot that you first hold a finger down on and then when use place a second finger the cursor jumps between the two points, you can then slide both fingers in and out to zoom (and set a variable so the first finger doesn't have to stop on the initial hot spot)... if you keep the first finger roughly the same height on the screen and rotate the second fingers height - or + you can rotate pics too.

edit... I'm keeping the hot spot at the moment, so you slide the second finger, just realised if you moved the first finger it'd quite like not work as the inbetween point might stay the same, doh!, so not as nice looking as the way true pinching works but still cool.

You can also do a fairly decent go at multi touch too, as long as you carefully choose your zones for left/right/up/down/fire etc you can fairly easily get 3 buttons working, you just work out depending where the cursor is what buttons/zones must be being pressed, as the cursor always ends up between the points... so you handle the zones for where you have your buttons drawn and then work out all combinations of where the zones the cursor will end up in if certain combinations are being pressed... not as good as multi touch as there's a bit of jittering if you don't press hard enough but still good enough...

For app stuff it's handy to have a little mouse cursor at top left say that you can tap to change between Left click/Right click and scroll wheel.

edit, wouldn't know how to do two finger scrolling mind

 

[Tempel]

Yeah, it's lower level, doesn't consider pointer location at all. I wasn't planning on explaining resistive touchpads, but then I really wanted to, so here goes:

They're basically made up of two perpendicular sets of pressure sensors, one running vertically (detecting movement in the horizontal direction), the other running horizontally (detecting movement in the vertical direction).

When you put down one finger, a pressure distribution is found on each axis. If wide lines mean higher detected pressure, then the horizontal axis sees something like this:

While the vertical axis sees something like this:

And the system overlays the two pressure distributions to find a central contact point:

If you then put down a second finger, the two axes will look like:

But when we overlay the two:

There are four apparent intersection points, even though you've only put down two fingers, so you can't tell exactly where each of the two fingers are.

But for pinch-to-zoom, that doesn't matter. All that matters is that you're describing the corner of a box, and with that resistive pressure layout, you can see the box plain as day. If you move either of your fingers, the box will change accordingly. For a quick implementation, it seems like it would be pretty easy to watch for that box and give scroll wheel events whenever it changes size, but I have no idea how to do that myself. I'm guessing it would involve reading lots of ill-formatted raw data from somewhere in /dev.

And you may have figured it out already, but for two-finger scrolling, the driver simply watches for a wide horizontal pressure distribution, then turns movements into scroll wheel events.

And all of this should follow with the caveat that this is just my understanding, as I am no expert. But I invite any experts to come in and correct me (or, even better, come in and tell me they're going to implement some pinch-to-zoom!).

 

[yaKC]

Thanks for that Tempel... very interesting indeed, not relating to pinch but for scrolling problem I was having

The way I'm doing it on the GP2X is just hacking around around with ideas, as soon as I started coding I realised you could do multitouch (to a degree) and pinching etc on a normal touchscreen but was at a bit of a loss as to how to do touch scrolling.

I've just added smoothing to my code and perhaps you could do scrolling with only one finger using the past variables stored for smoothing? I wasn't sure about scrolling as what happens if you touch anywhere and then quickly move, and you have an application button - then it would be clicked, but using the variables from a smoothing algorythm and checking for movement/speed you could discount a click (reset the actual click var and not send to apps/gadgets)in favour of scrolling? should work quite simply, shouldn't need any low level stuff as long as you code for it... I'll test it out when I wake up.


edit...

Can I just ask if anyone has experience with touchscreen smoothing, I've only added a 5 frame smoothing but by dividing the x and y over the past 5 frames I now have the cursor lag slightly behind where I press... is this normal... if the movement is larger than 20 pixels I reset the buffers and it jumps to where the pressure is but I can see circumstances whereby you need the cursor to be constantly under where you press... Is this inevitable with smoothing?