Mouse buttons on shoulders without losing shoulder modifiers


_wb_

Microbe
Staff member
Joined
Apr 5, 2012
Messages
5,390
Age
42
Location
Brussels, Belgium
One potential solution would be to give two shoulder buttons a dual use: when you tap them (not in combination with another key), they are mouse clicks, but in combination with other keys, they are modifiers.

E.g. say L1 and R1 are still Shift and Ctrl like on the Pandora, and L2 and R2 are Meta/LeftClick and Alt/RightClick.

How would it work?

- press L2 down, immediately release it: the software generates a left click (mouse button press followed by release)

- press L2 down, press another key: as soon as another key is pressed, the software generates a Meta keydown event just before the other key event is sent, locking the L2 trigger into "modifier mode"

- press L2 down, move the mouse: as soon as the mouse (left nub) is moved, the software generates a left mouse button down event and locks the L2 trigger into "mouse button" mode.

So basically the context determines how it behaves. Just pressing L2 down does nothing until you do something else or release it. So e.g. if you have software that highlights Alt-shortcuts when you hold Alt (like Ubuntu does with menu bars), that won't work, because the actual Alt keydown event only gets sent just before you hit a letter key. You can still use the keyboard Meta and Alt keys for that though.

It probably requires some software hacking to make it behave like that, so there's some software effort to be done. But in this way, we could have both the modifiers keys and the mouse buttons on the same key without really losing anything. You can still do mouse dragging (both left and right), you can still do Meta+Alt combos and a Left+Right click. You can do Shift+click/drag and Ctrl+click/drag (both left and right), but for Alt+click/drag you can't use the shoulder Alt, you have to use the keyboard Alt (i.e. START).

For games that use four shoulder buttons, you would have to disable the click functionality so they immediately send the keydown event instead of waiting for context. But that's no big deal: for games input mechanisms are different anyway. I don't think there are many games that expect four shoulder buttons and a mouse -- either it's a PC game which expects a mouse and a keyboard but no shoulder buttons, or it's a game console game which expects shoulder buttons and analog nubs/joysticks but no mouse. It's also backwards compatible with the Pandora since the Pandora doesn't have L2, R2 anyway.

Actually, with or without clickable nubs, I'm starting to think that this is how I want to have it.
What do you think? Did I miss something? Is this a good idea?
 
Make a config for Pandora and test it.

As an option, why not.

Nice idea.

I'm already used to Pandora's nub mouse clicks so I will set mine as it's on Pandora.
 
Make a config for Pandora and test it.

As an option, why not.

Nice idea.

I'm already used to Pandora's nub mouse clicks so I will set mine as it's on Pandora.
Wasn't this an idea because theres a chance that the Pyra nubs will be non clickable?
 
Last edited by a moderator:
Make a config for Pandora and test it.

As an option, why not.

Nice idea.

I'm already used to Pandora's nub mouse clicks so I will set mine as it's on Pandora.
Wasn't this an idea because theres a chance that the Pyra nubs will be non clickable?
Pandora's nubs are also non-clickable. On the Pandora, the default config is:

- left nub to move the mouse

- right nub to do clicks: move it left for left click, right for right click, up for double left click, down for middle click.

This is of course also a possibility for the Pyra. However, I think I'm not the only one who doesn't really like the Pandora default mechanism (I use it, but it's not ideal). I would prefer the right nub to be a scroll wheel (if possible a 2D one, that can also scroll horizontally).
 
Every solution is better than the one onto the Pandora, where you "click" with an bad analog nub which usualy don't work good in my case. The fact that the Pandora nubs tend to de-calibrate during usage doesn't make it better.  I only use nubs for desktop mouse navigation (works better while holding the Pandora than using the touchscreen) so for Pyra I would like to use nubs as mouse but would prefer shoulder- or action buttons for mouse buttons.
 
Every solution is better than the one onto the Pandora, where you "click" with an bad analog nub which usualy don't work good in my case. The fact that the Pandora nubs tend to de-calibrate during usage doesn't make it better.  I only use nubs for desktop mouse navigation (works better while holding the Pandora than using the touchscreen) so for Pyra I would like to use nubs as mouse but would prefer shoulder- or action buttons for mouse buttons.
The re-calibration is especially bad because people tend to need the left click more often, which means that the nub auto-calibration will think its sensor data is getting biased to the left, and it can re-calibrate itself in a very poor way because of that. At least that's my hypothesis of what happens.
 
With Nubs that may have more stable movement in the Pyra, I don't think I would mind the Pandora Nub clicking method. I got used to it after a while. Having a button mode would be good too, I like variety. 
 
Last edited by a moderator:
Can't say no to options, but as a default config: No.

People often seem to forget the gaming aspect in the... gaming controls when making such proposals. Most games simply won't let you map a mouse button to a function, they're often restricted to keyboard inputs.
Porters can adjust that, yes, but adding unnecessary effort to do the ports isn't the way to go imo.

Nubs as mouse button works for me mostly, swapped clicking to my worse nub, though. I'd be totally fine with that controlling scheme once the nubs work perfectly.
 
Sounds very tricky if you want to trigger different actions on key and/or mouse up/down events.

I don't think a keyboard where the key is determined from sequencing context is the proper way to go.

It is hard to explain the proper conditions and it is more difficult to tackle from code.

Sorry that I do not share your enthusiasm for this idea, but my experience so far with systems that attempt

to guess the proper intention of the user from context (like this setup) is that they are "ok-ish" at best when

they get it right and horribly annoying whenever they get it wrong.

Better to use the two new gamepad buttons for this and delegate insert/delete to Fn keypresses.
 
I think you misunderstand my proposal. I don't propose to modify all software to do this overloading trick. I propose to do that at a sufficiently low level so that software just sees mouse clicks and Meta / Alt keypresses. The only thing the low level driver has to do is postpone L2/R2 down events (not transmit them immediately) until the context (next event) disambiguates the intended behavior. Only at that point it either sends either a mouse button keydown event or a Meta/Alt keydown event.

So all software would automatically do the right thing, no way to "get it wrong".

People often seem to forget the gaming aspect in the... gaming controls when making such proposals. Most games simply won't let you map a mouse button to a function, they're often restricted to keyboard inputs.
Yes, games that want to use the L2/R2 shoulder buttons for something other than mouse clicks would need to put something in their runscript to map L2/R2 to something those games can use (probably something else than Meta for L2, since they probably don't know about Meta either). This is not a problem specific to my proposal: wherever we put the mouse buttons (e.g. the two new action buttons), if you want to use those as game buttons, something will have to be done to bridge the gap between what the game expects or can use and what key events the Pyra controls produce.
 
I propose to do that at a sufficiently low level so that software just sees mouse clicks and Meta / Alt keypresses. The only thing the low level driver has to do is postpone L2/R2 down events (not transmit them immediately) until the context (next event) disambiguates the intended behavior. Only at that point it either sends either a mouse button keydown event or a Meta/Alt keydown event.

So all software would automatically do the right thing, no way to "get it wrong".
 I understood that you would tackle this at the driver level, which is the problem itself. It presents an inconsistent view on the hardware and interferes with user customization through xmodmap. This way there will always be two keys that behave odd and cannot be customized through the usual means (i.e. they need to be customized in the driver itself).

It is an unneeded layer of complexity, there are perfectly viable other solutions to this (e.g. map the mouse onto the two new buttons, use meta/meta-lock combinations or perhaps even use the Pandora scheme).

You have made many plenty of good keyboard layout proposals, I know that you can come up with a simpler and better solution than this.
 
I propose to do that at a sufficiently low level so that software just sees mouse clicks and Meta / Alt keypresses. The only thing the low level driver has to do is postpone L2/R2 down events (not transmit them immediately) until the context (next event) disambiguates the intended behavior. Only at that point it either sends either a mouse button keydown event or a Meta/Alt keydown event.


So all software would automatically do the right thing, no way to "get it wrong".
 
I understood that you would tackle this at the driver level, which is the problem itself. It presents an inconsistent view on the hardware and interferes with user customization through xmodmap. This way there will always be two keys that behave odd and cannot be customized through the usual means (i.e. they need to be customized in the driver itself).


It is an unneeded layer of complexity, there are perfectly viable other solutions to this (e.g. map the mouse onto the two new buttons, use meta/meta-lock combinations or perhaps even use the Pandora scheme).


You have made many plenty of good keyboard layout proposals, I know that you can come up with a simpler and better solution than this.
Implemented at a low level doesn't have to imply that it should be difficult to reconfigure. I think we should have a user-friendly button remapping/configuration tool anyway, where you can assign functions to all input hardware in an easy way. Sure, the nubs, keyboard keys and action buttons will probably be implemented in different ways, but the user does not have to know about that. I'm imagining a gui that looks like a Pyra picture, where you can click on every key or button to assign a function to it from a list of options, where options include all keyboard keys (A-Z, 0-9, modifiers, symbols, etc), all mouse buttons, and overloaded combinations of mouse buttons and keyboard modifiers. The nubs can map to mouse movement, scrolling, mouse buttons, etc.

Dedicating two buttons to mouse clicks is obviously a cleaner solution than what I'm proposing, but it means sacrificing two perfectly fine keyboard keys. Using the right nub for clicks like on the Pandora means sacrificing a perfectly fine nub. Overloading shoulder buttons to be both clicks and modifiers may not be the most clean or conceptually simple solution, but it is the most efficient solution: nothing is wasted, it's basically as if you have three pairs of shoulder buttons instead of two. I think it should at least be available as an option, and as far as I'm concerned it could be the default option.
 
Having mouseclicks on shoulders only means no tabletop(*corrected) mode at all, for which considerations have already been made in making the shoulders what they are, that would crumble the whole idea. Almost as much as taking away usable buttons would.

Shoulder mousepress strikes me as something of a non-balanced design, where you instantly see compromises as flaws.

Mouse buttons on the nub used for mouse events is non complex and as intuitive as it gets.

And if you use your right hand for mouse movement, you can have left nub be left click. Perfectly viable, try it.

The downside is you have to let go of the nub (it wont move anywhere, cursor still in place) to rightclick.

If somehow that turns out to not work, or you have another preference, even the pandora style of clicks is preferable to even more vague solutions, because it works.

As an option i think shoulderclicks is ok, because then you can use only your left or right hand to do mouse events.

We dont know yet if the shoulders are going to be good. We however do know that the pandora way works.

With clickable nubs you also get a potential way to do middleclick, (without losing scroll) with nub ← being leftclick and nub → being rightclick. (on the nub you arent using for mouse movement.)

Option1:

                                                         (scroll)                                                                                                  (mouse movement)

                          leftclick   L E F T (middleclick) N U B  rightclick                (mouse movement)      R I G H T (menu button) N U B    (mouse movement)

                                                         (scroll)                                                                                                 (mouse movement)

   Option 2:                                       (scroll)                                                                                                  (mouse movement)

                          leftclick   L E F T (middleclick) N U B   doubleclick             (mouse movement)      R I G H T (rightclick) N U B    (mouse movement)

                                                         (scroll)                                                                                                 (mouse movement)

   Option 3:                                       (scroll)                                                                                                  (mouse movement)

                          (scroll)     L E F T (leftclick) N U B   (scroll)                            (mouse movement)      R I G H T (rightclick) N U B    (mouse movement)

                                                         (scroll)                                                                                                 (mouse movement)

Option 1 is perhaps the strongest for those who are right handed and use the stylus for mousing, since you then have rightlclick available with your right hand.

Option 2 is a mental hack to get the most use in an advanced type setting

Option 3 i think is the most intuitive, could run middleclick on shoulders since it is a more advanced thing, and not required. Horizontal scroll is nice in gimp, but pressing control and scrolling up and down for horizontal is the semi-good solution there. Clicking left nub (which is leftclick) twice to do doubeclick is a no-brainer.

With clickable nubs, the extra possibilites are good, and they make for less problems, whereas the only decisively different metric on the non-clickables is they travel farther. And thats 3mm when the pandora nubs feel and work (when they do) much better at 1.3mm

Getting sued by nintendo for using something they have a patent on in a handheld game console is a risk that i cant see how could possibly be worth the reward when the 3ds nubs are prone to error, mechanically weaker, and don't feel like quality nubs do.

The nubs with the shortest travel (1mm) have the easiest click, are the mechanically strongest, and of the kind that lasts the longest and they have discs that dont have to be custom made.
 
Last edited by a moderator:
What are the benefits of having a key/function devoted to double clicks?  I really don't double click much unless using a Windows box.  Even on the Pandora I don't recall double clicking.  I am not against having that as an option, but I think it could be better used for something else.  I usually only use double clicking to shade windows, and if I have that as a button I don't need it.  I don't have desktop icons, and I frequently use arrows and other keys for getting around, so maybe it just seems less useful to me because of my habits and how I set things up.  When I do need to double click I usually just hit the button twice, however I often disable the requirement for double clicks.

comradekingu, I would prefer a more robust option than 3DS nubs as well, but I don't think anyone will get sued for using the same manufacturer as Nintendo.  If they didn't want anyone else to use them I don't think they would be available.  It would be nice if we could test the short travel clicking nubs.  I am guessing since ED doesn't see them as a good option that they may not be that great for games.
 
Why do you have movement on the right nub? It's left on the pandora and in my opinion it is a good choice, since a gamepad has left side for movement and right side for actions. It's like the cursor is your game character you move around, manipulating the desktop game world.
 
Having mouseclicks on shoulders only means no desktop mode at all, for which considerations have already been made in making the shoulders what they are, that would crumble the whole idea. Almost as much as taking away usable buttons would.
Can you explain this please? I don't get what you mean.

Maybe the confusion is because of the word "desktop". What do you mean by "desktop mode"? Using the Pyra for desktop applications like word processing and other office stuff, full web browsing, etc? (basically anything non-gaming) Or using the Pyra for anything while putting it on top of a desk, so you can't really use the shoulder buttons? Let's say "desktop applications" to describe the former and "tabletop mode" to describe the latter. The term "desktop mode" is just too confusing for me.

If it's the former, then I don't see the problem, because my proposal does not take anything away from the shoulders as modifiers (which I do think are essential for efficient typing and keyboard shortcuts) so I don't really see what whole idea is getting crumbled here.

If it's the latter, then please explain the problem. I don't think nubs are suitable input devices for mouse movement in tabletop mode anyway (they should be optimized for thumb operation while in tabletop mode you would probably use your index finger), so hard to reach mouse buttons are not the biggest issue in terms of built-in mouse usability. Of course if you intend to work a lot in tabletop mode without attaching an external USB mouse, then you probably want to map those mouse buttons elsewhere. I think we should optimize for handheld mode though, at least when deciding on defaults.

Of course clickable nubs create more possibilities (it's too extra buttons after all, with some limitations on when they can be used), so I'm in favor of that, but my understanding was that perhaps the original plan of having clickable nubs would have to be revised.

A potential problem with clickable nubs as mouse buttons is that you may get accidental nub movement (causing accidental mouse movement or accidental scrolling) when clicking or holding down the button. Using the shoulder buttons does not have that problem. Also left and right click can be in logical positions regardless of which nub is used for mouse movement, and you can have both left and right mouse dragging.
 
I don't think nubs are suitable input devices for mouse movement in tabletop mode anyway (they should be optimized for thumb operation while in tabletop mode you would probably use your index finger), so hard to reach mouse buttons are not the biggest issue in terms of built-in mouse usability.
Pandora nubs work fine for this for at least one Pandora owner.  Mouse buttons are input using A and B by this user, since using the Pandora on a table is more difficult with the right nub as mouse buttons.  Index finger on left nub and another finger reaching over to click is not a problem.

I think we should optimize for handheld mode though, at least when deciding on defaults.
I agree, having it work well in the default usage scenario (handheld) needs to work first.  Without the clicking nubs we essentially lost 2 buttons, so that needs to be sorted before considering anything else.  Sure, it would be nice to get all the key input we have been trying to cram onto this tiny device, but shit happens.  The default Pandora method works, but is not ideal.  I am one who really wants a dedicated delete key, but I would rather lose my delete key to be able to click with something other than a nub.  The problem with that is if the new (tiny) buttons are used for one of the main forms of interaction it may seem (and feel) less than ideal to some.
 
Last edited by a moderator:
Adding a limitation which excludes tabletop mode (which is what i meant earlier, ty) is not optimizing handheld mode.

The problem of only being able to mousclick with shoulders doesnt excuse nubs that dont click, because there is nothing wrong with using index finger for mousing in tabletop mode.

Scrolling with nubs is what you would do this way in for example a pdf-document, commonly done in a off-hands setting where tabletop mode is natural. If you then cant mouseclick the nub, you end up having to reach around the device needlessly, to a position that isnt a intuitive, or particularly natural (?) position for mouseclicks.

By all means, as an addition its a good idea, but alone it isn't sufficient. Taking other usable buttons is really ugly though. It immediately looks shoehorned.

I think its premature, yet valid to imagine nub-clicks will have jittery effects. It needs to be tested, but in my mind you are holding the nub while clicking, so it wont skip. Unless there there is horrible button-wobble.

I do see rightclicking (when available) with shoulder as slightly quicker, since you dont have to let go of the nub (so that it can go back to centre) to rightclick.

However it could be that the newer hall effects can click anywhere, since the manual is has a bigger button than the manual for the bigger nubs.

In Xfce singleclick selects, and doubleclick enters. Clicking a selected folder does nothing, so it has to be done quickly. I think its better to change this so that clicking something that is selected also opens.
 
Last edited by a moderator:
Adding a limitation which excludes tabletop mode (which is what i meant earlier, ty) is not optimizing handheld mode.
Well if L2 and R2 are above L1 and L2 (which is how I imagine it to be, so L1 and R1 are in the same position as on the Pandora), then you should be able to press them even in tabletop mode. Not very comfortable, but not impossible.

In most tabletop situations, I would imagine most people to use an external keyboard and mouse (and screen) anyway.

Of course it would be necessary to be able to easily reconfigure the bindings. In tabletop mode without any external input, it would make sense to sacrifice the scroll nub or the Ins/Del keys to get more comfortable mouse buttons. I just don't think that sacrifice should be the default.

And yes, if we can get nubs that can comfortably click anywhere, then of course all of this is moot. In that case, clearly the most intuitive thing to do is to use nub clicks as mouse buttons.

I also agree with your comment that doubleclick should be avoided in interface design. Still, rapid clicking might sometimes be necessary anyway, e.g. in a FPS game where clicking is shooting.
 
Back
Top