Repo rating system: 5 stars or binary ratings?

[KickAss]

@ekianjo:

you're right, ppl just don't vote enough.

ppl could vote all the time, but somehow they don't.

pandoralive.info, for example, does a great job in keeping us all updated and motivated.

still, i doubt that the overall participation in the rating system will change much in recent time.

but why is that?

imho the pandora community is a bunch of ppl who like to tinker with stuff wholeheartedly.

maybe we're not all technical experts on some field, but we like to mess around and make things work.

in the end, we do not only want things to work, we do want to understand how they work best.

heck, the pandora is a total niche product and so is it's community.

i download something and mess with it. end of story. i dont care what others do.

i dont want an apple style repo to tell me what i want. i'll just grab what i want.

tbh, i don't vote much myself. the opinions of others on the matter are rarely of interest to me (there's only a few exceptions).

in my eyes the repo is a distribution channel. and that's it for me. i believe many do share this point of view.

rating is important, yes, and it's just a few clicks away. still, i don't think it's necessary most of the time (which may be wrong).

@ptitseb:

you're awesome and you know it

we all love your work and there's no exception to that!

iirc the only votes i ever took concerned your work.

without your contributions and commitment <--- all of this here ---> would be a lot less interesting.

don't focus too much on the ratings. you know how important you are.

@ekianjo + wizardstan:

i do believe ppl like ccf vote the way they do, because they dont know what it means.

that's why i'd change to 3 stars.

give them a guideline on how to vote and as a result the feedback for devs will be a lot more useful.

i mean, who's interested in 1 star votes of flappy bird, cuz the voter would like "flappy cat" way better, or the voter doesnt like the background color of the main menu...

also be aware, that after someone voted 1 star and the issue could get resolved, the vote can be changed.

 

[b_o_b]

Okay, we're fourteen pages in now.


What have we achieved? How is the current rating system being changed?


D.

Focus on the journey, not the destination. Joy is found not in finishing an activity but in doing it.

 

Greg Anderson b_o_b

 

[milkshake]

Wow this is why I have no idea on what should be done because everyone has a different spin or opinion on this, let's not forget what the purpose of ratings is for.


To help users decide and or find an application.


Although the rating may also impact developer moral, the rating is for the app not a "how much they like the developer rating".


Should we have a separate developer rating?

 

[iprice]

Should we have a separate developer rating?

But with ports who would you be rating - the original devs or the person that made it possible to be played on the Pandora?

A Hit/Miss/Maybe (thumbs up, thumbs down, shaky hand) option would be my addition to the multiple choices outlined above.

I'm not bothered about ratings so much anyway - the comments feedback (as mentioned) is always more beneficial to the dev, as long as the comments themselves aren't of the "Whoever left the lights on in the white Limousine..." sort (seriously, I had that EXACT feedback in a webOS form).

All reviews/ratings are subjective anyway and you're never really sure what the person is rating with the star system - is it the graphics, gameplay, coding, sound, the ability to play an old fave on the Pandora, the ease of use, the usefullness, the stalker that loves a particular dev, the fact that a game actually works etc. etc.?

Although in saying that, I don't think ratings are of use only to the user - if there are no feedback comments, and the rating is low (and the download count), then surely the dev will wonder why and either review it again or ask for feedback.

 

[WizardStan]

@ekianjo + wizardstan:

i do believe ppl like ccf vote the way they do, because they dont know what it means.

that's why i'd change to 3 stars.

give them a guideline on how to vote and as a result the feedback for devs will be a lot more useful.

All of his 1 star are basically the same comment: "couldn't make it work, it's garbage" when everyone around him is saying it works fine, sometimes with caveats of needing to follow instructions. No matter what guideline you come up with on what each star level means "not working at all" is definitely going to rank a single star. From the perspective of a competent user he is more or less using the star system correctly. He is not a competent user, he cannot follow simple directions. If you come up with a complex system of guidelines and feedback he would still rank the ones that he has as low as possible and leave comments about how they don't work.

 

[rygD]

I thought I saw a few of the "1 star because I don't like this game, it's garbage" variety as well, meaning games that CCF doesn't like get the same rating as games that CCF can't get working.

Oddly, while trying to get an example of this I found CCF giving 2 stars, and a very detailed comment:

http://repo.openpandora.org/?page=detail&app=hyperrogue&dp=true

 

[thatgui]

Wow this is why I have no idea on what should be done because everyone has a different spin or opinion on this, let's not forget what the purpose of ratings is for.


To help users decide and or find an application.

Thats why you should implement my proposal (and fix the download numbers while your at it) :

• remove the rating system (or at least make it invisible)
  This may be a good thing if user base and participation is high enough. But even with "motiviational" actions, beeing it positive (like Christoph.Krn suggests) or negative (like ekianjo suggest) ones, a rise in a scale that really matters won't happen. Splitting that up into several ratings (game itself, gfx, funfactor, whatever), reducing the number of stars or renaming them (-2 to +2) won't change that either, it only may reduce the possibility that different people, interpret the question "what do I actually rate" differently
  
• keep the comments
  Comments are usefull and a mechanism that gives a dev means to react to as usually people tend to be more "elaborate" if they have mostly negative feelings
  
• introduce "curation" (ake a "recommended" tag)
  People need something they can use as a compass if they enter unknown waters, even if these waters aren't really big. Newbies often as not ask specifically for recommended software here on the boards (or use pandoralive) instead of staring at cold numbers/stars in the repo. People need something to relate to, and this system gives them the possibility, as they can orient themself on a person, even if their choices are rather minimal. Replace the top rated list with the top curator list. This caters to ekianjo and wiz (guidance is good) and ptitseb (keep it positive).
  

 

[ekianjo]

Wow this is why I have no idea on what should be done because everyone has a different spin or opinion on this, let's not forget what the purpose of ratings is for.


To help users decide and or find an application.

Thats why you should implement my proposal (and fix the download numbers while your at it) :

• remove the rating system (or at least make it invisible)
  This may be a good thing if user base and participation is high enough. But even with "motiviational" actions, beeing it positive (like Christoph.Krn suggests) or negative (like ekianjo suggest) ones, a rise in a scale that really matters won't happen. Splitting that up into several ratings (game itself, gfx, funfactor, whatever), reducing the number of stars or renaming them (-2 to +2) won't change that either, it only may reduce the possibility that different people, interpret the question "what do I actually rate" differently
  
• keep the comments
  Comments are usefull and a mechanism that gives a dev means to react to as usually people tend to be more "elaborate" if they have mostly negative feelings
  
• introduce "curation" (ake a "recommended" tag)
  People need something they can use as a compass if they enter unknown waters, even if these waters aren't really big. Newbies often as not ask specifically for recommended software here on the boards (or use pandoralive) instead of staring at cold numbers/stars in the repo. People need something to relate to, and this system gives them the possibility, as they can orient themself on a person, even if their choices are rather minimal. Replace the top rated list with the top curator list. This caters to ekianjo and wiz (guidance is good) and ptitseb (keep it positive).
  

I don't like removing the ratings that much but at least your propostion makes sense and the curator list can provide some replacement. The only issue is that it's very likely again the most downloaded list and top curated list will look very, very similar. I don't see why it would be otherwise, since top curation is going to be like a popularity vote, not a quality-based vote. 

In any case, there are numerous improvements for the actual repo page that should be tried before changing the overall system (css magic, removing the categories or at least making them invisible instead of displaying them prominently, showing the latest comments somewhere and latest rated applications, etc...). 

 

[thatgui]

First, I never intended the "curation" idea to be a "drop in" replacment for the rating system, I always thought of it to be more of a guideline. The idea was that user x traverses through several of this lists and eventually decides which one (or the suggestion made on it) he followes based on how much the curators and his/her own taste overlapp. To make this more easy either the curator or is his list should have a small explantory text that outlines the taste/scope. Additionally I thought of evaluating wether the recommandation has a comment bound to, with the consequence that elaborate recommendations weigh more in an overall score of the curator. Maybe even the users itself could add to this overall score by beeing able to recommend a curator.

I don't like removing the ratings that much but at least your propostion makes sense and the curator list can provide some replacement. The only issue is that it's very likely again the most downloaded list and top curated list will look very, very similar. I don't see why it would be otherwise, since top curation is going to be like a popularity vote, not a quality-based vote.

This will probably happen, regardless of which rating/recommendation as long as the user has a certain amount leeway on what/how he is actually supposed to rate.

In any case, there are numerous improvements for the actual repo page that should be tried before changing the overall system (css magic, removing the categories or at least making them invisible instead of displaying them prominently, showing the latest comments somewhere and latest rated applications, etc...).

I wonder if milkshakes overhoul of the repos start page, he showed a preview pic some time ago, was only an idea, or was actually "beta - ready"

 

[Rockthesmurf]

Let's not direct any anger at Milkshake, lots of people seem to be saying he should at least remove the current and or replace with solution X, however there is no agreement at the moment, so Milkshake doesn't know what to do for the best!

 

[ingoreis]

Maybe its Time that one say


I love your Repo Milkshake.


Its fast,stable,you find 99,9Percent on Pandora Software on it and i dont want to life without it.


Thx for your hard work for us all with that


Dont be Angry when the People not know what they realy want for a Rating System.


Maybe you have nothing to Change on the Repo itself.


My Intention is that the PND Manager should allows more easily Ratings for us all.


We just need more Rates.


Good Work what you have done

 

[bzar]

My Intention is that the PND Manager should allows more easily Ratings for us all.

We just need more Rates.

You mean a mass rating view? That could be doable, but I have little time in the near future because Reasons.

 

[DrHAX]

I use the PND manager.. I just search and try to find things that fit my limited criteria... I would love to try more software but my limited dl like limit hinders this. I think a comment to rating system helps. Like on Amazon or hsn... helps figure out if the problem is the product is reboxed doggy doo or if grandma can't figure out where to put the SD card.


Also ratings like. Needs better documentation, etc could be helpful. But there should a way to flag unhelpful ratings without valid reasons, like


" LOL PND sucks big ones 1!1!!!11"


Or even have a common issue for a tag for the ratings at first glance 1 star Lack of documentation


2 stars program was unresponsive.


5 stars exceeded expectations. (Which could also be bad depending on expectations!. So explaination is required to be useful)


I think the issue is being able to know why something is. If the software says for 1ghz. Only an they use a cc and vote a single star that's not helpful to the rest of us by them being idiots.., not enless it only states that in the read me and not in the repo.

 

[ingoreis]

My Intention is that the PND Manager should allows more easily Ratings for us all.


We just need more Rates.

You mean a mass rating view? That could be doable, but I have little time in the near future because Reasons.

Yes maybe one PND Rating with PND Manager per IP Adress/Pandora UsernameCheck ,not per User

To Cheat this,you must have another IP Adress and another PandoraUserName because PND Manager Rekognize it and save it on Appdata too

Not Login your Account,just Vote and write a little Comment with PND Manager :rolleyes:

 

[bzar]

That... would allow anyone vote as many times as they want if only implemented in PNDManager. It would need to be implemented in the repo, at which point because you can't trust the pandora username you've just substituted repo accounts with distinct IP addresses. It's dead simple to just call a web interface all you want no matter how much PNDManager would like to limit it.

 

[ingoreis]

Thats True.


It will be open Thigs for Cheaters.


I thought there where a more Easy Way to Rate PNDs.


Maybe one "one Time login" for the PND Manager?


Then one Person,can make one Rating per PND.


The copying of the HTML Key or what that is,sounds complicated.


After redownloading some PNDs,PNDManager can then Remind you to Rate the redownloaded unrated PNDs.

 

[levi]

Perhaps if the PND manager added an interface that accepted your log in details for the repo and automatically added your key from the website into its configuration would make that process seem less daunting. Personally I didn't find it at all difficult and it worked first time for me, and I appreciated how I didn't have to log in to some random software I barely knew at the time, but for everyone else, perhaps it's a better way?

 

[rygD]

It really isn't that difficult.  If everything worked it would be a simple, quick process.  I like the idea that my username and password aren't stored on my SD card/Pandora in plaintext just so I can rate and make comments.

I still haven't figured out why it didn't work for me.  Maybe I will try again in a little while, since I need to check for updates and download a couple things.  

 

[bzar]

Going off on a tangent, but just to comment on the login/API key stuff in PNDManager:

Spoiler

A real login-based API key fetching interface in PNDManager would require a coordinated effort from me (adding the user interface and qtpndman support), Milkshake (adding a public API call for exchanging login credentials for an API key) and Cloudef (adding support for the API call to libpndman).

Another possibility would be to use some of the features in latest Qt versions for digging up information from a WebView (so you'd log in to the website inside PNDManager, then the view could be redirected to the page hosting the API key and it could be scraped off the page content), but this is really really clunky, not to mention would add considerably to PNDManager's already hefty PND size, also not to mention there aren't any recent Qt5 builds floating around with working QtQuick support AFAIK.

In any case, the precious resource it needs the most is dev time. I currently have little to offer because of other projects, and even if I did my time alone wouldn't fix the issue

 

[thatgui]

Perhaps if the PND manager added an interface that accepted your log in details for the repo and automatically added your key from the website into its configuration would make that process seem less daunting. Personally I didn't find it at all difficult and it worked first time for me, and I appreciated how I didn't have to log in to some random software I barely knew at the time, but for everyone else, perhaps it's a better way?

I thought the sole purpose for having the api key was that no credentials have to be exchanged between repo and client ?