ekianjo
Hardcore Member
you don't have to rate. You can comment too. Yet we have very few comments on many applications as well. Why not comment then ?
Last edited by a moderator:
Repo rating system: 5 stars or binary ratings?
- Thread starter _wb_
- Start date
ekianjo
Hardcore Member
5 stars has the meaning you give it, and it does not matter how you rate it yourself, as long as you explain your motivations behind the rating.
The current rating system would be massively more useful if we forced comments along with stars ratings - that way we get the rating + the qualitative feedback from the person as to why they thought it was great or not a good as they expected.
bzar
A Commando
harmful/non-functional/working/awesome?
ekianjo
Hardcore Member
I don't know, I think there is space for 2 ratings somehow.
1. Quality of the port (for ports only)
2. Actual quality of the game / application (not related to the porter).
That way Ptitseb could get 5/5 for the first category since he does a lot to make ports work very well on Pandora, and he would not be at fault for the ratings in 2 which are independent from him.
ptitSeb
Serial Porter
Yeah, that would be good, but I'm afraid we don't have enough vote®s for this to work.
ekianjo
Hardcore Member
Sure, that does not eliminate the non-participating problem. We have to tackle both anyway. There's no single bullet solution.
rygD
Nihilistic Mystic
Something like this is what I was thinking after looking at the first few posts after mine. Another option is to replace the stars with meaningful values. As example:
1 = does not work at all, or has severe problems making it unusable
2 = functions ok but has problems, meh game/program
3 = very few issues, i mess with it occasionally even though it isn't really for me
4 = little to no issues, i use it semi-regularly and like it
5 = perfect performance, really useful or fun
This could be expanded to 2 tier rating, where anything over 1 on performance, meaning you got it to work, unlocks the ratings for value/fun. As has been stated, this is all useless if people aren't voting and commenting. I think it might help with certain people giving things a 1 because they don't enjoy the game, and also giving a 1 because they didn't get it to work for whatever reason. Maybe a rating of 1/unusable/not working could also require feedback or notification to the maintainer that it isn't working for some people.
ekianjo
Hardcore Member
The problem with this kind of rating is that is mixes technical aspects with application-specific design aspects. It's like measuring apple and oranges at the same time.
What if something has severa problems but it's really fun ? where does it fit in your scale ? 3 ? We can see the issue with mixing 2 different kind of evaluations.
milkshake
Advanced Member
Just reading through this topic its easy to see that while people agree the rating mechanics could be different there is no agreement or stand out suggestion in place, so until there is some agreed preferred choice perhaps it would be worth while trying to encourage rating participation on the current library of apps on the repo.
in regards to 5 star rating although it may not be the best, its a pretty simple concept, I am pretty amazed people find it so confusing as if they have never used a star rating system before.
in regards to 5 star rating although it may not be the best, its a pretty simple concept, I am pretty amazed people find it so confusing as if they have never used a star rating system before.
Another Sollution.
When you download PNDs with PNDManager,then they will be saved in a List and sometime Renembering to Rate the PNDs with Stars.
But without registering an Account.
I think that many Users who download do not have an Account and i dont think that they will register an Account only for Rating the PND.
For do not Excessive Cheating,maybe you can buildin an Option for one Rating this PND per IP Adress.
Make the Ratings more easy for all,
let us do Ratings without an Account.
And maybe a renembering Rating System in the PND Manager
Then would we all have more Ratings.
When you download PNDs with PNDManager,then they will be saved in a List and sometime Renembering to Rate the PNDs with Stars.
But without registering an Account.
I think that many Users who download do not have an Account and i dont think that they will register an Account only for Rating the PND.
For do not Excessive Cheating,maybe you can buildin an Option for one Rating this PND per IP Adress.
Make the Ratings more easy for all,
let us do Ratings without an Account.
And maybe a renembering Rating System in the PND Manager
Then would we all have more Ratings.
Ziz
Advanced Member
- Joined
- Jan 15, 2006
- Messages
- 3,583
I have four and a half ideas:
- We remove the 5 stars rating
- We count the downloads of every version of a PND. The biggest count every reached for any version is the "max. Download"-value, used as "rating"
- This would also open interesting statistics about growing or shrinking download rates from version to version
- We remove the 5 stars rating
- We implement the proposed likes
- But to balanced the likes we weight them with the proposed "max. Download"-value
- If a PND is liked 30 times, but downloaded 300, and another liked 20 times, but downloaded only 50 times, we can assume, the rating of the second PND is better
- We stay with the 5 stars rating, but reset it.
- Every rating has to be coupled with a comment.
- So you like application xyz? Why?
- More important: So you don't like application xyz? Why?
- Spamish comments could just be removed from the moderators (and with them the rating).
- We stay with the 5 stars rating
- Bad rating count less.
- So let's weight the rating with itself.
- 1 star weights 0.4, 2 stars 0.55, 3 stars 0.7, 4 starts 0.85, 5 stars 1
- For any propose:
- Ratings with comments could be weighted higher
Last edited by a moderator:
ekianjo
Hardcore Member
rygD
Nihilistic Mystic
Originally 3 and 4 said the same thing technically, which were basically a slight improvement over 2. I scrapped that just before submitting so that I wouldn't get a response of "2 through 4 are almost the same"... It is a valid point either way, and the reason I shared it at all was really so I could show how it could be split into 2 ratings. If we were going with my original "has some issues" the best game in the world with some technical flaws would get a 4.
Unfortunately most of my ideas related to this revolve around 2 people. I want to keep one happy so that they don't throw up their arms and say "screw it" because the other is the only person rating and commenting, and at times they are, in my opinion, overly harsh in their ratings.
Edit: It seems too many of us are focused on the second person mentioned in the above paragraph. Regardless of if changes are made to the rating, I think the biggest need is a way to make rating/commenting easier. If that can't be accomplished then we need to get others rating/commenting, including myself. With the exception of ptitSeb's stuff I have rated probably less than 10% of the stuff I have or have tried.
Last edited by a moderator:
rygD
Nihilistic Mystic
Sorry for the double post.
Second sounds ok, and has the potential to be better than the first, but something with a few downloads and a few likes ends up being "the best in the repo", even if it has a limited audience/user base.
Third is probably a good approach, but I will rate things less. I go through my collection from time to time, and remove things I dislike or never use. If there is something I like but don't know why I can't rate it without spamming "i like this because i play it sometimes" (which I already do too much), so I will probably not rate anything unless I think it kicks ass, and I use it all the time.
Fourth defeats the purpose of having stars, doesn't it?
4.5 that may work. A comment is more valuable to some than a rating anyway (but what ekianjo said...however, this is exactly why comments are more valuable).
The issue is everyone rates things differently. I will probably never give a 1 star rating in the repo, as I reserve such things for when I get screwed over financially (i pay and never get anything, or when I do it is worthless). I probably would never give a 2 star for similar reasons.
First one sounds good.
Second sounds ok, and has the potential to be better than the first, but something with a few downloads and a few likes ends up being "the best in the repo", even if it has a limited audience/user base.
Third is probably a good approach, but I will rate things less. I go through my collection from time to time, and remove things I dislike or never use. If there is something I like but don't know why I can't rate it without spamming "i like this because i play it sometimes" (which I already do too much), so I will probably not rate anything unless I think it kicks ass, and I use it all the time.
Fourth defeats the purpose of having stars, doesn't it?
4.5 that may work. A comment is more valuable to some than a rating anyway (but what ekianjo said...however, this is exactly why comments are more valuable).
Last edited by a moderator:
ptitSeb
Serial Porter
The count per release thingy is interesting. That's a seperate topic than Star rating, but it may be interesting to judge the "popularity" of a Package (max download per release).
Linking Comment and Rating, why not, it's how big repo are doing. But still, this kind of system is usefull when you have huge base (both software & user), and when software cost money. It's not the case here, most are free, we have an active user base of what, less than 200 people?!!!, and we have less then 2000 PND...
Linking Comment and Rating, why not, it's how big repo are doing. But still, this kind of system is usefull when you have huge base (both software & user), and when software cost money. It's not the case here, most are free, we have an active user base of what, less than 200 people?!!!, and we have less then 2000 PND...
Neelix
Insecticidal Maniac
You make it sound like people don't understand the concept. I've not seen any of that kind of confusion at all. The problem isn't that the concept isn't understood, the problem is rather one of interpretation. Given that I think I've seen more opinions on said interpretation here than there are participants in the thread, confusion seems only natural.
- Neelix
read my Post
More easy Replies and Votings without havin an Account will Result in more Ratings from Users and
one Bad Rating from CCF is not more that Important because many others Rated it shure better
Login here login there,having an Account here,havin an Account there.
Sometimes i forgot because so many Accounts my Passwords or even my Account itself.
I think i have too much Accounts in too much Things and for every little Thing Account Login again?
I dont want to make an Amazon Account,i was forced to make an Google Accound and lot of other Accounts.
Shurly dont like other Users too making here and there ever a new Account.
We have shurly more Ratings on the Repo when everyone can Rate without login in everytime.
Maybe to the Fear of the Spambots,maybe we can Post in the Comments simply no Links easily.
Only login for Paid PNDs is ok or Upload PNDs or Change Things on the Additional Texts,but for Rating and Commenting realy needed?
More easy Replies and Votings without havin an Account will Result in more Ratings from Users and
one Bad Rating from CCF is not more that Important because many others Rated it shure better
Login here login there,having an Account here,havin an Account there.
Sometimes i forgot because so many Accounts my Passwords or even my Account itself.
I think i have too much Accounts in too much Things and for every little Thing Account Login again?
I dont want to make an Amazon Account,i was forced to make an Google Accound and lot of other Accounts.
Shurly dont like other Users too making here and there ever a new Account.
We have shurly more Ratings on the Repo when everyone can Rate without login in everytime.
Maybe to the Fear of the Spambots,maybe we can Post in the Comments simply no Links easily.
Only login for Paid PNDs is ok or Upload PNDs or Change Things on the Additional Texts,but for Rating and Commenting realy needed?
We already know that appealing to users to participate more doesn't work (unless someone has novel ways of motivation).
IMO anything that infuriates ptitseb the way the current rating system (CCF is just a user of it) did requires prompt action. Ptitseb isn't known to be particularly touchy. If there is no general consensus about the direction of changes, the feature should be deactivated/hidden until there is a solution. I haven't seen any comment indicating that the repo would be significantly less usable without the rating feature, but actually there were several of the 'I don't look at them anyway' variety.
It's simply not just another abstract problem to discuss at leisure. AFAICS the alternatives for ptitseb would be to stop creating/supporting PNDs or at least to stop publishing them on the repo (and perhaps use this board instead, where there is enough activity that there is a good chance to overwhelm lazy/incompetent voices with useful feedback). Both could solve his problem, but IMO that would be very sad solutions.
Rockthesmurf
Advanced Member
Could we default to a different average, one that eliminates votes which vary heavily from the norm? So if an app has 10 lots of 4/5* star rating and a single 1*, the 1* is ignored (could have an option to view average with this included). If the number of 1* ratings increases it could be considered again. I don't know exactly what weighting algorithm would work best, but some that can isolate edge case votes and eliminate them. I understand theoretically this could create dilemmas but maybe in our cases at the moment it could be fine.
