Rating software on repo.openpandora.org

[kuru]

OK, so this is nothing big, but has been pinching my side for quite some while now.


I have been following the repo since completion. The only gripe I have with it, are some of it's users!


What's the deal you ask? People come to the repo and find new software, try it, rate it one star (the lowest possible) and just move on, leaving behind big red question marks hovering in the air.


So, please, for the community's sake and that of the developers, if you rate something down on the repo, leave behind a little hint of what you disliked or report technical problems you may have encountered. That way everyone profits.


Thank you for your attention.

 

[clop]

I couldn't agree more

 

[Binky]

Maybe a frontend like panorama could keep track of the games/apps that you've downloaded, but not rated yet - then you could just go through the list once a week or so.


People will do things if its easy...

 

[diligentcircle]

It's a bit scary to say why you rated something 1 star sometimes. For example, I won't say what it is, but I recently rated something 1 star because I tried it and found it to be absolutely stupid. And to be frank, unless something drastically different was done, I don't see any way it could be better. Sure, I could tell the author what this is, but I tend to think I would more likely just offend the developer.


Honestly, though, I think every website with an anonymous rating system, including the Repo, should use like/dislike buttons rather than a 5-star scale. I started doing this on my website last December and haven't looked back. In a nutshell, my reasoning is this: people see a scale and try to manipulate it; if they think it should be a smaller number, they rate it 1 star, and if they think it's a larger number, they rate it 5 stars. With likes/dislikes, though, you can split it to two numbers that only increase, so if someone thinks that something is good, but not as good as everyone says, they can't decrease the number of "likes"; my conjecture (which personal experience tells me correct) is that they will not feel inclined to increase the "dislikes" counter in these situations in an attempt to balance it out, and instead they'll just not vote at all, since that's all they have the power to do. I explain more in depth in my blog post about this, which can be found here:


http://onpon4.blogspot.com/2011/12/5-star-ratings-vs-likesdislikes-why.html


Another big advantage of likes/dislikes for the Repo is that one thumbs-down doesn't look the same as 1,000 thumbs-downs at all, whereas one 1-star rating looks exactly the same, at a glance, as over 9,000 1-star ratings. People might look at something with an average rating of 1 star and dismiss it, but I don't think they would be as likely to be so dismissive about something that has only a single thumbs-down.

 

[bismuthdrummer]

Honestly, though, I think every website with an anonymous rating system, including the Repo, should use like/dislike buttons rather than a 5-star scale.

The tradeoff for using likes/dislikes is that you have no gauge as to how good something is. Does it provide fully supported functionality? Is it polished for what it does? Maybe I am only looking for those apps with near perfect functionality, or maybe I am willing to try anything above one star. With likes/dislikes, you have no sense of "how much" something is appreciated, only that it is or it isn't. Popularity is somewhat of a gauge itself, but beyond that there is no more data on the app's worthiness. Rainy Day might be a good example of an app that would presumably get lots of likes but not the full five stars since it isn't a fully functioning game. Something with lots of 5 star votes, like PCSX ReARMed, tells me that off-the-shelf it is near perfection.


Maybe you could require a comment alongside a vote, but that would just be asking for garbage commentary. Or maybe you could attach the usernames to the votes so people can see who voted the one star. I am sure there are downsides to that as well.

 

[kuru]

people see a scale and try to manipulate it; if they think it should be a smaller number, they rate it 1 star, and if they think it's a larger number, they rate it 5 stars.

That's true. i have found myself doing exactly that a few times in another place. These mechanics have noticeable drawbacks for sure.

It's a bit scary to say why you rated something 1 star sometimes.

I guess it comes down to your choice of words. Pretty much everything can be said in a multitude of ways, of which at least one is contributive and non-offensive.


Another option for the repo: It could be made mandatory to leave a comment if you want to rate something. On the other hand that could be quite BS-inducing, too. But at least ppl have to put down their names for it, which isn't too bad I guess. Maybe that would encourage ppl to put more thought into the ratings they give, or maybe even keep them from rating if they're not quite sure about what to say about a particular piece of software.


Edit: Meh, ninja'd ^^

 

[diligentcircle]

The tradeoff for using likes/dislikes is that you have no gauge as to how good something is.

That assumes people actually rate correctly. People tend to rate in an attempt to get the average to conform to what they think it should be, i.e. if the average is 4 stars and they think it should be 2 stars, they'll rate it 1 star. This skews the results.


This is actually not just conjecture on my part; Google found that very few people used the 2, 3, and 4 star ratings, and some YouTubers even openly admitted to what I just described. It's a pretty strong hypothesis.

Another option for the repo: It could be made mandatory to leave a comment if you want to rate something. On the other hand that could be quite BS-inducing, too. But at least ppl have to put down their names for it, which isn't too bad I guess. Maybe that would encourage ppl to put more thought into the ratings they give, or maybe even keep them from rating if they're not quite sure about what to say about a particular piece of software.

That would be a review, not a simple rating. While I think that's an excellent idea, I think simple anonymous ratings also serve a purpose, and if they aren't available, you can easily get useless reviews saying things like "dis is awsume!!!!111".


One particular purpose is my 1-star rating I mentioned earlier: I couldn't have given constructive criticism. I know I'm being vague here (don't want to accidentally offend the author), but in this case, there is no way that a review from me would be helpful. It is an app that I don't think is worth even existing, so any comment by me is not going to be helpful to the developer.

 

[ZXDunny]

Anything I don't like, or don't like the look of from the screenshots gets an automatic 1-star rating. If a program looks lazily done, I vote it down even if I don't download it.


D.

 

[bismuthdrummer]

I agree with you onpon4, and I see what you're saying.


I'm sure that statistical distortion happens... maybe you could hide the actual rating until each user votes? Keep the ranking, just no rating for you unless you've tried it. I don't know, just spitballing here.


Myself, I would take away the anonymity, so you can look at the vote list and see where the one star came from. My guess is you'd have a lot of the case of: *looks at the one star*... "Oh, THAT guy." Same goes with the five star voters, although I think the cynical one starrers tend to stick around more.

 

[diligentcircle]

If a program looks lazily done, I vote it down even if I don't download it.

How does something look "lazily done" just from a screenshot?

 

[ZXDunny]

If a program looks lazily done, I vote it down even if I don't download it.

How does something look "lazily done" just from a screenshot?

It's using 640x480 for no good reason other than the porter can't be arsed to change it. It uses a terminal-style interface. That sort of thing.


D.

 

[hideki]

It's using 640x480 for no good reason other than the porter can't be arsed to change it. It uses a terminal-style interface. That sort of thing.

Suppose some of my ports have been lucky to escape your wrath then, lol (mind you, CP/M and X-Tree gold can't very well use anything but a terminal interface)

 

[ZXDunny]

It's using 640x480 for no good reason other than the porter can't be arsed to change it. It uses a terminal-style interface. That sort of thing.

Suppose some of my ports have been lucky to escape your wrath then, lol (mind you, CP/M and X-Tree gold can't very well use anything but a terminal interface)

You spent a large amount of effort getting E-UAE to run Workbench in 800x480 hicolour modes. You added touch-controls to Elite-TNK to fill the gaps created. Terminal interfaces are fine when they were originally written for that interface - it's modern stuff that uses it as a lazy way of providing an interface that gets my goat.


D.

 

[diligentcircle]

EDIT: Actually, never mind... I'm not going to argue with ZX on how to vote or how to (pre)judge apps. If we had a likes/dislikes system, it wouldn't matter anyway.

 

[Neelix]

The tradeoff for using likes/dislikes is that you have no gauge as to how good something is. Does it provide fully supported functionality? Is it polished for what it does? Maybe I am only looking for those apps with near perfect functionality, or maybe I am willing to try anything above one star. With likes/dislikes, you have no sense of "how much" something is appreciated, only that it is or it isn't. Popularity is somewhat of a gauge itself, but beyond that there is no more data on the app's worthiness. Rainy Day might be a good example of an app that would presumably get lots of likes but not the full five stars since it isn't a fully functioning game. Something with lots of 5 star votes, like PCSX ReARMed, tells me that off-the-shelf it is near perfection.

I don't see a tradeoff here. The trouble with this theory is that a simple star rating conveys no such useful information. I have always found star ratings based on popular votes to be utterly useless because there is no common basis for what each level represents.


- Neelix

 

[kuru]

There could be a rating system with multiple categories like eBay does for their sellers now


Something like


Visuals


Sound


Originality


Stability


Controls

 

[diligentcircle]

There could be a rating system with multiple categories like eBay does for their sellers now

Overly complicated. That type of thing should be reserved for proper reviews, IMO.

 

[thatgui]

What's the point of having a like / dislike - System ?


Maybe I don't get it (or I refuse to), but what does this kind of rating say about an application ? Except the obvious of course, that there are x people that like it and that there are y people that don't like it. There is no in between, just the two extremes. What about the people that though: it was well made/well thought/a nice idea/looks promsing but is just not my cup of tea ?


I would vote for an extension of the current system by pointing out how many people voted for every number of stars (like 2 people voted 5 stars, 34 people voted 4 stars,...., 2345 voted 1 star).


When looking at a five star system like the one Amazon uses - I usually ignore the one and the five star ratings: One star raters usually are "nag" happy or are not capable of fully understanding the product/software they are rating. In contrast five star raters are usually overexited, or are dropping their complains (and there sure is always something to complain about) because they are too "nice" or want to encourage the author.


That doesn't mean their ratings are wrong in general, or that these people are stupid. Their rating just doesn't help me with my decision - and ZXDunny was nice enough to prove this right here in this very thread.


I also didn't understand how the manipulation of a star rating system should work. They got one vote to give, this may influence the overall vote if only a small number of people voted, but the bigger the count of voters gets the more a single vote gets averaged out - maybe someone could explan it ?

 

[sebt3]

As I guessed (well an app being rated 1 star without any download help to guess), some users rate before testing which is a shame !


For me a low rating mean this app/game sucks which also mean I wont bother spend more of my free time improving it.

 

[relliker]

May I suggest that the repo software description also be made visible on the pandora built-in pndstore? I don't look much at the ratings before I download and try software but I'd like an idea of what I would be downloading other than just it's name and installed/repo version numbers as I use the repo mostly directly from the Pandora, rather than from a browser.