Irritated with Rating system

[gordon daniels, modificado 10 Anos atrás.]

It would be nice if Liferay made it a policy that employees of companies that develop portlets be prohibited from rating and commenting about portlets. I've downloaded several portlets with 5 star ratings, only to have them either corrupt my server or are terrible in usability. So what gives with the ratings? Well, in ever single case the users who gave such glowing ratings to the portlet were in fact employees or the developer themselves. I find this unethical. Yes, that is harsh but that is how I see it. I've done a few reviews of portlets on the marketplace. I don't know any of the developers and I am genuinely trying to give feedback that is constructive.

Shame on you people who give high ratings to your own products!

[James Falkner, modificado 10 Anos atrás.]

gordon daniels:

It would be nice if Liferay made it a policy that employees of companies that develop portlets be prohibited from rating and commenting about portlets. I've downloaded several portlets with 5 star ratings, only to have them either corrupt my server or are terrible in usability. So what gives with the ratings? Well, in ever single case the users who gave such glowing ratings to the portlet were in fact employees or the developer themselves. I find this unethical. Yes, that is harsh but that is how I see it. I've done a few reviews of portlets on the marketplace. I don't know any of the developers and I am genuinely trying to give feedback that is constructive.

Shame on you people who give high ratings to your own products!

Hey Gordon - I completely agree about the ethics (or lack thereof) of self-ratings. We could easily stop people from rating their own (or their company's) apps, and that will be a good start (I hope!) but abuse is still possible without some sort of crowd-sourced feedback mechanism. If enough people thumbs-down a clearly biased review, it will have less impact on future potential users. The same issue exists at tripadvisor.com and is a difficult one. We have the 'thumbs down' capability now, but we could also shove the more thumbs-down'd reviews to the bottom, and/or hide those that reach a certain threshold of dislike.

[gordon daniels, modificado 10 Anos atrás.]

James; Those are all good thoughts. Yes, we could thumbs down, however, some portlets I've tried are actually good but still self-rated. So, thumbs down doesn't work in those cases. I could post on those portlets pointing out that its self-rated but did not want to get into a flame war with anyone. So, I don't have an answer.

thanks for your reply.

[Corné Aussems, modificado 10 Anos atrás.]

gordon daniels:

IWell, in ever single case the users who gave such glowing ratings to the portlet were in fact employees or the developer themselves.
Shame on you people who give high ratings to your own products!

Haha you can also comment that, i just received 5 thumbs up for that.
(btw i thumbed up my comment, just now )

James Falkner:

We could easily stop people from rating their own (or their company's) apps, and that will be a good start (I hope!)

And generate even more phony accounts on liferay. Thats a way to grow the community!

James Falkner:

If enough people thumbs-down a clearly biased review, it will have less impact on future potential users. The same issue exists at tripadvisor.com and is a difficult one. We have the 'thumbs down' capability now, but we could also shove the more thumbs-down'd reviews to the bottom, and/or hide those that reach a certain threshold of dislike.

Also this makes no sense because a team can easily generated more thumbs up than an individual.

James Falkner:

but abuse is still possible without some sort of crowd-sourced feedback mechanism.

Indeed we need mass and i see a new community program looming 'Community Plugin Critics Team'.

Honestly the only way you should address this is to give constructive advice. I do not think a company/person intends to publish something crappy on their name and vote for it.
So either you have problems implementing it, for which you need help, or the software is not as good as it should be and your advice is very much welcomed by the developers as well as the next person evaluating this. Lets learn from each other!
Like Tripadvisor one will value the most elaborate comments more than some comment bashing the owner.

I think Liferay should also add the option to remove a comment when things improved!

I am dead serious about the team just rally up and lets start evaluating these plugins!

C.heers

[Corné Aussems, modificado 10 Anos atrás.]

Corné Aussems:

I think Liferay should also add the option to remove a comment when things improved!

Wow James that is quick it already works!
If you review again you can change the comment and rating

[gordon daniels, modificado 10 Anos atrás.]

Corne: I certainly know how to implement the portlets. However, there are several that do not work as advertised. You may be right about them wanting input but who knows. Hopefully, people will view this and think twice.

[David H Nebinger, modificado 10 Anos atrás.]

How about weighting the rating? If an app is downloaded 5,000 times but only rated by 3 folks, the weight of that rating is pretty darn low.

[gordon daniels, modificado 10 Anos atrás.]

David: Not bad. Its certainly not fool proof but better than current state.

[Brian Kim, modificado 10 Anos atrás.]

I was thinking another option would be to give more weight to specific users that are more respected authorities. Kind of like the "Yelp Elite". Basically, certain people in the community are trusted to not give biased reviews either because they're very vested in the community or because they been elected as a trusted authority. We could have those Liferay Elite review's highlighted or weighted over other reviews.

[gordon daniels, modificado 10 Anos atrás.]

Brian: I like this idea alot. An example for me personally was looking at a portlet that Corne gave a thumbs up. I tried it because of that and was pleased with the portlet. You might be on to something here. :-)

[Corné Aussems, modificado 10 Anos atrás.]

gordon daniels:

Brian: I like this idea alot. An example for me personally was looking at a portlet that Corne gave a thumbs up. I tried it because of that and was pleased with the portlet. You might be on to something here. :-)

Knowing this is making me more cautious before i vote.
Also because i rate stuff sometimes because i like a guy.
But how can you see how many stars i gave? This would be important then.

For 'Critics' you need more criteria that need to be rated separately
Such a list already existed for the nominees and winners of last years contest.
"Judging Criteria"
(1) Innovative and Unique idea;
(2) Integration;
(3) User Interface;
(4) User Experience;
(5) Content

Adding to that
- Quality of Coding (5 Content?)
- Proper documentation (5 Content?)
- Availability of Source (5 Content?)
- Translations supported (3 UI tells something too)
- Availability of support
- Added Business Value (hard to do)

Other cool thing would be blogging about comparison of 'newsletter plugins';

I suggest creating a WIki page or a (yet to come) User Group 'Critics' where we can aggregate and discuss these criteria and such.
Anyone adhering to these rules can become a Critic, spit in the hand and swear solemnly not to misuse their powers.
Hmm thinking of that maybe some peer review would be better.
Full reports could be made available there too, etc etc

[gordon daniels, modificado 10 Anos atrás.]

Corne: You have given me much to think about with this post. Such as how far do we want to take this? I try out new portlets all the time on my test server. It amazes me how many of them break the server. I tried them in the first place based on what their funtionality was and by ratings or reviews. I really would like to be able to trust the Marketplace rating system to narrow down what I test. None of the 'bad' portlets did me harm, just wasted my time, which is precious now days :-).

Thank you for your thoughtful reply.

[David H Nebinger, modificado 10 Anos atrás.]

gordon daniels:

Corne: You have given me much to think about with this post. Such as how far do we want to take this?

I wouldn't mind seeing it taken up also, but honestly I'd rather be able to get paid for my plugins first.

[gordon daniels, modificado 10 Anos atrás.]

David: LOL. I can understand that. How will that work? I certainly would buy one of your plugins if I needed its functionality, but what happens when you purchase one and don't like it. Is there a refund policy? And with yours David, we might need a refund policy :-)

[Jack Bakker, modificado 10 Anos atrás.]

I think the main point here is towards devising a social way in the liferay community in order to minimize noise and what the community sees as unethical posts (selling one's own book via repeated annoying community forum posts, a plugin, service, bashing Liferay to sway to competitor, etc.

Avoiding SPAM really.

For mail systems there is spamassassin and related to avoid SPAM. However, an automated liferay rbl according to mutually? agreed spam-posters might be pretty harsh.

David recently posted word SPAM in response to a post. I thought that was clear, because I like David's efforts towards building instead of breaking.

Am dreaming on not a thumbs up or down for a particular post but a thumbs up or down on the poster by username (whether real or not) ; this might collect some good data prior to a yelp elite

[David H Nebinger, modificado 10 Anos atrás.]

gordon daniels:

David: LOL. I can understand that. How will that work? I certainly would buy one of your plugins if I needed its functionality, but what happens when you purchase one and don't like it. Is there a refund policy? And with yours David, we might need a refund policy :-)

Apple, Amazon, Google, M$ and others have figured these issues out, it's certainly not rocket science anymore. Yes, I understand that it's a legal nightmare, but the marketplace was announced what, almost 2 years ago?

Independent developers create apps for the other stores because they can (hopefully) get paid for the work that they do. With that not being an option, I'm too cynical to believe that the available apps are free of some sort of agenda. Nothing in the world is free, so it's unclear what the cost of the plugins actually are...

[Brian Kim, modificado 10 Anos atrás.]

gordon daniels:

David: LOL. I can understand that. How will that work? I certainly would buy one of your plugins if I needed its functionality, but what happens when you purchase one and don't like it. Is there a refund policy? And with yours David, we might need a refund policy :-)

This is taking us a little bit off-topic, but there will be an option for developers to offer 30-day trials of their apps.

[gordon daniels, modificado 10 Anos atrás.]

that would be a good thing.

[James Falkner, modificado 10 Anos atrás.]

Brian Kim:

I was thinking another option would be to give more weight to specific users that are more respected authorities. Kind of like the "Yelp Elite". Basically, certain people in the community are trusted to not give biased reviews either because they're very vested in the community or because they been elected as a trusted authority. We could have those Liferay Elite review's highlighted or weighted over other reviews.

+1 on the Liferay Marketplace Critic's Club idea ;-)

The same can be applied to the forums based on quality, not quantity, of posts!

[gordon daniels, modificado 10 Anos atrás.]

+1 from me. Like it. I like the critic idea. If you look at my ranking, I am Liferay Master. That's a joke. I am not a master. However, over the years I've posted alot. That should not be the criteria for ranking. Just saying.

[David H Nebinger, modificado 10 Anos atrás.]

gordon daniels:

If you look at my ranking, I am Liferay Master. That's a joke. I am not a master. However, over the years I've posted alot. That should not be the criteria for ranking. Just saying.

Totally disagree with your self assessment, Gordon. I'd put money on you over someone w/ a post count less than 100 any day of the week.

[gordon daniels, modificado 10 Anos atrás.]

David: you are too kind :-)

[Jack Bakker, modificado 10 Anos atrás.]

I nominate David gains weight

[David H Nebinger, modificado 10 Anos atrás.]

I nominate David gains weight

I'm gaining weight all on my own, Jack, don't need your help...

[Joseph Toman, modificado 10 Anos atrás.]

Wouldn't it be simpler to just make the name of the reviewer a link to their profile, the same as right here in the forums? That way one click would tell you who they are, do they work for the company that wrote the portlet, are they friends with the developer, etc. . Maybe add another rating widget that shows your friends' ratings, because those are the people that you trust, not some group of users that you may never have heard of but have been assigned some elite status as far as marketplace rating goes.

[gordon daniels, modificado 10 Anos atrás.]

Joseph: the concept is good but in practice not sure how useful it is. For instance, my profile says nothing about me, so if I reviewed a product you would not know who I work for, but you could see who my friends are I guess. I like the concept just not sure of execution.

thanks for the reply