Sign up FREE & get 250 points
 More options
Points2Shop Forums » Suggestions » Quality Score needs a rework
Created: 2016-06-09 12:49:54
Wannapla17
US platinum
Senior Poster
Total posts: 204
Before we begin I'd like to point out that I will be using my own personal experience this week with the "dangers" of the quality score. This post is not intended to be a rant of any kind, but rather important facts that need to be addressed on my opinion about how the Quality Score system is flawed.

I received a notice yesterday that certain features had been blocked on my account. Why? Because I managed to fail a trap question on a survey the other day. While I agree 100% that there SHOULD be trap questions to help filter the rushers from the people who try to provide quality survey answers, the system is flawed and here is why.

Over the last few weeks I have managed to complete around 65 surveys. That is completed, not counting the ones I was DQ'd from due to them reaching enough people in my demographics, etc. In just those 65 surveys completed I managed to hit 3 trap questions. So if we just took into account the 65 I actually got to complete, I managed to hit a trap question 4.6% of the time.

I was informed that I was "rushing" through surveys and I could be banned. (A member who has referred over 250 people to this website and been a member for close to 7 years) could be banned because I failed 4.6% of the time of a 3-4 week period. Less than that when you count all the surveys I got half way through before being DQ'd.

I can assure you I never rush through surveys, so if I hit a trap question its most likely a miss click, and my stats prove my case. Most of those trap questions are hidden inside a bunch of others and while I see them just fine a miss click can happen. I however did agree with support about the matter and told them I understand the situation. However, something needs to be changed about how the Quality Score really works.

I agree that there should be something there to keep rushers at bay, but to have a system so tight that a veteran member who has put years into this website can be banned forever due to a 4.6% fail rate is absurd and just isn't smart. We are humans, we are going to make mistakes.

My suggestion? Quality Score should reset once a month, or even once a week. Have a cap of 100 showing you are doing a perfect job. If during that week or month (whichever sounds better) you drop below say a score of 70 then you obviously have a stronger case to deal with. You guys openly admit that it doesn't matter how many surveys we complete or how big our QS, all it takes is one or two fails and we can be banned. Whats the point of a QS with no cap if that number carries no weight when your credibility is being addressed? It needs a cap, it needs to have weight.

What do you guys think? Any other suggestions to help make this system a little bit better?
killerseed
UK legend
Senior Poster
Total posts: 208
Date of Birth: 1981-10-19
Total Earnings: $6206.64
the system is fine .. you need to stop rushing a read properly. those traps are inside surveys for that specific reason, to make sure you dont just click any old box and are giving the providers what they pay good money for "Quality Data"

Lesson learned

i may also add the Quality system was always there, it just was not visible to the sites members :)
ldpx3000
US moderator
Points2shop is my specialty
Total posts: 1117
Thanks for your input. The QS system is definitely not perfect (after all, we are humans and make mistakes too) so we are constantly tweaking and adjusting it so that it still does the job we need it to while at the same time trying to be fair to our users.

While I can't say exactly what all factors go into a user being flagged, sufficed to say there are numerous moving parts and that means that there can be false positives from time to time and we will continue to work to make it better in the future :)
'Build a better idiot proof system and the universe will build a better idiot...'
Wannapla17
US platinum
Senior Poster
Total posts: 204
Thanks for your input. The QS system is definitely not perfect (after all, we are humans and make mistakes too) so we are constantly tweaking and adjusting it so that it still does the job we need it to while at the same time trying to be fair to our users.

While I can't say exactly what all factors go into a user being flagged, sufficed to say there are numerous moving parts and that means that there can be false positives from time to time and we will continue to work to make it better in the future :)ldpx3000


Not a problem LDP, I agree we need the system, hope it gets better over time.
the system is fine .. you need to stop rushing a read properly. those traps are inside surveys for that specific reason, to make sure you dont just click any old box and are giving the providers what they pay good money for "Quality Data"

Lesson learned

i may also add the Quality system was always there, it just was not visible to the sites members :)killerseed


I've been here for years, I know how the system works. Unless you have a real suggestion instead of assuming I "rush" my surveys when my completion rate says otherwise please don't bother posting here. The topic is about tweaking the QS system, not my credibility.
Bambiandblake
US (MA) gold
Junior Poster
Total posts: 76
Total Earnings: $3560.03
A week or so ago, I was hit with two quality score reductions at a rate of -10 points, per hit. Like you, I was perplexed.

I took it to Shout, and as suggested by the community, I filed a ticket thinking there was a problem with the system, like you and me, somebody had been hit with more than one reductions in a short amount of time.

I filed the ticket wondering if it was there an error or mine. The Shout suggestion and the follow up to that suggestion yielded something I should have considered before, that it was *I* who made the mistakes: presumably with an external survey and another for an issue with one of my profilers.

I was very fortunate that admin who handled my query was very patient and didn't, at least outwardly, remark that I was running to admin to avoid penalty of my carelessness (which would never be allowed to happen), but they gave me answers, stabilized my profilers, and assured me that (for now) my account was in good standing. This is something you should address with admin very honestly.

What I realize in this that this medium is based on competition and people are very competitive for the most points, to be at the top of the contests for rewards/awards, the most accruals, the most referrals, and sometimes that could cloud a person's judgment. Sometimes mere exhaustion can cloud your judgement, which is why if, from now on, I'm not feeling well, tired, or annoyed that I stop doing the surveys (and barking Shout) until I regain my senses. Then, of course, you have p2s looters but that is another story and one that is not applicable in your case.

I've seen it stated in Shout that regardless of your tenure here and a high QS that you're not immune to consequences of oversights - even a slipped click. We are all fallible.

If there is indeed a systematic problem, I trust the admin to be professional enough to research it, There may be some problems.

Best wishes.
Globie

You are not logged in.

Chat History

will load when you scroll to the bottom