Add a weight to head-fi score?

The idea is simple: head-fi collects user reviews, which are usually straight impressions and sometimes more biased than those from trusted reviewers. Another, maybe more important thing, is that head-fi at times has only 1 or 2 reviews of a product. With the above two points together, I think it is reasonable to weight down the head-fi score when calculating the average reviewer score, according to how many user reviews (and may be their lengths) it provides. It might be also good to do this adjustment to the "web search" score; the AI reviews also look unconvincing to me at times.

Agree, I am thinking for a longer time to improve the calculation of the average score in terms of weighting. One problem in general are IEMs with only a few reviews can land pretty high in the list when one reviewer is hyping it because the weight is higher when there are fewer reviews. The head-fi scores doesn't make it better. It is definitely on the to-do list.

Endoki wrote: Agree, I am thinking for a longer time to improve the calculation of the average score in terms of weighting. One problem in general are IEMs with only a few reviews can land pretty high in the list when one reviewer is hyping it because the weight is higher when there are fewer reviews. The head-fi scores doesn't make it better. It is definitely on the to-do list.

I got it, so the head-fi and AI scores are intended as a counter-measure for the "single reviewer hype." For this general problem, how about adding the median score which is more robust to outliers than the average. Or, if we just think products with few reviews should not be ranked very high (make sense), we can add a carefully designed penalty by review count.

Yeah, you got this right. To be honest the AI scores are absolutely a counter-measure to that problem. But also a non-optimal thing, most people hate such AI Sludge which I can totally understand. I will think about median, but this is also a unusual thing to determine meta-scores. The problem in general is that for a true "meta-critic" style site, we have not enough reviewers. Which is to be expected in a niche like this. So my general advice is: Take this whole thing with a grain of salt. I will try my best to optimize things and implement good suggestions, but we are still far away from a perfect solution.

Sure, the lack of data is the real thing.