Home / Articles / Product Reviews

How to Read Product Reviews Critically

Why online reviews are unreliable, what to look for, and how aggregation helps.

The Problem with Online Reviews

Online reviews are incredibly unreliable for predicting product quality. This isn't because reviewers are dishonest (though some are), but because review systems have inherent biases and problems.

Selection Bias

People are more likely to review products they love or hate. Satisfied customers with average experiences often don't bother reviewing. This creates a U-shaped distribution: lots of 5-star reviews from enthusiasts, lots of 1-star reviews from disappointed customers, and few 3-star reviews from people with normal experiences.

A product with average 4.5-star reviews might actually be mediocre. It's just that mediocre people didn't review it.

Fake Reviews

Paid fake reviews are common. Sellers buy reviews to boost their rating. Competitors post fake negative reviews to damage ratings. Review platforms spend billions trying to detect and remove fake reviews, but it's a constant struggle.

Detecting fake reviews is hard for individual consumers. ReviewMeta tries to algorithmically identify suspicious patterns, but it's an arms race between review buyers and detection systems.

Incentivised Reviews

Amazon has rules against paying for reviews, but incentivised reviews still happen. Sellers offer discounts in exchange for reviews, or they provide products to influencers who may be biased toward positive coverage.

Incentivised reviews aren't always fake, but they're biased toward positivity. Someone who got a product at 50% discount is more forgiving of flaws than someone who paid full price.

The Recency Bias Problem

Reviews reflect current manufacturing quality, not long-term durability. A product might have excellent recent reviews but serious durability problems that only show up after six months. Those reviews won't appear until later.

This is a bigger problem for products purchased infrequently (furniture, appliances) than consumables (groceries, books).

What to Look For in Reviews

If you're reading individual reviews, look for:

The Review Count Problem

More reviews can mean a product is popular, or it can mean it's been heavily reviewed-bombed or artificially boosted. A product with 10,000 reviews and 4.8 stars is suspicious. A product with 1,000 verified purchase reviews and 4.3 stars might be more honest.

The Law of Large Numbers

With enough reviews, averages tend toward true quality. A product with 100 reviews and 4.8 stars could be genuinely great or heavily manipulated. A product with 100,000 verified purchase reviews and 4.3 stars is probably actually pretty good, because it's harder to manipulate that many reviews.

Aggregation Reduces Bias

Combining reviews from multiple sources reduces individual source bias. If a product has 4.2 stars on Amazon, 4.1 stars on retailer websites, and 4.3 stars on specialist review sites, the aggregated score of 4.2 is more reliable than any single source.

Aggregation also smooths out manipulation. Fake reviews on Amazon matter less if the product also has genuine reviews on other platforms.

How RefDat Scores Work

RefDat calculates product scores by aggregating reviews from multiple independent sources. The score is weighted by review volume, recency, verified purchase status, and source reputation. This approach reduces the impact of individual source bias or manipulation.

A RefDat Score of 7.5 means that after aggregating reviews from multiple sources and weighting by reliability, the product ranks 7.5 out of 10. This is more reliable than a single source's 5-star rating.

The Australian Consumer Law Perspective

The Australian Consumer Law prohibits false or misleading claims about products. The ACCC (Australian Competition and Consumer Commission) has published guidelines on fake reviews and unfair consumer practices.

If a product has obviously fake reviews, you can report it to the ACCC. But individual consumers usually can't do much except distrust reviews and rely on aggregated scores.

Price and Quality Correlation

Price is not perfectly correlated with quality. Expensive products can be disappointing. Cheap products can be excellent. But extremely cheap products are usually cheap for a reason, and extremely expensive products are usually expensive for a reason.

When comparing products, consider price as a data point. A cheaper product with 4.0 stars might be better value than an expensive product with 4.5 stars.

Your Own Testing

Online reviews are just one input. Your own testing and experience matter. If a product doesn't feel right when you hold it, or if it doesn't fit your needs, no review score will make it good for you.

Visit RefDat Product Reviews for aggregated scores of Australian products, or read the product review methodology.

scraper trap unlimited downloads