Thread: On "inflated" star ratings

Posts: 88
Page: prev 1 2 3 4 5 6 7 8 9 next

Post by Arnaldo August 6, 2012 (51 of 88)
Geohominid said:

I take exception to that cynical statement. As the newest of the Site Reviewers, I get a very few free discs from companies. I purchase nearly all the discs I review, and thanks to a long experience in choosing likely winners for my collection, I'm mostly lucky in my choices, and have been surprised and pleased at how many 5 stars I have to award. In fact, I have had to review my own critical faculties to make sure that I am not being seduced into generosity in some way. Like Steve I have found that the standard of performance and engineering is significantly higher with SACD than RBCD...

Choice of recordings to add to one's collection is entirely the responsibility of the purchaser, not of the reviewer. I do the work to choose and buy my discs, I only write to express my enthusiasm or otherwise, not to satisfy a company who sends me freebies, and I am sure this is true for my fellow reviewers, whose integrity is impeccable...

In reference to my statement, "the beef is with official site reviewers' praising the Lord for almost every new freebie that comes their way," while itmay sound cynical to you, I see it as quite accurate in describing the standards of the official site reviewers, at least from the perspective of a reader. Kind of an Austin Power's Mini-Me version of Émile Zola's "J'accuse."

Still, you differentiate yourself from your colleagues in that you buy some, if not all, of your SACDs. Actually, it seems that you'd like to align yourself with the "occasional reviewer" category, which as I wrote, "are usually motivated to review only those recordings that stir very strong emotions, generally positive, thus their elevated percentage of high ratings." But you have published 464 reviews in 6 years, so occasional reviewer you're not. And then, a vast majority of these 464 reviews are 4.5 and 5 stars, again, a statistical improbability in the real and unreal world, whether in SACD or RBCD.

So, if your reply to my question, "reviewing, marketing or schmoozing?" is that you're reviewing, then quite frankly, your standards are way too low, whether in ratings or in text. In fact, you might consider disqualifying yourself as an official reviewer and join us commoners. Just compare your average opinions to professional reviewers to get a proper critical perspective. This also reveals a lack of experience with how the media world works. It's naive to think that companies, whether film studios or recording labels, send freebies or invite you to receptions to openly buy your stamp of approval. Instead, it's the more subtle schmoozing and coziness factors. If you don't know what I mean, then a quick course in media ethics might be in order.

Unfortunately, your comment that the "choice of recordings to add to one's collection is entirely the responsibility of the purchaser, not of the reviewer," very aptly describes how some of us perceive official reviews: as utterly irrelevant and lacking any kind of critical analysis. In other words, just like plain vanilla marketing. In the end, while I appreciate your reply, your reasoning seems more tailored to convince yourself than those critical of the official site reviewers' work.

Post by wehecht August 6, 2012 (52 of 88)
Arnaldo said:

1. Still, you differentiate yourself from your colleagues in that you buy some, if not all, of your SACDs.

2. So, if your reply to my question, "reviewing, marketing or schmoozing?" is that you're reviewing, then quite frankly, your standards are way too low, whether in ratings or in text.

3. ...If you don't know what I mean, then a quick course in media ethics might be in order.

4. In the end, while I appreciate your reply, your reasoning seems more tailored to convince yourself than those critical of the official site reviewers' work.

1. Do you know for a fact that the other site reviewers review nothing but freebies? That's what your statement implies.

2. Mr. Miller's reviews are extremely detailed, and each one is obviously the product of hours of work. They represent his honest opinion about each disc. If you think his opinion is not valuable to you don't bother to read his reviews.

3. I'm sure every record company sending out review copies hopes to engender favorable reviews, or at least a quotable blurb, but they also know they may get just the opposite. Comparing a $15 free sacd with a Hollywood press junket and one on one interview time with whatever airhead the studio is hyping is patently ridiculous. Further your implication that the site reviewers are ethically challenged is gratuitously insulting.

4. There is no reasoning that will convince you on this subject since you are determined to see everything from statistical improbabilities to ethical lapses to nefarious plots, all intended to deprive you of the guidance you seem to require.

This has gotten really old, why not give it a rest?

Post by krisjan August 6, 2012 (53 of 88)
Arnaldo said:

...a vast majority of these 464 reviews are 4.5 and 5 stars, again, a statistical improbability in the real and unreal world, whether in SACD or RBCD.

I believe there is a faulty premise in your reasoning: you infer that SACD ratings should be normally distributed about a given average(not sure what your understanding of statistics is, but I'm assuming you know what a "normal" distribution is). With that assumption, the average rating would be ~2.5 stars with very few 1's or 5's. But I don't think a normal distribution is the best model for this reality.

I would expect the performance and sonic ratings of SACD's to be much closer to a Poisson's distribution where the majority of items are ranked near the high end with a tail that drops down quickly to the low ratings (look up Poisson's distribution if you aren't familiar with it). Why? Because we are talking about ratings of recordings done by high level professionals (both those performing and those doing the recording) so there is an implicit expectation of high standards. Yes, an occaissional clunker gets released every now and then (hence the tail of the Poisson) but the large majority are high quality (and hence, highly rated) products. With this statistical model, the "average" can easily be quite high (say, 4.5 out of 5) and thus I don't believe there is a "problem" with the site reviewers ratings.

Post by Polly Nomial August 6, 2012 (54 of 88)
krisjan said:

I believe...

+1

Post by Beagle August 6, 2012 (55 of 88)
+1

--Or perhaps it is a Probability Cloud, such as defines the location of an electron around a nucleus: the electron MIGHT suddenly appear near Alpha centauri but it MUCH MORE PROBABLY will appear very near the nucleus (i.e., the disc you purchase might be filled with wikileaks from the Pentagon, but it probably has a professional performance of thorough-composed music professionally recorded, packaged and shipped).

Post by sacd_fan_2007 August 6, 2012 (56 of 88)
I'm not volunteering to enter review scores into Minitab and run a statistical analysis. :-) Instead, I'm asking the editorial board to more clearly identify their top choices.

In the 5 years since getting a first SACD player and joining this forum, I've collected/purchased about 120 SACD's and jettisoned 15-20. There are reviews from my earlier days on this forum that I'd be pleased to see disappear. There are discs in my SACD collection that are dull compared to some RBCD's I've kept for years. I perceive new SACD's as more consistently recorded now then they were in 2007. Disc players are much more accurate for the money now. My second SACD disc player is much higher quality than the first and makes many RBCD's sound nice. With more experienced ears, more live music experiences, and better equipment, I would re-calibrate my past reviews.

If this were your first day on SA-CD.NET where do you start? Are the Top Recommendations still valid in terms of 2012 quality? How many of those are now for sale only in a secondary market for the price of a cheap tablet computer?

Post by canonical August 7, 2012 (57 of 88)
krisjan said:

I would expect the performance and sonic ratings of SACD's to be much closer to a Poisson's distribution.

Sounds fishy to me.

First, the Poisson has an infinite right tail, whereas sa-cd.net only allows a top mark of 5.

Second, if X ~ Poisson(lambda):

a) if lambda is small, the distribution would be skewed to the right (meaning that most SACDs would score 0 or 1 ... which is exactly the opposite of what is observed), so that doesn't work, and

b) alternatively, if lambda is large, the Poisson tends towards a Normal, which is what you are arguing against, so that doesn't work either.

Post by Polly Nomial August 7, 2012 (58 of 88)
Use http://mathworld.wolfram.com/BetaDistribution.html with alpha>beta; that gives you a negatively skewed distribution.

Post by abind August 7, 2012 (59 of 88)
Zeus,

Please don't change anything. I like the current system exactly as it is.

More information is always better, and right now there are three different sources of information: The numbers of recommendations, the reviews with their star ratings, and the discussion. Each of these offers information, and I find that by reading all of those sources, I can usually get a fairly good idea of the quality of a particular disc.

Reviews of all kinds, whether of music, movies, or restaurants, need to be read with some skepticism. One must take into account the reviewer's point of view, and experience, and taste. Nonetheless, I find that the current system works very well for me.

Yes, there might be an over-enthusiastic review with 5 stars. But if, for example, the numerical recommendations are only 14/19, then I know there might be a problem with that particular disc. I understand that some people might object to the 5 star rating, and some might feel that 14 / 19 is too high a rating for a poor recording. Nonetheless, if one has been reading a variety of reviews over time, he can interpret what those numbers mean.
The number of stars is all relative. Complaining about the number of stars that others give reminds me of the guitarist in "Spinal Tap" who made his amplifier louder by changing the numbers on the volume dial.

Post by hiredfox August 7, 2012 (60 of 88)
krisjan said:


I would expect the performance and sonic ratings of SACD's to be much closer to a Poisson's distribution

Usually in shoals? But that's three dimensional...

Page: prev 1 2 3 4 5 6 7 8 9 next

new post