Thread: Debunking Meyer and Moran

Posts: 178
Page: 1 2 3 4 5 6 ... 18 next

Post by canonical September 24, 2009 (1 of 178)
Over in the BIS thread, fafnir referred to a paper by Meyer and Moran:

-----------------------------------------------------------------------

> In fact, tests were conducted on RBCD vs SACD (stereo mode only) by knowledgeable people
> and was published, I believe, by a chapter of the AES. ... It is available on:

> http://drewdaniels.com/audible.pdf

> There may be some moaning and gnashing of teeth after reading, but it certainly is thought-provoking.

----------------------------------------------------------------------


Knowledgeable people? Thought-provoking? You seem a little gullible, Mr Fafnir. It only takes a little thought and a little research to unravel this puzzle ...


Short Version:
______________

If you go to just a little trouble, you can actually find the list of SACDs that Meyer and Moran used in their test. And, if you do so, and check them out, .... it turns out, in a case of almost absolute academic incompetence, that the authors included in their list of test SACDs many discs that are NOT hi-rez at all /// Many of their test discs are actually 1980s early digital recordings, with no more resolution than CD itself. It's almost laughable.

Their 'study' is flawed and manifestly incompetent. It is equally puzzling how this could ever have got past a referee, assuming the journal uses same?



Long version
____________


The Meyer / Moran paper proceeds by theoretically starting with:

(a) a high-rez SACD, and then

(b) downsamples that to CD quality,

and then plays both (a) and (b) to audiences.



The difference between (a) and (b) should be:

(i) the loss of high frequency information above 20kHz (CD only supports up to 20kHz, whereas SACD theoretically supports uo to 100kHz and practically 50kHz after filtering). Even though the human ear does not hear above 20 kHz, the interplay of harmonics etc can effect frequencies which we do hear.

(ii) increased quantisation, loss of information etc from downsampling from hi-rez DSD resolution (say effectively 20 bit 192kHz) to CD quality resolution (16 bit 44.1 kHz).


The first indication of a problem with the Meyer / Moran paper is that it is fundamentally inconsistent with the earlier results of

T. Oohashi, E. Nishina, M. Honda, Y. Yonekura, Y. Fuwamoto, N. Kawai, T. Maekawa, S. Nakamura, H. Fukuyama, and H. Shibasaki. Inaudible high-frequency sounds affect brain activity: Hypersonic effect. Journal of Neurophysiology, 83(6):3548–3558, 2000.

http://jn.physiology.org/cgi/content/full/83/6/3548


which finds that, for subjects in double-blind tests:
*** EEG monitoring of their brain activity showed statistically significant enhancement in alpha-wave activity in their brains when the high-frequency sounds were included,

*** The subjects in the study found the sound containing the high-frequency components to be more pleasant than the same sound lacking the high-frequencies.
.

Thus, item (i) alone ... the high-frequency spectrum in SACDs (and other hi-rez recordings) ... should give rise to a statistically significant difference in the Meyer and Moran paper too. This is before any effect from the quantisation downsampling is added in. And yet, Meyer / Moran fail to reference this paper, and fail to explain or account why they cannot reproduce the earlier results.


Resolving Meyer / Moran
____________________

So why do Meyer and Moran fail to reproduce earlier double-blind tests? And fail to find any differences? It turns out that many of the recordings that Meyer and Moran selected to play their testers were NOT hi-rez to start with. This provides the ridiculous situation that they start with CD quality recording, albeit sold on an SACD disc, and then 'convert/downsample' it back to CD. It almost beggars belief. A listing of recordings used by the authors is provided here:

http://www.bostonaudiosociety.org/explanation.htm

For example, the Perahia Mozart SACD -- there is no hi-rez Mozart concerti recording by Perahia. There is one old analogue recording converted to DSD, and an early digital recording in the 80s that is basically CD quality to start with. Neither are appropriate to use in the test. Pink Floyd's Dark Side of the Moon is also analogue, and would not contain high frequency sounds. Nor is it acoustic. Same goes for Alan Parsons project. The BSO, Saint-Saens, Organ Symphony SACD (RCA 82876-61387-2 RE1) is a 1959 recording.
Steely Dan, Gaucho is NOT Hi-rez: it is originally a 1980 LP /1984 RBCD release.
The Carlos Heredia, Gypsy Flamenco is not hi-rez either: it is a mid-1990s recording that pre-dates the entire SACD format.


Under the circumstances, it is difficult to find anything of interest or thought-provoking about the Meyer / Moran paper ... The only real quandary is perhaps how this ever got published? The whole thing is frankly somewhat professionally embarrassing for all concerned.

Post by DSD September 24, 2009 (2 of 178)
Great investigation work Colin, Meyer and Moran must have purposely avoided DSD recorded SACDs to get the results they wanted.

Imagine if they used Telarc DSD recorded SACDs in the tests who use microphones with frequency response to 50kHz? Or a DSD recorded SACD using the new Sanken CO-100K microphone which goes to 100kHz. How different the results would have been.

Sadly it seems they were out to prove that CDs and SACDs sound the same.

Post by fafnir September 24, 2009 (3 of 178)
canonical said:

And yet, Meyer / Moran fail to reference this paper, and fail to explain or account why they cannot reproduce the earlier results.

As Kal Rubinson pointed out in a previous post, "None of Oohashi's papers have been replicated or confirmed by any other lab. In addition, there have been many criticisms of his methods and extracted conclusions." Perhaps a little research would also have revealed this, and prehaps you too might reconsider including yourself among the "gullible." Don't take this too seriously - I don't consider myself to be the least bit gullible, and so I must retaliate to a limited degree.

In any event, I stated that I found the test interesting, not that I completely agreed with it and that its methodology was good, which it was, apart, most likely, from the disc selection. I was deeply suspicious that the tests did not show an increased dynamic range amoung the SACDs. The increased bit depth of SACD should have been obvious if the discs were well-chosen. As far as hypersonic frequency relevance is concerned: It hasn't been proven to my satisfaction and I don't believe in it.

Maybe someday there will be better test that will satisfy everyone, but I'm inclined to doubt it.

Post by Kal Rubinson September 24, 2009 (4 of 178)
canonical said:
Thus, item (i) alone ... the high-frequency spectrum in SACDs (and other hi-rez recordings) ... should give rise to a statistically significant difference in the Meyer and Moran paper too.

It does not predict that. Putting aside my doubts about M&M and also about Oohashi's conclusions, a statistically significant change in EEG does not necessitate any conscious or measurable change in perception. As they say in the cop-out discussions, "the two sets are not inconsistent with each other" but neither are they corroborative or causal.

Kal

Post by trntbl September 25, 2009 (5 of 178)
Why isn't an old analog master tape high rez? I have many SACDs made from analog tapes and they sound fabulous, even on stereo.

Post by audioholik September 25, 2009 (6 of 178)
trntbl said:

Why isn't an old analog master tape high rez? I have many SACDs made from analog tapes and they sound fabulous, even on stereo.

trntbl, analog tape is analog tape, the study says there's no difference between SACD and 16/44.1

Post by fafnir September 25, 2009 (7 of 178)
trntbl said:

Why isn't an old analog master tape high rez? I have many SACDs made from analog tapes and they sound fabulous, even on stereo.

There is no reason why the original analog tapes cannot contain some hypersonic information. However, I doubt that many of them do - mics and analog recorders being what they were. The issue is how many of discs used in the test had some hypersonics; this is not really known, but there is a high probability that some - perhaps even a majority -did not.

In all fairness to the authors of the tests, who have been vilified on this forum, I contacted Mr. Meyer by email last evening, and he was kind enough to respond at length.

Herewith is his reply unedited except for an introductory paragraph that was omitted at his request and the last paragraph, which is not germane:

"About all I can say about our tests is what the paper says: We found no one who could consistently tell the difference between the source and the CD-quality version of it -- not even one single score reaching the 95% confidence level -- no matter what source we used, or which of the four systems (plus some headphones) we used, or what material we chose. Given the way the differences have been described, that is a surprise.

We have been accused of trying to produce a null result, which is both insulting and untrue. We were looking to find ANY combination of the above variables that produced positive results, and we broke the usual forced-choice testing rules to bias the experiment in favor of finding a correlation -- switching multiple times among the two sources and the unknown before asking for an identification; letting the subject control the switchbox and choose source material and specific musical passages he or she thought were the most revealing, and so on. We really wanted to zero in on any detectable differences.

The ABX box maintains double-blind conditions for everyone in the room, so even the experimenter can take the test, which I frequently did, though I didn't count my answers when I was also running the test for others because I couldn't concentrate that well. Obviously, I never achieved positive results either, even when I took the test alone in a properly relaxed and leisurely way, except under the special conditions we stated in the paper (no music playing, or only room tone, and the gain cranked way up so the CD link's noise was audible). I never heard a difference on music at realistic playback levels. If I had done so I would have pursued my own experiments and included the results.

The question of audibility of hypersonic sounds should have died years ago, as there is plenty of research indicating they are inaudible, but negative results are never as conclusive as positive ones in perceptual testing, so the matter will not die. There was a Pioneer paper from the early '90s (I think) that keeps popping up; I tried to reach the authors at the time to find out more about it but they never responded. They found no evidence that their subjects were consciously aware of differences, only that some vaguely described measure of brain activity as seen on an EEG was different."

Post by audioholik September 25, 2009 (8 of 178)
fafnir said:

"About all I can say about our tests is what the paper says: We found no one who could consistently tell the difference between the source and the CD-quality version of it"

but what was the source? a DSD recording?

it looks like they set their confidence level at 95%! but at the same time allowed for introducing great variablity in the source material :/

Post by trntbl September 25, 2009 (9 of 178)
audioholik said:

trntbl, analog tape is analog tape, the study says there's no difference between SACD and 16/44.1

Yes, I understand that. My point is, this study shouldn't be ignored because they used SACDs made from old analog master tapes instead of SACDs from original hi-rez digital recordings. The difference between 16/44.1 and DSD transfer should have been audible, if major difference exists.

I find the result of study not surprising at all. I would in fact find it surprising if there was a person capable of telling stereo SACDs from CDs with 95% success rate or better. (My own comparisions tell me that there is slight lack of grain with stereo-SACD layer, but I'm quite sure that I wouldn't be able to tell it in blind test.)

Post by audioholik September 25, 2009 (10 of 178)
trntbl said:

My point is, this study shouldn't be ignored because they used SACDs made from old analog master tapes instead of SACDs from original hi-rez digital recordings.

trntbl, I'm sorry but I don't see the point of using analog recordings from 50s and/or early digital recordings in a test comparing different recording formats (in this case DSD and 16/44.1), don't get me wrong, I don't have anything against putting 24/44.1 recordings on SACDs for example, but if you want to make final conclusions about the format it seems logical to use DSD and DXD recorded SACDs.

Page: 1 2 3 4 5 6 ... 18 next

Closed