|
Post by monkumonku on Aug 3, 2015 20:54:21 GMT -5
... and the conclusion would be that there was no difference since that's what 99% of the sample population reported. That conclusion would be unwarranted. The only conclusion that can be drawn from that result is that any differences present are undetectable by 99% of listeners. Whether a difference actually exists is not dependent upon the judgments of listeners, no matter how many of them you test. That is true, but on the other hand what is the purpose of A/B/X testing? Perhaps there may be a difference but if 99% or a significant percentage of people cannot detect a difference, then isn't that a valid conclusion from doing the test?
|
|
|
Post by garbulky on Aug 3, 2015 21:41:12 GMT -5
I know that the ABX testing is used in a wide variey of applications. But what makes us think its ability to judge subtle details or subtle differences AT level matched volumes works in audio perception? Which is a very different type of perception. So we have to do a bunch of stuff. 1. Find out if subtle differences can be identified in a double blind test FIRST. We can't just ASSUME these will be identified. What is smallest discrepancy that can be identified in double blind testing?
2. You have to test the ability of other non double blind tests to identify subtle differences. For instance....is there any kind of test that identifies differences better? For instance, if the signal was not level matched. If the listener knew what component was happening. Etc. etc. The goal is to see if there is a better measure. If there is... for instance if people did better with non level matched sources then we have to investigate why.
3. You have to test each assumption - Level matching, AB, ABX, and double blind testing.
What I see so far is just an assumption that double blind testing must work for the smallest difference the human ear can actually tell.
|
|
|
Post by yves on Aug 3, 2015 22:05:27 GMT -5
Ah but statistics DO lie. In this example, the 99% couldn't tell a difference. Maybe they couldn't hear high frequencies? Maybe they had presbyacusis? Maybe they didn't want to be taking the test at all & just didn't care? But if even ONE person can reliably hear a difference, then the difference DOES exist. The other 99 just didn't hear it. The one might have been lucky on the first pass & guessed correctly, but if the one can consistently tell A from B, then it can NOT be ascribed to random chance - A & B ARE different - Period. NO. If 99% could tell no difference there was no (significant) difference. Welcome to science. "{snip} In the extreme case where several or all systems are found to be fully transparent, {snip} "Welcome to a new and improved version of science.
|
|
|
Post by garym on Aug 3, 2015 22:22:04 GMT -5
Perhaps there may be a difference but if 99% or a significant percentage of people cannot detect a difference, then isn't that a valid conclusion from doing the test? Sure it is. Those listeners would be unwise to pay more for a component with (say) lower measurable distortion if they can't tell the difference between 0.1% THD and 0.5%. But it doesn't mean that difference doesn't exist, or that *some* people may be able to hear that difference.
|
|
|
Post by monkumonku on Aug 3, 2015 23:12:10 GMT -5
Perhaps there may be a difference but if 99% or a significant percentage of people cannot detect a difference, then isn't that a valid conclusion from doing the test? Sure it is. Those listeners would be unwise to pay more for a component with (say) lower measurable distortion if they can't tell the difference between 0.1% THD and 0.5%. But it doesn't mean that difference doesn't exist, or that *some* people may be able to hear that difference. I agree. That's the factual part but I guess my point is practically speaking, if 99% can't hear a difference then it's not something to make a big deal about. Except for the 1% who can hear a difference, of which 99% of those 1% are golden-eared audiophiles.
|
|
|
Post by drtrey3 on Aug 4, 2015 10:56:31 GMT -5
I have never come across anything in the audiophile press indicating that a writer had fear of ABX testing.
I have read lots of article either lauding the process or critiquing it.
Perhaps the "fear" is a projection from someone who has a hard on for ABX and little tolerance for people who do not.
Trey
|
|
|
Post by thepcguy on Aug 4, 2015 15:32:34 GMT -5
People say I'm crazy doing what I'm doing, Well they give me all kinds of warnings to save me from ruin, When I say that I'm o.k. they look at me kind of strange, Surely your not happy now you no longer play the game, People say I'm lazy dreaming my life away, Well they give me all kinds of advice designed to enlighten me, When I tell that I'm doing Fine watching shadows on the wall, Don't you miss the big time boy you're no longer on the ball? I'm just sitting here watching the wheels go round and round", I really love to watch them roll, No longer riding on the merry-go-round, I just had to let it go
|
|