klinemj
Emo VIPs 
Official Emofest Scribe
Posts: 15,096
|
Post by klinemj on Dec 24, 2016 15:08:35 GMT -5
So there's a difference. Do you choose the component you like better or you've convinced yourself is more "accurate" whatever that means? A positive double blind test doens't tell you which one is better. Just which one there is a difference. I suggest picking the gear that sounds better to you after using it as you normally would if the differences are worthwhile to you. Actually, if you are referring to a test like the ABX comparator test, you are right. If you are referring to all types of double blind testing, you are wrong. There are forms of blind testing designed to assess "preference" and they are used all the time in consumer research. I have done them hundreds of times in my work. I have long advocated, here and in other forums, that proper audio blind tests need to asess preference and preference for specific attributes...otherwise they are worthless. And in an earlier post you mentioned that a null finding does not mean there is no difference. To those that did not understand, this point is spot on. If no difference is detected, there may actually be one. But, the test did not identify it. That said, the difference might be very very small and so its importance may be questionable in terms of real world relevance. Mark |
|
|
|
Post by Boomzilla on Dec 24, 2016 15:12:12 GMT -5
You IMO are completely incorrect!....The one DAC review I have read that was blind was very controlled and confirmed that very few if any folks could distinguish high end DAC's... And that's EXACTLY what our "informal" test found. So why is the blind test valid but ours not? |
|
Deleted
Deleted Member
Posts: 0
|
Post by Deleted on Dec 24, 2016 15:13:49 GMT -5
You IMO are completely incorrect!....The one DAC review I have read that was blind was very controlled and confirmed that very few if any folks could distinguish high end DAC's... And that's EXACTLY what our "informal" test found. So why is the blind test valid but ours not? Because one is scientific. The other is guessing Let's be honest boom. You are anti scientific. You refuse to take for measurements to integrate subwoofers. And you refuse DBT is valid. Which of course is fine if you want to be less than accurate with your system and your claims of what gear is better. After all I have said it a bunch. You LOVE THE JOURNEY. most love the destination. Inn this crazy hobby |
|
|
|
Post by garbulky on Dec 24, 2016 15:18:34 GMT -5
Thanks klinemj yes I meant double blind in terms of listening. Also in terms of relevance of a null result I see that you use the words might and may. Not "is" and "does". Which is correct in your usage and an important distinction. |
|
DYohn
Emo VIPs
Posts: 18,494
|
Post by DYohn on Dec 24, 2016 15:27:30 GMT -5
And by the way boomzilla, never use the word "absolutely" when discussing something that is dependent on human perception. A statement like that is almost always wrong.  |
|
|
|
Post by knucklehead on Dec 24, 2016 15:35:24 GMT -5
And by the way boomzilla, never use the word "absolutely" when discussing something that is dependent on human perception. A statement like that is almost always wrong. So true!!! However scientific data can be had from subjective human perception. |
|
|
|
Post by copperpipe on Dec 24, 2016 15:39:00 GMT -5
Agreed, you haven't proved there is no difference. But again, it's an extremely strong indicator. Keep in mind, the lower your overall score in spotting a difference, the higher the probability that you didn't actually get the ones right that you thought you did. If your overall score is (say) 80% on spotting differences, then it's extremely likely there are differences and you are quite often picking them out. They are significant. If your overall score is (say) 20% on spotting differences, then it's a very high probability that you didn't spot any difference at all, you just guessed right a few times. That probability of getting lucky goes up as your overall score goes down. The closer you get to 50% probability, the closer your score is to somebody making a random guess. The other stuff is more statistical which I'll bow out of. I don't know enough of. Statistics aren't as cut and dry as they may appear to be. If your overall score is 20% then there is a high probability that there is a problem with the experiment or you if you are doing worse than random chance. Still doesn't tell you there isn't a difference in real world listening. A positive result PROVES there is a difference. Garbulky, a positive result does not PROVE there is a difference. You can randomly guess it and get lucky as well. You really should brush up on stats since it can help you here. There is no absolute proof here one way or the other, only (forgive me) "50 shades of grey". The lower the score, the less likely there is a difference to you. Who cares about the equipment at that point, or absolute proofs and nulls etc, you can't reliably tell the difference and you are the one buying the shiit. It only matters if you can spot the difference. And that is what the DBT will tell you. |
|
|
|
Post by garbulky on Dec 24, 2016 15:54:49 GMT -5
The closer you get to 50% probability, the closer your score is to somebody making a random guess. The other stuff is more statistical which I'll bow out of. I don't know enough of. Statistics aren't as cut and dry as they may appear to be. If your overall score is 20% then there is a high probability that there is a problem with the experiment or you if you are doing worse than random chance. Still doesn't tell you there isn't a difference in real world listening. A positive result PROVES there is a difference. Garbulky, a positive result does not PROVE there is a difference. You can randomly guess it and get lucky as well. You really should brush up on stats since it can help you here. There is no absolute proof here one way or the other, only (forgive me) "50 shades of grey". The lower the score, the less likely there is a difference to you. Who cares about the equipment at that point, or absolute proofs and nulls etc, you can't reliably tell the difference and you are the one buying the shiit. It only matters if you can spot the difference. And that is what the DBT will tell you. I get what you are saying my friend. Also appreciate that you've been very cordial in our interactions. I have to disagree with what you posit. Because DBT tells me one thing - a null result. The rest of my experience tells me another. Neither really disagree with each other imo. A null result doesn't contradict my experience. Though it certainly doesn't confirm it either. Subjective listening makes more of a real difference to my experience than a DBT test saying null. I believe I obtained better audio - for me - that way. YMMV. I go with what makes a difference to me. If DBT matches your experience or your intepretation of a null result then I recommend you to go with that experience, as you arrive at audio nirvana much easier than I would! You would not need to fiddle aroudn with gear. Get one that meets reasonable specifications - not hard at all in this day! Then concentrate on what matters to you. |
|
DYohn
Emo VIPs
Posts: 18,494
|
Post by DYohn on Dec 24, 2016 16:01:56 GMT -5
And one more point that may have already been made but I'm too lazy to read everything. Perceptual testing of any sort, "blind" or otherwise, can never be used to "prove" anything. At best it can provide statistically significant correlations or group consensus opinions.
|
|
|
|
Post by geebo on Dec 24, 2016 16:06:17 GMT -5
Let me ask it this way: Now that you have listened to them and have heard differences in them that you can identify, if one of them was put into your system could tell which one it was just by listening to whatever and for however long you needed? And could you do this for all three? Maybe? It would be easier if it is one or the other. It would be easier than a DBT test. Probably closest to getting "objective" with my subjective listening style. It would hard for me to do so because it would mean my wife would have to figure out how to hook up audio component and cover it with a sheet for a month. And she's already told me "are you kidding me?" Then she'd have to adjust the volume as I wouldn't be able to look at the remote control and universal remotes are confusing. Also the cloth would block the remote in the first place. So there are practicalities for me...as well as the desire! I'm happy with how I listen. Good dancer.  |
|
|
|
Post by sahmen on Dec 24, 2016 16:27:29 GMT -5
2 Questions: 1. Do Double blind testing of all forms on DACs tend to lead to inconclusive results at best. (My answer = Yes, it appears so.). 2. Does that mean or prove that the differences between the sound signatures of different DACs do not actually exist? (My answer: Absolutely not!) I am not actually sure whether anyone is drawing conclusion 2 (differences don't exist) from premise 1 (inconclusiveness of Double blind testing), but, from what I have read here, I am also not sure whether someone is not drawing that conclusion, hence my own input which follows: I do not know about all the testing (with their various charts, graphs, curves, gradients, and other stats and diagrams) but I do know that there are differences between the sq of different DACs, and I do not need any science to prove that to me. Proof: let's keep things simple. The XDA-2 and the DC-1 do not sound the same. I find that to be self-evident enough not to require any sophisticated testing or instruments to tell me that, except my own ears of course. One day, I attempted to replace the DC-1 with the XDA-2 in my Airmotiv 6s, desktop near-field system, because I needed to use the DC-1 in my second HT system. The sonic result of the swap was sooo disappointing each time I tried it that I ended up leaving things where they were until I was able to replace the DC-1 with an Audio gd headamp/DAC combo that gave me the results I was looking for, although it also presented another set of sonic differences. That is for speakers. When using headphones, I find the differences to be even more strongly affecting, maybe because headphones engage our nervous system in a different way than speakers do, given the relatively more intense immediacy of their presentation. Something like excessive treble harshness or glare on headphones, can, depending on their intensity, produce nervous, and even muscular, tensions in my body that I cannot tolerate for more than a few seconds at a time, even at relatively low listening levels. Conversely I will have to crank up the volume on speakers to levels of high distortion in order for the sound to affect me in a similar unbearable way. Now to come back to the subject of DACS, I just did a DSP upgrade on my Audiogd reference 5.32 , and also upgraded the USB32 DAC module on it to an Amanero384 USB combo. Those two upgrades are supposed to bring major improvements, and in many ways, they do. The DAC has become a little more resolving in the mids and highs, which have, in turn, been brought a little more forward than before, but at some expense of bass impact, the presence of which has slightly receded, even though it still reaches as deep as it previously did when called upon to do so by music from the source . Detail retrieval is slightly better than before, but some tendency toward treble glare/harshness has crept into the sound field, and this heightened inclination to treble harshness was not there before. I used to love this particular DAC for its R-2-R smoothness, but now that relaxing smoothness is all but gone, because of these two upgrades. It is to the point where I have to EQ down the treble on some songs in Amarra software in order to make listening to certain tracks tolerable, even though the same tracks used to play smoothly before the upgrade. This upgrade was meant to improve the sq performance of the DAC, but in spite of the improvements, I have been brought more than once to the point of considering to sell my Audiogd stack altogether, because of the occasional treble glare which I cannot stand, especially, when using headphones. Fortunately, I have DIRAC room/headphone eq functionality in the Amarra Symphony software that is installed on the mac mini in this particular rig, so I have the option of re-calibrating the rig Post-upgrade in order, hopefully, to bring things back to settings that are more pleasing to my ears. If that recalibration does not work, then I have to seriously consider selling the Augiogd stack, with the DAC upgrade and all. In this example, I chose the same DAC (AGD Ref. 5.32) on purpose, in order to show how a couple of tweaks in one and the same DAC. can cause dramatic changes in its sound signature. The example also has the advantage, hopefully, of confounding skeptics who have the habit of explaining every perception of difference in sq between different DACS as a placebo effect, or as symptoms of confirmation/expectation biases based on price difference (this is essentially the same DAC with two different USB modules, and the DSP upgrade is free). If still in doubt, just get yourself two Audiogd Ref. 5.32 DACs, one with the DSP 7 firmware and Amanero384 upgrades, and one without. Test them in your system and let me know whether you're hearing the same sound or not. If that sounds too complicated, you can simply do a DC-1/XDA-2 face-off  . Do I believe different DACs produce different Sound signatures and sound qualities? I frankly don't see how anyone can doubt that. But, YMMV, as in anything audio. Wow! Did I sit here and type all of that? I can't believe it!    |
|
|
|
Post by yves on Dec 24, 2016 16:30:45 GMT -5
However, the real reason the golden ear folks don't like blind tests is because the blind tests embarrass them and prove them wrong in most cases.IMO the real reason is because these tests often turn out inconclusive, and, in almost every case that I am aware of there's been little or no hard data published to be able to properly investigate if the test procedure actually was VALID. While on top of that, all sorts of ridiculous conclusions are being based on inconclusive test results by many. To illustrate the problem, the Meyer & Moran study, which CLEARLY wasn't a valid double-blind listening test, has caused a lot of self-defined objectivists to hang on to their strong belief that this study is reliable evidence in support of their claim that listening to Hi Rez audio (as opposed to standard Redbook CD) is pointless to any and all humans. What's more, now that Meridian's Bob Stuart and colleagues have published evidence that points in the opposite direction, the self-defined objectivists CLEARLY aren't going to accept it because... well, that's what self-defined objectivists are like. If they see something that confirms their own theory, they immediately call it reliable evidence. If they see something that refutes that, they immediately call it unreliable. What's truly embarrassing here is that this kind of manipulative behavior of theirs is what they actually refer to as SCIENCE. |
|
|
|
Post by Boomzilla on Dec 24, 2016 17:01:41 GMT -5
Nothing I've seen in this thread has moved me one iota from my original post.
|
|
Deleted
Deleted Member
Posts: 0
|
Post by Deleted on Dec 24, 2016 17:07:09 GMT -5
Monku, I have to mention one point here. I know of many folks who have spent for mainly psychological reasons thousands of IMO useless dollars on $1000 DAC's for example plus expensive speaker connectors, cables, and other very questionable gear, way more than I have spent on my entire system! Count me among them. Not thousands of dollars, but certainly a fair amount!  Plus all the gear you have still hidden from Monku Wife out in the garage! |
|
klinemj
Emo VIPs
Official Emofest Scribe
Posts: 15,096
|
Post by klinemj on Dec 24, 2016 17:08:04 GMT -5
Thanks klinemj yes I meant double blind in terms of listening. Also in terms of relevance of a null result I see that you use the words might and may. Not "is" and "does". Which is correct in your usage and an important distinction. Uh...listening? Well, yes, that is what we are talking about. I am referring to a specific research methodolgy whic applies to listening, taste, visuals, anything...there are techniques to asses liking/preference thatgo way beyond assessing if someone can tell a difference. On "might/may" versus more definitive words, all experiments have error. There is not definitive answer from almost all experiments due to error. There is this little science called statisitcs that explains it...sort of. About the only definitive answer is when someone lights a stick of dynamite, holds it against their chest, and lets it explode. The observers witness the event and one says, "did that idiot actually just blow himself up". And the other says, "yep" Mark |
|
klinemj
Emo VIPs
Official Emofest Scribe
Posts: 15,096
|
Post by klinemj on Dec 24, 2016 17:09:04 GMT -5
Nothing I've seen in this thread has moved me one iota from my original post. None of us expected you to change your mind. We know you are stubborn. Mark |
|
Deleted
Deleted Member
Posts: 0
|
Post by Deleted on Dec 24, 2016 17:20:18 GMT -5
So there's a difference. Do you choose the component you like better or you've convinced yourself is more "accurate" whatever that means? A powsitive double blind test doens't tell you which one is better. Just which one there is a difference. I suggest picking the gear that sounds better to you after using it as you normally would if the differences are worthwhile to you. The ones I have read and conducted all determine if there is a difference AND which one is preferred. |
|
|
|
Post by garbulky on Dec 24, 2016 17:22:37 GMT -5
A powsitive double blind test doens't tell you which one is better. Just which one there is a difference. I suggest picking the gear that sounds better to you after using it as you normally would if the differences are worthwhile to you. The ones I have read and conducted all determine if there is a difference AND which one is preferred. So have mine. But I didn't have to cajole my wife to switch in DACs. (Sorry, I'm just playing here. My wife left me long ago when I put in the room treatments. ). |
|
|
|
Post by beardedalbatross on Dec 24, 2016 17:23:26 GMT -5
Nothing I've seen in this thread has moved me one iota from my original post. I don't see why it would. You said it yourself: I would be very surprised if there is anyone in the world who can hear the difference between a DC-1 and a Schiit Yggdrasil in any real scientific test. Heck, I don't think Jason Stoddard would claim he can hear the difference between the Yggdrasil and Gugnir. |
|
Deleted
Deleted Member
Posts: 0
|
Post by Deleted on Dec 24, 2016 17:23:40 GMT -5
Then how about this: Live with each DAC for whatever period of time you need one at a time. A week, a month, whatever it takes to learn those subtle differences that differentiate the three. You would know which one you were listening at any given time. Then have someone put one of the three back into your system and again listen to it for whatever time you need. But this time you would not know which one you were listening to. Would you be able to identify which one it was? 100% of the time? No! About 50% of the time! |
|
.
