I would definitely agree that, when you engage the ASRC, it produces a very slight, but sometimes audible, change in the "sound".
Most people who've tried it both ways, on both the DC-1 and the XDA-2, agree that there is a subtle but sometimes noticeable difference.
As with virtually all complex devices, the ASRC does in fact introduce errors to the signal, as well as reducing jitter.
However, according to the manufacturer's specifications, the errors introduced by the ASRC are on the order of 130 dB below full output.
(This is FAR below the resolution of a Red Book CD, and very close to the best possible resolution that can be resolved a 24 bit file.)
Therefore, by those specs, it seems far more likely that the difference is due to a reduction in jitter.
Of course, if you do some research, you will find that the audible contribution made by jitter, and whether it is audible or not, is also a hotly debated topic.
Also note that the reduction in jitter offered by the ASRC will be different, depending on the input and source you're using, and the jitter characteristics of the source itself.
The other thing is that the audible effects of jitter are very subtle, and difficult to describe, and "less artificial" also isn't something we can measure or specify.
Therefore it's actually possible that the sound with the ASRC engaged, which you consider to sound "less natural", is in fact the more accurate choice of the two.
The bottom line of all this is that you are not the only one who hears a difference when the ASRC is engaged.
And, since we can't clearly define exactly what the difference is, whether it is more or less natural in sound is a subjective judgment call.
Therefore, it's a good thing that we can disable the ASRC, so we can hear both options, and choose the one that we prefer.
What I would note, however, is that the difference WILL depend on the type of input you're using and your source.
Therefore, your mileage will almost certainly vary, and it's really something that must be decided on a case by case basis.
Also, as a generalization, engaging the ASRC removes jitter, which is one way in which sources may vary from each other.
So, with the ASRC DISENGAGED, you are more likely to notice differences between sources - for better or worse.
And, with the ASRC engaged, you should expect differences between sources to be eliminated, or at least reduced.
I have had this setting turned on with my DC-1 since I got it, I always just assumed it was providing a benefit and never really questioned it or played with the setting. After getting the Bluesound Node 2i and listening to its internal DAC a little, I started to notice that while the Node's DAC didn't quite have the clarity of the Stealth, it tended to sound a little more "natural". Long story short, this lead to me playing around with the stealth and ultimately turning off the ASRC, now the Stealth maintains its clarity but has a less artificial sound to it. It is a very very subtle change but one I cannot ignore. Has anyone else experienced this?
