I have had some problems like that with the drill mill but not the Tom Senior. Two possibilities spring to mind, either the spindle bearings in the DM are not as good, of the speeds being different can affect vibration and some kind of resonance.
These problems are difficult to diagnose without a lot of time consuming testing.
Even taking the tool out of the collet and refitting it might make a difference.