Some interesting ideas here. I use centre drills mainly for creating holes for centres when turning, and for starting drills in the lathe. Hadn't thought of using spotting drills in the lathe for starting drills, although I do use them on the mills.
Gray: My drill chuck isn't keyless, athough that's not to say I haven't been damaging the ends. I'll have to have a look nexy time I use one.
The bits of the centre drill I broke last night came out easily with a pair of small pliers. The one I broke a few weeks ago was removed by plunging in with a centre cutting carbide endmill in the drill chuck. Goes through HSS with no bother. Fortunately I needed a bigger hole in the part anyway.
Spot drills are usually carbide, so much stiffer than HSS. They're also used to provide a chamfer on the hole as well as providing a start for a twist drill; two for the price of one.
I'm pretty sure my tailstock is on centre; the lathe turns parallel to within a few tenths when using tailstock support.
I've extracted the centre drills from the bin, and transferred them to the 'Broken HSS' bin instead.
I think that part of the problem is the 'quality' of the no name centre drills. They might be hard but they seem to be brittle. I've got boxes of small centre drills for use on the repetition lathe, and I've never had a problem with them. But they're Dormer, and also all left handed.
Basically I'll have to be more careful in the future. 
Regards,
Andrew