My reading is that the bored part of the hole is guaranteed to be concentric, and will then guide the D Bit/reamer to ensure the rest is as concentric as possible, in case the original N drill has wandered a bit.

I didn't build mine from scratch as I bought an almost completed one; the vendor suggested I looked to check out the spindle and bearings, but didn't say why.
It seems that he's not machined the bore and taper in situ using the grinding head bearings.
The taper was about 10 thou off centre, as was the through hole, so I re-machined it back to concentricity.
I did struggle to get the bore fully concentric all the way along, but it's OK for the part which supports/locates the wheel arbors.
The bore is now a tad oversize compared to the drawbar, and the supplies arbors, but I reversed the whole spindle head in the fixed steady and made sure there was a concentric section at the left hand end; this fits a small stepped section on the drawbar handle/knob.

Whilst the topslide was still offset for the internal taper, I also made a dozen or so blank arbors.
This ensured the tapers are the same as the hole, and the parallel sections now suite the new larger bore.

So that I could keep the thick end of the blank arbors in the chuck (a Griptru, which helps) I ran the lathe backwards and turned the arbors' tapers on the far side, using a boring bar from what I recall.

I was using a Myford clone (Warco 720) for this whole job, which obviously has the potential issue of the chuck unscrewing when running the lathe backwards.
My 6" 4 jaw has a separate backplate, where the hole through the backplate is smaller than the chuck's throat, so it's an easy matter to use a large washer, a nut, and some threaded rod through the Myford spindle as a drawbar to keep the chuck in place.

The 4" Griptru is slightly more difficult, but fortunately I have one with a backplate, rather than a threaded body.
I've adapted the backplate slightly; the outer of the register remains, but there is a recess in the centre, deep enough to take a washer between the front of the register and the back of the chuck. (there should always be a gap there anyway.
A have a washer where I've filled two flats on opposing sides, so it fits up the bore of the chuck/backplate on edge, but swivels and drops into the aforementioned recess.
I can then carry on as before with nut and all-thread as a drawbar.

Bill

Is it worth making a replica spindle nose (internal) on an MT2 shank so it will go concentric in the headstock while you're set up? Then could use this to machine the mounting end of arbors for grinding wheels.

Run it forwards but with the tool upside down at the back. Also good for internal screw cutting, the chips fall off the tool better and it's the same direction of feed as external so less confusing

Just a quick photo of the collet chuck for the Quorn, to take ¼" & 6mm mounted grinding points, or even carbide burrs.

I'd forgotten, It looks like I already had a photo in the album of the one on a long spindle/arbor too.
The lower right one is the Clarkson adaptor for their milling chuck(s)

Bill