Thanks for all the input. After tea, I went out to the shed .. and got more confused.
Firstly, The Spindle is R8 and it's the only R8 machine I've got so I can't easily do comparisons.
I have checked the runout inside the throat of the spindle – this with a standard 0.1mm indicator and it looks to be of the order of 4 microns. No photo for this, I'm afraid.
By finger feel there is some faint scratch or scuff in the throat, but impossible to actually see it.
I tried to get a good look inside with an endoscope attached to a laptop. I could really see anything – the wrong tool for the job, I think, however the top of the spindle looked like it had rust spots. On the sides as well as the top. Not sure if that would have an effect.
I then took a series of measurements at 100 mm down from the throat of the spindle.
The first was with a 20mm silver steel bar which I hope is straight in a 20mm collet. This shows the general arrangement

I rotated the spindle by hand, mainly but also under power a couple of times. The results were much the same.
The results here were encouraging. Apparently around 50 microns.
Min

Max

I was greatly encouraged!
Next I tried a piece of 6mm rod from a scanner – which is supposed to be generally accurate. This was terrible, about 0.3mm of wobble (as I call it).
Let's try the drill chuck again. I have two pieces of tool steel bar, one 4mm and one 6mm. The 4mm went in and i set up to measure
Min 25 microns

Max 520 microns
That's half a millimeter
I also measured the run out on the drill chuck 
To

That's about 0.03 mm which is a far cry from what's going on just under the chuck!
I then noticed that you could rattle the bar in the chuck. Not a lot, but definitely a movement. It clicked when it moved.
I tried the same with my 6mm tool steel bit and the results were about the same.
In the interests of repeatability, I then put the 20mm bar back in with the same 20mm collet.
This give quite different results.

and

about 300 microns (0.3mm) wobble.
I also noticed at this point that if I turned the spindle backwards it didn't run smoothly, instead it seemed to twitch every 10 degrees. This did not seem to be good. Nice forward, nasty back.
Finally, I took another 20mm collet (I happen to have a small selection of these) and put the bar in for that with similar results to the last one (that is, bad).
So the first take away is that the drill chuck is bad. It's not holding the drill straight. WIth a 0.03 run out on the chuck body and 0.3 on the drill rod, there's a problem with the chuck. Not sure if I should dissasemble and clean (there's no obvious swarf in there) or try and grind it smooth again. Or buy another chuck.
ON the bars and collets, I'm lost. There's something wrong going on here, but I can't work out what. There seems to be some problem – getting such different results in the same session with the same hardware is weird.
One think of note. I initially had trouble with collets not fitting in which turned out to be fixed by setting back the set screw which locates the collet, However even now some tools (collets) are harder to tighten up – in the last centimetre or so the drawbar gets stiff – but only at some angles. I don't quite see how this could related, but thought it worth mentioning.
I don't know what to try next, so any help – as always -will be much appreciated.
Iain