Collets vs drill-chuck are mostly a time trade-off on my MT mill, though collets are also slightly more accurate.
Drill-chuck wins when many holes of different sizes can be drilled in a sequence because drill chucks can load a wide range of drill sizes in any order. Keyless chucks are best for this – keyed chucks are much slower.
Changing from a collet chuck to a drill chuck just to drill a few holes is a waste of time. With a drawbar system It's quicker to change collets than the whole chuck. (MT is not the fastest tool-holding system available!)
I prefer collets because they can both drill and mill, allowing a mix of operations to be done without changing the chuck. As it's unwise to mill with a drill chuck, they're unsuited to my normal mixed work-flow – jobs where I typically drill several holes of the same size (one collet), and mill with a relatively few cutter diameters. Every so often a job calls for many different drill diameters and that's the only time it's worth me changing to a drill chuck. But my once in a blue moon job might be the sort of work others do most of the time, in which case their preferred chuck is the other way round to mine.
Bet there are people who never fit a drill chuck to their mill, and others who never use collets.
Dave