Modern soluble oils do not seem to suffer from bacterial growth and nasty niffs, at least in my experience. And I have coolant sitting in the sump for months at a time.
I use coolant on my CNC mill for two main purposes when cutting aluminium, one to wash away swarf to prevent re-cutting, and two to prevent swarf sticking to the cutter. If you have decent cutters they won't be affected by coolant or no coolant in terms of life.
The practical problem, identified by Nick, is that swarf, and coolant, will get everywhere. Careful guarding can reduce the issue, but I still seem to end up with coolant on the floor. On aluminium I can't see any reason not to be running the mill at full speed, so I'm not sure there's a solution in terms of preventing coolant spray. In terms of coolant I use Hysol XF, which is fine on aluminium.
To remove swarf you might consider air blast, no coolant to contend with, but there will be the noise of a compressor; not that anyone will hear it over the loud music. Selection of cutters intended for aluminium (generally uncoated, polished and a maximum of three flutes) can help with the sticking problem.
Unfortunately there is no simple answer to the problem.
Andrew