Also backing off the drill edges to give them a shallower, or nil, rake will reduce the chance of the brass grabbing and trying to climb the flutes.
Depending on the size you need, a small-diameter step-drill may be better than a twist-drill, though might not be available down to the diameter you need. (These are made for drilling sheet material.)
Drill into a suitable backing plate: e.g. some scrap plywood, MDF, off-cut of laminated chipboard.