I think only you can answer the question ultimately, Pailo.
It depends a bit on what size mill you have. If it's a full-size machine it might pretty much replace a pillar drill. Mine is not the smallest you can get but would still be in the mini-mill class and there are things that I can get under the drill that I couldn't fit under the mill, particularly when using a large (and long) drill bit. When those circumstances arise, however occasionally, the pillar drill is a necessity for me.
Also, I can get more raw torque from the drill than the mill which is useful if I want to make, say, a big hole with a holesaw to remove metal prior to boring. Or other "rough" jobs that are beyond the capacity of the mill or would risk stripping the its gears.
On top of that, the pillar drill doesn't really take up that much floor space anyway – or at least, if I got rid of it I can't think of much that would fit in that space. In fact, I have mine next to the bandsaw and wind the drill table down to support long lengths when sawing them.
If you do lose the pilllar drill, what will you do about the 2%?