I have sheets of aluminium 1050 and it is officially specified as soft, with poor machinability (see: http://www.aalco.co.uk/datasheets/?gId=1). The 1050A is grade 'O' which is the softest, and I also have grade 'H' which is tempered. You do need to be quite careful when specifying which grade you need when you buy it. Grade O is easy to bend and shape, while grade H14 has a bit more stiffness. 1050 is lovely stuff, but only for forming, bending, etc.
At the risk of offering a counsel of perfection, I would punch holes in sheet up to about 2mm and would not normally attempt to drill it. I consider 3mm and over to be plate. The Q-Cut cutters work ok on the thin sheet, but they need to be sharp. They do, however, produce a nice curve into the hole, on one side of the sheet. At a pinch, you can use woodworking techniques, and I have seen, and tried, some simple techniques like using a woodworking gouge or chisel to punch a hole, then using a shaped fibre or plastic punch to finish the edges with a flare. It may seem like butchery, but produces lovely finished work. I have a Metalman 'corking tool' which is a chisel-style punch with various heads for that kind of job, and its a handy tool.
Bosch produce sheet metal hole saws with very fine teeth, and those may work on 1050, but they are expensive.
I also use some of the 6000 grade bars, plate and flats (6082, 6060, etc) and cast tooling plate. 6032 was formerly called HE30 and it machines quite well. The cast tooling plate is a joy. I would not use a standard milling cutter, though. In my opinion, the only cutter you should use is one produced with a geometry specifically to suit aluminium — carbide with a single tooth and very aggressive angles. Accupro (sold by MSC Industrial) work very well indeed, but are very expensive. The finish is like chrome, even when machining dry. Use a high spindle speed and feed rate. Lathe tools are the same – you really must use a polished carbide, uncoated tipped tool for good results.
Its a lovely metal to work with, though.
Marcus