Phil, that flange looks just ripe for a couple of holes, or slots, to match the pitch of the Tee slots on the Mill table.
I had to extend the slots on my HV6 and use smaller studs that I thought (M8 instead of M10 ) for clamping. But a M8 stud and nut are quite capable of exerting enough force to crack a table, so hand pressure on an ordinary spanner will provide sufficient clamping.
I Asuming that your table has a 2 MT bore, you can hold a 3 or 4 jaw chuck on it with one of the 2MT Myford arbors thatv are available.
Mine is tapped M6 and is supplied with a top hat washer to allow the arbor to be tightened into the taper.
Breaking the taper is simple, with the table face vertical. Slacken the capscrew and tap itn with a mallet usually suffices.
With the table horizontal, you can't access the screw, so don't use it, just tap the arbor home firmly. But removing it, without taking the RT off the mill table requires a puller of the crude sort that I mentioned in MEW a while ago. It was just a bit of angle iron with a suitable sized slot and two M8 nuts welded on, and the setscrews drilled for brass pads, to prevent damage to the table.
AS you say, the direction of cut has to be such that the chuck tightens onto the arbor, not the other way round. Sometimes this means drilling a hole so that the end mill or slop drill has a starting point with in the metal.
HTH
Howard