Well, you do seem to admit pushing the tool and machine beyond recommended!
The tool setting needs to be spot-on, and the whole set-up and machine rigid. Some, including me, swear by using a rear tool-post but others would say that's not necessary. I think that may be so on a large lathe but one problem on a smaller machine is the cross-slide having sufficient engagement for rigidity when an over-long parting-tool is used in the normal tool-post.
If you were using an HSS blade I'd ask if you'd made the mistake I did for a long time, of grinding the tool on an angle to try to obviate a large pip on the parted-off side. It should be square across. You are using inserts though, so the tool shape would be all-correct.
Right tip for the material? Carbide inserts come in variety counts that leave Heinz standing, and the wrong tip type for the metal might affect things by demanding slightly more force than should be necessary.
Trying to part off in a single cut? I generally ease matters by cutting a wider groove in steps so some of the blade flank is in the open air, but beware of chips jamming in the gap. They shouldn't but still keep an eye on things.
Don't set the blade out from the holder all in one go, but in a few steps to minimise the amount of metal removed at full stretch.
Is the blade-holder correct for the blade and tool-post? The tool may be fully square to the lathe axis but a slight angle from the tip's vertical centre-line will upset its geometry, putting one corner very slightly high and the other very slightly low and possibly causing the tool flank to rub on the groove wall. That could well push the tool off the straight-and-narrow. I am not sure what else it may do, but it won't help matters anyway!
Use plenty of cutting-fluid. That often recommended for aluminium is paraffin or WD-40 – if nothing else they are are handy for removing the lubricating-oil from the slides. A light, neat cutting-oil or even thin lubricating-oil may be better as it lubricates the blade as well as the tip itself, and tends to tarry longer in the cut.
The dishing effect is very hard to avoid , and sometimes I resign myself to parting a bit over-length and facing off. It comes from the blade flexing, possibly due to that vertical-plane error or to axial square-ness being a tiny bit out; or from the cross-slide rotating a tiny amount by wear or mis-adjustment. I have probably encountered the latter when facing a large diameter. Both problems will be magnified by very long parting-tool overhangs.
Watch out for the whole tool-post creeping round. It can happen – as I have found – due to the cutting force being a long way from the tool-post pin, and highly off-set from it.
Hope this lot helps!