I’ve just found out that epicyclic axial gearboxes are available whose output rotation is in the same direction as their input rotation.
So if the smaller pulley still results in a wheel that is rotating too fast, you could fit a suitable epicyclic gearbox onto the motor shaft, and fix the drive pulley onto that. No layshafts, no brackets, no new motor, no VFD with all the wiring.
I think you would need one simple metal strap to stop the gearbox outer ring rotating.