As I see it, case hardening followed by quenching is going to leave the steel through-hardened. Oil quench might be acceptable, but I’m not too expert on this. Case hardening is usually carried out on non-hardenable steel.
Unless there is a large surface contact area, between the cam follower and push rod, point contact may well lead to rapid wear if the case hardening fails at a highly loaded point contact.
Tempering at 550 Celsius seems rather high to me.
Andrew’s suggestion seems the best option.
Hardening is always prone to distortion. Surface finish should be after, heat treatment, not before.
How are the rods fitted? It looks like that end, at least should be tempered – more so than the follower end.