I'm impressed by anyone who works a mill hard enough to polish the vice with an edge-finder!
I doubt the smoothness is the cause though. Assuming the finder is one of this type, photo filched from Arc Euro:
The bottom half of the finder is free to float relative to the top except for a light spring that keeps the two halves from falling apart. When the finder is spun by the mill, the lower part is first nudged into alignment with the spindle's axis; the balance is both accurate and delicate. When the lower surface of the finder touches an edge, it flies off the axis. How far it goes is restrained by the spring.
As the spring is unlikely to change much, I suspect dirt has got inside and is restricting it's travel. On my finder the lower part can be pulled down far enough to allow in a squirt of WM40 or similar, which should dislodge any muck inside.
Dave