Speed isn't quite the right word, books all use 'amplitude'. Much the same thing though.
Amplitude is a measure of how high a pendulum rises, usually expressed as the angle swept by the rod. High amplitude gives high speed, not useful, and the amplitude is kept low.
At the top of each a swing, both sides, the bob comes briefly to a stop, speed zero. Then gravity accelerates the bob downwards, and it's travelling at top speed at Bottom Dead Centre. After passing BDC, the bob de-accelerates to speed zero as it climbs to the other side, then repeats. The pendulum oscillates, and the time taken to go from side to side depends almost entirely on the length of the rod.
Almost! Period also varies slightly with amplitude, enough to be a problem. A pendulum loses energy each time it swings, causing the amplitude to fall, altering the period. And to keep going, the clock has to replace the lost energy by impulsing the pendulum, which also causes the amplitude to vary.
For accuracy, it pays to have the impulse apply a constant amount of energy to the pendulum each time. Unfortunately, in a spring powered clock, the impulse weakens as the spring unwinds, causing noticeable drift. A fusee fixes this by regulating the power delivered by the spring; it keeps impulse power nearly constant between rewinds.
The impulse (hence amplitude and period) is also disturbed by winding, so better mechanical clocks feature a remontoire – an independent source of energy that keeps impulses constant during winding.
The purpose of the fusee and remontoire is to keep the pendulum swinging at the same amplitude (speed at BDC), no matter what the input energy source is up to. Then the clock keeps better time.
Dave