Battery lifetime is closely related to temperature. The hotter they run the faster the battery ages with the associated reduction in capacity.
High current draw increases the internal heating and the continuous running associated with things like lawnmowers doesn't give the battery any breaks to let it cool down.
For any given current draw a larger capacity battery should run cooler than a smaller one. So all things being equal you'd expect a bigger battery to have a longer overall lifetime than a smaller one doing the same job. But if you exploit the extra battery capacity by making the larger battery work harder, drawing a disproportionately larger current, it will get even hotter inside leading to even shorter life.
It's a balancing act really as big enough is big enough. My guess is that if the battery takes longer to charge than it does to run down odds are you are asking too much of it.
It's noticeable that 3 Ah batteries run perceptibly hotter to touch than 5 Ah ones in my twin 18 volt Makita hedge cutter and strimmer. Once, on a very hot day, I managed to get one of the 3 Ah batteries so warm that the dual fast charger refused to charge it until I'd let it cool down for maybe 10 minutes. The other battery wasn't quite so warm to the touch and did accept charge. Charging almost immediately after the strimmer stopped might not have been the best idea in the world as not only did it take longer than usual but the charged battery was still quite warm. Memory says the second one was only a minute or three behind reaching full charge despite the 10 minute cool-down wait. Makita batteries have a temperature sensor inside to help monitor charging.
That little episode convinced me that sharing the two 3 Ah batteries from my drill and driver set with the hedge cutter and trimmer might not have been the best idea in the world. So I splashed out on two new 5 Ah ones for the garden tool department. Hafta say the extra life is nice. Two and a half charges to do all my hedges, one charge to do the strimming if I don't leave it too long between goes.
Anecdotal evidence so not conclusive but it may be relevant to note that Tesla makes serious efforts to keep their car battery packs cool.
Clive