You most certainly do need to worry about this.
I have the same type of push on/off tap on the Austin Sevens' under-bonnet tanks, that if they leak (mine do), can deliver very accurately a drip directly onto the hot exhaust. Then it's whoosh! No car and perhaps no me.
Don't faff about with wine bottle corks, or Teflon inserts (I have and it doesn't work) or with trying to over-tighten the joint with Mole Grips. Get new seals or, better, a replacement tap. These are readily available for the Seven and I expect they are from the specialist suppliers for whatever it is you have.
Good Luck
Martyn