I may be wrong but I think the larger health problem is that these two metals in non-enriched form are very poisonous!
Any amount of p36 about – we carry a significant amount of a compound of it, ourselves every day. There is a lot of it, happily married to Carbon and Oxygen, in the landscape too, with some ending up in our kettles or miniature steam-engine boilers!
Quite a number of those metals are of increasing importance now of course, with the drive to make life as electrical and electronic as possible. Many are called “rare earth” but are not actually ever so rare themselves. The difficulty for us is that their ores are scattered thinly, and some are tied up in complex silicates with other metals.