Much depends where you are. Some areas are replete with local engineering stockists, others have none.
Not all locals suppliers are interested in selling small quantities of metal and they may have contracted out their scrap disposal arrangements – so no off-cuts. Some people are lucky enough to know a friendly foreman who let's them root through company scrap bins. Where I worked it was a sacking offence to give away anything and it was also a sacking offence to give permission to another employee to give stuff away. Some companies just want to get rid of scrap as fast as they can. It helps to be in the trade to know which is which.
Scrapyards may be a cheap-ish source provided they'll let you through the gate. My local scrappy used to welcome passing trade: now there is razor wire, an angry dog, and a big "No Visitors" sign.
Once inside buying from a scrapyard is a tricky business unless you know what you're doing. The trouble is you don't know what scrap is made of: as beginner most of my bad experiences were due to using odd bits of metal all of which turned out to be unsuitable. You may be lucky: some people on the forum an excellent relationship with their scrapyard.
Buying new metal in small quantities isn't cheap. I'm lucky enough to have a local supplier, but his range is limited. When he fails the internet is my friend: in addition to ebay and amazon etc there are several metal and plastic suppliers who will happily sell you stuff. Again not cheap, but you usually get what you ask for. Try searching for MetalSupermarket and similar. There are suppliers catering for Model Engineers such as Noggins End and College Engineering. Horologists have sources too.
Old printers and scanners contain useful metal rods and other bits. Anything that breaks in my house gets taken apart to see if there's anything useful in it. Don't think your lathe is useless if it won't cut the stuff though: much of what's in an appliance is far from being general purpose.
The problem gets easier as you build up a scrap box and know what to look for. When making anything I try to over buy metal in order to build up a stock. It hurts at first, but money is saved thereafter.
Dave