Alan,
I started off many years ago with a similar ethos, and bought one of the mini combi machines, thinking that would satisfy my desires, I very soon realised I was wrong, that sold (now regretted btw), I am now on my fourth lathe, a Warco GH1330 which satisfies the majority of my needs although I still hanker after a smaller machine for some jobs.
Similar story with Milling machines, one piece of advice I would give in this respect is get the largest you can accomodate/afford. I now have a Ajax AJT4, picked up cheap from a workshop clearance and never regretted, it has more 'meat' than a similar Bridgeport turret mill and plenty of capacity (54" x 11" table and 40 Int tooling)
As Neil says, think about the biggest project you imagine you will tackle and start sizing your machines from there.
If you tell us where you are then there may be people or clubs local to you where you can get a taste of various machines.
Graeme