Yep, used is the best way. There is such a choice of used gear these days that you will always find something decent within your budget. I recently got a full set (3-SW) set of Callaway X-14 Pro Series irons for €176 (about $225) off Ebay. Stick a new set of grips on and they play as well as anything you can buy today. And for about €1200 less than they originally cost.
Just realised you were really looking for specific advice on what to get, not where to get it. Go for something aimed at high handicappers if you're just starting out, something forgiving with a decent cavity back. Run away from any set with the word blades in it, great for single figure handicap players, a nightmare for a beginner. Have a look online, there are tons of sites offering help and advice on what to buy. One problem with asking other golfers is 10 people asked will probably give you at least 8 different answers, it's very much a question of personal preference. If at all possible try the clubs before buying. If playing with the actual ones for sale is not possible maybe you know someone who has the same clubs and will let you have a go. The main thing is to find something you feel comfortable with. I agree with YJ about balls, no point getting new and expensive balls yet, decent quality lake balls will do fine.
And have fun, it's a great way to spend a couple or more hours.