Two things before we get started:
1. This is important, if you take anything away from this post it should be this principle: Don't Spend What You Don't Have. It is a good business practice, and not how you want to start off your business. Yes, sometimes you have to spend money to make money, but if you don't have it period, don't spend it.
2. I almost always recommend making do with what you have to start off. Sometimes it can be better to start with more basic equipment and learn how to rock it before you upgrade. You'll be a better photographer in the long run.
With that, here are 6 tips on purchasing affordable equipment, and where I would recommend buying it.

1. Sell Things You Don't Use
Lenses hold their value pretty well, but they are definitely cheaper used vs. brand new, especially much older lenses. I have bought all of my lenses used, and have had almost zero issues with them (one time I had to fix something on my 50mm, but it wasn't because I bought it used). Camera bodies lose their value relatively quickly compared to lenses, but this is a GREAT reason to buy them used. You can often get a used camera in "like new" or "gently used" condition that is a lot less expensive than brand new.
3. Buy Prime
New camera models come out frequently. You can buy a used camera body that isn't very old, that has a low shutter count (THIS is important. I should aim for under 40k if you can help it) and deliver great photos with it for years. Yes, updates are nice, but most people will never know you were shooting with last years model. Lenses come out less frequently, but there are often multiple versions of each type of lens. The older models are usually significantly cheaper, and they usually do a pretty adequate job. Just make sure that your camera model is compatible with the certain lens model (usually you can just google it).

5. Buy From Other Photographers or People Who Bought Equipment and Didn't Use It
Most of the time buying from other photographers is great, because the equipment has probably been well taken care of. Common reasons for photographers to sell equipment is because of upgrades or switching over to a different system (I got my Nikon full frame several years ago from a photographer who was switching to Canon).
Also, people buy cameras all the time and then don't understand them, and so eventually just sell them trying to recoup a little of their money. I've found some great deals on lenses this way.
Here are a few articles already on TheYoungTog as well to get you started:
Why You Should Start With a 50mm Lens
Invest in Lenses Before Cameras
The Basic Difference Between Lenses
a) They have some of the best prices on the internet.
b) They usually have tons of reviews, which are really helpful to read when trying to make a purchase.
c) They have excellent customer service which gives me a peace of mind when buying used equipment. If something is wrong, they will take care of it.
Amazon is the affiliate linking program I use to fund this site because I think they are the absolute best. If you do buy from there, pretty please go through that link? :)
2. Craigslist
I have actually found some great deals on craigslist on photography equipment. Though craigslist can get a bad rap sometimes, usually the sellers are just normal people trying to make a little money off their stuff. Often I have found that people are willing to negotiate with me (although be respectful when/if you do this) which is a plus.  And this gives you the opportunity to meet up in person and try the equipment out before you purchase it. On the downside, if anything goes wrong, there is no guarantee that you will get your money back. But personally I have only had good experiences. Always meet in a public place and bring others with you when meeting someone from craigslist. And if you are a minor always get permission from your parents first.
3. Ebay
I have found that Ebay is a great place to sell camera equipment. Due to the auction system sometimes people will overbid what things are worth (and sometimes no one bids, so it depends...) and you can get a lot more of your cash back that way. The biggest key I have found to selling on ebay is having good photos (it helps to be a photographer ;) and detailed descriptions. I've found great deals on equipment as well. A few things I have found useful to buying in ebay is: 1) Being patient 2) Watching auctions closely and figuring out what is a truly good deal, and only bidding on those. 3) Not bidding until the end of the auction. While ebay doesn't have the stellar service of Amazon, they do have a customer protection policy if you don't end up getting what you paid for, or it damaged.