If I only have the budget for one HF antenna, should I chose a vertical or a horizontal antenna?
If you can have only one antenna and can chose between a vertical HF antenna or a relatively high horizontal HF antenna, go for the horizontal antenna.
In January 2008, L. B. Cebik, W4RNL (SK) published what was going to be the last entry of his seminal 10-10 News series. It turned out to be an interesting gain comparison of single element 28.4MHz antennas modelled over various ground types of which a summary is presented here (Table 1). His findings seem to fit well with what John Devoldere, ON4UN modelled and published for equivalent antennas on the lower HF bands.
Horizontal HF antennas: near-by ground gain
At a height of about λ/2, the nearby ground reflection of a horizontal HF antenna will start to be constructive at interesting take-off angles for long- distance ionospheric contacts. This will provide a net gain over the antenna in free-space.
Vertical HF antennas: near-by ground loss
This is not the case with vertical HF antennas. Nearby ground only contributes loss. This even more so when the ground forms part of the return path of the radiating structure. Even when far-away ground reflections may cause the directivity of a vertical HF antenna at low take-off angles to be much higher than that of a horizontal HF antenna, its net gain will still be lower at those angles. This makes the horizontal HF antenna a clear winner, at least for what transmission is concerned. Note that gain and directivity are not synonyms; gain takes into account losses, directivity does not.
Vertical HF antennas still remain useful
Vertical HF antennas do have their merit though. At the lower end of the HF spectrum, the λ/2 height requirement for horizontal antennas can become cumbersome (even though a horizontal phased array may weaken this requirement by allowing somewhat lower heights). A vertical HF antenna can get away with a height of only λ/4. Furthermore, the directivity of a vertical HF antenna can be effectively employed at the reception end to cancel out high-angle interference by near-by stations. This is why some stations use receive-only phased-arrays of verticals on the low bands.
Polarisation at HF
Polarisation is not really an issue at HF (apart from lightning and man-made noise considerations). This is so because the ionosphere is mainly an anisotropic medium, i.e. it messes up polarisation.
You're asking about a topic called polarization, which is really important in VHF and UHF. It's the reason why rotating your HT can improve or harm your reception, and it's also the reason why some 3D glasses and sunglasses work. But because of the way HF propagation works, it doesn't really matter. Any ionospheric propagation gets the polarization randomized anyway. Very short range (ground wave) HF works best with a vertical antenna, but it's very rare that you care more about short-range than long-range.
In terms of limited space and cost, it's probably easier to set up a horizontal antenna (or an inverted V like a G5RV or OCF) than a vertical simply because it's easier to string a wire through some trees than it is to secure a flagpole or tower against gravity and wind loads.
There are useful tidbits in all the answers so far. Here are a couple more. Not every ham has the good fortune to have conveniently spaced tall trees to hang horizontal antennas from. On 80 meters to function well, a dipole antenna would be about 130 feet long (a half wave). Distance above ground for optimal performance would be the same distance. Probably not practical for most hams. The situation improves the higher in frequency you go--i.e 40 meters where the half wave length is about 66 feet with the corresponding reduction in optimal height. I guess what I'm saying is that, on 80 put up a dipole and get it as high as you can. It will still work at lower heights--just not as well. A vertical on 80 is also probably also impractical (unless you use "tricks" which will limit it's performance) since the vertical radiator would need to be a quarter wavelength long (66 feet) and many, many ground radials which would also be about that long. Commercially produced multi-band verticals make rather fantastic claims as to their performance. They tend to be poor performers when compared to a full sized horizontal antenna. They're also outrageously expensive. You can homebrew a dipole for far less than $50.00.
It matters only for local RF interference purposes. Once the signal hits the ionosphere, the polarization is randomized, thus it really doesn't make a difference.
Polarization of RF Interference tends to be vertically polarized, but that's such a small effect that it really probably doesn't matter which way you do the antenna.
Very good answers posted about polarization. I must point out an important caveat about polarization - while not so important for distant skip contacts ... it IS a vital consideration for LOCAL contacts. Cost and convenience is a prime factor for the popularity of rooftop vertical HF antennae. If your local friends are using verticals - you need to also be on a vertical (polarization) otherwise they may not hear you clearly or at all. As noted in another answer, vertical does draw more interference in a typical cityscape, but it varies depending on your location and proximity to interference sources.
If local contacts are important, and if other locals are using verticals, then you should consider a vertical. If DX is more important, then other horizontal options present themselves (which may even work well for local contacts).
A different factor to consider is the construction of the particular vertical you intend to use. Some are simply a 50 ohm "dummy load" with a wavelength/fraction wire sticking out it. Some have grounding radials. Some have coils. Just classifying it a vertical antenna is not entirely accurate.
If I could have only one antenaa, any antenna, it be a 150' Big Bertha with stacked tribanders :)
Before a good answer can be made, we need some more information:
1) What bands are you interested in working? 2) Can you get a horizontal antenna at least λ/4 high and preferably λ/2 high? 3) Do you have an antenna tuner?
If I were flat broke and had high trees, I would make a fan dipole with one leg on 80m/75m, one leg 40m (gives you 15m for free as it's the 3rd harmonic) and one leg 10m. Depending on the situation, you could probably pull 20m out of either 40m leg or 10m with a close enough match to not need a tuner. Having one would tighten the match, but probably wouldn't make a noticable difference. Not counting the optional tuner, you would be hard pressed to spend $50 on this solution. You'd have resonant antennas on the primary bands and could work the world no problem.
If soldering 6 wires is a problem, and you had a tuner, a G5RV or clone would be an option. It's an all band solution, but is a compromise on all bands and a tuner is required for a good match. You can work a lot of countries with one, but it will set you back $75-$100 plus a tuner.
If I had no trees, or low trees or no good way to hang a dipole, I would go with a multiband vertical like the GAP Challenger DX. It uses stub matching and gives a no tuner between than 1.5:1 on all the HF bands plus 6m and 2m. It's a bit of a compromise on 80m and is worthless on 160m though. It uses 3 couterpoise wires instead of a large radial field and you can work the work 40m and up no problem (I have one). But it costs $350. There are cheaper ones, but they are more and more of a compromise especially the ones that use coils to make up for their physical size.
I have both a vertical and dipoles and I find the noise difference to be negligible, but that's certainly something that is going to vary by location. In a city or dense suburban area, RF trash from the neighbor's plasma TV might rule out a vertical (as a previous poster noted, most man-made RFI is vertically polarized).
Well, speaking from a budget point of view, you can cover 80-6 for $100 with a multiband OCF dipole, but any HF vertical is going to require a lot more budget than that - probably at least 5 times that much, and more if you need to bury radials.
Most everything posted here is not relevant in light of the enormous cost difference between a well-performing dipole and a well-performing vertical for HF.
However, one of the exceptions is 160 meters. If you are mostly interested on making longer distance QSOs (especially DX) contacts on 160, vertical polarization is best on that band. Period.
Check out this page. In a nutshell: What is effective on 160m are vertically-polarized antennas for transmitting, using separate antennas for receiving.
Per W8JI on qrz.com:
"Nearly all especially successful stations on 160 use a vertically polarized antenna of some type for transmitting. That's just a fact."
"The fact is....... an Inverted L with 20 or more radials at least 50 feet and hopefully 100 feet long will absolutely smoke any normal height loop antenna or dipole antenna at nearly any distance on 160 meters. The possible exception is between 20 and 200 miles."
"I have a full size 160 dipole at 300 feet, and it is never really much better than a 1/4 wave vertical at any distance in any direction. As a matter of fact, the dipole is 10-20 dB weaker than the vertical off the dipole ends. The dipole only beats the vertical broadside to the dipole, and then only rarely!! And this is with the dipole 300 feet above ground."
"As a matter of fact a low full wave loop has no gain, any horizontal wire has increased earth induced loss as it is made longer when close to earth."
Having said that about 160 meters...
On 80 through 10, both vertical and horizontal antennas can be good choices. You should pick whichever is convenient to erect and/or will accomplish your objective.