I am assuming two things in my answer and both may be wrong assumptions:
- Your shoe-string budget rig is a transceiver, probably QRP, maybe even single band.
- Rig is CW only (typical of small shoe-string budget rigs).
Thus, do you really need a side-tone oscillator so that you can listen to your own sending (assuming CW).
In my opinion, the answer is yes. I learned CW when I was 9 years old (or, 58 years ago) and today, my main operating mode that I use every day, is CW. My copy speed is a comfortable 20 to 24 wpm error free but I can do contests up to 35 wpm and copy call signs (usually) at 38 to 40 wpm. But, one thing I can't do is send without using a side tone so that I can hear what I send. I assume you are just beginning at CW so therefore, I am pretty sure you can't effectively send without hearing what you send.
You ask if you can use a buzzer or some external oscillator. Answer is yes you can. Now, whether there would be interference in driving two separate circuits from a single switch (that is, using your single telegraph key with the transmission as well as separate side tone oscillator depends on the circuits themselves. A high-impedance interface to the side-tone oscillator should be safe (~10 times higher than the impedance of the keying circuit of the transmitter unless that impedance is also high).
You will be missing a nice feature of the built-in side tone oscillator and that is the ability to zero-beat a signal. Actually, there are people who can do this accurately without a side-tone generator but I can't (though, haven't tried in years). But, if you are a beginner at CW, then maybe zero beating a signal is not the most important feature. If you were to participate in CW traffic nets (as I do) then the net control would often require zero beat (or, ask for it). They will send out QNZ which means "Zero Beat your frequency with mine".
If you fail to zero beat a signal it is possible that you may not be heard if you send to him (or, her). The reason is that he may have squeezed down his filters to a point that you have to be very close to his frequency in order to be heard. For example, when I am on the hunt for CW stations, I keep my filter fairly wide at 1 KHz. But, once the QSO starts I narrow down my filter to anywhere between 200 and 400 Hz. If you are more than 400 Hz away in that situation then I probably would not hear you. Now, I use a K3 that has nice sharp roofing filters: 200 Hz, 400 Hz, 1000 Hz, 1800 Hz, and 2700 Hz. Note that my rig does not do AM since I took out my 6000 Hz AM filter.
As I said, my rig is an Elecraft K3 and it has a nifty feature. It can do a zero beat on the incoming signal by the mere push of a button (called the Spot button). I also have an LED display that shows how far off (low or high side) from center frequency that the signal is which is kind of nice. And, I have the P3 pan adapter that allows me to sort of zero beat visually on the screen. Nice rig but often never a first rig.