Can an amplified antenna be connected to a typical boombox radio with a telescoping antenna without damaging the radio?
You're incredibly unlikely to damage a robust boombox receiver by upgrading the antenna. Yes it will be "out of spec" for the Boombox and with enough gain you'll over power the circuit in the boombox but you're very unlikely to damage the circuit.
Cheapest way to boost that signal? take an alligator clip with a lot of wire and clip that baby to the end of the boombox antenna. If you're trying to tune into FM radio (85Mhz - 99.5Mhz) you're looking at about 10 total feet of antenna length for the best signal. Many antenna's are 1/4 Wavelength or similar fraction (1/6th, 1/8th etc) because it makes the product cheaper while keep adequate performance. You can improve by increasing the length and if you aren't satisfied you can also add a passive LNA (Low Noise Amplifier) which will generally form a regenerative circuit and increase the gain while automatically limiting noise. If you're hearing too much static you can also implement a filter to narrow what your Boombox is receiving.
Note that your boombox likely already has a proprietary LNA and filter, best to look for a schematic online of your device to develop a better plan.
If you're feeling ambitious or bored you're welcome to learn and experience some circuit prototyping. This link provides schematics for a regenerative radio circuit and as my friend once told me: "it's a VERY worthwhile project". Thanks, hope I've helped!
Not likely that it could be damaged by an amplified antenna, no.
Keep in mind that the boombox is designed to use that built-in antenna, so an amplified antenna would likely overdrive the receiver and the result would be a distorted signal.
Unless they have external antenna connectors your only option would be to clip it to the whip itself. Again, the boombox isn't designed for that.
So if you do try an amplified antenna and the results aren't great, then you might want to fall back to just clipping on some extra length of wire to the whip. But the antenna on the boombox is designed to be the "right" length, so even that may not help.