Note that the actual radios can pretty much be arbitrarily small – in the end, your phone is a fully fledged radio too, just for other bands and other modes than amateur radio uses! (And technically, it does things that are way more impressive than any amateur radio you can buy does.)
Of course, once you want a radio that has very much output power (some amateur radios do 100 W or more of RF power – to me as a communications engineer, that is crazy much power.), that power also unfortunately produces heat, and that heat needs to go somewhere, so these higher-power radios needs some heatsinks, fans and whatnot to keep them cool.
But technically, your radio can really without problem fit in your hand.
The thing that can't technically be made arbitrarily small is the antenna. An effective antenna is always in length somewhere in the order of magnitude of a quarter of a wavelength – you can build smaller ones or larger ones, but the smaller, the more trade-offs the antenna has to do.
That's why people in amateur radio also don't usually say "the 14 MHz band", they say "the 20m band", even if you put in the frequency in your radio in Hz, not in m. 20 m is roughly the wavelength for that band. So, if you see someone with an antenna in their yard that's say 4 to 20 m in size, chances are they are trying to have a very good antenna for the 20m band. Of course, again, a smaller antenna works, but it "costs" you received signal quality, and that means you can't successfully receive transmissions from further away or with lower power.
Many amateur radio users use very low-frequency bands, and through the speed of light, the frequency and the wavelength are linked ($\lambda = \frac cf$); the lower the frequency, the larger the wavelength. But, really, the 2m band (144 MHz and around) takes you very far – and an antenna for that isn't necessarily cumbersome.