It's quite difficult to get simpler than an SDR design such as the Softrock Lite 20m. It's little more than a quadrature mixer based on standard 74 series logic and an analog switch, an op-amp, a crystal oscillator, and a really simple input filter. The kit, including the PCB and all the components (but not a case) can be had for $30.
Of course, it achieves this simplicity by putting all the complicated parts in the computer, to which it attaches. However, this has some distinct advantages, like extreme flexibility, and if learning is your objective, SDR provides a great platform to experiment with modulation techniques. You get enough of the radio fundamentals in hardware with the kit, you can work out the higher-level concepts in software, then come back and implement an SSB receiver in hardware later, if you want. Performance can be really great because it's easy to build good filters in software. And, you can watch a big slice of the band all at once, which is really cool. And it's cheap, and easy to build.