Repeaters need to receive and transmit at the same time. The offset needs to be large enough to make it possible for the filters to separate the transmit and receive frequencies.
In my opinion 600 kHz is a very narrow spacing, it results in filters that are enormous, usually six cavities each 150 x 500 mm. Even so, they have several dB of loss in both bands.
5 MHz is much more reasonable, the filters can be made much smaller by helical loading of the coaxial elements inside. This lowers the Q somewhat, but with the larger spacing, it's much easier. A filter for a 5 MHz spacing might be 1/2 the length and diameter, and probably with less loss.
Reducing the offset below 600 kHz would be very difficult, the filters would be unstable over temperature changes, and the losses would be enormous.
Now the reason - I suspect - 600 kHz was chosen is because the amateur band is only 2 MHz wide in many countries. 600 allows some repeaters and some simplex traffic, space, etc. Possibly the lower parts of the band (144.0 - 144.5) were already allocated for CW, SSB and so on, so when FM came along, the repeaters needed to squeeze into the top 1.2 MHz.