Various organizations have issued guidelines on maximum permissible exposure. Below a certain power, the station can be assumed harmless. Above that power, some more careful design is prudent. Your station is mostly below that threshold, except possibly on the upper HF bands.
Thresholds below which a station does not need to be evaluated, according to the ARRL:
- 40 meters and lower frequencies: 500 watts
- 30 meters: 425 watts
- 20 meters: 225 watts
- 17 meters: 125 watts
- 15 meters: 100 watts
- 12 meters: 75 watts
- 10 meters, and all VHF: 50 watts
- 70 cm: 70 watts
- 33 cm: 150 watts
- 23 cm: 200 watts
- all higher frequencies: 250 watts
The lower thresholds around VHF are due to the increased energy absorption of human tissue at those frequencies. The cornea is especially at risk because it's small and has no blood flow to cool it.
As you say you intend to operate 100 watts on HF, and 50 watts on VHF, you are below these thresholds except on 12 and 10 meters. So if you want to be safe, reduce power a little bit on these bands and there should be no issue at all having the shack near the antenna.