I think that tube would hook into a vapor reccovery system on a 140 (the charcoal cannister thingy which collects vapors when the engine is off, then feeds them to the engine when it is started). On a 120 the tanks are vented only through the little valve in the gas cap. So you could just block the extra hose and make sure your gas cap has a vent.

If your trunk continues to smell like gas look for small rust holes in the top of the tank. This is not terrribly uncommon on old Volvos. The top of the tank lives much of its life underneath a rubber mat which holds in the moisture. I've fixed minor rust holes like this by smearing a small sheet metal screw with gas proof sealer, then screwing it into the pinhole.