You have a dirty, gummed up starter. Exact same symptoms as my wife's 850 Turbo Wagon with about 100K on it. The Bendix is not advancing into the full engaged position when the starter is engaged (thus the delay before the current starts to flow to the starter). Then, after it starts, the superior rpm of the Volvo engine fails to spin the Bendix away from the flywheen, thus the grinding noise as the starter gears tear metal off the flywheel teeth. Not good.

Your options are buying a rebuilt starter from Volvo for $300 plus exchange or removing your starter, cleaning it, re-lubing it with white lubriplate grease or a similar starter grease and reassembling it and driving it for another 80-100K before doing it again. It is a geared, sealed uint and the graphite from the brushes gets in the grease.

Oh, by the way, get yourself a new set of brushes. I found mine half or more worn at 100K but being 300 miles from a Volvo garage left them in. The starter has functioned perfectly for two years since. There was no discernable wear on the starter gears, Bendix or bearings, so I saw no merit in installing the $300.00 plus rebuilt Volvo starter I had ordered in "just in case." Your mileage may vary on starter wear, so check it over well.