The whine/hum sounds like the starter is turning, but not engaged to the flywheel (if it was, the engine would start). The solenoid connects power to the starter, but also pushes the gear into the flywheel.
Not sure why the starter didn't stop when you removed the key, that should have removed power from the solenoid. The problem seems to lie in the solenoid. perhaps it is sticking or not assembled correctly.
In any case I would attempt to return the starter. Your mechanic should be able to confirm that the solenoid is receiving power through the small wire.
If you are near Boston, I know a great rebuilder in Waltham, otherwise I have gotten rebuilt Boschs from FCP.
--
84 242Ti IPD bars&springs, 89 745 16v M46 IPD bars, 89 744 16v M46 IPD bars, 90 745 AW70, 91 245SE AW70 IPD bars, 93 245 CLassic M47