Strain it. Turn on every electrical thing you can think of in the car, then test the voltage across the terminals. Do your lights dim a lot with things like the heater turned on, power windows, etc? If you're still getting a reading in the high 13's-14, your alternator is just fine...for now anyway. If it drops to around 12 or 13, it's probably on its way out.

I've been trying to track down a similar noise with my mother's Honda that I've been driving for a month now until I go home for Christmas. It has an annoying whine in the morning that isn't the alternator, and it disappears entirely for the next day until the engine is cold again.