I have had the same sort of problem, but I know the switch is shot in mine. it's a poor design and really flimsy so eventually the plastic that the switch pivots on seems to wear and voila- broken switch.

All my other switches work so I have been slow to replace it, however yesterday was warm so I thought I would use a pen to slide the switch mech in order to allow the window to roll down. It did and then wouldn't go back up- infact no windows works, nor did the mirrors or sunroof. So I immed. figured I blew a fuse. Tried to track it down, but none are listed as power window switch-so I checked all that i could then appeared a-ok. Went back to the switch and started fiddling with it. Bear in mind my actual toggle is gone so i can see the orange slide that moves forward and back when the switch is activated. I noticed the light (green power) was out. Began working the slide up and down- at one point I got a loud-grrrrr sound like the motor was jaming. Eventually I got the light to stay on w/o the grrrr sound coming and the window rolled up and everything else returned to normal. So long story short- either a) swap that switch with a known good one (i.e., a friends) or b) buy a replacement out right ($110 FCP Groton). It sounds like you have the same problem as I.

Mike