If I'm reading this right....
The reason you fit the starter button is stop you breaking keys when fighting the strong spring in the switch when you turn the key to start the engine. You only need to click the key over to "ignition on" and then use the button to start the engine. If you're saying that the key snaps back it sounds as if you're turning the key past the first position and trying to start "normally" whilst pushing the start button. Depending where the button is you might need both hands for this.