The very first thing you need to tell (if you want an answer) is what is happening/not happening when the key is turned the first few times. "hit the starter" and "comes to life" won't cut it. Does the starter motor not run? or is the engine cranking but not starting until the 4th time?

If the starter motor is not engaging, then voltage will have to be checked at the "+50" yellow/green wire at the starter with the key in start position. Anything more than .75volts below battery voltage and you'd need to check the same circuit at the back of the ignition switch. Similar drop = defective ign switch, voltage good means wiring, connectors or neutral switch.

My own car did something similar every few months which I finally traced to a dirty 2-pin connector going to the safety switch in the console.