Audio Delay in Simon

I think that I have either fulfilled or am close to fulfilling most of the user stories for the Simon project but I have an issue with audio. For reference, here is my implementation.

I initially built and tested in Safari and found in issue with use story #5: I hear a sound that corresponds to each button both when the series of button presses plays, and when I personally press a button.

If I only press one button, I do hear the sound. However there is a significant delay. This sometimes causes button sounds to be skipped, especially when the same button is pressed twice in a row since there tends to be less time between clicks in this case. I also notice that this sometimes happens when the sequence is being played for the player to remember.

There is much less delay when I open the project in Chrome or Firefox. However, I can still double click the button to cause the same problem (button lights up twice, but only makes the sound once).