Currently, your HTML shows:
<div class="quote" id="quoteDisplay">
<!-- Quotes will display here -->
<button class="getNewQuote" onclick="newQuote()">New Quote</button>
Your newQuote function has the following:
document.getElementById('quoteDisplay').innerHTML = quote[randomNum];
When you assign something using .innerHTML, all existing HTML inside of what you target is replaced with what you assign it. Your button is inside the quoteDisplay div, so it gets replace with the new quote. The solution is very simple, so I will leave it to you to figure out what to do.
P.S. You have an unnecessary in your HTML which you should remove.