If not now, then when? You are only going to get older :).
Seriously, if you don't go for it now, not only will you always be thinking "what if I did...", but two years down the line, when you feel like you want to give it a go again, you will be two years older than you are now. And believe me, you will keep wanting to give it a go, some things just don't go away.
Programming is one of these things that no matter how old you are, if you get good at it, nothing can stop you. A lot of it is done remotely, so people won't even know how old you are unless you tell them. And even if you do tell them, your age and past experiences will work in your favour - you can always turn them into advantages. At least that's how I see it.
I'm not that much younger than you (35), I have 3 kids and the career of a housewife. I've got a psychology degree and I used to have a really good job before, but that all changed when the kids came. And now I'm fed up with being dependent on others, and while I love my family, it's time to do my own thing. A number will not stop me. And neither should it stop you.