Something like, "I'm really sorry I missed our date night. I know they mean a lot to you, especially since we've both been working a lot" is a much better thing to hear. He's upset that you're upset.