Madrid brings me back really good memories, I went there the first time with my husband and sisters, but, I have been there a couple of times and I found it really cahotic -sorry madrid lovers-.
Even though I really love the museums they have, and the Parque del Retiro area is really beautiful; plus do not forget the theatre, there are some great productions now, like The Lion King. So, this are in my opinion, the highlights of the city:
If you go there more than 2 days, go to Toledo, is a lovely city 40 minutes away from Madrid by train from Atocha and is not expensive at all.
There are a lot of lovely places to eat, El Columpio is one of my favourites, for breakfast or lunch.