Somewhere warm to travel over New Years?

My husband and I haven't been able to get our time off to line up so that we can take our honeymoon, which means we are either travelling over Christmas and/or New Years, or waiting until next summer and going for our 1st anniversary, so I'm looking into both options.

It has to be somewhere warm (ideally warm enough to swim, but as long as it's warmer than Canada will be I'm happy). We want to do a mix of relaxing and exploring, taking in some culture, food, etc. as well (a resort is fine if we can go off resort some days).

Budget is a little flexible – Ideally around $4-5k, but could go a little higher if it's something really awesome and worth it, or less and roll the remainder into savings for our next trip in a year or two.