At HAV, stay airside—limited amenities landside and slow processing favor staying near your gate.
A 3-6 hour layover to Havana is not recommended. Jose Marti Airport is 25km from Old Havana, the taxi takes 30-45 minutes each way in good traffic, and immigration can be slow. You would spend more time in transit than in the city. Stay at the airport unless you have a very smooth immigration track record in Cuba.
Six to ten hours makes a Havana trip worthwhile if you plan it carefully. Take an official Cubataxi from the airport ($25-30) directly to Old Havana. Walk Plaza de la Catedral and the streets nearby - the colonial architecture is extraordinary. Walk to Plaza Vieja (5 min) and climb to any rooftop bar for a mojito and city views. The Malecon waterfront promenade is 15 minutes walk from the plaza and essential. Head back to the airport 2 hours before your flight. Have CUP cash ready - USD and credit cards do not work.
With 10+ hours, you can absorb Havana properly. Hire a 1950s classic car for a city tour ($30-40/hour) - the drivers know every neighborhood and make the experience. See the art galleries around Obispo Street. Walk through the Vedado neighborhood for a more local feel: the Hotel Nacional rooftop bar has Mafia-era history and excellent daiquiris. The Museum of the Revolution (free with CUP) gives essential context. Eat at a paladar (private restaurant) - La Guarida in Centro Habana is the most famous (book ahead), but many excellent cheaper options exist. Leave 2.5 hours before your flight to clear airport security and formalities.