Yes it can be warm in Dec-Jan but it can also be very cold. It's chancy. We have a fireplace that burns wood. The only sell the wood at that time of the year in small bundles at Publix. We usually have about three weeks of cold weather with no rain. We average about three days per year when the temperature hits 32 degrees. Those days are usually in Dec or Jan.
The weeks with the fewest visitors here are usually the first two weeks of the year. The best weather here IMO is November and February-March. I guess it all depends upon your definition of the "dead of winter". We have BBQed on the beach on Christmas Day some years and stayed indoors due to the cold in other years. (Our homes really don't have what you would call a furnace) It's just plain luck if you hit beach weather at that time of year.