It will be empty, but that's not necessarily a bad thing.
Check with Colonial Williamsburg to see what is open vs. closed. They shut down some of the restaurants and exhibits. You will have no crowds to deal with though. Go to the Williamsburg Lodge during the evening and have a drink or hot chocolate in front of the fire place.
Busch Gardens is closed. The college is on winter break so nothing to do there except maybe go to a basketball game.
Jamestown, Yorktown and pretty much everything else is open. Both will be fun with the run of the place.
Many restaurants offer good deals during this time of year.
FWIW, I'm going down there for the long MKL weekend so I'm not afraid....