3 days minimum, 4 ideal. Worth the time.
I've been during different times, and Fall "foliage peak weeks" are by far the most beautiful, and temps are great. You can ask when that time occurs, or check it out online, but I think it's usually mid-October to early November. All of Colonial Wmsbg is spectacular, and the half-day trips in any direction are glorious (there are several outstanding ones).
One trip is to Shirley Plantation and then Berkeley Plantation---both are historic, both are really fun, both are within easy driving distance for a half-day leisurely outing, and the trees along the way are breathtaking. (Forget about any other historic homes along the way---those two are superior and plenty enough to do.)
See links below:
Minimize boxes by clicking little arrow to top right of big blue box, and also lose the bottom pics by clicking on double arrow on bottom right near word "Explore".....also can ZOOM in/out, or click & drag to enhance map picture.
Driving directions/locations
https://goo.gl/maps/XC9YL2EQTw92
Shirley Plantation
http://www.shirleyplantation.com/
Berkeley Plantation
http://www.berkeleyplantation.com/
Another GREAT trip is to Monticello in Charlottesville, Va, spectacular historic home of Thomas Jefferson, our most favorite half-day excursion from Wmsbg. Monticello is a very short 2-hr drive from Wmsbg, and you'll have ample time to tour the home and estate to keep it an easy half-day. Beautiful! Tavern at bottom of mountain near entrance is very nice for lunch afterward. Fall is awesome there.
Monticello
http://www.monticello.org/
Colonial Williamsburg is fantastic---lots to see and do. Spend as much time there as you like....definitely do the Courthouse thing.
Colonial Wmsbg
http://www.colonialwilliamsburg.com/
http://www.colonialwilliamsburg.com/do/revolutionary-city/tour-the-city/courthouse/
Jamestown is fun, but if you run short on time, or get "historied out", skip Jamestown and do the other things!
Jamestown
http://historicjamestowne.org/
The world-class Dewitt Wallace Museum in Colonial Wmsbg is also worth the couple hours of time.
http://www.history.org/history/museums/dewitt_gallery.cfm
Wonderful restaurants around Wmsbg and in Colonial Wmsbg....I have a list if needed.
Spring is great too----we spent a week in April twice, but nothing can compare to Fall in Virginia. Summer is awful, oppressively hot, AVOID (unless you have kids and want to do Water Country, but you won't have kids this time, you said). Winter can be difficult if temps plunge, if it snows, or the area experiences an ice storm. Power outages happen occasionally, not worth the risk, imho. Lots of stuff closed down during that season, but you will hear a lot about "Grand Illumination", which is a pretty cool thing to experience (but the Fall experience supercedes anything in Winter, imho).
http://www.colonialwilliamsburg.com/plan/calendar/hol-grand-illumination/
Good luck, a great destination.
.