Audobon was never at the Audobon house, but it is a good visit.
One of my favorite day trips was the ride to Fort Jefferson, but that might not be fun for a 4 year old.
Re: Audubon - but you can see (and buy) a $5000 Audubon original there (in case you were looking to take home a KW souvenir of the trip - haha)
Ft Jeff is a great trip, but it consumes a large part of the day - also not sure how grandma would do on that excursion. So you might try Ft Zach right there in KW - you can get a little bit of a fort experience (no doubt Ft Jeff is the much better fort, but you don't have 4+ hrs of boat travel to visit Ft Zach and its a lot cheaper), hit the beach right there next to Ft Zach and go for the tri-fecta by visiting the Eco Discovery Center there at the Truman Waterfront.
There are beaches in KW, but they are not anything to write home about. Smathers beach is nearby Windward Pointe - its manmade and is a sand beach, but also the most popular one for college spring breakers (so it may be more crowded than normal the time of your visit.) The majority of people say that the best beach in KW is the one at the state park there next to Ft Zach - its a little rocky so water shoes are a good bet.
The Eco Discovery Center is one of KW best kept secrets, it isn't a large facility but has a variety of interesting and neat displays - expect it would be enjoyed by all ages. (There's no admission, but they do accept donations).
Beaglemoms suggestion on Latitudes is a good call - I'd vote for lunch thou in lieu of dinner. While on the subject of dinner, Sarabeth's on Simonton is also great for a nice dinner. For a little more "old time KW" meal, hit HogFish Bar & Grille on Stock Island - staying at WP, its easier/quicker to go Hogfish than any of the restaurants in Old Town KW. Stock Island is not scenic, but the Hogfish taco is well worth it! Kelly's Caribbean and Grunt's are both nice settings for an evening meal (are across the street from one another on Caroline St). Also on Caroline (about 4/5 blocks from Kelly's/Grunts) is B.O.'s Fish Wagon - doesn't look like much, but its a great fish sandwich and a piece of Old KW. Another Caroline eatery is Pepe's - more popular for breakfast, but they're open all day. On Petronia St is Blue Heaven - great for breakfast or dinner (expect BH to get the nod by most to be a "KW quintessential must do"). Also have Santiago's Bodega and La Creperie there on Petronia.
The Sunset Celebration at Mallory Square is must for 1st visitors - you can catch the Cat Man as well as a variety of other street performers and artists all while taking in the popular KW sunset.
The Custom House Museum (you can't miss the big red brick building next to the Westin and across the street from Mel Fishers) is the home of KW Art & Historical Society - has a permanent Hemingway and Flagler exhibit as well as some interesting traveling exhibits.
While not really an official tour site, a visit to Casa Marina (now a Waldorf Astoria Resort) is neat little place for a short walk thru - beautiful property and provides a great flavor of what it was like in Flagler's heyday.
If grandma is going to stay at WP with the 4 yo one evening and you're going to spend some time enjoying KW night time entertainment, Smokin Tuna has a decent menu and the music venue is wonderful - try and catch Caffeine Carl or Rusty Lemmon or Joel Rush when they're playing. La Trattoria for dinner and Virgilio's for music and drinks is always good (LT in the front and Virgilio's is in the back). There's a lot of great live (and free) music in KW- the best way to find who is playing where is the website 6toejam.com. FYI Regardless of the time of day, if you're on Duval St and grandma wants to check out the view from the rooftop of the Bull and Whistle (ie: The Garden of Eden) - don't go!! lol
Expect you'll like WP, the units are very nice and you can easily spend a day enjoying the pool and property (the balconies are large and all have a nice view out over the water). If you were traveling with another couple and spending a lot of time on Duval St, you'd probably dislike the commute btwn WP and Duval (while the island is only 4 miles, its surprising how unenjoyable the commute is).
Ultimately, your biggest problem will be too many options for food, activities, site seeing & entertainment and not enough days.