Your resort review rating methodology

[gravityrules]

How do you determine your 1 to 10 rating of a resort when submitting a review? To me the template implies using a more objective standard (10 is world class resort if I'm remembering correctly) but I see a lot of very numerically high reviews of resorts that are in no way world class.
I think many of the reviews ratings are more of a satisfaction rating.
We have stayed at some very good resorts that I ended up rating as an 8 or so, not because we weren't satisfied with it, but because it wasn't close to a 'world class' resort.
If we have reviews using both methods doesn't this compromise the ratings somewhat?
I also see there is a lot of compression in the ratings. Is there an overall distribution of ratings available?

 

[pedro47]

How do you determine your 1 to 10 rating of a resort when submitting a review? To me the template implies using a more objective standard (10 is world class resort if I'm remembering correctly) but I see a lot of very numerically high reviews of resorts that are in no way world class.
I think many of the reviews ratings are more of a satisfaction rating.
We have stayed at some very good resorts that I ended up rating as an 8 or so, not because we weren't satisfied with it, but because it wasn't close to a 'world class' resort.
If we have reviews using both methods doesn't this compromise the ratings somewhat?
I also see there is a lot of compression in the ratings. Is there an overall distribution of ratings available?

Please define a "world class resort and how many timeshare resorts would fall into that category as a "world class resort?"

 

[gravityrules]

That's a question for @TUGBrian
I expect the answer is 'not many'. How many hotels are considered 'world class'? Star rankings are used for hotels, there has been discussion here that TS top out at 4 stars (1 to 5 scale). Perhaps the few 4 star timeshares are the only ones where a '10' rating on the TUG scale is possible.
Another way to look at this, are you going to rate a Worldmark resort that you like as a '10', the same '10' rating you give the absolute nicest Hyatt or Marriott? Or does the Worldmark top out at something less than 10 just because it's a different resort class?

 

[dioxide45]

I seem to recall the rankings kind of having a more defined set of criteria in the past. I don't recall how long ago that was. I suspect everyone rates a resort differently. Some have a bad sales experience, it gets a low rating regardless of how great the resort was. Someone may rank an average resort with no onesite food options or activities with a very high rating.

 

[TUGBrian]

they do, right on the review page!

 

[TUGBrian]

I would also agree that based on some reviews ive read...many 8-10 ratings should likely be a full point lower...

 

[gravityrules]

I've only rated 1 resort as a 10, Grand Lodge on Peak 7. But my experience with top tier resorts is limited compared to others. The lowest rating I've given is a 5.5 and I felt bad about giving a rating that low although it could have been a half point less.

It seems these resort ratings are like children from Lake Wobegone, all are above average!

 

[TUGBrian]

also have to contend with owners of the resorts they are reviewing in many cases like or even LOVE their resort and that alone is worth an extra half point or more in a rating that otherwise would have been a bit lower if it were a random resort they were visiting vs one they own at.

 

[jp10558]

I was honestly thinking about this a few weeks (and months) ago. I somehow missed that rating info when making a rating, but mostly understood the same sort of thing except for 5 = would not go back. Honestly, I think 5 / average should not imply would not go back. The real world isn't lake wobegotten??? where everyone is above average.

That said, taking that sort of scale to mind, I have had slightly higher ratings because of that. It's also been hard to figure what "average" was in my first 5 or so timeshare stays. Now I feel way more comfortable being at about 10 stays at places from Vacation Village to HGVC to Diamond to Wyndham so quite a spread IMHO.

Ok, so my reviews probably should be edited down (and I felt like I was going lower than many for the resorts before). On that scale the HIVC Oak and Spruce Resort should certainly be a 6 at best, and I'd like to deduct a point for no laundry in the units or even the building, but I don't want to give it a 5 = would not go back because honestly, while the rooms were old and there was no laundry in the unit, I had a blast with the mini golf and arcade and ping-pong, and probably would have enjoyed more if it wasn't like 95 the week I was there. But it's rated 8.18 which ... I could see on some metrics and not others. Like, it's a pretty meh location IMHO, I'd give that like a 4 - the least interesting location I've been to in the US and matching Wynchnor Park in the UK for sort of nothing going on in the area. The units are pretty old, on par with Wynchnor Park or Smuggs or the like - I'd give that a 5. Wyndham and HGVC are certainly a step up IMO. When I get my video cut together you'll see the "interesting" sort of themeing in the rooms. OTOH, it had a lot of activites, a lot of on site stuff to do that was fun and more that would be fun for kids too.

IDK exactly how to score these varied axis. Ideally there would be multiple ratings that then were combined automatically into the top value based on some algorithm in TUG2. But that's a major overhaul, and IDK how interested Brian is in that level of granularity.

I'm sure a lot of people also have varied expectations - so if something meets those expectations or the value is right, the rating is higher. I will say, if I'm staying somewhere for $600 for a week in a 2BR, I'll be inclined to overlook more things than if I'm paying $2,000.

Then there's the other question - is the scale based on our current timeshare experience - i.e. HGVC Ocean Enclave was the fanciest timeshare I've yet stayed in - does that push it towards a 10 based on just my personal knowledge? Or is the scale based on my imagined range of common timeshare quality? Or is it based on any resort in the world, including very expensive cash only ones?

Personally, I'd like to try and only rate within timeshare systems - If there's only one 10 timeshare in the world, that's not super useful IMO - squeezing all units realistically into the 5-8 range kind of makes the range a little misleading - why is there no difference between 3 and 4 or 1 and 2? Those should then collapse into a single number, not 4 numbers and point 5s. Then again, maybe we're all thinking too hard about this.

Anyway, I'll try and keep that scale in mind going forward.

 

[TUGBrian]

I believe it would be extremely difficult to categorize or differentiate between 1-4....all of which are varying levels of "you dont want to buy, trade, rent, or even stay here if you can avoid it."

i also see very few 1-3 ratings...the ones that do show up in most cases seem to earn it even if some are basing it on a poor experience or poor reaction to a problem vs the resort itself!

 

[SmithOp]

I prefer to read the reviews rather than rely on the number rating. It's so subjective, by reading you get a sense of what the reviewer thinks is important. I admit to giving my resort a 10 because that's why we own it, we like everything about it and go there often.

I read several 7 reviews and the reasons for the lower number were odd, like being turned off because they thought the developers were renting it out like a hotel. The developers have that right for the units they still own, why use that as a criteria to downgrade the rating? Another had an exchange into the least desirable unit, well that's on the person depositing the unit they got and the exchange company.

 

[WaikikiFirst]

How do you determine your 1 to 10 rating of a resort when submitting a review?

Not that I have looked at them, but the situation sounds like restaurant reviews on tripadvisor and reviews of something else on some other website that I can't remember right now. They are totally subjective and the #1 factor seems to be "Did I enjoy it or was there a problem?" followed by #2 factor being Price vs Quality rather than absolute quality.
As with the "People dinging free TSs" thread, every demographic has its own set of priorities, so consider the demographic and realize the priorities make it subjective.

 

[WaikikiFirst]

I think 5 / average should not imply would not go back. The real world isn't lake wobegotten??? where everyone is above average

over the years, I have had 2 people tell me they do these TYPE of reviews (not TUG specifically) this way:
"10" = "A"
"9" = "B"
and so on, so you see, basically everything IS above avg , if you consider "5" to be avg.
I was speechless, but so it goes.

 

[klpca]

Owner reviews will probably always be higher because they receive better units, and probably have some other perks that tip the scale. As they should, btw. The owner reviews at a particular resort we stayed at were glowing - so I booked an exchange there. We received the "exchangers unit" and it was just awful. I spoke to another tugger who received the exact same unit two months before me. She had the same experience. My review of the place looks like an outlier compared to the owner reviews. I believe their reviews, but mine is also accurate. We had two different experiences.

Reviews are inherently subjective. Personal experience is different from person to person. I don't like huge resorts but will sometimes stay at one for various reasons. The resort could be spectacular but I will never give it a "10" because it will always feel too big for me. And things that others may like ("amazing pools" or "great activities") will not get a higher score for me because I probably didn't spend any time at the pool, and I certainly didn't do a scheduled activity.

The review score is probably good for ranking resorts within a geographic location, but not very accurate for comparing resorts overall. There is no substitute for reading the reviews and making your own decision, especially when people have taken the time to write out a review.

 

[susieq]

I prefer to read the reviews rather than rely on the number rating. It's so subjective, by reading you get a sense of what the reviewer thinks is important. I admit to giving my resort a 10 because that's why we own it, we like everything about it and go there often.

I read several 7 reviews and the reasons for the lower number were odd, like being turned off because they thought the developers were renting it out like a hotel. The developers have that right for the units they still own, why use that as a criteria to downgrade the rating? Another had an exchange into the least desirable unit, well that's on the person depositing the unit they got and the exchange company.

I also prefer to read the reviews and go by what people think. I feel ratings mean different things to different people ~ everyone has different expectations. Some people are "pickier" than others, and some people exchange for location ............... whatever floats your boat!!

 

[jp10558]

Owner reviews will probably always be higher because they receive better units, and probably have some other perks that tip the scale. As they should, btw. The owner reviews at a particular resort we stayed at were glowing - so I booked an exchange there. We received the "exchangers unit" and it was just awful. I spoke to another tugger who received the exact same unit two months before me. She had the same experience. My review of the place looks like an outlier compared to the owner reviews. I believe their reviews, but mine is also accurate. We had two different experiences.

Reviews are inherently subjective. Personal experience is different from person to person. I don't like huge resorts but will sometimes stay at one for various reasons. The resort could be spectacular but I will never give it a "10" because it will always feel too big for me. And things that others may like ("amazing pools" or "great activities") will not get a higher score for me because I probably didn't spend any time at the pool, and I certainly didn't do a scheduled activity.

The review score is probably good for ranking resorts within a geographic location, but not very accurate for comparing resorts overall. There is no substitute for reading the reviews and making your own decision, especially when people have taken the time to write out a review.

This is something else I have trouble with - I just can't rank "kid friendliness" cause I don't have kids, and haven't traveled with kids yet. I kind of hope to get my cousin to come along and "let me borrow his and his young kids perspective" some time, but IDK when of it it'll actually happen. I personally only know about the pool via my BIL who lives in the hot tub / pool, and really his criteria is - IS THERE A POOL open inside? We never do activities - getting another scheduled thing in while juggling resting, fun, and remote work just makes it feel like even more work, and I'm already also trying to get youtube footage for the video reviews now. Often the rec center is going to vary a lot also - this last one had ping pong and mini golf free - both things I actually will do. But the paid arcade was fun but expensive - I could as well have gone to a casino and maybe gotten something out of it!

We tend to travel for stuff NOT AT THE RESORT so that makes that part a little tricky for me. I'm now making a "youtube work" point to try and grab video and at least see the playgrounds or whatever on site, but "naturally"? I'll be at the nature park, or the outlet malls, or a local store or restaurant. In about an hour you can see what I'm talking about here:

 

[gravityrules]

I understand the points about reviews tending to be subjective based on the reviewer's individual experience, but my point is the TUG review template is trying to reduce that subjectivity by having criteria for the numerical rating ... but it seems most of us are not rating based on the criteria.

As an experiment, is there any circumstance where you would rate a condo resort with no amenities (other than the room) highly? Based on the review criteria I'm thinking it should not rate higher than 7.5 , no matter how much I liked it or enjoyed my stay.

 

[jp10558]

I understand the points about reviews tending to be subjective based on the reviewer's individual experience, but my point is the TUG review template is trying to reduce that subjectivity by having criteria for the numerical rating ... but it seems most of us are not rating based on the criteria.

As an experiment, is there any circumstance where you would rate a condo resort with no amenities (other than the room) highly? Based on the review criteria I'm thinking it should not rate higher than 7.5 , no matter how much I liked it or enjoyed my stay.

Not by the criteria I don't think. Which going forward I will try and follow better (I swear I hadn't seen that before, IDK how I've missed it / where it is lol). Actually that is part of the confusion I have - it almost seems like to even get a 6 there must be *some* planned activity. At least one during the week.

It's also unclear to me if an 8 or higher means there must be no extra charges for like a beach chair or bike or something. Does that also apply to activities where like they charge for the t-shirt for tie dying? Does HGVC Ocean Enclave rate a 7.5 because there's a beach chair fee - and if HGVC doesn't technically provide any of that, and it's a third party rental company (like it seemed to be) then does it now jump to a 9.5 (because it seems like the only "major" thing in Myrtle Beach is *the beach* and it has it right outside the door?

Ideally, while I know no one wants a huge rules lawyering, or maybe few would even know about what the ratings are supposed to mean - it would be really helpful to those who "do care" to have some of the ambiguities removed.

 

[Panina]

I always read what was written. I never just look at the rating. Sometimes someone gives a low rating but when you read the narrative you realize what they wrote really has no merit and vice versa.

 

[gravityrules]

I always read what was written. I never just look at the rating. Sometimes someone gives a low rating but when you read the narrative you realize what they wrote really has no merit and vice versa.

Although this thread is specifically about TUG reviews much of it applies to any numerical review system. I have a hard time believing you never look at ratings (here or elsewhere). I suspect you are discounting their value, weighting the actual review words more than the numerical rating (which I do too). But I don't completely ignore the ratings.

Other than the previously discussed owner bias I don't think anyone is 'gaming the system' for monetary gain here on TUG (plenty of games with Amazon ratings, maybe TripAdvisor too for all I know). If someone is making money through advertising or otherwise based on a rating then the temptation to game the system is immense.

JD Powers seems to have a category that every auto manufacturer can use to advertise they are tops in something ...

 

[TUGBrian]

I believe the rating chart was simply created MANY years ago on the forum for a general guideline on how to rate resorts...vs any sort of 10 commandments of timesharing.

different people will always have different experiences! I could absolutely see rating a resort a 10 that didnt have any activities if the resort were outstanding and I was on vacation without my kids!

could also see a really nice resort designed for adults getting a lower than expected rating because a family of 5 spent the entire time having to find ways to entertain their kids outside the resort etc.

I will say ive only ever seen ONE obvious review in my time that I felt was questionable only because the person who posted it had also just posted a rental at that same resort. I normally wouldnt have even noticed one or the other, but it was a 10 rating for a resort that had never gotten a 10 before...and then I saw the rental and made the connection.

 

[jp10558]

I believe the rating chart was simply created MANY years ago on the forum for a general guideline on how to rate resorts...vs any sort of 10 commandments of timesharing.

I'd be up for us maybe working out an update in 2024. Or at least doing some hashing out between the existing ambiguities. TBH, but I know this might be a lot of work for someone on the back end of TUG2, I'd love to see more like the tripadvisor ratings where there are a few categories. Which I also would put up for debate on the forum here.

For instance, I'd kind of love to have the main # either calculated from the other scales below or made "the vibe" number for the rater, i.e. how much do you LIKE the resort. But under it I'd like to see

• Scale of your attempted objective rating within timeshares
• Unit / Furnishing scale (i'd include recentness of refresh, cleanliness, design / floor layout, functionality of appliances, ease of access - stairs/elevator/walkup)
• Unit / accesses to amenities and activites scale - (Easy walking distance / safe for young teens to get there, Tricky walking ("complicated elevators, 3+ flights of stairs with roads or parking lots interleaved / age 15+ to feel comfortable, practically unwalkable distance measured in miles / must drive people to the location)
• Activities (None, Some - low quality, Some - average quality+, Calendar booked - Low quality, Calendar booked - average quality +).
• Amenities (None; Pool / hot tub; multiple pools, hot tubs, saunas; Arcade - Shuffleboard - Billiards - Ping Pong - Mini Golf - etc; On Site event center - trails - amusement park - water park - go karts - etc)
• Food / Shopping / Dining (None; Convenience Corner; Full Convenience Store - Restaurant; Multiple / stores - restaurants)
• Recommended for Adults Only, Adults with Teens, Adults with younger kids, Something for everyone

This is a quick rough idea just after waking up - I could also think about just trying to make the current scale clearly inclusive(gets better as it goes up) and remove the confusions I pointed out above. I might also clarify that if a resort is limited to a 7 because it charges for extras, a 7.5 is the "would be a 9 but is limited by the extra charges", or just the big number is a category, and a .5 above it means it's a resort I'd rate higher except that it's category limited.

I'd also argue to open up 9 to just being good resorts, 9.5 being like top 10% of resorts if they also meet the other criteria, and 10 being top 20 resorts in the world that you've seen or read about, over and above the rest of the "category" requirements of a 9.5.

IDK, I might just like making categories and rating systems too much.

 

[TUGBrian]

not against changing the criteria at all.

though i dont think we could switch from 1-10 to 1-5 without some severe issues there on the back end...but who knows.

perhaps just making the existing rating system more visible on the review page vs it being something you have to click on to view would improve things.

 

[TUGBrian]

will say this is one of the topics for saturdays newsletter, so we shall see how the rest of our TUG family responds!

 

[austdav]

Ask 10 questions. add them together and divide by 10. Walla you have your rating. Myself i would like to see 2 fee based questions. 1 for owners & 1 for people trading in. Also have a definite score. example. if no fees a 10 score, if less than $100 an 8, $150 - $100 a 7 and on down, that way the score is based on fees not what someone deems reasonable. Don't care what scale just that there ought to be one.