Yes, they kept calling. It showed up on our credit and knocked about 100 points. We initially decided: That's OK, we don't need to use credit for a few years, but I worried about the (very unlikely) potential of them suing us eventually. Also, one of us is going through a job search now, and we really don't need this sht in case a prospective employer pulls the report. We finally decided to just eat it and pay and actually use the vacation club; it turns out that the points offer some pretty options for some domestic trips. We're still getting ripped off a bit, but I can sleep. I also reserved a website domain and will put a legally well-vetted blog of my experiences up shortly, and try to encourage others to stay away from this hideous industry.
My best guess is that you would be OK, other than the credit score hit. I don't think you would get sued. I *think* the credit score drop and the harassment will be the worst of it. You can send them a letter telling them they cannot call, and then they can't, they can only send letters. Another post on the TUG site indicates 2 lawsuits out of 95 defaults, one with extenuating circumstances I doubt you have. You may want to find a consumer protection lawyer who will give you a free review of the realistic risks. Write a terse summary of your situation and the sales scam in advance. I saw a few online, and that was going to be my next step before deciding to just eat it. Stay away from the ones who specialize in timeshares; so far they seem like they're part of the cabal, to me anyway.
I'm so sorry you're going through this. Sending you good vibes for peace and closure on this.