I love the ease and cost of the pre-fab. And from the information, I've received they are actually built better and stronger than site built.
Here's my problem with them; they tend to use all light weight finish materials so they can be moved with their trucks. The sheetrock is 1/4 or less, their cabinets are cheap, their sinks and tubs plastic. Also, they don't have a good pitch in their roofs. They are made short to go under power lines.
If you could buy one without the finishes, and then simply have the finishes put in once it was set, or buy a used one and have it gutted and re-finished, I think that would be the best.
They are a good product, structurally and weather wise. I just don't like the light-weight siding and finish work. If it doesn't bother you, you'll get a good product at a great price and so much easier than site built.