<p>I have some design diagrams only on paper. Scanning them to bitmaps is easy, but I've had no luck getting useful vectors out of them. I've tried vectorizers in programs like gimp, and a few online services. Generally, I end up with enormous numbers of spurious vectors (from dust, dotted lines, text on the  diagram, slightly variations in scanning contrast, etc).</p>

<p>What tools and/or techniques can I use to get a more useful vector result, that I can then modify in a normal CAD tool without spending absurd amounts of time cleaning it all up first?</p>

There is the capable but somewhat expensive
Scan2CAD
.
Otherwise, if you're happy with outlines and not centre lines, scan b&w, aggressively clean up macules, mask off text, and then vectorize in potrace, autotrace, etc. Alternatively, load the bitmap at the correct resolution into a drawing package as a raster layer and draw the lines/objects you want over it. This avoids the horrors of dotted lines.
Both ways are quite a bit of work, sadly.