When I'm the recipient of a Linked-In (or other "social networking") invite, I always refuse the invite, then send a quick / polite note to the requester (if I have a relationship with them) informing them that I don't use the service.
I find it very presumptuous when someone that I barely know at all sends me a Linked-In invite. I don't bother to inform those folks that I don't use the service, I just ignore it (thanks for the spam).
To me, a relationship is built on history and long-term trust. The fact that you "know" someone else that I know -- what do I care?
And I'm certainly not interested to enable people to similarly spam a bunch of other people that are part of my "network." A friend of mine who has used Linked-In extensively tells me of common situations where someone that he barely knows (who he's directly connected to) asks for a recommendation to someone else who's twice removed (but connected via other relationships). At a certain point, you have to question the logic of how these are real "relationships."
I'm in the white pages with 750,000 other residents ... I guess you could say we're all part of a "social network" too.