Every political group attracts a few crackpots, but no group attracts as high a percentage of crackpots as groups with unpopular opinions. Outcasts are drawn to fringe groups — and fringe groups have little enough support that they don’t want to alienate any potential supporters.
I’ve been aware for a long time that libertarian and some conservative groups attract crazy people. Many times, these crazy people are highly intelligent, very weird and often obsessed with something strange. Those folks aren’t generally going to be accepted among the mainstream parties, because those groups have plenty of support and it’s easy to edge the weirdos out. But fringe groups accept the weirdos more readily.
And why not? One of the core libertarian beliefs is that people have the right to be whatever they want to be. If somebody’s life centers around promoting drinking colloidal silver to cure every ailment under the sun — or trying to communicate with aliens or researching conspiracies about how the Bilderbergers rule the world — hey, that’s his business, even if he’s nuts. As long as he agrees with us that other people have the right to believe and act as they choose, he’s welcome in the “liberty tent.”

Should a rational person question orthodox assumptions on climate?
What makes good science fiction? Aya Katz and I discuss ‘Podkayne’
Hypocritical Republicans wimp out on free market when politics calls
What really caused me to run from a ‘haunted house’ long ago?
Jobs are created from ‘selfish’ acts; they don’t just exist on their own
The real crime is how CNN is trying to manipulate what you believe