Is a man wearing a cardigan a complete turn-off?

I confess to owning one but my doctor says he thinks he can cure me.