That's definitely strange. I haven't experienced a decrease in my traffic; I'm actually increasing in traffic for my site. Perhaps it's the amount of times the link appears on a forum? I know you have more posts than I do on WB.
[EDIT] I may test out un-linking on WB for a bit to see if my traffic increases. We'll see.