Well my first one was just last year to be honest. I'd never attended one before then. As far as the Summit goes there aren't as many affiliates there as there are affiliate networks, lead generation companies, etc. but overall it's a good experience. You get to network with folks, share ideas and there are even some great after parties.
The Affiliate Summit has 2 days of exhibits where companies setup booths and you just walk around talking to folks. They also have other seminars going on in the "meeting rooms" on various topics (traffic building, affiliate programs, etc.) So there's a lot going on and if you have a full pass you can attend any event you want.
I got lucky, ShareASale gave me a free, full-conference pass ($1,000) so all I had to pay for was my flight and hotel fees.