He's thinking: "What...? Do they think I look like a loser, which is why they're not letting me in on that?"

(Well, Dude...I hate to tell you, but you shouldn't rule out that possibility. Otherwise, you would be in on that.)