Knowing When to Give Up

When I'm playing chess (you think a turtle can't play chess?) and my opponent castles, I just yield right then and there. The way I see it, my opponent either:

1. Knows the rules for castling, which automatically makes them a much better chess player than me. Or,
2. Is cheating.