Climbers wear small sized shoes because climbing shoes are meant to be very minimal and snug. When climbers wear climbing shoes that are slightly smaller than their actual shoe size, these attributes are slightly enhanced, as is the performance a climber can gain from wearing smaller climbing shoes.
This can be a double-edged sword, though. If a climber wears climbing shoes that are too small, they may not gain any of the benefits climbing shoes provide, and may actually worsen their performance, not to mention injure or damage their feet.
Finding the right climbing shoes and the right size are not as simple as some may think. It takes going through a few shoes and trying new ones out until you find the right pair for you.