Your lats and your back would do most of the work while climbing.

Something beginner climbers usually experience is that sensation of discovering "new" muscles in their arms. These are just muscles that they have not used before, or not as often as they do in climbing. Many may think that this means that your arms are the most worked muscles in climbing, since they hurt the most, but, it's just because they are normally not utilized in the same way as they are when climbing.

Your back muscles, on the other hand, are those which actually pull you up. Your arms are used for keeping you on the wall, but what lifts you up, other than your feet which should be pushing you up, are your lats and back muscles.

Think of it like a pull-up. You can hang from the bar for quite a while, but, the moment you start to lift your self up, and do this repeatedly, your back will begin to slowly fail, as it is the back muscles that are bringing you up.