Theoretical studies predict hydrophobic matching between transmembrane domains of proteins and bilayer lipids to be a physical mechanism by which membranes laterally self-organize. We now experimentally study the direct consequences of mismatching of transmembrane peptides of different length with bilayers of different thicknesses at the molecular level. In both model membranes and simulations we show that cholesterol critically constrains structural adaptations at the peptide-lipid interface under mismatch. These constraints translate into a sorting potential and lead to selective lateral segregation of peptides and lipids according to their hydrophobic length.