Through major histocompatibility complex class Ia leader sequence-derived (VL9) peptide binding and CD94/NKG2 receptor engagement, human leucocyte antigen E (HLA-E) reports cellular health to NK cells. Previous studies demonstrated a strong bias for VL9 binding by HLA-E, a preference subsequently supported by structural analyses. However, Mycobacteria tuberculosis (Mtb) infection and Rhesus cytomegalovirus-vectored SIV vaccinations revealed contexts where HLA-E and the rhesus homologue, Mamu-E, presented diverse pathogen-derived peptides to CD8+ T cells, respectively. Here we present crystal structures of HLA-E in complex with HIV and Mtb-derived peptides. We show that despite the presence of preferred primary anchor residues, HLA-E-bound peptides can adopt alternative conformations within the peptide binding groove. Furthermore, combined structural and mutagenesis analyses illustrate a greater tolerance for hydrophobic and polar residues in the primary pockets than previously appreciated. Finally, biochemical studies reveal HLA-E peptide binding and exchange characteristics with potential relevance to its alternative antigen presenting function in vivo.