Cross-reactive CD4+ T cells that recognize severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) are more commonly detected in the peripheral blood of unexposed individuals compared with SARS-CoV-2–reactive CD8+ T cells. However, large numbers of memory CD8+ T cells reside in tissues, feasibly harboring localized SARS-CoV-2–specific immune responses. To test this idea, we performed a comprehensive functional and phenotypic analysis of virus-specific T cells in tonsils, a major lymphoid tissue site in the upper respiratory tract, and matched peripheral blood samples obtained from children and adults before the emergence of COVID-19 (coronavirus disease 2019). We found that SARS-CoV-2–specific memory CD4+ T cells could be found at similar frequencies in the tonsils and peripheral blood in unexposed individuals, whereas functional SARS-CoV-2–specific memory CD8+ T cells were almost only detectable in the tonsils. Tonsillar SARS-CoV-2–specific memory CD8+ T cells displayed a follicular homing and tissue-resident memory phenotype, similar to tonsillar Epstein-Barr virus–specific memory CD8+ T cells, but were functionally less potent than other virus-specific memory CD8+ T cell responses. The presence of preexisting tissue-resident memory CD8+ T cells in unexposed individuals could potentially enable rapid sentinel immune responses against SARS-CoV-2.