Findings are inconsistent about whether tobacco, alcohol, and human papillomavirus (HPV) are two independent HNC risk factor groups that distinguish an infection-associated cancer from a tobacco/alcohol-associated HNC. We found that cancer in the oral cavity risk was greater in HPV-E6/E7 seropositive/heavy tobacco users (adjusted OR = 3.5) than in HPV-seronegative/heavy tobacco users (adjusted OR = 1.4); and HPV-seropositive/heavy alcohol users (adjusted OR = 9.8) had greater risk than HPV-seronegative/heavy alcohol users (adjusted OR = 3.1). In contrast, the risk of oropharyngeal cancer was greater in the HPV-seronegative/heavy tobacco (adjusted OR = 11.0) than in HPV-seropositive/heavy tobacco (adjusted OR = 4.7) users and greater in HPV-seronegative/heavy alcohol users (adjusted OR = 24.3) compared to HPV-seropositive/heavy alcohol users (adjusted OR = 8.5). Disease-specific and recurrence-free adjusted survival were significantly worse in oropharyngeal HPV-seronegative cases with no survival differences by HPV status seen in oral cavity cases. The association between tobacco/alcohol, HPV, and tumor site is complex. There appear to be distinct tumor site differences in the combined exposure risks, suggesting that different molecular pathways are involved.