Discovery of durable memory B cell (MBC) subsets against neutralizing viral epitopes is critical for determining immune correlates of protection from SARS-CoV-2 infection. Here, we identified functionally distinct SARS-CoV-2-reactive B cell subsets by profiling the repertoire of convalescent COVID-19 patients using a high-throughput B cell sorting and sequencing platform. Utilizing barcoded SARS-CoV-2 antigen baits, we isolated thousands of B cells that segregated into discrete functional subsets specific for the spike, nucleocapsid protein (NP), and open reading frame (ORF) proteins 7a and 8. Spike-specific B cells were enriched in canonical MBC clusters, and monoclonal antibodies (mAbs) from these cells were potently neutralizing. By contrast, B cells specific to ORF8 and NP were enriched in naïve and innate-like clusters, and mAbs against these targets were exclusively non-neutralizing. Finally, we identified that B cell specificity, subset distribution, and affinity maturation were impacted by clinical features such as age, sex, and symptom duration. Together, our data provide a comprehensive tool for evaluating B cell immunity to SARS-CoV-2 infection or vaccination and highlight the complexity of the human B cell response to SARS-CoV-2.