Many temperate forests of the Northeastern United States and Europe have received significant anthropogenic acid and nitrogen (N) deposition over the last century. Although temperate hardwood forests are generally thought to be N-limited, anthropogenic deposition increases the possibility of phosphorus (P) limiting productivity in these forest ecosystems. Moreover, inorganic P availability is largely controlled by soil pH and biogeochemical theory suggests that forests with acidic soils (i.e., <pH 5) are particularly vulnerable to P limitation. Results from previous studies in these systems are mixed with evidence both for and against P limitation. We hypothesized that shifts in mycorrhizal colonization and community structure help temperate forest ecosystems overcome an underlying P limitation by accessing mineral and organic P sources that are otherwise unavailable for direct plant uptake. We examined arbuscular mycorrhizal (AM) and ectomycorrhizal (EcM) communities and soil microbial activity in an ecosystem-level experiment where soil pH and P availability were manipulated in mixed deciduous forests across eastern Ohio, USA. One year after treatment initiation, AM root biomass was positively correlated with the most available P pool, resin P, while AM colonization was negatively correlated. In total, 15,876 EcM root tips were identified and assigned to 26 genera and 219 operational taxonomic units (97% similarity). Ectomycorrhizal richness and root tip abundance were negatively correlated with the moderately available P pools, while the relative percent of tips colonized by Ascomycetes was positively correlated with soil pH. Canonical correspondence analysis revealed regional, but not treatment, differences in AM communities, while EcM communities had both treatment and regional differences. Our findings highlight the complex interactions between mycorrhizae and the soil environment and further underscore the fact that mycorrhizal communities do not merely reflect the host plant community.