Polymer modification is a promising approach to enhancing the performance of asphalt binders. However, a major issue plaguing polymer mixing with asphalt is the phase separation of these two components post-mixing. In the field of polymer science, the Flory-Huggins theory has been long established as the way to describe the conditions under which a mixture is stable or unstable. This Flory-Huggins interaction parameter χ and the molecular weights of the components are required under this theory. This work uses molecular dynamics simulations to calculate χ between various polymers found in waste plastics and asphalt. The compatibility of these polymers with the asphalt binder is discussed as a function of binder composition, molecular weight, and temperature. Four common polymers (polyethylene, polystyrene, polypropylene, and polybutadiene) were tested. While none of the polymers showed good compatibility with asphalt, PE was the most compatible. Stable compatibilization of plastics with asphalt will require additives or chemical modification of the plastic.