We show that by manipulating quantum interference in a multi-atom cavity quantum electrodynamics (CQED) system, the nonlinear excitation of the cavity-atom polariton can be resonantly enhanced while the linear excitation is suppressed. Under the appropriate conditions, it is possible to selectively enhance or suppress the polariton excitation with two free-pace laser fields. We report on an experiment with cold Rb atoms in an optical cavity and present experimental results that demonstrate such interference control of the CQED excitation and its direct application to studies of all-optical switching and cross-phase modulation of the cavity-transmitted light.