We present an adaptive optical neural network based on a large-scale event-driven architecture. In addition to changing the synaptic weights (synaptic plasticity), the optical neural network's structure can also be reconfigured enabling various functionalities (structural plasticity). Key building blocks are wavelength-addressable artificial neurons with embedded phase-change materials that implement nonlinear activation functions and nonvolatile memory. Using multimode focusing, the activation function features both excitatory and inhibitory responses and shows a reversible switching contrast of 3.2 decibels. We train the neural network to distinguish between English and German text samples via an evolutionary algorithm. We investigate both the synaptic and structural plasticity during the training process. On the basis of this concept, we realize a large-scale network consisting of 736 subnetworks with 16 phase-change material neurons each. Overall, 8398 neurons are functional, highlighting the scalability of the photonic architecture.