Although Casimir forces are inseparable from their fluctuations, little is known about these fluctuations in soft matter systems. We use the membrane stress tensor to study the fluctuations of the membrane-mediated Casimir-like force. This method enables us to recover the Casimir force between two inclusions and to calculate its variance. We show that the Casimir force is dominated by its fluctuations. Furthermore, when the distance d between the inclusions is decreased from infinity, the variance of the Casimir force decreases as -1/d2. This distance dependence shares a common physical origin with the Casimir force itself.