Mitochondria, which are essential organelles in resting and replicating cells, can vary in number, mass and shape. Past research has primarily focused on short-term molecular mechanisms underlying fission/fusion. Less is known about longer-term mitochondrial behavior such as the overall makeup of cell populations' morphological patterns and whether these patterns can be used as biomarkers of drug response in human cells. We developed an image-based analytical technique to phenotype mitochondrial morphology in different cancers, including cancer cell lines and patient-derived cancer cells. We demonstrate that (i) cancer cells of different origins, including patient-derived xenografts, express highly diverse mitochondrial phenotypes; (ii) a given phenotype is characteristic of a cell population and fairly constant over time; (iii) mitochondrial patterns correlate with cell metabolic measurements and (iv) therapeutic interventions can alter mitochondrial phenotypes in drug-sensitive cancers as measured in pre- versus post-treatment fine needle aspirates in mice. These observations shed light on the role of mitochondrial dynamics in the biology and drug response of cancer cells. On the basis of these findings, we propose that image-based mitochondrial phenotyping can provide biomarkers for assessing cancer phenotype and drug response.