Four-dimensional (4D) radiotherapy is the explicit inclusion of the temporal changes in anatomy during the imaging, planning, and delivery of radiotherapy. One key component of 4D radiotherapy planning is the ability to automatically ("auto") create contours on all of the respiratory phase computed tomography (CT) datasets comprising a 4D CT scan, based on contours manually drawn on one CT image set from one phase. A tool that can be used to automatically propagate manually drawn contours to CT scans of other respiratory phases is deformable image registration. The purpose of the current study was to geometrically quantify the difference between automatically generated contours with manually drawn contours. Four-DCT data sets of 13 patients consisting of ten three-dimensional CT image sets acquired at different respiratory phases were used for this study. Tumor and normal tissue structures [gross tumor volume (GTV), esophagus, right lung, left lung, heart and cord] were manually drawn on each respiratory phase of each patient. Large deformable diffeomorphic image registration was performed to map each CT set from the peak-inhale respiration phase to the CT image sets corresponding with subsequent respiration phases. The calculated displacement vector fields were used to deform contours automatically drawn on the inhale phase to the other respiratory phase CT image sets. The code was interfaced to a treatment planning system to view the resulting images and to obtain the volumetric, displacement, and surface congruence information; 692 automatically generated structures were compared with 692 manually drawn structures. The auto- and manual methods showed similar trends, with a smaller difference observed between the GTVs than other structures. The auto-contoured structures agree with the manually drawn structures, especially in the case of the GTV, to within published interobserver variations. For the GTV, fractional volumes agree to within 0.2+/-0.1, center of mass displacements agree to within 0.5+/-1.5 mm, and agreement of surface congruence is 0.0+/-1.1 mm. The surface congruence between automatic and manual contours for the GTV, heart, left lung, right lung and esophagus was less than 5 mm in 99%, 94%, 94%, 91% and 89%, respectively. Careful assessment of the performance of automatic algorithms is needed in the presence of 4D CT artifacts.