Tumorigenesis can induce adaptive T-cell-mediated immune responses against malignant cells. Such cellular immune responses are actively suppressed by cancer cells via mechanisms of immune tolerance. We studied T-cell responses against tumor growth by examining tumor-infiltrating lymphocytes (TILs) in upper gastrointestinal (GI) cancers. The number of T-bet(+) TILs correlates with better survival of esophageal cancer patients. Using well-defined mouse models, we have further shown that T-bet and Eomes are both required for the adaptive anti-tumor immunity by regulating T-cell trafficking into the tumor tissue and their effector functions inside the tumor microenvironment. In order to gain further insight into the tumor immune microenvironment in the upper GI cancer, we have also studied expression levels of co-inhibitory molecules such as B7-H1/PD-L1 and B7-H4 in tissue specimens of esophageal and gastric cancers. These inhibitory B7 molecules were expressed at high but variable levels by cancer cells. The overexpression of these molecules correlates with poor clinicopathological parameters and shorter patient survival time. The number of CD3(+) and CD8(+) TILs correlates inversely with expression levels of B7-H4 in samples from esophageal cancer, supporting a role of active immune suppression by inhibitory B7 molecules in the tumor microenvironment. In addition, TILs show functional exhaustion and express high levels of PD-1 and Tim-3. We propose that metabolic competition mediated by phosphatidylinositol 3-kinases (PI3Ks) characterizes the immune suppression within cancer tissues. Future tumor vaccine design should combine blockade of B7 inhibitory molecules and enhancement of T-bet and Eomes levels within the tumor microenvironment.