A quantum network that consists of several components should ideally work on a single physical platform. Neutral alkali atoms have the potential to be very well suited for this purpose due to their electronic structure, which involves long-lived nuclear spins and very sensitive highly excited Rydberg states. In this Letter, we describe a fabrication method based on quartz glass to structure arbitrary shapes of microscopic vapor cells. We show that the usual spectroscopic properties known from macroscopic vapor cells are almost unaffected by the strong confinement.