Bone marrow (BM) is an organ responsible for crucial processes in living organs, e. g., hematopoiesis. In recent years, Organ-on-a-Chip (OoC) devices have been used to satisfy the need for in vitro systems that better mimic the phenomena occurring in the BM microenvironment. Given the growing interest in these systems and the diversity of developed devices, an integrative systematic literature review is required. We have performed this review, following the PRISMA method aiming to identify the main characteristics and assess the effectiveness of the devices that were developed to represent the BM. A search was performed in the Scopus, PubMed, Web of Science and Science Direct databases using the keywords (("bone marrow" OR "hematopoietic stem cells" OR "haematopoietic stem cells") AND ("organ in a" OR "lab on a chip" OR "microfluidic" OR "microfluidic*" OR ("bioreactor" AND "microfluidic*"))). Original research articles published between 2009 and 2020 were included in the review, giving a total of 21 papers. The analysis of these papers showed that their main purpose was to study BM cells biology, mimic BM niches, model pathological BM, and run drug assays. Regarding the fabrication protocols, we have observed that polydimethylsiloxane (PDMS) material and soft lithography method were the most commonly used. To reproduce the microenvironment of BM, most devices used the type I collagen and alginate. Peristaltic and syringe pumps were mostly used for device perfusion. Regarding the advantages compared to conventional methods, there were identified three groups of OoC devices: perfused 3D BM; co-cultured 3D BM; and perfused co-cultured 3D BM. Cellular behavior and mimicking their processes and responses were the mostly commonly studied parameters. The results have demonstrated the effectiveness of OoC devices for research purposes compared to conventional cell cultures. Furthermore, the devices have a wide range of applicability and the potential to be explored.