Rat bite fever is rare in Western countries. It can be very difficult to diagnose as blood cultures are typically negative and a history of rodent exposure is often missed. Unless a high index of suspicion is maintained, the associated polyarthritis can be mistaken for rheumatoid arthritis. We report a case of culture-positive rat bite fever in a 46-year-old female presenting with fever and polyarthritis. The clinical presentation mimicked rheumatoid arthritis. Infection was complicated by discitis, a rare manifestation. We discuss the diagnosis and management of this rare zoonotic infection. We also review nine reported cases of rat bite fever, all of which had an initial presumptive diagnosis of a rheumatological disorder. Rat bite fever is a potentially curable infection but can have a lethal course if left untreated.