Background: The distribution of body mass in populations of Western countries differs from that of populations of East Asian countries. In East Asian countries, fewer people have a high body mass index than those in Western countries. In Japan, the country with the highest number of older adults worldwide, many people have a low body mass index. Therefore, this study aimed to determine the association between a low body mass index and mortality in patients with sepsis in Japan. Methods: We conducted this retrospective analysis of 548 patients with severe sepsis from a multicenter prospective observational study. Multivariate logistic regression analyses determined the association between body mass index and 28-day mortality adjusted for age, sex, preexisting conditions, the occurrence of septic shock, Acute Physiology and Chronic Health Evaluation II scores, and Sequential Organ Failure Assessment scores. Furthermore, the association between a low body mass index and 28-day mortality was analyzed. Results: The low body mass index group represented 18.8% of the study population (103/548); the normal body mass index group, 57.3% (314/548); and the high body mass index group, 23.9% (131/548), with the 28-day mortality rates being 21.4% (22/103), 11.2% (35/314), and 14.5% (19/131), respectively. In the low body mass index group, the crude and adjusted odds ratios (95% confidence intervals) for 28-day mortality relative to the non-low body mass index (normal and high body mass index groups combined) group were 2.0 (1.1-3.4) and 2.3 (1.2-4.2), respectively. Conclusion: A low body mass index was found to be associated with a higher 28-day mortality than the non-low body mass index in patients with sepsis in Japan. Given that older adults often have a low body mass index, these patients should be monitored closely to reduce the occurrence of negative outcomes.