We investigate the annealing temperature dependence of the magnetic properties of CoFeB/MgO stacks with different buffer materials (Mo, Ta, and W). For Mo and W, bcc-crystalline and amorphous-like films are prepared by changing the deposition conditions. A relatively small saturation magnetization is maintained after annealing up to 400 °C for the samples with bcc-W, bcc-Mo, and amorphous-like Mo buffers. A small variation in magnetic dead layer thickness with annealing is observed for the samples with bcc-crystalline buffer layers. The interfacial anisotropy is found to mainly depend on the element of the buffer layer used regardless of its crystalline structure, and is larger for the samples with W and Mo buffers than those with Ta buffer. The sample with bcc-Mo buffer shows the highest robustness against annealing among the studied systems. We give a systematic picture based on the thermochemistry that can reasonably explain the observed buffer layer dependence of the variations in magnetic properties with annealing.