Although the atomic structure of amorphous alloys, which lacks long-range translational symmetry, may appear homogeneous at the macroscopic scale, their local dynamic and/or static properties however vary significantly according to the recent experimental and simulation results. In the literature of amorphous alloys, the nature of such local heterogeneities is currently an issue under debate. More importantly, since amorphous alloys are in a thermodynamically nonequilibrium state, their local structures constantly evolve during structural relaxation, physical aging and mechanical deformation. As such, local structural heterogeneities, which vary with the thermal and mechanical history of amorphous alloys, could provide a key to understand the structural origin of their mechanical behavior, such as anelasticity, viscoelasticity, plasticity and fracture. In this review article, we first review mechanical spectroscopy or dynamic mechanical analyses as an important tool to study the relaxation dynamics in amorphous alloys, with a focus on the possible correlation between the secondary (also called β)relaxation and the local structural heterogeneities of amorphous alloys. After that, we discuss the recent advances on the understanding of structural heterogeneities in metallic supercooled liquids and the influence of the structural heterogeneities on the overall mechanical properties of the corresponding amorphous alloys. Finally, we briefly discuss the further development of research on this subject.