Glassiness is ubiquitous and diverse in characteristics in nature. Understanding their differences and classification remains a major scientific challenge. Here, we show that scaling of magnetic memories with time can be used to classify magnetic glassy materials into two distinct classes. The systems studied are high temperature superconductor-related materials, spin-orbit Mott insulators, frustrated magnets, and dilute magnetic alloys. Our bulk magnetization measurements reveal that most densely populated magnets exhibit similar memory behavior characterized by a relaxation exponent of 1-n\approx 0.6(1) 1 - n ≈ 0.6 (1). This exponent is different from 1-\boldsymbol\approx 1/3 1 - n ≈ 1 / 3 of dilute magnetic alloys that was ascribed to their hierarchical and fractal energy landscape, and is also different from 1-\boldsymbol=1 1 - n = 1 of the conventional Debye relaxation expected for a spin solid, a state with long range order. Furthermore, our systematic study on dilute magnetic alloys with varying magnetic concentration exhibits crossovers among the two glassy states and spin solid.