It is found that an icosahedral quasicrystalline phase is formed in the Zr-Al-Ni-Cu glassy alloy by addition of Nb, Ta or V elements. The icosahedral phase was confirmed as a primary precipitation phase in the melt-spun Zr65Al7.5Ni10Cu12.5X5 (X = Nb, Ta or V) glassy alloys with a two-stage crystallization process after the distinct glass transition. The onset temperature of the transformation from glass to icosahedral phase is 705 K for Nb, 710 K for Ta and 702 K for V at the heating rate of 0.67 Ks-1. The size of the icosahedral particles is in the range of 10 to 50 nm. The second crystallization reaction results in the formation of Zr2Cu+Zr2Ni+Zr3Al2 phases through a sharp exothermic reaction. The formation of a nano scale icosahedral phase in the Zr-Al-Ni-Cu glassy alloy by addition of Nb, Ta or V, as well as noble metals, indicates that the increase of the nucleation rate contributes to the precipitation of the icosahedral phase, leading to the concept that the icosahedral short-range order exists in the glassy state in the Zr-Al-Ni-Cu alloy.