Local atomic arrangements of dielectric ϵ-Ga2O3 epitaxial thin films grown on indium doped tin oxide (ITO) and α-Al2O3 substrates were investigated by combining transmission electron microscopy (TEM) and X-ray fluorescence holography (XFH). TEM showed the orthorhombic lattice structure for the ϵ-Ga2O3 thin film on both the substrates and a significant in-plane Ga ion displacement on α-Al2O3. However, XFH revealed a partial disorder of Ga vacancies and a large displacement along the a-axis on ITO. On the other hand, highly ordered Ga vacancies and a displacement of Ga ions toward the Ga vacancy sites were observed for the film on α-Al2O3 by XFH. Such a high degree of freedom in atomic sites and displacements is believed to contribute to a high dielectric constant of ϵ-Ga2O3.