We performed optical spectroscopy on a thin film of FeSe, in which orbital ordering shows up in the orthorhombic phase below Ts. The optical conductivity spectrum for FeSe exhibits no gap feature in the orbitally ordered state, in contrast to those for iron pnictides showing the gap opening in the magnetostructurally ordered phase. Instead, we observed a gradual suppression of the coherent carrier density with temperature. This highlights a peculiar metallic state in FeSe that the Fermi surfaces are gradually modified below Ts. We also found an anomalous behavior of the optical phonon mode, indicating the intimate connection between the orbital and lattice degrees of freedom. The result implies the orbital origin of the structural transition.