For the development of three-dimensional devices, selective epitaxial growth (SEG) technology has attracted much attention. SEG has been applied to fabricate many devices and it is expected to be used in future manufacturing processes. Therefore, its characteristics must be examined in detail to extend its application. For the fabrication of a three-dimensional device structure, the selectivity of epitaxial growth must be accurately controlled not only on Si and SiO2, but also on different impurity-type silicon surfaces. In this work, we investigated some characteristics of the SEG process, especially focusing on the surface roughness after SEG. Both vapor phase epitaxy (VPE) and solid phase epitaxy (SPE) were performed on ion-implanted silicon-on-insulator (SOI) thin wafers. It was often reported that epitaxial growth is very sensitive to the crystal condition of the substrate on which the films are deposited. However, we first revealed that the impurity type (p- or n-type) and its concentration at the substrate surface markedly changed the roughness and incubation times of the deposition. From our results, SPE with the oxide cap layer formation is effective for maintaining almost the same flatness as the original wafer surface. It is also effective to employ the low-temperature H2/Xe plasma treatment after the SEG to reduce roughness.