The ability of visual self-recognition in animals and infants is considered a hallmark of the domain-general cognitive representation of the self, which also underpins higher social ability. Cortical regions activated during self-face recognition in human adults have been accordingly expected to play the domain-general role in self-processing. However, there is no evidence of the involvement of this network in non-face domains. We compared cortical responses during face and name recognition of self, a friend, and an unfamiliar person, using functional magnetic resonance imaging (fMRI). Recognition of the self-face activated the right inferior frontal, precentral, supramarginal, and bilateral ventral occipitotemporal regions, consistent with previous findings, whereas these regions did not show self-specific activation during name recognition. During both face and name recognitions, increased activation for the friend and unfamiliar person than for the self was observed in the bilateral temporoparietal regions, and higher activation for the self and friend than for the unfamiliar person was observed in the medial cortical structures. These results suggest that the role of the self-specific networks during face recognition is not domain-general, but rather face-specific, and that the medial cortical structures, which are also implicated in self-referential processes, are not relevant to self-other distinction during face or name recognition. Instead, the reduced temporoparietal activation is a domain-general characteristic of the cortical response during self-recognition, which may reflect suppression of an automatic preparatory process for social interaction, possibly paralleling the disappearance of social behavior to the mirrored self-image at the emergence of self-recognition in animals and infants.