Person recognition has been assumed to entail many types of person-specific cognitive responses, including retrieval of knowledge, episodic recollection, and emotional responses. To demonstrate the cortical correlates of this modular structure of multimodal person representation, we investigated neural responses preferential to personally familiar people and responses dependent on familiarity with famous people in the temporal and parietal cortices. During functional magnetic resonance imaging (fMRI) measurements, normal subjects recognized personally familiar names (personal) or famous names with high or low degrees of familiarity (high or low, respectively). Effects of familiarity with famous people (i.e., high-low) were identified in the bilateral angular gyri, the left supramarginal gyrus, the middle part of the bilateral posterior cingulate cortices, and the left precuneus. Activation preferentially relevant to personally familiar people (i.e., personal-high) was identified in the bilateral temporo-parietal junctions, the right anterolateral temporal cortices, posterior middle temporal gyrus, posterior cingulate cortex (with a peak in the posterodorsal part), and the left precuneus; these activation foci exhibited varying degrees of activation for high and low names. An equivalent extent of activation was observed for all familiar names in the bilateral temporal poles, the left orbito-insular junction, the middle temporal gyrus, and the anterior part of the posterior cingulate cortex. The results demonstrated that distinct cortical areas supported different types of cognitive responses, induced to different degrees during recognition of famous and personally familiar people, providing neuroscientific evidence for the modularity of multimodal person representation.