Genome-wide association studies have suggested that allelic variations in the CACNA1C gene confer susceptibility to schizophrenia and bipolar disorder only in women. Here we investigated the sex-specific effects of the CACNA1C variant rs1024582 on psychiatry-related traits, brain activity during tasks and rest, and brain volume in 1207 normal male and female subjects. After correcting for multiple comparisons, there were significant interaction effects between sex and the minor allele of this polymorphism on the hostile behavior subscale scores of the Coronary-Prone Type Scale mediated by higher scores in female carriers of the minor allele. Imaging analyses revealed significant interaction effects between sex and the minor allele on fractional amplitude of low-frequency fluctuations in the right dorsolateral prefrontal cortex and on brain activity during the 2-back task in areas of the right posterior cingulate cortex, right thalamus, and right hippocampus, which were all mediated by reduced activity in female carriers of the minor allele. Our results demonstrated that the rs1024582 risk variant of CACNA1C is associated with reduced activity in the frontolimbic regions at rest and during a working memory task as well as with greater hostility in females in the healthy population.