To trace the fate of individual pollen grains through pollination processes, we determined genotypes of single pollen grains deposited on Hemerocallis stigmas in an experimental mixed-species array. Hemerocallis fulva, pollinated by butterflies, has diurnal, reddish and unscented flowers, and H. citrina, pollinated by hawkmoths, has nocturnal, yellowish and sweet scent flowers. We observed pollinator visits to an experimental array of 24 H. fulva and 12 F2 hybrids between the two species (H. fulva and H. citrina) and collected stigmas after every trip bout of swallowtail butterflies or hawkmoths. We then measured selection by swallowtail butterflies or hawkmoths through male and female components of pollination success as determined by single pollen genotyping. As expected, swallowtail butterflies imposed selection on reddish color and weak scent: the number of outcross pollen grains acquired is a quadratic function of flower color with the maximum at reddish color, and the combined pollination success was maximal at weak scent (almost unrecognizable for human). This explains why H. fulva, with reddish flowers and no recognizable scent, is mainly pollinated by swallowtail butterflies. However, we found no evidence of hawkmoths-mediated selection on flower color or scent. Our findings do not support a hypothesis that yellow flower color and strong scent intensity, the distinctive floral characteristics of H. citrina, having evolved in adaptations to hawkmoths. We suggest that the key trait that triggers the evolution of nocturnal flowers is flowering time rather than flower color and scent.