We discuss the environmental dependence of the galaxy evolution based on deep panoramic imaging of two distant clusters, CL0016+1609 at z = 0.55 and RXJ0152.7-1257 at z = 0.83, taken with Suprime-Cam on Subaru as part of the PISCES project. By combining with the SDSS data as a local counterpart for comparison, we construct a large sample of galaxies that spans wide ranges in environment, time, and stellar mass (or luminosity). Based on local and global densities of galaxies, we classify three environments: field, group, and cluster. Then we focus on the color-magnitude relation of galaxies. In particular, we quantify how the color-magnitude relation is built-up as a function of time, environment, and mass. In the field environment, there is no clear color-magnitude relation at z = 0.83, while a clear relation is seen at lower redshifts, especially at the bright end. In groups, a relation is visible only at the bright end (MV <-20) at z=0.83. The relation is found to extend down to our magnitude limit in groups at lower redshifts. Clusters have a clear relation down to the magnitude limit at all redshifts considered here. We confirm the 'down-sizing' in the star formation recently reported both at low and high redshifts, where massive galaxies complete their star formation earliest and the truncation of star formation is propagated to smaller objects as time progresses. Our results suggest that 'down-sizing' is yet at its earlier stage in lower density regions, while it is more proceeded in high density regions. All in all, it is likely that the formation and evolution of galaxies are the earliest in massive systems and in high density regions, and it proceeds to less massive systems and to lower density regions.