The color and patterns of animal-pollinated flowers are known to have effects on pollinator attraction. In this study, the relative importance of flower color and color contrast patterns on pollinator attraction was examined in two pollinator groups, swallowtail butterflies and hawkmoths using two Hemerocallis species; butterfly-pollinated H. fulva and hawkmoth-pollinated H. citrina, having reddish and yellowish flowers in human vision, respectively. Flowers of both species have UV bullseye patterns, composed of UV-absorbing centers and UV-reflecting peripheries, known to function as a typical nectar guide, but UV reflectance was significantly more intense in the peripheries of H. citrina flowers than in those of H. fulva flowers. Comparison based on the visual systems of butterflies and hawkmoths showed that the color contrast of the bullseye pattern in H. citrina was more intense than that in H. fulva. To evaluate the relative importance of flower color and the color contrast of bullseye pattern on pollinator attraction, we performed a series of observations using experimental arrays consisting of Hemerocallis species and their hybrids. As a result, swallowtail butterflies and crepuscular/nocturnal hawkmoths showed contrasting preferences for flower color and patterns: butterflies preferred H. fulva-like-colored flowers whereas the preference of hawkmoths was affected by the color contrast of the bullseye pattern rather than flower color. Both crepuscular and nocturnal hawkmoths consistently preferred flowers with stronger contrast of the UV bullseye pattern, whereas the preference of hawkmoths for flower color was incoherent. Our finding suggests that hawkmoths can use UV-absorbing/reflecting bullseye patterns for foraging under light-limited environments and that the intensified bullseye contrast of H. citrina evolved as an adaptation to hawkmoths. Our results also showed the difference of visual systems between pollinators, which may have promoted floral divergence.