Understanding the genetic basis of local adaptation requires insight in the fitness effects of individual loci under natural field conditions. While rapid progress is made in the search for genes that control differences between plant populations, it is typically unknown whether the genes under study are in fact key targets of habitat-specific natural selection. Using a quantitative trait loci (QTL) approach, we show that a QTL associated with flowering-time variation between two locally adapted wild barley populations is an important determinant of fitness in one, but not in the other population's native habitat. The QTL mapped to the same position as a habitat-specific QTL for field fitness that affected plant reproductive output in only one of the parental habitats, indicating that the genomic region is under differential selection between the native habitats. Consistent with the QTL results, phenotypic selection of flowering time differed between the two environments, whereas other traits (growth rate and seed weight) were under selection but experienced no habitat-specific differential selection. This implies the flowering-time QTL as a driver of adaptive population divergence. Our results from phenotypic selection and QTL analysis are consistent with local adaptation without genetic trade-offs in performance across environments, i.e. without alleles or traits having opposing fitness effects in contrasting environments.