Do large populations always outcompete smaller ones? Does increasing the mutation rate have a similar effect to increasing the population size, with respect to the adaptation of a population? How important are substitutions in determining the adaptation rate? In this study, we ask how population size and mutation rate interact to affect adaptation on empirical adaptive landscapes. Using such landscapes, we do not need to make many ad hoc assumption about landscape topography, such as about epistatic interactions among mutations or about the distribution of fitness effects. Moreover, we have a better understanding of all the mutations that occur in a population and their effects on the average fitness of the population than we can know in experimental studies. Our results show that the evolutionary dynamics of a population cannot be fully explained by the population mutation rate NμN\mu; even at constant NμN\mu, there can be dramatic differences in the adaptation of populations of different sizes. Moreover, the substitution rate of mutations is not always equivalent to the adaptation rate, because we observed populations adapting to high adaptive peaks without fixing any mutations. Finally, in contrast to some theoretical predictions, even on the most rugged landscapes we study, small population size is never an advantage over larger population size. These result show that complex interactions among multiple factors can affect the evolutionary dynamics of populations, and simple models should be taken with caution.