Much work has been devoted to understanding the evolutionary processes shaping genetic variation across genomes. Studies have found that neutral polymorphism is reduced close to genes and in regions of low recombination, suggesting the effects of natural selection. However, the effect of selection on neutral sequence divergence between species remains ambiguous. While studies have reported correlations between divergence and recombination, theoretical arguments suggest selection may not affect divergence at linked neutral sites. Here we address these outstanding issues by examining how natural selection has affected divergence between distantly related species. We show that neutral divergence is negatively correlated with functional content and positively correlated with estimates of background selection from primates. These patterns persist even when comparing humans and mice, species that split 75 million years ago. Further, neutral divergence is positively correlated with recombination rate. The correlation increases when focusing on genic regions, and biased gene conversion cannot explain all of this correlation. These signatures suggest that natural selection has affected linked divergence between distantly related species. Coalescent models indicate that background selection can generate these patterns. Even when the contribution of ancestral polymorphism to divergence is small, background selection in the ancestral population can still explain a large proportion of the variance in divergence across the genome. Thus, the view that selection does not affect divergence at linked neutral sites needs to be reconsidered. Our work has important implications for understanding evolution of genomes and interpreting patterns of genetic variation.