Increasing evidence shows that phenotypic variance is genetically controlled, and the variance itself is a quantitative trait. The precise mechanism of genetic control over the variance, however, remains to be determined. Here, using complex trait analysis of gene expression, we show that common genetic variation contributes to increasing gene expression variability via distinct modes of action–e.g., epistasis and decanalization. We focused on expression variability QTLs (evQTLs), i.e., genetic loci associated with gene express variance, in the human genome. We found that a quarter of evQTLs could be attributed to the presence of “third-party” eQTLs. These SNPs are associated with gene expression in a fraction, rather than the entire set, of samples. Many additional evQTLs do not interact with other SNPs and are thus unexplained by the epistasis model; these are attributable to the decanalizing effect of evQTL variants. Here we present the decanalization model, which predicts that evQTLs influence gene expression variability through modulating the sensitivity of transcriptional machinery to environmental perturbation. To validate the model we measured the discordant gene expression between monozygotic twins, and also estimated the amplitude of stochastic gene expression noise using repeated RT-qPCR assays on single samples. Both measures were found to be associated with genotypes of evQTLs explained by the decanalization model. Together, our results suggest that genetic variants work interactively or independently to influence gene expression variability. We anticipate our analysis to be a starting point for more sophisticated mechanistic analyses and opens a new, variability-centered research avenue for mapping complex traits.