Most spontaneous mutations affecting fitness are likely to be deleterious, but the strength of selection acting on them might be impacted by environmental stress. Such stress-dependent selection could expose hidden genetic variation, which in turn might increase the adaptive potential of stressed populations. On the other hand, this variation might represent a genetic load and thus lead to population extinction under stress. Previous studies to determine the link between stress and mutational effects on fitness, however, have produced inconsistent results. Here, we determined the net change in fitness in 29 genotypes of the green algae Chlamydomonas reinhardtii that accumulated mutations in the near absence of selection for approximately 1,000 generations across two stress gradients, increasing NaCl and decreasing phosphate. We found mutational effects to be magnified under extremely stressful conditions, but such effects were specific both to the type of stress as well as to the genetic background. The detection of stress-dependent fitness effects of mutations depended on accurately scaling relative fitness measures by generation times, thus offering an explanation for the inconsistencies among previous studies.