We use genome-wide data from the third generation respondents of the Framingham Heart Study to estimate heritability in body mass index using different quantities of the measured genotype. Heritability decreases rapidly when SNPs implicated by a genome-wide association study are removed but shows essentially no decline when SNPs implicated by a gene-environment interaction in a second genome-wide analysis are removed. This second result is highlighted by our additional finding that the SNPs which explain heritability amongst a subsample defined by higher educational attainment explain no heritability of the heritability in the lower education group, and vice-versa. Finally, we do find consistent heritability estimates when we compare family-based estimates versus those based on measured genotype.