Outbreeding depression: Avenues for further research

A response to Razib Khan

Written by Peter Frost.

Razib Khan has written an interesting reply to my piece on outbreeding, specifically my contention that fertility starts to decrease beyond marriages with fourth cousins (Frost, 2024; Khan, 2024). This decrease, I argued, is “the canary in the coal mine.” It is the first sign that something is going wrong as outbreeding brings together increasingly different gene systems.

In my piece, I reviewed the evidence on outbreeding and decreased fertility:

• Helgason et al. (2008) used a genealogical database to examine data on 160,811 Icelandic couples born between 1800 and 1964. They found evidence of both inbreeding depression and outbreeding depression, with fertility peaking at marriages between third or fourth cousins. The children of such marriages likewise had the same peak fertility. The inverted V-shaped association between fertility and parental relatedness held up even when the data were broken down into 25-year intervals.

• Laboriau and Amorim (2008) used the Danish Central Personal Register to examine data on 22,298 Danish women born in 1954 and still living in that country in 1969. They likewise found evidence of both inbreeding depression and outbreeding depression, with fertility peaking at marriages between partners separated by a distance of 75 km between their home parishes. The relationship between fertility and distance was not explained by education, family income, urbanicity, or mother's age at first birth.

• Joffe (2010) argued that an increase in outbreeding explains a century-long decline in sperm quality and a corresponding increase in testicular cancer. This explanation seems to be a better "fit" to the geographic and temporal distribution of sperm quality decline than the more popular explanation of increased exposure to estrogenic compounds.

• Davenport and Steggerda (1928) administered psychological tests in Jamaica to 300 adults and some 1,200 children of Black, White, and biracial origin. The biracial participants had results midway between those of the Black and White participants, with the exception of certain psychological tests where they had to compare or imagine objects in two or three dimensions. On such tests, their failure rates exceeded those of the other participants.

Keep in mind that decreased fertility is just that – a lower chance of having a baby due to a higher risk of embryonic or fetal miscarriage. In most cases, patience and persistence will pay off. Nonetheless, those embryos and fetuses are miscarrying for a reason, and the reason seems to be an increase in genetic incompatibilities with lower parental relatedness.

An extreme example is provided by crossings between modern humans and archaic humans, like the Neandertals. At some point, Neandertal genes entered our gene pool, either directly 50,000 to 60,000 years ago when modern humans left Africa or indirectly as early as 120,000 years ago via an intermediate population in the Middle East (the Skhul-Qafzeh hominins).

Whatever the scenario, much of that archaic ancestry has been selected out of our gene pool, particularly in the case of genes associated with fertility:

One line of evidence for reduced fertility in male hybrids is that the proportion of archaic ancestry in modern humans is significantly reduced on chromosome X compared to the autosomes. This is suggestive of reduced male fertility as loci contributing to this phenotype are concentrated on chromosome X in hybrids of other species. We confirm an extreme reduction of Neanderthal ancestry on chromosome X (16%–34% of the autosomes depending on the population) and find a quantitatively similar reduction of Denisovan ancestry (21% of the autosomes in Oceanians).

The second line of evidence in support of the hypothesis of reduced fertility in hybrids is that there is a reduction of archaic ancestry in genes that are disproportionately expressed in testes, a known characteristic of male hybrid fertility. (Sankaraman et al., 2016)

This reduction of archaic ancestry was not confined to genes associated with fertility. Petr et al. (2019) conclude that “selection against [Neandertal] introgression was strongest in regulatory and conserved noncoding regions compared with protein-coding sequence (CDS).” It seems, then, that the incompatibilities were not in the genes that manufacture the proteins of human tissue. Rather, they were at a higher level – in genes that regulate growth and development.

Eventually, those incompatibilities were purged from the gene pool through successive generations of purifying selection. How many? Petr et al. (2019) estimate ten or so: “Similar to Harris and Nielsen, we observed abrupt removal of Neandertal alleles from the modern human population during the first ~10 generations after introgression, followed by quick stabilization of Neandertal ancestry levels.” That estimate, however, is based on the assumption of a single introgression some 55,000 years ago. Many academics argue for multiple introgressions that go farther back in time.

But let’s suppose that estimate is right. Should we feel reassured knowing that any harmful effects of outbreeding will be purged from the gene pool in ten generations? Let’s also keep in mind how much stronger natural selection was 55,000 years ago. Today, infant mortality is largely gone and we’re making progress in reducing fetal mortality. The gene pool would now take a lot longer to purge itself. 

At this point, you might ask: “But wouldn’t there be a lot fewer harmful effects from crossings between different human populations?” Yes, of course. But “a lot fewer” doesn’t mean “almost none.”  Indeed, it may be worthwhile to look for signs of purifying selection in introgressions from one human population into another. Has anyone done such a study?

According to Razib, someone has:

In 2014 I specifically asked a postdoctoral fellow in David Reich’s laboratory if they had looked to see if the pattern with Neanderthal introgression could be seen more modestly in the African forager genomes that they had early access to through the Simons Foundation, and was told they had detected no such signature. (Khan, 2023)

There are three problems with this study:

1. It’s inside information. As such, it’s difficult to evaluate, all the more so because the source is unnamed. This is why research findings are normally published in academic journals, where you’re forced to provide the data, explain the methodology, spell out the assumptions, and acknowledge the shortcomings. After publication, you’re also supposed to make yourself available for questions about your paper.

   The problem here is not that an unnamed source may lie (although that does happen). The problem is that an unnamed source may speak in good faith and still be wrong. It’s often only when you put pen to paper that you notice faults in your reasoning. More faults may become evident during peer review and then after publication.

2. The methodology is a lot less precise. The Icelandic and Danish studies directly measured the effects of outbreeding on fertility. This kind of methodology is generally superior to one where the effects are estimated through a model where each calculation is plagued by uncertain assumptions and margins of error.

   A related problem is that it is easier to measure purifying selection after the Neandertal introgression into the human gene pool than, say, after the non-forager introgression into the gene pool of African foragers (i.e., pygmies and Khoisans). The Neandertals died out some 40,000 years ago, so we don’t have to control for subsequent admixture. We do, however, in the case of the pygmies and the Khoisans, who have been in continual contact with various populations up to the present. Yes, there are ways to remove recent admixture from the calculations, but they are subject to error.

3. The genome of the introgressing population is known only approximately. This is not the case with the Neandertals, whose genome has been fully sequenced on the basis of ancient DNA from 50,000 years ago — which is close enough to the presumed introgression at 55,000 BP. In contrast, we have not yet reconstructed the genomes of the various non-Khoisan groups that have introgressed into the Khoisan gene pool over the past 150,000 years. Some of those groups have died out, and not enough of their DNA has been retrieved to reconstruct their genomes. The other groups are still around but have undergone significant evolutionary change in the meantime. Keep in mind that Africans are not “living fossils.” They have pursued their own trajectories of evolutionary change, just like humans elsewhere (Frost, 2023).

In sum, it is difficult to measure the purifying selection that follows an introgression from one human population into another. There is too much noise in the data. We can measure the purifying selection after the Neandertal introgression because the impact on human fertility was much more dramatic, and thus easier to make out against the noise. Also, we have fully sequenced the Neandertal genome. This is not the case with the genomes of most human populations that have existed in the past. The main obstacles here are lack of funding (and interest) and the difficulty of retrieving ancient DNA in tropical environments, where DNA degrades much faster.

Razib’s take on the Icelandic study

In my opinion, the best avenue for future research would be to replicate the Icelandic and Danish findings with another genealogical database, like BALZAC here in French Canada.

Razib does mention the Icelandic study but rejects its findings because the genetic differences between degrees of consanguinity are “minimal.” He again refers to an unnamed source:

I asked a friend who has worked more recently with Icelandic genomic data about that conclusion, and his intuition was the same as mine: the genetic differences between this kinship group and those further out are minimal, so it is likely not a biological dynamic. (Khan, 2023)

How “minimal” are these genetic differences? How many more genes become different, on average, with each successive degree of consanguinity? Unfortunately, we don’t know, since that chart has never been plotted. Please correct me if I’m wrong.

Razib may be referring to the finding that, on average, any two individuals are 99.9% genetically identical – as measured by nucleotide sequences. So what’s left to measure?

Those same sequences, however, show that humans and chimpanzees are 98 to 99% genetically identical. Now, a gene is an assemblage of nucleotide sequences. So each and every human gene could be 2% different from its chimp counterpart, and a 2% difference may significantly alter how each and every human gene functions. Furthermore, the human-chimp difference increases to 4-5% if we allow for differences in deletions and duplications (Suntsova and Buzdin, 2020). Finally, if we disregard nucleotide sequences entirely and look at the products of genes, i.e., proteins, we find that 80% of them differ between humans and chimps:

The chimpanzee is our closest living relative. The morphological differences between the two species are so large that there is no problem in distinguishing between them. However, the nucleotide difference between the two species is surprisingly small. The early genome comparison by DNA hybridization techniques suggested a nucleotide difference of 1–2%. Recently, direct nucleotide sequencing confirmed this estimate. These findings generated the common belief that the human is extremely close to the chimpanzee at the genetic level. However, if one looks at proteins, which are mainly responsible for phenotypic differences, the picture is quite different, and about 80% of proteins are different between the two species. (Glazko et al., 2005)

The point here is that we are only starting to quantify the number of genes that differ between groups – be they species or groups below the species level. To date, no one has measured how this number increases with each degree of consanguinity. The size of each increase may be much larger than what most of us would surmise. Even identical twins differ from each other, on average, at 5.2 genes (Jonsson et al., 2021).

Broader questions

On a final note, I wish to go beyond my disagreement with Razib and address the broader epistemological questions I have encountered with this topic: How much do we really know? Is this line of enquiry legitimate? And why should kinship matter?

How much do we really know? Even if one accepts that fertility starts to decrease beyond marriages between fourth cousins, the decrease might be nothing to worry about. Most couples wish to have only two or three children, and that family size is achievable for the range of consanguinity shown by Icelandic and Danish couples. But whether the higher risk of developmental failure applies after birth or only at the embryonic and fetal stages is something we don’t know. Such risks remain uncertain in both their magnitude and nature.

We need more data on the inverted V-shaped relationship between consanguinity and fertility. Does fertility always peak at marriages with third or fourth cousins? And does it decline at a steady rate further out? Or does it taper off? These questions could be explored in other genealogical databases, such as those for French Canada, Finland or Estonia. Any such study should clearly distinguish between outbreeding depression and inbreeding depression, since the genetic problems in each case are different.

Is this line of enquiry legitimate? Whatever one’s political affiliation, there is a widespread feeling that personal choices in sexual relationships are just that: personal. They are not to be questioned by others, even on an abstract level. This view is widely shared by libertarians on the right and social liberals on the left. It is not incompatible, however, with the view that personal freedom is served not only by a wider range of choices but also by more information. Ultimately, freedom is best served by informed choices.

Why should kinship matter? We cannot understand outbreeding without understanding kinship – a concept that has become alien to many of us. A term like “degree of consanguinity” means nothing unless we know who our ancestors were and who our cousins are. But how many of us do?

At the University of Washington, the anthropologist Pierre van den Berghe would each year ask his new students to name all four of their grandparents. Most couldn’t. His finding has been confirmed by an American study, which found that only 47% of the respondents could correctly name all four of their grandparents (Melore, 2022). I myself would have trouble naming all four. It was only when I came to Quebec that I encountered tons of people who could name not only their grandparents but also their cousins. They valued kinship as something that concerned them personally, and they were just ordinary people.

Without that understanding of kinship, issues relating to consanguinity will be understood only in a vague, abstract sense.

Peter Frost has a PhD in anthropology from Université Laval. His main research interest is the role of sexual selection in shaping highly visible human traits, notably skin color, hair color, and eye color. Other research interests include gene-culture coevolution. Find his Newsletter here.

---

Support Aporia with a $6 monthly subscription and follow us on Twitter.

---

References

Davenport, C.B. and Steggerda, M. (1928). Race Crossing in Jamaica. Washington: Carnegie Institution, Publication no. 395. http://www.velesova-sloboda.info/archiv/pdf/davenport-race-crossing-in-jamaica.pdf    

Frost, P. (2023). The Ghosts of Africa. Africa had its own Neanderthals. Aporia Magazine, September 7.

Frost, P. (2024). The Goldilocks zone between inbreeding and outbreeding. Aporia Magazine, January 10.

Glazko, G., Veeramachaneni, V., Nei, M., & Makalowski, W. (2005). Eighty percent of proteins are different between humans and chimpanzees. Gene 346, 215-219. https://doi.org/10.1016/j.gene.2004.11.003  

Helgason, A., Pálsson, S., Guðbjartsson, D.F., Kristjánsson, þ., and Stefánsson, K. (2008). An association between the kinship and fertility of human couples. Science 319(5864): 813-816. https://doi.org/10.1126/science.1150232

Joffe, M. (2010). What has happened to human fertility? Human Reproduction 25(2): 295-307. https://doi.org/10.1093/humrep/dep390   

Jonsson, H., Magnusdottir, E., Eggertsson, H. P., Stefansson, O. A., Arnadottir, G. A., Eiriksson, O., ... & Stefansson, K. (2021). Differences between germline genomes of monozygotic twins. Nature Genetics 53(1): 27-34. https://doi.org/10.1038/s41588-020-00755-1

Khan, R. (2024). The Genetic Time Machine: Neanderthals, genomics and hybridization priors. Aporia Magazine, January 16.

Labouriau, R., and Amorim, A. (2008). Comment on "An Association Between the Kinship and Fertility of Human Couples." Science 322(5908): 1634. https://doi.org/10.1126/science.1161907

Melore, C. (2022). Family tree stumped: Most Americans can’t name all 4 of their grandparents! Family & Relationships News, April 6, StudyFinds. https://studyfinds.org/family-tree-name-grandparents/

Petr, M., Pääbo, S., Kelso, J., and Vernot, B. (2019). Limits of long-term selection against Neandertal introgression. Proceedings of the National Academy of Sciences 116(5): 1639-1644. https://doi.org/10.1073/pnas.1814338116  

Sankararaman, S., Mallick, S., Patterson, N., Reich, D., Djebali, S., Tilgner, H., Guernec, G., Martin, D., Merkel, A., Knowles, D.G., et al. (2016). The combined landscape of Denisovan and Neanderthal ancestry in present-day humans. Current Biology 26(9): 1241-1247. https://doi.org/10.1016/j.cub.2016.03.037

Suntsova, M.V., and Buzdin, A.A. (2020). Differences between human and chimpanzee genomes and their implications in gene expression, protein functions and biochemical properties of the two species. BMC Genomics 21 (Suppl 7), 535. https://doi.org/10.1186/s12864-020-06962-8