Revisiting Dominance in Population Genetics

GBE Revisiting Dominance in Population Genetics Chenlu Di1, Kirk E. Lohmueller 1,2,3, * 1 Department of Ecology and Evolutionary Biology, University of California, Los Angeles, CA, USA 2 Interdepartmental Program in Bioinformatics, University of California, Los Angeles, CA, USA 3 Department of Human Genetics, David Geffen School of Medicine, Los Angeles, CA, USA Accepted: June 24, 2024 Abstract Dominance refers to the effect of a heterozygous genotype relative to that of the two homozygous genotypes. The degree of dominance of mutations for fitness can have a profound impact on how deleterious and beneficial mutations change in fre quency over time as well as on the patterns of linked neutral genetic variation surrounding such selected alleles. Since dom inance is such a fundamental concept, it has received immense attention throughout the history of population genetics. Early work from Fisher, Wright, and Haldane focused on understanding the conceptual basis for why dominance exists. More re cent work has attempted to test these theories and conceptual models by estimating dominance effects of mutations. However, estimating dominance coefficients has been notoriously challenging and has only been done in a few species in a limited number of studies. In this review, we first describe some of the early theoretical and conceptual models for under standing the mechanisms for the existence of dominance. Second, we discuss several approaches used to estimate domin ance coefficients and summarize estimates of dominance coefficients. We note trends that have been observed across species, types of mutations, and functional categories of genes. By comparing estimates of dominance coefficients for dif ferent types of genes, we test several hypotheses for the existence of dominance. Lastly, we discuss how dominance influ ences the dynamics of beneficial and deleterious mutations in populations and how the degree of dominance of deleterious mutations influences the impact of inbreeding on fitness. Key words: population genetics, dominance, deleterious mutations, inference, natural selection. Significance Dominance refers to the phenotype of the heterozygous genotype relative to that of the two homozygous genotypes. It is a foundational quantity in population genetics because dominance affects how natural selection changes the frequen cies of mutations. Despite intense study over 100 years, the dominance effects of mutations in different organisms re main mostly unknown. Further, the reasons for why some mutations are recessive are not fully understood. In this review, we describe conceptual models for the existence of dominance and discuss some methods that have been used to estimate dominance coefficients. Introduction Mutations may affect the fitness of individuals who carry them. In diploid organisms, mutations can be carried in het erozygous or homozygous genotypes. Dominance describes the effect of a mutant heterozygote relative to the two homozygotes. Some mutations only have a pheno typic effect when homozygous; these are known as reces sive mutations. In contrast, dominant mutations have the © The Author(s) 2024. Published by Oxford University Press on behalf of Society for Molecular Biology and Evolution. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial License (https://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact for reprints and translation rights for reprints. All other permissions can be obtained through our RightsLink service via the Permissions link on the article page on our site—for further information please contact . Genome Biol. Evol. 16(8) https://doi.org/10.1093/gbe/evae147 Advance Access publication 8 August 2024 1 *Corresponding author: Email: . GBE Di and Lohmueller Here, we review population genetic aspects of domin ance. We begin by defining dominance from a population genetic perspective. We then revisit the historical and more recent explanations of dominance and recessiveness. We review methods for estimating dominance coefficients and gather estimates of dominance coefficients from the literature from different methods and species. Next, we dis cuss how these estimates support or conflict with models for dominance. Finally, we review how dominance influ ences the dynamics of selected mutations and the evolution of populations, especially focusing on inbreeding depres sion, introgression, and adaptation. Defining Dominance The concept of dominance is as old as genetics itself. In Mendel’s pea plant experiment, homozygous purple flow ers were crossed with homozygous white flowers, and all the heterozygous offspring had purple flowers. In this case, the allele determining the purple flower phenotype is dominant (Mendel 1865) while the allele determining the white flower phenotype is recessive. Since these early days of genetics, dominance has been described in the con text of Mendelian genetic traits, quantitative genetics, and population genetics. Here, we focus primarily on popula tion genetic aspects of dominance. In population genetics, the dominance coefficient (h) quantifies the fitness of heterozygotes relative to that of Fig. 1. Dominance refers to the fitness of the heterozygous genotype compared with that of the homozygous genotypes. The left y-axis indicates the fitness of different genotypes (x-axis). a) For deleterious mutations, the ancestral homozygote (circle) has the highest fitness and the derived homozygote has the lowest fitness (triangle). The fitness of the ancestral homozygous genotype is 1, the derived (mutant) homozygote is 1−s, and the heterozygote is 1−hs. If the fitness of the heterozygote is the same as that of the ancestral homozygote (top square), h = 0 and the deleterious mutation is recessive. If the fitness of the heterozygote is the average of the ancestral and the derived homozygotes (middle square), h = ½, and the deleterious mutation is additive. If the fitness of the heterozygote is the same as the derived homozygote, h = 1, and the deleterious mutation is dominant (bottom square). b) For beneficial mutations, the an cestral homozygote has the lowest fitness (bottom left circle) and the derived homozygote has the highest fitness (upper right triangle). If the fitness of the heterozygote is the same as the ancestral homozygote (bottom circle), h = 0, the beneficial mutation is recessive (bottom square). If the fitness of the hetero zygote is the average of the derived and ancestral homozygotes, the mutation is additive (middle square). If the fitness of the heterozygote is as high as that of the derived homozygote, the mutation is dominant and h = 1 (upper square). 2 Genome Biol. Evol. 16(8) https://doi.org/10.1093/gbe/evae147 Advance Access publication 8 August 2024 (...truncated)