Behavior Evolves

monochrome photo of flock of flying birds
Photo by Aleksandar Pasaric on

Introduction to Behavior

Behavior is the way an organism acts according to a particular situation or stimulus and it plays a role in survival, health, and physical well-being. Some behaviors are instinctive, while other are conscious driven. Organism behaves as a result from an interaction between genetic processes and the environment. Behavior includes physical actions, emotional reactions, and social interactions. Behavior can change as the organism age and gain experience. Most behaviors are perceived to be learned and culturally transmitted. However, behavior can also be genetically based thus, behavioral evolution can occur by the process of natural selection.

Making the Connection between Genes and Behavior

Linking mutations to a particular change in behavioral traits is one of the major goals in studying behavioral evolution. Changes in sensory systems of the brain or changes in the structures used to carry out behavior may be related to the evolution of behavior (Sheehan, et. Al. 2018). The complexity often demands significant effort to tell differences in behavior between species, let alone the process of mapping the genes. However, careful analysis of the difference in behavior between species is crucial to linking mutations to changes in phenotypes related to behavior. Recent study about two closely related species of deer mouse (Peromyscus maniculatus and Peromyscus polionotus) revealed that one locus is associated with the burrow-digging motivation and provides a possible link between genes and behavior (Metz, et. Al. 2017).

There are four different processes where behavior can evolve. These processes include sensory perception, stimuli response threshold, template for behavioral output, and structural/physiological effectors of the behavior (Figure 1). In behavioral genomics studies, identification of the mechanism of behavioral evolution possesses a common problem. For example, the two closely related species of adult deer mice usually dig burrows when opportunity arise which indicates the differences in sensory perception may not be the cause. An earlier identified quantitative trait locus analysis describes one deer mouse species built an escape tunnel while the other did not, which suggests the two have differential burrowing behavioral templates.

Figure 1. Behavioral evolution can occur between stimuli detection through the action of behaviors. In most cases, behavioral evolution will include selection on one or more steps (Sheehan, et. Al. 2018).

Genetic differences have been reported at the early stages of burrowing behavior between the two mouse species. P. polionotus started burrowing about two weeks earlier in addition to building a larger and more complex burrows. Several juvenile P. polionotus built miniature versions of adult burrows including the escape tunnel feature. These architecturally complete mini burrows continued even in juvenile mice separated from their biological parents. This provides additional evidence that differences in burrow-building may be associated with genes and not by learning.

Researchers observed that adult burrowing behavior like P. polionotus is genetically associated with the early stages of burrowing behavior (Sheehan, et. Al. 2018). This could indicate pleiotropy and that a single gene is controlling both development and adult performance. Most juvenile P. polionotus show strong burrowing behavior at the age of only 17 days. Most behaviors, including begging and courtship, are observed at a specific stage of life as such, we would not expect that the early stages of burrowing and adult burrowing behavior are genetically correlated. The genetic correlation complicates how to understand the selective pressure leading to the burrowing behavior of P. polionotus. Nevertheless, the relationship between the early and adult burrowing behaviors suggests the underlying quantitative trait locus analysis may influence burrow-digging motivation.

Observing Behavioral Evolution with Digital Model

Group behavior is the attitude, feeling, and action of a collection of organisms that can be observed. Examples of collective behavior include herding ungulates, schooling fishes, pods of dolphins, and nest building ants. One study investigated grouping behavior called swarming, the coordinate movement of a species to maintain cohesiveness (Olson, et. Al. 2016). Swarming behavior comes with fitness costs including higher risk of predation and required sharing of food with the group. Nevertheless, swarming behavior may possess benefits such as mating success improvement, increase food-hunting effectiveness, and better problem solving. Researchers proposed that swarming behavior serves as a protection from predators by improving vigilance, reductions of being attacked, and by confusing the predators.

Data scientist Randal S. Olson and his colleagues from Michigan State University used digital model of predator-prey coevolution to investigate the selfish herd hypothesis (Olson, et. Al. 2016). The selfish herd hypothesis explains that an individual moves to a specific position, usually towards the center, within the group when being attacked hoping to increase their chance of survival. The appearance of a cohesive swarm is the result of individuals moving continuously toward a common point in the group. The researchers created a simulation and placed virtual preys. Each virtual prey is programmed so that it can decide how to make movements based on sensory information. Every simulation time step, virtual preys compute information from their sensors and take actions for example, move, or turn. Virtual prey can detect predators and other members of the group. Researchers also utilized digital genome which contains information needed to describe a program for the virtual prey. The digital genome is programmed to undergo evolutionary changes including point mutations, deletions, duplications, and crossovers.

The study simulates a predator that always attack preys on the outer edges of the group (Figure 2). This type of predation shows significant effect on the selfish herd evolution. When preys that are on the outer part of the group are being consistently attacked by predators, the prey group evolves quickly to have a cohesive swarming behavior (Figure 3). The finding gives a better explanation of how natural selection plays a role in behavioral evolution.

Figure 2. An illustration of predation mode (outside attack) which shows predators (X) attacking only the outer edges of the prey group (Olson, et. Al. 2016).
Figure 3. Relationship between the number of generation and mean nearby prey. Generation refers to the simulation time step while the mean nearby prey refers to the number of preys within 30 virtual meters of each other. The graph shows that outside attack predation mode (black squares) allows the prey group to evolve cohesive swarming behavior at a faster rate compared to other predation modes not mentioned in this article. Prey group without predation (star) do not evolve cohesive swarming behavior (Olson, et. Al. 2016).

Genetic Basis of Behavioral Evolution

Parental care is an evolutionary behavior acquired by some organisms involving investment being made to the fitness of offspring. It is a behavior necessary for survival, especially in mammals. Andres Bendensky from Harvard University and his colleagues show that two closely related mice, Oldfield and Deer mouse, have significant and heritable differences in parental behavior (Bendesky, et. Al. 2017). They identified 12 regions of the genome which affects parental care. Of the 12 regions, 8 have sex specific effects, indicating parental care can be different between males and females. Some genomic regions broadly affect parental care, while other regions affect a particular behavior for example, nest building. The nest building behavior is correlated with the expression of vasopressin from the hypothalamus of the two related mouse species. Higher levels of vasopressin expression were observed in less frequent nest builders. Using pharmacology and chemogenetics, researchers show that vasopressin only inhibits the nest-building behavior but not the other parental care behavior.

Animal Domestication in Evolutionary Terms

Phenotypic changes can be the result of random mutations. Dogs may have diverged from the wolf only around 13,500 years ago (Herre 1959). However, genetic variability has amassed at substantial rates disproportionate with random mutations. Consequently, studying genetic variation under domestication can be very intriguing.

One variation feature correlated with domestication is its similar pattern in various domesticated mammals. When two different animals are subjected to domestication, they evolve in the same manner. Both lost the species-specific wild type responses to humans. Their reproductive system became more active and both animals acquired the ability to breed in any season. Conversely, activity related to hormonal regulator of stress and adaptation are reduced in some domesticated animal studies. Research suggests that regulatory changes in genetic activity are correlated with the exceptional level of diversity and its similar pattern in domesticated animals (Herre 1959). Presumably, the regulatory changes were correlated by selecting animals for specific behavior. Selection for tameability and adaptation to human social environment may have also caused changes in regulatory processes. Thus, we can say that any model of domestication can be called forced evolution. Regardless of the mechanisms of genetic activity changes within the domestication process, the findings indicate that these changes can be the product of tameability selection.

Understanding the Loss of Schooling Behavior in Cavefish

Figure 4. Astyanax mexicanus surface fish (A) and blind cavefish (B). Photographs by Yoshiyuki Yamamoto.

Many types of fish exhibit schooling behavior giving them the benefit of reduced predatory encounters and food hunting improvement. Nevertheless, schooling behavior can be disadvantageous for example, during food scarcity when dispersed food hunting strategy is necessary. Fishes that form a school rely on the capability to sense one another. Visual information has been correlated with schooling behavior. The evolution of schooling behavior in many fishes is not well understood. Astyanax mexicanus provides contributions to our understanding in the evolution of schooling behavior. There are two forms of A. mexicanus, sighted surface-dwelling and non-sighted cave-dwelling. The cave form has more taste buds and water sensors, regressed eyes, and reduced melanin pigmentation. Cavefishes also have several altered behaviors which includes reduced aggressiveness and sleeping time, weak responses to intraspecific signals, improved vibrational attractions, altered feeding behavior, and the lack of schooling behavior. Many of these behaviors have been previously studied, but its relationship to genetics is yet to be fully understood.

Researchers observed that the sighted surface-dwelling fishes actively clusters into schools while the cavefishes do not. The evident lack of predators in the caves eases a selective pressure and that cavefishes are not motivated to form in schools. On the other hand, food scarcity in most caves possibly influences cavefishes to establish a scattered distribution.

Researchers determined that the evolutionary regulation of schooling behavior is related to both visual and non-visual information (Kowalko, et. Al 2013). However, the loss of sight in blind cavefishes has the strongest correlation with the reduced schooling behavior. Surface fishes were observed to have a reduced schooling behavior when placed in the dark. Additionally, fish with one of the eyes degraded shoaled farther from one another.

Cavefishes, compared to surface fishes, have been observed to have increased level of brain serotonin and dopamine biosynthesis enzyme. Both elevated pathways are speculated to function in cavefishes by spending more time hunting for food. These pathways were thought to have a pleiotropic effect on schooling behavior. The result was supported by an adjoining experiment where the levels of serotonin and dopamine in surface fish were increased. Data from the adjoining experiment revealed that the schooling tendency in surface fish was reduced.

Overall, the absence of schooling behavior in cavefish population has a genetic basis and that vision is necessary for schooling behavior in surface fish. Differential metabolic pathways also have been observed between cavefishes and surface fishes.


Bendesky, A., Kwon, YM., Lassance, JM. (2017). The genetic basis of parental care evolution in monogamous mice. Nature 544, 434–439.

Emlen D.J., Zimmer, C. (2020). Evolution: Making Sense of Life. W.H. Freeman MacMillan Learning 3rd Edition. Chapter 16. Page 1294. ISBN 978-1-319-23093-7 (epub). LCCN: 2019937273

Herre WK. (1959). Today’s status of domestication research. Naturwissenschaftlich Rundschau; 12:87–

Kowalko, J. E., Rohner, N., Rompani, S. B., Peterson, B. K., Linden, T. A., Yoshizawa, M., Kay, E. H., Weber, J., Hoekstra, H. E., Jeffery, W. R., Borowsky, R., & Tabin, C. J. (2013). Loss of schooling behavior in cavefish through sight-dependent and sight-independent mechanisms. Current biology: CB, 23(19), 1874–1883.

Metz, H. C., Bedford, N. L., Pan, Y. L., & Hoekstra, H. E. (2017). Evolution and Genetics of Precocious Burrowing Behavior in Peromyscus Mice. Current biology: CB, 27(24), 3837–3845.e3.

National Institutes of Health (NIH). Information about the Science of Healthy Behaviors. Accessed: November 28, 2020.

Olson, R. S., Knoester, D. B., & Adami, C. (2016). Evolution of Swarming Behavior Is Shaped by How Predators Attack. Artificial life, 22(3), 299–318.

Sheehan, M. J., Miller, C. H., Vogt, C. C., & Ligon, R. A. (2018). Behavioral Evolution: Can You Dig It? Current biology: CB, 28(1), R19–R21.

Trut, L., Oskina, I., & Kharlamova, A. (2009). Animal evolution during domestication: the domesticated fox as a model. BioEssays: news and reviews in molecular, cellular, and developmental biology, 31(3), 349–360.

Related Links

Research Summary: Order in Spontaneous Behavior

ABSTRACT Brains are usually described as input/output systems: they transform sensory input into motor output. However, the motor output of brains (behavior) is notoriously variable, even under identical sensory conditions. The question of whether this behavioral variability merely reflects residual deviations due to extrinsic random noise in such otherwise deterministic systems or an intrinsic, adaptive … Continue reading

Research Summary: The Transcription Factor Ultraspiracle Influences Honey Bee Social Behavior and Behavior-Related Gene Expression

Abstract Behavior is among the most dynamic animal phenotypes, modulated by a variety of internal and external stimuli. Behavioral differences are associated with large-scale changes in gene expression, but little is known about how these changes are regulated. Here we show how a transcription factor (TF), ultraspiracle….

Research Summary: A Novel Mutant Allele of Pw1/Peg3 Does Not Affect Maternal Behavior or Nursing Behavior

ABSTRACT Parental imprinting is a mammalian-specific form of epigenetic regulation in which one allele of a gene is silenced depending on its parental origin. Parentally imprinted genes have been shown to play a role in growth, metabolism, cancer, and behavior. Although the molecular mechanisms underlying parental imprinting have been largely elucidated, the selective advantage of … Continue reading

Why is changing health-related behaviour so difficult?
Kelly MP, Barker M.

Challenges of a small world analysis for the continuous monitoring of behavior in mice. Bermudez Contreras E, Sutherland RJ, Mohajerani MH, Whishaw IQ.

Leave a Reply