Les Houches Lectures on Community Ecology: From Niche Theory to Statistical Mechanics

Ecosystems are among the most interesting and well-studied examples of self-organized complex systems. Community ecology, the study of how species interact with each other and the environment, has a rich tradition. Over the last few years, there has been a growing theoretical and experimental interest in these problems from the physics and quantitative biology communities. Here, we give an overview of community ecology, highlighting the deep connections between ecology and statistical physics. We start by introducing the two classes of mathematical models that have served as the workhorses of community ecology: Consumer Resource Models (CRM) and the generalized Lotka-Volterra models (GLV). We place a special emphasis on graphical methods and general principles. We then review recent works showing a deep and surprising connection between ecological dynamics and constrained optimization. We then shift our focus by analyzing these same models in "high-dimensions" (i.e. in the limit where the number of species and resources in the ecosystem becomes large) and discuss how such complex ecosystems can be analyzed using methods from the statistical physics of disordered systems such as the cavity method and Random Matrix Theory.

Emergent competition shapes top-down versus bottom-up control in multi-trophic ecosystems

Ecosystems are commonly organized into trophic levels-organisms that occupy the same level in a food chain (e.g., plants, herbivores, carnivores). A fundamental question in theoretical ecology is how the interplay between trophic structure, diversity, and competition shapes the properties of ecosystems. To address this problem, we analyze a generalized Consumer Resource Model with three trophic levels using the zero-temperature cavity method and numerical simulations. We derive the corresponding mean-field cavity equations and show that intra-trophic diversity gives rise to an effective "emergent competition" term between species within a trophic level due to feedbacks mediated by other trophic levels. This emergent competition gives rise to a crossover from a regime of top-down control (populations are limited by predators) to a regime of bottom-up control (populations are limited by primary producers) and is captured by a simple order parameter related to the ratio of surviving species in different trophic levels. We show that our theoretical results agree with empirical observations, suggesting that the theoretical approach outlined here can be used to understand complex ecosystems with multiple trophic levels.

Metabolic similarity and the predictability of microbial community assembly

When microbial communities form, their composition is shaped by selective pressures imposed by the environment. Can we predict which communities will assemble under different environmental conditions? Here, we hypothesize that quantitative similarities in metabolic traits across metabolically similar environments lead to predictable similarities in community composition. To that end, we measured the growth rate and by-product profile of a library of proteobacterial strains in a large number of single nutrient environments. We found that growth rates and secretion profiles were positively correlated across environments when the supplied substrate was metabolically similar. By analyzing hundreds of in-vitro communities experimentally assembled in an array of different synthetic environments, we then show that metabolically similar substrates select for taxonomically similar communities. These findings lead us to propose and then validate a comparative approach for quantitatively predicting the effects of novel substrates on the composition of complex microbial consortia.

Midwave resonant cavity infrared detectors (RCIDs) with suppressed background noise

We report a resonant cavity infrared detector (RCID) with an InAsSb/InAs superlattice absorber with a thickness of only ≈ 100 nm, a 33-period GaAs/Al0.92Ga0.08As distributed Bragg reflector bottom mirror, and a Ge/SiO2/Ge top mirror. At a low bias voltage of 150 mV, the external quantum efficiency (EQE) reaches 58% at the resonance wavelength λres ≈ 4.6 µm, with linewidth δλ = 19-27 nm. The thermal background current for a realistic system scenario with f/4 optic that views a 300 K scene is estimated by integrating the photocurrent generated by background spanning the entire mid-IR spectral band (3-5 µm). The resulting specific detectivity is a factor of 3 lower than for a state-of-the-art broadband HgCdTe device at 300 K, where dark current dominates the noise. However, at 125 K where the suppression of background noise becomes critical, the estimated specific detectivity D* of 5.5 × 1012 cm Hz½/W is more than 3× higher. This occurs despite a non-optimal absorber cut-off that causes the EQE to decrease rapidly with decreasing temperature, e.g., to 33% at 125 K. The present RCID's advantage over the broadband device depends critically on its low EQE at non-resonance wavelengths: ≤ 1% in the range 3.9-5.5 µm. Simulations using NRL MULTIBANDS indicate that impact ionization in the bottom contact and absorber layers dominates the dark current at near ambient temperatures. We expect future design modifications to substantially enhance D* throughout the investigated temperature range of 100-300 K.

Gamma knife capsulotomy for intractable OCD: Neuroimage analysis of lesion size, location, and clinical response

Obsessive-compulsive disorder (OCD) affects 2-3% of the population. One-third of patients are poorly responsive to conventional therapies, and for a subgroup, gamma knife capsulotomy (GKC) is an option. We examined lesion characteristics in patients previously treated with GKC through well-established programs in Providence, RI (Butler Hospital/Rhode Island Hospital/Alpert Medical School of Brown University) and São Paulo, Brazil (University of São Paolo). Lesions were traced on T1 images from 26 patients who had received GKC targeting the ventral half of the anterior limb of the internal capsule (ALIC), and the masks were transformed into MNI space. Voxel-wise lesion-symptom mapping was performed to assess the influence of lesion location on Y-BOCS ratings. General linear models were built to compare the relationship between lesion size/location along different axes of the ALIC and above or below-average change in Y-BOCS ratings. Sixty-nine percent of this sample were full responders (≥35% improvement in OCD). Lesion occurrence anywhere within the targeted region was associated with clinical improvement, but modeling results demonstrated that lesions occurring posteriorly (closer to the anterior commissure) and dorsally (closer to the mid-ALIC) were associated with the greatest Y-BOCS reduction. No association was found between Y-BOCS reduction and overall lesion volume. GKC remains an effective treatment for refractory OCD. Our data suggest that continuing to target the bottom half of the ALIC in the coronal plane is likely to provide the dorsal-ventral height required to achieve optimal outcomes, as it will cover the white matter pathways relevant to change. Further analysis of individual variability will be essential for improving targeting and clinical outcomes, and potentially further reducing the lesion size necessary for beneficial outcomes.

Emergent competition shapes the ecological properties of multi-trophic ecosystemss

Ecosystems are commonly organized into trophic levels - organisms that occupy the same level in a food chain (e.g., plants, herbivores, carnivores). A fundamental question in theoretical ecology is how the interplay between trophic structure, diversity, and competition shapes the properties of ecosystems. To address this problem, we analyze a generalized Consumer Resource Model with three trophic levels using the zero-temperature cavity method and numerical simulations. We find that intra-trophic diversity gives rise to "emergent competition" between species within a trophic level due to feedbacks mediated by other trophic levels. This emergent competition gives rise to a crossover from a regime of top-down control (populations are limited by predators) to a regime of bottom-up control (populations are limited by primary producers) and is captured by a simple order parameter related to the ratio of surviving species in different trophic levels. We show that our theoretical results agree with empirical observations, suggesting that the theoretical approach outlined here can be used to understand complex ecosystems with multiple trophic levels.

Diverse communities behave like typical random ecosystems

In 1972, Robert May triggered a worldwide research program studying ecological communities using random matrix theory. Yet, it remains unclear if and when we can treat real communities as random ecosystems. Here, we draw on recent progress in random matrix theory and statistical physics to extend May's approach to generalized consumer-resource models. We show that in diverse ecosystems adding even modest amounts of noise to consumer preferences results in a transition to "typicality," where macroscopic ecological properties of communities are indistinguishable from those of random ecosystems, even when resource preferences have prominent designed structures. We test these ideas using numerical simulations on a wide variety of ecological models. Our work offers an explanation for the success of random consumer resource models in reproducing experimentally observed ecological patterns in microbial communities and highlights the difficulty of scaling up bottom-up approaches in synthetic ecology to diverse communities.

Niche Theory for Mutualism: A Graphical Approach to Plant-Pollinator Network Dynamics

AbstractContemporary niche theory is a useful framework for understanding how organisms interact with each other and with their shared environment. Its graphical representation, popularized by Tilman's resource ratio hypothesis, facilitates analysis of the equilibrium structure of complex dynamical models, including species coexistence. This theory has been applied primarily to resource competition since its early beginnings. Here, we integrate mutualism into niche theory by expanding Tilman's graphical representation to the analysis of consumer-resource dynamics of plant-pollinator networks. We graphically explain the qualitative phenomena previously found by numerical simulations, including the effects on community dynamics of nestedness, adaptive foraging, and pollinator invasions. Our graphical approach promotes the unification of niche and network theories and deepens the synthesis of different types of interactions within a consumer-resource framework.

Machine Learning as Ecology

Machine learning methods have had spectacular success on numerous problems. Here we show that a prominent class of learning algorithms - including Support Vector Machines (SVMs) - have a natural interpretation in terms of ecological dynamics. We use these ideas to design new online SVM algorithms that exploit ecological invasions, and benchmark performance using the MNIST dataset. Our work provides a new ecological lens through which we can view statistical learning and opens the possibility of designing ecosystems for machine learning.

The Nanomechanical Bit

The applicability of nanomechanical devices for computational approaches is reviewed. The focus is on the representation and processing of information based on nanomechanical bits. Several device concepts are discussed ranging from nano-electromechanical systems in silicon to circuits based on carbon nano-tube switches, combinations of nanomechanical resonators and traditional transistors, and integration into a computing architecture. The strengths of mechanical systems include their scalability, robustness to external electrical shocks, and their low-energy consumption. Hence, they may lead the way to new forms of ultradense memory and alternative routes of computing. In conjunction with quantum mechanical single electron circuits, nano-electromechanical systems may also have potential for quantum computational circuits.

Effect of Resource Dynamics on Species Packing in Diverse Ecosystems

The competitive exclusion principle asserts that coexisting species must occupy distinct ecological niches (i.e., the number of surviving species cannot exceed the number of resources). An open question is to understand if and how different resource dynamics affect this bound. Here, we analyze a generalized consumer resource model with externally supplied resources and show that-in contrast to self-renewing resources-species can occupy only half of all available environmental niches. This motivates us to construct a new schema for classifying ecosystems based on species packing properties.

The Minimum Environmental Perturbation Principle: A New Perspective on Niche Theory

AbstractFifty years ago, Robert MacArthur showed that stable equilibria optimize quadratic functions of the population sizes in several important ecological models. Here, we generalize this finding to a broader class of systems within the framework of contemporary niche theory and precisely state the conditions under which an optimization principle (not necessarily quadratic) can be obtained. We show that conducting the optimization in the space of environmental states instead of population sizes leads to a universal and transparent physical interpretation of the objective function. Specifically, the equilibrium state minimizes the perturbation of the environment induced by the presence of the competing species, subject to the constraint that no species has a positive net growth rate. We use this "minimum environmental perturbation principle" to make new predictions for evolution and community assembly, where the minimum perturbation increases monotonically under invasion by new species. We also describe a simple experimental setting where the conditions of validity for this optimization principle have been empirically tested.

The thermodynamic uncertainty relation in biochemical oscillations

Living systems regulate many aspects of their behaviour through periodic oscillations of molecular concentrations, which function as 'biochemical clocks.' The chemical reactions that drive these clocks are intrinsically stochastic at the molecular level, so that the duration of a full oscillation cycle is subject to random fluctuations. Their success in carrying out their biological function is thought to depend on the degree to which these fluctuations in the cycle period can be suppressed. Biochemical oscillators also require a constant supply of free energy in order to break detailed balance and maintain their cyclic dynamics. For a given free energy budget, the recently discovered 'thermodynamic uncertainty relation' yields the magnitude of period fluctuations in the most precise conceivable free-running clock. In this paper, we show that computational models of real biochemical clocks severely underperform this optimum, with fluctuations several orders of magnitude larger than the theoretical minimum. We argue that this suboptimal performance is due to the small number of internal states per molecule in these models, combined with the high level of thermodynamic force required to maintain the system in the oscillatory phase. We introduce a new model with a tunable number of internal states per molecule and confirm that it approaches the optimal precision as this number increases.

Data-driven modeling reveals a universal dynamic underlying the COVID-19 pandemic under social distancing

We show that the COVID-19 pandemic under social distancing exhibits universal dynamics. The cumulative numbers of both infections and deaths quickly cross over from exponential growth at early times to a longer period of power law growth, before eventually slowing. In agreement with a recent statistical forecasting model by the IHME, we show that this dynamics is well described by the erf function. Using this functional form, we perform a data collapse across countries and US states with very different population characteristics and social distancing policies, confirming the universal behavior of the COVID-19 outbreak. We show that the predictive power of statistical models is limited until a few days before curves flatten, forecast deaths and infections assuming current policies continue and compare our predictions to the IHME models. We present simulations showing this universal dynamics is consistent with disease transmission on scale-free networks and random networks with non-Markovian transmission dynamics.

The Community Simulator: A Python package for microbial ecology

Natural microbial communities contain hundreds to thousands of interacting species. For this reason, computational simulations are playing an increasingly important role in microbial ecology. In this manuscript, we present a new open-source, freely available Python package called Community Simulator for simulating microbial population dynamics in a reproducible, transparent and scalable way. The Community Simulator includes five major elements: tools for preparing the initial states and environmental conditions for a set of samples, automatic generation of dynamical equations based on a dictionary of modeling assumptions, random parameter sampling with tunable levels of metabolic and taxonomic structure, parallel integration of the dynamical equations, and support for metacommunity dynamics with migration between samples. To significantly speed up simulations using Community Simulator, our Python package implements a new Expectation-Maximization (EM) algorithm for finding equilibrium states of community dynamics that exploits a recently discovered duality between ecological dynamics and convex optimization. We present data showing that this EM algorithm improves performance by between one and two orders compared to direct numerical integration of the corresponding ordinary differential equations. We conclude by listing several recent applications of the Community Simulator to problems in microbial ecology, and discussing possible extensions of the package for directly analyzing microbiome compositional data.

A minimal model for microbial biodiversity can reproduce experimentally observed ecological patterns

Surveys of microbial biodiversity such as the Earth Microbiome Project (EMP) and the Human Microbiome Project (HMP) have revealed robust ecological patterns across different environments. A major goal in ecology is to leverage these patterns to identify the ecological processes shaping microbial ecosystems. One promising approach is to use minimal models that can relate mechanistic assumptions at the microbe scale to community-level patterns. Here, we demonstrate the utility of this approach by showing that the Microbial Consumer Resource Model (MiCRM) - a minimal model for microbial communities with resource competition, metabolic crossfeeding and stochastic colonization - can qualitatively reproduce patterns found in survey data including compositional gradients, dissimilarity/overlap correlations, richness/harshness correlations, and nestedness of community composition. By using the MiCRM to generate synthetic data with different environmental and taxonomical structure, we show that large scale patterns in the EMP can be reproduced by considering the energetic cost of surviving in harsh environments and HMP patterns may reflect the importance of environmental filtering in shaping competition. We also show that recently discovered dissimilarity-overlap correlations in the HMP likely arise from communities that share similar environments rather than reflecting universal dynamics. We identify ecologically meaningful changes in parameters that alter or destroy each one of these patterns, suggesting new mechanistic hypotheses for further investigation. These findings highlight the promise of minimal models for microbial ecology.

The in vivo genetic program of murine primordial lung epithelial progenitors

Multipotent Nkx2-1-positive lung epithelial primordial progenitors of the foregut endoderm are thought to be the developmental precursors to all adult lung epithelial lineages. However, little is known about the global transcriptomic programs or gene networks that regulate these gateway progenitors in vivo. Here we use bulk RNA-sequencing to describe the unique genetic program of in vivo murine lung primordial progenitors and computationally identify signaling pathways, such as Wnt and Tgf-β superfamily pathways, that are involved in their cell-fate determination from pre-specified embryonic foregut. We integrate this information in computational models to generate in vitro engineered lung primordial progenitors from mouse pluripotent stem cells, improving the fidelity of the resulting cells through unbiased, easy-to-interpret similarity scores and modulation of cell culture conditions, including substratum elastic modulus and extracellular matrix composition. The methodology proposed here can have wide applicability to the in vitro derivation of bona fide tissue progenitors of all germ layers.

Constrained optimization as ecological dynamics with applications to random quadratic programming in high dimensions

Quadratic programming (QP) is a common and important constrained optimization problem. Here, we derive a surprising duality between constrained optimization with inequality constraints, of which QP is a special case, and consumer resource models describing ecological dynamics. Combining this duality with a recent "cavity solution," we analyze high-dimensional, random QP where the optimization function and constraints are drawn randomly. Our theory shows remarkable agreement with numerics and points to a deep connection between optimization, dynamical systems, and ecology.

Active regeneration unites high- and low-temperature features in cooperative self-assembly

Cytoskeletal filaments are capable of self-assembly in the absence of externally supplied chemical energy, but the rapid turnover rates essential for their biological function require a constant flux of adenosine triphosphate (ATP) or guanosine triphosphate (GTP) hydrolysis. The same is true for two-dimensional protein assemblies employed in the formation of vesicles from cellular membranes, which rely on ATP-hydrolyzing enzymes to rapidly disassemble upon completion of the process. Recent observations suggest that the nucleolus, p granules, and other three-dimensional membraneless organelles may also demand dissipation of chemical energy to maintain their fluidity. Cooperative binding plays a crucial role in the dynamics of these higher-dimensional structures, but is absent from classic models of one-dimensional cytoskeletal assembly. In this paper, we present a thermodynamically consistent model of active regeneration with cooperative assembly, and compute the maximum turnover rate and minimum disassembly time as a function of the chemical driving force and the binding energy. We find that these driven structures resemble different equilibrium states above and below the nucleation barrier. In particular, we show that the maximal acceleration under large binding energies unites infinite-temperature local fluctuations with low-temperature nucleation kinetics.

Available energy fluxes drive a transition in the diversity, stability, and functional structure of microbial communities

A fundamental goal of microbial ecology is to understand what determines the diversity, stability, and structure of microbial ecosystems. The microbial context poses special conceptual challenges because of the strong mutual influences between the microbes and their chemical environment through the consumption and production of metabolites. By analyzing a generalized consumer resource model that explicitly includes cross-feeding, stochastic colonization, and thermodynamics, we show that complex microbial communities generically exhibit a transition as a function of available energy fluxes from a “resource-limited” regime where community structure and stability is shaped by energetic and metabolic considerations to a diverse regime where the dominant force shaping microbial communities is the overlap between species’ consumption preferences. These two regimes have distinct species abundance patterns, different functional profiles, and respond differently to environmental perturbations. Our model reproduces large-scale ecological patterns observed across multiple experimental settings such as nestedness and differential beta diversity patterns along energy gradients. We discuss the experimental implications of our results and possible connections with disorder-induced phase transitions in statistical physics.

The diversity, stability and functional structure of microbial communities have dramatic effects on the health of humans and of ecosystems. The complexity of these communities has so far precluded the development of a general predictive model that would capture the dependence of these features on environmental conditions. We confronted this challenge by constructing a flexible simulation framework, and randomly sampling parameters under a variety of modeling assumptions to identify generic patterns. We found two qualitatively distinct regimes of community structure, which reproduce observed patterns of biodiversity, and make new predictions about stability and function.

Limits of predictions in thermodynamic systems: a review

The past twenty years have seen a resurgence of interest in nonequilibrium thermodynamics, thanks to advances in the theory of stochastic processes and in their thermodynamic interpretation. Fluctuation theorems provide fundamental constraints on the dynamics of systems arbitrarily far from thermal equilibrium. Thermodynamic uncertainty relations bound the dissipative cost of precision in a wide variety of processes. Concepts of excess work and excess heat provide the basis for a complete thermodynamics of nonequilibrium steady states, including generalized Clausius relations and thermodynamic potentials. But these general results carry their own limitations: fluctuation theorems involve exponential averages that can depend sensitively on unobservably rare trajectories; steady-state thermodynamics makes use of a dual dynamics that lacks any direct physical interpretation. This review aims to present these central results of contemporary nonequilibrium thermodynamics in such a way that the power of each claim for making physical predictions can be clearly assessed, using examples from current topics in soft matter and biophysics.

Dynamics of phosphoinositide conversion in clathrin-mediated endocytic traffic

Vesicular carriers transport proteins and lipids from one organelle to another, recognizing specific identifiers for the donor and acceptor membranes. Two important identifiers are phosphoinositides and GTP-bound GTPases, which provide well-defined but mutable labels. Phosphatidylinositol and its phosphorylated derivatives are present on the cytosolic faces of most cellular membranes. Reversible phosphorylation of its headgroup produces seven distinct phosphoinositides. In endocytic traffic, phosphatidylinositol-4,5-biphosphate marks the plasma membrane, and phosphatidylinositol-3-phosphate and phosphatidylinositol-4-phosphate mark distinct endosomal compartments. It is unknown what sequence of changes in lipid content confers on the vesicles their distinct identity at each intermediate step. Here we describe 'coincidence-detecting' sensors that selectively report the phosphoinositide composition of clathrin-associated structures, and the use of these sensors to follow the dynamics of phosphoinositide conversion during endocytosis. The membrane of an assembling coated pit, in equilibrium with the surrounding plasma membrane, contains phosphatidylinositol-4,5-biphosphate and a smaller amount of phosphatidylinositol-4-phosphate. Closure of the vesicle interrupts free exchange with the plasma membrane. A substantial burst of phosphatidylinositol-4-phosphate immediately after budding coincides with a burst of phosphatidylinositol-3-phosphate, distinct from any later encounter with the phosphatidylinositol-3-phosphate pool in early endosomes; phosphatidylinositol-3,4-biphosphate and the GTPase Rab5 then appear and remain as the uncoating vesicles mature into Rab5-positive endocytic intermediates. Our observations show that a cascade of molecular conversions, made possible by the separation of a vesicle from its parent membrane, can label membrane-traffic intermediates and determine their destinations.

Far-from-equilibrium distribution from near-steady-state work fluctuations

A long-standing goal of nonequilibrium statistical mechanics has been to extend the conceptual power of the Boltzmann distribution to driven systems. We report some new progress towards this goal. Instead of writing the nonequilibrium steady-state distribution in terms of perturbations around thermal equilibrium, we start from the linearized driven dynamics of observables about their stable fixed point, and expand in the strength of the nonlinearities encountered during typical fluctuations away from the fixed point. The first terms in this expansion retain the simplicity of known expansions about equilibrium, but can correctly describe the statistics of a certain class of systems even under strong driving. We illustrate this approach by comparison with a numerical simulation of a sheared Brownian colloid, where we find that the first two terms in our expansion are sufficient to account for the shear thinning behavior at high shear rates.

Cusp kernels for velocity-changing collisions

We introduce an analytical kernel, the "cusp" kernel, to model the effects of velocity-changing collisions on optically pumped atoms in low-pressure buffer gases. Like the widely used Keilson-Storer kernel [J. Keilson and J. E. Storer, Q. Appl. Math. 10, 243 (1952)], cusp kernels are characterized by a single parameter and preserve a Maxwellian velocity distribution. Cusp kernels and their superpositions are more useful than Keilson-Storer kernels, because they are more similar to real kernels inferred from measurements or theory and are easier to invert to find steady-state velocity distributions.

Temperature-insensitive laser frequency locking near absorption lines

Combined magnetically induced circular dichroism and Faraday rotation of an atomic vapor are used to develop a variant of the dichroic atomic vapor laser lock that eliminates lock sensitivity to temperature fluctuations of the cell. Operating conditions that eliminate first-order sensitivity to temperature fluctuations can be determined by low-frequency temperature modulation. This temperature-insensitive gyrotropic laser lock can be accurately understood with a simple model, that is in excellent agreement with observations in potassium vapor at laser frequencies in a 2 GHz range about the 770.1 nm absorption line. The methods can be readily adapted for other absorption lines.

A time-limited behavioral group for treatment of obsessive-compulsive disorder

In vivo exposure with response prevention is an effective treatment for obsessive-compulsive disorder (OCD) either alone or combined with pharmacotherapy. Widespread application of this technique has been limited by lack of trained therapists and the expense of intensive individual behavioral therapy. This report describes a time-limited 10-session behavioral therapy group for OCD whose key elements are exposure, response prevention, therapist and participant modeling, and cognitive restructuring. In a naturalistic open trial of 90 patients meeting DSM-III-R criteria for OCD who completed the 10-session group, self-administered Yale-Brown Obsessive-Compulsive Scale scores (mean +/- SD) were 21.8 +/- 5.6 at baseline and 16.6 +/- 6.4 after the 10-week treatment, a significant decrease. A descriptive analysis of the therapeutic elements of the group and its advantages over individual behavioral treatment are presented.