A domain-agnostic approach for characterization of lifelong learning systems

Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to "real world" events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed. This critical gap may be addressed through the development of "Lifelong Learning" systems that are capable of (1) Continuous Learning, (2) Transfer and Adaptation, and (3) Scalability. Unfortunately, efforts to improve these capabilities are typically treated as distinct areas of research that are assessed independently, without regard to the impact of each separate capability on other aspects of the system. We instead propose a holistic approach, using a suite of metrics and an evaluation framework to assess Lifelong Learning in a principled way that is agnostic to specific domains or system techniques. Through five case studies, we show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems. We highlight how the proposed suite of metrics quantifies performance trade-offs present during Lifelong Learning system development - both the widely discussed Stability-Plasticity dilemma and the newly proposed relationship between Sample Efficient and Robust Learning. Further, we make recommendations for the formulation and use of metrics to guide the continuing development of Lifelong Learning systems and assess their progress in the future.

Intelligent Agents for the Optimization of Atomic Layer Deposition

Atomic layer deposition (ALD) is a highly controllable thin film synthesis approach with applications in computing, energy, and separations. The flexibility of ALD means that it can access a massive chemical catalogue; however, this chemical and process diversity results in significant challenges in determining processing parameters that result in stable and uniform film growth with minimal precursor consumption. In situ measurements of the ALD growth per cycle (GPC) can accelerate process development but it still requires expert intuition and time-consuming trial and error to identify acceptable processing parameters. This procedure is made more difficult by the presence of experimental noise in the GPC values and the complexity of ALD surface chemistries. A need exists for efficient optimization approaches capable of autonomously determining processing conditions resulting in optimal ALD film growth. In this work, we present the development of three optimization strategies and compare their performance in optimizing four simulated ALD processes. Furthermore, the effect of noise in the GPC measurements on optimization convergence is studied.

Exploring Neuromodulation for Dynamic Learning

A continual learning system requires the ability to dynamically adapt and generalize to new tasks with access to only a few samples. In the central nervous system, across species, it is observed that continual and dynamic behavior in learning is an active result of a mechanism known as neuromodulation. Therefore, in this work, neuromodulatory plasticity is embedded with dynamic learning architectures as a first step toward realizing power and area efficient few shot learning systems. An inbuilt modulatory unit regulates learning based on the context and internal state of the system. This renders the system an ability to self modify its weights. In one of the proposed architectures, ModNet, a modulatory layer is introduced in a random projection framework. ModNet's learning capabilities are enhanced by integrating attention along with compartmentalized plasticity mechanisms. Moreover, to explore modulatory mechanisms in conjunction with backpropagation in deeper networks, a modulatory trace learning rule is introduced. The proposed learning rule, uses a time dependent trace to modify the synaptic connections as a function of ongoing states and activations. The trace itself is updated via simple plasticity rules thus reducing the demand on resources. The proposed ModNet and learning rules demonstrate the ability to learn from few samples, train quickly, and perform few-shot image classification in a computationally efficient manner. The simple ModNet and the compartmentalized ModNet architecture learn benchmark image classification tasks in just 2 epochs. The network with modulatory trace achieves an average accuracy of 98.8%±1.16 on the omniglot dataset for five-way one-shot image classification task while requiring 20x fewer trainable parameters in comparison to other state of the art models.

Probing the Atomic-Scale Structure of Amorphous Aluminum Oxide Grown by Atomic Layer Deposition

Atomic layer deposition (ALD) is a well-established technique for depositing nanoscale coatings with pristine control of film thickness and composition. The trimethylaluminum (TMA) and water (H₂O) ALD chemistry is inarguably the most widely used and yet to date, we have little information about the atomic-scale structure of the amorphous aluminum oxide (AlO_x) formed by this chemistry. This lack of understanding hinders our ability to establish process-structure-property relationships and ultimately limits technological advancements employing AlO_x made via ALD. In this work, we employ synchrotron high-energy X-ray diffraction (HE-XRD) coupled with pair distribution function (PDF) analysis to characterize the atomic structure of amorphous AlO_x ALD coatings. We combine ex situ and in operando HE-XRD measurements on ALD AlO_x and fit these experimental data using stochastic structural modeling to reveal variations in the Al-O bond length, Al and O coordination environment, and extent of Al vacancies as a function of growth conditions. In particular, the local atomic structure of ALD AlO_x is found to change with the substrate and number of ALD cycles. The observed trends are consistent with the formation of bulk Al₂O₃ surrounded by an O-rich surface layer. We deconvolute these data to reveal atomic-scale structural information for both the bulk and surface phases. Overall, this work demonstrates the usefulness of HE-XRD and PDF analysis in improving our understanding of the structure of amorphous ALD thin films and provides a pathway to evaluate how process changes impact the structure and properties of ALD films.

The chemical physics of sequential infiltration synthesis-A thermodynamic and kinetic perspective

Sequential infiltration synthesis (SIS) is an emerging materials growth method by which inorganic metal oxides are nucleated and grown within the free volume of polymers in association with chemical functional groups in the polymer. SIS enables the growth of novel polymer-inorganic hybrid materials, porous inorganic materials, and spatially templated nanoscale devices of relevance to a host of technological applications. Although SIS borrows from the precursors and equipment of atomic layer deposition (ALD), the chemistry and physics of SIS differ in important ways. These differences arise from the permeable three-dimensional distribution of functional groups in polymers in SIS, which contrast to the typically impermeable two-dimensional distribution of active sites on solid surfaces in ALD. In SIS, metal-organic vapor-phase precursors dissolve and diffuse into polymers and interact with these functional groups through reversible complex formation and/or irreversible chemical reactions. In this perspective, we describe the thermodynamics and kinetics of SIS and attempt to disentangle the tightly coupled physical and chemical processes that underlie this method. We discuss the various experimental, computational, and theoretical efforts that provide insight into SIS mechanisms and identify approaches that may fill out current gaps in knowledge and expand the utilization of SIS.

Characterizing the field of Atomic Layer Deposition: Authors, topics, and collaborations

This paper describes how Atomic Layer Deposition (ALD) has evolved over time using a combination of bibliometric, social network, and text analysis. We examined the rate of knowledge production as well as changes in authors, journals, and collaborators, showing a steady growth of ALD research. The study of the collaboration network of ALD scientists over time points out that the ALD research community is becoming larger and more interconnected, with a largest connected component that spans 90% of the authors in 2015. In addition, the evolution of network centrality measures (degree and betweenness centrality) and author productivity revealed the central figures in ALD over time, including new "stars" appearing in the last decade. Finally, the study of the title words in our dataset is consistent with a shift in focus on research topics towards energy applications and nanotechnology.

An instrument for in situ coherent x-ray studies of metal-organic vapor phase epitaxy of III-nitrides

We describe an instrument that exploits the ongoing revolution in synchrotron sources, optics, and detectors to enable in situ studies of metal-organic vapor phase epitaxy (MOVPE) growth of III-nitride materials using coherent x-ray methods. The system includes high-resolution positioning of the sample and detector including full rotations, an x-ray transparent chamber wall for incident and diffracted beam access over a wide angular range, and minimal thermal sample motion, giving the sub-micron positional stability and reproducibility needed for coherent x-ray studies. The instrument enables surface x-ray photon correlation spectroscopy, microbeam diffraction, and coherent diffraction imaging of atomic-scale surface and film structure and dynamics during growth, to provide fundamental understanding of MOVPE processes.

A modular reactor design for in situ synchrotron x-ray investigation of atomic layer deposition processes

Synchrotron characterization techniques provide some of the most powerful tools for the study of film structure and chemistry. The brilliance and tunability of the Advanced Photon Source allow access to scattering and spectroscopic techniques unavailable with in-house laboratory setups and provide the opportunity to probe various atomic layer deposition (ALD) processes in situ starting at the very first deposition cycle. Here, we present the design and implementation of a portable ALD instrument which possesses a modular reactor scheme that enables simple experimental switchover between various beamlines and characterization techniques. As first examples, we present in situ results for (1) X-ray surface scattering and reflectivity measurements of epitaxial ZnO ALD on sapphire, (2) grazing-incidence small angle scattering of MnO nucleation on silicon, and (3) grazing-incidence X-ray absorption spectroscopy of nucleation-regime Er2O3 ALD on amorphous ALD alumina and single crystalline sapphire.

New Insights into Sequential Infiltration Synthesis

Sequential infiltration synthesis (SIS) is a process derived from ALD in which a polymer is infused with inorganic material using sequential, self-limiting exposures to gaseous precursors. SIS can be used in lithography to harden polymer resists rendering them more robust towards subsequent etching, and this permits deeper and higher-resolution patterning of substrates such as silicon. Herein we describe recent investigations of a model system: Al₂O₃ SIS using trimethyl aluminum (TMA) and H₂O within the diblock copolymer, poly(styrene-block-methyl methacrylate) (PS-b-PMMA). Combining in-situ Fourier transform infrared absorption spectroscopy, quartz-crystal microbalance, and synchrotron grazing incidence small angle X-ray scattering with high resolution scanning transmission electron microscope tomography, we elucidate important details of the SIS process: 1) TMA adsorption in PMMA occurs through a weakly-bound intermediate; 2) the SIS kinetics are diffusion-limited, with desorption 10× slower than adsorption; 3) dynamic structural changes occur during the individual precursor exposures. These findings have important implications for applications such as SIS lithography.

Influence of the angular distribution function of incident particles on the microstructure and anomalous scaling behavior of thin films

The microstructure and the scaling properties of films grown by plasma enhanced chemical vapor deposition are reproduced with a discrete model that takes into account the angular distribution function of the particles and the lateral growth of the films. Both the experimental and simulated surfaces exhibit a granular microstructure and an anomalous scaling behavior characterized by values of the growth exponent beta that vary with the scale of measurement. Depending on the angular distribution function used in the model, values of beta ranging from 0.86 to 0.2 are obtained.

Influence of the excited states on the electron-energy distribution function in low-pressure microwave argon plasmas

In this work the influence of the excited states on the electron-energy distribution function has been determined for an argon microwave discharge at low pressure. A collisional-radiative model of argon has been developed taking into account the most recent experimental and theoretical values of argon-electron-impact excitation cross sections. The model has been solved along with the electron Boltzmann equation in order to study the influence of the inelastic collisions from the argon excited states on the electron-energy distribution function. Results show that under certain conditions the excited states can play an important role in determining the shape of the distribution function and the mean kinetic energy of the electrons, deplecting the high-energy tail due to inelastic processes from the excited states, especially from the 4s excited configuration. It has been found that from the populations of the excited states an excitation temperature can be defined. This excitation temperature, which can be experimentally determined by optical emission spectroscopy, is lower than the electron kinetic temperature obtained from the electron-energy distribution function.