Weighted p -Bits for FPGA Implementation of Probabilistic Circuits

Translating weighted probabilistic bits to synthetic genetic circuits

Synthetic genetic circuits in plants could be the next technological horizon in plant breeding, showcasing potential for precise patterned control over expression. Nevertheless, uncertainty in metabolic environments prevents robust scaling of traditional genetic circuits for agricultural use, and studies show that a deterministic system is at odds with biological randomness. We analyze the necessary requirements for assuring Boolean logic gate sequences can function in unpredictable intracellular conditions, followed by interpreted pathways by which a mathematical representation of probabilistic circuits can be translated to biological implementation. This pathway is utilized through translation of a probabilistic circuit model presented by Pervaiz that works through a series of bits; each composed of a weighted matrix that reads inputs from the environment and a random number generator that takes the matrix as bias and outputs a positive or negative signal. The weighted matrix can be biologically represented as the regulatory elements that affect transcription near promotors, allowing for an electrical bit to biological bit translation that can be refined through tuning using invertible logic prediction of the input to output relationship of a genetic response. Failsafe mechanisms should be introduced, possibly through the use of self-eliminating CRISPR-Cas9, dosage compensation, or cybernetic modeling (where CRISPR is clustered regularly interspaced short palindromic repeats and Cas9 is clustered regularly interspaced short palindromic repeat-associated protein 9). These safety measures are needed for all biological circuits, and their implementation is needed alongside work with this specific model. With applied responses to external factors, these circuits could allow fine-tuning of organism adaptation to stress while providing a framework for faster complex expression design in the field.

CMOS Single-Photon Avalanche Diode Circuits for Probabilistic Computing

Intrinsically random hardware devices are increasingly attracting attention for their potential use in probabilistic computing architectures. One such device is the single-photon avalanche diode (SPAD) and an associated functional unit, the variable-rate SPAD circuit (VRSC), recently proposed by us as a source of randomness for sampling and annealing Ising and Potts models. This work develops a more advanced understanding of these VRSCs by introducing several VRSC design options and studying their tradeoffs as implemented in a 65-nm CMOS process. Each VRSC is composed of a SPAD and a processing circuit. Combinations of three different SPAD designs and three different types of processing circuits were evaluated on several metrics such as area, speed, and variability. Measured results from the SPAD design space show that even extremely small SPADs are suitable for probabilistic computing purposes, and that high dark count rates are not detrimental either, so SPADs for probabilistic computing are actually easier to integrate in standard CMOS processes. Results from the circuit design space show that the time-to-analog-based designs introduced in this work can produce highly exponential and analytical transfer functions, but that the less analytically tractable output of the previously proposed filter-based designs can achieve less variability in a smaller footprint. Probabilistic bits (P-bits) composed of the fabricated VRSCs achieve bit flip rates of 50 MHz and allow at least one order of magnitude of control over their simulated annealing temperature.

Direct design of ground-state probabilistic logic using many-body interactions for probabilistic computing

In this work, an innovative design model aimed at enhancing the efficacy of ground-state probabilistic logic with a binary energy landscape (GSPL-BEL) is presented. This model enables the direct conversion of conventional CMOS-based logic circuits into corresponding probabilistic graphical representations based on a given truth table. Compared to the conventional approach of solving the configuration of Ising model-basic probabilistic gates through linear programming, our model directly provides configuration parameters with embedded many-body interactions. For larger-scale probabilistic logic circuits, the GSPL-BEL model can fully utilize the dimensions of many-body interactions, achieving minimal node overhead while ensuring the simplest binary energy landscape and circumventing additional logic synthesis steps. To validate its effectiveness, hardware implementations of probabilistic logic gates were conducted. Probabilistic bits were introduced as Ising cells, and cascaded conventional XNOR gates along with passive resistor networks were precisely designed to realize many-body interactions. HSPICE circuit simulation results demonstrate that the probabilistic logic circuits designed based on this model can successfully operate in free, forward, and reverse modes, exhibiting the simplest binary probability distributions. For a 2-bit × 2-bit integer factorizer involving many-body interactions, compared to the logic synthesis approach, the GSPL-BEL model significantly reduces the number of consumed nodes, the solution space (in the free-run mode), and the number of energy levels from 12, 4096, and 9-8, 256, and 2, respectively. Our findings demonstrate the significant potential of the GSPL-BEL model in optimizing the structure and performance of probabilistic logic circuits, offering a new robust tool for the design and implementation of future probabilistic computing systems.

Experimental demonstration of an on-chip p-bit core based on stochastic magnetic tunnel junctions and 2D MoS2 transistors

Probabilistic computing is a computing scheme that offers a more efficient approach than conventional complementary metal-oxide-semiconductor (CMOS)-based logic in a variety of applications ranging from optimization to Bayesian inference, and invertible Boolean logic. The probabilistic bit (or p-bit, the base unit of probabilistic computing) is a naturally fluctuating entity that requires tunable stochasticity; by coupling low-barrier stochastic magnetic tunnel junctions (MTJs) with a transistor circuit, a compact implementation is achieved. In this work, by combining stochastic MTJs with 2D-MoS2 field-effect transistors (FETs), we demonstrate an on-chip realization of a p-bit building block displaying voltage-controllable stochasticity. Supported by circuit simulations, we analyze the three transistor-one magnetic tunnel junction (3T-1MTJ) p-bit design, evaluating how the characteristics of each component influence the overall p-bit output. While the current approach has not reached the level of maturity required to compete with CMOS-compatible MTJ technology, the design rules presented in this work are valuable for future experimental implementations of scaled on-chip p-bit networks with reduced footprint.

Energy-efficient superparamagnetic Ising machine and its application to traveling salesman problems

The growth of artificial intelligence leads to a computational burden in solving non-deterministic polynomial-time (NP)-hard problems. The Ising computer, which aims to solve NP-hard problems faces challenges such as high power consumption and limited scalability. Here, we experimentally present an Ising annealing computer based on 80 superparamagnetic tunnel junctions (SMTJs) with all-to-all connections, which solves a 70-city traveling salesman problem (TSP, 4761-node Ising problem). By taking advantage of the intrinsic randomness of SMTJs, implementing global annealing scheme, and using efficient algorithm, our SMTJ-based Ising annealer outperforms other Ising schemes in terms of power consumption and energy efficiency. Additionally, our approach provides a promising way to solve complex problems with limited hardware resources. Moreover, we propose a cross-bar array architecture for scalable integration using conventional magnetic random-access memories. Our results demonstrate that the SMTJ-based Ising computer with high energy efficiency, speed, and scalability is a strong candidate for future unconventional computing schemes.

Enhanced convergence in p-bit based simulated annealing with partial deactivation for large-scale combinatorial optimization problems

This article critically investigates the limitations of the simulated annealing algorithm using probabilistic bits (pSA) in solving large-scale combinatorial optimization problems. The study begins with an in-depth analysis of the pSA process, focusing on the issues resulting from unexpected oscillations among p-bits. These oscillations hinder the energy reduction of the Ising model and thus obstruct the successful execution of pSA in complex tasks. Through detailed simulations, we unravel the root cause of this energy stagnation, identifying the feedback mechanism inherent to the pSA operation as the primary contributor to these disruptive oscillations. To address this challenge, we propose two novel algorithms, time average pSA (TApSA) and stalled pSA (SpSA). These algorithms are designed based on partial deactivation of p-bits and are thoroughly tested using Python simulations on maximum cut benchmarks that are typical combinatorial optimization problems. On the 16 benchmarks from 800 to 5000 nodes, the proposed methods improve the normalized cut value from 0.8 to 98.4% on average in comparison with the conventional pSA.

Fast-Converging Simulated Annealing for Ising Models Based on Integral Stochastic Computing

Probabilistic bits (p-bits) have recently been presented as a spin (basic computing element) for the simulated annealing (SA) of Ising models. In this brief, we introduce fast-converging SA based on p-bits designed using integral stochastic computing. The stochastic implementation approximates a p-bit function, which can search for a solution to a combinatorial optimization problem at lower energy than conventional p-bits. Searching around the global minimum energy can increase the probability of finding a solution. The proposed stochastic computing-based SA method is compared with conventional SA and quantum annealing (QA) with a D-Wave Two quantum annealer on the traveling salesman, maximum cut (MAX-CUT), and graph isomorphism (GI) problems. The proposed method achieves a convergence speed a few orders of magnitude faster while dealing with an order of magnitude larger number of spins than the other methods.

A quantum-inspired probabilistic prime factorization based on virtually connected Boltzmann machine and probabilistic annealing

Probabilistic computing has been introduced to operate functional networks using a probabilistic bit (p-bit), broadening the computational abilities in non-deterministic polynomial searching operations. However, previous developments have focused on emulating the operation of quantum computers similarly, implementing every p-bit with large weight-sum matrix multiplication blocks and requiring tens of times more p-bits than semiprime bits. In addition, operations based on a conventional simulated annealing scheme required a large number of sampling operations, which deteriorated the performance of the Ising machines. Here we introduce a prime factorization machine with a virtually connected Boltzmann machine and probabilistic annealing method, which are designed to reduce the hardware complexity and number of sampling operations. From 10-bit to 64-bit prime factorizations were performed, and the machine offers up to 1.2 × 108 times improvement in the number of sampling operations compared with previous factorization machines, with a 22-fold smaller hardware resource.

Biasing the quantum vacuum to control macroscopic probability distributions

Quantum field theory suggests that electromagnetic fields naturally fluctuate, and these fluctuations can be harnessed as a source of perfect randomness. Many potential applications of randomness rely on controllable probability distributions. We show that vacuum-level bias fields injected into multistable optical systems enable a controllable source of quantum randomness, and we demonstrated this concept in an optical parametric oscillator (OPO). By injecting bias pulses with less than one photon on average, we controlled the probabilities of the two possible OPO output states. The potential of our approach for sensing sub-photon-level fields was demonstrated by reconstructing the temporal shape of fields below the single-photon level. Our results provide a platform to study quantum dynamics in nonlinear driven-dissipative systems and point toward applications in probabilistic computing and weak field sensing.

Probabilistic Circuit Implementation Based on P-Bits Using the Intrinsic Random Property of RRAM and P-Bit Multiplexing Strategy

Probabilistic computing is an emerging computational paradigm that uses probabilistic circuits to efficiently solve optimization problems such as invertible logic, where traditional digital computations are difficult to solve. This paper proposes a true random number generator (TRNG) based on resistive random-access memory (RRAM), which is combined with an activation function implemented by a piecewise linear function to form a standard p-bit cell, one of the most important parts of a p-circuit. A p-bit multiplexing strategy is also applied to reduce the number of p-bits and improve resource utilization. To verify the superiority of the proposed probabilistic circuit, we implement the invertible p-circuit on a field-programmable gate array (FPGA), including AND gates, full adders, multi-bit adders, and multipliers. The results of the FPGA implementation show that our approach can significantly save the consumption of hardware resources.

Thermodynamic State Machine Network

We describe a model system—a thermodynamic state machine network—comprising a network of probabilistic, stateful automata that equilibrate according to Boltzmann statistics, exchange codes over unweighted bi-directional edges, update a state transition memory to learn transitions between network ground states, and minimize an action associated with fluctuation trajectories. The model is grounded in four postulates concerning self-organizing, open thermodynamic systems—transport-driven self-organization, scale-integration, input-functionalization, and active equilibration. After sufficient exposure to periodically changing inputs, a diffusive-to-mechanistic phase transition emerges in the network dynamics. The evolved networks show spatial and temporal structures that look much like spiking neural networks, although no such structures were incorporated into the model. Our main contribution is the articulation of the postulates, the development of a thermodynamically motivated methodology addressing them, and the resulting phase transition. As with other machine learning methods, the model is limited by its scalability, generality, and temporality. We use limitations to motivate the development of thermodynamic computers—engineered, thermodynamically self-organizing systems—and comment on efforts to realize them in the context of this work. We offer a different philosophical perspective, thermodynamicalism, addressing the limitations of the model and machine learning in general.

Hardware Design for Autonomous Bayesian Networks

Directed acyclic graphs or Bayesian networks that are popular in many AI-related sectors for probabilistic inference and causal reasoning can be mapped to probabilistic circuits built out of probabilistic bits (p-bits), analogous to binary stochastic neurons of stochastic artificial neural networks. In order to satisfy standard statistical results, individual p-bits not only need to be updated sequentially but also in order from the parent to the child nodes, necessitating the use of sequencers in software implementations. In this article, we first use SPICE simulations to show that an autonomous hardware Bayesian network can operate correctly without any clocks or sequencers, but only if the individual p-bits are appropriately designed. We then present a simple behavioral model of the autonomous hardware illustrating the essential characteristics needed for correct sequencer-free operation. This model is also benchmarked against SPICE simulations and can be used to simulate large-scale networks. Our results could be useful in the design of hardware accelerators that use energy-efficient building blocks suited for low-level implementations of Bayesian networks. The autonomous massively parallel operation of our proposed stochastic hardware has biological relevance since neural dynamics in brain is also stochastic and autonomous by nature.

Implementation of an efficient magnetic tunnel junction-based stochastic neural network with application to iris data classification

Stochastic neuromorphic computation (SNC) has the potential to enable a low power, error tolerant and scalable computing platform in comparison to its deterministic counterparts. However, the hardware implementation of complementary metal oxide semiconductor (CMOS)-based stochastic circuits involves conversion blocks that cost more than the actual processing circuits. The realization of the activation function for SNCs also requires a complicated circuit that results in a significant amount of power dissipation and area overhead. The inherent probabilistic switching behavior of nanomagnets provides an advantage to overcome these complexity issues for the realization of low power and area efficient SNC systems. This paper presents magnetic tunnel junction (MTJ)-based stochastic computing methodology for the implementation of a neural network. The stochastic switching behavior of the MTJ has been exploited to design a binary to stochastic converter to mitigate the complexity of the CMOS-based design. The paper also presents the technique for realizing stochastic sigmoid activation function using an MTJ. Such circuits are simpler than existing ones and use considerably less power. An image classification system employing the proposed circuits has been implemented to verify the effectiveness of the technique. The MTJ-based SNC system shows area and energy reduction by a factor of 13.5 and 2.5, respectively, while the prediction accuracy is 86.66%. Furthermore, this paper investigates how crucial parameters, such as stochastic bitstream length, number of hidden layers and number of nodes in a hidden layer, need to be set precisely to realize an efficient MTJ-based stochastic neural network (SNN). The proposed methodology can prove a promising alternative for highly efficient digital stochastic computing applications.

Integer factorization using stochastic magnetic tunnel junctions

Conventional computers operate deterministically using strings of zeros and ones called bits to represent information in binary code. Despite the evolution of conventional computers into sophisticated machines, there are many classes of problems that they cannot efficiently address, including inference, invertible logic, sampling and optimization, leading to considerable interest in alternative computing schemes. Quantum computing, which uses qubits to represent a superposition of 0 and 1, is expected to perform these tasks efficiently^1-3. However, decoherence and the current requirement for cryogenic operation⁴, as well as the limited many-body interactions that can be implemented, pose considerable challenges. Probabilistic computing^1,5-7 is another unconventional computation scheme that shares similar concepts with quantum computing but is not limited by the above challenges. The key role is played by a probabilistic bit (a p-bit)-a robust, classical entity fluctuating in time between 0 and 1, which interacts with other p-bits in the same system using principles inspired by neural networks⁸. Here we present a proof-of-concept experiment for probabilistic computing using spintronics technology, and demonstrate integer factorization, an illustrative example of the optimization class of problems addressed by adiabatic⁹ and gated² quantum computing. Nanoscale magnetic tunnel junctions showing stochastic behaviour are developed by modifying market-ready magnetoresistive random-access memory technology^10,11 and are used to implement three-terminal p-bits that operate at room temperature. The p-bits are electrically connected to form a functional asynchronous network, to which a modified adiabatic quantum computing algorithm that implements three- and four-body interactions is applied. Factorization of integers up to 945 is demonstrated with this rudimentary asynchronous probabilistic computer using eight correlated p-bits, and the results show good agreement with theoretical predictions, thus providing a potentially scalable hardware approach to the difficult problems of optimization and sampling.