Revisiting neural information, computing and linking capacity

Neural information theory represents a fundamental method to model dynamic relations in biological systems. However, the notion of information, its representation, its content and how it is processed are the subject of fierce debates. Since the limiting capacity of neuronal links strongly depends on how neurons are hypothesized to work, their operating modes are revisited by analyzing the differences between the results of the communication models published during the past seven decades and those of the recently developed generalization of the classical information theory. It is pointed out that the operating mode of neurons is in resemblance with an appropriate combination of the formerly hypothesized analog and digital working modes; furthermore that not only the notion of neural information and its processing must be reinterpreted. Given that the transmission channel is passive in Shannon's model, the active role of the transfer channels (the axons) may introduce further transmission limits in addition to the limits concluded from the information theory. The time-aware operating model enables us to explain why (depending on the researcher's point of view) the operation can be considered either purely analog or purely digital.

On the Role of Speed in Technological and Biological Information Transfer for Computations

In all kinds of implementations of computing, whether technological or biological, some material carrier for the information exists, so in real-world implementations, the propagation speed of information cannot exceed the speed of its carrier. Because of this limitation, one must also consider the transfer time between computing units for any implementation. We need a different mathematical method to consider this limitation: classic mathematics can only describe infinitely fast and small computing system implementations. The difference between mathematical handling methods leads to different descriptions of the computing features of the systems. The proposed handling also explains why biological implementations can have lifelong learning and technological ones cannot. Our conclusion about learning matches published experimental evidence, both in biological and technological computing.

Towards Generalizing the Information Theory for Neural Communication

Neuroscience extensively uses the information theory to describe neural communication, among others, to calculate the amount of information transferred in neural communication and to attempt the cracking of its coding. There are fierce debates on how information is represented in the brain and during transmission inside the brain. The neural information theory attempts to use the assumptions of electronic communication; despite the experimental evidence that the neural spikes carry information on non-discrete states, they have shallow communication speed, and the spikes' timing precision matters. Furthermore, in biology, the communication channel is active, which enforces an additional power bandwidth limitation to the neural information transfer. The paper revises the notions needed to describe information transfer in technical and biological communication systems. It argues that biology uses Shannon's idea outside of its range of validity and introduces an adequate interpretation of information. In addition, the presented time-aware approach to the information theory reveals pieces of evidence for the role of processes (as opposed to states) in neural operations. The generalized information theory describes both kinds of communication, and the classic theory is the particular case of the generalized theory.

Which scaling rule applies to large artificial neural networks

Experience shows that cooperating and communicating computing systems, comprising segregated single processors, have severe performance limitations, which cannot be explained using von Neumann’s classic computing paradigm. In his classic “First Draft,” he warned that using a “too fast processor” vitiates his simple “procedure” (but not his computing model!); furthermore, that using the classic computing paradigm for imitating neuronal operations is unsound. Amdahl added that large machines, comprising many processors, have an inherent disadvantage. Given that artificial neural network’s (ANN’s) components are heavily communicating with each other, they are built from a large number of components designed/fabricated for use in conventional computing, furthermore they attempt to mimic biological operation using improper technological solutions, and their achievable payload computing performance is conceptually modest. The type of workload that artificial intelligence-based systems generate leads to an exceptionally low payload computational performance, and their design/technology limits their size to just above the “toy” level systems: The scaling of processor-based ANN systems is strongly nonlinear. Given the proliferation and growing size of ANN systems, we suggest ideas to estimate in advance the efficiency of the device or application. The wealth of ANN implementations and the proprietary technical data do not enable more. Through analyzing published measurements, we provide evidence that the role of data transfer time drastically influences both ANNs performance and feasibility. It is discussed how some major theoretical limiting factors, ANN’s layer structure and their methods of technical implementation of communication affect their efficiency. The paper starts from von Neumann’s original model, without neglecting the transfer time apart from processing time, and derives an appropriate interpretation and handling for Amdahl’s law. It shows that, in that interpretation, Amdahl’s law correctly describes ANNs.

How Amdahl's Law limits the performance of large artificial neural networks : why the functionality of full-scale brain simulation on processor-based simulators is limited

With both knowing more and more details about how neurons and complex neural networks work and having serious demand for making performable huge artificial networks, more and more efforts are devoted to build both hardware and/or software simulators and supercomputers targeting artificial intelligence applications, demanding an exponentially increasing amount of computing capacity. However, the inherently parallel operation of the neural networks is mostly simulated deploying inherently sequential (or in the best case: sequential–parallel) computing elements. The paper shows that neural network simulators, (both software and hardware ones), akin to all other sequential–parallel computing systems, have computing performance limitation due to deploying clock-driven electronic circuits, the 70-year old computing paradigm and Amdahl’s Law about parallelized computing systems. The findings explain the limitations/saturation experienced in former studies.

Regulatory T cells in peripheral blood of patients with mixed connective tissue disease

OBJECTIVE

To determine the percentage and absolute number of CD4+ regulatory T-cells (Treg) in 48 patients with mixed connective tissue disease (MCTD).

METHODS

Data were classified on the basis of the stage of the disease: 17 patients were in the active stage and 31 in the inactive stage. The absolute number of CD4+/intracellular interleukin-10+ (IL-10+) and CD4+CD25+high Treg cells was determined by flow cytometry. The percentage of CD4+CD25+high suppressor T-cells was determined on the basis of Foxp3 expression.

RESULTS

The percentage and the absolute number of CD4+CD25+high Treg cells were lower in patients than in healthy controls (p<0.04), and were further decreased in patients with active MCTD and were lower than in the inactive stage (p<0.01). There was an increase in the percentage and absolute number of CD4+IL-10+ Treg cells in patients with MCTD compared to the healthy controls (p<0.02). The percentage of CD4+IL-10+ Treg cells was higher in the active stage of MCTD than in the inactive stage of the disease (p<0.005). However, we did not find any significant difference in the absolute number of CD4+IL-10+ Treg cells between the patients in the active and inactive stages of MCTD.

CONCLUSION

Our results suggest that the decrease in the number of CD4+CD25+high Treg cells in an important factor in the immunoregulatory disturbance in patients with MCTD. We suggest that the increase in the number of CD4+IL-10+ Treg cells is a compensatory mechanism aiming to restore the balance between type 1 and type 2 cytokines in MCTD.

Catastrophic antiphospholipid syndrome in cancer

Antiphospholipid syndrome is characterized by the presence of antiphospholipid antibodies resulting in arterial and venous thromboembolism. Apart from primary cases, this syndrome is often associated with autoimmune diseases. Around 50 cases of catastrophic antiphospholipid antibody syndrome have been reported as yet. Authors describe the first case of catastrophic antiphospholipid syndrome associated with gastric cancer. Apart from presenting the clinical case, authors also discuss the possible pathomechanism of this associated disorder including the role of immunological factors, as well as antiphospholipid antibodies.

[Catastrophic antiphospholipid antibody syndrome in a cancer patient]

Antiphospholipid syndrome is characterized by the presence of antiphospholipid antibodies resulting in arterial and venous thromboembolism. Apart from primary cases, this syndrome is often associated with autoimmune diseases. Around 50 cases of catastrophic antiphospholipid antibody syndrome have been reported as yet. Authors describe a case of a female patients with catastrophic antiphospholipid syndrome associated with gastric cancer. This may be the first case of such association in the literature. Authors also discuss the possible pathomechanism of this disorder, as well as the available therapeutic approaches.