Some Information Measures Properties of the GOS-Concomitants from the FGM Family

In this paper we recall, extend and compute some information measures for the concomitants of the generalized order statistics (GOS) from the Farlie–Gumbel–Morgenstern (FGM) family. We focus on two types of information measures: some related to Shannon entropy, and some related to Tsallis entropy. Among the information measures considered are residual and past entropies which are important in a reliability context.

From Chaos to Ordering: New Studies in the Shannon Entropy of 2D Patterns

Properties of the Voronoi tessellations arising from random 2D distribution points are reported. We applied an iterative procedure to the Voronoi diagrams generated by a set of points randomly placed on the plane. The procedure implied dividing the edges of Voronoi cells into equal or random parts. The dividing points were then used to construct the following Voronoi diagram. Repeating this procedure led to a surprising effect of the positional ordering of Voronoi cells, reminiscent of the formation of lamellae and spherulites in linear semi-crystalline polymers and metallic glasses. Thus, we can conclude that by applying even a simple set of rules to a random set of seeds, we can introduce order into an initially disordered system. At the same time, the Shannon (Voronoi) entropy showed a tendency to attain values that are typical for completely random patterns; thus, the Shannon (Voronoi) entropy does not distinguish the short-range ordering. The Shannon entropy and the continuous measure of symmetry of the patterns demonstrated the distinct asymptotic behavior, while approaching the close saturation values with the increase in the number of iteration steps. The Shannon entropy grew with the number of iterations, whereas the continuous measure of symmetry of the same patterns demonstrated the opposite asymptotic behavior. The Shannon (Voronoi) entropy is not an unambiguous measure of order in the 2D patterns. The more symmetrical patterns may demonstrate the higher values of the Shannon entropy.

Tight and Scalable Side-Channel Attack Evaluations through Asymptotically Optimal Massey-like Inequalities on Guessing Entropy

The bounds presented at CHES 2017 based on Massey’s guessing entropy represent the most scalable side-channel security evaluation method to date. In this paper, we present an improvement of this method, by determining the asymptotically optimal Massey-like inequality and then further refining it for finite support distributions. The impact of these results is highlighted for side-channel attack evaluations, demonstrating the improvements over the CHES 2017 bounds.

Entropy of Badminton Strike Positions

The aim of the current study was twofold: (i) to investigate the distribution of the strike positions of badminton players while quantifying the corresponding standard entropy and using an alternative metric (spatial entropy) related to winning and losing points and random positions; and (ii) to evaluate the standard entropy of the receiving positions. With the datasets of 259 badminton matches, we focused on the positions of players’ strokes and the outcome of each point. First, we identified those regions of the court from which hits were most likely to be struck. Second, we computed the standard entropy of stroke positions, and then the spatial entropy, which also considers the order and clustering of the hitting locations in a two-dimensional Euclidean space. Both entropy quantifiers revealed high uncertainty in the striking position; however, specific court locations (i.e., the four corners) are preferred over the rest. When the outcome of each point was taken into account, we observed that the hitting patterns with lower entropy were associated with higher probabilities of winning points. On the contrary, players striking from more random positions were more prone to losing the points.

Information-based decoding of the coupling among human brain activity and movement paths

BACKGROUND

Walking like many other actions of a human is controlled by the brain through the nervous system. In fact, if a problem occurs in our brain, we cannot walk correctly. Therefore, the analysis of the coupling of brain activity and walking is very important especially in rehabilitation science. The complexity of movement paths is one of the factors that affect human walking. For instance, if we walk on a path that is more complex, our brain activity increases to adjust our movements.

OBJECTIVE

This study for the first time analyzed the coupling of walking paths and brain reaction from the information point of view.

METHODS

We analyzed the Shannon entropy for electroencephalography (EEG) signals versus the walking paths in order to relate their information contents.

RESULTS

According to the results, walking on a path that contains more information causes more information in EEG signals. A strong correlation (p= 0.9999) was observed between the information contents of EEG signals and walking paths. Our method of analysis can also be used to investigate the relation among other physiological signals of a human and walking paths, which has great benefits in rehabilitation science.

Exploiting the Massey Gap

In this paper, we find new refinements to the Massey inequality, which relates the Shannon and guessing entropies, introducing a new concept: the Massey gap. By shrinking the Massey gap, we improve all previous work without introducing any new parameters, providing closed-form strict refinements, as well as a numerical procedure improving them even further.

A New Fractional Particle Swarm Optimization with Entropy Diversity Based Velocity for Reactive Power Planning

Optimal Reactive Power Dispatch (ORPD) is the vital concern of network operators in the planning and management of electrical systems to reduce the real and reactive losses of the transmission and distribution system in order to augment the overall efficiency of the electrical network. The principle objective of the ORPD problem is to explore the best setting of decision variables such as rating of the shunt capacitors, output voltage of the generators and tap setting of the transformers in order to diminish the line loss, and improve the voltage profile index (VPI) and operating cost minimization of standard electrical systems while keeping the variables within the allowable limits. This research study demonstrates a compelling transformative approach for resolving ORPD problems faced by the operators through exploiting the strength of the meta-heuristic optimization model based on a new fractional swarming strategy, namely fractional order (FO)-particle swarm optimization (PSO), with consideration of the entropy metric in the velocity update mechanism. To perceive ORPD for standard 30 and 57-bus networks, the complex nonlinear objective functions, including minimization of the system, VPI improvement and operating cost minimization, are constructed with emphasis on efficacy enhancement of the overall electrical system. Assessment of the results show that the proposed FO-PSO with entropy metric performs better than the other state of the art algorithms by means of improvement in VPI, operating cost and line loss minimization. The statistical outcomes in terms of quantile-quantile illustrations, probability plots, cumulative distribution function, box plots, histograms and minimum fitness evaluation in a set of autonomous trials validate the capability of the proposed optimization scheme and exhibit sufficiency and also vigor in resolving ORPD problems.

Stochastic Comparisons of Weighted Distributions and Their Mixtures

In this paper, various stochastic ordering properties of a parametric family of weighted distributions and the associated mixture model are developed. The effect of stochastic variation of the output random variable with respect to the parameter and/or the underlying random variable is specifically investigated. Special weighted distributions are considered to scrutinize the consistency as well as the usefulness of the results. Stochastic comparisons of coherent systems made of identical but dependent components are made and also a result for comparison of Shannon entropies of weighted distributions is developed.

Some Remarks about Entropy of Digital Filtered Signals

The finite numerical resolution of digital number representation has an impact on the properties of filters. Much effort has been done to develop efficient digital filters investigating the effects in the frequency response. However, it seems that there is less attention to the influence in the entropy by digital filtered signals due to the finite precision. To contribute in such a direction, this manuscript presents some remarks about the entropy of filtered signals. Three types of filters are investigated: Butterworth, Chebyshev, and elliptic. Using a boundary technique, the parameters of the filters are evaluated according to the word length of 16 or 32 bits. It has been shown that filtered signals have their entropy increased even if the filters are linear. A significant positive correlation (p < 0.05) was observed between order and Shannon entropy of the filtered signal using the elliptic filter. Comparing to signal-to-noise ratio, entropy seems more efficient at detecting the increasing of noise in a filtered signal. Such knowledge can be used as an additional condition for designing digital filters.

Investigation of Early Warning Indexes in a Three-Dimensional Chaotic System with Zero Eigenvalues

A rare three-dimensional chaotic system with all eigenvalues equal to zero is proposed, and its dynamical properties are investigated. The chaotic system has one equilibrium point at the origin. Numerical analysis shows that the equilibrium point is unstable. Bifurcation analysis of the system shows various dynamics in a period-doubling route to chaos. We highlight that from the evaluation of the entropy, bifurcation points can be predicted by identifying early warning signals. In this manner, bifurcation points of the system are analyzed using Shannon and Kolmogorov-Sinai entropy. The results are compared with Lyapunov exponents.

Entropy Rate Estimation for English via a Large Cognitive Experiment Using Mechanical Turk

The entropy rate h of a natural language quantifies the complexity underlying the language. While recent studies have used computational approaches to estimate this rate, their results rely fundamentally on the performance of the language model used for prediction. On the other hand, in 1951, Shannon conducted a cognitive experiment to estimate the rate without the use of any such artifact. Shannon’s experiment, however, used only one subject, bringing into question the statistical validity of his value of h = 1.3 bits per character for the English language entropy rate. In this study, we conducted Shannon’s experiment on a much larger scale to reevaluate the entropy rate h via Amazon’s Mechanical Turk, a crowd-sourcing service. The online subjects recruited through Mechanical Turk were each asked to guess the succeeding character after being given the preceding characters until obtaining the correct answer. We collected 172,954 character predictions and analyzed these predictions with a bootstrap technique. The analysis suggests that a large number of character predictions per context length, perhaps as many as 10 3 , would be necessary to obtain a convergent estimate of the entropy rate, and if fewer predictions are used, the resulting h value may be underestimated. Our final entropy estimate was h ≈ 1.22 bits per character.

A comparative study of sequence- and structure-based features of small RNAs and other RNAs of bacteria

Small RNAs (sRNAs) in bacteria have emerged as key players in transcriptional and post-transcriptional regulation of gene expression. Here, we present a statistical analysis of different sequence- and structure-related features of bacterial sRNAs to identify the descriptors that could discriminate sRNAs from other bacterial RNAs. We investigated a comprehensive and heterogeneous collection of 816 sRNAs, identified by northern blotting across 33 bacterial species and compared their various features with other classes of bacterial RNAs, such as tRNAs, rRNAs and mRNAs. We observed that sRNAs differed significantly from the rest with respect to G+C composition, normalized minimum free energy of folding, motif frequency and several RNA-folding parameters like base-pairing propensity, Shannon entropy and base-pair distance. Based on the selected features, we developed a predictive model using Random Forests (RF) method to classify the above four classes of RNAs. Our model displayed an overall predictive accuracy of 89.5%. These findings would help to differentiate bacterial sRNAs from other RNAs and further promote prediction of novel sRNAs in different bacterial species.

Experimental Investigation of Secondary Flow Structures Downstream of a Model Type IV Stent Failure in a 180° Curved Artery Test Section

The arterial network in the human vasculature comprises of ubiquitously present blood vessels with complex geometries (branches, curvatures and tortuosity). Secondary flow structures are vortical flow patterns that occur in curved arteries due to the combined action of centrifugal forces, adverse pressure gradients and inflow characteristics. Such flow morphologies are greatly affected by pulsatility and multiple harmonics of physiological inflow conditions and vary greatly in size-strength-shape characteristics compared to non-physiological (steady and oscillatory) flows (1 - 7). Secondary flow structures may ultimately influence the wall shear stress and exposure time of blood-borne particles toward progression of atherosclerosis, restenosis, sensitization of platelets and thrombosis (4 - 6, 8 - 13). Therefore, the ability to detect and characterize these structures under laboratory-controlled conditions is precursor to further clinical investigations. A common surgical treatment to atherosclerosis is stent implantation, to open up stenosed arteries for unobstructed blood flow. But the concomitant flow perturbations due to stent installations result in multi-scale secondary flow morphologies (4 - 6). Progressively higher order complexities such as asymmetry and loss in coherence can be induced by ensuing stent failures vis-à-vis those under unperturbed flows (5). These stent failures have been classified as "Types I-to-IV" based on failure considerations and clinical severity (14). This study presents a protocol for the experimental investigation of the complex secondary flow structures due to complete transverse stent fracture and linear displacement of fractured parts ("Type IV") in a curved artery model. The experimental method involves the implementation of particle image velocimetry (2C-2D PIV) techniques with an archetypal carotid artery inflow waveform, a refractive index matched blood-analog working fluid for phase-averaged measurements (15 - 18). Quantitative identification of secondary flow structures was achieved using concepts of flow physics, critical point theory and a novel wavelet transform algorithm applied to experimental PIV data (5, 6, 19 - 26).

Computational prediction of miRNAs and their targets in Phaseolus vulgaris using simple sequence repeat signatures

BACKGROUND

MicroRNAs (miRNAs) are endogenous, noncoding, short RNAs directly involved in regulating gene expression at the post-transcriptional level. In spite of immense importance, limited information of P. vulgaris miRNAs and their expression patterns prompted us to identify new miRNAs in P. vulgaris by computational methods. Besides conventional approaches, we have used the simple sequence repeat (SSR) signatures as one of the prediction parameter. Moreover, for all other parameters including normalized Shannon entropy, normalized base pairing index and normalized base-pair distance, instead of taking a fixed cut-off value, we have used 99% probability range derived from the available data.

RESULTS

We have identified 208 mature miRNAs in P. vulgaris belonging to 118 families, of which 201 are novel. 97 of the predicted miRNAs in P. vulgaris were validated with the sequencing data obtained from the small RNA sequencing of P. vulgaris. Randomly selected predicted miRNAs were also validated using qRT-PCR. A total of 1305 target sequences were identified for 130 predicted miRNAs. Using 80% sequence identity cut-off, proteins coded by 563 targets were identified. The computational method developed in this study was also validated by predicting 229 miRNAs of A. thaliana and 462 miRNAs of G. max, of which 213 for A. thaliana and 397 for G. max are existing in miRBase 20.

CONCLUSIONS

There is no universal SSR that is conserved among all precursors of Viridiplantae, but conserved SSR exists within a miRNA family and is used as a signature in our prediction method. Prediction of known miRNAs of A. thaliana and G. max validates the accuracy of our method. Our findings will contribute to the present knowledge of miRNAs and their targets in P. vulgaris. This computational method can be applied to any species of Viridiplantae for the successful prediction of miRNAs and their targets.

Information theory and the ethylene genetic network

The original aim of the Information Theory (IT) was to solve a purely technical problem: to increase the performance of communication systems, which are constantly affected by interferences that diminish the quality of the transmitted information. That is, the theory deals only with the problem of transmitting with the maximal precision the symbols constituting a message. In Shannon's theory messages are characterized only by their probabilities, regardless of their value or meaning. As for its present day status, it is generally acknowledged that Information Theory has solid mathematical foundations and has fruitful strong links with Physics in both theoretical and experimental areas. However, many applications of Information Theory to Biology are limited to using it as a technical tool to analyze biopolymers, such as DNA, RNA or protein sequences. The main point of discussion about the applicability of IT to explain the information flow in biological systems is that in a classic communication channel, the symbols that conform the coded message are transmitted one by one in an independent form through a noisy communication channel, and noise can alter each of the symbols, distorting the message; in contrast, in a genetic communication channel the coded messages are not transmitted in the form of symbols but signaling cascades transmit them. Consequently, the information flow from the emitter to the effector is due to a series of coupled physicochemical processes that must ensure the accurate transmission of the message. In this review we discussed a novel proposal to overcome this difficulty, which consists of the modeling of gene expression with a stochastic approach that allows Shannon entropy (H) to be directly used to measure the amount of uncertainty that the genetic machinery has in relation to the correct decoding of a message transmitted into the nucleus by a signaling pathway. From the value of H we can define a function I that measures the amount of information content in the input message that the cell's genetic machinery is processing during a given time interval. Furthermore, combining Information Theory with the frequency response analysis of dynamical systems we can examine the cell's genetic response to input signals with varying frequencies, amplitude and form, in order to determine if the cell can distinguish between different regimes of information flow from the environment. In the particular case of the ethylene signaling pathway, the amount of information managed by the root cell of Arabidopsis can be correlated with the frequency of the input signal. The ethylene signaling pathway cuts off very low and very high frequencies, allowing a window of frequency response in which the nucleus reads the incoming message as a varying input. Outside of this window the nucleus reads the input message as an approximately non-varying one. This frequency response analysis is also useful to estimate the rate of information transfer during the transport of each new ERF1 molecule into the nucleus. Additionally, application of Information Theory to analysis of the flow of information in the ethylene signaling pathway provides a deeper insight in the form in which the transition between auxin and ethylene hormonal activity occurs during a circadian cycle. An ambitious goal for the future would be to use Information Theory as a theoretical foundation for a suitable model of the information flow that runs at each level and through all levels of biological organization.

Information flow in plant signaling pathways

Systems biology and mathematical approaches are required for understanding how genetic regulatory networks process information from the environment. A typical genetic communication channel is conformed by: 1) an encoder, which is a specific membrane receptor that perceives the environmental information in the form of a concentration of a specific phytohormone. In the particular case of the ethylene signaling pathway, the encoder is the ETR1,2 specific receptor to ethylene; 2) a transmitting pathway, which is a signaling pathway. In the case, the ethylene signaling pathway; 3) a decoder, which is the molecular transcriptional machinery associated with the ERF1 and downstream genes and 4) an effector, which is the molecular translational machinery associated to the ethylene genetic network. Every communication channel is subject to noise, i.e., any physicochemical process that can alter the message carried from the encoder to the decoder and effector. Noise introduces a certain amount of uncertainty in any message spread through the communication channel. The amount of uncertainty in the content of a message is measured with the Shannon's entropy function H and, consequently, the amount of information actually carried by the message can be measured with the information function I = Hmax-H. Genetic networks are composed of a relative low and fluctuating amount of molecules and this characteristic, together with the effect of noise, produces a genetic response at time t with a probability p(t) of being correct with respect to the input message, and a probability 1-p(t) of been incorrect. From these probability values, H and I functions can be evaluated and, for the first time, it is possible to assign a measure of information content to each message associated to a given concentration of phytohormone. This type of analysis can be applied to any other plant genetic regulatory network.

On the entropy of a hidden Markov process

We study the entropy rate of a hidden Markov process (HMP) defined by observing the output of a binary symmetric channel whose input is a first-order binary Markov process. Despite the simplicity of the models involved, the characterization of this entropy is a long standing open problem. By presenting the probability of a sequence under the model as a product of random matrices, one can see that the entropy rate sought is equal to a top Lyapunov exponent of the product. This offers an explanation for the elusiveness of explicit expressions for the HMP entropy rate, as Lyapunov exponents are notoriously difficult to compute. Consequently, we focus on asymptotic estimates, and apply the same product of random matrices to derive an explicit expression for a Taylor approximation of the entropy rate with respect to the parameter of the binary symmetric channel. The accuracy of the approximation is validated against empirical simulation results. We also extend our results to higher-order Markov processes and to Rényi entropies of any order.

Sensitive ultrasonic detection of dystrophic skeletal muscle in patients with duchenne muscular dystrophy using an entropy-based signal receiver

The dystrophinopathies comprise a group of X-linked genetic diseases that feature dystrophin deficiency. Duchenne and Becker muscular dystrophy are characterized by progressive weakness and wasting of skeletal, smooth, and/or cardiac muscle. Duchenne muscular dystrophy (DMD) is the most severe dystrophinopathy, with an incidence of 1:3500 male births. Despite understanding the structural and genetic basis for DMD, the pathogenesis and clinical basis for more severe involvement in specific skeletal muscle groups and the heart are poorly understood. Current techniques, such as strength testing for monitoring progress of disease and therapy in DMD patients, are imprecise and physically demanding for test subjects. Ultrasound is well-suited to detect changes in structure and organization in muscle tissue in a manner that makes low demands on the patient. Therefore, we investigated the use of ultrasound to quantitatively phenotype the remodeling process in patients with DMD. Beam-formed radio-frequency (RF) data were acquired from the skeletal muscles of nine DMD and five normal subjects imaged with a clinical imaging system (HDI5000 w/7 MHz probe applied above left biceps muscle). From these data, images were reconstructed using B-mode (log of analytic signal magnitude) and information-theoretic receivers (H(f)-receiver). H(f) images obtained from dystrophic muscle contained extensive "mottled" regions (i.e., areas with heterogeneous image contrast) that were not readily apparent from the B-Mode images. The 2-D autocorrelation of DMD H(f) images have broader peaks than those of normal subjects, which is indicative of larger scatterer sizes, consistent with pathologic changes of fibers, edema and fatty infiltration. Comparison of the relative peak widths (full width measured at 60% maximum) of the autocorrelation of the DMD and normal H(f) images shows a quantitative difference between the two groups (p < 0.005, student two-tailed paired t-test). Consequently, these imaging techniques may prove useful for longitudinal monitoring of disease progression and therapy.

Non-Rigid Multi-Modal Image Registration Using Cross-Cumulative Residual Entropy

In this paper we present a new approach for the non-rigid registration of multi-modality images. Our approach is based on an information theoretic measure called the cumulative residual entropy (CRE), which is a measure of entropy defined using cumulative distributions. Cross-CRE between two images to be registered is defined and maximized over the space of smooth and unknown non-rigid transformations. For efficient and robust computation of the non-rigid deformations, a tri-cubic B-spline based representation of the deformation function is used. The key strengths of combining CCRE with the tri-cubic B-spline representation in addressing the non-rigid registration problem are that, not only do we achieve the robustness due to the nature of the CCRE measure, we also achieve computational efficiency in estimating the non-rigid registration. The salient features of our algorithm are: (i) it accommodates images to be registered of varying contrast+brightness, (ii) faster convergence speed compared to other information theory-based measures used for non-rigid registration in literature, (iii) analytic computation of the gradient of CCRE with respect to the non-rigid registration parameters to achieve efficient and accurate registration, (iv) it is well suited for situations where the source and the target images have field of views with large non-overlapping regions. We demonstrate these strengths via experiments on synthesized and real image data.

Unique folding of precursor microRNAs: quantitative evidence and implications for de novo identification

MicroRNAs (miRNAs) participate in diverse cellular and physiological processes through the post-transcriptional gene regulatory pathway. Hairpin is a crucial structural feature for the computational identification of precursor miRNAs (pre-miRs), as its formation is critically associated with the early stages of the mature miRNA biogenesis. Our incomplete knowledge about the number of miRNAs present in the genomes of vertebrates, worms, plants, and even viruses necessitates thorough understanding of their sequence motifs, hairpin structural characteristics, and topological descriptors. In this in-depth study, we investigate a comprehensive and heterogeneous collection of 2241 published (nonredundant) pre-miRs across 41 species (miRBase 8.2), 8494 pseudohairpins extracted from the human RefSeq genes, 12,387 (nonredundant) ncRNAs spanning 457 types (Rfam 7.0), 31 full-length mRNAs randomly selected from GenBank, and four sets of synthetically generated genomic background corresponding to each of the native RNA sequence. Our large-scale characterization analysis reveals that pre-miRs are significantly different from other types of ncRNAs, pseudohairpins, mRNAs, and genomic background according to the nonparametric Kruskal-Wallis ANOVA (p<0.001). We examine the intrinsic and global features at the sequence, structural, and topological levels including %G+C content, normalized base-pairing propensity P(S), normalized minimum free energy of folding MFE(s), normalized Shannon entropy Q(s), normalized base-pair distance D(s), and degree of compactness F(S), as well as their corresponding Z scores of P(S), MFE(s), Q(s), D(s), and F(S). The findings will promote more accurate guidelines and distinctive criteria for the prediction of novel pre-miRs with improved performance.

Two-phase unfolding pathway of ribonuclease A during denaturation induced by dithiothreitol

The dynamics of the unfolding process of bovine pancreatic ribonuclease A (RNase A) unfolded by dithiothreitol (DTT) at a low concentration of 1:30 were investigated in alkaline phosphate-buffered saline solutions at 303K and 313K by using proton nuclear magnetic resonance ((1)H NMR) spectra. Three NMR spectral parameters including Shannon entropy, mutual information, and correlation coefficient were introduced into the analysis. The results show that the unfolding process of RNase A was slowed to the order of many hours, and the kinetics of the unfolding pathway described by the three parameters is best fit by a model of two consecutive first-order reactions. Temperature greatly influences the rate constants of the unfolding kinetics with different temperature effects observed for the fast and the slow processes. The consistencies and the differences between the three sets of parameters show that they reflect the same relative denaturation pathway but different spectra windows of the unfolding process of RNase A. The results suggest that the unfolding process of RNase A induced by low concentrations of DTT is a two-phase pathway containing fast and slow first-order reactions.