The Arcade Learning Environment: An Evaluation Platform for General Agents

In this article we introduce the Arcade Learning Environment (ALE): both a challenge problem and a platform and methodology for evaluating the development of general, domain-independent AI technology. ALE provides an interface to hundreds of Atari 2600 game environments, each one different, interesting, and designed to be a challenge for human players. ALE presents significant research challenges for reinforcement learning, model learning, model-based planning, imitation learning, transfer learning, and intrinsic motivation. Most importantly, it provides a rigorous testbed for evaluating and comparing approaches to these problems. We illustrate the promise of ALE by developing and benchmarking domain-independent agents designed using well-established AI techniques for both reinforcement learning and planning. In doing so, we also propose an evaluation methodology made possible by ALE, reporting empirical results on over 55 different games. All of the software, including the benchmark agents, is publicly available.

A pedigree-based study of mitochondrial D-loop DNA sequence variation among Arabian horses

Through DNA sequence comparisons of a mitochondrial D-loop hypervariable region, we investigated matrilineal diversity for Arabian horses in the United States. Sixty-two horses were tested. From published pedigrees they traced in the maternal line to 34 mares acquired primarily in the mid to late 19th century from nomadic Bedouin tribes. Compared with the reference sequence (GenBank X79547), these samples showed 27 haplotypes with altogether 31 base substitution sites within 397 bp of sequence. Based on examination of pedigrees from a random sampling of 200 horses in current studbooks of the Arabian Horse Registry of America, we estimated that this study defined the expected mtDNA haplotypes for at least 89% of Arabian horses registered in the US. The reliability of the studbook recorded maternal lineages of Arabian pedigrees was demonstrated by haplotype concordance among multiple samplings in 14 lines. Single base differences observed within two maternal lines were interpreted as representing alternative fixations of past heteroplasmy. The study also demonstrated the utility of mtDNA sequence studies to resolve historical maternity questions without access to biological material from the horses whose relationship was in question, provided that representatives of the relevant female lines were available for comparison. The data call into question the traditional assumption that Arabian horses of the same strain necessarily share a common maternal ancestry.

Matrix metalloproteinase production in regenerating axolotl spinal cord

In urodele amphibian spinal cord regeneration, the ependymal cells lining the central canal remodel the lesion site to favor axonal regrowth. We profiled the production of matrix metalloproteinases by injury-reactive mesenchymal ependymal cells in vivo and in vitro and found that matrix metalloproteinases are involved in this remodeling process in the axolotl (Ambystoma mexicanum). The production of cell-associated matrix metalloproteinases in vivo was shown to be identical to that in our cultured ependymal cell model system. Activated and zymogen forms of matrix metalloproteinases were identified using zymography, chemical inhibitors of matrix metalloproteinases, and cleavage of propeptides by organomercurials. The principal cellular proteinases consisted of matrix metalloproteinase-2 (gelatinase A) and matrix metalloproteinase-1 (type I collagenase), which display characteristic shifts in molecular weight following proenzyme processing by organomercurials. In addition, ependymal cell conditioned medium contained secreted forms of the enzyme undetectable in situ. Matrix metalloproteinase-9 (gelatinase B) as well as matrix metalloproteinase-2 and matrix metalloproteinase-1 were secreted and casein substrate zymography showed the presence of a small amount of a very high molecular weight matrix metalloproteinase-3 (prostromelysin) secreted into the culture medium. Matrix metalloproteinases were still present at 4 weeks post-lesioning when the ependymal cells have just re-epithelialized, but decreased near the completion of regeneration (8 weeks post-lesioning). Zymography showed no detectable matrix metalloproteinases in unlesioned cord but the presence of tissue inhibitor of metalloproteinase-1 in intact cord was seen by Western blotting. This study shows that matrix metalloproteinases are associated with urodele spinal cord regeneration and validates the use of our ependymal cell tissue culture model system to evaluate ependymal cell behavior during spinal cord regeneration.

Effect of restricted feed intake in broiler breeder hens on their stress levels and the growth and immunology of their offspring

The prenatal environment has been shown to have significant effects on the lifelong health of offspring in humans and other species. Such effects have not been studied extensively in avian species but could prove important, especially in the case of severe feed restriction imposed on broiler breeder hens to prevent obesity and reduce rate of lay. Feed restriction can potentially affect not only nutrient supply to the embryo but stress hormone levels within the hen. This study investigated the impact of nutrient restriction of the breeder hen on growth rate and immune responses in the progeny with the objective to measure the impact of feed restriction of broiler breeder hens on growth and immune response of the progeny. Broiler breeder hens were feed restricted from 24 wk of age and maintained at three bodyweights; 3.4, 3.6, and 4.0 kg until 43 wk of age and behavioral and physiological measures of stress recorded. Chicks were hatched from each hen treatment and at day 7 vaccinated for infectious bronchitis virus (IBV) and at 16, 18, and 20 d old given an immune challenge of lipopolysaccharide. Growth and immune responses of these birds were then recorded. Sex ratio was affected by hen bodyweight, with a significantly increased proportion of males hatched from heavy hens. Growth rate from 35 to 42 d of age was reduced in male progeny from low bodyweight hens. Female progeny from heavy hens responded to an immune challenge by reduced live weight and increased heterophil: lymphocyte ratio, suggesting a more robust immune response in these birds than in the progeny from lower bodyweight hens. Overall, progeny from heavy hens had increased antibodies at day 35 to the vaccination of IBV compared with progeny of low bodyweight hens, also suggesting an improved immune response in these birds. Breeder hens restricted to the lowest feed level showed behaviors indicative of increased stress (object pecking) and an increased heterophil: lymphocyte ratio. Feed restriction of broiler breeder hens increased indices of stress in hens and resulted in offspring that have reduced growth rate and immune response in a sex-dependent way.

Existence of Multiagent Equilibria with Limited Agents

Multiagent learning is a necessary yet challenging problem as multiagent systems become more prevalent and environments become more dynamic. Much of the groundbreaking work in this area draws on notable results from game theory, in particular, the concept of Nash equilibria. Learners that directly learn an equilibrium obviously rely on their existence. Learners that instead seek to play optimally with respect to the other players also depend upon equilibria since equilibria are fixed points for learning. From another perspective, agents with limitations are real and common. These may be undesired physical limitations as well as self-imposed rational limitations, such as abstraction and approximation techniques, used to make learning tractable. This article explores the interactions of these two important concepts: equilibria and limitations in learning. We introduce the question of whether equilibria continue to exist when agents have limitations. We look at the general effects limitations can have on agent behavior, and define a natural extension of equilibria that accounts for these limitations. Using this formalization, we make three major contributions: (i) a counterexample for the general existence of equilibria with limitations, (ii) sufficient conditions on limitations that preserve their existence, (iii) three general classes of games and limitations that satisfy these conditions. We then present empirical results from a specific multiagent learning algorithm applied to a specific instance of limited agents. These results demonstrate that learning with limitations is feasible, when the conditions outlined by our theoretical analysis hold.

Developmental programming: a new frontier for the poultry industry?

Increasing evidence that the maternal environment influences the programming of developing embryos and fetuses through epigenetic mechanisms has significant potential application in the broiler industry. The broiler breeder hen is subjected to restricted-feeding regimes to maximise egg quantity and quality, but the genetically high-intake potential of these birds makes this regime a stressful one. We propose that this stress is signalled to the developing embryo via changes in yolk composition as an evolutionary adaptation to changing environments, and that exposure to high levels of corticosteroids in ovo is associated with developmental reprogramming, which has effects on the behaviour, health and growth of the progeny. The present paper describes some preliminary results from a series of trials designed to elucidate the relationship between breeder hen diet and egg composition, and the growth, behaviour and immune function of the progeny. We conclude that manipulation of the breeder hen diet is an untapped opportunity to maintain the competitiveness of the chicken meat industry and further, that achieving improved productivity by this means may be compatible with improved animal welfare outcomes for the hen and her progeny.

A model for analyzing the aging of local populations: Illinois counties between 1950 and 1970

The article develops a model which explains the aging of local populations, expressed in terms of percent of the population over sixty-five years, in terms of the net movement of the young and the elderly, as well as socio-economic characteristics of the local population. The model is illustrated for data from Illinois counties for the period 1950-1960 and 1960-1970. The model represents the data for the 1960's more accurately than the data for the 1950's.

Primary health care for children of mothers with intellectual limitations

Intellectually impaired mothers can present a challenge to pediatric nurses. The lack of basic organizational and decision-making skills needed to provide positive nurturing can interfere with the health and development of their children. This descriptive study of the well child needs of 62 two-year-old children from low income families found that children of mothers with intellectual limitations had more frequent referrals for medical or developmental problems, accidents, and problems with hygiene and discipline than children of mothers without intellectual limitations. The needs of families with an intellectually limited mother must be recognized and evaluated if appropriate health care of children is to be accomplished.

FC18-02 - Cognitive distortions among online gamblers

Addictive disorders are being increasingly influenced by technology and one of the most recent developments is for gamblers to access games via the Internet. Prevalence data show that up to 10% of the population gamble online and studies have consistently indicated that Internet gamblers are particularly susceptible to developing gambling problems. Therefore, the purpose of this study was to explore differences between Internet and non-Internet gamblers to help determine why online gamblers are more likely to have gambling problems. Three hundred and seventy four participants (143 online gamblers, 172 males) from a large Canadian university completed an online questionnaire to investigate demographic, medium-related, comorbid psychological and cognitive factors with strong empirical support for contributing to problem gambling severity. Variables that significantly differentiated Internet and non-Internet gamblers in a univariate analyses were entered into a logistic regression to predict online gambling. A test of the full model was statistically significant, correctly classifying 77% of gamblers (64% of Internet gamblers and 85% of non-Internet gamblers). Cognitive distortions made an independent contribution to predicting Internet gamblers from those that had never wagered online. A hierarchical linear regression analysis revealed that cognitive distortions added significantly to problem gambling severity among online gamblers after controlling for other contributing variables. The findings have implications for clinicians working with Internet gamblers to specifically address thoughts related to luck, perseverance and illusion of control. As gambling technologies change and evolve, research needs to inform practice by identifying possible causal factors contributing to problem severity.

TREATMENT OF BREAST CANCER: IMO STATE NIGERIA VERSUS INDIANA, USA WOMEN - COMPARATIVE ANALYTIC STUDY

BACKGROUND

Women with breast cancer undergo multimodal treatment for best outcome. This study seeks to identify the treatment challenges for such women in Imo State, Nigeria vis-à-vis similar women in Indiana USA. We compared the treatment modalities of both groups; noting predictors of compliance for subsequent action.

SETTING

Federal Medical Centre, Owerri; Imo State, Imo State University, Orlu, Nigeria and Indiana University Hospital, Indiana, USA.

DESIGN

A retrospective study.

METHODOLOGY

From 2000-2013, 100 randomly pulled charts of patients treated for pathologically confirmed breast cancer in Imo, Nigeria Federal Medical Centre Owerri, Imo State University Hospital; and Indiana University Hospital U.S. respectively were reviewed. The demographics, clinical and pathological data of the patients with confirmed breast cancer were obtained. The data were formatted and analyzed with SPSS version 16.0. The clinical features, management options, outcomes and specific features were compared for both groups using Wilcoxon Rank Sum tests (age, parity) and chi-square tests for all other variables. A 5% significance level was used for all tests.

RESULTS

One hundred patients were included for each group. The mean/minimum ages; Imo, Nigeria 41.7/21 (SD/SE 15.3/1.5) vs. Indiana, U.S.56.4/29 (SD 12.4/SE 1.2) p<0.0001. Histology for Indiana USA women was predominantly ductal carcinoma in situ (DCIS) P<0.0001 while that of Imo, Nigeria was invasive ductal carcinoma inflammatory cancer P<0.0326. Women in both locations received chemotherapy and surgery. Imo women received less radiotherapy. Toxicity from chemotherapy remained constant features for both groups, P<0.0001. In Indiana USA, the 5year survival exceeded 85%; In Imo Nigeria it was 10%. This study showed that Women on both locations who were likely to be compliant were those receiving mastectomy; Imo, Nigeria 44(56%) <0.013 vs. Indiana, U.S. 74(80%) p<0.0186; women with cosmesis given; Imo, Nigeria 41(42%) vs. Indiana, U.S. 91 (94%) p<0.0001. Sample sizes were inadequate to perform multivariable models.

CONCLUSION

The multimodal treatment regimen implied that there was need for an algorithm protocol for breast cancer women. Thus the need to improve the quality of treatment particularly in Nigeria by improved treatment documentation to overcome key barriers involving information exchange.