Guiding the retraining of convolutional neural networks against adversarial inputs

Background

When using deep learning models, one of the most critical vulnerabilities is their exposure to adversarial inputs, which can cause wrong decisions (e.g., incorrect classification of an image) with minor perturbations. To address this vulnerability, it becomes necessary to retrain the affected model against adversarial inputs as part of the software testing process. In order to make this process energy efficient, data scientists need support on which are the best guidance metrics for reducing the adversarial inputs to create and use during testing, as well as optimal dataset configurations.

Aim

We examined six guidance metrics for retraining deep learning models, specifically with convolutional neural network architecture, and three retraining configurations. Our goal is to improve the convolutional neural networks against the attack of adversarial inputs with regard to the accuracy, resource utilization and execution time from the point of view of a data scientist in the context of image classification.

Method

We conducted an empirical study using five datasets for image classification. We explore: (a) the accuracy, resource utilization, and execution time of retraining convolutional neural networks with the guidance of six different guidance metrics (neuron coverage, likelihood-based surprise adequacy, distance-based surprise adequacy, DeepGini, softmax entropy and random), (b) the accuracy and resource utilization of retraining convolutional neural networks with three different configurations (one-step adversarial retraining, adversarial retraining and adversarial fine-tuning).

Results

We reveal that adversarial retraining from original model weights, and by ordering with uncertainty metrics, gives the best model w.r.t. accuracy, resource utilization, and execution time.

Conclusions

Although more studies are necessary, we recommend data scientists use the above configuration and metrics to deal with the vulnerability to adversarial inputs of deep learning models, as they can improve their models against adversarial inputs without using many inputs and without creating numerous adversarial inputs. We also show that dataset size has an important impact on the results.

Mining user reviews of COVID contact-tracing apps: An exploratory analysis of nine European apps

CONTEXT

More than 78 countries have developed COVID contact-tracing apps to limit the spread of coronavirus. However, many experts and scientists cast doubt on the effectiveness of those apps. For each app, a large number of reviews have been entered by end-users in app stores.

OBJECTIVE

Our goal is to gain insights into the user reviews of those apps, and to find out the main problems that users have reported. Our focus is to assess the "software in society" aspects of the apps, based on user reviews.

METHOD

We selected nine European national apps for our analysis and used a commercial app-review analytics tool to extract and mine the user reviews. For all the apps combined, our dataset includes 39,425 user reviews.

RESULTS

Results show that users are generally dissatisfied with the nine apps under study, except the Scottish ("Protect Scotland") app. Some of the major issues that users have complained about are high battery drainage and doubts on whether apps are really working.

CONCLUSION

Our results show that more work is needed by the stakeholders behind the apps (e.g., app developers, decision-makers, public health experts) to improve the public adoption, software quality and public perception of these apps.

Impact of Students' Presence and Course Participation on Learning Outcome in Co-Operative Online-based Courses

Socio-constructive instructional designs for online-based learning focus on interaction and communication of students to allow in-depth learning. The objective of this study is to analyze whether increased interaction of students in online-based learning settings may contribute to better outcome. We developed indicators for presence, participation, and interactivity of students. We extracted log data from the learning management system for 31 students in 10 online courses (n=123 course attendances). We correlated indicators to final grades and also applied a decision tree based machine learning approach. We found only weak to moderate correlations between the indicators and final grades, but acceptable results concerning prediction of students' success based on the indicators. Our results support the theory that student presence and participation in online-based courses is related to learning outcome.

Building a Community of Inquiry Within an Online-Based Health Informatics Program: Instructional Design and Lessons Learned

The rapid medical and technological progress and the resulting need for life-long learning lead to the proliferation of online-based programs in health informatics for health care and health informatics professionals. Effective learning in these online-based programs demands close interaction and cooperation between students and instructors. In such cooperative online-based programs, special emphasis is thus put on fostering a community of inquiry between all participants. The online-based master program on Health Information Management at the University UMIT is firmly based on the constructivist theory of situated learning in such a community of inquiry. Online activities in this master program are designed to promote social presence, cognitive presence and teacher presence as preconditions for successful learning. To verify whether these three forms of presences are visible and whether learning was successful, the first four six-week online-based modules with overall 60 participants were analyzed in depth, combining data from community of inquiry surveys, student evaluations, workload assessments and final assessment results. Results indicate that it was possible to build a community of inquiry (with scores of 4.2 and higher) and to foster effective learning. We conclude with recommendations on how to build a community of inquiry in online-based cooperative learning and also discuss some constraints.

Developing and Evaluating Collaborative Online-Based Instructional Designs in Health Information Management

The number of students enrolled in online courses is increasing steadily. Distance education offers many advantages, but also has inherent challenges. Successful distance education needs a thoughtfully designed instructional strategy where students are supported to actively create knowledge. We present the design and evaluation of three online-based courses in health informatics. The courses were based on a collaborative instructional strategy. The evaluation comprised workload analysis, student evaluation, student interviews and student reflections. Students expressed high satisfaction with online learning, despite a high workload, and high perceived learning outcomes. Using the Community of Inquiry framework as reference, we found very high levels of teaching presence, social presence and cognitive presence. Summarizing, we found that the chosen instructional strategy supported student-centered, collaborative learning. We conclude by presenting lesson learned for online-based instructional design.