1
|
Im J, Lee J, Lee S, Kwon HY. Data pipeline for real-time energy consumption data management and prediction. Front Big Data 2024; 7:1308236. [PMID: 38562648 PMCID: PMC10983847 DOI: 10.3389/fdata.2024.1308236] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Accepted: 03/01/2024] [Indexed: 04/04/2024] Open
Abstract
With the increasing utilization of data in various industries and applications, constructing an efficient data pipeline has become crucial. In this study, we propose a machine learning operations-centric data pipeline specifically designed for an energy consumption management system. This pipeline seamlessly integrates the machine learning model with real-time data management and prediction capabilities. The overall architecture of our proposed pipeline comprises several key components, including Kafka, InfluxDB, Telegraf, Zookeeper, and Grafana. To enable accurate energy consumption predictions, we adopt two time-series prediction models, long short-term memory (LSTM), and seasonal autoregressive integrated moving average (SARIMA). Our analysis reveals a clear trade-off between speed and accuracy, where SARIMA exhibits faster model learning time while LSTM outperforms SARIMA in prediction accuracy. To validate the effectiveness of our pipeline, we measure the overall processing time by optimizing the configuration of Telegraf, which directly impacts the load in the pipeline. The results are promising, as our pipeline achieves an average end-to-end processing time of only 0.39 s for handling 10,000 data records and an impressive 1.26 s when scaling up to 100,000 records. This indicates 30.69-90.88 times faster processing compared to the existing Python-based approach. Additionally, when the number of records increases by ten times, the increased overhead is reduced by 3.07 times. This verifies that the proposed pipeline exhibits an efficient and scalable structure suitable for real-time environments.
Collapse
Affiliation(s)
- Jeonghwan Im
- Graduate School of Data Science, Seoul National University of Science and Technology, Seoul, Republic of Korea
| | - Jaekyu Lee
- Graduate School of Data Science, Seoul National University of Science and Technology, Seoul, Republic of Korea
| | - Somin Lee
- Department of Global Technology Management, Seoul National University of Science and Technology, Seoul, Republic of Korea
| | - Hyuk-Yoon Kwon
- Graduate School of Data Science, Seoul National University of Science and Technology, Seoul, Republic of Korea
| |
Collapse
|
2
|
Artificial Neural Network Training Using Structural Learning with Forgetting for Parameter Analysis of Injection Molding Quality Prediction. INFORMATION 2022. [DOI: 10.3390/info13100488] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
The analysis of influential machine parameters can be useful to plan and design a plastic injection molding process. However, current research in parameter analysis is mostly based on computer-aided engineering (CAE) or simulation which have been demonstrated to be inadequate for analyzing complex behavioral changes in the real injection molding process. More advanced approaches using machine learning technology specifically with artificial neural networks (ANNs) brought promising results in terms of prediction accuracy. Nevertheless, the black box and distributed representation of ANN prevent humans from gaining an insight into which process parameters give a significant influence on the final prediction output. Therefore, in this paper, we develop a simpler ANN model by using structural learning with forgetting (SLF) as the algorithm for the training process. Instead of typical backpropagation which generated a fully connected layer of the ANN model, SLF only reveals the important neurons and connections. Hence, the training process of SLF leaves only influential connections and neurons. Since each of the neurons specifically on the input layer represent each of the injection molding parameters, the ANN-SLF model can be further investigated to determine the influential process parameters. By applying SLF to the ANN training process, this experiment has successfully extracted a set of significant injection molding process parameters.
Collapse
|
3
|
Monitoring System for Tracking a PV Generator in an Experimental Smart Microgrid: An Open-Source Solution. SUSTAINABILITY 2021. [DOI: 10.3390/su13158182] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
Smart grids and smart microgrids (SMGs) require proper monitoring for their operation. To this end, measuring, data acquisition, and storage, as well as remote online visualization of real-time information, must be performed using suitable equipment. An experimental SMG is being deployed that combines photovoltaics and the energy carrier hydrogen through the interconnection of photovoltaic panels, electrolyser, fuel cell, and load around a voltage bus powered by a lithium battery. This paper presents a monitoring system based on open-source hardware and software for tracking the temperature of the photovoltaic generator in such an SMG. In fact, the increases in temperature in PV modules lead to a decrease in their efficiency, so this parameter needs to be measured in order to monitor and evaluate the operation. Specifically, the developed monitoring system consists of a network of digital temperature sensors connected to an Arduino microcontroller, which feeds the acquired data to a Raspberry Pi microcomputer. The latter is accessed by a cloud-enabled user/operator interface implemented in Grafana. The monitoring system is expounded and experimental results are reported to validate the proposal.
Collapse
|
4
|
Lavalle A, Teruel MA, Maté A, Trujillo J. Fostering Sustainability through Visualization Techniques for Real-Time IoT Data: A Case Study Based on Gas Turbines for Electricity Production. SENSORS 2020; 20:s20164556. [PMID: 32823870 PMCID: PMC7472268 DOI: 10.3390/s20164556] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/09/2020] [Revised: 08/10/2020] [Accepted: 08/13/2020] [Indexed: 11/29/2022]
Abstract
Improving sustainability is a key concern for industrial development. Industry has recently been benefiting from the rise of IoT technologies, leading to improvements in the monitoring and breakdown prevention of industrial equipment. In order to properly achieve this monitoring and prevention, visualization techniques are of paramount importance. However, the visualization of real-time IoT sensor data has always been challenging, especially when such data are originated by sensors of different natures. In order to tackle this issue, we propose a methodology that aims to help users to visually locate and understand the failures that could arise in a production process.This methodology collects, in a guided manner, user goals and the requirements of the production process, analyzes the incoming data from IoT sensors and automatically derives the most suitable visualization type for each context. This approach will help users to identify if the production process is running as well as expected; thus, it will enable them to make the most sustainable decision in each situation. Finally, in order to assess the suitability of our proposal, a case study based on gas turbines for electricity generation is presented.
Collapse
Affiliation(s)
- Ana Lavalle
- Lucentia Research, DLSI, University of Alicante, Carretera San Vicente del Raspeig s/n, 03690 Alicante, Spain; (M.A.T.); (A.M.); (J.T.)
- Lucentia Lab, Avda. Pintor Pérez Gil, N-16, 03540 Alicante, Spain
- Correspondence:
| | - Miguel A. Teruel
- Lucentia Research, DLSI, University of Alicante, Carretera San Vicente del Raspeig s/n, 03690 Alicante, Spain; (M.A.T.); (A.M.); (J.T.)
- Lucentia Lab, Avda. Pintor Pérez Gil, N-16, 03540 Alicante, Spain
| | - Alejandro Maté
- Lucentia Research, DLSI, University of Alicante, Carretera San Vicente del Raspeig s/n, 03690 Alicante, Spain; (M.A.T.); (A.M.); (J.T.)
- Lucentia Lab, Avda. Pintor Pérez Gil, N-16, 03540 Alicante, Spain
| | - Juan Trujillo
- Lucentia Research, DLSI, University of Alicante, Carretera San Vicente del Raspeig s/n, 03690 Alicante, Spain; (M.A.T.); (A.M.); (J.T.)
- Lucentia Lab, Avda. Pintor Pérez Gil, N-16, 03540 Alicante, Spain
| |
Collapse
|
5
|
Sharma R, Jabbour CJC, Lopes de Sousa Jabbour AB. Sustainable manufacturing and industry 4.0: what we know and what we don't. JOURNAL OF ENTERPRISE INFORMATION MANAGEMENT 2020. [DOI: 10.1108/jeim-01-2020-0024] [Citation(s) in RCA: 59] [Impact Index Per Article: 14.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/05/2023]
Abstract
PurposeThe emergence the fourth industrial revolution, known as well as industry 4.0, and its applications in the manufacturing sector ushered a new era for the business entities. It not only promises enhancement in operational efficiency but also magnify sustainable operations practices. This current paper provides a thorough bibliometric and network analysis of more than 600 articles highlighting the benefits in favor of the sustainability dimension in the industry 4.0 paradigm.Design/methodology/approachThe analysis begins by identifying over 1,000 published articles in Scopus, which were then refined to works of proven influence and those authored by influential researchers. Using rigorous bibliometric software, established and emergent research clusters were identified for intellectual network analysis, identification of key research topics, interrelations and collaboration patterns.FindingsThis bibliometric analysis of the field helps graphically to illustrate the publications evolution over time and identify areas of current research interests and potential directions for future research. The findings provide a robust roadmap for mapping the research territory in the field of industry 4.0 and sustainability.Originality/valueAs the literature on sustainability and industry 4.0 expands, reviews capable of systematizing the main trends and topics of this research field are relevant.
Collapse
|
6
|
Abstract
Radio frequency identification (RFID) is an automated identification technology that can be utilized to monitor product movements within a supply chain in real-time. However, one problem that occurs during RFID data capturing is false positives (i.e., tags that are accidentally detected by the reader but not of interest to the business process). This paper investigates using machine learning algorithms to filter false positives. Raw RFID data were collected based on various tagged product movements, and statistical features were extracted from the received signal strength derived from the raw RFID data. Abnormal RFID data or outliers may arise in real cases. Therefore, we utilized outlier detection models to remove outlier data. The experiment results showed that machine learning-based models successfully classified RFID readings with high accuracy, and integrating outlier detection with machine learning models improved classification accuracy. We demonstrated the proposed classification model could be applied to real-time monitoring, ensuring false positives were filtered and hence not stored in the database. The proposed model is expected to improve warehouse management systems by monitoring delivered products to other supply chain partners.
Collapse
|
7
|
Improvement of Temperature Control Performance of Thermoelectric Dehumidifier Used Industry 4.0 by the SF-PI Controller. Processes (Basel) 2019. [DOI: 10.3390/pr7020098] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
This paper proposes the series connected fuzzy-proportional integral (SF-PI) controller, which is composed of the fuzzy control and the PI controller to improve temperature control performance of dehumidifier using a thermoelectric element. The control of conventional PI controller usually uses fixed gain. For that reason, it is limited in achieving satisfactory control performance in both transient-state and steady-state. The fuzzy control within SF-PI controller adjusts the input value of PI controller according to operating condition. The PI controller within the SF-PI controller controls the temperature of the thermoelectric element using that value. The SF-PI controller can achieve more accurate temperature control than a conventional PI controller for that reason. The SF-PI controller has been tested for various indoor environmental conditions such as temperature and relative humidity conditions. The average temperature error of the SF-PI controller between the reference temperature and the thermoelectric element temperature is 22% of traditional PI’s value and consumption power is reduced by about 10%. Therefore, the SF-PI controller proposed in this paper can improved the performance of temperature control of dehumidifier using thermoelectric element. The power consumed by buildings accounts for a significant portion of the total power consumption, of which the air conditioner represents the largest energy consumer. In this paper, it is possible to reduce the energy consumption by improving the performance of the dehumidifier, one of the air conditioners, and it can be applied to various control fields in the future.
Collapse
|
8
|
An Affordable Fast Early Warning System for Edge Computing in Assembly Line. APPLIED SCIENCES-BASEL 2018. [DOI: 10.3390/app9010084] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Maintaining product quality is essential for smart factories, hence detecting abnormal events in assembly line is important for timely decision-making. This study proposes an affordable fast early warning system based on edge computing to detect abnormal events during assembly line. The proposed model obtains environmental data from various sensors including gyroscopes, accelerometers, temperature, humidity, ambient light, and air quality. The fault model is installed close to the facilities, so abnormal events can be timely detected. Several performance evaluations are conducted to obtain the optimal scenario for utilizing edge devices to improve data processing and analysis speed, and the final proposed model provides the highest accuracy in terms of detecting abnormal events compared to other classification models. The proposed model was tested over four months of operation in a Korean automobile parts factory, and provided significant benefits from monitoring assembly line, as well as classifying abnormal events. The model helped improve decision-making by reducing or preventing unexpected losses due to abnormal events.
Collapse
|
9
|
OpenZmeter: An Efficient Low-Cost Energy Smart Meter and Power Quality Analyzer. SUSTAINABILITY 2018. [DOI: 10.3390/su10114038] [Citation(s) in RCA: 34] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Power quality and energy consumption measurements support providers and energy users with solutions for acquiring and reporting information about the energy supply for residential, commercial, and industrial sectors. In particular, since the average number of electronic devices in homes increases year by year and their sensitivity is very high, it is not only important to monitor the total energy consumption, but also the quality of the power supplied. However, in practice, end-users do not have information about the energy consumption in real-time nor about the quality of the power they receive, because electric energy meters are too expensive and complex to be handled. In order to overcome these inconveniences, an innovative, open source, low-cost, precise, and reliable power and electric energy meter is presented that can be easily installed and managed by any inexperienced user at their own home in urban or rural areas. The system was validated in a real house over a period of two weeks, showing interesting results and findings which validate our proposal.
Collapse
|
10
|
Analysis of Influencing Factors of Big Data Adoption in Chinese Enterprises Using DANP Technique. SUSTAINABILITY 2018. [DOI: 10.3390/su10113956] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Globally, many enterprises are currently focusing on big data technology to improve their performance and operations. Recent literature points out several factors that influence the adoption of big data. However, enterprises often resist using the business value of big data due to a lack of knowledge. The purpose of this study is to investigate the factors influencing big data adoption by Chinese enterprises and to develop an indicator system based on the Motivation–Opportunity–Ability (MOA) model. Moreover, the Decision-Making Trial and Evaluation Laboratory (DEMATEL) method is used to construct a network relationship map and to analyze its effects. Using the DEMATEL-based Analytic Network Process (ANP) (DANP) method to identify the weight distribution of index, this study quantitatively evaluates the influencing factors. The results show that leadership support, perceived usefulness, financial support, data resources, industrial development, data talents, and technical capability are key elements affecting the application of big data. Accordingly, some targeted suggestions are proposed.
Collapse
|
11
|
Syafrudin M, Alfian G, Fitriyani NL, Rhee J. Performance Analysis of IoT-Based Sensor, Big Data Processing, and Machine Learning Model for Real-Time Monitoring System in Automotive Manufacturing. SENSORS (BASEL, SWITZERLAND) 2018; 18:E2946. [PMID: 30181525 PMCID: PMC6164307 DOI: 10.3390/s18092946] [Citation(s) in RCA: 138] [Impact Index Per Article: 23.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/03/2018] [Revised: 08/30/2018] [Accepted: 09/03/2018] [Indexed: 12/20/2022]
Abstract
With the increase in the amount of data captured during the manufacturing process, monitoring systems are becoming important factors in decision making for management. Current technologies such as Internet of Things (IoT)-based sensors can be considered a solution to provide efficient monitoring of the manufacturing process. In this study, a real-time monitoring system that utilizes IoT-based sensors, big data processing, and a hybrid prediction model is proposed. Firstly, an IoT-based sensor that collects temperature, humidity, accelerometer, and gyroscope data was developed. The characteristics of IoT-generated sensor data from the manufacturing process are: real-time, large amounts, and unstructured type. The proposed big data processing platform utilizes Apache Kafka as a message queue, Apache Storm as a real-time processing engine and MongoDB to store the sensor data from the manufacturing process. Secondly, for the proposed hybrid prediction model, Density-Based Spatial Clustering of Applications with Noise (DBSCAN)-based outlier detection and Random Forest classification were used to remove outlier sensor data and provide fault detection during the manufacturing process, respectively. The proposed model was evaluated and tested at an automotive manufacturing assembly line in Korea. The results showed that IoT-based sensors and the proposed big data processing system are sufficiently efficient to monitor the manufacturing process. Furthermore, the proposed hybrid prediction model has better fault prediction accuracy than other models given the sensor data as input. The proposed system is expected to support management by improving decision-making and will help prevent unexpected losses caused by faults during the manufacturing process.
Collapse
Affiliation(s)
- Muhammad Syafrudin
- Department of Industrial and Systems Engineering, Dongguk University, Seoul 100-715, Korea.
| | - Ganjar Alfian
- u-SCM Research Center, Nano Information Technology Academy, Dongguk University, Seoul 100-715, Korea.
| | - Norma Latif Fitriyani
- Department of Industrial and Systems Engineering, Dongguk University, Seoul 100-715, Korea.
| | - Jongtae Rhee
- Department of Industrial and Systems Engineering, Dongguk University, Seoul 100-715, Korea.
| |
Collapse
|
12
|
Alfian G, Syafrudin M, Ijaz MF, Syaekhoni MA, Fitriyani NL, Rhee J. A Personalized Healthcare Monitoring System for Diabetic Patients by Utilizing BLE-Based Sensors and Real-Time Data Processing. SENSORS 2018; 18:s18072183. [PMID: 29986473 PMCID: PMC6068508 DOI: 10.3390/s18072183] [Citation(s) in RCA: 61] [Impact Index Per Article: 10.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/04/2018] [Revised: 06/25/2018] [Accepted: 07/05/2018] [Indexed: 12/18/2022]
Abstract
Current technology provides an efficient way of monitoring the personal health of individuals. Bluetooth Low Energy (BLE)-based sensors can be considered as a solution for monitoring personal vital signs data. In this study, we propose a personalized healthcare monitoring system by utilizing a BLE-based sensor device, real-time data processing, and machine learning-based algorithms to help diabetic patients to better self-manage their chronic condition. BLEs were used to gather users’ vital signs data such as blood pressure, heart rate, weight, and blood glucose (BG) from sensor nodes to smartphones, while real-time data processing was utilized to manage the large amount of continuously generated sensor data. The proposed real-time data processing utilized Apache Kafka as a streaming platform and MongoDB to store the sensor data from the patient. The results show that commercial versions of the BLE-based sensors and the proposed real-time data processing are sufficiently efficient to monitor the vital signs data of diabetic patients. Furthermore, machine learning–based classification methods were tested on a diabetes dataset and showed that a Multilayer Perceptron can provide early prediction of diabetes given the user’s sensor data as input. The results also reveal that Long Short-Term Memory can accurately predict the future BG level based on the current sensor data. In addition, the proposed diabetes classification and BG prediction could be combined with personalized diet and physical activity suggestions in order to improve the health quality of patients and to avoid critical conditions in the future.
Collapse
Affiliation(s)
- Ganjar Alfian
- U-SCM Research Center, Nano Information Technology Academy, Dongguk University, Seoul 100-715, Korea.
| | - Muhammad Syafrudin
- Department of Industrial and Systems Engineering, Dongguk University, Seoul 100-715, Korea.
| | - Muhammad Fazal Ijaz
- Department of Industrial and Systems Engineering, Dongguk University, Seoul 100-715, Korea.
| | - M Alex Syaekhoni
- Department of Industrial and Systems Engineering, Dongguk University, Seoul 100-715, Korea.
| | - Norma Latif Fitriyani
- Department of Industrial and Systems Engineering, Dongguk University, Seoul 100-715, Korea.
| | - Jongtae Rhee
- Department of Industrial and Systems Engineering, Dongguk University, Seoul 100-715, Korea.
| |
Collapse
|