KR-102962326-B1 - VIRTUAL SENSOR SYSTEM FOR DIGITAL TWIN APPLICATION

KR102962326B1KR 102962326 B1KR102962326 B1KR 102962326B1KR-102962326-B1

Abstract

The virtual sensor system for digital twin applications of the present invention is characterized by comprising: an edge gateway that operates a virtual sensor to collect data collected from a real sensor in a real environment and apply the collected data to a virtual sensor model to configure a digital twin environment; and a virtual sensor framework that learns a virtual sensor model using data measured from a real sensor from the edge gateway and deploys the virtual sensor model to the edge gateway.

Inventors

최원규
김세한
박현
정재영
권성수
박수진
배지훈
한유진

Assignees

한국전자통신연구원

Dates

Publication Date: 20260508
Application Date: 20230329

Claims (19)

An edge gateway that operates a virtual sensor to configure a digital twin environment by collecting data from a real sensor in a real environment and applying the collected data to a virtual sensor model; and A virtual sensor framework that trains a virtual sensor model using data measured from the physical sensor from the edge gateway and deploys the virtual sensor model to the edge gateway, The above virtual sensor framework is A data database that stores preprocessed data from the edge gateway; A virtual sensor learning model module that generates the virtual sensor by training the virtual sensor model with data stored in the above data database; A generated signal error analysis module that corrects the error of the virtual sensor by comparing the data stored in the above data database with the data of the virtual sensor input from the above virtual sensor learning model module; and A virtual sensor system for a digital twin application characterized by including a data and error monitoring visualization engine that monitors data stored in the data database and errors received from the generated signal error analysis module to determine whether an update to the virtual sensor model is required, and requests data collection for learning from the edge gateway based on the determination result.
In paragraph 1, the virtual sensor model A virtual sensor system for digital twin applications characterized by predicting time-series data based on the correlation characteristics of data collected from the physical sensor and data collected from the physical sensor at the current time, in order to configure a digital twin environment in the bridge field.
In paragraph 1, the virtual sensor model A virtual sensor system for a digital twin application characterized by predicting time-series data of an area where the actual sensor is not installed by learning based on data from the actual sensor that has a mutual correlation.
In paragraph 1, the edge gateway is A data database for storing data collected from the above physical sensor; A data preprocessing module that preprocesses data stored in the above data database; A virtual sensor database storing the above virtual sensor model; and A virtual sensor system for a digital twin application characterized by including a virtual sensor operation module that operates the virtual sensor through the virtual sensor model using data preprocessed in the data preprocessing module.
In paragraph 4, the edge gateway A virtual sensor system for a digital twin application, characterized by further including an abnormal signal detection module that requests an update of the virtual sensor model to the virtual sensor framework according to the changed data distribution when the data preprocessed by the data preprocessing module is determined to have changed data characteristics.
In paragraph 5, the above abnormal signal detection module is A virtual sensor system for a digital twin application characterized by transmitting an alarm to an administrator to check for abnormalities in the actual sensor if the data preprocessed in the above data preprocessing module is an abnormal signal.
In paragraph 6, the above abnormal signal detection module is It includes a deep learning model for detecting abnormal signals, and the RNN part based on a recurrent neural network autoencoder of the deep learning model A virtual sensor system for a digital twin application characterized by measuring the error between time series data measured by the above-mentioned real sensor and time series data restored by a deep learning model for detecting abnormal signals, comparing it with a preset reference value, determining it as an abnormal signal if the compared error is greater than the reference value, and determining it as a normal signal if it is smaller than the reference value.
delete
In paragraph 1, the virtual sensor learning model module is It includes an encoder that receives time series data of a first physical sensor as input and a decoder that receives time series data of a second physical sensor as input, A virtual sensor system for a digital twin application, characterized by setting the time series data of the first physical sensor as the input of the encoder, setting the time series data of the second physical sensor as the input of the decoder, setting the result of applying Teacher Forcing to the input sequence of the decoder as the target sequence of the decoder, and inputting the last internal state of the encoder as the initial state of the decoder to proceed with the learning of the virtual sensor model.
In paragraph 1, the virtual sensor model It includes a recurrent neural network-based encoder and a recurrent neural network-based decoder, The encoder and the decoder are composed of a stacked cell recurrent neural network, and A virtual sensor system for digital twin applications characterized in that the cell recurrent neural network is implemented as at least one of SimpleRNN (Recurrent Neural Network), LSTM (Long Short Term Memory), and GRU (Gated Recurrent Unit).
In item 10, the above virtual sensor model The final hidden state of the encoder is stored in the Input Representation (state) layer and then input into the initial state of the decoder; the input sequence of the decoder is shifted by one time step to generate a target sequence, and then the output signal passing through the time-distributed fully connected layer is set as the target signal to proceed with learning. A virtual sensor system for a digital twin application, characterized in that the above time distribution fully connected layer performs learning so that the decoder can know what the next target signal is at each time step.
In Clause 10, the above-mentioned generated signal error analysis module is A virtual sensor system for a digital twin application characterized by calculating the absolute value of the error between the data of the virtual sensor and the data of the real sensor, calculating the absolute error average over a set period, and then performing an exponential moving average on the absolute error average to monitor the error of the virtual sensor.
A data database that collects data from physical sensors; A data preprocessing module that preprocesses data stored in the above data database; A virtual sensor database that stores virtual sensor models; An anomaly detection module that requests an update of the virtual sensor model to the virtual sensor framework according to the changed data distribution when it is determined that the data preprocessed by the data preprocessing module has changed data characteristics; and It includes a virtual sensor operation module that operates the virtual sensor through the virtual sensor model using data preprocessed in the data preprocessing module above, and The above virtual sensor framework is A data database that stores preprocessed data from the edge gateway; A virtual sensor learning model module that generates the virtual sensor by training the virtual sensor model with data stored in the above data database; A generated signal error analysis module that corrects the error of the virtual sensor by comparing the data stored in the above data database with the data of the virtual sensor input from the above virtual sensor learning model module; and A data and error monitoring visualization engine comprising: a data and error monitoring visualization engine that monitors data stored in the data database and errors received from the generated signal error analysis module to determine whether an update to the virtual sensor model is required, and requests data collection for learning from the edge gateway based on the determination result. A virtual sensor system for digital twin applications characterized by the following.
In Clause 13, the above abnormal signal detection module is It includes a deep learning model for detecting abnormal signals, and the RNN part based on a recurrent neural network autoencoder of the deep learning model A virtual sensor system for a digital twin application characterized by measuring the error between time series data measured by the above-mentioned real sensor and time series data restored by a deep learning model for detecting abnormal signals, comparing it with a preset reference value, determining it as an abnormal signal if the compared error is greater than the reference value, and determining it as a normal signal if it is smaller than the reference value.
A data database that stores preprocessed data from the edge gateway; A virtual sensor learning model module that generates a virtual sensor by training a virtual sensor model with data stored in the above data database; and It includes a generated signal error analysis module that corrects the error of the virtual sensor by comparing the data stored in the above data database with the data of the virtual sensor input from the above virtual sensor learning model module, A virtual sensor system for a digital twin application, characterized by further including a data and error monitoring visualization engine that monitors data stored in the data database and errors received from the generated signal error analysis module to determine whether an update of the virtual sensor model is required, and requests data collection for learning from the edge gateway according to the determination result.
delete
In item 15, the virtual sensor learning model module above is It includes an encoder that receives time series data of a first physical sensor as input and a decoder that receives time series data of a second physical sensor as input, A virtual sensor system for a digital twin application, characterized by setting the time series data of the first physical sensor as the input of the encoder, setting the time series data of the second physical sensor as the input of the decoder, setting the result of applying Teacher Forcing to the input sequence of the decoder as the target sequence of the decoder, and inputting the last internal state of the encoder as the initial state of the decoder to proceed with the learning of the virtual sensor model.
In Clause 17, the above virtual sensor model It includes a recurrent neural network-based encoder and a recurrent neural network-based decoder, The encoder and the decoder are composed of a stacked cell recurrent neural network, and The final hidden state of the encoder is stored in the Input Representation (state) layer and then input into the initial state of the decoder; the input sequence of the decoder is shifted by one time step to generate a target sequence, and then the output signal passing through the time-distributed fully connected layer is set as the target signal to proceed with learning. A virtual sensor system for a digital twin application, characterized in that the above time distribution fully connected layer performs learning so that the decoder can know what the next target signal is at each time step.
In item 15, the above-mentioned generated signal error analysis module is A virtual sensor system for a digital twin application characterized by calculating the absolute value of the error between the data of the virtual sensor and the data of the real sensor, calculating the absolute error average over a set period, and then performing an exponential moving average on the absolute error average to monitor the error of the virtual sensor.

Description

Virtual Sensor System for Digital Twin Application The present invention relates to a virtual sensor system for digital twin applications. Virtual sensors can be implemented based on mathematical theory or models (First Principle), pure data (Black-box), or a combination of models and data (Gray-box). In addition to these concepts, virtual sensors can also be implemented based on digital twins. Beyond digital twin applications, virtual sensors can be usefully employed in environments where installing and operating sensors is difficult due to economic reasons or physical installation and maintenance. Virtual sensors can be used for purposes such as backup, replacement, observation, and fault detection and diagnosis. In addition, the level of virtual sensor implementation can be divided into four stages. Stage 1 is the stage of implementing a single physical sensor into the same virtual sensor, and Stage 2 is the stage of implementing a homogeneous combined sensor based on several homogeneous sensors and data. This is used as a method to implement a high-reliability virtual sensor. Stage 3 is the stage of implementing a virtual sensor by statically combining different heterogeneous sensors and data, and Stage 4 is the stage of implementing a virtual sensor by dynamically combining different heterogeneous sensors and data. To implement high-reliability virtual sensors, precise environmental modeling is required, along with a large amount of high-fidelity data containing accurate environmental information. To collect this large amount of high-fidelity data, high-reliability IoT sensors are necessary. Sensors such as acceleration sensors, expansion joint sensors, and tilt sensors can be used to monitor bridge safety. However, these sensors are expensive, and significant budgets are required for their installation and maintenance due to harsh external environments. Although various sensors must be installed and operated on bridges to periodically monitor safety, IoT sensor-based bridge safety management is not currently being implemented due to economic constraints. As prior art related to this, U.S. patent application US20210247752A1 covers remote structural health monitoring and fault (defect) state detection. However, it does not provide a solution for estimating sensor values by dynamically fusing heterogeneous data through the combination of a digital twin model and data. delete FIG. 1 is a conceptual diagram of a digital twin technology for bridge applications according to one embodiment of the present invention. FIG. 2 is a drawing showing an example of installing a real bridge environment sensor according to one embodiment of the present invention. FIG. 3 is a diagram showing an example of installation and operation of a bridge environment virtual sensor according to an embodiment of the present invention. FIG. 4 is a drawing showing another example of the installation and operation of a bridge environment virtual sensor according to one embodiment of the present invention. FIG. 5 is a system configuration diagram for creating, managing, and operating a virtual sensor according to an embodiment of the present invention. FIG. 6 is a diagram showing an example of predicting a target signal using a recurrent neural network in a bridge environment according to one embodiment of the present invention. Figure 7 is a detailed structural diagram of the virtual sensor learning model of Figure 6. FIG. 8 is a conceptual diagram of signal generation of an inference model of a virtual sensor output layer (decoder) according to one embodiment of the present invention. FIG. 9 is a configuration diagram of an error analysis model for a virtual sensor generated signal according to one embodiment of the present invention. FIG. 10 is a diagram showing a deep learning model for detecting abnormal signals in sensor data according to one embodiment of the present invention. FIG. 11 is a diagram illustrating an example for detecting abnormal signals in sensor data according to one embodiment of the present invention. The following describes an embodiment of a virtual sensor system for digital twin applications according to an embodiment of the present invention. In this process, the thickness of lines or the size of components depicted in the drawings may be exaggerated for clarity and convenience of explanation. Furthermore, the terms described below are defined considering their functions in the present invention, and these may vary depending on the intention or convention of the user or operator. Therefore, the definitions of these terms should be based on the content throughout this specification. Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the