CN-115713925-B - Wood piano playing detection method based on photoelectric information

CN115713925BCN 115713925 BCN115713925 BCN 115713925BCN-115713925-B

Abstract

The application relates to the technical field of piano teaching, in particular to an wood piano playing detection method based on photoelectric information. The method comprises the steps of installing a sensor, sampling a comparison playing process through the sensor to obtain a photoelectric comparison signal set, respectively carrying out intensity analysis and rhythm analysis on each photoelectric comparison signal to obtain a photoelectric comparison signal analysis result, establishing a playing detection model, sampling an actual playing process through the sensor to obtain a photoelectric actual signal set, inputting the photoelectric actual signal set into the playing detection model, and carrying out intensity detection and rhythm detection on each photoelectric actual signal in the photoelectric actual signal set by taking the photoelectric comparison signal set as a detection basis by the playing detection model, and outputting a detection result about keys based on time sequence in the playing process. The playing detection model provided by the application detects whether the key is accurately pressed or not and the intensity and rhythm of the corresponding photoelectric actual signal after the key is pressed, so that comprehensive detection can be realized.

Inventors

Lou Gangning
Zheng Yuanzhang
LI JUN
Yuan Jingxi
WANG YINHE
ZHU XUSHENG
ZHANG ZONGDA
Xue Xingzheng
WU JIANHAO
JIANG LINYI

Assignees

吉林大学

Dates

Publication Date: 20260505
Application Date: 20221114

Claims (7)

1. An wood piano playing detection method based on photoelectric information, characterized in that the method comprises: s1, installing a sensor according to keys in an wood piano; S2, determining a plurality of target tracks, and sampling a comparison playing process when the target tracks are correctly played by the wood piano through a sensor based on a preset sampling period to obtain a photoelectric comparison signal set about the pressed key based on time sequence in the comparison playing process of each target track; s3, respectively carrying out intensity analysis and rhythm analysis on each photoelectric control signal in the photoelectric control signal set to obtain a photoelectric control signal analysis result, and establishing a playing detection model based on a plurality of target tracks based on the photoelectric control signal analysis result; S4, sampling an actual playing process of the wood piano when the target track is actually played through a sensor based on a preset sampling period, and obtaining a photoelectric actual signal set of the pressed key based on time sequence in the actual playing process; s5, inputting a photoelectric actual signal set to a playing detection model, wherein the playing detection model takes a photoelectric comparison signal set as a detection basis to detect intensity and rhythm of each photoelectric actual signal in the photoelectric actual signal set, and outputs a detection result about keys based on time sequence in the playing process; The step S5 includes: s51, inputting a photoelectric actual signal set into a playing detection model, wherein the playing detection model takes a preset sampling period as a mapping basis between a photoelectric comparison signal set and a photoelectric actual signal set, determines the similarity between each photoelectric comparison signal and a corresponding photoelectric actual signal, and respectively obtains a plurality of similarity values; S52, determining a similarity threshold value, comparing the similarity value with the similarity threshold value respectively, and if the similarity value is larger than the similarity threshold value, confirming that the photoelectric actual signal corresponding to the similarity value is a correct playing signal and correspondingly confirming that the key pressed in the playing process is correct; S53, repeating the step S52 until all similarity values are compared with the similarity threshold value, and outputting detection results comprising keys which are correctly pressed and keys which are incorrectly pressed based on time sequence in the playing process; the step S51 includes: s551, inputting a photoelectric actual signal set into a playing detection model, wherein the playing detection model uses a preset sampling period as a mapping basis between a photoelectric comparison signal set and the photoelectric actual signal set, and determines the strong and weak characteristic similarity and the rhythm characteristic similarity between each photoelectric comparison signal and the corresponding photoelectric actual signal; S552, determining a comprehensive similarity index between each photoelectric control signal and the corresponding photoelectric actual signal according to the strong and weak feature similarity and the rhythm feature similarity between each photoelectric control signal and the corresponding photoelectric actual signal, and respectively obtaining a plurality of similarity values; The calculation formula of the comprehensive similarity index is as follows: ; In the formula, The similarity of the strong and weak features is represented, The similarity of the characteristics of the cadence is indicated, And Weight coefficient sum for respectively representing similarity weight coefficient of strong and weak characteristics and similarity of rhythm characteristics ; If Y is more than or equal to 0.75, confirming that the photoelectric actual signal corresponding to the comprehensive similarity index is a correct playing signal, and correspondingly confirming that the key pressed in the playing process is correct; If Y <0.75, it is confirmed that the photoelectric actual signal corresponding to the integrated similarity index is the wrong playing signal, and accordingly it is confirmed that the key pressed during the playing is wrong.
2. The method for detecting the play of an wood piano based on photoelectric information according to claim 1, wherein said step S1 comprises: S11, installing sensors according to keys in the wood piano, wherein each key corresponds to two sensors; and, the step S2 includes: S21, determining a plurality of target tracks, and sampling a comparison playing process when the target tracks are correctly played by the wood piano through a sensor based on a preset sampling period to obtain an initial photoelectric comparison signal set which is based on time sequence and comprises a plurality of initial photoelectric comparison signals about the pressed keys in the comparison playing process of each target track; Each initial photoelectric control signal comprises initial photoelectric control sub-signals respectively obtained by two sensors corresponding to the key pressed in the playing process in a preset sampling period; S22, sequentially carrying out denoising treatment, standardization treatment and integration treatment on a plurality of initial photoelectric control sub-signals in the initial photoelectric control signal set to obtain a plurality of photoelectric control signals, and correspondingly obtaining a photoelectric control signal set; and, the step S4 includes: S41, sampling an actual playing process of the wood piano when the target track is actually played through a sensor based on a preset sampling period, and obtaining an initial photoelectric actual signal set which is based on time sequence and comprises a plurality of initial photoelectric actual signals and is related to the pressed key in the actual playing process; Each initial photoelectric actual signal comprises initial photoelectric actual sub-signals respectively obtained by two sensors corresponding to the pressed key in the playing process in a preset sampling period; s42, sequentially carrying out denoising treatment, standardization treatment and integration treatment on a plurality of initial photoelectric actual sub-signals in the initial photoelectric actual signal set to obtain a plurality of photoelectric actual signals, and correspondingly obtaining a photoelectric actual signal set.
3. The method for detecting the play of an wood piano based on photoelectric information according to claim 2, wherein said step S22 comprises: s221, denoising the plurality of initial photoelectric contrast sub-signals based on a denoising processing formula of a wavelet transformation principle to obtain a plurality of denoising photoelectric contrast sub-signals and corresponding denoising photoelectric contrast signal sets; the denoising processing formula based on the wavelet transformation principle is as follows: S(t)=f(t)+ke(t),t=0,1,2,...n; Wherein S (t) represents an initial photoelectric contrast sub-signal containing a noise signal or an initial photoelectric actual sub-signal containing a noise signal, f (t) represents a denoising photoelectric contrast sub-signal or a denoising photoelectric actual sub-signal, e (t) is a noise signal, k is noise intensity, and t is the number of layers of wavelet decomposition; In the operation process, e (t) in the formula is eliminated through the wavelet transformation principle and used for reconstructing f (t) so that S (t) =f (t); s222, carrying out standardization processing on the denoising photoelectric control sub-signals according to a standardization formula of the photoelectric signals to obtain a plurality of standard photoelectric control sub-signals and corresponding standard photoelectric control signal sets; The normalized formula for the photo signal is shown below: f(t)'=(f(t)-max)/(max-min); Wherein max represents the at-maximum value of the denoising photoelectric contrast sub-signal or represents the maximum value of the denoising photoelectric actual sub-signal, min represents the at-minimum value of the denoising photoelectric contrast sub-signal or represents the minimum value of the denoising photoelectric actual sub-signal; And S223, integrating two standard photoelectric control sub-signals corresponding to each key to obtain a plurality of photoelectric control signals and corresponding photoelectric control signal sets.
4. The method for detecting the play of an wood piano based on photoelectric information according to claim 3, wherein said step S223 comprises: S2231, determining two standard photoelectric control sub-signals corresponding to each key, and calculating the correlation between the two standard photoelectric control sub-signals corresponding to each key to obtain a correlation matrix; the formula for calculating the correlation matrix is as follows: ; In the formula, Representing a correlation matrix between two standard photo-electronic signals corresponding to each key or a correlation matrix between two standard photo-electronic signals corresponding to each key; And Representing two standard photo-electronic control sub-signals corresponding to each key or representing two standard photo-electronic actual sub-signals corresponding to each key, respectively; represent the first The key of the key is used for the key, Representing the total number of keys; s2232, establishing a corresponding covariance matrix according to the correlation matrix, and normalizing the covariance matrix to obtain an absolute value of covariance; S2233, determining that the absolute value threshold of the covariance is 0.8, comparing the absolute value of the covariance with 0.8, and if the absolute value of the covariance is larger than 0.8, combining two standard photoelectric control sub-signals corresponding to each key to obtain photoelectric control signals; If the value is smaller than 0.8, calculating a fusion value of two standard photoelectric control sub-signals corresponding to each key, and taking the fusion value as a photoelectric control signal; the formula for calculating the fusion value is as follows: ; , ; Wherein U corresponds to the fusion value of the two standard photoelectric control sub-signals of each key or represents the fusion value of the two standard photoelectric actual sub-signals of each key; , Respectively representing the mean and variance of the two standard photo-electronic signals corresponding to each key or respectively representing the mean and variance of the two standard photo-electronic signals corresponding to each key; And Respectively representing the estimated values of the two standard photo-electronic signals corresponding to each key or respectively representing the estimated values of the two standard photo-electronic signals corresponding to each key.
5. The method for detecting the play of the wood piano based on the photoelectric information according to claim 1, wherein the calculation formula of the strong and weak feature similarity is as follows: ; ; In the formula, Representing the peak-to-peak value of the photo-control signal; Representing the average value of all the photoelectric control signals in the photoelectric control signal set; representing the intensity characteristic value of the photoelectric actual signal; representing the average value of the intensity characteristic values of all the photoelectric actual signals in the photoelectric actual signal set; and the standard deviation of the intensity characteristic values of all the photoelectric control signals in the photoelectric control signal set is represented.
6. The method for detecting the playing of an wood piano based on photoelectric information according to claim 5, wherein the calculation formula of the similarity of the rhythm features is: ; ; In the formula, Representing the peak-to-peak frequency of the photo-control signal per unit time; Representing the rhythm characteristic value of the photoelectric actual signal; representing the average value of the rhythm characteristic values of all the photoelectric control signals in the photoelectric control signal set; representing the standard deviation of the cadence characteristic values of all of the photo-control signals in the set of photo-control signals.
7. The method for detecting the play of an wood piano based on photoelectric information according to claim 1, wherein said play detection model comprises different color flashes for performing different color display based on whether the key is pressed correctly or pressed incorrectly; The method comprises the following steps: And S6, playing back the target track and displaying the detection result by flashing lights with different colors based on time sequence in the playing back process.

Description

Wood piano playing detection method based on photoelectric information Technical Field The application relates to the technical field of piano teaching, in particular to an wood piano playing detection method based on photoelectric information. Background In the field of piano education, it is often difficult for players to quantitatively grasp factors such as the accuracy, strength, and rhythm of keys at the time of playing, when practicing alone or in a class. For this reason, some detection technology devices for detecting whether the playing is correct or not by the key are appeared on the market. However, the products in the same field on the market are mostly concentrated on devices of electric pianos and electronic organ, and the wood pianos have the following drawbacks because of the particularities of their manufacturing processes, which make it difficult to add related detecting devices thereon, even if the related detecting devices are barely added to the wood pianos: 1. the coupling degree of the detection device and the original mechanical structure of the wood piano is too high. When the wood piano is modified, there may be a risk of damaging the original mechanical structure of the wood piano. 2. Fewer information elements are collected. The current technology for detecting the keys of the wood piano or the electronic organ can only feed back the accuracy and rhythm of the tones, and cannot detect information elements such as strength, emotion colors and the like. Disclosure of Invention The application provides an wood piano playing detection method based on photoelectric information, which can solve the problems of too high coupling degree with the wood piano and less acquired information elements existing in the existing playing detection device applied to the wood piano. The technical scheme of the application is a method for detecting the playing of an wood piano based on photoelectric information, which comprises the following steps: s1, installing a sensor according to keys in an wood piano; S2, determining a plurality of target tracks, and sampling a comparison playing process when the target tracks are correctly played by the wood piano through a sensor based on a preset sampling period to obtain a photoelectric comparison signal set about the pressed key based on time sequence in the comparison playing process of each target track; s3, respectively carrying out intensity analysis and rhythm analysis on each photoelectric control signal in the photoelectric control signal set to obtain a photoelectric control signal analysis result, and establishing a playing detection model based on a plurality of target tracks based on the photoelectric control signal analysis result; S4, sampling an actual playing process of the wood piano when the target track is actually played through a sensor based on a preset sampling period, and obtaining a photoelectric actual signal set of the pressed key based on time sequence in the actual playing process; S5, inputting the photoelectric actual signal set to a playing detection model, wherein the playing detection model takes the photoelectric comparison signal set as a detection basis to detect intensity and rhythm of each photoelectric actual signal in the photoelectric actual signal set, and outputs a detection result about keys based on time sequence in the playing process. Optionally, the step S1 includes: S11, installing sensors according to keys in the wood piano, wherein each key corresponds to two sensors; and, the step S2 includes: S21, determining a target track, and sampling a comparison playing process when the target track is correctly played by the wood piano through a sensor based on a preset sampling period to obtain an initial photoelectric comparison signal set which comprises a plurality of initial photoelectric comparison signals and is based on time sequence and related to a pressed key in the comparison playing process; Each initial photoelectric control signal comprises initial photoelectric control sub-signals respectively obtained by two sensors corresponding to the key pressed in the playing process in a preset sampling period; S22, sequentially carrying out denoising treatment, standardization treatment and integration treatment on a plurality of initial photoelectric control sub-signals in the initial photoelectric control signal set to obtain a plurality of photoelectric control signals, and correspondingly obtaining a photoelectric control signal set; and, the step S4 includes: S41, sampling an actual playing process of the wood piano when the target track is actually played through a sensor based on a preset sampling period, and obtaining an initial photoelectric actual signal set which is based on time sequence and comprises a plurality of initial photoelectric actual signals and is related to the pressed key in the actual playing process; Each initial photoelectric actual signal comprises initial photoelectric actual sub