CN108944750B - Active sound simulation device based on Void-Kalman filter - Google Patents

Active sound simulation device based on Void-Kalman filter Download PDF

Info

Publication number
CN108944750B
CN108944750B CN201810804305.6A CN201810804305A CN108944750B CN 108944750 B CN108944750 B CN 108944750B CN 201810804305 A CN201810804305 A CN 201810804305A CN 108944750 B CN108944750 B CN 108944750B
Authority
CN
China
Prior art keywords
module
sound
order
kalman filter
void
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201810804305.6A
Other languages
Chinese (zh)
Other versions
CN108944750A (en
Inventor
马芳武
葛林鹤
史津竹
代凯
蒲永锋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jilin University
Original Assignee
Jilin University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jilin University filed Critical Jilin University
Priority to CN201810804305.6A priority Critical patent/CN108944750B/en
Publication of CN108944750A publication Critical patent/CN108944750A/en
Application granted granted Critical
Publication of CN108944750B publication Critical patent/CN108944750B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R16/00Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
    • B60R16/02Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
    • B60R16/037Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for occupant comfort, e.g. for automatic adjustment of appliances according to personal settings, e.g. seats, mirrors, steering wheel
    • B60R16/0373Voice control

Landscapes

  • Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Soundproofing, Sound Blocking, And Sound Damping (AREA)

Abstract

The invention relates to an active sound simulation device based on a Void-Kalman filter, which comprises: the system comprises an FPGA active sound design module, an MCU control sound production module and a PC upper computer; the FPGA active sound design module is mainly used for extracting order signals of an engine sound source by using a Void-Kalman filter, and the extracted order signals are downloaded to the MCU control sounding module in real time through the SPI interface; meanwhile, the FPGA active sound design module receives a parameter signal of the MCU control sound production module through the SPI interface; the MCU control sound production module is used for finishing and controlling the real-time generation of the sound of the analog engine according to the order signal extracted by the FPGA active sound design module; and the PC upper computer downloads control data to the FPGA active sound design module, and requests and receives data sent by the FPGA active sound design module. The device draws the sound order signal through Void-Kalman filter, and MCU main control module control sound generates, obtains the sound that is rich in the change, reminds the pedestrian, reduces the potential safety hazard, lets the driver experience the enjoyment of driving simultaneously.

Description

Active sound simulation device based on Void-Kalman filter
Technical Field
The invention belongs to the technical field of active sound production of electric vehicles and hybrid electric vehicles, and particularly relates to an active sound simulation device based on a Void-Kalman filter.
Background
In recent years, with the emphasis of the nation on energy conservation, emission reduction and environmental protection, new energy vehicles develop rapidly, the sales volume is continuously increased, and some countries even go out of the prohibition schedule of fuel vehicles, so that the electric and intelligent automobile development is a necessary trend, and electric automobiles are incorporated into development strategies of many countries.
At present, a plurality of countries have related technical indexes, and regulations limit the upper limit value of the noise of the automobile; however, with the development of new energy automobiles, new problems of automobile NVH appear. The sound is used for transmitting information, is one of important transmission media, automobile noise can prompt pedestrians, and can provide vehicle state information and driving experience for drivers, and some racing cars and sports cars even carry out reasonable structural design on an engine and an exhaust pipe to generate engine roaring sound of surge so as to add driving pleasure. However, the electric vehicle is quite quiet at low speed, the electric drive system hardly generates any noise, ordinary pedestrians can easily ignore the existence of the vehicle, and people with visual impairment are difficult to feel the existence of the vehicle, so that potential safety hazards are caused; in addition, because the noise of the electric vehicle is too low, drivers in the electric vehicle lack necessary information for judging the motion state of the vehicle through sound, so that the potential safety hazard is increased, and the drivers who pursue the sound of the engine feel that the electric vehicle loses the driving pleasure.
The potential safety hazard caused by low noise of the electric vehicle at low speed is increasingly prominent, and especially some blind people and visual disorders are more sensitive to the potential safety hazard; in order to solve the pedestrian safety problem caused by the over-low noise of the electric automobile, a pedestrian warning sound system is produced at the right moment, and the pedestrian warning sound system is only used as a storage technology in a mainstream host factory abroad in the last few years, does not realize industrialization, and is more blank in the domestic market. However, with the force of domestic whole vehicle enterprises and the successive national governments coming out of the regulatory standards for installing sound warning systems, pedestrian warning sound systems are more and more emphasized, the market space is wide, and although the corresponding systems are now to be industrialized, how to design a set of sound meeting the brand image of the pedestrian still is a challenging problem. In addition, the existing corresponding system cannot realize on-line adjustment of the component size of each order signal simulating the engine sound, and the sound frequency component cannot change along with the vehicle working condition, so that the sound looks excessively monotonous and boring.
Disclosure of Invention
The invention aims to provide an active sound simulation device based on a Void-Kalman filter, which aims to solve the technical problems that the existing pedestrian warning sound system or the in-vehicle active sound production system cannot realize the on-line adjustment of the component size of each order signal of the simulated engine sound, and the sound frequency component cannot change along with the vehicle working condition. The active sound simulation device can provide automobile sound which accords with brand images for an electric vehicle pedestrian warning sound system or an in-vehicle active sound system, and realizes the online change and control of the simulated engine sound order energy, so that the sound accords with aesthetic requirements better.
In order to realize the purpose, the invention is realized by adopting the following technical scheme:
an active sound simulation device based on Void-Kalman filter, comprising: the system comprises an FPGA active sound design module, an MCU control sound production module and a PC upper computer; the FPGA active sound design module is used for completing calculation intensive tasks in the design process, mainly extracts order signals of an engine sound source by using a Void-Kalman filter, and downloads the extracted order signals to the MCU control sound production module in real time through the SPI interface; meanwhile, the FPGA active sound design module receives a parameter signal of the MCU control sound production module through the SPI interface; the MCU control sound production module is used for finishing and controlling the real-time generation of the sound of the analog engine according to the order signal extracted by the FPGA active sound design module; the PC upper computer is used as a man-machine interaction interface, downloads control data to the FPGA active sound design module, and requests and receives data sent by the FPGA active sound design module;
wherein, MCU control sound production module includes: the device comprises an MCU main control module, a power supply module, a protection circuit, a CAN bus module, an FPGA and MCU communication module, a DAC module, an audio amplifier and a loudspeaker;
one end of the MCU master control module is connected with the FPGA active sound design module through an SPI bus, the other end of the MCU master control module is connected with the CAN bus module through a CAN bus, the MCU master control module is used for acquiring engine sound order signals calculated and extracted by the FPGA active sound design module, acquiring vehicle speed, acceleration and accelerator pedal opening degree signals from the CAN bus module, controlling sound generation in real time according to the acquired signals, transmitting the calculated result to the DAC module through an SPI interface, converting digital signals into analog signals by the DAC module, amplifying the acquired analog signals through an audio amplifier, and outputting the amplified analog signals to a loudspeaker to generate sound;
the power supply module is connected with the protection circuit and used for providing reliable power supply for the MCU main control module and other external equipment;
the CAN bus module is used for acquiring vehicle speed, acceleration and accelerator pedal opening information from a whole vehicle CAN bus and sending the information to the MCU main control module;
and the FPGA and MCU communication module is used for the MCU main control module to read corresponding engines and engine order audio files from the FPGA active sound design module.
Preferably, the order signal extraction process of the Void-Kalman filter in the FPGA active sound design module to the engine sound source is as follows. For the sake of clarity in the description of the steps of the present application, the following description is given using relational mathematical expressions, but expressions representing mathematical expressions or models appearing in the present application are not intended to represent the only expressions of the technical solution of the present application, but are merely possible and preferred embodiments.
Step S1, constructing a Kalman filter, wherein the Kalman state equation and the measurement equation are as follows:
x(n)=Fnx(n-1)+Bnu(n)+ω(n) (1)
y(n)=Hnx(n)+v(n) (2)
equation (1) is a state equation, where x (n) is a state quantity, u (n) is an input, ω (n) is system noise, Fn、BnIs a matrix; equation (2) is a measurement equation, where y (n) is the measurement result, v (n) is the measurement noise, HnFor the matrix, both the system noise and the measurement noise are assumed to be white noise, ω (n) is the system noise, its mean is zero, and its covariance matrix is Q according to a multivariate normal distributionn(ii) a v (n) is the measurement noise, the mean is zero, and the covariance matrix is R according to the multivariate normal distributionnWhen the measurement output y (n) is obtained, a Kalman filter is used to obtain the optimal estimation of the state x; the filtering process comprises two steps: prediction and update of a Kalman filter;
and (3) prediction:
x(n|n-1)=Fnx(n-1|n-1)+Bnu(n) (3)
Figure GDA0002234410080000031
equation (3) is a state prediction equation, where x (n | n-1) is the prediction of the state of n steps under the state of known n-1 steps, and x (n-1| n-1) is the estimated value of the state of the n-1 step; formula (4) is a covariance matrix prediction equation, P (n | n-1) is the state covariance prediction of the nth step in the (n-1) step, and P (n-1| n-1) is the state covariance estimation of the (n-1) step;
updating:
Figure GDA0002234410080000032
Figure GDA0002234410080000033
wherein,for measuring the residual amount, SnTo measure the residual covariance matrix, KnCalculating the obtained optimal kalman gain; updating the filter variables x and P using the above equations;
Figure GDA0002234410080000036
P(n|n)=(I-KnHn)P(n|n-1) (9)
where x (n | n) is the estimate of state x at the current time and P (n | n) is the estimate of the covariance matrix of the state at the current time;
step S2, constructing a state equation and a measurement equation of the Void-Kalman filter;
the k-th order component of the vibration signal is expressed in the form:
yk(n)=xk(n)exp(jΘk(n)) (10)
wherein, exp (j theta)k(n) is positive complex indexA rotation signal; x is the number ofk(n) is a complex exponential signal, exp (j Θ)k(n)) amplitude and phase modulated signals of the complex exponential positive rotation signal;wherein ω (i) represents the rotation speed, and Δ t is the sampling time; the "measurement equation" of the Void-Kalman filter is called the data equation, which can be expressed as:
y(n)=∑xk(n)exp(jΘk(n))+η(n) (11)
wherein η (n) represents noise;
for modulated signal xk(t) expressed using a polynomial of lower order, which satisfies
Figure GDA0002234410080000041
Assuming that s is 3, the discretization of the expression can be expressed as:
xk(n+1)-3xk(n)+3xk(n-1)-xk(n-2)=ak(n) (13)
the above formula can be obtained through transformation
Figure GDA0002234410080000042
Order to
Figure GDA0002234410080000043
To obtain
Xk(n+1)=MXk(n)+Ak(n) (15)
Generalizing the above formula to multiple orders, as shown in the following formula:
order toTo obtain
X(n+1)=FX(n)+ω(k) (17)
Equation (17) is the "state equation" corresponding to the Void-Kalman filter, and is referred to herein as the structural equation;
step S3, acquiring an augmentation data equation of the Void-Kalman filter;
transforming the formula (11) to obtain the data equation augmentation form of the formula (18)
y(n)=H(n)X(n)+η(n) (18)
Wherein H (n) ═ H1(n),H2(n),H3(n)...Hk(n)],Hk(n)=[0 0 θk];
Step S4, filtering by using a filter to obtain time domain signals of each order;
applying the structural equation and the augmentation data equation to a Kalman filtering frame to obtain state estimation X of each order to be trackedk(n) by the specific procedure of letting F in the formula (1)n=F,Bn0, u (n) 0, H in formula (2)nAnd (h) (n), substituting the formula (17) and the formula (18) into the formulas (1) - (9), obtaining the estimation of the state x by using the same formula symbols as the formulas (1) and (2), finally obtaining the time domain signals of each order, and downloading the obtained time domain signals of each order to the MCU control sounding module through the SPI communication interface in time to ensure the real-time performance of the system.
As a preferred aspect of the present invention, the process of the MCU main control module controlling sound generation in real time is as follows: the MCU main control module controls each order signal and source sound sample output according to the speed, the acceleration and the opening degree signal of the accelerator pedal obtained from the CAN bus module, controls the sound gain and the tone of each order channel, and finally outputs the signals to the loudspeaker through integral volume control, gradual-in/gradual-out control, a filtering module, a DAC module and an audio amplifier; each order signal is obtained by calculation of a Void-Kalman filter; the channel sound gain is obtained by multiplying the original order signal by a coefficient, and the coefficient is the gain; the pitch control is realized by a linear resampling algorithm;
when the sound gain of the corresponding order signal channel is set to-1, the phase thereof is just opposite to that of the corresponding order component in the sound source, and theoretically, the order is attenuated to zero; when the gain of the corresponding order channel is larger than zero, the order component is increased; when the gain of the order channel is between-1 and 0, the order is attenuated, the control of each order component is realized by controlling the gain of each order channel, the gain is a function changing along with the speed, and the function of the corresponding order channel gain changing along with the speed, the acceleration and the acceleration is made into a lookup table, namely the enhancement or the attenuation of each order component along with the speed is realized in real time, and finally the sound rich in the change is obtained.
As a further preferable aspect of the present invention, the gain is expressed by a relationship among a speed, an acceleration, and an accelerator opening as follows:
CH_1_volume=f_voice_1(s)+k1’×pa_1(a)+k2’×pp_1(p)
CH_2_volume=f_voice_2(s)+k3’×pa_2(a)+k4’×pp_2(p)
CH_3_volume=f_voice_3(s)+k5’×pa_3(a)+k6’×pp_3(p)
……
CH_i_volume=f_voice_i(s)+k(2i-1)’×pa_i(a)+k(2i)’×pp_i(p)
……
CH_n_volume=f_voice_n(s)+k(2n-1)’×pa_n(a)+k(2n)’×pp_n(p)
wherein, CH _ i _ volume represents the gain of the ith channel (or order), s is the speed, a is the acceleration, and p is the opening degree of the accelerator pedal; the functions f _ voice _ i(s), pa _ i (a) and pp _ i (p) describe the relationship between the gain and speed of the ith channel, the acceleration and the pedal opening; k (2 i-1)' is the switching value of the acceleration weight, and when the switching value is 1, the gain of the ith channel is changed along with the acceleration; when the channel gain is 0, the channel gain does not change along with the acceleration; k (2 i)' is an accelerator pedal opening weight of the ith channel, and when the weight is 1, the gain of the ith channel is changed along with the accelerator pedal, and when the weight is 0, the gain of the ith channel is not changed along with the accelerator pedal.
As a further preferred aspect of the present invention, the tone control is implemented to simulate the variation of the engine sound tone with speed, acceleration and pedal opening, and the tone control is implemented by resampling the source sound and playing it at the same playing rate, so that when the sampling rate is increased, the sound tone is decreased, the sampling rate is decreased, the sound tone is increased, and the relationship between the sound tone and speed is described as follows, as the volume control of the channel is the same:
CH_1_pitch=F_voice_1(s)+K1×PA_1(a)+K2×PP_1(p)
CH_2_pitch=F_voice_2(s)+K3×PA_2(a)+K4×PP_2(p)
CH_3_pitch=F_voice_3(s)+K5×PA_3(a)+K6×PP_3(p)
……
CH_i_pitch=F_voice_i(s)+K(2i-1)×PA_i(a)+K(2i)×PP_i(p)
……
CH_n_pitch=F_voice_n(s)+K(2n-1)×PA_n(a)+K(2n)×PP_n(p)
wherein CH _ i _ pitch represents the pitch of the ith channel (or order), s is the velocity, a is the acceleration, and p is the accelerator pedal opening, and the functions F _ voice _ i(s), PA _ i (a), PP _ i (p) characterize the relationship between the pitch of the ith channel and the velocity, acceleration, and pedal opening; k (2i-1) is the switching value of the acceleration weight, and when the switching value is 1, the tone of the ith channel is changed along with the acceleration; when 0, it means that the channel tone does not vary with the acceleration; k (2i) is an accelerator opening weight of the ith passage, and when it is 1, it indicates that the tone of the ith passage varies with the accelerator pedal, and when it is 0, it indicates that the tone of the ith passage does not vary with the accelerator pedal.
As a further preferred aspect of the present invention, the overall volume control is for controlling the sound pressure level of the final output sound; the filtering module adopts a low-pass filter to filter high-frequency noise; the DAC module adopts a 16-bit DAC module; the fade-in/fade-out control process is: when the system starts to work, the volume of the system is gradually increased, so that excessive awkwardness and scaring of pedestrians are avoided; when the system is out of operation, the volume is gradually reduced rather than suddenly reduced to avoid scaring the pedestrian.
As a further preferable mode of the invention, the MCU control sound module further comprises an LED indicator, the LED indicator is connected to the MCU main control module through an LED driver, and when the MCU main control module fails, the LED indicator flashes on or off to remind the failure.
As a further preferable aspect of the present invention, the PC upper computer mainly implements setting of system parameters, and mainly includes a sound loading menu, a Void-Kalman filter parameter menu, an order gain VS vehicle speed menu, and other parameter menus; the voice loading menu mainly realizes loading of the recorded engine voice; the Void-Kalman filter parameter menu mainly realizes the setting of Void-Kalman filter parameters, and is convenient for extracting and obtaining time domain signals of each required order; the order gain VS speedometer menu is mainly used for designing a curve graph of each order signal changing along with time; the other parameter menu mainly controls other parameters of the system, such as overall volume control and the like.
The invention has the advantages and positive effects that: the active sound simulation device provided by the invention utilizes the Void-Kalman filter to extract the sound order signal of the engine, and then controls the generation of the sound in real time through the MCU master control module, so that the engine sound rich in variation is obtained, the sound can not only remind pedestrians and reduce potential safety hazards, but also can enable a driver to feel the pleasure of driving; in addition, the driver can know the working conditions of the vehicle, such as the vehicle speed, the acceleration and other information in time through the sound, so that the potential safety hazard is further reduced; the simulation device can provide automobile sound which accords with brand images for a pedestrian warning sound system or an in-vehicle active sound system of the electric vehicle, and realizes the online change and control of the simulated engine sound order energy, so that the sound accords with aesthetic requirements; because the simulation device has simple structure and flexible installation, the whole car factory can freely configure and design the simulated engine sound according with the characteristics of the brand.
Drawings
FIG. 1 is a graph of sound spectrum for a steady engine condition.
FIG. 2 is a schematic diagram of the simulation apparatus according to the present invention.
FIG. 3 is a flow chart of the Void-Kalman filter extraction of the order signal of the engine sound source.
Fig. 4 is a control schematic diagram of the MCU master control module.
FIG. 5 is a diagram of a PC upper computer human-computer interaction interface.
FIG. 6 is a graph of the engine sound spectrum rich in variations obtained by the present invention.
Detailed Description
In order to make the technical solutions and advantages of the present invention more apparent to those skilled in the art, the technical solutions of the present invention will be described clearly and completely with reference to the accompanying drawings.
Referring to fig. 2, the active sound simulator based on the Void-Kalman filter according to the present invention includes: the FPGA active sound design module 1, the MCU control sound production module 2 and the PC upper computer 3; the FPGA active sound design module 1 is used for completing calculation intensive tasks in the design process, mainly extracts order signals of an engine sound source by using a Void-Kalman filter, and downloads the extracted order signals to the MCU control sound production module in real time through an SPI interface; meanwhile, the FPGA active sound design module 1 receives a parameter signal of the MCU control sound production module 2 through the SPI interface; the MCU control sound production module 2 is used for finishing and controlling the real-time generation of the sound of the analog engine according to the order signal extracted by the FPGA active sound design module; the PC upper computer 3 is used as a man-machine interaction interface, downloads control data to the FPGA active sound design module, and requests and receives data sent by the FPGA active sound design module;
wherein, MCU control sound production module 2 includes: the system comprises an MCU main control module 21, a power supply module 22, a protection circuit 23, a CAN bus module 24, an FPGA and MCU communication module 25, a DAC module 26, an audio amplifier 27, a loudspeaker 28 and an LED indicator light 29;
one end of the MCU main control module 21 is connected with the FPGA active sound design module 1 through an SPI bus, the other end of the MCU main control module is connected with a CAN bus module 24 through a CAN bus, the MCU main control module is used for acquiring engine sound order signals calculated and extracted by the FPGA active sound design module, acquiring vehicle speed, acceleration and accelerator pedal opening degree signals from the CAN bus module 24, controlling sound generation in real time according to the acquired signals, transmitting the calculated result to a DAC module 26 through an SPI interface, converting digital signals into analog signals by the DAC module 26, amplifying the acquired analog signals through an audio amplifier 27, outputting the amplified analog signals to a loudspeaker 28 and generating sound;
the power supply module 22 is connected with the protection circuit 23 and used for providing reliable power supply for the MCU main control module 21 and other external devices;
the CAN bus module 24 is used for acquiring the speed, the acceleration and the opening degree information of an accelerator pedal from a CAN bus of the whole vehicle and sending the information to the MCU main control module 21;
the FPGA and MCU communication module 25 is used for the MCU master control module to read corresponding engines and engine order audio files from the FPGA active sound design module;
the LED indicator light 29 is connected with the MCU main control module 21 through an LED driver, and when the MCU main control module 21 breaks down, the LED indicator light is turned on or off to remind that the fault occurs; when the LED indicator lamp fails, the LED driver transmits a failure signal to the MCU main control module 21.
Because the numerical signal processing related to sound design is very resource-consuming, the MCU master control module 21 cannot independently complete work, in order to meet the real-time requirement, the FPGA-based active sound design module 1 is needed, and the main calculation work completed by the FPGA-based active sound design module 1 is a Void-Kalman filter.
Referring to fig. 3, in order to obtain the engine sound rich in variation, the order signal extraction process of the Void-Kalman filter in the FPGA active sound design module to the engine sound source is as follows:
step S1, constructing a Kalman filter, wherein the Kalman state equation and the measurement equation are as follows:
x(n)=Fnx(n-1)+Bnu(n)+ω(n) (1)
y(n)=Hnx(n)+v(n) (2)
equation (1) is a state equation, where x (n) is a state quantity, u (n) is an input, ω (n) is system noise, Fn、BnIs a matrix; equation (2) is a measurement equation, where y (n) is the measurement result, v (n) is the measurement noise, HnFor the matrix, both the system noise and the measurement noise are assumed to be white noise, ω (n) is the system noise, its mean is zero, and its covariance matrix is Q according to a multivariate normal distributionn(ii) a v (n) is the measurement noise, the mean is zero, and the covariance matrix is R according to the multivariate normal distributionnWhen the measurement output y (n) is obtained, a Kalman filter is used to obtain the optimal estimation of the state x; the filtering process comprises two steps: prediction and update of a Kalman filter;
and (3) prediction:
x(n|n-1)=Fnx(n-1|n-1)+Bnu(n) (3)
Figure GDA0002234410080000091
equation (3) is a state prediction equation, where x (n | n-1) is the prediction of the state of n steps under the state of known n-1 steps, and x (n-1| n-1) is the estimated value of the state of the n-1 step; formula (4) is a covariance matrix prediction equation, P (n | n-1) is the state covariance prediction of the nth step in the (n-1) step, and P (n-1| n-1) is the state covariance estimation of the (n-1) step;
updating:
Figure GDA0002234410080000092
Figure GDA0002234410080000094
wherein,
Figure GDA0002234410080000095
for measuring the residual amount, SnTo measure the residual covariance matrix, KnCalculating the obtained optimal kalman gain; updating the filter variables x and P using the above equations;
Figure GDA0002234410080000096
P(n|n)=(I-KnHn)P(n|n-1) (9)
where x (n | n) is the estimate of state x at the current time and P (n | n) is the estimate of the covariance matrix of the state at the current time;
step S2, constructing a state equation and a measurement equation of the Void-Kalman filter;
the k-th order component of the vibration signal is expressed in the form:
yk(n)=xk(n)exp(jΘk(n)) (10)
wherein, exp (j theta)k(n) is a complex exponential positive rotation signal; x is the number ofk(n) is a complex exponential signal, exp (j Θ)k(n)) amplitude and phase modulated signals of the complex exponential positive rotation signal;
Figure GDA0002234410080000097
wherein ω (i) represents the rotation speed, and Δ t is the sampling time; the "measurement equation" of the Void-Kalman filter is called the data equation, which can be expressed as:
y(n)=∑xk(n)exp(jΘk(n))+η(n) (11)
wherein η (n) represents noise;
for modulated signal xk(t) expressed using a polynomial of lower order, which satisfies
Figure GDA0002234410080000098
Assuming that s is 3, the discretization of the expression can be expressed as:
xk(n+1)-3xk(n)+3xk(n-1)-xk(n-2)=ak(n) (13)
the above formula can be obtained through transformation
Figure GDA0002234410080000101
Order to
Figure GDA0002234410080000102
To obtain
Xk(n+1)=MXk(n)+Ak(n) (15)
Generalizing the above formula to multiple orders, as shown in the following formula:
Figure GDA0002234410080000103
order toTo obtain
X(n+1)=FX(n)+ω(k) (17)
Equation (17) is the "state equation" corresponding to the Void-Kalman filter, and is referred to herein as the structural equation;
step S3, acquiring an augmentation data equation of the Void-Kalman filter;
transforming the formula (11) to obtain the data equation augmentation form of the formula (18)
y(n)=H(n)X(n)+η(n) (18)
Wherein H (n) ═ H1(n),H2(n),H3(n)...Hk(n)],Hk(n)=[0 0 θk];
Step S4, filtering by using a filter to obtain time domain signals of each order;
applying the structural equation and the augmentation data equation to a Kalman filtering frame to obtain state estimation X of each order to be trackedk(n) by the specific procedure of letting F in the formula (1)n=F,Bn0, u (n) 0, H in formula (2)nH (n), the formulae (17) andsubstituting the formula (18) into the formulas (1) - (9), obtaining the estimation of the state x by using other formula symbols which are the same as the formulas (1) and (2), finally obtaining the time domain signals of each order, downloading the obtained time domain signals of each order to the MCU control sounding module in time through the SPI communication interface, ensuring the real-time performance of the system, and controlling the sound generation in real time through the MCU main control module 21.
Referring to fig. 4, the process of controlling the sound generation by the MCU main control module 21 in real time is as follows: the MCU main control module 21 controls each order signal and source sound sample output according to the speed, acceleration and accelerator pedal opening degree signals obtained from the CAN bus module 24, controls the sound gain and tone of each order channel, and finally outputs the signals to the loudspeaker 28 through integral volume control, gradual-in/gradual-out control, a filtering module, a DAC module 26 and an audio amplifier 27; each order signal is obtained by calculation of a Void-Kalman filter; the channel sound gain is obtained by multiplying the original order signal by a coefficient, and the coefficient is the gain; the pitch control is realized by a linear resampling algorithm;
when the sound gain of the corresponding order signal channel is set to-1, the phase thereof is just opposite to that of the corresponding order component in the sound source, and theoretically, the order is attenuated to zero; when the gain of the corresponding order channel is larger than zero, the order component is increased; when the gain of the order channel is between-1 and 0, the order is attenuated, the control of each order component is realized by controlling the gain of each order channel, the gain is a function changing along with the speed, and the function of the corresponding order channel gain changing along with the speed, the acceleration and the acceleration is made into a lookup table, namely the enhancement or the attenuation of each order component along with the speed is realized in real time, and finally, the sound rich in the change is obtained, and the engine sound spectrogram rich in the change is shown in figure 6.
The gain of the present invention as a function of speed, acceleration and accelerator pedal opening can be expressed by the following relationship:
CH_1_volume=f_voice_1(s)+k1’×pa_1(a)+k2’×pp_1(p)
CH_2_volume=f_voice_2(s)+k3’×pa_2(a)+k4’×pp_2(p)
CH_3_volume=f_voice_3(s)+k5’×pa_3(a)+k6’×pp_3(p)
……
CH_i_volume=f_voice_i(s)+k(2i-1)’×pa_i(a)+k(2i)’×pp_i(p)
……
CH_n_volume=f_voice_n(s)+k(2n-1)’×pa_n(a)+k(2n)’×pp_n(p)
wherein, CH _ i _ volume represents the gain of the ith channel (or order), s is the speed, a is the acceleration, and p is the opening degree of the accelerator pedal; the functions f _ voice _ i(s), pa _ i (a) and pp _ i (p) describe the relationship between the gain and speed of the ith channel, the acceleration and the pedal opening; k (2 i-1)' is the switching value of the acceleration weight, and when the switching value is 1, the gain of the ith channel is changed along with the acceleration; when the channel gain is 0, the channel gain does not change along with the acceleration; k (2 i)' is an accelerator pedal opening weight of the ith channel, and when the weight is 1, the gain of the ith channel is changed along with the accelerator pedal, and when the weight is 0, the gain of the ith channel is not changed along with the accelerator pedal.
The tone control of the invention mainly realizes the simulation of the change of the engine sound tone along with the speed, the acceleration and the pedal opening, the tone control is realized by resampling the source sound and playing the source sound at the same playing speed, the sampling rate is improved, the sound tone is reduced, the sampling rate is reduced, the sound tone is raised, the tone is the same as the volume control of a channel, and the relationship between the tone and the speed can be described as follows:
CH_1_pitch=F_voice_1(s)+K1×PA_1(a)+K2×PP_1(p)
CH_2_pitch=F_voice_2(s)+K3×PA_2(a)+K4×PP_2(p)
CH_3_pitch=F_voice_3(s)+K5×PA_3(a)+K6×PP_3(p)
……
CH_i_pitch=F_voice_i(s)+K(2i-1)×PA_i(a)+K(2i)×PP_i(p)
……
CH_n_pitch=F_voice_n(s)+K(2n-1)×PA_n(a)+K(2n)×PP_n(p)
wherein CH _ i _ pitch represents the pitch of the ith channel (or order), s is the velocity, a is the acceleration, and p is the accelerator pedal opening, and the functions F _ voice _ i(s), PA _ i (a), PP _ i (p) characterize the relationship between the pitch of the ith channel and the velocity, acceleration, and pedal opening; k (2i-1) is the switching value of the acceleration weight, and when the switching value is 1, the tone of the ith channel is changed along with the acceleration; when 0, it means that the channel tone does not vary with the acceleration; k (2i) is an accelerator opening weight of the ith passage, and when it is 1, it indicates that the tone of the ith passage varies with the accelerator pedal, and when it is 0, it indicates that the tone of the ith passage does not vary with the accelerator pedal.
The integral volume control is used for realizing the control of the sound pressure level of the final output sound; the filtering module adopts a low-pass filter to filter high-frequency noise; the DAC module 26 adopts a 16-bit DAC module; the fade-in/fade-out control process is: when the system starts to work, the volume of the system is gradually increased, so that excessive awkwardness and scaring of pedestrians are avoided; when the system is out of operation, the volume is gradually reduced rather than suddenly reduced to avoid scaring the pedestrian.
Referring to fig. 5, the PC upper computer 3 of the present invention mainly implements the setting of system parameters, and mainly includes a sound loading 31 menu, a Void-Kalman filter parameter 32 menu, an order gain VS vehicle speed 33 menu, and other parameter 34 menus; the voice loading 31 menu mainly realizes loading of the recorded engine voice; the Void-Kalman filter parameter 32 menu mainly realizes the setting of Void-Kalman filter parameters, and is convenient for extracting time domain signals of each required order; the order gain VS speedometer 33 menu is mainly used for designing a curve graph of each order signal changing along with time; the other parameters 34 menu primarily controls other parameters of the system, such as overall volume control, etc. In addition, the order of the Void-Kalman filter can be set on a human-computer interface on the PC, the order of the engine sound to be extracted can be specified, and a time domain graph of each order signal is displayed in a coordinate.

Claims (8)

1. An active sound simulator based on Void-Kalman filter, comprising: the system comprises an FPGA active sound design module, an MCU control sound production module and a PC upper computer; the FPGA active sound design module is used for completing calculation intensive tasks in the design process, mainly extracts order signals of an engine sound source by using a Void-Kalman filter, and downloads the extracted order signals to the MCU control sound production module in real time through the SPI interface; meanwhile, the FPGA active sound design module receives a parameter signal of the MCU control sound production module through the SPI interface; the MCU control sound production module is used for finishing and controlling the real-time generation of the sound of the analog engine according to the order signal extracted by the FPGA active sound design module; the PC upper computer is used as a man-machine interaction interface, downloads control data to the FPGA active sound design module, and requests and receives data sent by the FPGA active sound design module;
wherein, MCU control sound production module includes: the device comprises an MCU main control module, a power supply module, a protection circuit, a CAN bus module, an FPGA and MCU communication module, a DAC module, an audio amplifier and a loudspeaker;
one end of the MCU master control module is connected with the FPGA active sound design module through an SPI bus, the other end of the MCU master control module is connected with the CAN bus module through a CAN bus, the MCU master control module is used for acquiring engine sound order signals calculated and extracted by the FPGA active sound design module, acquiring vehicle speed, acceleration and accelerator pedal opening degree signals from the CAN bus module, controlling sound generation in real time according to the acquired signals, transmitting the calculated result to the DAC module through an SPI interface, converting digital signals into analog signals by the DAC module, amplifying the acquired analog signals through an audio amplifier, and outputting the amplified analog signals to a loudspeaker to generate sound;
the power supply module is connected with the protection circuit and used for providing reliable power supply for the MCU main control module and other external equipment;
the CAN bus module is used for acquiring vehicle speed, acceleration and accelerator pedal opening information from a whole vehicle CAN bus and sending the information to the MCU main control module;
and the FPGA and MCU communication module is used for the MCU main control module to read corresponding engines and engine order audio files from the FPGA active sound design module.
2. The active sound simulator based on Void-Kalman filter of claim 1, wherein the order signal extraction process of Void-Kalman filter to engine sound source in FPGA active sound design module is as follows:
step S1, constructing a Kalman filter, wherein the Kalman state equation and the measurement equation are as follows:
x(n)=Fnx(n-1)+Bnu(n)+ω(n) (1)
y(n)=Hnx(n)+v(n) (2)
equation (1) is a state equation, where x (n) is a state quantity, u (n) is an input, ω (n) is system noise, Fn、BnIs a matrix; equation (2) is a measurement equation, where y (n) is the measurement result, v (n) is the measurement noise, HnFor the matrix, both the system noise and the measurement noise are assumed to be white noise, ω (n) is the system noise, its mean is zero, and its covariance matrix is Q according to a multivariate normal distributionn(ii) a v (n) is the measurement noise, the mean is zero, and the covariance matrix is R according to the multivariate normal distributionnWhen the measurement output y (n) is obtained, a Kalman filter is used to obtain the optimal estimation of the state x; the filtering process comprises two steps: prediction and update of a Kalman filter;
and (3) prediction:
x(n|n-1)=Fnx(n-1|n-1)+Bnu(n) (3)
Figure FDA0002234410070000021
equation (3) is a state prediction equation, where x (n | n-1) is the prediction of the state of n steps under the state of known n-1 steps, and x (n-1| n-1) is the estimated value of the state of the n-1 step; formula (4) is a covariance matrix prediction equation, P (n | n-1) is the state covariance prediction of the nth step in the (n-1) step, and P (n-1| n-1) is the state covariance estimation of the (n-1) step;
updating:
Figure FDA0002234410070000028
Figure FDA0002234410070000023
wherein,
Figure FDA0002234410070000024
for measuring the residual amount, SnTo measure the residual covariance matrix, KnCalculating the obtained optimal kalman gain; updating the filter variables x and P using the above equations;
Figure FDA0002234410070000025
P(n|n)=(I-KnHn)P(n|n-1) (9)
where x (n | n) is the estimate of state x at the current time and P (n | n) is the estimate of the covariance matrix of the state at the current time;
step S2, constructing a state equation and a measurement equation of the Void-Kalman filter;
the k-th order component of the vibration signal is expressed in the form:
yk(n)=xk(n)exp(jΘk(n)) (10)
wherein, exp (j theta)k(n) is a complex exponential positive rotation signal; xk (n) is a complex exponential signal, exp (j Θ)k(n)) amplitude and phase modulated signals of the complex exponential positive rotation signal;wherein ω (i) represents the rotation speed, and Δ t is the sampling time; the "measurement equation" of the Void-Kalman filter is called the data equation, which can be expressed as:
y(n)=∑xk(n)exp(jΘk(n))+η(n) (11)
wherein η (n) represents noise;
for modulated signal xk(t) expressed using a polynomial of lower order, which satisfies
Figure FDA0002234410070000027
Assuming that s is 3, the discretization of the expression can be expressed as:
xk(n+1)-3xk(n)+3xk(n-1)-xk(n-2)=ak(n) (13)
the above formula can be obtained through transformation
Figure FDA0002234410070000031
Order to
Figure FDA0002234410070000032
To obtain
Xk(n+1)=MXk(n)+Ak(n) (15)
Generalizing the above formula to multiple orders, as shown in the following formula:
order to
Figure FDA0002234410070000034
To obtain
X(n+1)=FX(n)+ω(k) (17)
Equation (17) is the "state equation" corresponding to the Void-Kalman filter, and is referred to herein as the structural equation;
step S3, acquiring an augmentation data equation of the Void-Kalman filter;
transforming the formula (11) to obtain the data equation augmentation form of the formula (18)
y(n)=H(n)X(n)+η(n) (18)
Wherein H (n) ═ H1(n),H2(n),H3(n)...Hk(n)],Hk(n)=[0 0 θk];
Step S4, filtering by using a filter to obtain time domain signals of each order;
applying the structural equation and the augmentation data equation to a Kalman filtering frame to obtain state estimation X of each order to be trackedk(n) by the specific procedure of letting F in the formula (1)n=F,Bn0, u (n) 0, H in formula (2)nAnd (h) (n), substituting the formula (17) and the formula (18) into the formulas (1) - (9), obtaining the estimation of the state x by using the same formula symbols as the formulas (1) and (2), finally obtaining the time domain signals of each order, and downloading the obtained time domain signals of each order to the MCU control sounding module through the SPI communication interface in time to ensure the real-time performance of the system.
3. The active sound simulator based on Void-Kalman filter of claim 1, wherein the MCU master control module controls the sound generation process in real time as follows: the MCU main control module controls each order signal and source sound sample output according to the speed, the acceleration and the opening degree signal of the accelerator pedal obtained from the CAN bus module, controls the sound gain and the tone of each order channel, and finally outputs the signals to the loudspeaker through integral volume control, gradual-in/gradual-out control, a filtering module, a DAC module and an audio amplifier; each order signal is obtained by calculation of a Void-Kalman filter; the channel sound gain is obtained by multiplying the original order signal by a coefficient, and the coefficient is the gain; the pitch control is realized by a linear resampling algorithm;
when the sound gain of the corresponding order signal channel is set to-1, the phase thereof is just opposite to that of the corresponding order component in the sound source, and theoretically, the order is attenuated to zero; when the gain of the corresponding order channel is larger than zero, the order component is increased; when the gain of the order channel is between-1 and 0, the order is attenuated, the control of each order component is realized by controlling the gain of each order channel, the gain is a function changing along with the speed, and the function of the corresponding order channel gain changing along with the speed, the acceleration and the acceleration is made into a lookup table, namely the enhancement or the attenuation of each order component along with the speed is realized in real time, and finally the sound rich in the change is obtained.
4. A Void-Kalman filter-based active sound simulation apparatus according to claim 3, wherein the gain variation with speed, acceleration and accelerator pedal opening is expressed by the following relationship:
CH_1volume=f_voice_l(s)+k1’×pa_1(a)+k2’×pp_1(p)
CH_2volume=f_voice_2(s)+k3’×pa_2(a)+k4’×pp_2(p)
CH_3volume=f_voice_3(s)+k5’×pa_3(a)+k6’×pp_3(p)
......
CH_i_volume=f_voice_i(s)+k(2i-1)’×pa_i(a)+k(2i)’×pp_i(p)
......
CH_n_volume=f_voice_n(s)+k(2n-1)’×pa_n(a)+k(2n)’×pp_n(p)
wherein CH _ i _ volume represents the gain of the ith channel, s is the speed, a is the acceleration, and p is the opening degree of the accelerator pedal; the functions f _ voice _ i(s), pa _ i (a) and pp _ i (p) describe the relationship between the gain and speed of the ith channel, the acceleration and the pedal opening; k (2 i-1)' is the switching value of the acceleration weight, and when the switching value is 1, the gain of the ith channel is changed along with the acceleration; when the channel gain is 0, the channel gain does not change along with the acceleration; k (2 i)' is an accelerator pedal opening weight of the ith channel, and when the weight is 1, the gain of the ith channel is changed along with the accelerator pedal, and when the weight is 0, the gain of the ith channel is not changed along with the accelerator pedal.
5. The active voice simulator based on Void-Kalman filter of claim 3, characterized in that the tone control mainly realizes the simulation of the engine voice tone with the change of speed, acceleration and pedal opening, the tone control is realized by resampling the source voice and playing it out at the same playing speed, increasing the sampling rate, decreasing the voice tone, decreasing the sampling rate, increasing the voice tone, as the volume control of the channel, the relationship with speed is described as follows:
CH_1_pitch=F_voice_1(s)+K1×PA_1(a)+K2×PP_1(p)
CH_2pitch=F_voice_2(s)+K3×PA_2(a)+K4×PP_2(p)
CH_3pitch=F_voice_3(s)+K5×PA_3(a)+K6×PP_3(p)
......
CH_i_pitch=F_voice_i(s)+K(2i-1)×PA_i(a)+K(2i)×PP_i(p)
......
CH_n_pitch=F_voice_n(s)+K(2n-1)×PA_n(a)+K(2n)×PP_n(p)
wherein CH _ i _ pitch represents the pitch of the ith channel, s is the speed, a is the acceleration, D is the accelerator pedal opening, and the functions F _ voice _ i(s), PA _ i (a), PP _ i (p) describe the relationship between the pitch and speed of the ith channel, and the relationship between the acceleration and the accelerator pedal opening; k (2i-1) is the switching value of the acceleration weight, and when the switching value is 1, the tone of the ith channel is changed along with the acceleration; when 0, it means that the channel tone does not vary with the acceleration; k (2i) is an accelerator opening weight of the ith passage, and when it is 1, it indicates that the tone of the ith passage varies with the accelerator pedal, and when it is 0, it indicates that the tone of the ith passage does not vary with the accelerator pedal.
6. The active voice simulator based on Void-Kalman filter of claim 3, characterized in that the overall volume control is used to realize the control of the final output voice pressure level; the filtering module adopts a low-pass filter to filter high-frequency noise; the DAC module adopts a 16-bit DAC module; the fade-in/fade-out control process is: when the system starts to work, the volume of the system is gradually increased; when the system is out of operation, the volume will gradually decrease.
7. The active sound simulation device based on the Void-Kalman filter according to claim 3, characterized in that the MCU control sound production module further comprises an LED indicator light, the LED indicator light is connected with the MCU main control module through an LED driver, and when the MCU main control module fails, the LED indicator light is turned on or off to remind the MCU main control module of the failure.
8. The active sound simulator based on Void-Kalman filter of any claim 1 to 7, wherein the PC host computer mainly realizes the setting of system parameters, mainly comprising a sound loading menu, a Void-Kalman filter parameter menu, an order gain VS speed menu and other parameter menus; the voice loading menu mainly realizes loading of the recorded engine voice; the Void-Kalman filter parameter menu mainly realizes the setting of Void-Kalman filter parameters, and is convenient for extracting and obtaining time domain signals of each required order; the order gain VS speedometer menu is mainly used for designing a curve graph of each order signal changing along with time; the other parameter menu mainly controls other parameters of the system.
CN201810804305.6A 2018-07-20 2018-07-20 Active sound simulation device based on Void-Kalman filter Expired - Fee Related CN108944750B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810804305.6A CN108944750B (en) 2018-07-20 2018-07-20 Active sound simulation device based on Void-Kalman filter

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810804305.6A CN108944750B (en) 2018-07-20 2018-07-20 Active sound simulation device based on Void-Kalman filter

Publications (2)

Publication Number Publication Date
CN108944750A CN108944750A (en) 2018-12-07
CN108944750B true CN108944750B (en) 2020-02-18

Family

ID=64482053

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810804305.6A Expired - Fee Related CN108944750B (en) 2018-07-20 2018-07-20 Active sound simulation device based on Void-Kalman filter

Country Status (1)

Country Link
CN (1) CN108944750B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110481470B (en) * 2019-08-15 2021-01-15 中国第一汽车股份有限公司 Design method of active sound production system of electric automobile
CN111695204B (en) * 2020-06-30 2022-07-08 重庆长安汽车股份有限公司 Compensation method for working condition identification based on order rendering
CN115431871A (en) * 2022-08-30 2022-12-06 中国第一汽车股份有限公司 Sound control method considering both low-speed pedestrian prompting sound and electronic sound waves of electric vehicle
CN115662470B (en) * 2022-12-12 2023-05-26 科大讯飞(苏州)科技有限公司 Audio particle extraction method, sound wave synthesis device, equipment and medium

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10414337B2 (en) * 2013-11-19 2019-09-17 Harman International Industries, Inc. Apparatus for providing environmental noise compensation for a synthesized vehicle sound
CN103625395B (en) * 2013-12-05 2017-01-04 合肥工业大学 A kind of improve the system of kinesthesia sound quality in electric automobile
CN103895567B (en) * 2014-03-26 2016-06-29 北京长安汽车工程技术研究有限责任公司 The speech simulation vocal technique of a kind of electric automobile and device
FR3034904B1 (en) * 2015-04-09 2018-04-27 Psa Automobiles Sa. SOUND SYNTHESIS METHOD AND DEVICE FOR VEHICLE
CN106671875A (en) * 2017-01-25 2017-05-17 哈尔滨工业大学(威海) Electric car driving system sound acquisition sounding method and device

Also Published As

Publication number Publication date
CN108944750A (en) 2018-12-07

Similar Documents

Publication Publication Date Title
CN108944750B (en) Active sound simulation device based on Void-Kalman filter
EP3514790B1 (en) System and method for vehicle noise masking
CN103392202B (en) vehicle engine sound enhancement method and system
CN101916519B (en) Driving simulation test method for test bench of power system of electric automobile
CN106671875A (en) Electric car driving system sound acquisition sounding method and device
CN106910510A (en) Vehicle-mounted power amplifying device, vehicle and its audio play handling method
CN206417114U (en) A kind of electric vehicle low speed caution sound acoustical generator
CN107199946A (en) The engine noise analogy method and system of a kind of electronic and mixed electrical automobile
EP3733442A1 (en) Vehicle emulator, mobile device for a vehicle emulator and a method for emulating a vehicle experience
KR102126668B1 (en) Apparatus and method for controlling virtual engine sound in responding to autocruise speed settings
CN114103850B (en) System for realizing active sound enhancement of automobile
CN111976591A (en) AVAS audio control method of pedestrian warning sound system of electric vehicle
CN103119643B (en) Vehicle having an electric drive
CN109591693A (en) A kind of electric vehicle motion sound quality active sonification system
CN109460610B (en) Electronic sound effect system for simulating sound effect of fuel engine system
CN116160955B (en) Active sound wave enhancement method and device for vehicle, storage medium and equipment
CN112046423A (en) Electric coach vehicle system for simulating fuel vehicle
CN213422401U (en) Engine noise acquisition system
CN110956866A (en) Driving simulation system based on VR technology
CN206411909U (en) A kind of improved automotive driving simulator
CN115534844A (en) Vehicle-mounted atmosphere lamp music rhythm control method and system
CN114475487A (en) Automatic volume control method for automobile alarm prompt tone
CN213007990U (en) Electric automobile noise simulation system
EP1382956A3 (en) Procedure for simulating the driving performance of vehicles
CN112061046A (en) System for simulating noise decomposition and synthesis of fuel vehicle

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20200218

Termination date: 20200720

CF01 Termination of patent right due to non-payment of annual fee