CN109993155B - Characteristic peak extraction method for low signal-to-noise ratio ultraviolet Raman spectrum - Google Patents
Characteristic peak extraction method for low signal-to-noise ratio ultraviolet Raman spectrum Download PDFInfo
- Publication number
- CN109993155B CN109993155B CN201910327282.9A CN201910327282A CN109993155B CN 109993155 B CN109993155 B CN 109993155B CN 201910327282 A CN201910327282 A CN 201910327282A CN 109993155 B CN109993155 B CN 109993155B
- Authority
- CN
- China
- Prior art keywords
- raman
- peak
- spectrum
- region
- valley
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/65—Raman scattering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/02—Preprocessing
- G06F2218/04—Denoising
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/08—Feature extraction
- G06F2218/10—Feature extraction by analysing the shape of a waveform, e.g. extracting parameters relating to peaks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/12—Classification; Matching
- G06F2218/14—Classification; Matching by matching peak patterns
Landscapes
- Health & Medical Sciences (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
Abstract
The invention discloses a characteristic peak extraction method for low signal-to-noise ratio ultraviolet Raman spectroscopy, and belongs to the fields of spectrum detection and spectrum processing technologies and signal processing. The method collects the ultraviolet Raman signals in real time, counts the collected Raman signal data, and predicts the effective peak value of the Raman signals. And processing each collected Raman spectrum, dividing each Raman spectrum into each block area according to the obtained effective valley and judging the attribute of each divided block area. And aiming at each noise region, each point is equal to the minimum value of the region, and then, the Raman signal region and the processed noise region are spliced. And then, splicing the processed N +1 frames of spectra into a 2D image along a time axis, filtering the 2D image by an iterative bilateral filtering method, and superposing and normalizing the N +1 frames of spectra along the time axis after filtering to obtain a clean Raman characteristic peak spectral image.
Description
Technical Field
The invention relates to a characteristic peak extraction method for low signal-to-noise ratio ultraviolet Raman spectrum, in particular to a method for acquiring clean and effective Raman characteristic peak images for real-time low signal-to-noise ratio ultraviolet Raman signals, and belongs to the fields of spectrum detection and spectrum processing technology and signal processing.
Background
Raman spectroscopy is a non-destructive spectroscopic detection method based on inelastic light scattering (i.e., changes in the energy/frequency of the incident laser light) of the laser light interacting with matter. By measuring the specific Raman spectrum (namely Raman fingerprint spectrum) of the detected molecular system, the method can carry out quick, simple and repeatable non-contact nondestructive detection and quantitative analysis on the sample, and has the characteristics of short required time, small sample consumption, accurate measurement result and the like. However, the sensitivity of Raman spectroscopy detection is low (the intensity of Rayleigh scattered radiation is only 10 times that of the incident light intensity)-3Raman spectrum intensity is about 10 of Rayleigh line only-3) Especially for some small, portable or simple ultraviolet Raman spectrum detection systems, due to the factors of low laser power, unstable laser, large uncooled CCD noise and the like, the detection result has large noise influence, the signal-to-noise ratio of Raman signals is low, and Raman characteristic peaks are difficult to identify.
Some conventional signal processing methods can remove some random noise, thereby making the raman signature peaks prominent, but are not necessarily ideal for raman spectroscopy because many weak raman peaks, either in amplitude or waveform, are very close to the noise. Therefore, many weak Raman effective signals can be lost in the process of removing random noise, and the signal-to-noise ratio is not improved too much. Therefore, novel filtering methods in a time domain are proposed, and weak signals with certain waveforms which do not change along with time can be reserved, such as a prediction filtering method in t-x-y, a Bayesian prediction filtering algorithm and the like.
Because the raman system usually detects only a single sample at a time, the raman signal generated by the single sample has fixed raman characteristic peak position except for amplitude change, and the noise has randomness. Therefore, when the spectral image of a sample is collected in real time, noise signals which constantly change along with the time can be distinguished by counting the peak positions which do not change along with the time, the noise signals are filtered, and the interested effective Raman signals are highlighted. Meanwhile, by utilizing a 2D spectral image construction method on a time domain, longitudinally expanding a spectrum on an x axis along the t axis, wherein the t axis is a time axis, the x axis is a wavelength or a Raman shift axis, then selecting a spectral image of a counted effective noise area as an interested area, selecting a smooth filtering method with an edge protection characteristic to filter the interested area, and filtering noise random along time near an effective Raman peak, so as to obtain a clean and effective Raman peak image for subsequent operations such as identification matching and the like, thereby improving the accuracy of rear-end Raman signal identification.
In summary, statistical classification performed after the raman detection signal is expanded in the time domain and a 2D filtering method for the interested t-x spectral image region are effective methods for extracting the characteristic peak, obtaining a clean raman characteristic peak image without peak distortion, and solving the problem that the weak raman characteristic peak is difficult to identify and extract due to the low signal-to-noise ratio of the raman signal of a small, portable or simple ultraviolet raman spectral detection system.
Disclosure of Invention
The invention discloses a method for extracting characteristic peaks of low signal-to-noise ratio ultraviolet Raman spectrum, which aims to solve the technical problems that: aiming at real-time Raman spectrum signals, particularly aiming at a Raman spectrum detection system with low Raman signal-to-noise ratio, clean and effective Raman characteristic peak images are extracted after processing so as to be convenient for subsequent operations such as identification and matching, and the like, thereby improving the accuracy of rear-end Raman signal identification. The Raman spectrum detection system with low Raman signal-to-noise ratio is particularly a small, portable or simple ultraviolet Raman spectrum detection system.
The purpose of the invention is realized by the following technical scheme.
The invention discloses a characteristic peak extraction method for low signal-to-noise ratio ultraviolet Raman spectrum, which collects ultraviolet Raman signals in real time, counts the collected Raman signal data, and predicts effective peak values of the Raman signals, wherein the effective peak values comprise effective valleys and peaks. Processing each collected Raman spectrum, dividing each Raman spectrum into each block area according to the obtained effective valley, and respectively judging the attribute of each divided block area, wherein the area attribute refers to the attribute of a noise area or the attribute of an effective Raman signal area. And aiming at each noise region, each point is equal to the minimum value of the region, and then, the Raman signal region and the processed noise region are spliced. And then, splicing the processed N +1 frames of spectra into a 2D image along a time axis, filtering the 2D image by an iterative bilateral filtering method, and superposing and normalizing the N +1 frames of spectra along the time axis after filtering to obtain a clean Raman characteristic peak spectral image.
The invention discloses a characteristic peak extraction method for low signal-to-noise ratio ultraviolet Raman spectroscopy, which comprises the following steps:
the method comprises the following steps: and collecting ultraviolet Raman signals in real time, and counting the collected Raman signal data to predict effective peak values of the Raman signals, wherein the effective peak values comprise effective valleys and peaks.
The ultraviolet Raman signal is acquired in real time from a spectrometer in a front-end Raman detection system, and the real-time acquired Raman signal is transmitted to a receiverAnd the signal buffer area is used for keeping and accessing the spectrum of the N +1 frame, and when a new spectrum of one frame is acquired, the spectrum data of the first N +1 frame is deleted to finish the acquisition of the Raman signal data. And performing valley and peak statistics on the N +1 frame image, and listing a valley netlist and a peak netlist. Presetting a step length of a statistical classification allowable interval, dividing the whole spectrum into M allowable intervals along an x axis, wherein the x axis is a wavelength axis or a Raman shift axis, if a peak netlist of the N frames of spectra has peaks in each allowable interval and the absolute value of the slope at the left end or the right end of the interval is greater than a preset threshold value KmaxNamely N frames of spectra all satisfy the following condition,
the allowable interval has f' (x)m)＞Kmax&&f'(xm+1)≤0,m∈CmM1, 2, …, M the predetermined threshold KmaxEqual to the absolute value of the maximum slope that can be caused by random noise, the peak is marked as a valid peak, the valid peak is a Raman peak with high intensity, and the position of the Raman peak is recorded.
For the valley netlist of the N +1 frame spectrum, the judgment condition of the valley point is given by the following formula,
the allowable interval has f' (x)m)≤0&&f'(xm+1)≥0,m∈Cm，m＝1,2,…,M
Then, the true spectrum valley point is predicted by the following method, and the true spectrum valley point is predicted by totally dividing the following three conditions: in the first case, in an allowable interval step length, if a valley point exists in a spectrum of less than or equal to (N +1)/2 frames, the spectrum is not an effective valley; in a step length of an allowable interval, if a valley point exists in the spectrum of more than or equal to N frames, the spectrum is an effective valley, and the effective valley position is calibrated and recorded according to the average value or the weight average value; and thirdly, in a step of an allowable interval, a valley point is formed in the spectrum which is larger than (N +1)/2 and smaller than N frames, which is probably because a low-intensity peak is formed under the influence of noise, and whether the peak is a valid valley is judged through a preset prediction condition. And predicting a true spectrum valley point through the three conditions to obtain a valley netlist. And predicting the effective peak positions of the Raman signal through the statistical low-valley netlist and the statistical peak netlist.
The specific judgment method of the preset prediction condition for judging whether the prediction condition is the effective valley is as follows: and observing whether a high-intensity Raman peak recorded in the peak netlist exists in an allowable interval step region with a low valley value close to the left and right, recording the high-intensity Raman peak into a valley netlist if the high-intensity Raman peak exists in the peak netlist, if the high-intensity Raman peak exists in an effective valley, recording the high-intensity Raman peak into the valley netlist, if the high-intensity Raman peak does not exist in the effective valley, overlapping the N +1 frame spectrum, and if the low valley exists in the allowable interval step, recording the effective valley into the valley netlist.
Step one the step size of the statistical classification tolerance interval is preferably 3.
Step two: and (3) for each frame of Raman spectrum, dividing each frame of Raman spectrum into each block region through the effective valley obtained in the first step, and respectively judging the attribute of each divided block region, wherein the region attribute refers to the attribute of a noise region or the attribute of an effective Raman signal region. And aiming at each noise region, each point is equal to the minimum value of the region, and then, the Raman signal region and the processed noise region are spliced.
And C, dividing each frame of Raman spectrum acquired in the step I into each block area through the valley netlist of each frame of Raman spectrum predicted in the step I, judging the attribute of each area, and respectively determining the attribute of each block area, namely respectively determining whether each block area is a noise area or an effective Raman signal area. For a single region, there are two criteria: the first criterion is: when a Raman peak with high intensity recorded in the peak net list obtained in the first step exists in the region, determining the Raman peak as an effective Raman signal region; the second criterion is: selecting the length of p allowable intervals in the step one as a step length, performing least square method smoothing, judging the ratio of the standard deviation and the average value of the smoothed data in the region, if the ratio of the standard deviation and the average value is greater than a preset threshold value, determining the region as an effective Raman signal region, otherwise, determining the region as a noise region. The preset threshold is one half of the average value of the ratio of the standard deviation to the average value after the least square method smoothing by taking the length of the p allowable intervals in the step one as the step length in all the areas. The first judgment standard is used for detecting a strong peak area; the second criterion is used to make a decision on weak peaks.
After determining the attribute of each block area, aiming at each noise area, each point is equal to the minimum value of the area, and all the wavelength or Raman shift position information of the area which is judged as the noise attribute is recorded. Then, the effective Raman signal region and the filtered noise region are spliced.
In the second step, the number of p in the p allowable intervals is preferably 3.
The principle of the second judgment standard is that the Raman width of a weak peak is larger than the sampling width, the noise width is the sampling width, the least square filtering is carried out according to the estimated step length of the average width of the weak peak, the least square filtering smoothes the noise into an approximate straight line region, the weak peak can be effectively reserved, the ratio of the standard to the average value of the approximate straight line region after smoothing is obviously smaller than the ratio of the standard deviation to the average value of the weak peak region, and then the noise region or the effective Raman signal region is judged by calculating the ratio of the standard deviation to the average value of the region.
And step three, processing each frame of Raman spectrum acquired in the step one in the step two, splicing the N +1 frames of Raman spectrum processed in the step two into a 2D image along a time axis, filtering the 2D image by an iterative bilateral filtering method, and superposing and normalizing the N +1 frames of Raman spectrum along the time axis after filtering to obtain a clean Raman characteristic peak spectrum image.
The effective Raman signal area judged in the step two is not processed, so that noise still exists; the 2D construction and filtering principle of the spectrum on t-x is to spread the spectrum into a t-x image in time, due to the fact that noise has randomness on a time axis, and the peak position of an effective Raman signal is constant on the time axis. Therefore, the t-x image is subjected to smooth filtering, so that part of noise can be filtered on the conventional x axis, namely the wavelength or the Raman shift axis, so that the curve is smooth, the spectrum synthesized again becomes uniform and continuous, the noise changing along with time can be filtered on the t axis, namely the time axis, and the noise of an unprocessed effective Raman signal area in the step two can be reduced without influencing the characteristic peak value of the Raman signal.
And (3) processing each frame of Raman spectrum acquired in the step (I) in the step (II), splicing the N +1 frames of Raman spectrum processed in the step (II) into a 2D image along a time axis, wherein the x axis is a wavelength or Raman shift axis, the t axis is a time axis, and the z axis is Raman intensity. And filtering the 2D image by an iterative bilateral filtering method, and after filtering, superposing and normalizing the N +1 spectrum along a time axis to obtain a clean Raman peak spectrum image.
The principle of the bilateral filtering method is as follows: in filtering algorithms, the value of a pixel at a target point is usually determined by the value of a small local neighbor around the position of the pixel. The specific implementation in 2D gaussian filtering is to assign different gaussian weight values to the pixel values in the surrounding predetermined range, and obtain the final result of the current point after weighted averaging. And the gaussian weight factor is generated using a spatial distance relationship between two pixels. The formulation description is generally as follows:
where c is the Gaussian weight based on spatial distance and k isd(x) For unitizing the results.
In the low-pass filtering algorithm, only the relation of spatial positions among pixels is considered in the Gaussian filtering, so that the filtering result loses edge information, and the edge information is an effective Raman signal with stable peak position along with time. The bilateral filtering is to add another weight subsection in the gaussian filtering to solve the problem. The preservation of edges in bilateral filtering is achieved by the following expression:
where s is a Gaussian weight based on the degree of similarity between pixels, kr(x) The same is used to unitize the results. The two are combined to obtain bilateral filtering based on spatial distance and comprehensive consideration of similarity:
the unitized part k (x) in the above formula is obtained by combining two Gaussian weights, wherein the calculation of c and s is described in detail as follows:
and d (xi, x) ═ d (xi-x) | xi-x |
And has sigma (phi, f) as sigma (phi-f) phi-f phi
The expressions given above are all infinite integration in space, which needs to be discretized in a pixelized image. Pixels whose distance exceeds a predetermined level have practically little influence on the current target pixel and can be ignored. The discretization after the definition of the local sub-region is simplified to the following form:
the iterative bilateral filtering method comprises the following operation steps: (1) setting a sigma factor sigma _ s of a bilateral filter function spatial domain, wherein the spatial domain refers to a t-x region, namely a 2D region formed by time, wavelength or Raman shift, and the sigma factor of the spatial domain is selected to contain more frames of spectra as much as possible but needs to be smaller than the Raman characteristic peak width; (2) setting a sigma factor sigma _ r of a bilateral filter function pixel range region, wherein the pixel range region refers to a Raman intensity range at each wavelength or Raman intensity position on each frame of spectrum, each time the sigma _ r is set, the sigma _ r is halved or subtracted by a preset step length until the sigma factor sigma _ r is smaller than the set minimum value Smin, keeping the sigma _ r equal to the Smin unchanged, selecting an initial value of the sigma _ r, and obtaining the sigma _ r by reserving two decimal numbers according to a result of dividing the sum of the average values of all noise regions by the half of the standard deviation of all noise regions obtained in the step two; (3) bilateral filtering is carried out on the constructed 2D image; (4) and (5) calculating the standard deviation and the average value ratio of each noise area of the filtered image according to the position information of the noise area recorded in the step two, and if the ratio is larger than a preset threshold value, returning to the step (2). (5) And obtaining a final iteration result after filtering.
The bilateral filtering method has the advantages that the noise randomly changing along with time is greatly reduced on the basis of effectively retaining the effective Raman signal with stable peak position along with time. And the weight can be well set by adjusting the size of the sigma _ r parameter, the single smoothing precision is improved by iteration, and the effective Raman signal cannot be blurred when noise is smoothed out like one-time smoothing.
Has the advantages that:
1. the characteristic peak extraction method for the low signal-to-noise ratio ultraviolet Raman spectrum disclosed by the invention adopts statistical classification, 2D construction and iteration bilateral filtering methods according to the physical characteristics of ultraviolet Raman signals and random noise, can more quickly and effectively obtain a clean Raman characteristic peak extraction result spectrum image without distortion compared with a common spectrum superposition or characteristic matching extraction method, and is easy to transplant to various systems.
2. The invention discloses a characteristic peak extraction method for low signal-to-noise ratio ultraviolet Raman spectrum, which can effectively predict and judge effective valleys and peaks by analyzing the physical characteristics of random noise, effectively classify the spectrum after being divided, and independently process noise regions.
3. The existing Raman spectrum filtering method mainly focuses on a filtering method of a space domain, and the characteristic peak extraction method for the low signal-to-noise ratio ultraviolet Raman spectrum disclosed by the invention has the advantages that as the effective Raman signal is stable along with the change of the time peak position, an obvious edge straight line is formed on a time axis, the noise is randomly changed along with the time, a random noise point is formed on the time axis, and more physical characteristics are introduced on the time axis, so that the filtering analysis is more facilitated.
4. The characteristic peak extraction method for the low signal-to-noise ratio ultraviolet Raman spectrum disclosed by the invention utilizes the edge protection characteristic of the bilateral filtering algorithm, not only removes noise points which cannot form edges in a 2D constructed Raman spectrum image and keeps Raman characteristic peak signals which form the edges, but also can set a very small Gaussian weighting factor for bilateral filtering, can ensure filtering precision through an iteration method, and cannot cause the situation that too much filtering at one time causes distortion of a noise reduction spectrum result.
Drawings
Fig. 1 is a general flow chart of the method for extracting characteristic peaks of low signal-to-noise ratio uv raman spectroscopy of the present invention.
FIG. 2 is a flow chart illustrating steps of the present invention.
FIG. 3 is a flow chart illustrating the steps of the present invention.
Fig. 4 is a schematic flow chart of steps of the method, wherein fig. 4a is a schematic flow chart of 2D spectral image construction, and fig. 4b is a schematic flow chart of processing of an iterative bilateral filtering method.
Fig. 5 is a schematic diagram of an example of a real-time ultraviolet raman spectrum processing result of heroin according to the present invention, where fig. 5a is an unprocessed frame of raman spectrum original image, fig. 5b is a temporarily stored 10 frames of raman spectrum original image to be processed, and fig. 5c is a processed raman feature peak image extraction result.
Fig. 6 is a schematic diagram of an example of a real-time uv-raman spectrum processing result of another sample according to the present invention, where fig. 6a is an original drawing of an untreated frame of acetaminophen tablet raman spectrum, fig. 6b is an extraction result of a treated acetaminophen tablet raman characteristic peak image, fig. 6c is an original drawing of an untreated frame of cefixime dispersible tablet raman spectrum, and fig. 6d is an extraction result of a treated cefixime dispersible tablet raman characteristic peak image.
Detailed Description
For a better understanding of the objects and advantages of the present invention, reference should be made to the following detailed description taken in conjunction with the accompanying drawings and examples.
Example 1:
at present, a portable 266nm laser excitation Raman spectrum system self-developed in a laboratory is utilized to carry out Raman detection on Heroin Heroin samples, the acquisition integration time is 2 seconds, a baseline correction function is started, namely, the fluorescence background noise of the spectrum is removed when each frame of Raman spectrum is acquired, N is 9, namely, when a new frame of spectrum is received, the first 9 frames of spectrum are temporarily stored, the x axis is Raman shift, algorithm processing is started after the samples are stably placed for 6 seconds, and the processing flow of the characteristic peak extraction method for the low signal-to-noise ratio ultraviolet Raman spectrum is shown in figure 1.
As shown in fig. 1, the specific implementation method of the characteristic peak extraction method for the low signal-to-noise ratio uv-raman spectrum disclosed in this embodiment is as follows:
the method comprises the following steps: as shown in fig. 2, the ultraviolet raman signal of the heroin sample is collected in real time, and the data of the collected raman signal is counted, so as to predict the effective peak value of the raman signal, wherein the effective peak value comprises effective valleys and peaks.
The method comprises the steps of acquiring ultraviolet Raman signals of a heroin sample in real time from a spectrometer in a front-end Raman detection system, transmitting the acquired real-time Raman signals to a computer signal buffer area through USB communication, wherein the signal buffer area is used for keeping 10 frames of stored spectra, deleting the first 10 frames of spectral data when acquiring a new frame of spectrum, and finishing the acquisition of Raman signal data. And performing valley and peak statistics on the 10 frames of images, and listing a valley netlist and a peak netlist. Presetting a statistical classification allowable interval step size to be 3, if the peak netlist of the 9-frame spectrum has peaks in the interval with the step size of 3 and the absolute value of the slope at the left end or the right end of the interval is more than 240, marking the peak as an effective peak, wherein the effective peak is a Raman peak with high intensity, and recording the position of the effective peak. For the 10-frame spectrum valley netlist, the true spectrum valley point is predicted by the following method, and the true spectrum valley point is predicted under the following three conditions: in case one, if the spectrum of less than or equal to 5 frames has a valley in the step length 3, the spectrum is not an effective valley; in the second case, if the spectrum of more than or equal to 9 frames has a valley in one step length 3, the spectrum is an effective valley, and the Raman shift position of the valley is recorded; and thirdly, under the condition that a valley exists in a step length 3 of the spectrum which is more than 5 and less than 9 frames, observing whether the near step length around the valley value is in a region of 3, whether a Raman peak with high intensity recorded in the peak netlist exists, recording the Raman peak into the valley netlist if an effective valley exists, superposing the 10 frames of the spectrum if the effective valley exists in the region of 3, recording the effective valley into the valley netlist, predicting the true spectrum valley point according to the three conditions, and obtaining the valley netlist. And predicting the effective peak positions of the Raman signal through the statistical low-valley netlist and the statistical peak netlist.
Step two: as shown in fig. 3, for each frame of raman spectrum, dividing each frame of raman spectrum into each block region through the effective valley obtained in the first step, and respectively determining the attribute of each block region, where the region attribute refers to the attribute of a noise region or the attribute of an effective raman signal region. And aiming at each noise region, each point is equal to the minimum value of the region, and then, the Raman signal region and the processed noise region are spliced.
And C, dividing each frame of Raman spectrum acquired in the step I into each block area through the valley netlist of each frame of Raman spectrum predicted in the step I, judging the attribute of each area, and respectively determining the attribute of each block area, namely respectively determining whether each block area is a noise area or an effective Raman signal area. For a single region, there are two criteria: the first criterion is: when a Raman peak with high intensity recorded in the peak net list obtained in the first step exists in the region, determining the Raman peak as an effective Raman signal region; the second criterion is: and selecting the length 9 as a step length, carrying out least square method estimation smoothing, judging the ratio of the standard deviation and the average value of the smoothed data in the region, if the ratio of the standard deviation and the average value is more than 0.19, determining the region as an effective Raman signal region, otherwise, determining the region as a noise region.
After determining the attribute of each block area, aiming at each noise area, each point is equal to the minimum value of the area, and all the wavelength or Raman shift position information of the area which is judged as the noise attribute is recorded. Then, the effective Raman signal region and the filtered noise region are spliced.
And step three, as shown in fig. 4a, processing each frame of raman spectrum acquired in the step one in the step two, splicing the 10 frames of spectra processed in the step two into a 2D image along a time axis, filtering the 2D image by an iterative bilateral filtering method shown in fig. 4b, and after filtering, superposing and normalizing the 10 spectra along the time axis to obtain a clean raman characteristic peak spectrum image.
And (3) processing each frame of Raman spectrum acquired in the step (I) in the step (II), splicing the 10 frames of Raman spectrum processed in the step (II) into a 2D image along a time axis, wherein the x axis is a wavelength or Raman shift axis, the t axis is a time axis, and the z axis is Raman intensity. The 2D image is filtered by an iterative bilateral filtering method.
As shown in fig. 4b, the operation steps of the iterative bilateral filtering method are as follows: (1) setting a sigma factor sigma _ s of a bilateral filter function spatial domain to be 10; (2) setting a sigma factor sigma _ r of a bilateral filter function pixel range, selecting an initial value of the sigma _ r to be 0.16, halving the sigma _ r every time the set sigma _ r is returned until the set sigma _ r is less than 0.02, and keeping the sigma _ r constant to 0.02; (3) bilateral filtering is carried out on the constructed 2D image; (4) and (4) calculating the average difference value of the intensity of each position of the current processing result image and the last processing result image, and if the difference value is more than 0.0001, returning to the step (2). (5) And obtaining a final iteration result after filtering.
After filtering, overlapping and normalizing the 10 frames of spectra along a time axis to obtain a clean Raman characteristic peak spectrum image.
Fig. 5a shows an unprocessed heroin raman spectrum original image, fig. 5b shows a temporarily stored 10 heroin raman spectrum original images to be processed, and fig. 5c shows a heroin raman characteristic peak image extraction result processed by the method of the present invention.
Example 2:
the characteristic peak extraction method aiming at the low signal-to-noise ratio ultraviolet Raman spectrum of the patent is utilized to process real-time ultraviolet Raman detection signals of acetaminophen tablets and cefixime dispersible tablets respectively, and the acquisition method and conditions of the real-time ultraviolet Raman signals are consistent with those of the embodiment 1. When the characteristic peak extraction method for the low signal-to-noise ratio ultraviolet Raman spectrum is implemented on an acetaminophen sheet, the absolute value threshold of the slope at the left end of the interval selected in the step one is 35, the threshold of the ratio of the standard deviation to the average value in the step two is 0.175, the initialization value of sigma _ r in the step three is 0.04, and the selection of Smin is 0.016. When the method for extracting the characteristic peak of the low signal-to-noise ratio ultraviolet Raman spectrum is implemented on the cefixime dispersible tablets, the absolute value threshold of the slope at the left end of the selected interval in the first step is 50, the threshold of the ratio of the standard deviation to the average value in the second step is 0.12, the initialization value of sigma _ r in the third step is 0.1, and the Smin is 0.015.
FIG. 6a shows an original drawing of an unprocessed one-frame acetaminophen sheet Raman spectrum, and FIG. 6b shows a processed acetaminophen sheet Raman feature peak image extraction result; fig. 6c shows an original drawing of a raman spectrum of an untreated frame of cefixime dispersible tablets, and fig. 6d shows a raman characteristic peak image extraction result of treated cefixime dispersible tablets.
The above detailed description is intended to illustrate the objects, aspects and advantages of the present invention, and it should be understood that the above detailed description is only exemplary of the present invention and is not intended to limit the scope of the present invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (7)
1. The characteristic peak extraction method for the low signal-to-noise ratio ultraviolet Raman spectrum is characterized by comprising the following steps of: comprises the following steps of (a) carrying out,
the method comprises the following steps: collecting ultraviolet Raman signals in real time, and counting the collected Raman signal data to predict effective peak values of the Raman signals, wherein the effective peak values comprise effective valleys and peaks;
step two: for each frame of Raman spectrum, dividing each frame of Raman spectrum into each block region through the effective valley obtained in the first step, and respectively judging the attribute of each divided block region, wherein the region attribute refers to the attribute of a noise region or the attribute of an effective Raman signal region; aiming at each noise area, each point is equal to the minimum value of the area, and then the Raman signal area and the processed noise area are spliced;
step three, processing each frame of Raman spectrum acquired in the step one in the step two, splicing the N +1 frames of Raman spectrum processed in the step two into a 2D image along a time axis, filtering the 2D image by an iterative bilateral filtering method, and superposing and normalizing the N +1 frames of Raman spectrum along the time axis after filtering to obtain a clean Raman characteristic peak spectrum image;
wherein, the first realization method of the step is that,
acquiring an ultraviolet Raman signal in real time from a spectrometer in a front-end Raman detection system, and transmitting the acquired Raman signal to a signal cache region, wherein the signal cache region is used for keeping access to N +1 frames of spectra, and deleting the previous N +1 frames of spectral data when a new spectrum is acquired, so as to finish the acquisition of Raman signal data; performing low valley and peak statistics on the N +1 frame image, and listing a low valley netlist and a peak netlist; presetting a step length of statistical classification allowable interval, dividing the whole spectrum into M allowable intervals along an x-axis, wherein the x-axis is a wavelength axisOr Raman shift axis, if the peak net list of the N frames of spectra has peaks in each allowable interval and the absolute value of the slope at the left end or the right end of the interval is greater than the preset threshold KmaxNamely N frames of spectra all satisfy the following condition,
the allowable interval has f' (x)m)＞Kmax&&f'(xm+1)≤0,m∈CmM1, 2, …, M the predetermined threshold KmaxIf the value is equal to the absolute value of the maximum slope caused by random noise, marking the value as an effective peak, wherein the effective peak is a Raman peak with high intensity, and recording the position of the effective peak;
for the valley netlist of the N +1 frame spectrum, the judgment condition of the valley point is given by the following formula,
the allowable interval has f' (x)m)≤0&&f'(xm+1)≥0,m∈Cm，m＝1,2,…,M
Then, the true spectrum valley point is predicted by the following method, and the true spectrum valley point is predicted by totally dividing the following three conditions: in the first case, in an allowable interval step length, if a valley point exists in a spectrum of less than or equal to (N +1)/2 frames, the spectrum is not an effective valley; in a step length of an allowable interval, if a valley point exists in the spectrum of more than or equal to N frames, the spectrum is an effective valley, and the effective valley position is calibrated and recorded according to the average value or the weight average value; in a step length of an allowable interval, a valley point exists in a spectrum which is larger than (N +1)/2 and smaller than N frames, and whether the spectrum is a valid valley is judged by a preset prediction condition, wherein the valley point possibly exists in the spectrum which is formed by the influence of noise at a low intensity peak; predicting a true spectrum valley point through the three conditions to obtain a valley netlist; and predicting the effective peak positions of the Raman signal through the statistical low-valley netlist and the statistical peak netlist.
2. The method for extracting characteristic peaks in low signal-to-noise ratio ultraviolet raman spectroscopy according to claim 1, characterized in that: the second step is realized by the method that,
dividing each frame of Raman spectrum obtained in the first step into each block area through the valley netlist of each frame of Raman spectrum predicted in the first step, judging the attribute of each area, and respectively determining the attribute of each block area, namely respectively determining whether each block area is a noise area or an effective Raman signal area; for a single region, there are two criteria: the first criterion is: when a Raman peak with high intensity recorded in the peak net list obtained in the first step exists in the region, determining the Raman peak as an effective Raman signal region; the second criterion is: selecting the length of p allowable intervals in the step one as a step length, performing least square method smoothing, judging the ratio of the standard deviation and the average value of the smoothed data in the region, if the ratio of the standard deviation and the average value is greater than a preset threshold value, determining the region as an effective Raman signal region, otherwise, determining the region as a noise region; the preset threshold is one half of the average value of the ratio of the standard deviation to the average value after all the areas take the p allowable interval lengths in the step one as step lengths and are smoothed by the least square method; the first judgment standard is used for detecting a strong peak area; the second judgment standard is used for judging the weak peak;
after determining the attribute of each block area, aiming at each noise area, enabling each point to be equal to the minimum value of the area, and recording all the wavelength or Raman displacement position information of the area judged as the noise attribute; then, the effective Raman signal region and the filtered noise region are spliced.
3. The method for extracting characteristic peaks in low signal-to-noise ratio ultraviolet raman spectroscopy according to claim 2, characterized in that: the third step is to realize the method as follows,
processing each frame of Raman spectrum acquired in the first step by using a second step, splicing the N +1 frames of Raman spectrum processed in the second step into a 2D image along a time axis, wherein the x axis is a wavelength or Raman shift axis, the t axis is a time axis, and the z axis is Raman intensity; and filtering the 2D image by an iterative bilateral filtering method, and after filtering, superposing and normalizing the N +1 spectrum along a time axis to obtain a clean Raman peak spectrum image.
4. The method for extracting characteristic peaks in low signal-to-noise ratio ultraviolet raman spectroscopy according to claim 1,2 or 3, wherein: the specific judgment method of the preset prediction condition for judging whether the prediction condition is the effective valley is as follows: and observing whether a high-intensity Raman peak recorded in the peak netlist exists in an allowable interval step region with a low valley value close to the left and right, recording the high-intensity Raman peak into a valley netlist if the high-intensity Raman peak exists in the peak netlist, if the high-intensity Raman peak exists in an effective valley, recording the high-intensity Raman peak into the valley netlist, if the high-intensity Raman peak does not exist in the effective valley, overlapping the N +1 frame spectrum, and if the low valley exists in the allowable interval step, recording the effective valley into the valley netlist.
5. The method for extracting characteristic peaks in low signal-to-noise ratio ultraviolet Raman spectroscopy according to claim 4, wherein: step one, the step size of the statistical classification tolerance interval is selected to be 3.
6. The method for extracting characteristic peaks in low signal-to-noise ratio ultraviolet raman spectroscopy according to claim 2 or 3, wherein: selecting the number of p in the p allowable intervals in the step two as 3;
the principle of the second judgment standard is that the Raman width of a weak peak is larger than the sampling width, the noise width is the sampling width, the least square filtering is carried out according to the estimated step length of the average width of the weak peak, the least square filtering smoothes the noise into an approximate straight line region, the weak peak can be effectively reserved, the ratio of the standard to the average value of the approximate straight line region after smoothing is obviously smaller than the ratio of the standard deviation to the average value of the weak peak region, and then the noise region or the effective Raman signal region is judged by calculating the ratio of the standard deviation to the average value of the region.
7. The method for extracting characteristic peaks in low signal-to-noise ratio ultraviolet raman spectroscopy according to claim 1,2 or 3, wherein: the iterative bilateral filtering method comprises the following operation steps: (1) setting a sigma factor sigma _ s of a bilateral filter function spatial domain, wherein the spatial domain refers to a t-x region, namely a 2D region formed by time, wavelength or Raman shift, and the sigma factor of the spatial domain is selected to contain more frames of spectra as much as possible but needs to be smaller than the Raman characteristic peak value width; (2) setting a sigma factor sigma _ r of a bilateral filter function pixel range region, wherein the pixel range region refers to a Raman intensity range at each wavelength or Raman intensity position on each frame of spectrum, each time the sigma _ r is set, the sigma _ r is halved or subtracted by a preset step length until the sigma factor sigma _ r is smaller than the set minimum value Smin, keeping the sigma _ r equal to the Smin unchanged, selecting an initial value of the sigma _ r, and obtaining the sigma _ r by reserving two decimal numbers according to a result of dividing the sum of the average values of all noise regions by the half of the standard deviation of all noise regions obtained in the step two; (3) bilateral filtering is carried out on the constructed 2D image; (4) calculating the standard deviation and the average value ratio of each noise area of the filtered image according to the position information of the noise area recorded in the step two, and if the ratio is larger than a preset threshold value, returning to the step (2); (5) and obtaining a final iteration result after filtering.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910327282.9A CN109993155B (en) | 2019-04-23 | 2019-04-23 | Characteristic peak extraction method for low signal-to-noise ratio ultraviolet Raman spectrum |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910327282.9A CN109993155B (en) | 2019-04-23 | 2019-04-23 | Characteristic peak extraction method for low signal-to-noise ratio ultraviolet Raman spectrum |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109993155A CN109993155A (en) | 2019-07-09 |
| CN109993155B true CN109993155B (en) | 2021-01-01 |
Family
ID=67135082
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910327282.9A Active CN109993155B (en) | 2019-04-23 | 2019-04-23 | Characteristic peak extraction method for low signal-to-noise ratio ultraviolet Raman spectrum |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109993155B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20230050999A1 (en) * | 2020-01-24 | 2023-02-16 | Panasonic Intellectual Property Management Co., Ltd. | Raman spectroscopy method and raman spectroscopy support device |
| CN112161966B (en) * | 2020-09-29 | 2021-08-10 | 中国科学院长春光学精密机械与物理研究所 | Method and device for separating Raman spectrum of sample containing fluorescence spectrum |
| CN113567604B (en) * | 2021-07-22 | 2022-09-30 | 华谱科仪(大连)科技有限公司 | Detection and analysis method of chromatographic spectrogram and electronic equipment |
| CN113281324A (en) * | 2021-06-29 | 2021-08-20 | 江苏大学 | Characteristic information extraction method of small molecule volatile matter and portable detection system |
| CN113640445B (en) * | 2021-08-11 | 2024-06-11 | 贵州中烟工业有限责任公司 | Characteristic peak identification method based on image processing, computing device and storage medium |
| CN114113035B (en) * | 2021-11-18 | 2024-02-02 | 北京理工大学 | Identification method of transgenic soybean oil |
| CN114280002B (en) * | 2021-12-16 | 2023-05-30 | 宜宾五粮液股份有限公司 | Abnormal fermented grain spectrum screening method based on characteristic peak judgment |
| CN115078616B (en) * | 2022-05-07 | 2024-06-07 | 天津国科医疗科技发展有限公司 | Multi-window spectrum peak identification method, equipment, medium and product based on signal to noise ratio |
| CN115001571B (en) * | 2022-05-11 | 2023-09-12 | 青岛诺克通信技术有限公司 | Adaptive digital filtering method of optical time domain reflectometer and optical time domain reflectometer |
| CN115389481B (en) * | 2022-08-16 | 2024-05-24 | 华中科技大学 | Method for detecting uniformity of biomass surface coating based on Raman spectrum surface scanning |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107818298A (en) * | 2017-09-27 | 2018-03-20 | 厦门大学 | General Raman spectral characteristics extracting method for machine learning material recognition |
| CN108918499A (en) * | 2018-06-28 | 2018-11-30 | 华南师范大学 | The method of Raman baseline drift is removed in Raman map |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103743717B (en) * | 2013-12-03 | 2015-09-23 | 中国工程物理研究院化工材料研究所 | Explosive CL-20 crystal formation quantitative test raman signatures area determination method |
| CN105138826A (en) * | 2015-08-10 | 2015-12-09 | 厦门大学 | Raman signal reconstruction method under strong noise background |
| CN107179310B (en) * | 2017-06-01 | 2018-02-27 | 温州大学 | Raman spectrum characteristic peak recognition methods based on robust noise variance evaluation |
| CN107764797B (en) * | 2017-09-21 | 2020-03-03 | 天津大学 | Raman spectrum image data preprocessing method based on low-rank tensor algorithm |
-
2019
- 2019-04-23 CN CN201910327282.9A patent/CN109993155B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107818298A (en) * | 2017-09-27 | 2018-03-20 | 厦门大学 | General Raman spectral characteristics extracting method for machine learning material recognition |
| CN108918499A (en) * | 2018-06-28 | 2018-11-30 | 华南师范大学 | The method of Raman baseline drift is removed in Raman map |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109993155A (en) | 2019-07-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109993155B (en) | Characteristic peak extraction method for low signal-to-noise ratio ultraviolet Raman spectrum | |
| CN110032988B (en) | Real-time noise reduction enhancement method for ultraviolet Raman spectrum system | |
| Lu et al. | Robust and efficient saliency modeling from image co-occurrence histograms | |
| CN107179310B (en) | Raman spectrum characteristic peak recognition methods based on robust noise variance evaluation | |
| Uss et al. | Maximum likelihood estimation of spatially correlated signal-dependent noise in hyperspectral images | |
| CN110335204B (en) | Thermal imaging image enhancement method | |
| CN107657209B (en) | Template image registration mechanism based on finger vein image quality | |
| CN110763913B (en) | Derivative spectrum smoothing processing method based on signal segmentation classification | |
| CN116958036A (en) | Method for inspecting pollution state of insulator by unmanned aerial vehicle based on hyperspectral technology | |
| CN114758219B (en) | Trace identification method based on spectral data and infrared temperature data fusion | |
| US8189860B2 (en) | Systems and methods of using spatial/spectral/temporal imaging for hidden or buried explosive detection | |
| CN114998664A (en) | Rapid detection method and device for micro-plastic in seawater by multiple optical platforms | |
| Cappel et al. | Removing cosmic ray features from Raman map data by a refined nearest neighbor comparison method as a precursor for chemometric analysis | |
| KR101750520B1 (en) | Method for improving detection performance of hyperspectral target detection by using target signal exclusion | |
| CN111089856B (en) | Post-processing method for extracting Raman spectrum weak signal | |
| Bhushan et al. | An effective pre-processing algorithm for detecting noisy spectral bands in hyperspectral imagery | |
| Orczyk et al. | Fingerprint ridges frequency | |
| CN114330411A (en) | Self-adaptive windowed Raman spectrum identification method based on similarity | |
| Nguyen et al. | A novel automatic concrete surface crack identification using isotropic undecimated wavelet transform | |
| CN116777917A (en) | Defect detection method and system for optical cable production | |
| CN116503372A (en) | Spectrogram noise detection method for spectrometer | |
| Yan et al. | Non-intrusive fingerprints extraction from hyperspectral imagery | |
| CN111814727B (en) | Hyperspectrum-based potential fingerprint detection method | |
| Guan et al. | A new metric for latent fingerprint image preprocessing | |
| US10713787B2 (en) | System and method for automated stereology of cancer |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |