CN110246096B - Fitting correction method and device for scattered X-ray - Google Patents
Fitting correction method and device for scattered X-ray Download PDFInfo
- Publication number
- CN110246096B CN110246096B CN201910461574.1A CN201910461574A CN110246096B CN 110246096 B CN110246096 B CN 110246096B CN 201910461574 A CN201910461574 A CN 201910461574A CN 110246096 B CN110246096 B CN 110246096B
- Authority
- CN
- China
- Prior art keywords
- image
- light source
- point light
- irradiation
- diagram
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000012937 correction Methods 0.000 title claims abstract description 15
- 238000010586 diagram Methods 0.000 claims abstract description 89
- 238000012545 processing Methods 0.000 claims abstract description 33
- 238000004364 calculation method Methods 0.000 claims abstract description 19
- 230000005855 radiation Effects 0.000 claims abstract description 18
- 238000005286 illumination Methods 0.000 claims description 22
- 238000010606 normalization Methods 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 3
- 238000011084 recovery Methods 0.000 claims description 3
- 238000009499 grossing Methods 0.000 claims description 2
- 238000003672 processing method Methods 0.000 abstract description 2
- 238000010587 phase diagram Methods 0.000 description 5
- 238000005457 optimization Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 210000001015 abdomen Anatomy 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000005658 nuclear physics Effects 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000000988 bone and bone Anatomy 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000001678 irradiating effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G06T5/73—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10116—X-ray image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
Abstract
The invention discloses a fitting correction method and a fitting correction device for scattered X-ray rays, wherein the method comprises the following steps: acquiring an original image of X-ray, and extracting a low-resolution image of an Nth layer from the original image based on a pyramid hierarchical model; calculating the total brightness value of the initial radiation diagram according to the brightness value of each point light source in the low-resolution image, and adjusting the brightness of the total brightness values of all pixel points of the initial radiation diagram to obtain a first radiation diagram; carrying out nonlinear processing on the first scattered ray diagram to obtain an optimized second scattered ray diagram; and performing interpolation processing on the second scattered ray diagram to obtain a third scattered ray diagram with the same resolution as the original image, and performing difference operation on each pixel point of the original image and the third scattered ray diagram to obtain a corrected image. The method can obtain the scattered ray diagram through calculation in a mathematical operation mode, eliminate the influence of the scattered ray on the image and obtain a clear image, and is simpler and easier to understand compared with other image processing methods.
Description
Technical Field
The invention relates to the field of X-ray scattered rays, in particular to a fitting correction method and device for X-ray scattered rays.
Background
X-ray is widely applied to the technical field of existing medicine, and the X-ray is imaged on a detector through a human body. X-ray radiation produces scatter when it strikes the body. The thicker the area, the more easily scattering occurs. For example, the images of lumbar vertebrae and abdomen are more likely to generate scattering than the images of limbs. The adverse effects of scattering on the image are: scattered rays cause the shot image to be blurred, and the final contrast and details of the image are influenced, so that the image is similar to haze and information in the image cannot be accurately identified.
In the prior art, a method of physically eliminating scattered rays is generally adopted, and X-ray scattered rays are filtered by adding a grid in an instrument; and adopting software to eliminate scattered rays, eliminating the interference of the scattered rays on the image through an image processing method, finding out the influence of the scattered rays on the image, and removing the influence area of the scattered rays in the image to obtain an original real image, for example, simulating that X-ray irradiates an object and collides with a courtyard of the object by utilizing a nuclear physics principle to obtain a distribution model of the scattered rays so as to estimate a scattered image; the method of the nuclear physics principle in the prior art is based on a real physical model, but is complex in understanding and operation, is not beneficial to wide technical personnel in the field to understand and apply, is difficult to obtain a scattered ray diagram, and is difficult to eliminate the influence of X-ray scattered rays on the image.
Therefore, how to obtain an X-ray scattergram by a simple and easy-to-understand method and eliminate the influence of the X-ray scattergram on the image are research directions in the field.
Disclosure of Invention
The application provides a fitting correction method and device for scattered X-ray, which can solve the technical problems that a scattered ray image cannot be obtained through a simple image processing algorithm and the influence of the scattered X-ray on the image cannot be eliminated in the prior art.
The invention provides a method for correcting the fitting of scattered X-ray rays, which comprises the following steps:
acquiring an original image of X-ray, and extracting a low-resolution image of an Nth layer from the original image based on a pyramid hierarchical model;
determining an irradiation range corresponding to each point light source according to the brightness value of each point light source in the low-resolution image, obtaining the irradiation intensity of the corresponding point light source by using the irradiation range, determining the total brightness value of each pixel point in the irradiation range corresponding to the irradiation of each point light source according to the irradiation intensity, taking the total brightness value of each pixel point as the total brightness value of an initial ray scattering diagram, and performing brightness adjustment on the total brightness values of all the pixel points of the initial ray scattering diagram to obtain a first ray scattering diagram containing the pixel points of the irradiation range of all the point light sources;
carrying out nonlinear processing on the first scattered ray map to obtain an optimized second scattered ray map;
and performing interpolation processing on the second scattered ray image to obtain a third scattered ray image with the same resolution as the original image, and performing difference operation on each pixel point of the original image and the third scattered ray image to obtain a corrected image.
Optionally, the step of determining the illumination range corresponding to each point light source according to the brightness value of each point light source in the low-resolution image includes:
determining the maximum irradiation radius and the minimum irradiation radius of each point light source according to the brightness value of each point light source in the low-resolution image;
calculating a total irradiation radius of each point light source by using the brightness value of each point light source, the maximum irradiation radius and the minimum irradiation radius to obtain the irradiation range;
wherein, the calculation formula of the total irradiation radius is as follows:
R＝R_min+(R_max-R_min)/I_Detect_D6(i,j)
when R _ min =0, R = R _ max/I _ Detect _ D6 (I, j)
Wherein R represents the total irradiation radius, R _ min represents the minimum irradiation radius, R _ max represents the maximum irradiation radius, (I, j) represents a coordinate position of the point light source, and I _ Detect _ D6 (I, j) represents a luminance value of the point light source.
Optionally, the step of obtaining the illumination intensity of the corresponding point light source by using the illumination range includes:
obtaining a scale parameter by using the total irradiation radius of the irradiation range of the point light source, wherein the scale parameter is as follows:
wherein σ represents the scale parameter, R represents the total irradiation radius, and n represents a preset constant;
obtaining the illumination intensity of the point light source by using the scale parameters, wherein the formula is as follows:
wherein f (x) represents the irradiation intensity of the point light source, μ represents the position of the point light source, the coordinate of the point light source is (i, j), x represents the position of any pixel point in the irradiation range, and the coordinate (x, y) of any pixel point.
Optionally, the step of determining a total brightness value of each pixel point within an irradiation range corresponding to the irradiation of each point light source according to the irradiation intensity, and taking the total brightness value of each pixel point as a total brightness value of an initial radiation diagram includes:
and determining the brightness value of a pixel point in an irradiation range corresponding to the irradiation of the point light source according to the irradiation intensity, superposing the brightness values of the same pixel point from different point light sources, and taking the total brightness value of each pixel point as the total brightness value of the initial scattered ray diagram.
Optionally, the step of adjusting the brightness of the total brightness values of all the pixel points of the initial radiation diagram to obtain a first radiation diagram including the pixel points in the irradiation range of all the point light sources includes:
and after smoothing and filtering the initial scattering base map, carrying out threshold normalization processing, counting a histogram of the brightness values of the initial scattering base map and a histogram of the brightness values of the preset light source, obtaining a brightness coefficient according to the histograms, and carrying out brightness adjustment on the total brightness values of all pixel points of the initial scattering base map according to the brightness coefficient to obtain a first scattering base map containing the pixel points in the irradiation range of all point light sources.
Optionally, the step of performing nonlinear processing on the first scatter diagram to obtain an optimized second scatter diagram includes:
carrying out nonlinear processing on the first scattered ray diagram to obtain a second scattered ray diagram; the formula is as follows:
I_scatter_2＝a+b·log c (I_scatter_D6)
i _ scatter _2 represents the second scatter diagram, I _ scatter _ D6 represents the first scatter diagram, and a, b and c represent preset parameters;
and obtaining a candidate real image by using the low-resolution image and the second scattered ray image, and performing sorting according to the contrast of the candidate real image to obtain an optimized second scattered ray image.
Optionally, the step of obtaining the optimized second scattergram by obtaining a candidate real image from the low-resolution image and the second scattergram and performing sorting according to the contrast of the candidate real image includes:
obtaining a candidate real image by using the low-resolution image and the second scattered ray map, wherein the formula is as follows:
I_D6＝I_Detect_D6-I_scatter_2
wherein I _ D6 represents the candidate real image, I _ Detect _ D6 represents the low resolution image, and I _ scatter _2 represents the second scatter diagram;
selecting a preset area of the candidate real image to perform automatic histogram extension, calculating the variance of the candidate real image to obtain the contrast of the candidate real image,
calculating the variance of the candidate real image, wherein the formula is as follows:
where D represents the variance of the candidate real image, I _ D6 (x) 1 ,y 1 ) And I _ D6 (x) n ,y n ) A luminance value, I _ D6, representing a pixel point in said candidate real image average Representing the average brightness value of all pixel points of the candidate real image;
and selecting the contrast of the candidate real image, and selecting the candidate real image corresponding to the maximum contrast to obtain the optimized second radiation diagram.
Optionally, the step of performing interpolation calculation on the second scattergram to obtain a third scattergram having the same resolution as the original image includes:
and performing interpolation calculation on the optimized second scattered ray map by using a bilinear interpolation method to obtain a third scattered ray map with the same resolution as the original image, wherein the formula is as follows:
f(d+u,d+u)＝(1-d) 2 x f (d, d) + (1-d) x u x f (d, d + 1) + u x (1-u) x f (d +1, d) + u x f (d +1 ) wherein f (d + u ) represents the third scattergram, (d + u ) represents the resolution, the resolution of the optimized second scattergram is (d, d), and the optimized second scattergram f (d, d) = I _ scatter _1.
Optionally, the step of performing a difference operation on the original image and each pixel point of the third scattergram to obtain a corrected image includes:
calculating the corrected image according to the following formula:
I＝I_Detect-I_scatter
wherein I represents the corrected image, I _ Detect represents the original image, and I _ scatter represents the third scattergram.
Another aspect of the present application provides an apparatus for fitting and correcting scattered X-ray, comprising:
the data processing module is used for acquiring an original image of X-ray and extracting a low-resolution image of an Nth layer from the original image based on a pyramid hierarchical model;
the brightness processing module is used for determining the irradiation range corresponding to each point light source according to the brightness value of each point light source in the low-resolution image, obtaining the irradiation intensity of the corresponding point light source by using the irradiation range, determining the total brightness value of each pixel point in the irradiation range corresponding to the irradiation of each point light source according to the irradiation intensity, and taking the total brightness value of each pixel point as the total brightness value of the initial scattered ray diagram;
the image acquisition module is used for adjusting the brightness of the total brightness values of all the pixel points of the initial radiation diagram to obtain a first radiation diagram containing the pixel points in the irradiation range of all the point light sources;
the image comparison module is used for carrying out nonlinear processing on the first scattered ray image to obtain an optimized second scattered ray image;
the image calculation module is used for carrying out interpolation processing on the second scattered ray image to obtain a third scattered ray image with the same resolution as the original image;
and the image recovery module is used for performing difference operation on the original image and each pixel point of the third scattered ray diagram to obtain a corrected image.
The invention provides a fitting correction method and a fitting correction device for X-ray scattered rays, wherein an original image irradiated by X-ray is used as a target image based on a light model, one layer of image with low resolution is obtained, each calculation parameter is obtained through the brightness value of the target image, a first scattered ray image, a second scattered ray image and a third scattered ray image are obtained in sequence, and a corrected image is obtained through the difference operation of the original image and the third scattered ray image, so that the influence of the scattered rays in the original image is eliminated.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is an image with scattered ray effect of an acquired image according to an embodiment of the present application;
FIG. 2 is a positive phase diagram and a negative phase diagram of the collected image of the X-ray photographed human body of the present application;
FIG. 3 is a schematic diagram of a lighting model-based captured image of X-rays in an embodiment of the present application;
FIG. 4 is a real image after the influence of scattered rays is eliminated in the embodiment of the present application;
FIG. 5 is a step diagram of a method for calibrating scattered X-ray according to the present application;
fig. 6 is an architecture diagram of an apparatus for correcting scattered X-ray according to the present application.
Detailed Description
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The technical problems that a scattered ray image cannot be obtained through a simple image processing algorithm and the influence of X-ray scattered rays on the image cannot be eliminated in the prior art are solved.
Please refer to fig. 1, which illustrates an image under the influence of scattered rays collected in an embodiment of the present application; in the embodiment of the application, the X-ray passes through the human body to be imaged on the detector, and when the X-ray irradiates the human body structure, the thicker part is easier to generate scattering. If the X-ray is used for shooting lumbar vertebrae and abdomen in a human body, scattering is easy to generate, the scattered ray can affect an image of a shooting result, the brightness of the image is not balanced enough, the local brightness of the image is too high, a dynamic range is occupied, the contrast of the image is low, and the shooting result cannot be observed easily.
In order to solve the above technical problems, the present invention provides a method and an apparatus for fitting and correcting scattered X-ray.
Please refer to fig. 2, which is a positive phase diagram and a negative phase diagram of the collected images of the human body photographed by the X-ray according to the present application; in this embodiment, when the X-ray irradiates the human body, the positive phase diagram of the captured image shows that the brightness is lower in the thick and dense places; the inverse image shows a higher brightness where the volume is thick and dense, for example, the bone is white and the air region is black.
Please refer to fig. 3, which is a schematic diagram of an X-ray captured image based on a light model according to an embodiment of the present application; in an embodiment of the application, the inverse of the acquired image is: the thick or high density part shows higher brightness, larger scattering intensity and wider influence range, and the air area shows lower brightness and lower scattering intensity; therefore, the inverse image of the acquired image is taken as an original image, the original image is abstracted into a light model, and each pixel in the original image is understood as a lamp. In the light model, the brightness of the lamp is assumed to be the pixel brightness value, so that the stronger the light brightness is, the brighter the light irradiates the surroundings, and the wider the irradiation range is; further, in the light model in this embodiment, a pyramid hierarchical model is used to divide the resolution of the original image of the light model into a plurality of layers, where the image resolution of the first layer is 3072 × 3072, the image resolution of the second layer is 1560 × 1560, and the image resolution of the third layer is 768 × 768.... The image resolution of the sixth layer is 48 × 48; in this embodiment, a gaussian map of the sixth layer of the pyramid is used for calculation and is labeled as I _ Detect _ D6. The scatter plot, labeled I _ scatter _ D6, is a superimposed representation of the influence of each pixel (which can be understood as "each lamp", "each point source") in the gaussian plot of the sixth layer of the original image on the surroundings.
Please refer to fig. 4, which is a real image after eliminating the influence of scattered rays in the embodiment of the present application; in the embodiment of the application, in order to obtain a real image for eliminating the influence of scattered rays, an image of a human body or an object irradiated by X-rays is obtained firstly, the image is an acquired image, then, a related brightness parameter is calculated through the brightness of the acquired image through a series of algorithms, the scattered ray image is calculated according to the brightness parameter, the real image is obtained through the difference operation between the acquired image and the scattered ray image, the real image is the image with the influence of the scattered rays eliminated, the brightness in the image is more balanced, the contrast in the image is high, the image is clear and visible, and the image can be stretched.
Please refer to fig. 5, which is a flowchart illustrating a method for calibrating scattered X-ray according to the present application;
the invention provides a fitting correction method of scattered X-ray, which comprises the following steps:
s101, collecting an original image of X-ray, and extracting a low-resolution image of an Nth layer from the original image based on a pyramid hierarchical model;
s102, determining an irradiation range corresponding to each point light source according to the brightness value of each point light source in the low-resolution image, obtaining the irradiation intensity of the corresponding point light source by using the irradiation range, determining the total brightness value of each pixel point in the irradiation range corresponding to the irradiation of each point light source according to the irradiation intensity, taking the total brightness value of each pixel point as the total brightness value of an initial ray scattering diagram, and performing brightness adjustment on the total brightness values of all the pixel points of the initial ray scattering diagram to obtain a first ray scattering diagram containing the pixel points of the irradiation range of all the point light sources;
s103, carrying out nonlinear processing on the first scattered ray diagram to obtain an optimized second scattered ray diagram;
s104, performing interpolation processing on the second scattered ray diagram to obtain a third scattered ray diagram with the same resolution as the original image, and performing difference operation on each pixel point of the original image and the third scattered ray diagram to obtain a corrected image.
In the embodiment of the application, a normalization processing operation is performed on a brightness value range of the low-resolution image I _ Detect _ D6, the value range of the brightness value range is [0,1.0], and it is determined that the brightness value of the strongest light source is 1.0, the irradiation radius of the strongest light source is R _ max, the brightness value of the weakest light source is 0, and the irradiation radius of the weakest light source is R _ min; further, R _ max is an adjustable parameter, preferably, R _ max =9, R _min =0, and in this embodiment, the irradiation radius of the light source is in a linear relationship with the luminance value of the light source, and the irradiation radius of the light source increases with the increase of the luminance value of the light source.
Further, the step of determining the illumination range corresponding to each point light source according to the brightness value of each point light source in the low-resolution image includes:
determining the maximum irradiation radius and the minimum irradiation radius of each point light source according to the brightness value of each point light source in the low-resolution image;
calculating the total irradiation radius of each point light source by using the brightness value, the maximum irradiation radius and the minimum irradiation radius of each point light source to obtain an irradiation range;
wherein, the calculation formula of the total irradiation radius is as follows:
R＝R_min+(R_max-R_min)/I_Detect_D6(i,j)
when R _ min =0, R = R _ max/I _ Detect _ D6 (I, j)
Wherein, R represents the total irradiation radius, R _ min represents the minimum irradiation radius, R _ max represents the maximum irradiation radius, (I, j) represents the coordinate position of the point light source, and I _ Detect _ D6 (I, j) represents the brightness value of the point light source;
calculating the irradiation range by the following formula:
S＝πR2
wherein S represents the illumination range of the point light source, and R represents the total illumination radius.
Further, the step of obtaining the illumination intensity of the corresponding point light source by using the illumination range includes:
setting a scale parameter by using the total irradiation radius of the irradiation range of the point light source, wherein the scale parameter is set as follows:
where σ denotes a scale parameter, R denotes a total irradiation radius, and n denotes a preset constant, and in this embodiment, preferably, n =3;
the irradiation intensity of the point light source is obtained by using the scale parameters, and the formula is as follows:
wherein f (x) represents the irradiation intensity of the point light source, mu represents the position of the point light source, the coordinate of the point light source is (i, j), x represents the position of any pixel point in the irradiation range, and the coordinate (x, y) of any pixel point; in the present embodiment, each point light source with different brightness illuminates a range of the total illumination radius R, and the illumination intensity follows a normal distribution.
Further, the step of determining the total brightness value of each pixel point in the irradiation range corresponding to the irradiation of each point light source according to the irradiation intensity, and using the total brightness value of each pixel point as the total brightness value of the initial radiation diagram includes:
determining the brightness value of a pixel point in an irradiation range corresponding to the irradiation of the corresponding point light source according to the irradiation intensity, superposing the brightness values of the same pixel point from different point light sources, and taking the total brightness value of each pixel point as the total brightness value of the initial ray scattering diagram;
calculating the brightness value of the pixel point in the irradiation range corresponding to the point light source irradiation, wherein the formula is as follows:
I_add_(x) n ＝I_Detect_D6(i n ,j n )·f(x) n
wherein, I _ add _ (x) n (ii) brightness values of pixels in an illumination range corresponding to the point light source illumination, (i) n ,j n ) Indicating the coordinate position of the point light source, I _ Detect _ D6 (I) n ,j n ) A preset luminance value representing an illumination range corresponding to the point light source irradiation, n represents a preset constant, f (x) n Representing the illumination intensity of any point light source; in this embodiment, I _ Detect _ D6 (I, j) represents a point light source (I, j) with a certain brightness value in the low-resolution image, and the range with the total irradiation radius R centered on the pixel point (I, j) in the scatter diagram I _ scatter _ D6 is shownContributes to brightness;
calculating the total brightness value of the corresponding pixel point, wherein the formula is as follows:
wherein, I _ scatter _ D6 (x, y) delta represents the total brightness value generated by different light sources irradiating the same pixel point, I _ add (x) n The brightness value of the pixel point in the irradiation range corresponding to the point light source irradiation is represented;
and calculating the total brightness value of the initial scattered ray diagram, wherein the formula is as follows:
I_scatter_D6Δ＝I_scatter_D6(x 1 ,y 1 )Δ+I_scatter_D6(x 2 ,y 2 )Δ+...+I_scatter_D6(x n ,y n )Δ
wherein, I _ scatter _ D6 Δ The total brightness value of the initial scattered ray diagram and the brightness values of all pixel points in the illumination range corresponding to all the point light sources are represented, (x) n ,y n ) Representing any pixel within the illumination range.
Further, the step of adjusting the brightness of the total brightness values of all the pixels of the initial ray scattering diagram to obtain the first ray scattering diagram including the pixels of the irradiation range of all the point light sources includes:
carrying out smooth filtering on the initial scattering base map, carrying out threshold normalization processing, counting a histogram of the brightness values of the initial scattering base map and a histogram of the brightness values of a preset light source, obtaining a brightness coefficient according to the histograms, and carrying out brightness adjustment on the total brightness values of all pixel points of the initial scattering base map according to the brightness coefficient to obtain a first scattering base map containing the pixel points in the irradiation range of all point light sources;
calculating the brightness coefficient, wherein the formula is as follows:
Factor＝Index_middle_Detect/(n*Index_middle_Scatter)；
wherein, index _ middle _ Scatter represents the value of the median of the histogram of the brightness values of the initial scattered ray diagram, index _ middle _ Detect represents the value of the median of the histogram of the brightness values of the preset light source, and n is a constant;
and adjusting the brightness of the total brightness value of all the initial radiation diagrams according to the brightness coefficient, wherein the formula is as follows:
I_scatter_D6＝I_scatter_D6Δ·Factor
wherein, I _ scatter _ D6 Δ Showing an initial scattered ray diagram, and I _ scatter _ D6 showing a first scattered ray diagram;
in this embodiment, since the brightness of the first scattered ray pattern is much lower than that of the original image, the initial scattered ray pattern I _ scatter _ D6 is set to be the same as the original image Δ Is adjusted to 1/n of the luminance value of the low resolution image I _ Detect _ D6, n representing a constant, preferably, typically n =4 such that the luminance value of the first scatter diagram I _ scatter _ D6 is optimal in this embodiment.
In the present embodiment, the first scattergram is a scattergram.
Further, the step of performing nonlinear processing on the first scatter diagram to obtain an optimized second scatter diagram includes:
carrying out nonlinear processing on the first scattered ray map to obtain a second scattered ray map; the formula is as follows:
I_scatter_2＝a+b·logc(I_scatter_D6)
i _ scatter _2 represents a second scatter diagram, I _ scatter _ D6 represents a first scatter diagram, and a, b and c represent preset parameters; further, three parameters a, b and c form a three-dimensional parameter space, and an optimized parameter value is searched in the space;
obtaining a candidate real image by using the low-resolution image and the second scattered ray image, and sorting according to the contrast of the candidate real image to obtain an optimized second scattered ray image; in this embodiment, the adopted optimization method is a traversal optimization method, where a is in a range of [ -1.0,1.0], b is in a range of [0.1,0], c is in a range of [0.0001,0.01], further 100 calculation value parameters are evenly partitioned into the range of a, 50 calculation value parameters are evenly partitioned into the range of b, 100 calculation value parameters are evenly partitioned into the range of c, a second scattergram is calculated according to the result of multiple preset a, b, c parameters to obtain a candidate real image, and the contrast is calculated, so that after 500000 times of calculation of the second scattergram, the candidate real image is selected according to the contrast of the candidate real image, and the candidate real image corresponding to the maximum contrast is selected to determine the parameters a, b, and c; in this embodiment, the search speed and accuracy can be improved by combining with classical optimization methods such as ant colony algorithm and bird colony algorithm.
Further, the step of obtaining a candidate real image by using the low-resolution image and the second scattered ray image, and performing sorting according to the contrast of the candidate real image to obtain the optimized second scattered ray image includes:
obtaining a candidate real image by using the low-resolution image and the second scattered ray map, wherein the formula is as follows:
I_D6＝I_Detect_D6-I_scatter_2
wherein, I _ D6 represents a candidate real image, I _ Detect _ D6 represents a low-resolution image, and I _ scatter _2 represents a second ray diagram;
selecting a preset area of the candidate real image to perform automatic histogram extension, calculating the variance of the candidate real image to obtain the contrast of the candidate real image,
the formula of the variance of the candidate real image is as follows:
where D represents the variance of the candidate real image, I _ D6 (x) 1 ,y 1 ) And I _ D6 (x) n ,y n ) The brightness value of the pixel point in the candidate real image is represented, I _ D6 average Representing the average brightness value of all pixel points of the candidate real image; further, the larger the variance of the candidate real image is, the larger the contrast of the candidate real image is;
and selecting the contrast of the candidate real images, and selecting the candidate real image corresponding to the maximum contrast to obtain the optimized second scatter diagram.
In this embodiment, the histogram of the candidate real image passing through the ROI (region of interest) region is automatically stretched, and the larger the contrast of the result is, the better the image de-scattering effect is, which proves that the calculated second scatter diagram is closer to the real scatter diagram;
further, the histogram automatic stretching method comprises the following steps: firstly, selecting an effective range, obtaining the maximum value range and the minimum value range of a histogram, and taking 0.01 as a cut-off threshold value of each end, namely the value range of the effective range is [0.01,0.99]; second, the valid range of pixel values is mapped to the full value domain range.
In the embodiment of the present application, the second scattergram is a scattergram, which represents an image from which the influence of the scattered ray is eliminated in the embodiment of the present application.
Further, the step of performing interpolation calculation on the second scattered ray map to obtain a third scattered ray map having the same resolution as the original image includes:
and performing interpolation calculation on the optimized second scattered ray map by using a bilinear interpolation method to obtain a third scattered ray map with the same resolution as the original image, wherein the formula is as follows:
f(d+u,d+u)＝(1-d) 2 and x f (d, d) + (1-d) x u x f (d, d + 1) + u x (1-u) x f (d +1, d) + u x f (d +1 ), wherein f (d + u ) represents an original image, and (d + u ) represents resolution, the resolution of the optimized second scattered ray diagram is (d, d), and the optimized second scattered ray diagram f (d, d) = I _ scatter _1.
Further, the step of performing difference operation on the original image and each pixel point of the third scattergram to obtain a corrected image includes:
calculating a corrected image, the formula is as follows:
I＝I_Detect-I_scatter
wherein, I represents the corrected image, I _ Detect represents the original image, and I _ scatter represents the third scattered ray diagram.
Please refer to fig. 6, which is a diagram illustrating an architecture of an apparatus for calibrating scattered X-ray according to the present application;
another aspect of the present application provides an apparatus for fitting and correcting scattered X-ray, the apparatus 200 comprising:
the data processing module 201 is configured to acquire an original image of X-rays, and extract a low-resolution image of an nth layer from the original image based on a pyramid hierarchical model;
the brightness processing module 202 is configured to determine an irradiation range corresponding to each point light source according to a brightness value of each point light source in the low-resolution image, obtain the irradiation intensity of the corresponding point light source by using the irradiation range, determine a total brightness value of each pixel point within the irradiation range corresponding to the irradiation of each point light source according to the irradiation intensity, and take the total brightness value of each pixel point as a total brightness value of the initial scattergram;
the image acquisition module 203 is configured to perform brightness adjustment on the total brightness values of all the pixel points of the initial radiation diagram to obtain a first radiation diagram including the pixel points in the irradiation range of all the point light sources;
the image comparison module 204 is used for performing nonlinear processing on the first scattergram, obtaining optimal nonlinear transformation parameters a, b and c by a common optimization method, and obtaining an optimized second scattergram;
the image calculation module 205 is configured to perform interpolation processing on the second scattergram to obtain a third scattergram having the same resolution as the original image;
and the image recovery module 206 is configured to perform difference operation on the original image and each pixel point of the third scattergram to obtain a corrected image.
The invention provides a fitting correction method and a fitting correction device for scattered X-ray, wherein an original image irradiated by X-ray is used as a target image based on a light model, one layer of image with lower resolution is obtained, each calculation parameter is obtained through the brightness value of the target image, a first scattered ray image, a second scattered ray image and a third scattered ray image are obtained in sequence, and a corrected image is obtained through the difference operation of the original image and the third scattered ray image, so that the scattered ray influence in the original image is eliminated.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical division, and in actual implementation, there may be other divisions, for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one position, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It should be noted that for simplicity and convenience of description, the above-described method embodiments are shown as a series of combinations of acts, but it should be understood by those skilled in the art that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no acts or modules are necessarily required of the invention.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In view of the above description of the fitting and correction method and apparatus for scattered X-ray provided by the present invention, those skilled in the art will appreciate that there are variations in the embodiments and applications of the method and apparatus according to the teachings of the present invention.
Claims (9)
1. An X-ray scattered ray fitting correction method, characterized by comprising the following steps:
acquiring an original image of X-ray, and extracting a low-resolution image of an Nth layer from the original image based on a pyramid hierarchical model;
determining an irradiation range corresponding to each point light source according to the brightness value of each point light source in the low-resolution image, obtaining the irradiation intensity of the corresponding point light source by using the irradiation range, determining the brightness value of each pixel point in the irradiation range corresponding to the irradiation of the corresponding point light source according to the irradiation intensity, superposing the brightness values of different point light sources to the same pixel point to obtain the total brightness value of each pixel point in the irradiation range corresponding to the irradiation of each point light source, taking the total brightness value of each pixel point as the total brightness value of an initial ray scattering diagram, and performing brightness adjustment on the total brightness values of all the pixel points of the initial ray scattering diagram to obtain a first ray diagram containing the pixel points in the irradiation range of all the point light sources;
carrying out nonlinear processing on the first scattered ray diagram to obtain an optimized second scattered ray diagram;
and performing interpolation processing on the second scattered ray diagram to obtain a third scattered ray diagram with the same resolution as the original image, and performing difference operation on each pixel point of the original image and the third scattered ray diagram to obtain a corrected image.
2. The method of claim 1, wherein said step of determining the illumination range corresponding to each point light source in the low resolution image according to the brightness value of each point light source comprises:
determining the maximum irradiation radius and the minimum irradiation radius of each point light source according to the brightness value of each point light source in the low-resolution image;
calculating a total irradiation radius of each point light source by using the brightness value of each point light source, the maximum irradiation radius and the minimum irradiation radius to obtain the irradiation range;
wherein, the calculation formula of the total irradiation radius is as follows:
R＝R_min+(R_max-R_min)/I_Detect_D6(i,j)
when R _ min =0, R = R _ max/I _ Detect _ D6 (I, j)
Wherein R represents the total irradiation radius, R _ min represents the minimum irradiation radius, R _ max represents the maximum irradiation radius, (I, j) represents a coordinate position of the point light source, and I _ Detect _ D6 (I, j) represents a luminance value of the point light source.
3. The method of claim 1 wherein said step of using said illumination range to obtain the illumination intensity of said corresponding point light source comprises:
obtaining a scale parameter by using the total irradiation radius of the irradiation range of the point light source, wherein the scale parameter is as follows:
wherein σ represents the scale parameter, R represents the total irradiation radius, and n represents a preset constant;
obtaining the illumination intensity of the point light source by using the scale parameters, wherein the formula is as follows:
wherein f (x) represents the irradiation intensity of the point light source, μ represents the position of the point light source, the coordinate of the point light source is (i, j), x represents the abscissa position of any pixel point within the irradiation range, and the coordinate of any pixel point is (x, y).
4. The method according to claim 1, wherein said step of adjusting the brightness of the total brightness values of all the pixels in the initial scattergram to obtain the first scattergram including the pixels in the illumination range of all the point light sources comprises:
and carrying out value threshold normalization processing after smoothing and filtering the initial scattered ray diagram, counting a histogram of the brightness value of the initial scattered ray diagram and a histogram of the brightness value of a preset light source, obtaining a brightness coefficient according to the histograms, and carrying out brightness adjustment on the total brightness values of all the pixels of the initial scattered ray diagram according to the brightness coefficient to obtain a first scattered ray diagram of the pixels in the irradiation range including all the point light sources.
5. A method of fitting correction of scattered X-rays according to claim 1, wherein said step of non-linearly processing said first scatter plot to obtain an optimized second scatter plot comprises:
carrying out nonlinear processing on the first scattered ray diagram to obtain a second scattered ray diagram; the formula is as follows:
I_scatter_2＝a+b·log c (I_scatter_D6)
i _ scatter _2 represents the second scatter diagram, I _ scatter _ D6 represents the first scatter diagram, and a, b and c represent preset parameters;
and obtaining a candidate real image by using the low-resolution image and the second scattered ray image, and performing sorting according to the contrast of the candidate real image to obtain an optimized second scattered ray image.
6. The method according to claim 5, wherein the step of obtaining a candidate real image using the low resolution image and the second scattergram, and selecting the candidate real image according to the contrast of the candidate real image to obtain the optimized second scattergram comprises:
obtaining a candidate real image by using the low-resolution image and the second scattered ray map, wherein the formula is as follows:
I_D6＝I_Detect_D6-I_scatter_2
wherein I _ D6 represents the candidate real image, I _ Detect _ D6 represents the low resolution image, and I _ scatter _2 represents the second scatter diagram;
selecting a preset area of the candidate real image to perform automatic histogram extension, calculating the variance of the candidate real image to obtain the contrast of the candidate real image,
wherein, the variance of the candidate real image is represented by the following formula:
wherein, D tableShows the variance of the candidate real image, I _ D6 (x) 1 ,y 1 ) And I _ D6 (x) n ,y n ) A luminance value, I _ D6, representing a pixel point in said candidate real image average Representing the average brightness value of all pixel points of the candidate real image;
and selecting the candidate real image corresponding to the maximum contrast ratio by using the contrast ratio of the candidate real image to obtain the optimized second radiation diagram.
7. A method according to claim 1, wherein said step of interpolating said second scattergram to obtain a third scattergram having the same resolution as said original image comprises:
and performing interpolation calculation on the optimized second scattered ray map by using a bilinear interpolation method to obtain a third scattered ray map with the same resolution as the original image, wherein the formula is as follows:
f(d+u,d+u)＝(1-d) 2 ×f(d,d)+(1-d)×u×f(d,d+1)+u×(1-u)×f(d+1,d)+u×u×f(d+1,d+1)
wherein f (d + u ) represents the third scattergram, (d + u ) represents resolution, the resolution of the optimized second scattergram is (d, d), and the optimized second scattergram f (d, d) = I _ scatter _1.
8. The method according to claim 1, wherein the step of performing a difference operation on each pixel point of the original image and the third scattergram to obtain a corrected image comprises:
calculating the corrected image according to the following formula:
I＝I_Detect-I_scatter
wherein I represents the corrected image, I _ Detect represents the original image, and I _ scatter represents the third scattergram.
9. An apparatus for fitting correction of scattered X-ray radiation, comprising:
the data processing module is used for acquiring an original image of X-ray and extracting a low-resolution image of an Nth layer from the original image based on a pyramid hierarchical model;
the brightness processing module is used for determining an irradiation range corresponding to each point light source according to the brightness value of each point light source in the low-resolution image, obtaining the irradiation intensity of the corresponding point light source by using the irradiation range, determining the brightness value of each pixel point in the irradiation range corresponding to the irradiation of the corresponding point light source according to the irradiation intensity, superposing the brightness values of different point light sources to the same pixel point to obtain the total brightness value of each pixel point in the irradiation range corresponding to the irradiation of each point light source, and taking the total brightness value of each pixel point as the total brightness value of the initial ray scattering diagram;
the image acquisition module is used for adjusting the brightness of the total brightness values of all the pixel points of the initial radiation diagram to obtain a first radiation diagram containing the pixel points in the irradiation range of all the point light sources;
the image comparison module is used for carrying out nonlinear processing on the first scattered ray image to obtain an optimized second scattered ray image;
the image calculation module is used for carrying out interpolation processing on the second scattered ray image to obtain a third scattered ray image with the same resolution as the original image;
and the image recovery module is used for performing difference operation on the original image and each pixel point of the third scattered ray diagram to obtain a corrected image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910461574.1A CN110246096B (en) | 2019-05-30 | 2019-05-30 | Fitting correction method and device for scattered X-ray |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910461574.1A CN110246096B (en) | 2019-05-30 | 2019-05-30 | Fitting correction method and device for scattered X-ray |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110246096A CN110246096A (en) | 2019-09-17 |
| CN110246096B true CN110246096B (en) | 2023-03-10 |
Family
ID=67885301
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910461574.1A Active CN110246096B (en) | 2019-05-30 | 2019-05-30 | Fitting correction method and device for scattered X-ray |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110246096B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2015065991A (en) * | 2013-09-26 | 2015-04-13 | 朝日レントゲン工業株式会社 | Scattered radiation correction device, scattered radiation correction method, and x-ray imaging device |
| CN105225205A (en) * | 2014-06-27 | 2016-01-06 | 展讯通信（上海）有限公司 | Image enchancing method, Apparatus and system |
| CN106033598A (en) * | 2015-03-18 | 2016-10-19 | 北京纳米维景科技有限公司 | Digital filter wire grid imaging method used for eliminating scattered radiation influence |
| JP2017129561A (en) * | 2016-01-20 | 2017-07-27 | キヤノン株式会社 | Measuring system, information processing device, information processing method, and program |
| CN108335269A (en) * | 2018-01-16 | 2018-07-27 | 沈阳东软医疗***有限公司 | Scattering correction method, apparatus, control device and the storage medium of X-ray image |
| CN109493299A (en) * | 2018-11-14 | 2019-03-19 | 杭州雄迈集成电路技术有限公司 | A method of eliminating point light source illumination effect |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001299733A (en) * | 2000-04-27 | 2001-10-30 | Konica Corp | Pci radiation beam image processing apparatus, pci radiation beam image detecting and processing apparatus, pci radiation beam image outputting apparatus and pci radiation beam image diagnosis supporting apparatus |
| US7251056B2 (en) * | 2001-06-11 | 2007-07-31 | Ricoh Company, Ltd. | Image processing apparatus, image processing method and information recording medium |
| US6879715B2 (en) * | 2001-12-05 | 2005-04-12 | General Electric Company | Iterative X-ray scatter correction method and apparatus |
| CN100510725C (en) * | 2006-11-14 | 2009-07-08 | 北京国药恒瑞美联信息技术有限公司 | Virtual grid imaging method and system used for eliminating influence of scattered radiation |
| EP2188661B1 (en) * | 2008-04-01 | 2014-07-30 | Bauhaus-Universität Weimar | Method and illumination device for optical contrast enhancement |
| JP6145889B2 (en) * | 2014-03-24 | 2017-06-14 | 富士フイルム株式会社 | Radiation image processing apparatus and method, and program |
| JP6362914B2 (en) * | 2014-04-30 | 2018-07-25 | キヤノンメディカルシステムズ株式会社 | X-ray diagnostic apparatus and image processing apparatus |
| JP6156849B2 (en) * | 2014-09-30 | 2017-07-05 | 富士フイルム株式会社 | Radiation image processing apparatus, method and program |
| JP2017143943A (en) * | 2016-02-16 | 2017-08-24 | 富士フイルム株式会社 | Radiation image processing device, method, and program |
-
2019
- 2019-05-30 CN CN201910461574.1A patent/CN110246096B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2015065991A (en) * | 2013-09-26 | 2015-04-13 | 朝日レントゲン工業株式会社 | Scattered radiation correction device, scattered radiation correction method, and x-ray imaging device |
| CN105225205A (en) * | 2014-06-27 | 2016-01-06 | 展讯通信（上海）有限公司 | Image enchancing method, Apparatus and system |
| CN106033598A (en) * | 2015-03-18 | 2016-10-19 | 北京纳米维景科技有限公司 | Digital filter wire grid imaging method used for eliminating scattered radiation influence |
| JP2017129561A (en) * | 2016-01-20 | 2017-07-27 | キヤノン株式会社 | Measuring system, information processing device, information processing method, and program |
| CN108335269A (en) * | 2018-01-16 | 2018-07-27 | 沈阳东软医疗***有限公司 | Scattering correction method, apparatus, control device and the storage medium of X-ray image |
| CN109493299A (en) * | 2018-11-14 | 2019-03-19 | 杭州雄迈集成电路技术有限公司 | A method of eliminating point light source illumination effect |
Non-Patent Citations (5)
| Title |
|---|
| Derivation of elemental abundance maps at intermediate resolution from optical interpolation of lunar prospector gamma-ray spectrometer data;Yuriy G. Shkuratov et al;《Planetary and Space Science》;20051231;1287–1301 * |
| 光束阻挡技术校正X线图像散射的方法;李艳玲等;《光盘技术》;20070608(第03期);25-27 * |
| 微焦点X射线成像***扫描参数测量及投影图像校正研究;倪建平等;《核电子学与探测技术》;20010520;第21卷(第03期);165-168 * |
| 探伤图像的几何畸变校正方法;唐国维等;《大庆石油学院学报》;20081015;第32卷(第05期);71-74 * |
| 数字X线摄影中散射线计算机补偿方法的研究进展;张桂芳等;《中国医学物理学杂志》;20061030;第23卷(第05期);320-322 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110246096A (en) | 2019-09-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9349198B2 (en) | Robust artifact reduction in image reconstruction | |
| Tang et al. | Unpaired low-dose CT denoising network based on cycle-consistent generative adversarial network with prior image information | |
| Bal et al. | Metal artifact reduction in CT using tissue‐class modeling and adaptive prefiltering | |
| Hong et al. | End-to-end unpaired image denoising with conditional adversarial networks | |
| Karimi et al. | Segmentation of artifacts and anatomy in CT metal artifact reduction | |
| CN104978578B (en) | Mobile phone photograph text image method for evaluating quality | |
| Hasan et al. | Hybrid-collaborative noise2noise denoiser for low-dose CT images | |
| Banerjee et al. | In-camera automation of photographic composition rules | |
| US20230007835A1 (en) | Composition-guided post processing for x-ray images | |
| Zhu et al. | Metal artifact reduction for X-ray computed tomography using U-net in image domain | |
| Pezeshk et al. | Seamless insertion of pulmonary nodules in chest CT images | |
| Liu et al. | Enhancement of low illumination images based on an optimal hyperbolic tangent profile | |
| US6973157B2 (en) | Method and apparatus for weighted backprojection reconstruction in 3D X-ray imaging | |
| Guo et al. | Low-light image enhancement with joint illumination and noise data distribution transformation | |
| Azam et al. | Smoke removal and image enhancement of laparoscopic images by an artificial multi-exposure image fusion method | |
| Liao et al. | Noise estimation for single-slice sinogram of low-dose X-ray computed tomography using homogenous patch | |
| CN110246096B (en) | Fitting correction method and device for scattered X-ray | |
| JP7187131B2 (en) | Image generation device, X-ray computed tomography device and image generation method | |
| CN109934777B (en) | Image local invariant feature extraction method, device, computer equipment and storage medium | |
| Gottschalk et al. | Deep learning based metal inpainting in the projection domain: Initial results | |
| KR20200021156A (en) | High Quality Four Dimensional Cone Beam Computerized Tomography System Using Prior Image | |
| Liu et al. | X-ray layered refocusing imaging based on linear scanning | |
| Liu | Restoration method of motion blurred image based on feature fusion and particle swarm optimization algorithm | |
| Kipele et al. | Poisson noise reduction with nonlocal-pca hybrid model in medical x-ray images | |
| Rohleder et al. | Cross-domain metal segmentation for CBCT metal artifact reduction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |