CN110738224A - image processing method and device - Google Patents
image processing method and device Download PDFInfo
- Publication number
- CN110738224A CN110738224A CN201810796175.6A CN201810796175A CN110738224A CN 110738224 A CN110738224 A CN 110738224A CN 201810796175 A CN201810796175 A CN 201810796175A CN 110738224 A CN110738224 A CN 110738224A
- Authority
- CN
- China
- Prior art keywords
- image
- region
- area
- light spot
- identified
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 21
- 238000003062 neural network model Methods 0.000 claims abstract description 73
- 238000000034 method Methods 0.000 claims abstract description 29
- 238000012549 training Methods 0.000 claims abstract description 25
- 238000012545 processing Methods 0.000 claims description 26
- 238000005286 illumination Methods 0.000 claims description 12
- 238000001514 detection method Methods 0.000 claims description 10
- 230000015654 memory Effects 0.000 description 9
- 238000004590 computer program Methods 0.000 description 6
- 238000012544 monitoring process Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Geometry (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention provides image processing methods and devices, wherein the method comprises the steps of obtaining an image to be recognized containing a light spot area, the image to be recognized is an image collected in a shadowless lamp irradiation area, and recognizing the light spot area of the image to be recognized according to image information of the image to be recognized and shapes of all areas included in the image to be recognized through a pre-trained neural network model, wherein the neural network model is obtained through pre-training of a sample image containing the light spot area, the shape of the light spot area is a preset shape, and the image information of the light spot area meets preset conditions.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to image processing methods and apparatuses.
Background
In scenes in the medical field, such as an operating room, an examination room, an emergency room and the like, when a doctor performs operations on a patient, shadows can be generated on the head, hands, medical instruments and the like of the doctor and the shadows can influence the operation precision, so that the shadowless lamp is applied to .
Generally, operations in the medical field can be monitored by the image acquisition device for recording the operation process, however, due to the comparison of the monitoring range of the image acquisition device, the monitoring image acquired by the image acquisition device usually only has the spot area where the shadowless lamp is located as the area containing valid information.
Therefore, it is problems to be solved urgently how to detect the spot area in the image and further accurately acquire effective information in the image.
Disclosure of Invention
The embodiment of the invention aims to provide image processing methods and devices to realize detection of a light spot area in an image, and the specific technical scheme is as follows:
, an embodiment of the invention provides methods of image processing, the method comprising:
acquiring an image to be identified containing a light spot area; the image to be identified is an image collected in an illumination area of the shadowless lamp;
recognizing a light spot area of the image to be recognized according to the image information of the image to be recognized and the shape of each area included in the image to be recognized through a pre-trained neural network model;
the neural network model is obtained by training a sample image containing a light spot area in advance, the shape of the light spot area is a preset shape, and the image information of the light spot area meets a preset condition.
Optionally, after the light spot region of the image to be recognized is recognized through the pre-trained neural network model, the method further includes:
adjusting acquisition parameters of image acquisition equipment for acquiring the image to be identified based on the image information of the light spot area of the image to be identified; and/or
And processing the image to be recognized based on the image information of the light spot area of the image to be recognized.
Optionally, the step of adjusting acquisition parameters of an image acquisition device acquiring the image to be recognized based on the image information of the light spot region of the image to be recognized includes:
determining the brightness information of the light spot area of the image to be identified;
and adjusting exposure parameters of the image acquisition equipment according to the brightness information of the light spot area.
Optionally, the step of adjusting acquisition parameters of an image acquisition device acquiring the image to be recognized based on the image information of the light spot region of the image to be recognized includes:
determining the area of a light spot region of the image to be identified and the area of the image to be identified;
and adjusting the focal length of the image acquisition equipment according to the relation between the boundary of the light spot region and the boundary of the image to be identified, the area of the light spot region of the image to be identified and the area of the image to be identified.
Optionally, the step of processing the image to be recognized based on the image information of the light spot region of the image to be recognized includes:
determining a current region to be coded of the image to be identified, wherein the size of the current region to be coded is a preset value;
judging whether the current region to be coded is located in the speckle region;
when the current region to be coded is located in the speckle region, coding the current region to be coded by adopting a preset -th coding parameter;
and when the current region to be coded is not located in the speckle region, coding the current region to be coded by adopting a preset second coding parameter, wherein the second coding parameter is smaller than the th coding parameter.
Optionally, the step of determining whether the current region to be encoded is located in the speckle region includes:
identifying th regions overlapping the light spot region and remaining second regions in the current region to be encoded;
and when the area of the th region is larger than or equal to the area of the second region, determining that the region to be coded is positioned in the spot region, and when the area of the th region is smaller than the area of the second region, determining that the region to be coded is not positioned in the spot region.
Optionally, the training process of the neural network model includes:
acquiring a sample image containing a light spot area, wherein the sample image is an image acquired in an illumination area of a shadowless lamp, the shape of the light spot area is a preset shape, and the image information of the light spot area meets the preset condition;
and inputting the sample images into a preset initial neural network model, and obtaining the neural network model when the identified regions of the initial neural network model are matched with the speckle regions included in the corresponding sample images according to the image information of the sample images and the shapes of the regions included in the sample images.
Optionally, the preset conditions include at least that the brightness value of the light spot region and the brightness value of the remaining region of the sample image except for the light spot region satisfy a predetermined size relationship, the pixel value of the light spot region is within a preset pixel value range, and the brightness value of the light spot region is within a preset brightness value range.
In a second aspect, an embodiment of the present invention provides kinds of image processing apparatuses, including:
the image acquisition module is used for acquiring an image to be identified containing a light spot area; the image to be identified is an image collected in an illumination area of the shadowless lamp;
the area detection module is used for identifying the light spot area of the image to be identified according to the image information of the image to be identified and the shape of each area included in the image to be identified through a pre-trained neural network model;
the neural network model is obtained by training a sample image containing a light spot area in advance, the shape of the light spot area is a preset shape, and the image information of the light spot area meets a preset condition.
Optionally, the apparatus further comprises:
the parameter adjusting module is used for adjusting acquisition parameters of image acquisition equipment for acquiring the image to be identified based on the image information of the light spot area of the image to be identified; and/or
And the image processing module is used for processing the image to be identified based on the image information of the light spot area of the image to be identified.
Optionally, the parameter adjusting module includes:
the brightness information determining submodule is used for determining the brightness information of the light spot area of the image to be identified;
and the exposure parameter adjusting submodule is used for adjusting the exposure parameters of the image acquisition equipment according to the brightness information of the light spot area.
Optionally, the parameter adjusting module includes:
the area determination submodule is used for determining the area of a light spot area of the image to be identified and the area of the image to be identified;
and the focal length adjusting submodule is used for adjusting the focal length of the image acquisition equipment according to the relationship between the boundary of the light spot region and the boundary of the image to be identified, the area of the light spot region of the image to be identified and the area of the image to be identified.
Optionally, the image processing module includes:
the region determining submodule is used for determining a current region to be coded of the image to be identified, and the size of the current region to be coded is a preset value;
the area judgment submodule is used for judging whether the current area to be coded is positioned in the speckle area;
and the coding submodule is used for coding the current region to be coded by adopting a preset th coding parameter when the current region to be coded is positioned in the facula region, and coding the current region to be coded by adopting a preset second coding parameter when the current region to be coded is not positioned in the facula region, wherein the second coding parameter is smaller than the th coding parameter.
Optionally, the area determination sub-module is specifically configured to:
identifying th regions overlapping the light spot region and remaining second regions in the current region to be encoded;
and when the area of the th region is larger than or equal to the area of the second region, determining that the region to be coded is positioned in the spot region, and when the area of the th region is smaller than the area of the second region, determining that the region to be coded is not positioned in the spot region.
Optionally, the apparatus further comprises:
the device comprises a sample image acquisition module, a light spot area acquisition module and a light source module, wherein the sample image acquisition module is used for acquiring a sample image containing the light spot area, the sample image is an image acquired in a shadowless lamp irradiation area, the light spot area is in a preset shape, and the image information of the light spot area meets the preset condition;
and the neural network model training module is used for inputting the sample images into a preset initial neural network model, and obtaining the neural network model when the identified area of the initial neural network model is matched with the light spot area included in each corresponding sample image according to the image information of each sample image and the shape of each area included in each sample image.
Optionally, the preset conditions include at least that the brightness value of the light spot region and the brightness value of the remaining region of the sample image except for the light spot region satisfy a predetermined size relationship, the pixel value of the light spot region is within a preset pixel value range, and the brightness value of the light spot region is within a preset brightness value range.
In a third aspect, an embodiment of the invention provides electronic devices, including a processor and a memory;
the memory stores executable program code;
the processor executes a program corresponding to the executable program code by reading the executable program code stored in the memory, for executing the image processing methods as described in the above.
In a fourth aspect, an embodiment of the present invention provides computer-readable storage media, where a computer program is stored, and when the computer program is executed by a processor, the computer program implements the image processing methods described in the aforementioned aspect.
The embodiment of the invention provides image processing methods and devices, wherein the method comprises the steps of obtaining an image to be recognized containing a light spot area, the image to be recognized is an image collected in a shadowless lamp irradiation area, and recognizing the light spot area of the image to be recognized according to image information of the image to be recognized and shapes of all areas included in the image to be recognized through a pre-trained neural network model, wherein the neural network model is obtained through pre-training of a sample image containing the light spot area, the shape of the light spot area is a preset shape, and the image information of the light spot area meets preset conditions.
In the embodiment of the invention, the neural network model can be obtained by training the sample image containing the light spot region in advance, and then the light spot region in the image to be recognized can be detected through the neural network model when the image to be recognized is obtained, so that the detection of the light spot region in the image is realized.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flow chart of image processing methods according to an embodiment of the invention;
FIG. 2 is a schematic diagram of spot areas according to an embodiment of the present invention;
FIG. 3 is another flowchart of the image processing methods according to the embodiment of the present invention;
FIG. 4 is another flowchart of the image processing methods of embodiments of the invention;
FIG. 5 is another flowchart of the image processing methods of embodiments of the invention;
FIG. 6 is another flowchart of the image processing methods of embodiments of the invention;
FIG. 7 is another flowchart of the image processing methods of embodiments of the invention;
FIG. 8 is a schematic structural diagram of image processing apparatuses according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of electronic devices according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only partial embodiments of of the present invention, rather than all embodiments.
The present invention will be described in detail below with reference to specific examples.
Referring to fig. 1, a flow chart of image processing methods according to an embodiment of the present invention is shown, where the method may include the following steps:
s101, acquiring an image to be identified containing a light spot area; the image to be identified is an image collected in an illumination area of the shadowless lamp;
in the embodiment of the present invention, an image capturing device may be installed in a scene to be detected, so as to capture an image through the image capturing device, and obtain effective information through analyzing the image, where the scene to be detected may be, for example, scenes in the medical field, such as an operating room, an examination room, an emergency room, and the like, the image capturing device may be a ball machine, a video camera, a snapshot machine, and the like, and or more image capturing devices may be installed in the scene to be detected, which is not limited in the embodiment of the present invention.
In a medical field, a shadowless lamp is usually installed above an operation table, and in a monitoring image acquired by an image acquisition device, only a spot area where the shadowless lamp is located is usually an area containing valid information. Therefore, in the embodiment of the invention, when the image acquisition device is installed, the monitoring area of the image acquisition device can be set to at least contain the irradiation area of the shadowless lamp.
The image acquisition equipment installed in the scene to be detected can acquire images of the monitoring area. For example, the image capturing device may capture images continuously according to a preset period (e.g., 1 minute, 5 minutes, 10 minutes, etc.); alternatively, in order to reduce the storage capacity of the image capturing device, the image capturing device may capture the image only within a preset time period, for example, a start capturing condition and an end capturing condition may be set in the image capturing device in advance, and when the image capturing device detects that the start capturing condition is satisfied, the image capturing device starts capturing the image until the end capturing condition is detected to be satisfied.
Wherein, the start collecting condition may include: a person enters a scene to be detected, certain equipment is started and the like; accordingly, the end acquisition condition may include: a person leaves a scene to be detected, a certain device is turned off, and the like, which is not limited in the embodiment of the present invention.
The image processing method provided by the embodiment of the invention can be applied to any equipment with an image processing function. For example, the method can be used for image acquisition equipment installed in a scene to be detected; or, the image capturing device may be an electronic device other than an image capturing device, such as a desktop computer, a portable computer, an intelligent mobile terminal, a server, and the like. For convenience of description, the area detection method provided by the embodiment of the present invention is described by taking an electronic device applied to an image capturing device as an example.
For example, a wired connection may be established between the image capturing device and the electronic device through any wired connection manners, or a wireless connection may be established between the image capturing device and the electronic device through any long-distance wireless connection manners or Near Field Communication (NFC), bluetooth, or other short-distance wireless connection manners, which is not limited in this embodiment of the present invention.
In the embodiment of the invention, the electronic equipment can acquire the image to be identified containing the light spot area. For example, the electronic device may obtain an image currently acquired and sent by the image acquisition device as an image to be identified; alternatively, an image whose acquisition time is within a preset time period may be identified as an image to be identified in an image that has been sent by the image acquisition device, which is not limited in the embodiment of the present invention.
S102, identifying a light spot area of the image to be identified according to the image information of the image to be identified and the shape of each area included in the image to be identified through a pre-trained neural network model; the neural network model is obtained by training a sample image containing a light spot area in advance, the shape of the light spot area is a preset shape, and image information of the light spot area meets a preset condition.
In the embodiment of the invention, the electronic equipment can be trained in advance to obtain the neural network model, and then the light spot area in the image to be identified can be detected through the neural network model. Specifically, the electronic device may train through a sample image including the spot region to obtain the neural network model. The shape of the light spot region is a preset shape, such as a circle, a rectangle, and the like, and the image information of the light spot region meets a preset condition.
When image processing is performed, after the electronic device acquires an image to be recognized including a light spot region, the image to be recognized can be input into a neural network model obtained through pre-training, and the neural network model can recognize the light spot region of the image to be recognized according to image information of the image to be recognized and shapes of regions included in the image to be recognized.
For example, the neural network model may divide all possible regions in the image to be identified, and then in each region, identify a region whose image information and shape both satisfy the identification condition, that is, a spot region.
After the neural network model identifies the light spot area, the position information of the light spot area in the image to be identified can be output. For example, when the light spot region is rectangular, the electronic device may establish a two-dimensional coordinate system in the image to be recognized, and use coordinates of four vertices of the light spot region, that is, x and y coordinate values corresponding to the four vertices, as the position information of the light spot region in the image to be recognized.
When the light spot area is circular, the electronic equipment can establish a polar coordinate system in the image to be recognized, and the x and y coordinate values of the position of the circle center of the circle where the light spot area is located and the radius value of the circle are used as the position information of the light spot area in the image to be recognized; or, the electronic device may determine a circumscribed rectangle corresponding to the spot region, and use the position information of the circumscribed rectangle as the position information of the spot region in the image to be identified.
Fig. 2 shows schematic diagrams of the light spot region, in fig. 2, a solid line rectangular box represents an image to be recognized, a dotted line rectangular box represents a detected light spot region, and a solid line circle represents an actual light spot region.
In the embodiment of the invention, the neural network model can be obtained by training the sample image containing the light spot region in advance, and then the light spot region in the image to be recognized can be detected through the neural network model when the image to be recognized is obtained, so that the detection of the light spot region in the image is realized.
It is understood that the spot area in the image to be recognized is an area containing effective information. In the image acquisition process, how to acquire an image with better image quality in a light spot area; and when processing an image, how to keep the complete image information of the spot area as much as possible is an important factor influencing whether accurate effective information can be obtained.
As implementation manners of the embodiment of the present invention, as shown in fig. 3, image processing methods provided by the embodiment of the present invention may include the following steps:
s301, acquiring an image to be identified containing a light spot area; the image to be identified is an image collected in an illumination area of the shadowless lamp;
s302, identifying a light spot area of the image to be identified according to the image information of the image to be identified and the shape of each area included in the image to be identified through a pre-trained neural network model; the neural network model is obtained by training a sample image containing a light spot area in advance, the shape of the light spot area is a preset shape, and the image information of the light spot area meets a preset condition;
the steps S301 to S302 are substantially the same as the steps S101 to S102 in the embodiment shown in fig. 1, and are not repeated herein.
S303, adjusting acquisition parameters of image acquisition equipment for acquiring the image to be identified based on the image information of the light spot area of the image to be identified; and/or processing the image to be recognized based on the image information of the light spot area of the image to be recognized.
In the embodiment of the invention, after the electronic device detects the light spot area of the image to be recognized, the electronic device can adjust the acquisition parameters of the image acquisition device for acquiring the image to be recognized based on the image information of the light spot area, and/or process the image to be recognized.
For example, the electronic device may determine whether the image quality of the light spot region may be better based on the image information of the light spot region of the image to be recognized, and if so, may adjust the acquisition parameter of the image acquisition device that acquires the image to be recognized, so as to acquire the image with better image quality of the light spot region through the adjusted acquisition parameter. Or, the image to be recognized may be processed based on the image information of the light spot region of the image to be recognized, so as to retain the complete image information of the light spot region as much as possible.
It will be appreciated that the exposure parameters of an image capture device will affect the brightness of its captured image. Moreover, the brightness of the image directly affects the image quality of the image, and an image that is too bright or too dark will affect the effect of acquiring valid information.
Specifically, as shown in fig. 4, the step of adjusting, by the electronic device, the exposure parameter of the image capturing device for capturing the image to be recognized based on the image information of the light spot region of the image to be recognized may include:
s401, determining the brightness information of a light spot area of an image to be identified;
the luminance information may be, for example: average brightness, maximum brightness, minimum brightness, etc., which are not limited in the embodiments of the present invention. Specifically, the electronic device may perform statistics on the brightness value of each pixel in the spot region of the image to be recognized to obtain the brightness information of the spot region.
S402, adjusting exposure parameters of the image acquisition equipment according to the brightness information of the light spot area.
If the difference between the average brightness and the target brightness is greater than the preset threshold and the average brightness is lower than the target brightness, the exposure is considered too dark, in this case, the aperture, the block speed and the like can be increased to make the average brightness of the subsequently acquired image approach the target brightness, and if the difference between the average brightness and the target brightness is greater than the preset threshold and the average brightness is lower than the target brightness, the exposure is considered too bright, in this case, the aperture, the block speed and the like can be reduced to make the average brightness of the subsequently acquired image approach the target brightness.
It will be appreciated that for the same spot region, the focal length of the image capture device will affect the size of the spot region in the image captured by the image capture device.
Generally, only the light spot region in the image contains effective information, and the larger the area of the light spot region, the more accurate and effective information can be acquired. Therefore, in order to acquire accurate effective information, it is necessary to ensure that the image includes a complete spot region and the area of the spot region is maximized as much as possible.
Specifically, as shown in fig. 5, the step of adjusting the focal length of the image capturing device for capturing the image to be recognized by the electronic device based on the image information of the light spot region of the image to be recognized may include:
s501, determining the area of a light spot region of an image to be identified and the area of the image to be identified;
for example, when the spot area is rectangular, the electronic device may calculate the area of the spot area according to the length and the width of the spot area. Moreover, the area of the image to be recognized may be calculated according to the length and the width of the image to be recognized, or the area of the image to be recognized, which is stored in advance, may be acquired.
S502, adjusting the focal length of the image acquisition equipment according to the relation between the boundary of the light spot region and the boundary of the image to be identified, the area of the light spot region of the image to be identified and the area of the image to be identified.
For example, when at least boundaries of the light spot area coincide with the boundaries of the image to be recognized, the light spot area can be determined not to be completely contained in the image to be recognized, and when the boundaries of the light spot area do not coincide with the boundaries of the image to be recognized, the light spot area can be determined to be completely contained in the image to be recognized.
When the light spot area is not completely contained in the image to be recognized, it can be determined that the current light spot area is too large, and the target is considered to be closer, so that the image to be recognized cannot contain the complete light spot area. In this case, the focal length of the image capturing device can be reduced to reduce the size of the spot region in the subsequently captured image, and an image including the complete spot region is captured. Specifically, zooming and zooming can be performed by adjusting an electric lens built in the camera.
When the light spot region is completely included in the image to be recognized, the electronic device may further adjust the focal length of the image capturing device according to the area of the light spot region of the image to be recognized and the area of the image to be recognized, so as to avoid the area of the light spot region being too small.
Specifically, the electronic device may calculate the area of the remaining region in the image to be recognized, except the light spot region, according to the area of the light spot region and the area of the image to be recognized, and adjust the focal length of the image acquisition device according to the ratio of the area of the light spot region to the area of the remaining region; or the electronic device can directly adjust the focal length of the image acquisition device according to the ratio of the area of the light spot region to the area of the image to be identified. Wherein, the larger the above ratio is, the larger the light spot area is; a smaller ratio indicates a smaller spot area.
When the electronic device directly adjusts the focal length of the image acquisition device according to the ratio of the area of the light spot region to the area of the image to be recognized, specifically, the electronic device may calculate the ratio of the area of the region in the dotted-line rectangular frame to the area of the region in the solid-line rectangular frame in fig. 2, that is, the ratio of the area of the light spot region to the area of the image to be recognized. If the ratio is smaller than a preset ratio, such as 80%, 90%, 95%, etc., the target is considered to be far away, so that the area of the light spot region in the image to be recognized is too small. In this case, the focal length of the image pickup device may be increased to enlarge the spot area in the subsequently picked-up image. Specifically, zooming can be performed by adjusting an electric lens built in the camera.
It will be appreciated that typically only the spot area in the image to be identified contains valid information. Therefore, when the image to be recognized is processed, for example, when the image to be recognized is encoded, how to ensure that the information of the light spot region is not lost is an important factor influencing whether the original effective information can be acquired after the encoded image is decoded.
Specifically, as shown in fig. 6, the step of processing, by the electronic device, the image to be recognized based on the image information of the light spot of the image to be recognized may include:
s601, determining a current to-be-coded area of the image to be identified, wherein the size of the current to-be-coded area is a preset value;
for example, the electronic device may sequentially use each region in the image to be recognized as the region to be coded according to a preset size of a coding macro block, such as 8 × 8 pixels, 16 × 16 pixels, 32 × 32 pixels, and the like, so as to code the image to be recognized.
S602, judging whether the current region to be coded is located in the spot region; if yes, executing step S603, and if not, executing step S604;
it is understood that the encoding process applied to the image can reduce the size of the image, but at the same time, part of the original information in the image is lost. When the image is coded, coded images with different qualities are obtained by using different coding parameters.
In the embodiment of the invention, in order to ensure the size of the encoded image and ensure that the effective information of the light spot region is not lost as much as possible, different encoding parameters can be adopted to respectively encode the light spot region and the residual region. Specifically, the electronic device may determine whether the current region to be encoded is located in the speckle region, so as to encode the current region to be encoded by using the corresponding encoding parameter according to the determination result.
For example, the electronic device may identify th region overlapping the spot region and the remaining second region in the current region to be encoded, determine that the region to be encoded is located within the spot region when the th region has an area greater than or equal to the area of the second region, and determine that the region to be encoded is not located within the spot region when the th region has an area less than the area of the second region.
S603, encoding the current region to be encoded by adopting a preset th encoding parameter;
s604, the current region to be coded is coded by adopting a preset second coding parameter, wherein the second coding parameter is smaller than the th coding parameter.
When the electronic device determines that the region to be coded is located in the speckle region, a preset -th coding parameter may be used to code the current region to be coded, and when the electronic device determines that the region to be coded is not located in the speckle region, a preset second coding parameter may be used to code the current region to be coded.
That is to say, the light spot region may be encoded by using the high-quality encoding parameter, and the remaining region may be encoded by using the low-quality encoding parameter, so that it is ensured that the size of the encoded image satisfies the condition, and at the same time, it is ensured that the effective information of the light spot region is not lost as much as possible.
When the image to be identified is coded by the method, for the current region to be coded, which is partially located in the speckle region, when the part of the current region to be coded, which is located in the speckle region, is less than halves of the region to be coded, the current region to be coded is coded by using low-quality coding parameters.
Optionally, in the embodiment of the present invention, in order to further step improve the integrity of the effective information of the light spot region, before the electronic device encodes the image to be recognized, the light spot region may be appropriately enlarged, for example, the light spot region may be enlarged by a preset multiple, such as 5%, 10%, 15%, or the like, so that it is ensured that all the pixels in the light spot region can be encoded with high-quality encoding parameters, and the effective information of the light spot region is prevented from being lost.
As implementation manners of the embodiment of the present invention, as shown in fig. 7, the process of the electronic device to train the neural network model may include the following steps:
s701, obtaining a sample image containing a light spot area, wherein the sample image is an image collected in an illumination area of a shadowless lamp, the shape of the light spot area is a preset shape, and image information of the light spot area meets a preset condition;
in an embodiment of the invention, the electronic device may acquire an image of the sample containing the spot area. For example, an image acquired of a shadowless lamp illuminated area in an operating room scene may be taken as a sample image. The shape of the spot area may be a predetermined shape, such as a circle, a rectangle, or the like.
It will be appreciated that in the scene to be detected, the brightness, hue, etc. of the shadowless lamp illuminated area will generally be different from the remaining areas. Accordingly, in the sample image, the image information of the spot region is also generally different from the remaining region in the sample image except for the spot region. The image information may include any relevant information capable of distinguishing the spot region from the remaining region, such as brightness, pixel value, and the like, which is not limited in the embodiment of the present invention.
In the embodiment of the invention, the image information of the light spot region in the sample image satisfies a preset condition, the preset condition comprises at least items that the brightness value of the light spot region and the brightness value of the remaining region except the light spot region in the sample image satisfy a predetermined size relationship, for example, the brightness value of each pixel of the light spot region is greater than that of each pixel of the remaining region, the pixel value of the light spot region is within a preset pixel value range, and the brightness value of the light spot region is within the preset brightness value range.
And S702, inputting the sample images into a preset initial neural network model, and obtaining the neural network model when the initial neural network model matches the identified area with the corresponding speckle area in each sample image according to the image information of each sample image and the shape of the area in each sample image.
After the sample image is acquired, the electronic device may train a preset initial neural network model by using the sample image. For example, the sample images may be input into a preset initial neural network model, the initial neural network identifies each region according to the image information of each sample image and the shape of the region included in each sample image, and when the identified region matches the speckle region included in each corresponding sample image, a trained neural network model is obtained.
The training process of the specific neural network model can be carried out on an existing deep learning network platform or can be realized through a built program framework, and the training process is not limited herein.
The convolutional neural network generally comprises network layers such as convolutional layers, pooling layers, nonlinear layers and fully-connected layers, of course, the neural network model in the embodiment can also be a fully-convolutional neural network model, that is, a convolutional neural network without a fully-connected layer, and the embodiment of the present invention does not limit the type and/or structure of the neural network model.
In the embodiment, the neural network model can be obtained through training of the sample image, so that the light spot region in the image to be recognized can be recognized through the neural network model.
Correspondingly, an embodiment of the present invention further provides kinds of image processing apparatuses, as shown in fig. 8, an apparatus includes:
the image acquisition module 810 is configured to acquire an image to be identified, which includes a light spot region; the image to be identified is an image collected in an illumination area of the shadowless lamp;
the region detection module 820 is configured to identify, through a pre-trained neural network model, a light spot region of the image to be identified according to the image information of the image to be identified and shapes of regions included in the image to be identified;
the neural network model is obtained by training a sample image containing a light spot area in advance, the shape of the light spot area is a preset shape, and the image information of the light spot area meets a preset condition.
In the embodiment of the invention, the neural network model can be obtained by training the sample image containing the light spot region in advance, and then the light spot region in the image to be recognized can be detected through the neural network model when the image to be recognized is obtained, so that the detection of the light spot region in the image is realized.
implementations of an embodiment of the invention, the apparatus further comprising:
the parameter adjusting module is used for adjusting acquisition parameters of image acquisition equipment for acquiring the image to be identified based on the image information of the light spot area of the image to be identified; and/or
And the image processing module is used for processing the image to be identified based on the image information of the light spot area of the image to be identified.
embodiments of the present invention, the parameter adjustment module includes:
the brightness information determining submodule is used for determining the brightness information of the light spot area of the image to be identified;
and the exposure parameter adjusting submodule is used for adjusting the exposure parameters of the image acquisition equipment according to the brightness information of the light spot area.
embodiments of the present invention, the parameter adjustment module includes:
the area determination submodule is used for determining the area of a light spot area of the image to be identified and the area of the image to be identified;
and the focal length adjusting submodule is used for adjusting the focal length of the image acquisition equipment according to the relationship between the boundary of the light spot region and the boundary of the image to be identified, the area of the light spot region of the image to be identified and the area of the image to be identified.
implementation manners of the embodiment of the present invention, the image processing module includes:
the region determining submodule is used for determining a current region to be coded of the image to be identified, and the size of the current region to be coded is a preset value;
the area judgment submodule is used for judging whether the current area to be coded is positioned in the speckle area;
and the coding submodule is used for coding the current region to be coded by adopting a preset th coding parameter when the current region to be coded is positioned in the facula region, and coding the current region to be coded by adopting a preset second coding parameter when the current region to be coded is not positioned in the facula region, wherein the second coding parameter is smaller than the th coding parameter.
As implementation manners of the embodiment of the present invention, the area judgment sub-module is specifically configured to:
identifying th regions overlapping the light spot region and remaining second regions in the current region to be encoded;
and when the area of the th region is larger than or equal to the area of the second region, determining that the region to be coded is positioned in the spot region, and when the area of the th region is smaller than the area of the second region, determining that the region to be coded is not positioned in the spot region.
implementations of an embodiment of the invention, the apparatus further comprising:
the device comprises a sample image acquisition module, a light spot area acquisition module and a light source module, wherein the sample image acquisition module is used for acquiring a sample image containing the light spot area, the sample image is an image acquired in a shadowless lamp irradiation area, the light spot area is in a preset shape, and the image information of the light spot area meets the preset condition;
and the neural network model training module is used for inputting the sample images into a preset initial neural network model, and obtaining the neural network model when the identified area of the initial neural network model is matched with the light spot area included in each corresponding sample image according to the image information of each sample image and the shape of each area included in each sample image.
according to the embodiment of the present invention, the preset conditions include at least items that the brightness value of the flare region and the brightness value of the remaining region of the sample image except for the flare region satisfy a predetermined size relationship, the pixel value of the flare region is within a preset pixel value range, and the brightness value of the flare region is within a preset brightness value range.
Accordingly, the embodiment of the present invention further provides electronic devices, as shown in fig. 9, including a processor 910 and a memory 920;
the memory 920 stores executable program code;
the processor 910 executes a program corresponding to the executable program code by reading the executable program code stored in the memory 920, so as to execute image processing methods according to an embodiment of the present invention, where the image processing method includes:
acquiring an image to be identified containing a light spot area; the image to be identified is an image collected in an illumination area of the shadowless lamp;
recognizing a light spot area of the image to be recognized according to the image information of the image to be recognized and the shape of each area included in the image to be recognized through a pre-trained neural network model;
the neural network model is obtained by training a sample image containing a light spot area in advance, the shape of the light spot area is a preset shape, and the image information of the light spot area meets a preset condition.
In the embodiment of the invention, the neural network model can be obtained by training the sample image containing the light spot region in advance, and then the light spot region in the image to be recognized can be detected through the neural network model when the image to be recognized is obtained, so that the detection of the light spot region in the image is realized.
The Memory 920 may include a Random Access Memory (RAM) or a non-volatile Memory (NVM), such as at least disk memories, and optionally at least storage devices located remotely from the processor.
The Processor 910 may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Array (FPGA), other Programmable logic devices, a discrete , a transistor logic device, and discrete hardware components.
Such electronic devices include, but are not limited to, smart phones, computers, personal digital assistants, and the like.
Accordingly, the present invention further provides computer-readable storage media, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements image processing methods according to the present invention, where the image processing method includes:
acquiring an image to be identified containing a light spot area; the image to be identified is an image collected in an illumination area of the shadowless lamp;
recognizing a light spot area of the image to be recognized according to the image information of the image to be recognized and the shape of each area included in the image to be recognized through a pre-trained neural network model;
the neural network model is obtained by training a sample image containing a light spot area in advance, the shape of the light spot area is a preset shape, and the image information of the light spot area meets a preset condition.
In the embodiment of the invention, the neural network model can be obtained by training the sample image containing the light spot region in advance, and then the light spot region in the image to be recognized can be detected through the neural network model when the image to be recognized is obtained, so that the detection of the light spot region in the image is realized.
It should be noted that, in this document, relational terms such as , second and the like are only used to distinguish entities or operations from another entities or operations, and no necessarily requires or implies that any such actual relationship or order exists between the entities or operations.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the apparatus/electronic device/storage medium embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant points can be referred to the partial description of the method embodiment.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (16)
1, A method of image processing, the method comprising:
acquiring an image to be identified containing a light spot area; the image to be identified is an image collected in an illumination area of the shadowless lamp;
recognizing a light spot area of the image to be recognized according to the image information of the image to be recognized and the shape of each area included in the image to be recognized through a pre-trained neural network model;
the neural network model is obtained by training a sample image containing a light spot area in advance, the shape of the light spot area is a preset shape, and the image information of the light spot area meets a preset condition.
2. The method according to claim 1, wherein after the light spot region of the image to be recognized is recognized through a pre-trained neural network model, the method further comprises:
adjusting acquisition parameters of image acquisition equipment for acquiring the image to be identified based on the image information of the light spot area of the image to be identified; and/or
And processing the image to be recognized based on the image information of the light spot area of the image to be recognized.
3. The method according to claim 2, wherein the step of adjusting acquisition parameters of an image acquisition device acquiring the image to be recognized based on the image information of the light spot area of the image to be recognized comprises:
determining the brightness information of the light spot area of the image to be identified;
and adjusting exposure parameters of the image acquisition equipment according to the brightness information of the light spot area.
4. The method according to claim 2, wherein the step of adjusting acquisition parameters of an image acquisition device acquiring the image to be recognized based on the image information of the light spot area of the image to be recognized comprises:
determining the area of a light spot region of the image to be identified and the area of the image to be identified;
and adjusting the focal length of the image acquisition equipment according to the relation between the boundary of the light spot region and the boundary of the image to be identified, the area of the light spot region of the image to be identified and the area of the image to be identified.
5. The method according to claim 2, wherein the step of processing the image to be recognized based on the image information of the light spot region of the image to be recognized comprises:
determining a current region to be coded of the image to be identified, wherein the size of the current region to be coded is a preset value;
judging whether the current region to be coded is located in the speckle region;
when the current region to be coded is located in the speckle region, coding the current region to be coded by adopting a preset -th coding parameter;
and when the current region to be coded is not located in the speckle region, coding the current region to be coded by adopting a preset second coding parameter, wherein the second coding parameter is smaller than the th coding parameter.
6. The method of claim 5, wherein the step of determining whether the current region to be encoded is located in the speckle region comprises:
identifying th regions overlapping the light spot region and remaining second regions in the current region to be encoded;
and when the area of the th region is larger than or equal to the area of the second region, determining that the region to be coded is positioned in the spot region, and when the area of the th region is smaller than the area of the second region, determining that the region to be coded is not positioned in the spot region.
7. The method of any of , wherein the training process for the neural network model comprises:
acquiring a sample image containing a light spot area, wherein the sample image is an image acquired in an illumination area of a shadowless lamp, the shape of the light spot area is a preset shape, and the image information of the light spot area meets the preset condition;
and inputting the sample images into a preset initial neural network model, and obtaining the neural network model when the identified regions of the initial neural network model are matched with the speckle regions included in the corresponding sample images according to the image information of the sample images and the shapes of the regions included in the sample images.
8. The method according to any one of claims 1-6 and , wherein the preset conditions include at least that the brightness value of the spot region and the brightness value of the remaining region of the sample image except the spot region satisfy a predetermined size relationship, the pixel value of the spot region is within a preset pixel value range, and the brightness value of the spot region is within a preset brightness value range.
An image processing apparatus of the type 9, , comprising:
the image acquisition module is used for acquiring an image to be identified containing a light spot area; the image to be identified is an image collected in an illumination area of the shadowless lamp;
the area detection module is used for identifying the light spot area of the image to be identified according to the image information of the image to be identified and the shape of each area included in the image to be identified through a pre-trained neural network model;
the neural network model is obtained by training a sample image containing a light spot area in advance, the shape of the light spot area is a preset shape, and the image information of the light spot area meets a preset condition.
10. The apparatus of claim 9, further comprising:
the parameter adjusting module is used for adjusting acquisition parameters of image acquisition equipment for acquiring the image to be identified based on the image information of the light spot area of the image to be identified; and/or
And the image processing module is used for processing the image to be identified based on the image information of the light spot area of the image to be identified.
11. The apparatus of claim 10, wherein the parameter adjustment module comprises:
the brightness information determining submodule is used for determining the brightness information of the light spot area of the image to be identified;
and the exposure parameter adjusting submodule is used for adjusting the exposure parameters of the image acquisition equipment according to the brightness information of the light spot area.
12. The apparatus of claim 10, wherein the parameter adjustment module comprises:
the area determination submodule is used for determining the area of a light spot area of the image to be identified and the area of the image to be identified;
and the focal length adjusting submodule is used for adjusting the focal length of the image acquisition equipment according to the relationship between the boundary of the light spot region and the boundary of the image to be identified, the area of the light spot region of the image to be identified and the area of the image to be identified.
13. The apparatus of claim 10, wherein the image processing module comprises:
the region determining submodule is used for determining a current region to be coded of the image to be identified, and the size of the current region to be coded is a preset value;
the area judgment submodule is used for judging whether the current area to be coded is positioned in the speckle area;
and the coding submodule is used for coding the current region to be coded by adopting a preset th coding parameter when the current region to be coded is positioned in the facula region, and coding the current region to be coded by adopting a preset second coding parameter when the current region to be coded is not positioned in the facula region, wherein the second coding parameter is smaller than the th coding parameter.
14. The apparatus according to claim 13, wherein the region determining submodule is specifically configured to:
identifying th regions overlapping the light spot region and remaining second regions in the current region to be encoded;
and when the area of the th region is larger than or equal to the area of the second region, determining that the region to be coded is positioned in the spot region, and when the area of the th region is smaller than the area of the second region, determining that the region to be coded is not positioned in the spot region.
15. The apparatus of any of claims 9-14 and , further comprising:
the device comprises a sample image acquisition module, a light spot area acquisition module and a light source module, wherein the sample image acquisition module is used for acquiring a sample image containing the light spot area, the sample image is an image acquired in a shadowless lamp irradiation area, the light spot area is in a preset shape, and the image information of the light spot area meets the preset condition;
and the neural network model training module is used for inputting the sample images into a preset initial neural network model, and obtaining the neural network model when the identified area of the initial neural network model is matched with the light spot area included in each corresponding sample image according to the image information of each sample image and the shape of each area included in each sample image.
16. The apparatus according to any one of claims 9-14 and , wherein the predetermined conditions include at least that the brightness value of the spot region satisfies a predetermined size relationship with the brightness value of the remaining region of the sample image except the spot region, that the pixel value of the spot region is within a predetermined pixel value range, and that the brightness value of the spot region is within a predetermined brightness value range.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810796175.6A CN110738224A (en) | 2018-07-19 | 2018-07-19 | image processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810796175.6A CN110738224A (en) | 2018-07-19 | 2018-07-19 | image processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110738224A true CN110738224A (en) | 2020-01-31 |
Family
ID=69235600
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810796175.6A Pending CN110738224A (en) | 2018-07-19 | 2018-07-19 | image processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110738224A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111460844A (en) * | 2020-04-17 | 2020-07-28 | 支付宝(杭州)信息技术有限公司 | Method, device and equipment for detecting positioning light of code scanning equipment |
| CN111862035A (en) * | 2020-07-17 | 2020-10-30 | 平安科技（深圳）有限公司 | Training method of light spot detection model, light spot detection method, device and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101860757A (en) * | 2010-06-03 | 2010-10-13 | 无锡中星微电子有限公司 | Intelligent monitoring system and method for encoding and decoding images thereof |
| CN103366190A (en) * | 2013-07-26 | 2013-10-23 | 中国科学院自动化研究所 | Method for identifying traffic sign |
| CN103413124A (en) * | 2013-08-19 | 2013-11-27 | 中国科学院自动化研究所 | Method for detecting round traffic sign |
| CN104732543A (en) * | 2015-03-30 | 2015-06-24 | 中国人民解放军63655部队 | Infrared weak small target fast detecting method under desert and gobi background |
| CN105488468A (en) * | 2015-11-26 | 2016-04-13 | 浙江宇视科技有限公司 | Method and device for positioning target area |
-
2018
- 2018-07-19 CN CN201810796175.6A patent/CN110738224A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101860757A (en) * | 2010-06-03 | 2010-10-13 | 无锡中星微电子有限公司 | Intelligent monitoring system and method for encoding and decoding images thereof |
| CN103366190A (en) * | 2013-07-26 | 2013-10-23 | 中国科学院自动化研究所 | Method for identifying traffic sign |
| CN103413124A (en) * | 2013-08-19 | 2013-11-27 | 中国科学院自动化研究所 | Method for detecting round traffic sign |
| CN104732543A (en) * | 2015-03-30 | 2015-06-24 | 中国人民解放军63655部队 | Infrared weak small target fast detecting method under desert and gobi background |
| CN105488468A (en) * | 2015-11-26 | 2016-04-13 | 浙江宇视科技有限公司 | Method and device for positioning target area |
Non-Patent Citations (1)
| Title |
|---|
| 彭瑞云 等: "《形态计量与图像分析学》", 31 August 2012, 军事医学科学出版社 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111460844A (en) * | 2020-04-17 | 2020-07-28 | 支付宝(杭州)信息技术有限公司 | Method, device and equipment for detecting positioning light of code scanning equipment |
| CN111862035A (en) * | 2020-07-17 | 2020-10-30 | 平安科技（深圳）有限公司 | Training method of light spot detection model, light spot detection method, device and medium |
| WO2021120842A1 (en) * | 2020-07-17 | 2021-06-24 | 平安科技（深圳）有限公司 | Training method for facula detection model, method for facula detection, device and medium |
| CN111862035B (en) * | 2020-07-17 | 2023-07-28 | 平安科技（深圳）有限公司 | Training method of light spot detection model, light spot detection method, device and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3611915B1 (en) | Method and apparatus for image processing | |
| CN107886484B (en) | Beautifying method, beautifying device, computer-readable storage medium and electronic equipment | |
| US20190378247A1 (en) | Image processing method, electronic device and non-transitory computer-readable recording medium | |
| CN108810413B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| US9251439B2 (en) | Image sharpness classification system | |
| CN110650291B (en) | Target focus tracking method and device, electronic equipment and computer readable storage medium | |
| US20100214445A1 (en) | Image capturing method, image capturing apparatus, and computer program | |
| AU2016298076A1 (en) | Automatic fundus image capture system | |
| US20130147910A1 (en) | Mobile device and image capturing method | |
| JP5421727B2 (en) | Image processing apparatus and control method thereof | |
| CN111368819B (en) | Light spot detection method and device | |
| US20130169824A1 (en) | Blur detection system for night scene images | |
| CN108093170B (en) | User photographing method, device and equipment | |
| CN110738224A (en) | image processing method and device | |
| CN112887610A (en) | Shooting method, shooting device, electronic equipment and storage medium | |
| CN107547839A (en) | Remote control table based on graphical analysis | |
| CN110365897B (en) | Image correction method and device, electronic equipment and computer readable storage medium | |
| CN110191324B (en) | Image processing method, image processing apparatus, server, and storage medium | |
| CN113014876A (en) | Video monitoring method and device, electronic equipment and readable storage medium | |
| US11902533B2 (en) | Code rate control method and apparatus, image acquisition device, and readable storage medium | |
| US20120188437A1 (en) | Electronic camera | |
| JP2016092513A (en) | Image acquisition device, shake reduction method and program | |
| Chen et al. | A quality measure of mobile phone captured 2D barcode images | |
| CN108419047B (en) | Remote monitoring method based on cloud computing | |
| CN111275045A (en) | Method and device for identifying image subject, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200131 |
|
| RJ01 | Rejection of invention patent application after publication |