CN111800568B - Light supplement method and device - Google Patents
Light supplement method and device Download PDFInfo
- Publication number
- CN111800568B CN111800568B CN202010784875.0A CN202010784875A CN111800568B CN 111800568 B CN111800568 B CN 111800568B CN 202010784875 A CN202010784875 A CN 202010784875A CN 111800568 B CN111800568 B CN 111800568B
- Authority
- CN
- China
- Prior art keywords
- matching
- color temperature
- image
- temperature value
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 70
- 239000013589 supplement Substances 0.000 title claims abstract description 57
- 230000008569 process Effects 0.000 claims abstract description 26
- 230000001502 supplementing effect Effects 0.000 claims abstract description 16
- 238000004422 calculation algorithm Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 2
- 230000002123 temporal effect Effects 0.000 claims 3
- 238000012937 correction Methods 0.000 abstract description 8
- 238000005516 engineering process Methods 0.000 abstract description 7
- 238000004590 computer program Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 10
- 230000000694 effects Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 238000003384 imaging method Methods 0.000 description 4
- 230000007613 environmental effect Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/56—Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/74—Circuitry for compensating brightness variation in the scene by influencing the scene brightness using illuminating means
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3179—Video signal processing therefor
- H04N9/3182—Colour adjustment, e.g. white balance, shading or gamut
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
- Color Television Image Signal Generators (AREA)
Abstract
The invention discloses a light supplementing method and a light supplementing device, which are applied to mobile equipment, wherein the mobile equipment is provided with a light supplementing lamp and a plurality of cameras, and the light supplementing method comprises the following steps: shooting a first preset number of images through a first preset number of cameras to obtain a first color temperature value; matching the first target image with images in a preset image library to obtain a matching result, wherein the first target image is one of a first preset number of images; determining a second color temperature value according to the matching result; obtaining a target color temperature value according to the first color temperature value and the second color temperature value; and controlling a light supplement lamp to supplement light for the shooting process of the mobile equipment according to the target color temperature value. The invention solves the technical problem that the mobile terminal in the related technology has higher difficulty in color correction in the shooting process.
Description
Technical Field
The invention relates to the field of mobile terminal camera shooting, in particular to a light supplementing method and device.
Background
In the related art, the requirements of the current camera shooting scenes of the mobile terminal tend to be diversified, different camera shooting scenes have light sources with different color temperatures, and a camera needs to know the color temperatures of the scenes under different light sources, so that correct color correction is performed. The light supplement lamp of the terminal camera has a certain color temperature, the color temperature of the light supplement lamp is mixed with the environmental color temperature, and the difficulty of image color correction is increased when an image is generated.
In view of the above problems in the related art, no effective solution has been proposed.
Disclosure of Invention
The invention mainly aims to provide a light supplementing method and a light supplementing device, and aims to solve the technical problem that in the related art, a mobile terminal is difficult to correct colors in the shooting process.
In order to achieve the above object, according to an aspect of the present invention, there is provided a supplementary lighting method applied in a mobile device, where a supplementary lighting lamp and a plurality of cameras are provided on the mobile device, the method including: shooting a first preset number of images through a first preset number of cameras to obtain a first color temperature value; matching the first target image with images in a preset image library to obtain a matching result, wherein the first target image is one of a first preset number of images; determining a second color temperature value according to the matching result; obtaining a target color temperature value according to the first color temperature value and the second color temperature value; and controlling a light supplement lamp to supplement light for the shooting process of the mobile equipment according to the target color temperature value.
Further, before a first preset number of images are captured by a first preset number of cameras to obtain a first color temperature value, the method further includes: determining one camera in the multiple cameras as a first camera, wherein an image acquired by the first camera is a first target image; and determining a second preset number of cameras in the plurality of cameras as second cameras, wherein the second preset number is smaller than the first preset number, and the first cameras and the second cameras with the second preset number form the cameras with the first preset number.
Further, shooting a first preset number of images through a first preset number of cameras to obtain a first color temperature value includes: controlling a first preset number of cameras to shoot a first preset number of images under the condition that a light supplement lamp of the mobile device is turned off; acquiring a first preset number of color temperature values corresponding to a first preset number of images; and calculating the average value of the color temperature values of the first preset number, and determining the average value as the first color temperature value.
Further, matching the first target image with an image in a preset gallery, and acquiring a matching result includes: determining the shooting time of a first target image; determining a preset time range according to the shooting time; determining at least one matching image in a preset image library according to a preset time range, wherein the matching image is an image of which the shooting time in the preset image library belongs to the preset time range; determining the time similarity corresponding to at least one matching image according to the shooting time of at least one matching image, wherein the time similarity of the matching images is higher when the shooting time is closer to the shooting time of the first target image; and generating a first matching result according to the time similarity of at least one matching image.
Further, after generating a first matching result according to the time similarity of at least one matching image, matching the first target image with an image in a preset gallery, and acquiring the matching result further includes: converting the first target image and the at least one matching image into character strings; determining the content similarity of at least one matching image according to the character string and a preset hash algorithm, wherein the closer the content in the matching image is to the content in the target image, the higher the content similarity of the matching image is; and multiplying the time similarity of at least one matching image in the first matching result by the content similarity of the at least one matching image to obtain a matching result, wherein the matching result comprises at least one matching value corresponding to the at least one matching image.
Further, determining the second color temperature value according to the matching result comprises: determining the highest matching value in the at least one matching value, and judging whether the highest matching value is greater than or equal to a preset threshold value; if the highest matching value is larger than or equal to a preset threshold value, determining the matching image corresponding to the highest matching value as a second target image and determining the color temperature value corresponding to the second target image as a second color temperature value; and if the highest matching value is smaller than the preset threshold value, determining that the second target image does not exist in the preset image library and determining that the second color temperature value is zero.
Further, obtaining the target color temperature value according to the first color temperature value and the second color temperature value includes: if the preset image library contains a second target image, performing weighted average on the first color temperature value and the second color temperature value to obtain a weighted average value, and determining the weighted average value as a target color temperature value; and if the preset image library does not contain the second target image, determining the first color temperature value as the target color temperature value.
In order to achieve the above object, according to another aspect of the present invention, there is provided a light supplement device for use in a mobile device, the mobile device being provided with a light supplement lamp and a plurality of cameras, the device including: the device comprises a first acquisition unit, a second acquisition unit and a control unit, wherein the first acquisition unit is used for shooting a first preset number of images through a first preset number of cameras to acquire a first color temperature value; the second obtaining unit is used for matching the first target image with images in a preset image library to obtain a matching result, wherein the first target image is one of the images in the first preset number; the first determining unit is used for determining a second color temperature value according to the matching result; the third obtaining unit is used for obtaining a target color temperature value according to the first color temperature value and the second color temperature value; and the control unit is used for controlling the light supplementing lamp to supplement light for the shooting process of the mobile equipment according to the target color temperature value.
The invention adopts the following steps: shooting a first preset number of images through a first preset number of cameras to obtain a first color temperature value; matching the first target image with images in a preset image library to obtain a matching result, wherein the first target image is one of a first preset number of images; determining a second color temperature value according to the matching result; obtaining a target color temperature value according to the first color temperature value and the second color temperature value; according to the target color temperature value, the light supplementing lamp is controlled to supplement light for the shooting process of the mobile device, the technical problem that the difficulty of color correction of the mobile terminal in the shooting process in the related technology is high is solved, and the technical effect of improving the image imaging color is achieved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate an embodiment of the invention and, together with the description, serve to explain the invention and not to limit the invention. In the drawings:
fig. 1 is a flowchart of a light supplement method according to an embodiment of the present invention;
fig. 2 is a flowchart illustrating another specific step of a light supplement method according to an embodiment of the present invention; and
fig. 3 is a schematic diagram of a light supplement device according to an embodiment of the present invention.
Detailed Description
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict. The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged under appropriate circumstances in order to facilitate the description of the embodiments of the invention herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
According to an embodiment of the invention, a light supplement method is provided.
Fig. 1 is a flowchart of a light supplement method according to an embodiment of the present invention, where the method is applied to a mobile device, and the mobile device is provided with a light supplement lamp and a plurality of cameras, as shown in fig. 1, the method includes the following steps:
step S101, shooting a first preset number of images through a first preset number of cameras to obtain a first color temperature value;
step S102, matching a first target image with images in a preset image library to obtain a matching result, wherein the first target image is one of a first preset number of images;
step S103, determining a second color temperature value according to the matching result;
step S104, obtaining a target color temperature value according to the first color temperature value and the second color temperature value;
and S105, controlling a light supplement lamp to supplement light for the shooting process of the mobile equipment according to the target color temperature value.
In the above, in the present embodiment, a mobile device is provided, on which a plurality of cameras are disposed, and preferably, the plurality of cameras constitute a camera array, wherein the camera array may be a vertically arranged array, a horizontally arranged array, or an array arranged in multiple rows.
The embodiment of the application is based on the plurality of cameras, and in order to reduce the interference effect of the color temperature of the light supplement lamp on the current environment color temperature, the similar images in the plurality of cameras and the gallery are adopted to analyze the current environment color temperature respectively and perform targeted light supplement, so that the difficulty of color correction in the shooting process is reduced, and the color effect of image imaging is ensured.
The embodiment of the invention provides a light supplementing method, wherein a first preset number of cameras are used for shooting a first preset number of images to obtain a first color temperature value; matching the first target image with images in a preset image library to obtain a matching result, wherein the first target image is one of a first preset number of images; determining a second color temperature value according to the matching result; obtaining a target color temperature value according to the first color temperature value and the second color temperature value; according to the target color temperature value, the light supplementing lamp is controlled to supplement light for the shooting process of the mobile device, the technical problem that the difficulty of color correction of the mobile terminal in the shooting process in the related technology is high is solved, and the technical effect of improving the image imaging color is achieved.
Optionally, before the first preset number of images are captured by the first preset number of cameras to obtain the first color temperature value, the method further includes: determining one camera in the multiple cameras as a first camera, wherein an image acquired by the first camera is a first target image; and determining a second preset number of cameras in the plurality of cameras as second cameras, wherein the second preset number is smaller than the first preset number, and the first cameras and the second cameras with the second preset number form the cameras with the first preset number.
Specifically, in the light supplement method provided by the embodiment of the application, a plurality of cameras are adopted, and besides a conventional preview photographing function of a main camera, a plurality of cameras used for working are used for collecting images to estimate the color temperature of the environment.
The mobile equipment is provided with the plurality of cameras, so a main camera needs to be determined, wherein an image acquired by the main camera is a first target image, the first target image acquired by the main camera is matched with a picture in a preset image library to determine a second color temperature value, the plurality of second cameras need to be determined after the main camera is determined, and the color temperature under the current environment is estimated through the plurality of second cameras and the image acquired by the main camera.
In an alternative embodiment, if the plurality of cameras form a quadrilateral array, determining a camera in the geometric center position in the camera array as a first camera (i.e., a main camera), and determining four cameras located at four corners of an area where the camera array is located as second cameras, where images collected by the second cameras and the first cameras can estimate a current ambient color temperature, and the main camera and the plurality of second cameras constitute a first preset number of cameras for a user to work.
It should be noted that, in different devices and in different scenes, the specific main camera and the plurality of second cameras are determined according to specific situations, and the present application is not particularly limited.
Meanwhile, it should be noted that the number of the main cameras is generally one, the number of the second cameras is generally multiple, and the specific number is determined according to specific requirements.
Optionally, before the first preset number of images are captured by the first preset number of cameras to obtain the first color temperature value, the method further includes: after the mobile equipment is detected to be opened, parameter initialization is carried out on the cameras of the first preset number, and after the parameter initialization is completed, the first color temperature value and the second color temperature value are determined through images collected by the cameras of the first preset number.
Optionally, capturing a first preset number of images through a first preset number of cameras to obtain a first color temperature value includes: controlling a first preset number of cameras to shoot a first preset number of images under the condition that a light supplement lamp of the mobile device is turned off; acquiring a first preset number of color temperature values corresponding to a first preset number of images; and calculating the average value of the color temperature values of the first preset number, and determining the average value as the first color temperature value.
Specifically, after the cameras used for working are determined, a multi-camera color temperature estimation stage is started, a first preset number of cameras selected in the last stage and used for working are started, under the condition that a light supplement lamp is not started, the first preset number of cameras are controlled to capture images, information of the collected first preset number of images is analyzed, color temperature estimation is performed according to statistical information of all statistic points in all the images to obtain a plurality of color temperature values, the plurality of color temperature values are averaged, and therefore a first color temperature value is obtained, namely the plurality of images collected by the first preset number of cameras are used for estimating environmental color temperature to obtain an environmental color temperature value.
Optionally, matching the first target image with an image in a preset gallery, and obtaining a matching result includes: determining the shooting time of a first target image; determining a preset time range according to the shooting time; determining at least one matching image in a preset image library according to a preset time range, wherein the matching image is an image of which the shooting time in the preset image library belongs to the preset time range; determining the time similarity corresponding to at least one matching image according to the shooting time of at least one matching image, wherein the time similarity of the matching images is higher when the shooting time is closer to the shooting time of the first target image; and generating a first matching result according to the time similarity of at least one matching image.
After the ambient color temperature value is determined, an image matching stage is entered, in the image matching stage, the first target image is matched with the pictures in the preset picture library to generate a first matching result, wherein the matching image related to the first target image and existing in the preset picture library is determined by the shooting time of the first target image, specifically, the preset time range is determined by the shooting time of the first target image, whether the shooting time of the pictures in the preset picture library is within the preset time range is judged, if such a picture exists, such a picture is determined as the matching image of the first target image, wherein the closer the shooting time is to the shooting time of the first target image, the higher the time similarity of the matching image is.
For example, if the shooting time of the first target image is 2019, 7, 14:20, the preset time range is determined to be 2019, 7, 14:00-2019, 7, 14:30, so that the shooting time of which images in the preset image library are in the preset time range is judged through the determined preset time range, the images with the shooting time in the preset time range are determined as matching images, and the time similarity of the matching images is determined through the shooting time. For example, the time similarity of the image whose photographing time is 2019, 7, 14:10 is higher than that of the image whose photographing time is 2019, 7, 14: 05.
It should be noted that the preset gallery is generally a gallery of the mobile device itself, and may also be a gallery in communication connection with the mobile device in different embodiments.
Optionally, after generating a first matching result according to the time similarity of at least one matching image, matching the first target image with an image in a preset gallery, and acquiring the matching result further includes: converting the first target image and the at least one matching image into character strings; determining the content similarity of at least one matching image according to the character string and a preset hash algorithm, wherein the closer the content in the matching image is to the content in the target image, the higher the content similarity of the matching image is; and multiplying the time similarity of at least one matching image in the first matching result by the content similarity of the at least one matching image to obtain a matching result, wherein the matching result comprises at least one matching value corresponding to the at least one matching image.
In the above, after determining at least one matching image through the time similarity, the content similarity of the matching image needs to be obtained, where the specific means is to convert the at least one matching image and the first target image into a character string, preferably, convert the image into the character string through a perceptual hash algorithm, and determine the content similarity ranking of the matching image through the comparison between the character string of the matching image and the character string of the first target image.
After the content similarity and the time similarity of the matched images are determined through the steps, the time similarity of each matched image is multiplied by the content similarity to obtain at least one matching value, the matching values are sorted in a descending order, and the sorted matching values are determined as matching results.
Optionally, determining the second color temperature value according to the matching result includes: determining the highest matching value in the at least one matching value, and judging whether the highest matching value is greater than or equal to a preset threshold value; if the highest matching value is larger than or equal to a preset threshold value, determining the matching image corresponding to the highest matching value as a second target image and determining the color temperature value corresponding to the second target image as a second color temperature value; and if the highest matching value is smaller than the preset threshold value, determining that the second target image does not exist in the preset image library and determining that the second color temperature value is zero.
In the above, after at least one matching value is sorted, it is determined whether the first-ranked matching value is greater than 0.8, and if the first matching value is greater than 0.8, the image corresponding to the first-ranked matching value is determined as the second target image, where the color temperature value of the second target image is the second color temperature value, and the second color temperature value is used to estimate the target color temperature value. If the first ranked match value is less than or equal to 0.8, then it is determined that a second target image is not present in the gallery.
The color temperature value corresponding to the second target image is a second color temperature value.
Optionally, obtaining the target color temperature value according to the first color temperature value and the second color temperature value includes: if the preset image library contains a second target image, performing weighted average on the first color temperature value and the second color temperature value to obtain a weighted average value, and determining the weighted average value as a target color temperature value; and if the preset image library does not contain the second target image, determining the first color temperature value as the target color temperature value.
In the above, after the first color temperature value and the second color temperature value are determined, the final color temperature value (i.e., the target color temperature value) needs to be calculated according to the first color temperature value and the second color temperature value.
Namely, after entering a color temperature estimation stage, performing weighted average calculation on the first color temperature value and the second color temperature value to obtain a target color temperature value, if the second target image does not exist in the gallery, only the first color temperature value exists, the target color temperature value is numerically equal to the first color temperature value, after the target color temperature value is determined, transmitting the determined target color temperature value information to a light supplement lamp, and the light supplement lamp obtains a light supplement effect of the target color temperature value through different current ratios to supplement light in a shooting process.
Further, after the target color temperature value is determined, the purpose of adjusting the color temperature is achieved by adjusting the current of the LED driving circuit of the mobile device.
Fig. 2 is a flowchart illustrating another specific step of a light supplement method according to an embodiment of the present invention, where the specific step is shown in fig. 2.
By the light supplement method, the effects of estimating the current ambient color temperature through the plurality of cameras and determining the color temperature of the light supplement lamp according to the current ambient color temperature are achieved.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.
The embodiment of the present invention further provides a light supplement device, and it should be noted that the light supplement device of the embodiment of the present invention can be used for executing the light supplement method provided by the embodiment of the present invention. A light supplement device according to an embodiment of the present invention is described below.
Fig. 3 is a schematic diagram of a light supplement device according to an embodiment of the present invention. As shown in fig. 3, the apparatus is applied to a mobile device, and the mobile device is provided with a fill-in light and a plurality of cameras, and the apparatus includes: the device comprises a first acquisition unit, a second acquisition unit and a control unit, wherein the first acquisition unit is used for shooting a first preset number of images through a first preset number of cameras to acquire a first color temperature value; the second obtaining unit is used for matching the first target image with images in a preset image library to obtain a matching result, wherein the first target image is one of the images in the first preset number; the first determining unit is used for determining a second color temperature value according to the matching result; the third obtaining unit is used for obtaining a target color temperature value according to the first color temperature value and the second color temperature value; and the control unit is used for controlling the light supplementing lamp to supplement light for the shooting process of the mobile equipment according to the target color temperature value.
The light supplementing device provided by the embodiment of the invention is characterized in that the first obtaining unit is used for shooting a first preset number of images through a first preset number of cameras to obtain a first color temperature value; the second obtaining unit is used for matching the first target image with images in a preset image library to obtain a matching result, wherein the first target image is one of the images in the first preset number; the first determining unit is used for determining a second color temperature value according to the matching result; the third obtaining unit is used for obtaining a target color temperature value according to the first color temperature value and the second color temperature value; the control unit is used for controlling the light supplementing lamp to supplement light for the shooting process of the mobile device according to the target color temperature value, the technical problem that the difficulty of color correction of the mobile terminal in the shooting process is high in the related technology is solved, and the technical effect of improving the image imaging color is achieved.
Optionally, the apparatus further comprises: the device comprises a first determining unit, a second determining unit and a third determining unit, wherein the first determining unit is used for determining one camera in a plurality of cameras as a first camera before a first preset number of images are shot by the first preset number of cameras to obtain a first color temperature value, and the images collected by the first camera are first target images; and the fourth determining unit is used for determining a second preset number of cameras in the plurality of cameras as second cameras, wherein the second preset number is smaller than the first preset number, and the first cameras and the second cameras with the second preset number form the cameras with the first preset number.
Optionally, the first obtaining unit includes: the control subunit is used for controlling the cameras with the first preset number to shoot the images with the first preset number under the condition that the light supplement lamp of the mobile equipment is turned off; the first obtaining subunit is configured to obtain a first preset number of color temperature values corresponding to a first preset number of images; and the first determining subunit is used for calculating the average value of the color temperature values of the first preset number and determining the average value as the first color temperature value.
Optionally, the second obtaining unit includes: a second determining subunit, configured to determine a shooting time of the first target image; the third determining subunit is used for determining a preset time range according to the shooting time; the fourth determining subunit is configured to determine, according to the preset time range, at least one matching image existing in the preset gallery, where the matching image is an image in the preset gallery in which the shooting time belongs to the preset time range; the fifth determining subunit is configured to determine, according to the shooting time of the at least one matching image, a time similarity corresponding to the at least one matching image, where the time similarity of the matching image is higher when the shooting time is closer to the shooting time of the first target image; and the generating subunit is used for generating a first matching result according to the time similarity of at least one matching image.
Optionally, after generating the first matching result according to the time similarity of the at least one matching image, the second obtaining unit further includes: a conversion subunit, configured to convert the first target image and the at least one matching image into a character string; a sixth determining subunit, configured to determine, according to the character string and a preset hash algorithm, a content similarity of at least one matching image, where the closer the content in the matching image is to the content in the target image, the higher the content similarity of the matching image is; and the second obtaining subunit is configured to multiply the time similarity of the at least one matching image in the first matching result by the content similarity of the at least one matching image, so as to obtain a matching result, where the matching result includes at least one matching value corresponding to the at least one matching image.
Optionally, the first determination unit includes: the judging subunit is used for determining the highest matching value in the at least one matching value and judging whether the highest matching value is greater than or equal to a preset threshold value; a seventh determining subunit, configured to determine, when the highest matching value is greater than or equal to the preset threshold, the matching image corresponding to the highest matching value as the second target image and determine the color temperature value corresponding to the second target image as the second color temperature value; and the eighth determining subunit is configured to determine that the second target image does not exist in the preset map library and determine that the second color temperature value is zero, when the highest matching value is smaller than the preset threshold value.
Optionally, the third obtaining unit includes: a ninth determining subunit, configured to perform weighted average on the first color temperature value and the second color temperature value to obtain a weighted average value when the preset map library includes the second target image, and determine the weighted average value as the target color temperature value; and the tenth determining subunit is configured to determine the first color temperature value as the target color temperature value when the preset gallery does not include the second target image.
The first obtaining unit 301 and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.
The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to be one or more than one, and the technical problem that the mobile terminal in the related technology has high difficulty in color correction in the shooting process is solved by adjusting the kernel parameters.
The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.
The embodiment of the invention provides a computer-readable storage medium or a nonvolatile storage medium, and the program realizes a light supplement method when being executed by a processor.
The embodiment of the invention provides a processor, which is used for running a program, wherein a light supplementing method is executed when the program runs.
The embodiment of the invention provides equipment, which comprises a processor, a computer-readable storage medium or a nonvolatile storage medium and a program which is stored on the computer-readable storage medium or the nonvolatile storage medium and can run on the processor, wherein the processor executes the program to realize the following steps: shooting a first preset number of images through a first preset number of cameras to obtain a first color temperature value; matching the first target image with images in a preset image library to obtain a matching result, wherein the first target image is one of a first preset number of images; determining a second color temperature value according to the matching result; obtaining a target color temperature value according to the first color temperature value and the second color temperature value; and controlling a light supplement lamp to supplement light for the shooting process of the mobile equipment according to the target color temperature value.
Optionally, before the first preset number of images are captured by the first preset number of cameras to obtain the first color temperature value, the method further includes: determining one camera in the multiple cameras as a first camera, wherein an image acquired by the first camera is a first target image; and determining a second preset number of cameras in the plurality of cameras as second cameras, wherein the second preset number is smaller than the first preset number, and the first cameras and the second cameras with the second preset number form the cameras with the first preset number.
Optionally, capturing a first preset number of images through a first preset number of cameras to obtain a first color temperature value includes: controlling a first preset number of cameras to shoot a first preset number of images under the condition that a light supplement lamp of the mobile device is turned off; acquiring a first preset number of color temperature values corresponding to a first preset number of images; and calculating the average value of the color temperature values of the first preset number, and determining the average value as the first color temperature value.
Optionally, matching the first target image with an image in a preset gallery, and obtaining a matching result includes: determining the shooting time of a first target image; determining a preset time range according to the shooting time; determining at least one matching image in a preset image library according to a preset time range, wherein the matching image is an image of which the shooting time in the preset image library belongs to the preset time range; determining the time similarity corresponding to at least one matching image according to the shooting time of at least one matching image, wherein the time similarity of the matching images is higher when the shooting time is closer to the shooting time of the first target image; and generating a first matching result according to the time similarity of at least one matching image.
Optionally, after generating a first matching result according to the time similarity of at least one matching image, matching the first target image with an image in a preset gallery, and acquiring the matching result further includes: converting the first target image and the at least one matching image into character strings; determining the content similarity of at least one matching image according to the character string and a preset hash algorithm, wherein the closer the content in the matching image is to the content in the target image, the higher the content similarity of the matching image is; and multiplying the time similarity of at least one matching image in the first matching result by the content similarity of the at least one matching image to obtain a matching result, wherein the matching result comprises at least one matching value corresponding to the at least one matching image.
Optionally, determining the second color temperature value according to the matching result includes: determining the highest matching value in the at least one matching value, and judging whether the highest matching value is greater than or equal to a preset threshold value; if the highest matching value is larger than or equal to a preset threshold value, determining the matching image corresponding to the highest matching value as a second target image and determining the color temperature value corresponding to the second target image as a second color temperature value; and if the highest matching value is smaller than the preset threshold value, determining that the second target image does not exist in the preset image library and determining that the second color temperature value is zero.
Optionally, obtaining the target color temperature value according to the first color temperature value and the second color temperature value includes: if the preset image library contains a second target image, performing weighted average on the first color temperature value and the second color temperature value to obtain a weighted average value, and determining the weighted average value as a target color temperature value; and if the preset image library does not contain the second target image, determining the first color temperature value as the target color temperature value. The device herein may be a server, a PC, a PAD, a mobile phone, etc.
The invention also provides a computer program product adapted to perform a program for initializing the following method steps when executed on a data processing device: shooting a first preset number of images through a first preset number of cameras to obtain a first color temperature value; matching the first target image with images in a preset image library to obtain a matching result, wherein the first target image is one of a first preset number of images; determining a second color temperature value according to the matching result; obtaining a target color temperature value according to the first color temperature value and the second color temperature value; and controlling a light supplement lamp to supplement light for the shooting process of the mobile equipment according to the target color temperature value.
Optionally, before the first preset number of images are captured by the first preset number of cameras to obtain the first color temperature value, the method further includes: determining one camera in the multiple cameras as a first camera, wherein an image acquired by the first camera is a first target image; and determining a second preset number of cameras in the plurality of cameras as second cameras, wherein the second preset number is smaller than the first preset number, and the first cameras and the second cameras with the second preset number form the cameras with the first preset number.
Optionally, capturing a first preset number of images through a first preset number of cameras to obtain a first color temperature value includes: controlling a first preset number of cameras to shoot a first preset number of images under the condition that a light supplement lamp of the mobile device is turned off; acquiring a first preset number of color temperature values corresponding to a first preset number of images; and calculating the average value of the color temperature values of the first preset number, and determining the average value as the first color temperature value.
Optionally, matching the first target image with an image in a preset gallery, and obtaining a matching result includes: determining the shooting time of a first target image; determining a preset time range according to the shooting time; determining at least one matching image in a preset image library according to a preset time range, wherein the matching image is an image of which the shooting time in the preset image library belongs to the preset time range; determining the time similarity corresponding to at least one matching image according to the shooting time of at least one matching image, wherein the time similarity of the matching images is higher when the shooting time is closer to the shooting time of the first target image; and generating a first matching result according to the time similarity of at least one matching image.
Optionally, after generating a first matching result according to the time similarity of at least one matching image, matching the first target image with an image in a preset gallery, and acquiring the matching result further includes: converting the first target image and the at least one matching image into character strings; determining the content similarity of at least one matching image according to the character string and a preset hash algorithm, wherein the closer the content in the matching image is to the content in the target image, the higher the content similarity of the matching image is; and multiplying the time similarity of at least one matching image in the first matching result by the content similarity of the at least one matching image to obtain a matching result, wherein the matching result comprises at least one matching value corresponding to the at least one matching image.
Optionally, determining the second color temperature value according to the matching result includes: determining the highest matching value in the at least one matching value, and judging whether the highest matching value is greater than or equal to a preset threshold value; if the highest matching value is larger than or equal to a preset threshold value, determining the matching image corresponding to the highest matching value as a second target image and determining the color temperature value corresponding to the second target image as a second color temperature value; and if the highest matching value is smaller than the preset threshold value, determining that the second target image does not exist in the preset image library and determining that the second color temperature value is zero.
Optionally, obtaining the target color temperature value according to the first color temperature value and the second color temperature value includes: if the preset image library contains a second target image, performing weighted average on the first color temperature value and the second color temperature value to obtain a weighted average value, and determining the weighted average value as a target color temperature value; and if the preset image library does not contain the second target image, determining the first color temperature value as the target color temperature value.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The above are merely examples of the present invention, and are not intended to limit the present invention. Various modifications and alterations to this invention will become apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.
Claims (7)
1. The light supplement method is applied to mobile equipment, wherein a light supplement lamp and a plurality of cameras are arranged on the mobile equipment, and the light supplement method comprises the following steps:
shooting a first preset number of images through a first preset number of cameras to obtain a first color temperature value;
matching a first target image with images in a preset image library to obtain a matching result, wherein the first target image is one of the images in the first preset number;
determining a second color temperature value according to the matching result;
obtaining a target color temperature value according to the first color temperature value and the second color temperature value;
according to the target color temperature value, controlling the light supplement lamp to supplement light for the shooting process of the mobile device, matching a first target image with an image in a preset image library, and obtaining a matching result comprises:
determining the shooting time of the first target image;
determining a preset time range according to the shooting time;
determining at least one matching image in the preset image library according to the preset time range, wherein the matching image is an image of which the shooting time in the preset image library belongs to the preset time range;
determining time similarity corresponding to at least one matching image according to shooting time of the at least one matching image, wherein the time similarity of the matching image is higher when the shooting time is closer to the shooting time of the first target image;
generating a first matching result depending on the temporal similarity of at least one of the matching images,
after generating a first matching result according to the time similarity of at least one matching image, matching a first target image with an image in a preset image library, wherein obtaining the matching result further comprises:
converting the first target image and at least one of the matching images into a character string;
determining the content similarity of at least one matching image according to the character string and a preset hash algorithm, wherein the closer the content in the matching image is to the content in the target image, the higher the content similarity of the matching image is;
multiplying the time similarity of at least one of the matching images in the first matching result by the content similarity of at least one of the matching images to obtain the matching result, wherein the matching result comprises at least one matching value corresponding to at least one of the matching images,
determining a second color temperature value according to the matching result comprises:
determining the highest matching value of at least one matching value, and judging whether the highest matching value is greater than or equal to a preset threshold value;
if the highest matching value is larger than or equal to the preset threshold value, determining the matching image corresponding to the highest matching value as a second target image and determining the color temperature value corresponding to the second target image as the second color temperature value;
and if the highest matching value is smaller than the preset threshold value, determining that the second target image does not exist in the preset image library and determining that the second color temperature value is zero.
2. The method of claim 1, wherein prior to taking a first preset number of images with a first preset number of cameras to obtain a first color temperature value, the method further comprises:
determining one camera in the plurality of cameras as a first camera, wherein an image acquired by the first camera is the first target image;
determining a second preset number of cameras in the plurality of cameras as second cameras, wherein the second preset number is smaller than the first preset number, and the first cameras and the second cameras in the second preset number form the first preset number of cameras.
3. The method of claim 1, wherein capturing a first preset number of images with a first preset number of cameras to obtain a first color temperature value comprises:
controlling the cameras of the first preset number to shoot the images of the first preset number under the condition that the light supplement lamp of the mobile device is turned off;
acquiring a first preset number of color temperature values corresponding to the first preset number of images;
and calculating the average value of the color temperature values of the first preset number, and determining the average value as the first color temperature value.
4. The method of claim 1, wherein obtaining a target color temperature value based on the first color temperature value and the second color temperature value comprises:
if the preset image library contains the second target image, performing weighted average on the first color temperature value and the second color temperature value to obtain a weighted average value, and determining the weighted average value as the target color temperature value;
and if the second target image is not contained in the preset image library, determining the first color temperature value as the target color temperature value.
5. The utility model provides a light filling device, its characterized in that uses in the mobile device, be provided with light filling lamp and a plurality of camera on the mobile device, include:
the device comprises a first acquisition unit, a second acquisition unit and a control unit, wherein the first acquisition unit is used for shooting a first preset number of images through a first preset number of cameras to acquire a first color temperature value;
the second obtaining unit is used for matching a first target image with images in a preset image library to obtain a matching result, wherein the first target image is one of the images in the first preset number;
the first determining unit is used for determining a second color temperature value according to the matching result;
the third obtaining unit is used for obtaining a target color temperature value according to the first color temperature value and the second color temperature value;
the control unit is used for controlling the light supplementing lamp to supplement light for the shooting process of the mobile equipment according to the target color temperature value,
the second acquisition unit includes: a second determining subunit, configured to determine a shooting time of the first target image; the third determining subunit is used for determining a preset time range according to the shooting time; a fourth determining subunit, configured to determine, according to the preset time range, at least one matching image that exists in the preset gallery, where the matching image is an image in the preset gallery in which the shooting time belongs to the preset time range; a fifth determining subunit, configured to determine, according to a shooting time of at least one of the matching images, a time similarity corresponding to the at least one of the matching images, where the time similarity of the matching image is higher when the shooting time is closer to the shooting time of the first target image; a generating subunit, configured to generate a first matching result according to the temporal similarity of at least one of the matching images,
after generating a first matching result depending on the temporal similarity of at least one of the matching images, the second obtaining unit further includes: a conversion subunit, configured to convert the first target image and at least one of the matching images into a character string; a sixth determining subunit, configured to determine, according to the character string and a preset hash algorithm, a content similarity of at least one of the matching images, where the closer a content in the matching image is to a content in the target image, the higher the content similarity of the matching image is; a second obtaining subunit, configured to multiply a time similarity of at least one of the matching images in the first matching result by a content similarity of the at least one of the matching images to obtain the matching result, where the matching result includes at least one matching value corresponding to the at least one of the matching images,
the first determination unit includes: the judging subunit is used for determining the highest matching value in at least one matching value and judging whether the highest matching value is greater than or equal to a preset threshold value; a seventh determining subunit, configured to determine, when the highest matching value is greater than or equal to the preset threshold, a matching image corresponding to the highest matching value as a second target image and determine a color temperature value corresponding to the second target image as the second color temperature value; an eighth determining subunit, configured to determine that the second target image does not exist in the preset gallery and determine that the second color temperature value is zero, when the highest matching value is smaller than the preset threshold.
6. A computer-readable storage medium or a non-volatile storage medium, wherein the computer-readable storage medium or the non-volatile storage medium includes a stored program, and when the program runs, the computer-readable storage medium or the non-volatile storage medium is controlled by a device in which the computer-readable storage medium or the non-volatile storage medium is located to execute a light supplement method according to any one of claims 1 to 4.
7. A processor, configured to execute a program, wherein the program executes the method according to any one of claims 1 to 4.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010784875.0A CN111800568B (en) | 2020-08-06 | 2020-08-06 | Light supplement method and device |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010784875.0A CN111800568B (en) | 2020-08-06 | 2020-08-06 | Light supplement method and device |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111800568A CN111800568A (en) | 2020-10-20 |
CN111800568B true CN111800568B (en) | 2021-11-05 |
Family
ID=72828676
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010784875.0A Active CN111800568B (en) | 2020-08-06 | 2020-08-06 | Light supplement method and device |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111800568B (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112712475B (en) * | 2020-12-17 | 2022-11-15 | 珠海格力电器股份有限公司 | Image color difference compensation method and device, electronic equipment and storage medium |
CN112738957B (en) * | 2020-12-28 | 2023-02-28 | 河北谊安奥美医疗设备有限公司 | Method for controlling shading compensation of operation shadowless lamp |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2009021565A1 (en) * | 2007-08-10 | 2009-02-19 | Sony Ericsson Mobile Communications Ab | Electronic device comprising a display with adjustable brightness, chromaticity or both |
CN105791711A (en) * | 2016-04-29 | 2016-07-20 | 努比亚技术有限公司 | Shooting light supplement apparatus and method thereof |
CN106657947A (en) * | 2017-01-13 | 2017-05-10 | 奇酷互联网络科技(深圳)有限公司 | Image generation method and photographing device |
CN107302669A (en) * | 2017-08-25 | 2017-10-27 | 上海鼎为电子科技(集团)有限公司 | Light compensation method, device and mobile terminal |
CN108965835A (en) * | 2018-08-23 | 2018-12-07 | Oppo广东移动通信有限公司 | A kind of image processing method, image processing apparatus and terminal device |
CN109005360A (en) * | 2018-06-11 | 2018-12-14 | 北京小米移动软件有限公司 | Light compensation method, device and the computer readable storage medium of shooting environmental |
-
2020
- 2020-08-06 CN CN202010784875.0A patent/CN111800568B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2009021565A1 (en) * | 2007-08-10 | 2009-02-19 | Sony Ericsson Mobile Communications Ab | Electronic device comprising a display with adjustable brightness, chromaticity or both |
CN105791711A (en) * | 2016-04-29 | 2016-07-20 | 努比亚技术有限公司 | Shooting light supplement apparatus and method thereof |
CN106657947A (en) * | 2017-01-13 | 2017-05-10 | 奇酷互联网络科技(深圳)有限公司 | Image generation method and photographing device |
CN107302669A (en) * | 2017-08-25 | 2017-10-27 | 上海鼎为电子科技(集团)有限公司 | Light compensation method, device and mobile terminal |
CN109005360A (en) * | 2018-06-11 | 2018-12-14 | 北京小米移动软件有限公司 | Light compensation method, device and the computer readable storage medium of shooting environmental |
CN108965835A (en) * | 2018-08-23 | 2018-12-07 | Oppo广东移动通信有限公司 | A kind of image processing method, image processing apparatus and terminal device |
Also Published As
Publication number | Publication date |
---|---|
CN111800568A (en) | 2020-10-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110099222B (en) | Exposure adjusting method and device for shooting equipment, storage medium and equipment | |
CN108174118B (en) | Image processing method and device and electronic equipment | |
WO2021046715A1 (en) | Exposure time calculation method, device, and storage medium | |
CN107820020A (en) | Method of adjustment, device, storage medium and the mobile terminal of acquisition parameters | |
US20180109711A1 (en) | Method and device for overexposed photography | |
CN103617432A (en) | Method and device for recognizing scenes | |
CN111800568B (en) | Light supplement method and device | |
CN108897786A (en) | Recommended method, device, storage medium and the mobile terminal of application program | |
CN110633665B (en) | Identification method, device and storage medium | |
CN110443766B (en) | Image processing method and device, electronic equipment and readable storage medium | |
CN106651797B (en) | Method and device for determining effective area of signal lamp | |
CN109618102B (en) | Focusing processing method and device, electronic equipment and storage medium | |
WO2022142177A1 (en) | Method and apparatus for generating hdr image, and electronic device and readable storage medium | |
CN111654643B (en) | Exposure parameter determination method and device, unmanned aerial vehicle and computer readable storage medium | |
CN112383772B (en) | Camera performance automatic test method and device, electronic equipment and storage medium | |
CN112261292B (en) | Image acquisition method, terminal, chip and storage medium | |
CN111629146B (en) | Shooting parameter adjusting method, shooting parameter adjusting device, shooting parameter adjusting equipment and storage medium | |
CN113177886B (en) | Image processing method, device, computer equipment and readable storage medium | |
CN108564537B (en) | Image processing method, image processing device, electronic equipment and medium | |
CN110312083B (en) | Self-cruise adjusting device and method for automatic exposure and hardware accelerator | |
CN116506737A (en) | Method, device, equipment and storage medium for determining exposure parameters | |
CN116310662A (en) | Object recognition method and device | |
CN107454340B (en) | Image synthesis method and device based on high dynamic range principle and mobile terminal | |
JP2023078061A (en) | Imaging exposure control method and apparatus, device and storage medium | |
CN114143429B (en) | Image shooting method, device, electronic equipment and computer readable storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |