CN114390211B - Exposure convergence method, device, electronic equipment and storage medium - Google Patents

Exposure convergence method, device, electronic equipment and storage medium Download PDF

Info

Publication number
CN114390211B
CN114390211B CN202011126865.4A CN202011126865A CN114390211B CN 114390211 B CN114390211 B CN 114390211B CN 202011126865 A CN202011126865 A CN 202011126865A CN 114390211 B CN114390211 B CN 114390211B
Authority
CN
China
Prior art keywords
exposure
exposure index
index
convergence
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011126865.4A
Other languages
Chinese (zh)
Other versions
CN114390211A (en
Inventor
张旭
张文婷
孟德珅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Xiaomi Mobile Software Co Ltd
Original Assignee
Beijing Xiaomi Mobile Software Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Xiaomi Mobile Software Co Ltd filed Critical Beijing Xiaomi Mobile Software Co Ltd
Priority to CN202011126865.4A priority Critical patent/CN114390211B/en
Publication of CN114390211A publication Critical patent/CN114390211A/en
Application granted granted Critical
Publication of CN114390211B publication Critical patent/CN114390211B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70Circuitry for compensating brightness variation in the scene
    • H04N23/71Circuitry for evaluating the brightness variation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70Circuitry for compensating brightness variation in the scene
    • H04N23/73Circuitry for compensating brightness variation in the scene by influencing the exposure time

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

The disclosure relates to an exposure convergence method, an exposure convergence device, electronic equipment and a storage medium. Wherein the method is applied to an image signal processor assembled by electronic equipment, and comprises the following steps: when a camera application in the electronic equipment is started, acquiring current ambient light information; determining a target exposure index range corresponding to the current environment light information from a corresponding relation between a preset environment light range and an exposure index range; under the condition that the historical exposure index is in the target exposure index range, performing exposure convergence by taking the historical exposure index as an initial exposure index; wherein, the historical exposure index is: the camera applies a final exposure index that was used in at least one start before the current start. When the historical exposure index is determined to be in the exposure index range corresponding to the current ambient light information, the historical exposure index can be used as the initial exposure index for the current exposure convergence, and the purpose of rapid exposure convergence is achieved.

Description

Exposure convergence method, device, electronic equipment and storage medium
Technical Field
The present disclosure relates to the field of image processing, and in particular, to an exposure convergence method, an apparatus, an electronic device, and a storage medium.
Background
With the continuous development of imaging technology, more and more users are added into the line of photography and video shooting, and even the hot trend of the national photography is created.
In the related art, when a camera application is started, an exposure parameter of the camera application needs to be adjusted according to ambient light information of the environment, so that an acquired picture is exposed normally (i.e., light and dark moderately), and the adjustment process is also called exposure convergence.
At present, although the camera application can accurately determine the final exposure parameters for enabling the picture to be exposed normally according to the ambient light information, the initial exposure parameters adopted when the camera application is started cannot be reasonably set, so that the exposure convergence process duration is longer, and poor use experience is brought to users.
Disclosure of Invention
The disclosure provides an exposure convergence method, an exposure convergence device, an electronic device and a storage medium, which can reasonably determine an initial exposure index when a camera application is started so as to reduce the duration required by exposure convergence and achieve the purpose of rapid exposure convergence.
According to a first aspect of the present disclosure, there is provided an exposure convergence method applied to an image signal processor of an electronic device assembly, comprising:
When a camera application in the electronic equipment is started, acquiring current ambient light information;
determining a target exposure index range corresponding to the current environment light information from a corresponding relation between a preset environment light range and an exposure index range;
under the condition that the historical exposure index is in the target exposure index range, performing exposure convergence by taking the historical exposure index as an initial exposure index;
wherein, the historical exposure index is: the camera applies the final exposure index used in any one of the activations prior to the present activation.
According to a second aspect of the present disclosure, there is provided an exposure convergence device applied to an image signal processor of an electronic equipment assembly, comprising:
the acquisition unit is used for acquiring current ambient light information when a camera application in the electronic equipment is started;
the determining unit is used for determining a target exposure index range corresponding to the current environment light information from the corresponding relation between the preset environment light range and the exposure index range;
a convergence unit that performs exposure convergence with the historical exposure index as an initial exposure index when the historical exposure index is within the target exposure index range;
Wherein, the historical exposure index is: the camera applies the final exposure index used in any one of the activations prior to the present activation.
According to a third aspect of the present disclosure, there is provided an electronic device comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor implements the method of the first aspect by executing the executable instructions.
According to a fourth aspect of the present disclosure there is provided a computer readable storage medium having stored thereon computer instructions which when executed by a processor perform the steps of the method according to the first aspect.
In the technical scheme of the disclosure, the corresponding relation between the environment light range and the exposure index range is preset, so that the disclosure can determine the target exposure index range corresponding to the current environment light information on the basis of acquiring the current environment light information; further judging whether the obtained historical exposure index is in the target exposure index range or not by comparing the obtained historical exposure index with the determined target exposure index range; if yes, directly adopting the historical exposure index as an initial exposure index to carry out the exposure convergence. It is to be understood that, because the historical exposure index is an exposure index that makes the picture under the corresponding historical environmental light tend to be exposed normally, when the historical exposure index is in the range of the target exposure index corresponding to the current environmental light information, the current environmental light is proved to be relatively close to the historical environmental light, and correspondingly, the historical exposure index is relatively close to the target exposure index that makes the picture be exposed normally. Therefore, the historical exposure index is used as the initial exposure index for the current exposure convergence, so that the difference between the initial exposure index and the target exposure index is smaller, and the effect of rapid exposure convergence is achieved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.
FIG. 1 is a flow chart of an exposure convergence method shown in an exemplary embodiment of the present disclosure;
FIG. 2 is a flow chart of another exposure convergence method shown in an exemplary embodiment of the present disclosure;
FIG. 3 is a block diagram of an exposure convergence device shown in an exemplary embodiment of the disclosure;
FIG. 4 is a block diagram of another exposure convergence device shown in an exemplary embodiment of the disclosure;
fig. 5 is a schematic structural diagram of an electronic device in an exemplary embodiment of the present disclosure.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.
The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present disclosure. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.
With the continuous development of imaging technology, most of the current electronic devices already have camera functions, and shooting requirements of users can be met only by installing camera applications on the electronic devices. In the related art, when a camera application (hereinafter simply referred to as a camera for convenience of description) is started, an exposure parameter of the camera needs to be adjusted according to ambient light information of an environment in which the camera is located so that an acquired picture is exposed normally, and the adjustment process is also referred to as exposure convergence.
Since the exposure convergence process involves a large number of exposure parameters, a technician typically converts a plurality of exposure parameters into a unified standard parameter, which is called an exposure index, for describing the picture exposure degree under the corresponding exposure parameters. When the camera is started and performs exposure convergence, in order to avoid a problem that the number of exposure parameters is large, which results in difficulty in determining the value of the exposure parameters used for the exposure convergence, the exposure convergence is generally performed by determining the exposure index.
Currently, most cameras of electronic devices perform exposure convergence by adopting a cold start mode. In this case, when the camera in the electronic device is activated, the camera reads a preset initial exposure index (mostly based on a common ambient light setting), and determines a final exposure index by collecting ambient light information, thereby completing the convergence of the exposure from the initial exposure index to the final exposure index. In the cold start mode, if the preset initial exposure index is greatly different from the determined final exposure index, serious underexposure or overexposure occurs in the initial stage of exposure convergence. It appears to the user that the problem of excessive brightness or darkness of the picture occurs in the first few frames of the camera preview picture. Also, since the difference between the initial exposure index and the final exposure index is large, the duration of the exposure convergence process is long, and the camera preview picture needs a long time to recover to the exposure normal.
In addition to the cold start mode, a hot start camera start mode is also proposed in the related art. In this way, the final exposure index adopted by the current start is stored after each start of the camera, and the final exposure index is used as the initial exposure index for the next exposure convergence when the next camera is started, so as to perform exposure convergence. It is to be understood that when the hot start mode is adopted, if the ambient light at the time of starting the camera is closer to the ambient light at the time of starting the camera next time, the initial exposure index and the final exposure index of the next exposure convergence are closer, the exposure convergence duration is shorter, and the problem of overexposure or underexposure does not occur. However, when the ambient light difference between the two camera starts is large, the difference between the final exposure index used for convergence of the two exposures is large, that is, the difference between the initial exposure index used for convergence of the next exposure and the final exposure index is large, which also causes the problem of picture exposure in the cold start and the problem of long duration of exposure convergence.
Therefore, in the related art, whether the camera is started in a cold or hot mode, there is a high probability that short overexposure or underexposure occurs in the preview screen due to improper initial exposure index setting, and the duration of the exposure convergence process is long.
For this reason, the present disclosure proposes an exposure convergence method to solve the above-described problems. Before describing the technical solution of the present disclosure in detail, it should be emphasized that the camera startup procedure in the present disclosure refers to: the camera application of the electronic device is opened, and the image signal processor performs first exposure convergence so that a preview picture of the camera application tends to be exposed to normal process. After the first exposure convergence, the image signal processor automatically adjusts the exposure parameters according to the user adjustment operation or according to the ambient light, so that the exposure convergence is not considered as the exposure convergence in the starting process of the camera of the present disclosure, but rather as the exposure convergence in the running process of the camera.
Fig. 1 is a flowchart of an exposure convergence method according to an exemplary embodiment of the present disclosure, which is applied to an image signal processor. As shown in fig. 1, the method may include the steps of:
step 102, when a camera application in the electronic device is started, current ambient light information is acquired.
In the present disclosure, the image signal processor may be a component for image signal processing included in any electronic device. The component may be an image signal processor separately provided in the electronic device, or may be an image processing module included in a central processing unit of the electronic device. How to arrange the image signal processor in the electronic device in particular can be determined by a person skilled in the art according to the actual situation, which is not limited by the present disclosure.
Accordingly, the present disclosure is not limited to the electronic device in which the image signal processor is mounted. The electronic equipment can be a mobile terminal, such as a smart phone, a tablet personal computer and the like; the mobile terminal can also be a fixed terminal, such as a PC, an intelligent television and the like; but also professional photographic imaging devices such as digital cameras, video cameras, etc. It should be understood that any electronic device having a photographing function may be used as the electronic device to which the image signal processor is mounted in the present disclosure.
From the above, it is generally simpler to determine the initial exposure index in the exposure convergence process in the related art. When the camera is started in a cold mode, a preset initial exposure index is directly used as an initial exposure index for exposure convergence; and when the camera is started up, the final exposure index adopted in the last time of camera starting is directly used as the initial exposure index for current exposure convergence. The method and the device do not directly perform exposure convergence based on a preset initial exposure index or a last final exposure index, but determine an approximate exposure index range based on current ambient light information preferentially before exposure convergence, and then determine how to determine the initial exposure index according to the determined exposure index range.
It should be appreciated that, since the disclosure determines a rough exposure index range based on current ambient light information before exposure convergence and determines an initial exposure index according to the exposure index range, the determined initial exposure index can be relatively close to a final exposure index for normalizing the image exposure with a relatively high probability, so as to shorten exposure convergence time and avoid short-term improper exposure of the preview image.
In the present disclosure, the acquisition of current ambient light information typically requires the aid of a light sensor. Taking the example of separately providing the image signal processor in the electronic device: when the electronic equipment detects that a user turns on the camera, the central processing unit of the electronic equipment can send an image acquisition signal to the image signal processor; after receiving the image acquisition signal, the image signal processor can send a light acquisition instruction to the light sensor so that the light sensor acquires the current environment light signal and returns the current environment light signal; the image signal processor can generate current ambient light information based on the current ambient light signal after receiving the current ambient light signal returned by the light sensor. Of course, when the image signal processor is not separately set, the interaction process between the central processing unit and the image signal processor is only modified to be the interaction between a plurality of modules in the central processing unit, and the general interaction flow is not changed. It should be noted that the above process of acquiring current ambient light information is merely exemplary, and any manner of acquiring ambient light information may be applied to the present disclosure, which is not limited by the present disclosure.
In the present disclosure, the operation of acquiring the historical exposure index is also included. In an embodiment, the historical exposure index may be the same as the initial exposure index used for the hot start, which is the final exposure index used during the last start of the camera; alternatively, the historical exposure index may include: the camera uses a plurality of final exposure indexes used during a plurality of starts before the start. The historical exposure index may be an exposure index preset by a technician, in addition to the final exposure index employed in the historical exposure convergence process. Of course, the above examples are illustrative only, and the specific meaning of the historical exposure index may be determined by one skilled in the art based on actual circumstances, and the present disclosure is not limited thereto.
In practical applications, the image signal processor may save the final exposure index in the local storage space after the exposure convergence is completed, so that the image signal processor can read the final exposure index from the local storage space when the subsequent camera is started.
Step 104, determining a target exposure index range corresponding to the current ambient light information from a corresponding relation between a preset ambient light range and an exposure index range.
In the present disclosure, a correspondence between an ambient light range and an exposure index range is preconfigured to determine a target exposure index range corresponding to the current ambient light information. Specifically, a plurality of ambient light ranges and preset exposure index ranges corresponding to the respective ambient light ranges may be recorded in the correspondence. For example, the correspondence may be as shown in table 1 below:
ambient light range Exposure index range
0~13107 320~399
13108~26215 240~319
26216~39322 160~239
39323~52429 80~159
52430~65535 0~79
TABLE 1
In actual operation, after determining the current ambient light information, a target ambient light range corresponding to the current ambient light information may be determined from a plurality of ambient light ranges included in the corresponding relationship, and a preset exposure index range corresponding to the target ambient light range may be further determined as a target exposure index range corresponding to the current ambient light information.
Taking the correspondence in table 1 as an example, assuming that the value of the current ambient light information determined by the image signal processor is 14000, the determined target ambient light range is 13108-26215, and the further determined target exposure index range corresponding to the current ambient light information is 240-319.
After determining the target exposure index range corresponding to the current ambient light information, the acquired historical exposure index can be compared with the determined target exposure index range, and whether the historical exposure index is in the target exposure index range is judged.
In one embodiment, the historical exposure index is the final exposure index used by the image signal processor during the last exposure convergence. In this embodiment, it is only necessary to determine whether the historical exposure index is within a certain target exposure index range. Because the historical exposure index is unique, the judgment operation is simpler, and the efficiency is higher.
In another embodiment, the historical exposure index comprises: the camera uses a plurality of final exposure indexes used during a plurality of starts before the start. In this embodiment, the image signal processor may determine whether any final exposure index of the plurality of final exposure indexes belongs to the determined target exposure index range after reading the plurality of final exposure indexes. When it is determined that any final exposure index exists and belongs to the determined target exposure index range, the any final exposure index can be used as an initial exposure index for convergence of the current exposure. Compared with the previous embodiment, since the historical exposure index of the present embodiment includes a plurality of final exposure indexes, the probability that the historical exposure index can be used as the initial exposure index for the current exposure convergence is improved.
It should be understood that since the historical exposure index in the present embodiment includes a plurality of final exposure indexes, it is likely that a case where the plurality of final exposure indexes all belong to a certain target exposure index range occurs. For this case, a person skilled in the art may determine the final exposure index, which is the initial exposure index for the current exposure convergence, in different ways according to actual needs, which is not limited by the present disclosure. For example, the following two ways may be adopted:
in one mode, the image signal processor may compare the plurality of final exposure indexes with the determined target exposure index range one by one after acquiring the plurality of final exposure indexes, and once it is determined that any final exposure index is within the target exposure index range, stop the comparison process immediately, and use any final exposure index as the initial exposure index for convergence of the present exposure. In this way, when there are a plurality of final exposure indexes within the determined target exposure index range, it is not necessary to compare all the final exposure indexes contained in the history exposure index with the target exposure index range, and occupation of processing resources of the image signal processor by the comparison process is reduced.
In another manner, the image signal processor may still compare the plurality of final exposure indexes with the determined target exposure index range one by one after acquiring the plurality of final exposure indexes, but compare all final exposure indexes included in the historical exposure indexes with the determined target exposure index range to determine all final exposure indexes within the target exposure index range; on the basis, any determined final exposure index can be used as an initial exposure index for the current exposure convergence. Further, an intermediate value of the target exposure index range may be determined, and all the determined final exposure indexes are differed from the intermediate value, so that the final exposure index with the smallest difference from the intermediate value is used as the initial exposure index for convergence of the present exposure, so as to further improve the accuracy of the determined initial exposure index.
Step 106, under the condition that the historical exposure index is within the target exposure index range, performing exposure convergence by taking the historical exposure index as an initial exposure index; wherein, the historical exposure index is: the camera applies the final exposure index used in any one of the activations prior to the present activation.
It should be understood that, since the determined target exposure index range corresponds to the obtained current ambient light information, when the historical exposure index is within the determined target exposure index range, it can be proved that the current ambient light is closer to the historical ambient light where the image signal processor uses the historical exposure index. Obviously, the historical exposure index is also relatively close to the target exposure index determined based on the current ambient light information (i.e. the final exposure index adopted for exposure convergence in the current camera starting process). It can be seen that if the historical exposure index is determined as the initial exposure index for the current exposure convergence, the duration of the exposure convergence can be greatly reduced.
In addition, since the purpose of determining the target exposure index is to: the exposure of the preview picture is made to be normal. Obviously, when the historical exposure index is close to the target exposure index, the problem of overexposure or underexposure does not occur in the initial stage of the current exposure convergence.
It should be stated that exposure convergence in the present disclosure refers to: when the camera is started, the initial exposure index is adjusted to the target exposure index, so that the camera previews the picture to be in a normal exposure process. The process is a dynamic adjustment process. In practical application, after the image signal processor determines the target exposure index based on the current ambient light information, the actual exposure convergence process is as follows: the historical exposure index, which is the initial exposure index, is converged to the target exposure index.
In addition, since the exposure index is only one standard parameter for representing a plurality of exposure parameters. The image signal processor cannot make picture adjustment based on the exposure index in actual operation. Therefore, in practical application, each exposure index corresponds to at least one group of exposure parameters, so that after determining the initial exposure index and the target exposure index adopted by the current exposure convergence, the initial exposure parameters corresponding to the initial exposure index and the target exposure parameters corresponding to the target exposure index are further determined, and then the exposure convergence process is completed by adjusting the exposure parameters of the picture.
It should be understood that the exposure index, as a standard parameter representing a plurality of exposure parameters, characterizes the final exposure condition of the picture. In practical cases, the multiple exposure parameters can be combined by different values to present similar picture exposure conditions. Therefore, when the same exposure index corresponds to multiple groups of exposure parameters, the picture exposure conditions represented by the multiple groups of exposure parameters are basically consistent, and a person skilled in the art can determine any one group of exposure parameters corresponding to exposure convergence according to actual requirements, for example, the finally adopted exposure parameters can be determined in view of the influence of the exposure parameters on other picture dimensions such as depth of field, frame rate and the like.
In the present disclosure, the exposure parameters corresponding to the exposure index may include: at least one of exposure time, exposure gain, aperture, shutter, and sensitivity. The exposure time length and the exposure gain are mostly used as exposure parameters corresponding to the exposure index on mobile terminals such as smart phones; in many professional photographic imaging apparatuses such as digital cameras and video cameras, an aperture, a shutter, and a sensitivity are used as exposure parameters corresponding to exposure indexes. The exposure parameters corresponding to the exposure index may be determined by those skilled in the art according to actual situations, which is not limited in the present disclosure.
In the present disclosure, when it is determined that the historical exposure index is not within the determined target exposure index range, it may be determined that: when the historical exposure index is adopted, the environmental light of the image signal processor is greatly different from the current environmental light, if the historical exposure index is still used as the initial exposure index adopted by the exposure convergence, the problems that the exposure convergence time is too long and the exposure is unreasonable in the preview picture at the initial stage of the exposure convergence are likely to occur when the hot start is adopted in the related technology. Therefore, in this case, the history exposure index is no longer used as the initial exposure index for the current exposure convergence.
In this case, a preset initial exposure index corresponding to the current ambient light information may be further determined from the correspondence, so as to determine the preset initial exposure index as an initial exposure index adopted for the current exposure convergence. Specifically, the preset initial exposure index corresponding to each ambient light range may be further preset in the corresponding relationship. For example, it may be as shown in table 2:
ambient light range Exposure index range Presetting an initial exposure index
0~13107 320~399 360
13108~26215 240~319 280
26216~39322 160~239 200
39323~52429 80~159 120
52430~65535 0~79 40
TABLE 2
In actual operation, after determining a target ambient light range corresponding to the current ambient light information from a plurality of ambient light ranges included in the correspondence, a preset initial exposure index corresponding to the target ambient light range may be further determined as: and taking the preset initial exposure index corresponding to the current ambient light information as the initial exposure index for convergence of the current exposure.
It should be appreciated that the predetermined initial exposure index determined corresponds to the current ambient light information and the target exposure index is also determined based on the current ambient light information. In other words, the determined preset initial exposure index and the final determined target exposure index should be relatively close, so that if the preset initial exposure index is used as the initial exposure index for the current exposure convergence, the duration of exposure convergence can be reduced, the purpose of rapid exposure convergence can be achieved, and the problem that the preview picture is overexposed or underexposed in the initial exposure convergence stage can not occur.
In the present disclosure, the target exposure index may also be set manually by the user, in which case, in the case where the electronic device detects an exposure adjustment operation by the user, an exposure convergence instruction may be sent to the image signal processor; so that the image signal processor converges the determined historical exposure index to the target exposure index after reading the target exposure index contained in the exposure convergence instruction. By setting the target exposure index by the user, the exposure convergence process is controllable for the user, and the problem of inaccurate picture exposure caused by inaccurate automatic target exposure index determination by the equipment is avoided.
According to the technical scheme, the corresponding relation between the environment light range and the exposure index range is preset, so that the exposure index range corresponding to the current environment light information can be determined on the basis of acquiring the current environment light information; and further determining whether the historical exposure index is within the exposure index range by comparing the obtained historical exposure index with the determined exposure index range. If yes, directly adopting the historical exposure index as an initial exposure index to perform exposure convergence in the current camera starting process. It is to be understood that, because the historical exposure index is an exposure index that makes the picture under the corresponding historical environmental light tend to be exposed normally, when the historical exposure index is in the exposure index range corresponding to the current environmental light information, the current environmental light is proved to be relatively close to the historical environmental light, and correspondingly, the historical exposure index is relatively close to the final exposure index that makes the picture exposed normally. Therefore, the historical exposure index is used as the initial exposure index for the current exposure convergence, so that the difference between the initial exposure index and the target exposure index is smaller, and the effect of rapid exposure convergence is achieved; meanwhile, as the current ambient light is relatively close to the historical ambient light, when the historical exposure index is used as the initial exposure index, the problem of overexposure or underexposure of a preview picture in the related technology cannot occur at the initial stage (or the previous frames) of the current exposure convergence, and the visual experience of a user is improved.
Further, in the corresponding relation, corresponding preset initial exposure indexes are further configured for each environment light range, so that when the historical exposure index is not in the determined target exposure index range, the preset initial exposure index corresponding to the current environment light information can be used as the initial exposure index for current exposure convergence, and the determined target exposure index is relatively close to the determined preset initial exposure index because the target exposure index is determined based on the current environment light information. Therefore, when the historical exposure index is not within the determined target exposure index range, the method and the device can still realize rapid exposure convergence based on the preset initial exposure index configured in the corresponding relation, and meanwhile avoid the problems of overexposure or underexposure of a preview picture in the initial exposure convergence stage in the related technology.
For convenience of understanding, the technical scheme of the disclosure is described below by taking exposure convergence of the smart phone in a camera opening process as an example.
Fig. 2 is a flowchart of another exposure convergence method according to an exemplary embodiment of the present disclosure, which is applied to an image signal processor in a smart phone. As shown in fig. 2, the method may include the steps of:
Step 201, receiving an image acquisition instruction sent by a central processing unit.
In this embodiment, the smart phone may install a camera application in advance to take a photograph or take a video through the smart phone.
On the basis, the central processor of the smart phone can send an image acquisition signal to the image signal processor when detecting the click operation of the icon of the camera application by the user to start the camera application; the image signal processor can respond to the image acquisition signal and instruct the light sensor to acquire the current ambient light.
Step 202, sending a current ambient light collection instruction to a light sensor.
And 203, generating current ambient light information based on the current ambient light signals acquired by the light sensor.
In actual operation, the ambient light signal is collected by the light sensor, and the image signal processor needs to convert the ambient light signal into ambient light information after receiving the ambient light signal for further processing. Specifically, any conversion mode capable of realizing the "light signal to light information" may be adopted, which is not limited in the present disclosure.
Step 204, determining a target exposure index range and a target initial exposure index corresponding to the current ambient light information from the exposure index lookup table.
In this embodiment, the correspondence relationship described above is stored in the form of a table, which may be named as an exposure index lookup table.
Continuing with the example of table 2 above, assuming that the value of the determined current ambient light information is 14000, then the determined target ambient light range is 13108-26215, the determined target exposure index range corresponding to the current ambient light information is 240-319, and the preset initial exposure index corresponding to the current ambient light information is 280.
Step 205, a pre-stored historical exposure index is read.
Step 206, judging whether the historical exposure index is in the determined target exposure index range; if yes, go to step 207A, otherwise, go to step 207B.
In the present embodiment, the history exposure index is taken as an example of the final exposure index used in the previous exposure convergence process.
In step 207A, the exposure convergence is performed using the historical exposure index as the initial exposure index for the current exposure convergence.
Assuming that the read historical exposure index is 300, it is obvious that 300 is in the range of 240-319, i.e. the historical exposure index is in the range of the determined target exposure index, and then the historical exposure index of 300 can be used as the initial exposure index of the current exposure convergence to perform exposure convergence.
Obviously, because the historical exposure index is in the range of the determined target exposure index, the last environmental light is closer to the current environmental light, and the finally determined target exposure index converged by the current exposure is closer to the historical exposure index. For example, it may be further assumed that the determined target exposure index is 310, that is, when the camera is started, the exposure parameter of the first frame in the current exposure convergence process is set as: the exposure parameter corresponding to the exposure index 300 is used as the exposure parameter of the last frame of picture in the current exposure convergence process to perform exposure convergence, and the exposure parameter corresponding to the exposure index 310 is used as the exposure parameter of the last frame of picture in the current exposure convergence process.
In step 207B, the target initial exposure index is used as the initial exposure index for the current exposure convergence to perform exposure convergence.
Assuming that the read historical exposure index is 380, it is obvious that 380 is not in the range of 240-319, i.e. the historical exposure index is not in the range of the determined target exposure index, and then the 380 historical exposure index cannot be used as the initial exposure index of the current exposure convergence to perform exposure convergence, but the 280 determined target initial exposure index should be used as the initial exposure index of the current exposure convergence to perform exposure convergence
Obviously, because the historical exposure index is not in the range of the determined target exposure index, the difference between the environmental light where the last exposure is converged and the environmental light where the current exposure is converged is larger, and the finally determined target exposure index of the current exposure is also larger than the historical exposure index. However, since the current environmental light information is within the range of the target environmental light information 13108 to 26215, accordingly, the target exposure index finally determined based on the current environmental light information should also be within the corresponding target exposure index range 240 to 319. Still assuming that the determined target exposure index is 310, then when the camera is started, the exposure parameter of the first frame of picture in the current exposure convergence process will not be set as follows in a hot start mode: exposure parameters corresponding to the historical exposure index 380. But the exposure parameter of the first frame of picture in the current exposure convergence process is set as follows: and taking the exposure parameter corresponding to the exposure index 280 as the exposure parameter of the last frame of picture in the current exposure convergence process to carry out exposure convergence, wherein the exposure parameter corresponds to the exposure index 310. Obviously, compared with a hot start mode, the difference between the target exposure index and the initial exposure index can be greatly reduced, and the aim of rapid exposure convergence is achieved.
According to the technical scheme, whether the historical exposure index is in the exposure index range corresponding to the current environment light information or not can be determined, so that the purpose of rapid exposure convergence can be achieved.
Fig. 3 is a block diagram of an exposure convergence device according to an exemplary embodiment of the present disclosure. Referring to fig. 3, the apparatus includes an acquisition unit 301, a determination unit 302, and a convergence unit 303.
The acquisition unit 301 is configured to acquire current ambient light information when a camera application in the electronic device is started;
the determining unit 302 is configured to determine a target exposure index range corresponding to the current ambient light information from a corresponding relationship between a preset ambient light range and an exposure index range;
the convergence unit 303 performs exposure convergence with the history exposure index as an initial exposure index when the history exposure index is within the target exposure index range;
wherein, the historical exposure index is: the camera applies the final exposure index used in any one of the activations prior to the present activation.
Optionally, the determining unit 302 is further configured to:
determining a target ambient light range to which the current ambient light information belongs from a plurality of ambient light ranges contained in the corresponding relation;
and determining a preset exposure index range corresponding to the target environment light range as a target exposure index range.
Optionally, the historical exposure index is a final exposure index used in a last starting process of the camera application; or,
the historical exposure index includes: the camera uses a plurality of final exposure indexes used in a plurality of starting processes before the current starting.
As shown in fig. 4, fig. 4 is a block diagram of another exposure converging apparatus according to an exemplary embodiment of the present disclosure, which further includes, on the basis of the foregoing embodiment shown in fig. 3: a judgment unit 304, a first exponent determining unit 305, a second exponent determining unit 306, and a receiving unit 307.
Optionally, the method further comprises:
the judging unit 304 is configured to read the plurality of final exposure indexes; judging whether any final exposure index in the multiple final exposure indexes belongs to the target exposure index range;
Wherein, in the case where it is determined that there is any final exposure index belonging to the target exposure index range, the any final exposure index is taken as the initial exposure index.
Alternatively to this, the method may comprise,
the first index determining unit 305 is configured to determine a preset initial exposure index corresponding to the current ambient light information from the correspondence relationship; and under the condition that the historical exposure index is not in the target exposure index range, taking the determined preset initial exposure index as the initial exposure index for the current exposure convergence.
Alternatively to this, the method may comprise,
the second index determining unit 306 is configured to determine a target exposure index of the current exposure convergence based on the current ambient light information;
the convergence unit 303 is further equipped to: and converging the historical exposure index to the target exposure index.
Alternatively to this, the method may comprise,
the receiving unit 307 configured to receive an exposure convergence instruction transmitted by the electronic apparatus in a case where an exposure adjustment operation by a user is detected;
the convergence unit 303 is further configured to read a target exposure index included in the exposure convergence instruction and converge the historical exposure index to the target exposure index.
For the device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the objectives of the disclosed solution. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
Correspondingly, the disclosure also provides a signal processing device, which comprises: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement an exposure convergence method according to any of the above embodiments, for example the method may comprise: when a camera application in the electronic equipment is started, acquiring current ambient light information; determining a target exposure index range corresponding to the current environment light information from a corresponding relation between a preset environment light range and an exposure index range; under the condition that the historical exposure index is in the target exposure index range, performing exposure convergence by taking the historical exposure index as an initial exposure index; wherein, the historical exposure index is: the camera applies the final exposure index used in any one of the activations prior to the present activation.
Accordingly, the present disclosure also provides an electronic device including a memory, and one or more programs, where the one or more programs are stored in the memory, and configured to be executed by the one or more processors, the one or more programs include instructions for implementing the exposure convergence method according to any of the foregoing embodiments, for example, the method may include: when a camera application in the electronic equipment is started, acquiring current ambient light information; determining a target exposure index range corresponding to the current environment light information from a corresponding relation between a preset environment light range and an exposure index range; under the condition that the historical exposure index is in the target exposure index range, performing exposure convergence by taking the historical exposure index as an initial exposure index; wherein, the historical exposure index is: the camera applies the final exposure index used in any one of the activations prior to the present activation.
Fig. 5 is a block diagram illustrating an apparatus 500 for achieving exposure convergence according to an example embodiment. For example, the apparatus 500 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, or the like.
Referring to fig. 5, an apparatus 500 may include one or more of the following components: a processing component 502, a memory 504, a power supply component 506, a multimedia component 508, an audio component 510, an input/output (I/O) interface 512, a sensor component 514, and a communication component 516.
The processing component 502 generally controls overall operation of the apparatus 500, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 502 may include one or more processors 520 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 502 can include one or more modules that facilitate interactions between the processing component 502 and other components. For example, the processing component 502 can include a multimedia module to facilitate interaction between the multimedia component 508 and the processing component 502.
The memory 504 is configured to store various types of data to support operations at the apparatus 500. Examples of such data include instructions for any application or method operating on the apparatus 500, contact data, phonebook data, messages, pictures, videos, and the like. The memory 504 may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
The power supply component 506 provides power to the various components of the device 500. The power components 506 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the device 500.
The multimedia component 508 includes a screen between the device 500 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 508 includes a front-facing camera and/or a rear-facing camera. The front-facing camera and/or the rear-facing camera may receive external multimedia data when the apparatus 500 is in an operational mode, such as a photographing mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
The audio component 510 is configured to output and/or input audio signals. For example, the audio component 510 includes a Microphone (MIC) configured to receive external audio signals when the device 500 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 504 or transmitted via the communication component 516. In some embodiments, the audio component 510 further comprises a speaker for outputting audio signals.
The I/O interface 512 provides an interface between the processing component 502 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.
The sensor assembly 514 includes one or more sensors for providing status assessment of various aspects of the apparatus 500. For example, the sensor assembly 514 may detect the on/off state of the device 500, the relative positioning of the components, such as the display and keypad of the device 500, the sensor assembly 514 may also detect a change in position of the device 500 or a component of the device 500, the presence or absence of user contact with the device 500, the orientation or acceleration/deceleration of the device 500, and a change in temperature of the device 500. The sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 514 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 514 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 516 is configured to facilitate communication between the apparatus 500 and other devices in a wired or wireless manner. The apparatus 500 may access a wireless network based on a communication standard, such as WiFi,2G or 3G,4G LTE, 5G NR (New Radio), or a combination thereof. In one exemplary embodiment, the communication component 516 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 516 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the apparatus 500 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements for executing the methods described above.
In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 504, including instructions executable by processor 520 of apparatus 500 to perform the above-described method. For example, the non-transitory computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any adaptations, uses, or adaptations of the disclosure following the general principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
The foregoing description of the preferred embodiments of the present disclosure is not intended to limit the disclosure, but rather to cover all modifications, equivalents, improvements and alternatives falling within the spirit and principles of the present disclosure.

Claims (8)

1. An exposure convergence method, characterized by being applied to an image signal processor of an electronic equipment assembly, comprising:
When a camera application in the electronic equipment is started, acquiring current ambient light information;
determining a target exposure index range corresponding to the current environment light information from a corresponding relation between a preset environment light range and an exposure index range;
the final exposure index contained in the history exposure index in the target exposure index range is differed from the intermediate value of the target exposure index range, and the final exposure index with the smallest difference from the intermediate value is used as an initial exposure index for exposure convergence;
wherein, the historical exposure index is: the camera uses a plurality of final exposure indexes used in a plurality of starting processes before the current starting.
2. The method of claim 1, wherein determining the target exposure index range corresponding to the current ambient light information from a preset correspondence between ambient light ranges and exposure index ranges comprises:
determining a target ambient light range to which the current ambient light information belongs from a plurality of ambient light ranges contained in the corresponding relation;
and determining a preset exposure index range corresponding to the target environment light range as a target exposure index range.
3. The method as recited in claim 1, further comprising:
determining a preset initial exposure index corresponding to the current ambient light information from the corresponding relation;
and under the condition that the historical exposure index is not in the target exposure index range, taking the determined preset initial exposure index as the initial exposure index for the current exposure convergence.
4. The method of claim 1, wherein the step of determining the position of the substrate comprises,
further comprises: determining a target exposure index of the current exposure convergence based on the current ambient light information;
the step of performing exposure convergence by taking the historical exposure index as an initial exposure index comprises the following steps: and converging the historical exposure index to the target exposure index.
5. The method of claim 1, wherein the step of determining the position of the substrate comprises,
further comprises: receiving an exposure convergence instruction sent by the electronic equipment under the condition that the exposure adjustment operation of a user is detected;
the step of performing exposure convergence by taking the historical exposure index as an initial exposure index comprises the following steps: and reading a target exposure index contained in the exposure convergence instruction, and converging the historical exposure index to the target exposure index.
6. An exposure convergence device, characterized by being applied to an image signal processor of an electronic equipment assembly, comprising:
the acquisition unit is used for acquiring current ambient light information when a camera application in the electronic equipment is started;
the determining unit is used for determining a target exposure index range corresponding to the current environment light information from the corresponding relation between the preset environment light range and the exposure index range;
a convergence unit that makes a difference between a final exposure index included in the history exposure index within the target exposure index range and a middle value of the target exposure index range, and performs exposure convergence with a final exposure index having a smallest difference from the middle value as an initial exposure index;
wherein, the historical exposure index is: the camera uses a plurality of final exposure indexes used in a plurality of starting processes before the current starting.
7. An electronic device, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to implement the method of any of claims 1-5 by executing the executable instructions.
8. A computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the steps of the method according to any of claims 1-5.
CN202011126865.4A 2020-10-20 2020-10-20 Exposure convergence method, device, electronic equipment and storage medium Active CN114390211B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011126865.4A CN114390211B (en) 2020-10-20 2020-10-20 Exposure convergence method, device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011126865.4A CN114390211B (en) 2020-10-20 2020-10-20 Exposure convergence method, device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN114390211A CN114390211A (en) 2022-04-22
CN114390211B true CN114390211B (en) 2024-03-22

Family

ID=81193071

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011126865.4A Active CN114390211B (en) 2020-10-20 2020-10-20 Exposure convergence method, device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN114390211B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008148180A (en) * 2006-12-13 2008-06-26 Sony Corp Imaging apparatus, image processing device and method, and computer program
WO2017177693A1 (en) * 2016-04-15 2017-10-19 惠州Tcl移动通信有限公司 Method and system for controlling brightness when starting camera, and shooting terminal
CN108419025A (en) * 2018-05-30 2018-08-17 北京图森未来科技有限公司 A kind of exposure parameter method of adjustment, device and readable medium
CN109413336A (en) * 2018-12-27 2019-03-01 北京旷视科技有限公司 Image pickup method, device, electronic equipment and computer readable storage medium
CN109474790A (en) * 2018-11-05 2019-03-15 浙江大华技术股份有限公司 Exposure adjustment method, device and video camera and computer storage medium
CN109889731A (en) * 2019-02-22 2019-06-14 生迪智慧科技有限公司 The determination method, apparatus and storage medium of acquisition parameters
CN111435971A (en) * 2019-01-15 2020-07-21 阿里巴巴集团控股有限公司 Image acquisition parameter configuration method and device, terminal and readable storage medium

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105450936B (en) * 2014-05-30 2019-02-15 杭州海康威视数字技术股份有限公司 Intelligent adjusting method and its device when video camera automatic exposure

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008148180A (en) * 2006-12-13 2008-06-26 Sony Corp Imaging apparatus, image processing device and method, and computer program
WO2017177693A1 (en) * 2016-04-15 2017-10-19 惠州Tcl移动通信有限公司 Method and system for controlling brightness when starting camera, and shooting terminal
CN108419025A (en) * 2018-05-30 2018-08-17 北京图森未来科技有限公司 A kind of exposure parameter method of adjustment, device and readable medium
CN109474790A (en) * 2018-11-05 2019-03-15 浙江大华技术股份有限公司 Exposure adjustment method, device and video camera and computer storage medium
CN109413336A (en) * 2018-12-27 2019-03-01 北京旷视科技有限公司 Image pickup method, device, electronic equipment and computer readable storage medium
CN111435971A (en) * 2019-01-15 2020-07-21 阿里巴巴集团控股有限公司 Image acquisition parameter configuration method and device, terminal and readable storage medium
CN109889731A (en) * 2019-02-22 2019-06-14 生迪智慧科技有限公司 The determination method, apparatus and storage medium of acquisition parameters

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Neeru Narang et al.Gender and ethnicity classification using deep learning in heterogeneous face recognition.《2016 International Conference on Biometrics (ICB)》.2016,全文. *
一种自适应快速曝光控制方法;王雷斌;张爱武;叶泽田;侯明辉;曲文乾;;测绘科学;20090320(02);全文 *
基于图像处理的摄像机自动对焦和自动曝光技术的研究;李龙龙;《中国优秀硕士论文电子期刊网》;20160315;全文 *

Also Published As

Publication number Publication date
CN114390211A (en) 2022-04-22

Similar Documents

Publication Publication Date Title
US10674088B2 (en) Method and device for acquiring image, terminal and computer-readable storage medium
CN110796988B (en) Backlight adjusting method and device
KR101800101B1 (en) Method, apparatus, program, and recording medium for adjustmenting shooting parameter
US9661223B2 (en) Method and device for photographing including camera shake
CN107463052B (en) Shooting exposure method and device
CN108040204B (en) Image shooting method and device based on multiple cameras and storage medium
CN111586280B (en) Shooting method, shooting device, terminal and readable storage medium
CN108830194B (en) Biological feature recognition method and device
CN114390211B (en) Exposure convergence method, device, electronic equipment and storage medium
CN106775246B (en) Screen brightness adjusting method and device
CN111277754B (en) Mobile terminal shooting method and device
CN110874829B (en) Image processing method and device, electronic device and storage medium
CN111698414B (en) Image signal processing method and device, electronic device and readable storage medium
CN107707819B (en) Image shooting method, device and storage medium
CN107682623B (en) Photographing method and device
CN111625212A (en) Method and device for controlling screen-off of display screen of terminal equipment and storage medium
CN114339017B (en) Distant view focusing method, device and storage medium
CN116567411A (en) Power consumption control method, device, equipment and medium
CN111385486B (en) Control method and device of image acquisition equipment
CN117354413A (en) Display control method, display control device, electronic equipment and readable storage medium
CN116546314A (en) Power consumption control method, device, equipment and medium
CN116634266A (en) Method, device and storage medium for adjusting power consumption of camera
CN116366968A (en) Image shooting method and device, electronic equipment and storage medium
CN116489748A (en) Control method, control device, terminal and storage medium
CN117278849A (en) Focusing method, focusing device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant