CN106161967B

CN106161967B - Backlight scene panoramic shooting method and mobile terminal

Info

Publication number: CN106161967B
Application number: CN201610821714.8A
Authority: CN
Inventors: 胡鹏翔
Original assignee: Vivo Mobile Communication Co Ltd
Current assignee: Vivo Mobile Communication Co Ltd
Priority date: 2016-09-13
Filing date: 2016-09-13
Publication date: 2020-03-17
Anticipated expiration: 2036-09-13
Also published as: CN106161967A

Abstract

The invention provides a backlight scene panoramic shooting method and a mobile terminal, wherein the method comprises the following steps: receiving a panoramic shooting instruction; for each frame of initial preview image collected by a camera, detecting whether a shooting scene of the frame of initial preview image is a backlight scene; when the shooting scene of the initial preview image is a backlight scene, performing high dynamic range processing on the initial preview image to generate an intermediate preview image; and splicing the initial preview images acquired by the camera under the non-backlight scene and the processed intermediate preview images according to the acquisition sequence to generate a target panoramic image. According to the embodiment of the invention, each frame of image in a backlight scene is subjected to high dynamic range processing, and then the processed image and the initial preview image in a non-backlight scene are spliced according to the sequence of acquisition to obtain the panoramic image without high-brightness range exposure, so that the details of bright and dark places are clear and visible, and the panoramic shooting effect is improved.

Description

Backlight scene panoramic shooting method and mobile terminal

Technical Field

The invention relates to the technical field of communication, in particular to a backlight scene panoramic shooting method and a mobile terminal.

Background

The current panoramic synthesis method generally performs simple splicing on pictures, but if the method still performs simple picture synthesis in some special scenes, such as a dark environment and a backlight scene, the effect of the obtained panoramic picture is greatly reduced. If the panoramic photo is in a dark environment, the whole panoramic photo is darker; if the scene is a backlight scene, the panoramic photo is too dark in a dark place and too bright in a bright place. At present, when panoramic shooting is carried out in a dark environment, panoramic synthesis can be carried out after brightening and denoising treatment.

However, the current technology cannot solve the problem of panoramic shooting in a backlight scene. When shooting is carried out in a backlight scene, the situation that a part of pictures are overexposed, no details exist and a piece of pictures are white can occur; it may also happen that there are some details in the bright place, but too dark in the dark, no details at all visible, a piece of black. Panoramic pictures obtained by panoramic synthesis based on the photos without details and with poor quality have poor effects, and the quality of the pictures and the experience of viewing the pictures are influenced.

Disclosure of Invention

The embodiment of the invention provides a backlight scene panoramic shooting method and a mobile terminal, and aims to solve the problem that in the prior art, the quality of a picture is influenced because an overexposed or excessively dark scene appears in a panoramic image shot in a backlight scene.

In a first aspect, an embodiment of the present invention provides a method for panoramic shooting of a backlighted scene, which is applied to a mobile terminal having a camera, where the method includes:

receiving a panoramic shooting instruction;

for each frame of initial preview image collected by a camera, detecting whether a shooting scene corresponding to the frame of initial preview image is a backlight scene;

when the shooting scene corresponding to the initial preview image is detected to be a backlight scene, performing high dynamic range processing on the initial preview image to generate a middle preview image;

and carrying out image splicing on all initial preview images acquired by the camera under a non-backlight scene and all intermediate preview images acquired by the camera under a backlight scene according to the sequence of acquisition to generate a target panoramic image.

In a second aspect, an embodiment of the present invention further provides a mobile terminal, including a camera, where the mobile terminal further includes:

the receiving module is used for receiving a panoramic shooting instruction;

the detection module is used for detecting whether a shooting scene corresponding to each frame of initial preview image acquired by the camera is a backlight scene or not;

the processing module is used for performing high dynamic range processing on the initial preview image to generate a middle preview image when the detection module detects that the shooting scene corresponding to the initial preview image is a backlight scene;

and the generating module is used for splicing all the initial preview images acquired by the camera under the non-backlight scene and the intermediate preview images processed by the processing module under the backlight scene according to the acquisition sequence to generate a target panoramic image.

In this way, in the embodiment of the present invention, whether the current frame initial preview image is in a backlight scene is detected for each frame initial preview image during panoramic shooting, and when the current frame initial preview image is in the backlight scene, high dynamic range processing is performed for the current frame initial preview image to generate a middle preview image; the obtained intermediate preview image and the initial preview image in the non-backlight scene are spliced according to the image acquisition sequence to obtain the panoramic image, so that the obtained panoramic image does not have the condition of high-brightness range exposure, the details of bright and dark positions are clear and visible, the panoramic shooting effect and the panoramic shooting experience are improved, and the problem that the picture quality is influenced due to the fact that the panoramic image shot in the backlight scene in the prior art is over-exposed or over-dark is solved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive exercise.

Fig. 1 is a schematic diagram illustrating a panoramic shooting method for a backlit scene according to a first embodiment of the present invention;

fig. 2 is a schematic diagram illustrating a panoramic shooting method for a backlit scene according to a second embodiment of the present invention;

fig. 3 is a schematic diagram of a mobile terminal according to a third embodiment of the present invention;

fig. 4 is a schematic diagram of a mobile terminal according to a third embodiment of the present invention;

fig. 5 is a schematic diagram of a mobile terminal according to a third embodiment of the present invention;

fig. 6 is a schematic diagram of a mobile terminal according to a third embodiment of the present invention;

fig. 7 is a block diagram of a mobile terminal according to a fourth embodiment of the present invention;

fig. 8 is a block diagram of a mobile terminal according to a fifth embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Example one

As shown in fig. 1, a method for panoramic shooting in a backlit scene according to a first embodiment of the present invention is applied to a mobile terminal having a camera, and includes:

step 101, receiving a panoramic shooting instruction.

Firstly, a mobile terminal needs to receive a panoramic shooting instruction of a user, enters a panoramic shooting mode according to the panoramic shooting instruction, and utilizes a camera to collect images for panoramic shooting. After entering the panorama shooting mode, step 102 needs to be performed.

And 102, detecting whether a shooting scene corresponding to each frame of initial preview image acquired by the camera is a backlight scene or not.

After entering the panoramic shooting mode, the mobile terminal needs to detect whether a shooting scene of each frame of initial preview image collected by the camera is a backlight scene in the shooting process. When detecting whether a shooting scene of a current frame initial preview image is a backlight scene, acquiring gray information of the current frame initial preview image, forming a gray histogram according to the gray information of the current frame initial preview image, and dividing the gray histogram into N areas which are sequentially arranged and respectively comprise a corresponding number of gray values according to the ascending order of the gray values; wherein the number of gray scale values in each region may or may not be equal. In the gray histogram, each gray value corresponds to the number of corresponding pixel points, and the gray value information of the gray histogram is 0-255. And then determining whether the shooting scene of the current frame initial preview image is a backlight scene according to the gray information of the N areas, wherein N is an integer greater than or equal to 3.

And the boundary gray values of two adjacent areas are adjacent, i.e. the last boundary gray value of the previous area is adjacent to the start boundary gray value of the next area. For example: the gray level histogram is divided into three areas, wherein the gray level value of the first area is 0-36, the gray level value of the second area is 37-178, and the gray level value of the third area is 179-255.

And 103, when the shooting scene corresponding to the initial preview image is detected to be a backlight scene, performing high dynamic range processing on the initial preview image to generate a middle preview image.

After the shooting scene of the current frame initial preview image is determined to be a backlight scene, high dynamic range processing needs to be carried out on the current frame initial preview image, and the specific processing mode is that an overexposure preview image, an underexposure preview image and a normally exposed preview image are determined for the current frame initial preview image, and synthesis processing is carried out according to the three images to obtain an intermediate preview image.

It should be noted that, when obtaining preview images with different exposure parameters, if the current mobile terminal only includes one camera, corresponding processing needs to be performed on an overexposed region and an underexposed region of the current initial preview image. When the mobile terminal comprises two cameras, one camera can be used for outputting an overexposure preview image, and the other camera outputs a normal exposure preview image and an underexposure preview image. When the mobile terminal comprises three cameras, one camera can be used for outputting an overexposure preview image, the other camera outputs a normal exposure preview image, and the rest camera outputs an underexposure preview image.

And 104, splicing all the initial preview images acquired by the camera under the non-backlight scene and the processed intermediate preview images acquired by the camera under the backlight scene according to the acquisition sequence to generate a target panoramic image.

And after the processed intermediate preview images are obtained, carrying out panoramic stitching on all the initial preview images in the non-backlight scene and the processed intermediate preview images in the backlight scene according to the image acquisition sequence.

When performing panoramic shooting, detecting each frame of initial preview image, and detecting whether a shooting scene corresponding to the frame of initial preview image is a backlight scene, when the shooting scene of the frame of initial preview image is the backlight scene, processing the frame of initial image to generate an intermediate preview image, when performing panoramic stitching, discarding the unprocessed initial preview image in the backlight scene, and stitching by using the processed intermediate preview image in the backlight scene. When each frame of initial preview image is detected, and whether a shooting scene corresponding to the frame of initial preview image is a backlight scene is detected, if the shooting scene of the frame of initial preview image is a non-backlight scene, the frame of initial preview image can be adopted for panoramic stitching. Specifically, when each frame of image of the panoramic image is an image shot in a backlit scene, each frame of image needs to be processed, and each frame of image in the obtained panoramic image is a processed intermediate preview image.

If a partial image of the panoramic image is an image shot in a backlight scene, and the other part of the panoramic image is an image shot in a non-backlight scene, all the initial preview images shot in the non-backlight scene and all the processed intermediate preview images shot in the backlight scene need to be sequentially spliced according to the time sequence according to the acquisition time of each frame of image, so that the spliced panoramic image is obtained.

In the first embodiment of the invention, whether the current frame initial preview image is in a backlight scene or not is detected for each frame initial preview image during panoramic shooting, and when the current frame initial preview image is in the backlight scene, high dynamic range processing is carried out for the current frame initial preview image to generate a middle preview image; the obtained intermediate preview image and the initial preview image in the non-backlight scene are spliced according to the image acquisition sequence to obtain the panoramic image, so that the obtained panoramic image does not have the condition of high-brightness range exposure, the details of bright and dark positions are clear and visible, the panoramic shooting effect and the panoramic shooting experience are improved, and the problem that the picture quality is influenced due to the fact that the panoramic image shot in the backlight scene in the prior art is over-exposed or over-dark is solved.

Example two

As shown in fig. 2, a method for panoramic shooting in a backlit scene according to a second embodiment of the present invention is applied to a mobile terminal having a camera, and the method includes:

step 201, receiving a panoramic shooting instruction.

After receiving a start instruction of panoramic shooting, entering a panoramic shooting mode, and detecting whether a shooting scene of each frame of initial preview image acquired by the camera is in a backlight scene, where step 202 needs to be executed.

Step 202, for each frame of initial preview image collected by the camera, obtaining gray scale information of the frame of initial preview image.

Step 203, based on the gray level information, a gray level histogram is generated.

In the gray histogram, each gray value corresponds to the number of corresponding pixel points, the gray value information of the gray histogram is 0-255, and step 204 is executed after the gray histogram is generated.

And step 204, dividing the gray level histogram into N areas according to the ascending order of the gray level values.

And dividing the gray level histogram into N areas which are sequentially arranged according to the ascending order of the gray level value. Wherein each of the N regions includes a preset number of gray values. Wherein the number of gray scale values in each region may or may not be equal.

And the gray value information of the N areas is 0-255, when the gray histogram is divided into three areas, the gray value of the first area can include 0-78, the gray value of the second area can include 79-155, and the gray value of the third area can include 156-255.

Step 205, determining whether the shooting scene corresponding to the frame of initial preview image is a backlight scene based on the gray-scale values of the N regions.

After the gray level histogram is divided, it is necessary to determine whether the shooting scene of the current frame image is a backlight scene according to the gray levels of the N regions, and the modes include three types.

In a first mode

When the gray level histogram includes a first region, a second region and a third region which are sequentially arranged according to the ascending order of gray level values, based on the gray level values of the N regions, a manner of determining whether a shooting scene corresponding to the frame of initial preview image is a backlight scene is as follows:

counting the number of pixel points corresponding to each gray value in the first area, determining a first gray value with the maximum number of the pixel points, and counting the number N1 of the pixel points of the first gray value;

counting the number of pixel points corresponding to each gray value in the second area, determining a second gray value with the maximum number of the pixel points, and counting the number of the pixel points N2 with the second gray value;

counting the number of pixel points corresponding to each gray value in the third region, determining a third gray value with the maximum number of pixel points, and counting the number N3 of the pixel points of the third gray value;

when N1 is greater than N2 and N3 is greater than N2, it is determined that the shooting scene corresponding to the frame of the initial preview image is a backlit scene.

Specifically, when the gray level histogram is divided into three regions which are sequentially arranged, and the gray levels of the first region, the second region and the third region are sequentially increased, the number of the pixel points of each gray level in the first region needs to be counted, a first gray level corresponding to the maximum number of the pixel points in the first region is determined, and then the number N1 of the pixel points corresponding to the first gray level is determined. For example: the number of the gray values in the first region is 31, the corresponding gray value is 0-30, the number of the pixel points corresponding to each gray value needs to be counted, and the number of the pixel points corresponding to the first gray value with the largest number of the pixel points is determined. For example, if the number of pixels corresponding to the gray scale value 30 is the largest, the number of pixels corresponding to the gray scale value 30 needs to be determined, and the number is determined to be N1.

Meanwhile, the number of the pixel points of each gray value in the second region needs to be counted, a second gray value corresponding to the maximum number of the pixel points in the second region is determined, and then the number of the pixel points corresponding to the second gray value N2 is determined. For example: the number of the gray values in the second area is 151, the corresponding gray value is 31-181, the number of the pixel points corresponding to each gray value needs to be counted, and the number of the pixel points corresponding to the second gray value with the largest number of the pixel points is determined. For example, if the number of pixels corresponding to the gray-scale value 126 is the largest, the number of pixels corresponding to the gray-scale value 126 needs to be determined, and the number is determined to be N2.

The number of the pixel points of each gray value in the third region also needs to be counted, a third gray value corresponding to the maximum number of the pixel points in the third region is determined, and then the number N3 of the pixel points corresponding to the third gray value is determined. For example: the number of gray values in the third area is 74, the corresponding gray value is 182-255, the number of pixel points corresponding to each gray value needs to be counted, and the number of pixel points corresponding to the third gray value with the largest number of pixel points is determined. For example, if the number of pixels corresponding to the gray-scale value 224 is the largest, the number of pixels corresponding to the gray-scale value 224 needs to be determined, and the number is determined to be N3.

Then, comparing the sizes of N1, N2 and N3, and when N1 is greater than N2 and N3 is greater than N2, at this time, the maximum pixel number information of the first region, the second region and the third region of the gray histogram is in a U shape, and it can be determined that the shooting scene of the initial preview image of the frame is a backlight scene.

Mode two

calculating the sum of the number of all pixel points in the first area and the third area to obtain a first pixel point sum value;

calculating the sum of the number of all pixel points in the first area, the second area and the third area to obtain a second pixel point sum value;

calculating the ratio of the sum of the first pixel points to the sum of the second pixel points to obtain the ratio of the pixel points;

comparing the pixel point ratio with a preset threshold value;

and when the pixel point ratio is greater than a preset threshold value, determining that the shooting scene corresponding to the initial preview image of the frame is a backlight scene.

Specifically, when the gray level histogram is divided into three regions which are sequentially arranged, and the gray levels of the first region, the second region and the third region are sequentially increased, the number K1 of the pixels in the first region, the number K2 of the pixels in the second region and the number K3 of the pixels in the third region are counted. And calculating the sum of the number K1 of the pixels in the first area and the number K3 of the pixels in the third area to obtain a sum value of the first pixels. And calculating the sum of the number K1 of the pixels in the first region, the number K2 of the pixels in the second region and the number K3 of the pixels in the third region to obtain a second pixel sum value. And then calculating the ratio of the sum value of the first pixel point to the sum value of the second pixel point to obtain the ratio of the pixel points. And after the pixel point ratio is obtained, comparing the pixel point ratio with a preset threshold, and when the pixel point ratio is greater than the preset threshold, determining that the shooting scene of the frame of initial preview image is a backlight scene.

Mode III

calculating the sum of products of each gray value of the first area and the number of corresponding pixel points to obtain a first sum value;

calculating the ratio of the first sum value to the total number of the gray values of the first area to obtain a first reference value M1;

calculating the sum of products of each gray value of the second area and the number of corresponding pixel points to obtain a second sum value;

calculating the ratio of the second sum value to the total number of the gray values of the second area to obtain a second reference value M2;

calculating the sum of products of each gray value of the third area and the number of corresponding pixel points to obtain a third sum value;

calculating the ratio of the third sum value to the total number of the gray values of the third area to obtain a third reference value M3;

when M1 is smaller than M2 and M3 is larger than M2, it is determined that the shooting scene corresponding to the frame of the initial preview image is a backlit scene.

Specifically, when the gray level histogram is divided into three regions which are sequentially arranged, and the gray levels of the first region, the second region and the third region are sequentially increased, for the first region, the number of pixel points corresponding to each gray level of the first region is obtained, then, the product of each gray level and the corresponding number of pixel points is calculated, when the first region includes 31 gray levels, 31 products can be obtained, 31 products are accumulated to obtain a first sum, and the ratio of the first sum to the number of gray levels 31 corresponding to the first region is calculated to obtain M1.

Meanwhile, for the second region, the number of pixel points corresponding to each gray value of the second region is obtained, then the product of each gray value and the corresponding number of pixel points is calculated, when the second region includes 151 gray values, 151 products can be obtained, 151 products are accumulated to obtain a second sum value, and the ratio of the second sum value to the number 151 of gray values corresponding to the second region is calculated to obtain M2.

For the third region, the number of pixel points corresponding to each gray value of the third region is obtained, then the product of each gray value and the corresponding number of pixel points is calculated, when the third region includes 74 gray values, 74 products can be obtained, 74 products are accumulated to obtain a third sum value, and the ratio of the third sum value to the number of gray values 74 corresponding to the third region is calculated to obtain M3.

Then, the sizes of M1, M2 and M3 are compared, and when M1 is smaller than M2 and M3 is larger than M2, the shooting scene of the initial preview image of the frame is determined to be a backlight scene.

And step 206, when the shooting scene corresponding to the initial preview image is detected to be a backlight scene, performing high dynamic range processing on the initial preview image to generate a middle preview image.

When the shooting scene of the frame of initial preview image is determined to be a backlight scene, high dynamic range processing needs to be carried out on the frame of initial preview image, and when the high dynamic range processing is carried out on the frame of initial preview image, a processing mode needs to be determined according to the number of the mobile terminal cameras.

When the mobile terminal comprises a camera, extracting an overexposed area image and an underexposed area image in the initial preview image aiming at the frame image; reducing the brightness of the image in the overexposure area to a preset brightness value; carrying out brightness compensation processing on the underexposed area image; generating an intermediate preview image based on the normal exposure area image, the processed overexposed area image and the underexposed area image in the initial preview image; the image of the overexposure area is an image of which the exposure parameter exceeds a first preset threshold value, and the image of the underexposure area is an image of which the exposure parameter is lower than a second preset threshold value.

Because the mobile terminal only comprises one camera, a normal exposure area image, an underexposure area image and an overexposure area image need to be extracted from the current frame initial preview image, and then brightness reduction processing is performed on the overexposure area image, so that the brightness of the overexposure area image is reduced to a preset brightness value. For the underexposed area image, brightness compensation processing is required, and then the intermediate preview image is obtained by synthesizing the processed overexposed area image, the underexposed area image and the unprocessed normal exposure area image.

When the mobile terminal comprises a first camera and a second camera, before high dynamic range processing is carried out on a current frame image, an overexposed backlight level and an underexposed backlight level corresponding to a backlight scene need to be determined; the method for determining the overexposure backlight level and the underexposure backlight level corresponding to the backlight scene comprises the following steps:

in the first mode of step 205, the maximum number of pixels N1 in the first region, the maximum number of pixels N2 in the second region, and the maximum number of pixels N3 in the third region are obtained. The overexposure backlight level is then determined from the value of N1/N2 and the underexposure backlight level is determined from the value of N3/N2. For example, when N1/N2 is within a first numerical range of 0-10, N1/N2 is determined to belong to a first overexposure backlight level, when N1/N2 is within a second numerical range of 11-20, N1/N2 is determined to belong to a second overexposure backlight level … …, when N1/N2 is within a tenth numerical range of 91-100, and N1/N2 is determined to belong to a tenth overexposure backlight level.

When N3/N2 is located in a first numerical range of 0-10, determining that N3/N2 belongs to a first under-exposure backlight level, when N3/N2 is located in a second numerical range of 11-20, determining that N3/N2 belongs to a second under-exposure backlight level … …, when N3/N2 is located in a tenth numerical range of 91-100, and determining that N3/N2 belongs to a tenth under-exposure backlight level.

If the exposure levels which can be set by the camera are more, the range of the levels can be enlarged, and the corresponding number of the levels is determined according to the actual situation.

After determining the overexposure backlight level and the underexposure backlight level, the initial preview image needs to be subjected to high dynamic range processing to generate an intermediate preview image, which specifically comprises the following steps:

acquiring a frame of normal exposure preview image and a frame of underexposure preview image which are continuously acquired by a first camera;

acquiring a frame of overexposure preview image acquired by a second camera;

carrying out image synthesis on the normal exposure preview image, the underexposure preview image and the overexposure preview image to generate an intermediate preview image;

the method comprises the steps of obtaining an exposure parameter of an exposure image, obtaining an overexposure preview image, obtaining a normal exposure preview image, and obtaining a normal exposure preview image, wherein the overexposure preview image is an image of which the exposure parameter exceeds a first preset threshold value, the underexposure preview image is an image of which the exposure parameter is lower than a second preset threshold value, and the normal exposure preview image is an image of which the exposure parameter is a third.

Specifically, a first exposure parameter of a first camera in normal exposure and a normal exposure preview image output by the first camera are obtained; and setting the exposure parameters of the second camera according to the first exposure parameters and the overexposure backlight level, and acquiring an overexposure preview image output by the second camera. Because the exposure time of the overexposure preview image is longer, after the first camera outputs the normal exposure preview image, the exposure parameters of the first camera can be set according to the underexposure backlight level, and the underexposure preview image output by the first camera is obtained, so that the time for obtaining the normal, overexposure and underexposure preview image is shortest. And finally, synthesizing an intermediate preview image according to the normal exposure preview image, the overexposure preview image and the underexposure preview image.

When the mobile terminal comprises a first camera, a second camera and a third camera, before high dynamic range processing is carried out on a current frame image, an overexposed backlight level and an underexposed backlight level corresponding to a backlight scene need to be determined; the mode of determining the overexposure backlight level and the underexposure backlight level corresponding to the backlight scene is the same as the mode of the mobile terminal comprising two cameras.

acquiring a frame of normal exposure preview image acquired by a first camera, a frame of overexposure preview image acquired by a second camera and a frame of underexposure preview image acquired by a third camera;

carrying out image synthesis on the normal exposure preview image, the overexposure preview image and the underexposure preview image to generate an intermediate preview image;

Acquiring a first exposure parameter of a first camera in normal exposure and a normal exposure preview image output by the first camera; then setting exposure parameters of a second camera according to the first exposure parameters and the overexposure backlight level, and acquiring an overexposure preview image output by the second camera; finally, setting exposure parameters of a third camera according to the first exposure parameters and the underexposure backlight grade, and acquiring an underexposure preview image output by the third camera; and synthesizing an intermediate preview image according to the acquired normal exposure preview image, the acquired overexposure preview image and the acquired underexposure preview image.

And step 207, splicing all the initial preview images acquired by the camera in the non-backlight scene and the processed intermediate preview images acquired by the camera in the backlight scene according to the acquisition sequence to generate a target panoramic image.

After the intermediate preview images are obtained, all the obtained initial preview images in the non-backlight scene and the processed intermediate preview images shot in the backlight scene are subjected to panoramic stitching according to the image acquisition sequence.

When each frame of image of the panoramic image is an image shot in a backlight scene, each frame of image needs to be processed, and each frame of image in the obtained panoramic image is a processed intermediate preview image.

If a part of images of the panoramic image are images shot in a backlight scene, and the other part of images are images shot in a non-backlight scene, acquiring the acquisition time of all initial preview images shot in the non-backlight scene and the acquisition time of all processed intermediate preview images shot in the backlight scene are required, and sequentially splicing according to the time sequence to obtain the spliced panoramic image.

In the second embodiment of the invention, whether the current frame initial preview image is in a backlight scene or not is detected for each frame initial preview image during panoramic shooting, and when the current frame initial preview image is in the backlight scene, high dynamic range processing is carried out for the current frame initial preview image to generate a middle preview image; the obtained intermediate preview image and the initial preview image in the non-backlight scene are spliced according to the image acquisition sequence to obtain the panoramic image, so that the obtained panoramic image does not have the condition of high-brightness range exposure, the details of bright and dark positions are clear and visible, the panoramic shooting effect and the panoramic shooting experience are improved, and the problem that the picture quality is influenced due to the fact that the panoramic image shot in the backlight scene in the prior art is over-exposed or over-dark is solved.

EXAMPLE III

The following is an embodiment of a mobile terminal provided in the third embodiment of the present invention, and details that are not described in detail in the embodiment of the mobile terminal may refer to the above method embodiment.

The embodiment of the invention provides a mobile terminal, which can realize the details of the backlight scene panoramic shooting method in the first embodiment and achieve the same effect. As shown in fig. 3, the mobile terminal includes:

a receiving module 10, configured to receive a panoramic shooting instruction;

the detection module 20 is configured to detect, for each frame of initial preview image acquired by the camera, whether a shooting scene corresponding to the frame of initial preview image is a backlight scene;

the processing module 30 is configured to, when the detection module 20 detects that the shooting scene corresponding to the initial preview image is a backlight scene, perform high dynamic range processing on the initial preview image to generate an intermediate preview image;

and the generating module 40 is configured to perform image stitching on all initial preview images acquired by the camera in a non-backlit scene and all intermediate preview images acquired by the camera in a backlit scene and processed by the processing module 30 according to an acquisition sequence, so as to generate a target panoramic image.

As shown in fig. 4, the detection module 20 includes:

the first obtaining submodule 21 is configured to obtain gray scale information of the frame of initial preview image;

the generation submodule 22 is configured to generate a gray level histogram based on the gray level information acquired by the acquisition submodule 21;

a dividing submodule 23 configured to divide the grayscale histogram generated by the generating submodule 22 into N regions in an order of increasing grayscale values;

a determining submodule 24, configured to determine whether a shooting scene corresponding to the frame of initial preview image is a backlight scene based on gray-level values of the N regions divided by the dividing submodule 23;

wherein each of the N regions includes a preset number of gray values.

When the gray histogram includes a first region, a second region, and a third region sequentially arranged according to an increasing order of gray values, as shown in fig. 5, the determining sub-module 24 includes:

the first processing unit 2401 is configured to count the number of pixel points corresponding to each gray value in the first region, determine a first gray value with the largest number of pixel points, and count the number N1 of the pixel points with the first gray value;

the second processing unit 2402 is configured to count the number of pixel points corresponding to each gray value in the second region, determine a second gray value with the largest number of pixel points, and count the number N2 of pixel points with the second gray value;

the third processing unit 2403 is configured to count the number of pixel points corresponding to each gray value in the third region, determine a third gray value with the largest number of pixel points, and count the number N3 of the pixel points of the third gray value;

a first determining unit 2404, configured to determine that the shooting scene corresponding to the frame of the initial preview image is a backlit scene when N1 is greater than N2 and N3 is greater than N2.

Wherein, when the gray histogram includes a first region, a second region and a third region which are sequentially arranged according to an increasing order of gray values, the determining submodule 24 includes:

the first calculating unit 2405 is configured to calculate a sum of the number of all pixel points in the first region and the third region to obtain a first pixel point sum value;

a second calculating unit 2406, configured to calculate a sum of the numbers of all pixel points in the first region, the second region, and the third region, to obtain a second pixel point sum value;

a third calculating unit 2407, configured to calculate a ratio between the first pixel point sum value obtained by the first calculating unit 2405 and the second pixel point sum value obtained by the second calculating unit 2406, so as to obtain a pixel point ratio;

a comparing unit 2408, configured to compare the pixel point ratio obtained by the third calculating unit 2407 with a preset threshold;

a second determining unit 2409, configured to determine that a shooting scene corresponding to the initial preview image of the frame is a backlight scene when the comparison result of the comparing unit 2408 is that the pixel point ratio is greater than the preset threshold.

a fourth calculating unit 2410, configured to calculate a sum of products of each gray value of the first area and the corresponding number of pixel points to obtain a first sum;

a fifth calculating unit 2411, configured to calculate a ratio of the first sum obtained by the fourth calculating unit 2410 to the total number of the grayscale values of the first region, so as to obtain a first reference value M1;

a sixth calculating unit 2412, configured to calculate a sum of products of each gray value of the second region and the corresponding number of pixel points, to obtain a second sum;

a seventh calculating unit 2413, configured to calculate a ratio of the second sum obtained by the sixth calculating unit 2412 to the total number of the grayscale values of the second region, so as to obtain a second reference value M2;

an eighth calculating unit 2414, configured to calculate a sum of products of each gray value of the third area and the corresponding number of pixel points, to obtain a third sum;

a ninth calculating unit 2415, configured to calculate a ratio of the third sum obtained by the eighth calculating unit 2414 to the total number of the grayscale values of the third region, so as to obtain a third reference value M3;

a third determining unit 2416, configured to determine that the shooting scene corresponding to the frame of the initial preview image is a backlit scene when M1 is smaller than M2 and M3 is larger than M2.

Wherein, when the mobile terminal includes a camera, as shown in fig. 6, the processing module 30 includes:

an extraction submodule 31 configured to extract an overexposed area image and an underexposed area image in the initial preview image;

a first processing submodule 32, configured to reduce the brightness of the overexposed region image extracted by the extraction submodule 31 to a preset brightness value;

a second processing submodule 33, configured to perform brightness compensation processing on the underexposed area image extracted by the extraction submodule 31;

a third processing submodule 34, configured to generate an intermediate preview image based on the normal exposure area image, the processed overexposed area image, and the underexposed area image in the initial preview image;

the image of the overexposure area is an image of which the exposure parameter exceeds a first preset threshold value, and the image of the underexposure area is an image of which the exposure parameter is lower than a second preset threshold value.

Wherein, when the mobile terminal includes the first camera and the second camera, the processing module 30 includes:

the second obtaining submodule 35 is configured to obtain a frame of normal exposure preview image and a frame of underexposure preview image that are continuously collected by the first camera;

a third obtaining submodule 36, configured to obtain a frame of overexposure preview image acquired by the second camera;

a first synthesis submodule 37, configured to perform image synthesis on the normal exposure preview image and the underexposure preview image acquired by the second acquisition submodule 35 and the overexposure preview image acquired by the third acquisition submodule 36, so as to generate an intermediate preview image;

Wherein, when the mobile terminal includes the first camera, the second camera and the third camera, the processing module 30 includes:

a fourth obtaining submodule 38, configured to obtain a frame of normal exposure preview image acquired by the first camera, a frame of overexposure preview image acquired by the second camera, and a frame of underexposure preview image acquired by the third camera;

a second synthesis submodule 39, configured to perform image synthesis on the normal exposure preview image, the overexposure preview image, and the underexposure preview image acquired by the fourth acquisition submodule 38, and generate an intermediate preview image;

In the third embodiment of the invention, whether the current frame initial preview image is in a backlight scene or not is detected for each frame initial preview image during panoramic shooting, and when the current frame initial preview image is in the backlight scene, high dynamic range processing is carried out for the current frame initial preview image to generate a middle preview image; the obtained intermediate preview image and the initial preview image in the non-backlight scene are spliced according to the image acquisition sequence to obtain the panoramic image, so that the obtained panoramic image does not have the condition of high-brightness range exposure, the details of bright and dark positions are clear and visible, the panoramic shooting effect and the panoramic shooting experience are improved, and the problem that the picture quality is influenced due to the fact that the panoramic image shot in the backlight scene in the prior art is over-exposed or over-dark is solved.

Example four

Fig. 7 is a block diagram of a mobile terminal according to another embodiment of the present invention. The mobile terminal 700 shown in fig. 7 includes: at least one processor 701, a memory 702, at least one network interface 704, a user interface 703, and a camera 706. The various components in the mobile terminal 700 are coupled together by a bus system 705. It is understood that the bus system 705 is used to enable communications among the components. The bus system 705 includes a power bus, a control bus, and a status signal bus in addition to a data bus. But for clarity of illustration the various busses are labeled in figure 7 as the bus system 705.

The user interface 703 may include, among other things, a display, a keyboard, or a pointing device (e.g., a mouse, trackball, touch pad, or touch screen, among others.

It is to be understood that the memory 702 in embodiments of the present invention may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The non-volatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of example, but not limitation, many forms of RAM are available, such as Static random access memory (Static RAM, SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic random access memory (Synchronous DRAM, SDRAM), Double Data rate Synchronous Dynamic random access memory (ddr SDRAM ), Enhanced Synchronous SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct memory bus RAM (DRRAM). The memory 702 of the systems and methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

In some embodiments, memory 702 stores the following elements, executable modules or data structures, or a subset thereof, or an expanded set thereof: an operating system 7021 and application programs 7022.

The operating system 7021 includes various system programs, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks. The application 7022 includes various applications, such as a Media Player (Media Player), a Browser (Browser), and the like, for implementing various application services. Programs that implement methods in accordance with embodiments of the present invention can be included within application program 7022.

In the embodiment of the present invention, the processor 701 is configured to, by calling a program or an instruction stored in the memory 702, specifically, a program or an instruction stored in the application 7022: according to a panoramic shooting instruction received by the user interface 703; for each frame of initial preview image collected by a camera, detecting whether a shooting scene corresponding to the frame of initial preview image is a backlight scene; when the shooting scene corresponding to the initial preview image is detected to be a backlight scene, performing high dynamic range processing on the initial preview image to generate a middle preview image; and carrying out image splicing on all initial preview images acquired by the camera under a non-backlight scene and all intermediate preview images acquired by the camera under a backlight scene according to the sequence of acquisition to generate a target panoramic image.

The method disclosed in the above embodiments of the present invention may be applied to the processor 701, or implemented by the processor 701. The processor 701 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be implemented by integrated logic circuits of hardware or instructions in the form of software in the processor 701. The Processor 701 may be a general-purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable Gate Array (FPGA) or other programmable logic device, discrete Gate or transistor logic device, or discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory 702, and the processor 701 reads the information in the memory 702 and performs the steps of the above method in combination with the hardware thereof.

It is to be understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For a hardware implementation, the Processing units may be implemented within one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), general purpose processors, controllers, micro-controllers, microprocessors, other electronic units configured to perform the functions described herein, or a combination thereof.

For a software implementation, the techniques described herein may be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. The software codes may be stored in a memory and executed by a processor. The memory may be implemented within the processor or external to the processor.

Optionally, when detecting whether the shooting scene corresponding to the frame of initial preview image is a backlight scene, the processor 701 is further configured to: acquiring gray information of the frame of initial preview image; generating a gray level histogram based on the gray level information; dividing the gray level histogram into N areas according to the ascending order of the gray level values; determining whether a shooting scene corresponding to the frame of initial preview image is a backlight scene or not based on the gray values of the N areas; wherein each of the N regions includes a preset number of gray values.

Optionally, when the gray level histogram includes a first region, a second region, and a third region that are sequentially arranged according to an increasing order of gray levels, the processor 701 is further configured to, when determining whether a shooting scene corresponding to the frame of initial preview image is a backlight scene based on the gray levels of the N regions: counting the number of pixel points corresponding to each gray value in the first area, determining a first gray value with the maximum number of the pixel points, and counting the number N1 of the pixel points of the first gray value; counting the number of pixel points corresponding to each gray value in the second area, determining a second gray value with the maximum number of the pixel points, and counting the number of the pixel points N2 with the second gray value; counting the number of pixel points corresponding to each gray value in the third region, determining a third gray value with the maximum number of pixel points, and counting the number N3 of the pixel points of the third gray value; when N1 is greater than N2 and N3 is greater than N2, it is determined that the shooting scene corresponding to the frame of the initial preview image is a backlit scene.

Optionally, when the gray level histogram includes a first region, a second region, and a third region that are sequentially arranged according to an increasing order of gray levels, the processor 701 is further configured to, when determining whether a shooting scene corresponding to the frame of initial preview image is a backlight scene based on the gray levels of the N regions: calculating the sum of the number of all pixel points in the first area and the third area to obtain a first pixel point sum value; calculating the sum of the number of all pixel points in the first area, the second area and the third area to obtain a second pixel point sum value; calculating the ratio of the sum of the first pixel points to the sum of the second pixel points to obtain the ratio of the pixel points; comparing the pixel point ratio with a preset threshold value; and when the pixel point ratio is greater than a preset threshold value, determining that the shooting scene corresponding to the initial preview image of the frame is a backlight scene.

Optionally, when the gray level histogram includes a first region, a second region, and a third region that are sequentially arranged according to an increasing order of gray levels, the processor 701 is further configured to, when determining whether a shooting scene corresponding to the frame of initial preview image is a backlight scene based on the gray levels of the N regions: calculating the sum of products of each gray value of the first area and the number of corresponding pixel points to obtain a first sum value; calculating the ratio of the first sum value to the total number of the gray values of the first area to obtain a first reference value M1; calculating the sum of products of each gray value of the second area and the number of corresponding pixel points to obtain a second sum value; calculating the ratio of the second sum value to the total number of the gray values of the second area to obtain a second reference value M2; calculating the sum of products of each gray value of the third area and the number of corresponding pixel points to obtain a third sum value; calculating the ratio of the third sum value to the total number of the gray values of the third area to obtain a third reference value M3; when M1 is smaller than M2 and M3 is larger than M2, it is determined that the shooting scene corresponding to the frame of the initial preview image is a backlit scene.

Optionally, when the mobile terminal includes one camera, the processor 701 performs high dynamic range processing on the initial preview image, and when generating an intermediate preview image, is further configured to: extracting an overexposed area image and an underexposed area image in the initial preview image; reducing the brightness of the image in the overexposure area to a preset brightness value; carrying out brightness compensation processing on the underexposed area image; generating an intermediate preview image based on the normal exposure area image, the processed overexposed area image and the underexposed area image in the initial preview image; the image of the overexposure area is an image of which the exposure parameter exceeds a first preset threshold value, and the image of the underexposure area is an image of which the exposure parameter is lower than a second preset threshold value.

Optionally, when the mobile terminal includes a first camera and a second camera, the processor 701 performs high dynamic range processing on the initial preview image, and when generating an intermediate preview image, is further configured to: acquiring a frame of normal exposure preview image and a frame of underexposure preview image which are continuously acquired by a first camera; acquiring a frame of overexposure preview image acquired by a second camera; carrying out image synthesis on the normal exposure preview image, the underexposure preview image and the overexposure preview image to generate an intermediate preview image; the method comprises the steps of obtaining an exposure parameter of an exposure image, obtaining an overexposure preview image, obtaining a normal exposure preview image, and obtaining a normal exposure preview image, wherein the overexposure preview image is an image of which the exposure parameter exceeds a first preset threshold value, the underexposure preview image is an image of which the exposure parameter is lower than a second preset threshold value, and the normal exposure preview image is an image of which the exposure parameter is a third.

Optionally, when the mobile terminal includes the first camera, the second camera, and the third camera, the processor 701 performs high dynamic range processing on the initial preview image, and when generating the intermediate preview image, is further configured to: acquiring a frame of normal exposure preview image acquired by a first camera, a frame of overexposure preview image acquired by a second camera and a frame of underexposure preview image acquired by a third camera; carrying out image synthesis on the normal exposure preview image, the overexposure preview image and the underexposure preview image to generate an intermediate preview image; the method comprises the steps of obtaining an exposure parameter of an exposure image, obtaining an overexposure preview image, obtaining a normal exposure preview image, and obtaining a normal exposure preview image, wherein the overexposure preview image is an image of which the exposure parameter exceeds a first preset threshold value, the underexposure preview image is an image of which the exposure parameter is lower than a second preset threshold value, and the normal exposure preview image is an image of which the exposure parameter is a third.

The mobile terminal 700 can implement the processes implemented by the mobile terminal in the foregoing embodiments, and details are not repeated here to avoid repetition.

In the fourth embodiment of the invention, whether the current frame initial preview image is in a backlight scene or not is detected for each frame initial preview image during panoramic shooting, and when the current frame initial preview image is in the backlight scene, high dynamic range processing is carried out for the current frame initial preview image to generate a middle preview image; the obtained intermediate preview image and the initial preview image in the non-backlight scene are spliced according to the image acquisition sequence to obtain the panoramic image, so that the obtained panoramic image does not have the condition of high-brightness range exposure, the details of bright and dark positions are clear and visible, the panoramic shooting effect and the panoramic shooting experience are improved, and the problem that the picture quality is influenced due to the fact that the panoramic image shot in the backlight scene in the prior art is over-exposed or over-dark is solved.

EXAMPLE five

Fig. 8 is a schematic structural diagram of a mobile terminal according to another embodiment of the present invention. Specifically, the mobile terminal 800 in fig. 8 may be a mobile phone, a tablet computer, a Personal Digital Assistant (PDA), or a vehicle-mounted computer.

The mobile terminal 800 in fig. 8 includes a Radio Frequency (RF) circuit 810, a memory 820, an input unit 830, a display unit 840, a camera 850, a processor 860, an audio circuit 870, a wifi (wireless fidelity) module 880, and a power supply 890.

The input unit 830 may be used, among other things, to receive numeric or character information input by a user and to generate signal inputs related to user settings and function control of the mobile terminal 800. Specifically, in the embodiment of the present invention, the input unit 830 may include a touch panel 831. The touch panel 831, also referred to as a touch screen, can collect touch operations performed by a user on or near the touch panel 831 (e.g., operations performed by the user on the touch panel 831 using a finger, a stylus, or any other suitable object or accessory), and drive the corresponding connection device according to a preset program. Alternatively, the touch panel 831 may include two portions, i.e., a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 860, and can receive and execute commands sent by the processor 860. In addition, the touch panel 831 may be implemented by various types such as a resistive type, a capacitive type, an infrared ray, and a surface acoustic wave. In addition to the touch panel 831, the input unit 830 may include other input devices 832, and the other input devices 832 may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

Among other things, the display unit 840 may be used to display information input by the user or information provided to the user and various menu interfaces of the mobile terminal 800. The display unit 840 may include a display panel 841, and the display panel 841 may be alternatively configured in the form of an LCD or an Organic Light-Emitting Diode (OLED), or the like.

It should be noted that the touch panel 831 can overlay the display panel 841 to form a touch display screen, which, when it detects a touch operation thereon or nearby, is passed to the processor 860 to determine the type of touch event, and then the processor 860 provides a corresponding visual output on the touch display screen according to the type of touch event.

The touch display screen comprises an application program interface display area and a common control display area. The arrangement modes of the application program interface display area and the common control display area are not limited, and can be an arrangement mode which can distinguish two display areas, such as vertical arrangement, left-right arrangement and the like. The application interface display area may be used to display an interface of an application. Each interface may contain at least one interface element such as an icon and/or widget desktop control for an application. The application interface display area may also be an empty interface that does not contain any content. The common control display area is used for displaying controls with high utilization rate, such as application icons like setting buttons, interface numbers, scroll bars, phone book icons and the like.

The processor 860 is a control center of the mobile terminal 800, connects various parts of the entire mobile phone using various interfaces and lines, and performs various functions of the mobile terminal 800 and processes data by operating or executing software programs and/or modules stored in the first memory 821 and calling data stored in the second memory 822, thereby integrally monitoring the mobile terminal 800. Optionally, processor 860 may include one or more processing units.

In an embodiment of the present invention, the processor 860 is configured to, by invoking software programs and/or modules stored in the first memory 821 and/or data stored in the second memory 822: according to the panorama shooting instruction received by the input unit 830; for each frame of initial preview image collected by a camera, detecting whether a shooting scene corresponding to the frame of initial preview image is a backlight scene; when the shooting scene corresponding to the initial preview image is detected to be a backlight scene, performing high dynamic range processing on the initial preview image to generate a middle preview image; and carrying out image splicing on all initial preview images acquired by the camera under a non-backlight scene and all intermediate preview images acquired by the camera under a backlight scene according to the sequence of acquisition to generate a target panoramic image.

Optionally, when detecting whether the shooting scene corresponding to the frame of initial preview image is a backlight scene, the processor 860 is further configured to: acquiring gray information of the frame of initial preview image; generating a gray level histogram based on the gray level information; dividing the gray level histogram into N areas according to the ascending order of the gray level values; determining whether a shooting scene corresponding to the frame of initial preview image is a backlight scene or not based on the gray values of the N areas; wherein each of the N regions includes a preset number of gray values.

Optionally, when the grayscale histogram includes a first region, a second region, and a third region that are sequentially arranged according to an increasing order of grayscale values, the processor 860 is further configured to, when determining whether a captured scene corresponding to the frame of initial preview image is a backlight scene based on the grayscale values of the N regions: counting the number of pixel points corresponding to each gray value in the first area, determining a first gray value with the maximum number of the pixel points, and counting the number N1 of the pixel points of the first gray value; counting the number of pixel points corresponding to each gray value in the second area, determining a second gray value with the maximum number of the pixel points, and counting the number of the pixel points N2 with the second gray value; counting the number of pixel points corresponding to each gray value in the third region, determining a third gray value with the maximum number of pixel points, and counting the number N3 of the pixel points of the third gray value; when N1 is greater than N2 and N3 is greater than N2, it is determined that the shooting scene corresponding to the frame of the initial preview image is a backlit scene.

Optionally, when the grayscale histogram includes a first region, a second region, and a third region that are sequentially arranged according to an increasing order of grayscale values, the processor 860 is further configured to, when determining whether a captured scene corresponding to the frame of initial preview image is a backlight scene based on the grayscale values of the N regions: calculating the sum of the number of all pixel points in the first area and the third area to obtain a first pixel point sum value; calculating the sum of the number of all pixel points in the first area, the second area and the third area to obtain a second pixel point sum value; calculating the ratio of the sum of the first pixel points to the sum of the second pixel points to obtain the ratio of the pixel points; comparing the pixel point ratio with a preset threshold value; and when the pixel point ratio is greater than a preset threshold value, determining that the shooting scene corresponding to the initial preview image of the frame is a backlight scene.

Optionally, when the grayscale histogram includes a first region, a second region, and a third region that are sequentially arranged according to an increasing order of grayscale values, the processor 860 is further configured to, when determining whether a captured scene corresponding to the frame of initial preview image is a backlight scene based on the grayscale values of the N regions: calculating the sum of products of each gray value of the first area and the number of corresponding pixel points to obtain a first sum value; calculating the ratio of the first sum value to the total number of the gray values of the first area to obtain a first reference value M1; calculating the sum of products of each gray value of the second area and the number of corresponding pixel points to obtain a second sum value; calculating the ratio of the second sum value to the total number of the gray values of the second area to obtain a second reference value M2; calculating the sum of products of each gray value of the third area and the number of corresponding pixel points to obtain a third sum value; calculating the ratio of the third sum value to the total number of the gray values of the third area to obtain a third reference value M3; when M1 is smaller than M2 and M3 is larger than M2, it is determined that the shooting scene corresponding to the frame of the initial preview image is a backlit scene.

Optionally, when the mobile terminal includes a camera, the processor 860 performs high dynamic range processing on the initial preview image, and when generating an intermediate preview image, is further configured to: extracting an overexposed area image and an underexposed area image in the initial preview image; reducing the brightness of the image in the overexposure area to a preset brightness value; carrying out brightness compensation processing on the underexposed area image; generating an intermediate preview image based on the normal exposure area image, the processed overexposed area image and the underexposed area image in the initial preview image; the image of the overexposure area is an image of which the exposure parameter exceeds a first preset threshold value, and the image of the underexposure area is an image of which the exposure parameter is lower than a second preset threshold value.

Optionally, when the mobile terminal includes a first camera and a second camera, the processor 860 performs high dynamic range processing on the initial preview image, and when generating an intermediate preview image, is further configured to: acquiring a frame of normal exposure preview image and a frame of underexposure preview image which are continuously acquired by a first camera; acquiring a frame of overexposure preview image acquired by a second camera; carrying out image synthesis on the normal exposure preview image, the underexposure preview image and the overexposure preview image to generate an intermediate preview image; the method comprises the steps of obtaining an exposure parameter of an exposure image, obtaining an overexposure preview image, obtaining a normal exposure preview image, and obtaining a normal exposure preview image, wherein the overexposure preview image is an image of which the exposure parameter exceeds a first preset threshold value, the underexposure preview image is an image of which the exposure parameter is lower than a second preset threshold value, and the normal exposure preview image is an image of which the exposure parameter is a third.

Optionally, when the mobile terminal includes the first camera, the second camera, and the third camera, the processor 860 performs high dynamic range processing on the initial preview image, and when generating the intermediate preview image, is further configured to: acquiring a frame of normal exposure preview image acquired by a first camera, a frame of overexposure preview image acquired by a second camera and a frame of underexposure preview image acquired by a third camera; carrying out image synthesis on the normal exposure preview image, the overexposure preview image and the underexposure preview image to generate an intermediate preview image; the method comprises the steps of obtaining an exposure parameter of an exposure image, obtaining an overexposure preview image, obtaining a normal exposure preview image, and obtaining a normal exposure preview image, wherein the overexposure preview image is an image of which the exposure parameter exceeds a first preset threshold value, the underexposure preview image is an image of which the exposure parameter is lower than a second preset threshold value, and the normal exposure preview image is an image of which the exposure parameter is a third.

In the fifth embodiment of the invention, whether the current frame initial preview image is in a backlight scene or not is detected for each frame initial preview image during panoramic shooting, and when the current frame initial preview image is in the backlight scene, high dynamic range processing is carried out for the current frame initial preview image to generate a middle preview image; the obtained intermediate preview image and the initial preview image in the non-backlight scene are spliced according to the image acquisition sequence to obtain the panoramic image, so that the obtained panoramic image does not have the condition of high-brightness range exposure, the details of bright and dark positions are clear and visible, the panoramic shooting effect and the panoramic shooting experience are improved, and the problem that the picture quality is influenced due to the fact that the panoramic image shot in the backlight scene in the prior art is over-exposed or over-dark is solved.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

While the preferred embodiments of the present invention have been described, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the following claims.

Claims

1. A method for panoramic shooting of a backlight scene is applied to a mobile terminal with a camera, and is characterized by comprising the following steps:

receiving a panoramic shooting instruction;

for each frame of initial preview image collected by the camera, detecting whether a shooting scene corresponding to the frame of initial preview image is a backlight scene;

splicing all initial preview images acquired by the camera under a non-backlight scene and all intermediate preview images acquired by the camera under a backlight scene according to the sequence of acquisition to generate a target panoramic image;

when the mobile terminal comprises a first camera, a second camera and a third camera, the step of performing high dynamic range processing on the initial preview image to generate a middle preview image comprises the following steps:

acquiring a frame of normal exposure preview image acquired by the first camera, a frame of overexposure preview image acquired by the second camera and a frame of underexposure preview image acquired by the third camera;

synthesizing the normal exposure preview image, the overexposure preview image and the underexposure preview image to generate an intermediate preview image;

the method comprises the steps of obtaining an overexposure preview image, obtaining an underexposure preview image, and obtaining a normal exposure preview image, wherein the overexposure preview image is an image of which the exposure parameter exceeds a first preset threshold, the underexposure preview image is an image of which the exposure parameter is lower than a second preset threshold, and the normal exposure preview image is an image of which the exposure parameter is a third preset threshold.

2. The method according to claim 1, wherein the step of detecting whether the shot scene corresponding to the initial preview image of the frame is a backlight scene comprises:

acquiring gray information of the frame of initial preview image;

generating a gray level histogram based on the gray level information;

dividing the gray level histogram into N areas according to the ascending order of the gray level values;

determining whether a shooting scene corresponding to the frame of initial preview image is a backlight scene or not based on the gray values of the N areas;

wherein each of the N regions includes a preset number of gray values.

3. The method according to claim 2, wherein when the gray histogram includes a first region, a second region and a third region arranged in order of increasing gray value, the step of determining whether the captured scene corresponding to the frame of the initial preview image is a backlit scene based on the gray values of the N regions includes:

counting the number of pixel points corresponding to each gray value in the first region, determining a first gray value with the maximum number of pixel points, and counting the number of the pixel points N1 of the first gray value;

counting the number of pixel points corresponding to each gray value in the second region, determining a second gray value with the maximum number of pixel points, and counting the number of the pixel points N2 of the second gray value;

counting the number of pixel points corresponding to each gray value in the third region, determining a third gray value with the maximum number of pixel points, and counting the number of the pixel points N3 of the third gray value;

4. The method according to claim 2, wherein when the gray histogram includes a first region, a second region and a third region arranged in order of increasing gray value, the step of determining whether the captured scene corresponding to the frame of the initial preview image is a backlit scene based on the gray values of the N regions includes:

comparing the pixel point ratio with a preset threshold value;

and when the pixel point ratio is larger than the preset threshold value, determining that the shooting scene corresponding to the initial preview image of the frame is a backlight scene.

5. The method according to claim 2, wherein when the gray histogram includes a first region, a second region and a third region arranged in order of increasing gray value, the step of determining whether the captured scene corresponding to the frame of the initial preview image is a backlit scene based on the gray values of the N regions includes:

6. The method according to claim 1, wherein when the mobile terminal includes a camera, the step of performing high dynamic range processing on the initial preview image to generate an intermediate preview image includes:

extracting an overexposed area image and an underexposed area image in the initial preview image;

reducing the brightness of the overexposure area image to a preset brightness value;

performing brightness compensation processing on the underexposed area image;

generating the intermediate preview image based on the normal exposure area image, the processed overexposed area image and the underexposed area image in the initial preview image;

7. The method according to claim 1, wherein when the mobile terminal includes a first camera and a second camera, the step of performing high dynamic range processing on the initial preview image to generate an intermediate preview image comprises:

acquiring a frame of normal exposure preview image and a frame of underexposure preview image which are continuously acquired by the first camera;

acquiring a frame of overexposure preview image acquired by the second camera;

synthesizing the normal exposure preview image, the underexposure preview image and the overexposure preview image to generate an intermediate preview image;

8. A mobile terminal, comprising a camera, characterized in that the mobile terminal further comprises:

the receiving module is used for receiving a panoramic shooting instruction;

the processing module is used for performing high dynamic range processing on the initial preview image to generate an intermediate preview image when the detection module detects that the shooting scene corresponding to the initial preview image is a backlight scene;

the generating module is used for splicing all initial preview images acquired by the camera under a non-backlight scene and the intermediate preview images processed by the processing module under the backlight scene according to the acquisition sequence to generate a target panoramic image;

when the mobile terminal comprises a first camera, a second camera and a third camera, the processing module comprises:

a fourth obtaining submodule, configured to obtain a frame of normal exposure preview image collected by the first camera, a frame of overexposure preview image collected by the second camera, and a frame of underexposure preview image collected by the third camera;

the second synthesis submodule is used for carrying out image synthesis on the normal exposure preview image, the overexposure preview image and the underexposure preview image acquired by the fourth acquisition module to generate the intermediate preview image;

9. The mobile terminal of claim 8, wherein the detection module comprises:

the first obtaining submodule is used for obtaining the gray information of the frame of initial preview image;

the generation submodule is used for generating a gray level histogram based on the gray level information acquired by the acquisition submodule;

the dividing submodule is used for dividing the gray histogram generated by the generating submodule into N areas according to the ascending order of gray values;

the determining submodule is used for determining whether a shooting scene corresponding to the frame of initial preview image is a backlight scene or not based on the gray values of the N areas divided by the dividing submodule;

wherein each of the N regions includes a preset number of gray values.

10. The mobile terminal of claim 9, wherein when the histogram of gray levels includes a first region, a second region, and a third region arranged in order of increasing gray level, the determining sub-module comprises:

the first processing unit is used for counting the number of pixel points corresponding to each gray value in the first area, determining a first gray value with the largest number of pixel points, and counting the number N1 of the pixel points with the first gray value;

the second processing unit is used for counting the number of pixel points corresponding to each gray value in the second area, determining a second gray value with the largest number of pixel points, and counting the number N2 of the pixel points with the second gray value;

the third processing unit is used for counting the number of pixel points corresponding to each gray value in the third region, determining a third gray value with the largest number of pixel points, and counting the number N3 of the pixel points of the third gray value;

and a first determining unit, configured to determine that the shooting scene corresponding to the frame of the initial preview image is a backlight scene when N1 is greater than N2 and N3 is greater than N2.

11. The mobile terminal of claim 9, wherein when the histogram of gray levels includes a first region, a second region, and a third region arranged in order of increasing gray level, the determining sub-module comprises:

the first calculation unit is used for calculating the sum of the number of all pixel points in the first area and the third area to obtain a first pixel point sum value;

the second calculation unit is used for calculating the sum of the number of all pixel points in the first area, the second area and the third area to obtain a second pixel point sum value;

the third calculating unit is used for calculating the ratio of the first pixel point sum value obtained by the first calculating unit to the second pixel point sum value obtained by the second calculating unit to obtain a pixel point ratio;

the comparison unit is used for comparing the pixel point ratio obtained by the third calculation unit with a preset threshold;

and the second determining unit is used for determining that the shooting scene corresponding to the frame of initial preview image is a backlight scene when the comparison result of the comparing unit is that the pixel point ratio is greater than the preset threshold value.

12. The mobile terminal of claim 9, wherein when the histogram of gray levels includes a first region, a second region, and a third region arranged in order of increasing gray level, the determining sub-module comprises:

the fourth calculation unit is used for calculating the sum of products of all gray values of the first area and the number of corresponding pixel points to obtain a first sum value;

a fifth calculating unit, configured to calculate a ratio of the first sum obtained by the fourth calculating unit to the total number of the grayscale values of the first region, so as to obtain a first reference value M1;

a sixth calculating unit, configured to calculate a sum of products of each gray value of the second region and the corresponding number of pixel points to obtain a second sum;

a seventh calculating unit, configured to calculate a ratio of the second sum obtained by the sixth calculating unit to the total number of the grayscale values of the second region, so as to obtain a second reference value M2;

the eighth calculating unit is used for calculating the sum of products of all gray values of the third area and the number of corresponding pixel points to obtain a third sum;

a ninth calculating unit, configured to calculate a ratio of the third sum obtained by the eighth calculating unit to the total number of the grayscale values of the third region, so as to obtain a third reference value M3;

and a third determining unit, configured to determine that the shooting scene corresponding to the frame of the initial preview image is a backlight scene when M1 is less than M2 and M3 is greater than M2.

13. The mobile terminal of claim 8, wherein when the mobile terminal comprises a camera, the processing module comprises:

the extraction submodule is used for extracting an overexposed area image and an underexposed area image in the initial preview image;

the first processing submodule is used for reducing the brightness of the overexposed area image extracted by the extraction submodule to a preset brightness value;

the second processing submodule is used for carrying out brightness compensation processing on the underexposed area image extracted by the extraction submodule;

a third processing submodule, configured to generate the intermediate preview image based on a normal exposure area image, a processed overexposed area image, and an underexposed area image in the initial preview image;

14. The mobile terminal of claim 8, wherein when the mobile terminal comprises a first camera and a second camera, the processing module comprises:

the second acquisition submodule is used for acquiring a frame of normal exposure preview image and a frame of underexposure preview image which are continuously acquired by the first camera;

the third acquisition submodule is used for acquiring a frame of overexposure preview image acquired by the second camera;

the first synthesis sub-module is used for carrying out image synthesis on the normal exposure preview image, the underexposure preview image and the overexposure preview image acquired by the second acquisition sub-module and the third acquisition sub-module to generate the intermediate preview image;