CN116664385A

CN116664385A - Image processing method and device, electronic equipment and storage medium

Info

Publication number: CN116664385A
Application number: CN202210157074.0A
Authority: CN
Inventors: 王庆民; 赵雄; 马凯伦
Original assignee: Beijing Xiaomi Mobile Software Co Ltd
Current assignee: Beijing Xiaomi Mobile Software Co Ltd
Priority date: 2022-02-21
Filing date: 2022-02-21
Publication date: 2023-08-29

Abstract

The disclosure relates to an image processing method and device, electronic equipment and storage medium. Wherein the method comprises the following steps: acquiring at least three images to be processed, wherein the background content of the at least three images to be processed is consistent, and the positions of the same shot main body in the at least three images to be processed are different; carrying out inter-frame difference processing on each image to be processed and other images to be processed respectively to obtain a plurality of difference images; and performing pixel value ANDed operation between at least two differential images obtained based on the same image to be processed to obtain a target image containing a target area, and determining the position of the target area in the target image as the position of the shot subject in the same image to be processed.

Description

Image processing method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of image processing, and in particular, to an image processing method and apparatus, an electronic device, and a storage medium.

Background

With the advent of the national photography era, various photography functions, such as an image cloning function and background replacement, have been added to devices by large equipment manufacturers in order to increase the attractiveness of their own products to users.

In order to realize the above-described photographing function, the terminal device first needs to determine the position of the subject in the image. However, the related art generally determines the position of the subject by means of pre-modeling, which requires not only a large amount of preliminary preparation but also a large amount of processing resources.

Disclosure of Invention

The disclosure provides an image processing method and device, electronic equipment and storage medium, and the position of a shot subject in an image can be determined without a pre-established model.

According to a first aspect of the present disclosure, there is provided an image processing method including:

acquiring at least three images to be processed, wherein the background content of the at least three images to be processed is consistent, and the positions of the same shot main body in the at least three images to be processed are different;

carrying out inter-frame difference processing on each image to be processed and other images to be processed respectively to obtain a plurality of difference images;

and performing pixel value ANDed operation between at least two differential images obtained based on the same image to be processed to obtain a target image containing a target area, and determining the position of the target area in the target image as the position of the shot subject in the same image to be processed.

Alternatively to this, the method may comprise,

further comprises: performing binarization processing on pixel values in the differential images to obtain a plurality of binarized images;

the pixel value and operation between at least two differential images obtained based on the same image to be processed comprises the following steps: and performing pixel value AND operation between at least two binarized images obtained based on the same image to be processed.

Optionally, the method further comprises:

after obtaining target images respectively corresponding to the images to be processed, respectively carrying out background segmentation on the corresponding images to be processed based on target areas in the target images;

and acquiring a plurality of foreground parts corresponding to the shot main body obtained through background segmentation, and adding the plurality of foreground parts into any image to be processed according to the positions of the plurality of foreground parts in the image to be processed to which the foreground parts belong.

Optionally, the method further comprises:

after at least three unprocessed images are acquired, carrying out position registration on background pictures contained in the at least three unprocessed images so that the background contents contained in the background pictures are positioned at the same position in different registered images obtained through the position registration;

And taking the registered images obtained by position registration as the at least three images to be processed.

Optionally, the method further comprises:

and determining a background area in which the picture content in the registered image is lost, and complementing the picture content in the background area through an image complementing algorithm.

Optionally, the method further comprises:

and determining background areas with lost picture contents in each registered image, and carrying out picture completion on the background areas in each registered image according to the picture contents in other registered images with non-lost picture contents of the same background area.

Optionally, the pixel value is a gray value or a color value.

According to a second aspect of the present disclosure, there is provided an image processing apparatus including:

the device comprises an acquisition unit, a display unit and a display unit, wherein the acquisition unit acquires at least three images to be processed, the background content of the at least three images to be processed is consistent, and the positions of the same shot main body in the at least three images to be processed are different;

the difference unit is used for carrying out inter-frame difference processing on each image to be processed and other images to be processed to obtain a plurality of difference images;

an AND unit for performing an AND operation of pixel values between at least two differential images obtained based on the same image to be processed to obtain a target image containing a target region, and determining the position of the target region in the target image as the position of the subject in the same image to be processed

According to a third aspect of the present disclosure, there is provided an electronic device comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor implements the method of the first aspect by executing the executable instructions.

According to a fourth aspect of the present disclosure there is provided a computer readable storage medium having stored thereon computer instructions which when executed by a processor perform the steps of the method according to the first aspect.

In the technical scheme, after at least three images to be processed are obtained, the position of the shot main body in the images to be processed can be determined by only sequentially executing inter-frame difference and AND operation on the images to be processed, so that the problems that a large amount of early-stage preparation work is required and more processing resources are occupied due to the fact that the position of the shot main body needs to be determined by depending on a pre-established model in the related technology are avoided.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

FIG. 1 is a flow chart of an image processing method shown in an exemplary embodiment of the present disclosure;

FIG. 2 is a flow chart of an image cloning method according to an exemplary embodiment of the present disclosure;

FIG. 3 is a schematic illustration of a photograph of a fire extinguisher as a subject, according to an exemplary embodiment of the present disclosure;

FIG. 4 is a schematic diagram of a differential image shown in an exemplary embodiment of the present disclosure;

FIG. 5 is a schematic illustration of a binarized image, as shown in an exemplary embodiment of the present disclosure;

FIG. 6A is a schematic diagram of a target image including a target region, as shown in an exemplary embodiment of the present disclosure;

FIG. 6B is a schematic illustration of a clone image shown in an exemplary embodiment of the present disclosure;

fig. 7 is a block diagram of an image processing apparatus shown in an exemplary embodiment of the present disclosure;

FIG. 8 is a block diagram of another image processing apparatus shown in an exemplary embodiment of the present disclosure;

fig. 9 is a schematic structural view of an electronic device in an exemplary embodiment of the present disclosure.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.

The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present disclosure. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.

In the related art, in order to achieve shooting effects such as image cloning, background replacement, and the like, an electronic device needs to preferentially determine the position of a subject in an image through an image segmentation model. This approach requires preferential training of the image segmentation model, not only requires gathering a large number of sample images, but model accuracy depends to a large extent on the knowledge reserves of the operator. In addition, when the device runs the image segmentation model, a large amount of processing resources are required to be occupied, and a great burden is caused to the device, so that problems such as jamming and dead halt even occur in some cases.

Therefore, the disclosure proposes an image processing method to avoid the problems of the prior art that the model needs to be trained in advance and excessive processing resources are occupied when the model is run due to the need of determining the position of the subject in the image through the image segmentation model.

Fig. 1 is a diagram illustrating an image processing method according to an exemplary embodiment of the present disclosure. As shown in fig. 1, the method may include the steps of:

step 102, obtaining at least three images to be processed, wherein the background content of the at least three images to be processed is consistent, and the positions of the same shot subject in the at least three images to be processed are different.

As can be seen from the above description, the related art has a problem that the processing resources are occupied more, because the related art determines the position of the subject in the image through the image segmentation model. In view of this, the present disclosure no longer determines the position of the subject in the image by the image segmentation model.

In the present disclosure, at least three images to be processed may be preferentially acquired. The acquired at least three images to be processed comprise the same shot main body, and the positions of the same shot main body in different images to be processed are different. In addition, the background content contained in at least three images to be processed is consistent.

In some possible embodiments, the at least three images to be processed may be acquired in a variety of ways. For example, the user may perform multiple position adjustments on the subject in the same environment, and after each adjustment, the user may perform shooting on the subject at the same position and at the same angle through an electronic device having an image shooting function, so as to obtain the at least three images to be processed. For another example, the user may take a plurality of shots of the moving subject at the same position and at the same angle to acquire at least three images to be processed including the same subject. Of course, the above manner of acquiring the image to be processed is merely illustrative, and in particular, how to acquire the image to be processed may be determined by those skilled in the art according to actual situations, which is not limited in this disclosure.

After at least three images to be processed are acquired, the present disclosure may further perform inter-frame difference of pixel values between each image to be processed and other images to be processed, so as to obtain a plurality of difference images. It should be noted that, for any one of the images to be processed, the images to be processed other than itself are regarded as other images to be processed corresponding to the one of the images to be processed. If the position of the subject in a certain image to be processed is to be determined, in this step, the image to be processed needs to be at least subjected to inter-frame difference with two other images to be processed, respectively.

It should be understood that, since the background content of different images to be processed is consistent and the positions of the photographed subjects are different, this means that the pixel values of the areas other than the positions of the photographed subjects (i.e., the areas covered by the background content) are relatively close to each other or even the same among the different images to be processed, and the pixel values of the positions of the photographed subjects are relatively different, so that after the inter-frame difference of the pixel values is performed on any two images to be processed, two areas with relatively large differences between the pixel values and 0 in the obtained difference image can reflect the positions of the photographed subjects in the two images to be processed respectively.

Further, after the inter-frame difference is completed, if the plurality of difference images are obtained based on the same image to be processed, it means that there is one region for reflecting the position of the subject in the same image to be processed in two regions where the pixel value and the difference between 0 are larger included in any one of the plurality of difference images, so that if at least two of the plurality of difference images are subjected to the and operation, a target image including a unique target region where the pixel value and the difference between 0 are larger can be obtained, and the position of the target region in the target image can obviously reflect the position of the subject in the same image to be processed. Therefore, through the technical scheme, the position of the shot main body in the image can be determined on the premise of not using an image segmentation model.

In some possible embodiments, the user may not be able to ensure that the electronic device is in a completely fixed position and shooting angle during the image shooting process. In this case, there is a high possibility that the same background content is located at different positions in different images, resulting in a problem of poor difference effect. Therefore, after capturing a plurality of images, the background images included in the images may be further subjected to position registration. In the present disclosure, the captured image may be referred to as an unprocessed image, and then, after at least three unprocessed images are acquired, the background frames included in the at least three unprocessed images may be subjected to position registration, so that the background content included in the background frames is in the same position in different registered images obtained through position registration. On the basis, the obtained registered images can be used as the at least three images to be processed.

It should be appreciated that the position registration operation is likely to require a position shift of the background content contained in the background image, thereby resulting in blank areas in the registered image resulting from the position registration where the image content is missing. Therefore, in the present disclosure, the blank region in the registered image may also be subjected to screen complement, so that the background content in the obtained multiple images to be processed is consistent.

In an embodiment, a background area with missing picture content in the registered image can be preferentially determined, and the picture content of the background area is complemented by an image complement algorithm. For example, a corresponding image complement model may be generated based on the neural network, the image complement model may encode the image into feature vectors in the feature space, and then decode the image based on the respective feature vectors to obtain a complement image. Of course, this example is merely illustrative, and in some possible embodiments, any image-complementing algorithm may be used to complement the background area where the content of the image is missing, and in particular, what image-complementing algorithm is used may be determined by those skilled in the art according to the actual situation, which is not limited by the present disclosure.

In another embodiment, the background area where the picture content in each registered image is missing may be preferentially determined, and the picture complement is performed on the background area in each registered image according to the picture content in other registered images where the picture content in the same background area is not missing. It should be understood that, in each registered image obtained by the position registration, the missing position of the picture content is different, so that the picture content which is not missing in one registered image can be copied to the registered image with lost picture content in another same background area, and the picture complement of the other registered image is realized.

In the present disclosure, similar effects can be achieved by a cutting manner, except that the background in the image to be processed is made to be identical by a picture complement manner. For example, after the registered images are obtained, the registered images may be compared, and background portions with inconsistent content in the registered images may be cropped, so that the background content in the image to be processed obtained by cropping is consistent. It should be understood that after the position registration and the picture completion (or image cropping) operations are completed, the background content of the same position in each image to be processed can be ensured to be consistent, and then the only different regions (i.e. the regions where the photographed subject is located) in the image to be processed can be obtained by performing the inter-frame difference on the basis. In some possible embodiments, the image cropping mode is adopted to achieve the effect, so that the operation is simpler when the background content is consistent. In many cases, the screen complement method is used to match the background content when the screen size of the processed image is required to be unable to be reduced.

Of course, the above manner of processing the registered images so that the background content in the resulting processed images is consistent is merely illustrative, and how to operate specifically may be determined by one skilled in the art based on actual circumstances, which is not limited by the present disclosure.

And 104, respectively carrying out inter-frame difference processing on each image to be processed and other images to be processed to obtain a plurality of difference images.

As described above, in a differential image obtained by the inter-frame difference, it is generally divided into two parts, i.e., a region where the pixel value is 0 or close to 0 and a region where the difference between the pixel value and 0 is large. If the pixel value and operation is directly performed between at least two differential images obtained by processing the same image to be processed on the basis of the above-mentioned difference, there is a high possibility that the target region is not present in the target image obtained by the and operation or the target region is not clear due to a small difference in pixel values of two regions corresponding to the "subject in the same image to be processed" in the at least two differential images.

For example, it is assumed that the differential image a and the differential image B are both obtained based on the image 1 to be processed, and the differential image a includes two regions a and a 'having a large difference between the pixel values and 0, and the differential image B includes two regions B and B' having a large difference between the pixel values and 0. The position of the region a in the differential image a and the position of the region B in the differential image B can reflect the position of the subject in the image 1 to be processed. In an ideal case, after the difference images a and B are subjected to the summing operation, the summed target image may screen out the region a 'in the difference image a and the region B' in the difference image B, leaving only the overlapping region of the region a and the region B (i.e., the target region described above). However, since in some possible embodiments, the pixel values of the pixel points at the same position in the region a and the region B may be different from 0 by a large amount but have a small value, the obtained target image will screen out the regions a and B (or a part of the regions a and B) after the difference image a and the difference image B are summed, so that the position of the subject in the image 1 cannot be determined.

Therefore, after obtaining a plurality of differential images through inter-frame difference, the present disclosure may further perform binarization processing on the differential images, so that the obtained binarized images only include two pixel points with pixel values of 0 and 255. On the basis, pixel value and operation can be carried out between binarized images obtained based on the same image to be processed.

It should be understood that after the difference image is binarized, the pixel values of each pixel point in the obtained binarized image are all constant values, so that the problem that partial images are screened out by mistake due to small difference of numerical values between the difference images is avoided. For example, if the pixel value of a certain pixel in the region a is 245 and the pixel value of a pixel at the same position in the region b is 247, the difference between the two is 0 if the and operation is directly performed; after binarization, the values of the pixel point in the region a and the region b are 255, and the result obtained by performing the ANDed operation on the two values can be 255.

And 106, performing pixel value AND operation between at least two differential images obtained based on the same image to be processed to obtain a target image containing a target area, and determining the position of the target area in the target image as the position of the shot subject in the same image to be processed.

In the present disclosure, after the inter-frame difference is completed, a plurality of all the difference images may be obtained based on the same image to be processed, and then the target image including the target region may be obtained in various ways. For example, a preset number may be optionally selected from a plurality of differential images obtained based on the same image to be processed, and the preset number may be summed between the preset number of differential images to obtain the target image, and in some possible embodiments, the preset number may be determined by a person skilled in the art according to the actual situation, and may be 2, 3, etc.; for another example, a set of differential images with the number of the images being the preset number may be selected from a plurality of differential images obtained based on the same image to be processed, and an and operation may be performed between the preset number of differential images belonging to the same group to obtain a plurality of to-be-determined target images corresponding to the same image to be processed, on this basis, fusion may be performed on the plurality of to-be-determined target images, so that the obtained image is used as the target image corresponding to the same image to be processed, and in some possible embodiments, the above fusion may be implemented by performing a weighted average on pixel values of pixels at the same position in each to-be-determined target image. Of course, the manner in which the target image is obtained is merely illustrative, and how this is specifically performed may be determined by those skilled in the art according to actual circumstances, which is not limited by the present disclosure.

In the present disclosure, the target image corresponding to each image to be processed may be obtained in the above manner, and the position where the target area in each target image is located is determined as the position of the subject in the corresponding image to be processed. On the basis, various processing can be carried out on the images to be processed according to the positions of the shot main body in the images to be processed, so as to realize different image effects.

For example, the background segmentation may be performed on the corresponding image to be processed according to the target region in each target image, so as to obtain the foreground portion and the background portion of each image to be processed. Further, the positions of the foreground portions in the respective to-be-processed images can be obtained according to the segmentation, and the foreground portions can be added into any to-be-processed image. It should be understood that the foreground portion refers to: the portion of the image to be processed that reflects the subject picture, and thus, after each foreground portion is added to any one of the images to be processed, the cloning effect of the subject can be achieved.

Of course, this example is merely illustrative, and after determining the position of the subject in the image to be processed, how to process the image to be processed specifically according to the position to achieve different image effects may be determined by those skilled in the art according to actual needs, which is not limited by the present disclosure.

It is stated that the method can perform inter-frame difference and AND operation on the difference image according to the gray value of the pixel point; and carrying out inter-frame difference on the image to be processed according to the color value of the pixel point, and carrying out AND operation on the difference image. In other words, the pixel values in the present disclosure may be either gray values or color values, and of course, what pixel values are specifically used for image processing may be determined by those skilled in the art according to actual needs, which is not limited in the present disclosure.

It should be further stated that the execution body of the technical scheme of the present disclosure may be any type of electronic device, for example, the electronic device may be a mobile terminal such as a smart phone, a tablet computer, or a fixed terminal such as a smart television, a PC (personal computer ), or the like. It should be understood that, only the electronic device having the image processing function may be used as the execution body of the technical solution of the present disclosure, and specifically, which type of electronic device is used as the execution body of the technical solution of the present disclosure may be determined by those skilled in the art according to actual needs, which is not limited by the present disclosure.

According to the technical scheme, when the user shoots at least three to-be-processed images with consistent background content and different positions of the same shot main body, the to-be-processed images and other to-be-processed images can be subjected to inter-frame difference of pixel values respectively to obtain a plurality of difference images. On the basis, the pixel value can be further obtained and operated between at least two differential images obtained based on the same image to be processed, so that a target image containing a target area is obtained, and at the moment, the position of the target area in the target image can be determined as the position of the shot main body in the image to be processed.

It can be seen that, the technical solution of the present disclosure is equivalent to performing a pairwise inter-frame difference on all the images to be processed, and further performing an and operation of pixel values between the differential images obtained based on the same image to be processed after obtaining the differential images through the inter-frame difference, so as to determine the position of the target area obtained by the and operation as the position of the subject in the corresponding image to be processed.

It should be understood that, since the background content of each image to be processed is consistent, but the positions of the subject are different, that is, the pixel values of the areas where the background content is the same and the pixel values of the positions where the subject is located are different between different images to be processed, after the inter-frame difference of the pixel values is performed on any two images to be processed, the pixel value of the pixel point located in the "area where the background content is in both images to be processed" in the obtained difference image is 0 or close to 0, and the pixel value of the pixel point located in the "area where the subject is in both images to be processed" is a large difference from 0. It can be seen that the difference image obtained by performing the inter-frame difference on any two images to be processed includes two areas capable of reflecting the positions of the subject in the two images to be processed, respectively. On this basis, it is conceivable that one region exists in each of a plurality of differential images obtained based on the same image to be processed for reflecting the position of the subject in the same image to be processed, and therefore, the position of the obtained target region in the target image can be determined as the position of the subject in the same image to be processed by performing an and operation on at least two of the plurality of differential images.

Further, after the differential image is obtained, the differential image may be further binarized to obtain a plurality of binarized images. Wherein the AND operation is performed between at least two binarized images obtained based on the same image to be processed. It should be understood that by performing binarization processing on the differential image, the situation that the differential image is mistakenly screened out due to the fact that the pixel values of the pixel points at the same position of different differential images are not different but have different values can be avoided, and therefore the position accuracy of the determined shot main body is improved.

In addition, in the related art, since the determination of the position of the subject using the image segmentation model is usually performed based on a single image, and it is difficult for the apparatus to know which part of the image is the subject for which the user needs to perform the position determination with respect to the single image, it is generally necessary for the user to inform the apparatus of which part of the image is the subject for which the position determination is required by manually clicking or the like, and on the basis of this, the image segmentation model can determine the position of the subject in the image based on the instruction of the user. In the disclosure, the position of the shot subject is the inconsistent part of the picture content in the multiple images to be processed, so that the position of the shot subject in the image can be determined without executing additional indication operation by a user, and the problem that the position of the shot subject can be determined based on an image segmentation model only by requiring the user to additionally send the indication operation in the related art is avoided.

In the following, taking an example of realizing an image cloning effect in a smart phone, the technical scheme of the disclosure is introduced.

Fig. 2 is a flowchart illustrating an image cloning method according to an exemplary embodiment of the present disclosure. As shown in fig. 2, the method may include the steps of:

step 201, three photographs are taken of a subject.

In this embodiment, the user can fix the smart phone through tools like a tripod to guarantee that the position of the smart phone is fixed, and the shooting angle is fixed. On the basis, the user can adjust the position of the shot main body for a plurality of times, and take a picture every time the position is adjusted.

For example, assuming that the subject is a fire extinguisher, the three photos taken by the user in the above manner may be pictures x, y, and z as shown in fig. 3.

Step 202, each photo is respectively subjected to interframe difference with other photos.

In this embodiment, after three photographs are taken, the inter-frame difference of the gradation values can be performed between the respective photographs. It should be understood that the background content of the other areas except the subject is identical among the three photos, and therefore, any differential image obtained by the inter-frame difference can reflect the positions of the subject in the two images, respectively.

Taking the above example, this step can make the inter-frame differences for photo x and photo y, photo x and photo z, and photo y and photo z, respectively, to obtain the differential images xy, xz, yz as shown in fig. 4. Wherein the differential image xy can reflect the positions of the fire extinguisher in the photo x and the photo y respectively; the differential image xz may reflect the positions of the fire extinguisher in photo x and photo z, respectively; the differential image yz may reflect the location of the fire extinguisher in photo y and photo z, respectively.

Step 203, binarizing the three differential images obtained by the inter-frame difference.

In this embodiment, the binarization process may be further performed on the three differential images, so that the images after binarization are absolute black-and-white images, and thus the position of the subject in the photograph is better represented.

Taking the above example, binarized images xy ', xz ', yz ' as shown in fig. 5 can be obtained by binarizing.

Step 204, performing an and operation on each binarized picture and other binarized pictures.

Taking the above example, the gray value and operation may be performed on the binarized images xy 'and xz', so as to obtain a target image x 'including the target area as shown in fig. 6A, and obviously, the position of the target area in the target image x' may represent the position of the fire extinguisher in the photo x. Similarly, gray value and operation can be performed on the binarized images xy ' and yz ' to obtain a target image y ' corresponding to the photo y; the binarized images xz ' and yz ' may be subjected to a gray value-summing operation to obtain a target image z ' corresponding to the photograph z.

In step 205, three target images including the target region are obtained by performing the summation.

In step 206, the three photos are subjected to background separation based on the position of the target area in the target image.

Taking the above example, the photo x may be background separated based on the target region in the target image x' to obtain the foreground portion x″; performing background separation on the photo y based on a target region in the target image y 'to obtain a foreground part y'; the photo z is background separated based on the target region in the target image z' to obtain the foreground portion z ".

Step 207, adding the foreground part obtained by separating the background to any photo.

With the above examples in mind, the foreground part x ", the foreground part y" and the foreground part z "separated by the background can be added to any one of the photos x, y and z to realize the cloning of the fire extinguisher in the image. Of course, in some possible embodiments, since there is a fire extinguisher in photo x itself, if a clone of the fire extinguisher is implemented in photo x, only foreground portion y "and foreground portion z" need to be added to photo x, and accordingly, the and operation of binarizing images xy 'and xz' need not be performed in the foregoing steps. Of course, the cloning of the fire extinguisher in photo y or photo z is similar and will not be described in detail here. The clone image obtained in practice can be shown in FIG. 6B.

According to the technical scheme, under the condition that three photos are obtained through shooting, the position of the shot main body in the photo is determined through carrying out operations such as inter-frame difference, binarization and AND on the photo in sequence, so that the cloning effect of the shot main body in the photo is realized according to the position, and the problem that the image cloning effect needs to be realized by means of an image segmentation model in the related art is avoided.

Fig. 7 is a block diagram of an image processing apparatus shown in an exemplary embodiment of the present disclosure. Referring to fig. 7, the apparatus includes an acquisition unit 701, a differential unit 702, and an and unit 703.

An acquiring unit 701, configured to acquire at least three images to be processed, where background contents of the at least three images to be processed are consistent, and positions of the same subject in the at least three images to be processed are different;

the difference unit 702 performs inter-frame difference processing of pixel values on each image to be processed and other images to be processed respectively to obtain a plurality of difference images;

and an AND unit 703, configured to perform an AND operation of pixel values between at least two differential images obtained based on the same image to be processed, obtain a target image including a target region, and determine a position of the target region in the target image as a position of the subject in the same image to be processed.

Alternatively to this, the method may comprise,

the acquisition unit 701 is also used to: after at least three unprocessed images are acquired, carrying out position registration on background pictures contained in the at least three unprocessed images so that the background contents contained in the background pictures are positioned at the same position in different registered images obtained through the position registration; and taking the registered images obtained by position registration as the at least three images to be processed.

Optionally, the pixel value is a gray value or a color value.

As shown in fig. 8, fig. 8 is a block diagram of another image processing apparatus according to an exemplary embodiment of the present disclosure, which further includes, on the basis of the foregoing embodiment shown in fig. 7: a binarization unit 704, a segmentation unit 705, and a complementation unit 706.

Alternatively to this, the method may comprise,

further comprises: a binarization unit 704, configured to perform binarization processing on pixel values in the plurality of differential images, so as to obtain a plurality of binarized images;

the and unit 703 is further configured to: and performing pixel value AND operation between at least two binarized images obtained based on the same image to be processed.

Optionally, the method further comprises:

the segmentation unit 705 performs background segmentation on the corresponding images to be processed based on the target areas in the target images after obtaining the target images corresponding to the images to be processed respectively;

The acquisition unit 701 is also used to: and acquiring a plurality of foreground parts corresponding to the shot main body obtained through background segmentation, and adding the plurality of foreground parts into any image to be processed according to the positions of the plurality of foreground parts in the image to be processed to which the foreground parts belong.

Optionally, the method further comprises:

and a complementing unit 706, configured to determine a background area in the registered image, where the picture content is lost, and complement the picture content in the background area through an image complementing algorithm.

Optionally, the complementing unit 706 is further used for:

For the device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the objectives of the disclosed solution. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

Correspondingly, the disclosure also provides an image processing device, which comprises: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the image processing method according to any of the above embodiments, for example the method may comprise: acquiring at least three images to be processed, wherein the background content of the at least three images to be processed is consistent, and the positions of the same shot main body in the at least three images to be processed are different; carrying out inter-frame difference processing on each image to be processed and other images to be processed respectively to obtain a plurality of difference images; and performing pixel value ANDed operation between at least two differential images obtained based on the same image to be processed to obtain a target image containing a target area, and determining the position of the target area in the target image as the position of the shot subject in the same image to be processed.

Accordingly, the present disclosure also provides an electronic device including a memory, and one or more programs, where the one or more programs are stored in the memory, and configured to be executed by the one or more processors, the one or more programs including instructions for implementing the image processing method according to any of the foregoing embodiments, for example, the method may include: acquiring at least three images to be processed, wherein the background content of the at least three images to be processed is consistent, and the positions of the same shot main body in the at least three images to be processed are different; carrying out inter-frame difference processing on each image to be processed and other images to be processed respectively to obtain a plurality of difference images; and performing pixel value ANDed operation between at least two differential images obtained based on the same image to be processed to obtain a target image containing a target area, and determining the position of the target area in the target image as the position of the shot subject in the same image to be processed.

Fig. 9 is a block diagram illustrating an apparatus 900 for implementing an image processing method according to an exemplary embodiment. For example, apparatus 900 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, exercise device, personal digital assistant, or the like.

Referring to fig. 9, apparatus 900 may include one or more of the following components: a processing component 902, a memory 904, a power component 906, a multimedia component 908, an audio component 910, an input/output (I/O) interface 912, a sensor component 914, and a communication component 916.

The processing component 902 generally controls overall operations of the apparatus 900, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 902 may include one or more processors 920 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 902 can include one or more modules that facilitate interaction between the processing component 902 and other components. For example, the processing component 902 can include a multimedia module to facilitate interaction between the multimedia component 908 and the processing component 902.

The memory 904 is configured to store various types of data to support operations at the apparatus 900. Examples of such data include instructions for any application or method operating on the device 900, contact data, phonebook data, messages, pictures, videos, and the like. The memory 904 may be implemented by any type of volatile or nonvolatile memory device or combination thereof, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

The power supply component 906 provides power to the various components of the device 900. Power supply components 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for device 900.

The multimedia component 908 comprises a screen between the device 900 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 908 includes a front-facing camera and/or a rear-facing camera. The front-facing camera and/or the rear-facing camera may receive external multimedia data when the apparatus 900 is in an operational mode, such as a photographing mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

The audio component 910 is configured to output and/or input audio signals. For example, the audio component 910 includes a Microphone (MIC) configured to receive external audio signals when the device 900 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 904 or transmitted via the communication component 916. In some embodiments, the audio component 910 further includes a speaker for outputting audio signals.

The I/O interface 912 provides an interface between the processing component 902 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.

The sensor assembly 914 includes one or more sensors for providing status assessment of various aspects of the apparatus 900. For example, the sensor assembly 914 may detect the on/off state of the device 900, the relative positioning of the components, such as the display and keypad of the device 900, the sensor assembly 914 may also detect the change in position of the device 900 or one component of the device 900, the presence or absence of user contact with the device 900, the orientation or acceleration/deceleration of the device 900, and the change in temperature of the device 900. The sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 914 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 914 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 916 is configured to facilitate communication between the apparatus 900 and other devices in a wired or wireless manner. The apparatus 900 may access a wireless network based on a communication standard, such as WiFi,2G or 3G,4G LTE, 5G NR (New Radio), or a combination thereof. In one exemplary embodiment, the communication component 916 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component 916 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, apparatus 900 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements for executing the methods described above.

In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as a memory 904 including instructions executable by the processor 920 of the apparatus 900 to perform the above-described method. For example, the non-transitory computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any adaptations, uses, or adaptations of the disclosure following the general principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

The foregoing description of the preferred embodiments of the present disclosure is not intended to limit the disclosure, but rather to cover all modifications, equivalents, improvements and alternatives falling within the spirit and principles of the present disclosure.

Claims

1. An image processing method, comprising:

2. The method of claim 1, wherein the step of determining the position of the substrate comprises,

3. The method as recited in claim 1, further comprising:

4. The method as recited in claim 1, further comprising:

5. The method as recited in claim 4, further comprising:

6. The method as recited in claim 4, further comprising:

7. The method of claim 1, wherein the pixel value is a gray value or a color value.

8. A position recognition apparatus of a subject, characterized by comprising:

and the AND unit performs AND operation of pixel values between at least two differential images obtained based on the same image to be processed to obtain a target image containing a target area, and determines the position of the target area in the target image as the position of the shot main body in the same image to be processed.

9. An electronic device, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to implement the method of any of claims 1-7 by executing the executable instructions.

10. A computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the steps of the method of any of claims 1-7.