CN112188200A

CN112188200A - Image processing method, device, equipment and storage medium

Info

Publication number: CN112188200A
Application number: CN202011066241.8A
Authority: CN
Inventors: 丁晶晶
Original assignee: OneConnect Financial Technology Co Ltd Shanghai
Current assignee: OneConnect Smart Technology Co Ltd; OneConnect Financial Technology Co Ltd Shanghai
Priority date: 2020-09-30
Filing date: 2020-09-30
Publication date: 2021-01-05
Also published as: WO2022068357A1

Abstract

The embodiment of the application discloses an image processing method, an image processing device, image processing equipment and a storage medium. The method comprises the following steps: receiving to-be-processed video data sent by a first terminal, and acquiring an image type of a target image in the to-be-processed video data; if the image type of the target image is an adhesion image type, acquiring block codes of image blocks of a first image in the video data to be processed; determining the block code of the key image block of the target image according to the block code of the image block of the first image; restoring the frame code of the target image according to the block code of the key image block and the block code of the image block of the first image; and determining the playing sequence of the target image in the video data to be processed by adopting the frame coding of the target image. Adopt this application, can restore the playing order of the target image of adhesion image type in pending video data, this application is applicable to wisdom medical field, can further promote the construction in wisdom city.

Description

Image processing method, device, equipment and storage medium

Technical Field

The present application relates to the field of image processing technologies, and in particular, to an image processing method, an image processing apparatus, an image processing device, and a storage medium.

Background

In the prior art, due to the response time delay of the led display, the content of the previous frame does not disappear, the pixel of the content of the next frame starts to be displayed, and the contents of a plurality of frames of images are overlapped, so that the playing sequence of the images in the video data cannot be determined. In addition, the video quality sometimes becomes very poor after the video is transmitted, and the conventional frame coding method (such as characters, two-dimensional codes and bar codes) of each frame of image fails, which also results in that the playing sequence of the image in the video data cannot be determined. If a blood vessel of a patient is photographed to obtain a medical image of the patient, when the medical image of the patient is transmitted to the playing terminal, the frame number of the image in the medical image is lost due to extremely poor video quality, the medical image is blurred, and the patient cannot be diagnosed with a disease according to the medical image. The inability to determine the playing sequence of the images in the video data results in the inability to restore the playing speed of the images in the captured video data and the inability to achieve frame synchronization between the sending end and the receiving end. For example, after an image in an advertisement video of a certain product is encoded according to a conventional frame encoding method, when the advertisement video of the product is played and contents of multiple frames of images in the advertisement video of the product are overlapped, a playing sequence of the multiple frames of images with overlapped contents cannot be determined, frame synchronization between a video sending end and a video playing end cannot be realized, and a screen is blurred during playing (i.e., a video is blurred), so that the video cannot be basically viewed.

Disclosure of Invention

The embodiment of the application provides an image processing method, an image processing device, image processing equipment and a storage medium, which can restore the playing sequence of a target image in video data to be processed and realize frame synchronization.

In a first aspect, an embodiment of the present application provides an image processing method, including:

receiving video data to be processed sent by a first terminal, and acquiring an image type of a target image in the video data to be processed;

if the image type of the target image is an adhesion image type, acquiring block codes of image blocks of a first image in the video data to be processed; the block coding of the image block of the first image is obtained by coding the image block of the first image by the first terminal according to the position of the first image in the video data to be processed, the first image is the last frame image of the target image, and the type of the adhesion image is used for indicating that the target image is overlapped with a plurality of frame images in the video data to be processed;

determining the block code of the key image block of the target image according to the block code of the image block of the first image; the key image blocks are image blocks of the target image, which have different block codes from those of the image blocks of the first image;

restoring the frame code of the target image according to the block code of the key image block and the block code of the image block of the first image;

and determining the playing sequence of the target image in the video data to be processed by adopting the frame coding of the target image.

Wherein the determining of the block code of the key image block of the target image according to the block code of the image block of the first image comprises:

obtaining block codes of image blocks of the first image; the block coding of the image block of the first image is obtained by the first terminal according to the position of the first image in the video data to be processed;

the target image is subjected to blocking processing to obtain at least two candidate image blocks;

detecting the target image to obtain the definition of the target image;

determining key image blocks in the at least two candidate image blocks according to the definition of the target image and the block codes of the image blocks of the first image;

and acquiring the pixel value of the key image block, and determining the block code of the key image block according to the pixel value of the key image block.

Wherein the determining a key image block of the at least two candidate image blocks according to the definition of the target image and the block coding of the image block of the first image comprises:

determining the number of the images adhered to the target image according to the definition of the target image;

determining the position of the first image in the video data to be processed according to the block codes of the image blocks of the first image;

and determining key image blocks in the at least two candidate image blocks according to the positions and the number of the images adhered by the target image, wherein the number of the key image blocks is greater than or equal to the number of the images adhered by the target image.

Wherein determining a block code of the key image block according to pixel values of the key image block comprises:

if the pixel value of the key image block is larger than a first threshold, taking a first coding value as the block coding of the key image block;

and if the pixel value of the key image block is smaller than or equal to the first threshold, using the second coded value as the block code of the key image block.

The restoring the frame code of the target image according to the block code of the key image block and the block code of the image block of the first image includes:

acquiring the position of the key image block in the target image;

block codes of image blocks of the first image, the positions of which are different from the positions of the key image blocks in the target image, are used as block codes of residual image blocks in the target image, wherein the residual image blocks are image blocks in the target image except the key image blocks;

and combining the block codes of the residual image blocks in the target image and the block codes of the key image blocks to obtain the frame codes of the target image.

Wherein the method further comprises:

if the image type of the target image is a non-adhesive image type, dividing the target image to obtain a plurality of image blocks;

acquiring position information of each image block in the plurality of image blocks, wherein the position information is positioned in the target image;

determining a calibration image block from the plurality of image blocks according to the position information of each image block in the plurality of image blocks in the target image;

acquiring pixel values of the calibration image blocks and pixel values of image blocks in the target image except the calibration image blocks;

and determining the frame code of the target image according to the pixel values of the calibration image blocks and the pixel values of the image blocks in the target image except the calibration image blocks.

Determining a frame code of the target image according to the pixel values of the calibration image blocks and the pixel values of the image blocks in the target image except the calibration image blocks comprises the following steps:

acquiring the ratio of the pixel value of the calibration image block to the pixel value of the target image block; the target image block is any one of the image blocks of the target image except the calibration image block;

if the ratio is larger than a second threshold, taking a first coding value as the block coding of the target image block;

if the ratio is smaller than or equal to the second threshold, taking a second coded value as the block code of the target image block;

and determining the frame code of the target image according to the block code of the target image block.

In a second aspect, an embodiment of the present application provides an image processing apparatus, including:

the first acquisition module is used for receiving to-be-processed video data sent by a first terminal and acquiring the image type of a target image in the to-be-processed video data;

the second obtaining module is used for obtaining block codes of image blocks of a first image in the video data to be processed if the image type of the target image is an adhesion image type; the block coding of the image block of the first image is obtained by coding the image block of the first image by the first terminal according to the position of the first image in the video data to be processed, the first image is the last frame image of the target image, and the type of the adhesion image is used for indicating that the target image is overlapped with a plurality of frame images in the video data to be processed;

the first determining module is used for determining the block codes of the key image blocks of the target image according to the block codes of the image blocks of the first image; the key image blocks are image blocks of the target image, which have different block codes from those of the image blocks of the first image;

the restoring module is used for restoring the frame code of the target image according to the block code of the key image block and the block code of the image block of the first image;

and the second determining module is used for determining the playing sequence of the target image in the video data to be processed by adopting the frame coding of the target image.

In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor adapted to implement one or more instructions; and the number of the first and second groups,

a computer storage medium storing one or more instructions adapted to be loaded by the processor and to perform the steps of:

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, including: the computer storage medium stores one or more instructions adapted to be loaded by a processor and to perform the steps of:

In the method, the video data to be processed sent by a first terminal is received, and the image type of a target image in the video data to be processed is obtained; if the image type of the target image is an adhesion image type, acquiring block codes of image blocks of a first image in the video data to be processed; determining the block code of a key image block of a target image according to the block code of the image block of the first image; restoring the frame code of the target image according to the block code of the key image block and the block code of the image block of the first image, and quickly obtaining the frame code of the target image according to the block code of the first image and the block code of the key image block in the target image after obtaining the block code of the key image block in the target image because the block code of the image block of the first image is known; and determining the playing sequence of the target image in the video data to be processed by adopting the frame coding of the target image. By adopting the method and the device, the frame coding of the target image can be restored to determine the playing sequence (serial number) of the target image in the video data to be processed, so that the playing speed (frame rate) of multiple frames of images in the video data to be processed is obtained, the frame synchronization of a sending end and a receiving end is realized, and the problem that the serial number of the image is lost due to poor image definition caused by adhesion of the image or compression of the image, and the frame synchronization cannot be realized is avoided.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic flowchart of an image processing method according to an embodiment of the present application;

fig. 2 is a schematic diagram of a gray coding method provided in an embodiment of the present application;

fig. 3 is a schematic diagram of a method for encoding each frame of image in video data to be processed according to an embodiment of the present application;

fig. 4 is a schematic diagram of a block encoding method for acquiring a key image block of a target image according to an embodiment of the present application;

FIG. 5 is a schematic flowchart of another image processing method provided in the embodiments of the present application;

fig. 6 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of an electronic device according to another embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Referring to fig. 1, a flowchart of an image processing method according to an embodiment of the present application is shown, where the image processing method is executed by an electronic device, and the image processing method includes steps S101 to S104.

S101, receiving to-be-processed video data sent by a first terminal, and acquiring an image type of a target image in the to-be-processed video data.

The electronic equipment in the scheme can receive to-be-processed video data sent by the first terminal, wherein the to-be-processed video data is obtained by shooting through a camera module in the first terminal, and the to-be-processed video data comprises multi-frame images. The target image is any one of the multi-frame images of the video data to be processed sent by the first terminal and received by the electronic equipment. The image types of the target images comprise a blocking image type and a non-blocking image type, and the target images of the blocking image type are displayed as the target images begin to be displayed without losing the content of the previous frame of images, so that the content of the target images is overlapped with the content of other images, namely the target images are blocked. If the target images are adhered, the type of the target images is an adhered image type; and if the target image is not adhered, the type of the target image is a non-adhered image type.

And S102, if the image type of the target image is the adhesion image type, acquiring block codes of image blocks of a first image in the video data to be processed.

S103, determining the block code of the key image block of the target image according to the block code of the image block of the first image.

The first image belongs to video data to be processed, and the first image is a previous frame image of the target image. The first image refers to an image of a frame previous to a target image in an image of video data to be processed sent by a first terminal and received by an electronic device in this embodiment, that is, the target image is received after the first image is received. However, after each frame of image is sequenced according to the shooting time of each frame of image in the video data to be processed, the target image may be the next frame of image of the first image or the last n frames of image of the first image, and n is a positive integer greater than 1. For example, the video data to be processed is captured every 1ms, if the first image is the image captured in the 1 st ms, the target image may be the image captured in the 3 rd ms or the image captured in the 4 th ms, n is a positive integer greater than 1 and smaller than i, and i may be determined according to the number of overlapping target images and other images. If the image type of the target image is an adhesion image type, acquiring a block code of an image block of a first image in the video data to be processed, and determining a block code of a key image block of the target image according to the block code of the image block of the first image, wherein the key block of the target image is an image block which is not used by the block code of the image block of the target image and the block code of the image block of the first image. When each frame of image in the video data to be processed is shot and acquired by the first terminal, each frame of image is encoded according to the shooting time sequence (namely the serial number of each frame of image) of each frame of image, and the frame code of each frame of image is acquired.

Optionally, the first terminal may use gray coding, and encode each frame of image according to the shooting time sequence of each frame of image (i.e., the sequence number of each frame of image), to obtain the frame code of each frame of image. Gray code means that in the coding of a group of numbers, if any two adjacent codes have only one binary digit difference. Fig. 2 is a schematic diagram of a gray coding method provided in an embodiment of the present application. As shown in fig. 2, when the decimal number is 1, the four-bit binary code is 0001, and the four-bit gray code is 0001, and when the decimal number is 2, the four-bit binary code is 0010, and the four-bit gray code is 0011. When the decimal number 1 is changed to 2, the four-bit binary code is changed from 0001 to 0010, the binary numbers of the third bit and the fourth bit are changed, and the four-bit gray code is changed from 0001 to 0011, and only the binary number of the third bit is changed. Compared with the code 0011, the code 0001 has only the third bit binary number changed in the code 0011, i.e., from 0 to 1, so that the key binary number in the code 0011 is the third bit. The following codes can be continuously observed, and it can be seen that only one binary number in any two adjacent codes in the gray code changes. As shown in fig. 3, a schematic diagram of a method for encoding each frame of image in video data to be processed according to an embodiment of the present application is provided, and as shown in fig. 3, each frame of image in the video data to be processed may be uniformly divided, each frame of image may be uniformly divided into 12 image blocks, and one image block of the 12 image blocks may be used as a calibration image block, and for example, a sixth image block may be used as a calibration image block. And then acquiring the shooting time of each frame of image, sequencing each frame of image according to the shooting time of each frame of image, and acquiring the serial number of each frame of image, wherein the serial number of each frame of image is a decimal number. And carrying out gray coding on the sequence number of each frame of image to obtain 11-bit gray coding of each frame of image, and taking the 11-bit gray coding of each frame of image as the frame coding of each frame of image. As shown in fig. 2, a picture with sequence number 1 in the video data to be processed is gray-coded with sequence number 1 of the picture, and an 11-bit gray code 00000000001 of the frame picture is obtained. And obtaining block codes of 11 image blocks except the calibrated pixel block in each frame of image according to the 11-bit gray code of each frame of image, wherein one binary number in the 11-bit gray code corresponds to the block code of one image block in the 11 image blocks.

For example, as shown in fig. 3, 12 image blocks in each frame image have corresponding position information, and the sixth image block in the image is taken as a calibration image block. The 1 st image block is positioned at the upper left of the image, so that the image blocks are positioned from left to right and from top to bottom, and the 12 th image block is positioned at the lower right of the image. And taking the first binary number of the 11-bit gray code as the block code of the first image block in the image, taking the second binary number as the block code of the second image block in the image, and so on, taking the binary number of the twelfth bit as the block code of the twelfth image block in the image, thereby obtaining the block code of each image block in the 11 image blocks in each frame of image. For example, for a picture with sequence number 2, the 11-bit gray code corresponding to the picture is 00000000011, the first binary number 0 is used as the block code of the first image block in the picture, and so on, the 11 th binary number 1 is used as the block code of the 12 th image block in the picture. Compared with the previous 11-bit gray code 00000000001, the 11-bit gray code 00000000011 only changes the 10 th bit, i.e., from 0 to 1, so that the key image block in the image corresponding to the code 00000000011 is the tenth block. If the block code of the image block is 0, marking the pixel value of the image block as 255; a block of an image block is coded to 1, the pixel values of the image block are marked 0.

Fig. 4 is a schematic diagram of a block coding method for obtaining a key image block of a target image according to an embodiment of the present application, and as shown in fig. 4, the steps of obtaining the block coding method content of the key image block of the target image include S21 to S25.

S21, a block code of an image block of the first image is obtained.

And S22, performing blocking processing on the target image to obtain at least two candidate image blocks.

And S23, detecting the target image to obtain the definition of the target image.

The block coding of the image block of the first image is obtained by the first terminal coding the first image according to the position (i.e. the serial number of the first image) of the first image in the video data to be processed. And after receiving the target image, carrying out blocking processing on the target image to obtain at least two candidate image blocks in the target image. Specifically, the target image may be divided uniformly to obtain at least two image blocks with equal areas. And then, detecting the target image according to a definition algorithm to obtain the definition of the target image. Specifically, the sharpness algorithm includes: brenner (gradient function), Tenengrad (gradient function), Laplacian (gradient function), SMD (grayscale variance function), and the like.

S24, a key image block of the at least two candidate image blocks is determined based on the sharpness of the target image and the block coding of the image block of the first image.

S25, the pixel values of the key image block are obtained, and the block coding of the key image block is determined according to the pixel values of the key image block.

And determining a key image block of the at least two candidate image blocks according to the definition of the target image and the block coding of the image block of the first image. And after obtaining the key image blocks in the target image, acquiring pixel values in the target image, and determining block codes of the key image blocks in the target image according to the pixel values of the key image blocks in the target image.

Optionally, determining the number of the images adhered to the target image according to the definition of the target image; determining the position of a first image in video data to be processed according to the block coding of an image block of the first image; and determining key image blocks in the at least two candidate image blocks according to the positions and the number of the images adhered by the target image, wherein the number of the key image blocks is greater than or equal to the number of the images adhered by the target image.

And obtaining the definition of the target image according to a definition algorithm, and determining the number of the images adhered to the target image according to the definition of the target image. The poorer the definition of the target image is, the more the images adhered to the target image are; the clearer the definition of the target image is, the less the image adhered to the target image is. Therefore, the value of the definition of the target image can be obtained according to the definition algorithm, and the number of the images adhered to the target image is determined according to the value of the definition of the target image. And if the definition value of the target image is smaller than the first definition value, determining that the number of the images adhered to the target image is 2. And determining the position of the first image in the image data to be processed according to the block codes of the image blocks of the first image, determining key image blocks in at least two candidate image blocks according to the position of the first image in the image data to be processed and the number of the images adhered by the target image, wherein the number of the key image blocks is greater than or equal to the number of the images adhered by the target image. For example, if it is determined that the position of the first image in the video data to be processed is the first frame image according to the block coding of the image block of the first image, and if it is determined that the number of images to which the target image is stuck is 2 according to the definition of the target image, it may be predicted that the target image may be the fourth frame image in the video data to be processed according to the position of the first image in the video data to be processed and the number of images to which the target image is stuck. However, errors may occur in prediction, so that key image blocks of at least two candidate image blocks in the target image may be determined, the number of the key image blocks is greater than or equal to the number of images adhered to the target image, and the errors may be reduced by determining the codes of the target image according to the key image blocks in the target image.

For example, the sixth block in the image block of each frame of video in the video to be processed is the standard image block, and if the frame code of the first image is 00000000001, it can be known that the frame code of the next frame of image of the first image in the video data to be processed is 00000000011, and the key image block in the frame of image is the eleventh image block; the frame codes of the last two frames of the first image are 00000000010, and the key image block of the frame image is the twelfth image block; the frame of the last three frame images of the first image is encoded as 00000000110, and the key image block of the frame image is the tenth image block. If the number of the images adhered to the target image is 2, the number of the key image blocks in the target image may be determined to be 3 or more than 3. And if the number of the key image blocks in the target image is determined to be 3, taking the tenth image block, the eleventh image block and the twelfth image block as the key image blocks of the target image. By analogy, the number and the position of the key image blocks in the target image can be determined according to the number of the key image blocks in the target image and the position of the key image blocks in the image after the first image in the data to be processed.

Optionally, if the pixel value of the key image block is greater than the first threshold, the first encoded value is used as the block encoding of the key image block; and if the pixel value of the key image block is smaller than or equal to the first threshold, the second coded value is used as the block code of the key image block.

Acquiring pixel values of key image blocks in a target image, and if the pixel values of the key image blocks in the target image are larger than a first threshold, taking the first coding values as block codes of the key image blocks in the target image; and if the pixel value of the key image block in the target image is smaller than or equal to the first threshold, using the second coding value as the block coding of the key image block in the target image. Specifically, when a target image is obtained by shooting at a first terminal, the target image is subjected to uniform blocking processing to obtain 12 image blocks with equal areas, a sixth image block in the target image is used as a calibration image block, the pixel value of the calibration image block is marked as 128, and 11 images except the calibration image block in the target image are marked according to the frame code of the target image. After the target image is received, the key image block in the target image is obtained after the frame coding of the target image is lost due to the adhesion of the target image and other images. After the number and the positions of the key image blocks in the target image are obtained, graying and position calibration can be performed on each key image block (the position calibration refers to taking the middle part of the key image block and discarding the edge part of the key image block). After the position of the key image block is calibrated, the influence of interference factors in the key image block can be eliminated, and the pixel value of the key image block can be acquired more accurately. After the key image blocks after position calibration are obtained, filtering each key image block after position calibration to determine key pixels of each key image block, and then averaging the key pixels of each key image block to obtain a pixel value of each key image block, namely a pixel average value.

After the pixel values of the key image blocks are obtained, whether the pixel values of the key image blocks are larger than a first threshold value or not is judged to judge the block codes of the key image blocks. If the pixel value of the key image block in the target image is larger than a first threshold, taking the first coding value as the block coding of the key image block in the target image; and if the pixel value of the key image block in the target image is smaller than or equal to the first threshold, using the second coding value as the block coding of the key image block in the target image. For example, the first threshold may be 128, and if the pixel value of a key image block in the target image is greater than 128 and the pixel value tends to be 255, which indicates that the key image block has a trend of changing to the first encoded value 0, the first encoded value 0 is used as the block encoding of the key image block in the target image; if the pixel value of the key image block in the target image is less than or equal to 128 and the pixel value tends to 0, which indicates that the key image block has a tendency to change to the first encoded value 1, the second encoded value 1 is used as the block encoding of the key image block in the target image.

And S104, restoring the frame code of the target image according to the block code of the key image block and the block code of the image block of the first image.

And S105, determining the playing sequence of the target image in the video data to be processed by adopting the frame coding of the target image.

And obtaining the frame code of the target image according to the block code of the key image block and the block code of the residual image block in the target image except the key image block after obtaining the block code of the residual image block in the target image according to the block code of the key image block in the first image.

Optionally, acquiring the position of the key image block in the target image; block codes of image blocks of which the positions are different from the positions of the key image blocks in the target image in the image blocks of the first image are used as block codes of residual image blocks in the target image, wherein the residual image blocks are image blocks in the target image except the key image blocks; and combining the block codes of the residual image blocks in the target image and the block codes of the key image blocks to obtain the frame codes of the target image.

And acquiring the position of the key image block in the target image, and using the block code of the image block of the first image, the position of which is different from the position of the key image block in the target image, as the block code of the residual image block in the target image. And the residual image blocks are the image blocks in the target image except the key image block. After block codes of residual image blocks (except for calibration image blocks) in the target image are obtained, the block codes of the residual image blocks in the target image and the block codes of the key image blocks are combined to obtain frame codes of the target image. For example, the frame code of the first image is 00000000001, the key images in the target image are the tenth image block, the eleventh image block and the twelfth image block, and the tenth image block, the eleventh image block and the twelfth image block of the target image are 1, 1 and 0 respectively; and coding blocks of image blocks of the first image, the positions of which are different from the positions of the key image blocks in the target image, as block codes of the residual image blocks in the target image. I.e. the block codes of the first to ninth image blocks in the first image are used as the block codes of the first to ninth image blocks in the target image, the frame of the target image is coded 00000000110. The calibration image block is not encoded, that is, the calibration image block has no block encoding.

Optionally, the pixel value of each image block in the remaining image blocks (the calibrated image blocks) in the target image and the pixel value of the calibrated image block may also be obtained, and the block code of each image block in the remaining image blocks (the calibrated image blocks) in the target image is obtained according to the difference between the pixel value of each image block in the remaining image blocks (the calibrated image blocks) and the pixel value of the calibrated image block. And subtracting the pixel value of the calibrated image block from the pixel value of each image block in the residual image blocks (except the calibrated image block) in the target image to obtain the difference value between the pixel value of each image block in the residual image blocks (except the calibrated image block) in the calibrated image and the pixel value of the target image block, wherein the target image block is any one of the image blocks in the target image except the calibrated image block. And if the difference value is greater than 0, taking the first coding value 0 as the block coding of the target image block. And if the difference is less than or equal to 0, taking the second coded value 1 as the block code of the target image block, and so on to obtain the block code of each image block in the residual image blocks (divided by the nominal image block) in the target image.

And after the frame coding of the target image is obtained, namely 11-bit gray codes corresponding to the target image are obtained, and the decimal number corresponding to the target image is obtained according to the 11-bit gray codes corresponding to the target image. If the target image 11 is gray coded 000000000110, the decimal number corresponding to the target image is 4, and the playing sequence of the target image in the video data to be processed is determined to be the fourth according to the decimal number 4 corresponding to the target image. Therefore, the frame rate of the video frame in the to-be-processed video data sent by the first terminal can be reduced, so as to realize frame synchronization between the sending terminal (the first terminal) and the receiving terminal (the electronic device in the embodiment of the application). Due to the fact that the target image is adhered to other images, the definition of the target image is poor, the playing sequence of the target image in the video data to be processed cannot be determined, the traditional coding modes (such as characters, two-dimensional codes and bar codes) are invalid, the playing sequence of the target image in the video data to be processed cannot be determined, and therefore frame synchronization between a sending end and a receiving end cannot be achieved. The method and the device can not cause the coding failure of the target image, and can still obtain the frame coding of the target image, thereby determining the playing sequence of the target image in the video data to be processed and realizing the frame synchronization between the sending end and the receiving end. Frame synchronization refers to a process of providing a start mark of each frame at a transmitting end and detecting and acquiring the mark at a receiving end in order to correctly separate each path of time slot signals in a digital time division multiplexing communication system, which is called frame synchronization.

The method and the device can be used in remote medical treatment and remote monitoring scenes, after the first terminal shoots and acquires the medical image of the patient, each frame of image in the medical image of the patient can be coded by using Gray coding, and then the coded medical image is sent to the electronic equipment in the embodiment. After receiving the medical image sent by the first terminal, the electronic device in this embodiment obtains a frame code of each frame of image in the medical image according to a block code of a key image block of a target image in the medical image and a block code of a first frame of image if the medical image of the patient is adhered; and if the medical images of the patient are not adhered, obtaining the frame codes of each frame of image according to the pixel values of the image blocks of each frame of image in the medical images and the pixel values of the calibration pixel blocks. Therefore, frame synchronization between the first terminal (transmitting end) and the electronic equipment (receiving end) of the embodiment can be realized, so that remote medical treatment and remote monitoring of the patient can be realized.

In the embodiment of the application, the video data to be processed sent by a first terminal is received, and the image type of a target image in the video data to be processed is obtained; if the image type of the target image is an adhesion image type, acquiring block codes of image blocks of a first image in the video data to be processed; determining the block code of a key image block of a target image according to the block code of the image block of the first image; restoring the frame code of the target image according to the block code of the key image block and the block code of the image block of the first image, and quickly obtaining the frame code of the target image according to the block code of the first image and the block code of the key image block in the target image after obtaining the block code of the key image block in the target image because the block code of the image block of the first image is known; and determining the playing sequence of the target image in the video data to be processed by adopting the frame coding of the target image. By adopting the method and the device, the frame coding of the target image can be restored to determine the playing sequence (serial number) of the target image in the video data to be processed, so that the playing speed (frame rate) of multiple frames of images in the video data to be processed is obtained, the frame synchronization of a sending end and a receiving end is realized, and the problem that the serial number of the image is lost due to poor image definition caused by adhesion of the image or compression of the image, and the frame synchronization cannot be realized is avoided.

As shown in fig. 5, a schematic flow chart of another image processing method provided in this embodiment is shown, and as shown in fig. 5, the schematic flow chart of the other image processing method includes steps S201 to S210.

S201, receiving video data to be processed sent by a first terminal, and acquiring an image type of a target image in the video data to be processed.

S202, if the image type of the target image is the adhesion image type, acquiring block codes of image blocks of a first image in the video data to be processed.

And S203, determining the block code of the key image block of the target image according to the block code of the image block of the first image.

And S204, restoring the frame code of the target image according to the block code of the key image block and the block code of the image block of the first image.

S205, determining the playing sequence of the target image in the video data to be processed by adopting the frame coding of the target image.

The specific contents of steps S201 to S205 in the embodiment of the present application can refer to the contents in the embodiment of fig. 1, and will not be described herein again.

And S206, if the image type of the target image is a non-blocking image type, dividing the target image to obtain a plurality of image blocks.

If the image type of the target image is a non-blocking image, that is, the target image is not overlapped with other images, but the definition of the target image is deteriorated due to operations such as compression and the like in the transmission process, and the playing sequence of the target image in the image data to be processed cannot be determined. The target image is divided to obtain a plurality of image blocks of the target image. When the first terminal obtains the target image by shooting, the target image is coded according to the serial number of the target image. Specifically, the first terminal may encode the plurality of image blocks of the target image by using gray coding, and obtain a block code of each of the plurality of image blocks of the target image.

S207, position information of each image block in the plurality of image blocks in the target image is acquired.

And S208, determining a calibration image block from the plurality of image blocks according to the position information of each image block in the plurality of image blocks, wherein the position information is located in the target image.

The method comprises the steps of obtaining position information of each image block in a target image, and determining a calibration image block from the plurality of image blocks according to the position information of each image block in the target image. For example, as shown in fig. 3, the target image is divided uniformly, and 12 image blocks having the same area are obtained. And sequencing the 12 image blocks to obtain the position information of the 12 image blocks, for example, the position information of the first image block is the image block at the upper left corner of the target image. And one image block of the 12 image blocks of the target image is designated as a calibration image block, and if a sixth image block is designated as the calibration image block, other image blocks can also be designated as the calibration image blocks. And encoding a plurality of image blocks (except for the nominal image block) of the target image by adopting 11-bit gray coding, and obtaining the block code (except for the nominal image block) of each image block in the target image. And if the first terminal takes the sixth image block in the target image as the calibration image block, determining the sixth image block in the target image as the calibration image block in the position information of each image block in the plurality of image blocks, which is located in the target image.

S209, acquiring the pixel values of the calibration image blocks and the pixel values of the image blocks in the target image except the calibration image blocks.

After obtaining a calibration image block of the target image and image blocks in the target image except the calibration image block, graying the calibration image block and the image blocks except the calibration image block, and then carrying out position calibration on the grayed calibration image block and the image blocks except the calibration image block, wherein the position calibration refers to only taking the middle part of the image block and removing the edge part of each image block. And then filtering the calibrated image blocks after position calibration and the image blocks except the calibrated image blocks, taking out key pixels of each image block, and averaging the key pixels of each image block to obtain the pixels of the calibrated image blocks and the pixel values of the image blocks except the calibrated image blocks.

S210, determining the frame code of the target image according to the pixel values of the calibration image blocks and the pixel values of the image blocks in the target image except the calibration image blocks.

And determining the frame code of the target image according to the pixel values of the calibration image blocks and the pixel values of the image blocks in the target image except the calibration image blocks. When a plurality of image blocks in each frame of image in video data to be processed are coded according to gray coding, if the block coding of the image block is 0, marking the pixel value of the image block as 255; a block of an image block is coded to 1, the pixel values of the image block are marked 0. And the pixel values of the marked image blocks are marked as 128, the marked pixel values of the image blocks in each frame image do not change too much in the transmission process. Therefore, the block code of each image block can be obtained according to the pixel values of the calibration image blocks and the pixel values of the image blocks in the target image except the calibration image blocks, so that the frame code of the target image can be obtained.

Optionally, obtaining a ratio between a pixel value of the calibration image block and a pixel value of the target image block; the target image block is any one of the image blocks in the target image except the calibration image block; if the ratio is larger than a second threshold, the first coding value is used as the block coding of the target image block; if the ratio is smaller than or equal to a second threshold, the second coding value is used as the block coding of the target image block; and determining the frame code of the target image according to the block code of the target image block.

And subtracting the pixel value of the calibration image block from the pixel value of the target image block to obtain the ratio of the pixel value of the calibration image block to the pixel value of the target image block, wherein the target image block is any one of the image blocks in the target image except the calibration image block. And if the ratio is larger than a second threshold value, taking the first coding value as the block coding of the target image block, wherein the second threshold value is 1, and the first coding value is 0. I.e. the pixel values of the target image block are larger than the pixel values of the nominal image block, i.e. the pixels of the target image block are larger than 128, and therefore the first encoded value 0 is encoded as a block of the target image block. And if the ratio is smaller than or equal to the second threshold, taking the second coded value as the block code of the target image block, wherein the second coded value is 1. I.e. the pixel values of the target image block are smaller than or equal to the pixel values of the target image block, i.e. the pixels of the target image block are smaller than or equal to 128, and thus the first encoded value 0 is encoded as a block of the target image block. And by analogy, after the block codes of a plurality of image blocks (except the calibration image block) in the target image are obtained, combining the block codes of the plurality of image blocks (except the calibration image block) in the target image according to the position information of each image block, and obtaining the frame code of the target image.

In the method, the video data to be processed sent by a first terminal is received, and the image type of a target image in the video data to be processed is obtained; if the image type of the target image is an adhesion image type, acquiring block codes of image blocks of a first image in the video data to be processed; determining the block code of a key image block of a target image according to the block code of the image block of the first image; restoring the frame code of the target image according to the block code of the key image block and the block code of the image block of the first image, and quickly obtaining the frame code of the target image according to the block code of the first image and the block code of the key image block in the target image after obtaining the block code of the key image block in the target image because the block code of the image block of the first image is known; determining the playing sequence of the target image in the video data to be processed by adopting the frame coding of the target image; if the image type of the target image is a non-adhesive image type, dividing the target image to obtain a plurality of image blocks, and determining a calibration image block from the plurality of image blocks; acquiring pixel values of the calibration image blocks and pixel values of image blocks in the target image except the calibration image blocks; and determining the frame code of the target image according to the pixel values of the calibration image blocks and the pixel values of the image blocks in the target image except the calibration image blocks. By adopting the method and the device, the frame coding of the target image can be restored to determine the playing sequence (serial number) of the target image in the video data to be processed, so that the playing speed (frame rate) of multiple frames of images in the video data to be processed is obtained, the frame synchronization of a sending end and a receiving end is realized, and the problem that the serial number of the image is lost due to poor image definition caused by adhesion of the image or compression of the image, and the frame synchronization cannot be realized is avoided.

Fig. 6 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present disclosure, where the apparatus according to the embodiment of the present disclosure may be used in the above-mentioned electronic device. In this embodiment, the image processing apparatus includes:

the first obtaining module 11 is configured to receive video data to be processed sent by a first terminal, and obtain an image type of a target image in the video data to be processed;

a second obtaining module 12, configured to obtain a block code of an image block of a first image in the to-be-processed video data if the image type of the target image is an adhesion image type; the block coding of the image block of the first image is obtained by coding the image block of the first image by the first terminal according to the position of the first image in the video data to be processed, the first image is the last frame image of the target image, and the type of the adhesion image is used for indicating that the target image is overlapped with a plurality of frame images in the video data to be processed;

a first determining module 13, configured to determine a block code of a key image block of the target image according to a block code of an image block of the first image; the key image blocks are image blocks of the target image, which have different block codes from those of the image blocks of the first image;

wherein the first determining module 13 includes: the device comprises a first acquisition unit, a block processing unit, a detection unit, a first determination unit and a second determination unit.

A first obtaining unit configured to obtain a block code of an image block of the first image; the block coding of the image block of the first image is obtained by the first terminal according to the position of the first image in the video data to be processed;

the block processing unit is used for carrying out block processing on the target image to obtain at least two candidate image blocks;

the detection unit is used for detecting the target image to obtain the definition of the target image;

a first determining unit, configured to determine a key image block of the at least two candidate image blocks according to a definition of the target image and a block code of an image block of the first image;

and the second determining unit is used for acquiring the pixel values of the key image blocks and determining the block codes of the key image blocks according to the pixel values of the key image blocks.

Wherein the first determining unit is specifically configured to:

Wherein the second determining unit is specifically configured to:

The restoring module 14 is configured to restore the frame code of the target image according to the block code of the key image block and the block code of the image block of the first image;

wherein the reduction module 14 comprises: the second acquisition unit, the third determination unit and the combination unit.

The second acquisition unit is used for acquiring the position of the key image block in the target image;

a third determining unit, configured to use block codes of image blocks, of which positions in the image blocks of the first image are different from positions of the key image blocks in the target image, as block codes of remaining image blocks in the target image, where the remaining image blocks are image blocks in the target image other than the key image blocks;

and the combination unit is used for combining the block codes of the residual image blocks in the target image and the block codes of the key image blocks to obtain the frame codes of the target image.

A second determining module 15, configured to determine, by using frame coding of the target image, a playing order of the target image in the video data to be processed.

Wherein the apparatus further comprises:

the dividing processing module is used for dividing the target image to obtain a plurality of image blocks if the image type of the target image is a non-adhesion image type;

the third acquisition module is used for acquiring the position information of each image block in the plurality of image blocks, wherein the position information is positioned in the target image;

the third determining module is used for determining a calibration image block from the plurality of image blocks according to the position information of each image block in the plurality of image blocks in the target image;

the fourth obtaining module is used for obtaining the pixel values of the calibration image blocks and the pixel values of the image blocks in the target image except the calibration image blocks;

and the fourth determining module is used for determining the frame code of the target image according to the pixel values of the calibration image blocks and the pixel values of the image blocks in the target image except the calibration image blocks.

Wherein the fourth determining module comprises:

the third acquiring unit is used for acquiring the ratio of the pixel value of the calibration image block to the pixel value of the target image block; the target image block is any one of the image blocks of the target image except the calibration image block;

a fourth determining unit, configured to use the first encoded value as a block encoding of the target image block if the ratio is greater than a second threshold;

a fifth determining unit, configured to, if the ratio is smaller than or equal to the second threshold, use a second encoded value as block encoding of the target image block;

a sixth determining unit, configured to determine a frame code of the target image according to a block code of the target image block.

Please refer to fig. 7, which is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure, where the electronic device in the embodiment shown in fig. 7 may include: one or more processors 21; one or more input devices 22, one or more output devices 23, and a memory 24. The processor 21, the input device 22, the output device 23, and the memory 24 are connected by a bus 25.

The Processor 21 may be a Central Processing Unit (CPU), and may be other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The input device 22 may include a touch pad, a fingerprint sensor (for collecting fingerprint information of a user and direction information of the fingerprint), a microphone, etc., the output device 23 may include a display (LCD, etc.), a speaker, etc., and the output device 23 may output the data table after the correction process.

The memory 24 may include both read-only memory and random access memory and provides instructions and data to the processor 21. A portion of the memory 24 may also include a non-volatile random access memory, the memory 24 being for storing a computer program comprising program instructions, the processor 21 being for executing the program instructions stored by the memory 24 for performing an image processing method, namely for performing the following:

determining a block code of a key image block of the target image according to a block code of the image block of the first image, wherein the key image block is an image block of the target image, which is different from the block code of the image block of the first image;

Optionally, the processor 21 is configured to execute program instructions stored in the memory 24 for performing the following operations:

detecting the target image to obtain the definition of the target image;

acquiring the position of the key image block in the target image;

The processor 21, the input device 22, and the output device 23 described in this embodiment of the present application may execute the implementation described in the embodiment of the traffic sign identification method provided in this embodiment of the present application, and may also execute the implementation of the electronic device described in this embodiment of the present application, which is not described herein again.

Also provided in embodiments of the present application is a computer-readable storage medium storing a computer program, where the computer program includes program instructions, and the program instructions, when executed by a processor, implement the image processing method as shown in the embodiments of fig. 1 and 5.

The computer readable storage medium may be an internal storage unit of the electronic device according to any of the foregoing embodiments, for example, a hard disk or a memory of the control device. The computer-readable storage medium may also be an external storage device of the control device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like provided on the control device. Further, the computer-readable storage medium may also include both an internal storage unit and an external storage device of the control device. The computer-readable storage medium is used to store the computer program and other programs and data required by the control device. The computer readable storage medium may also be used to temporarily store data that has been output or is to be output.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application. It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the control device and the unit described above may refer to corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed control device and method may be implemented in other ways. For example, the above-described apparatus embodiments are illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed.

While the invention has been described with reference to specific embodiments, the scope of the invention is not limited thereto, and those skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the invention. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image processing method, comprising:

2. The method according to claim 1, wherein determining a block encoding of a key image block of the target image from a block encoding of an image block of the first image comprises:

detecting the target image to obtain the definition of the target image;

3. The method of claim 2, wherein determining key image blocks of the at least two candidate image blocks according to the sharpness of the target image and the block coding of the image block of the first image comprises:

4. The method according to claim 2, wherein said determining a block encoding of the key image block from pixel values of the key image block comprises:

5. The method according to claim 1, wherein restoring the frame code of the target image according to the block code of the key image block and the block code of the image block of the first image comprises:

acquiring the position of the key image block in the target image;

6. The method of claim 1, further comprising:

7. The method according to claim 6, wherein determining the frame coding of the target image according to the pixel values of the anchor image blocks and the pixel values of the image blocks in the target image other than the anchor image blocks comprises:

8. An image processing apparatus characterized by comprising:

9. A computer device, comprising: a processor, a memory, and a network interface;

the processor is connected with a memory and a network interface, wherein the network interface is used for providing data communication functions, the memory is used for storing a computer program, and the processor is used for calling the computer program to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program comprising program instructions which, when executed by a processor, perform the steps of the method according to any one of claims 1 to 7.