CN115052160B - Image coding method and device based on cloud data automatic downloading and electronic equipment - Google Patents

Image coding method and device based on cloud data automatic downloading and electronic equipment Download PDF

Info

Publication number
CN115052160B
CN115052160B CN202210425363.4A CN202210425363A CN115052160B CN 115052160 B CN115052160 B CN 115052160B CN 202210425363 A CN202210425363 A CN 202210425363A CN 115052160 B CN115052160 B CN 115052160B
Authority
CN
China
Prior art keywords
image
similar
images
hamming distance
coded
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210425363.4A
Other languages
Chinese (zh)
Other versions
CN115052160A (en
Inventor
张昊
王磊
刘亮
袁智敏
朱文林
孙祥洪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Tobacco Jiangxi Industrial Co Ltd
Original Assignee
China Tobacco Jiangxi Industrial Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Tobacco Jiangxi Industrial Co Ltd filed Critical China Tobacco Jiangxi Industrial Co Ltd
Priority to CN202210425363.4A priority Critical patent/CN115052160B/en
Publication of CN115052160A publication Critical patent/CN115052160A/en
Application granted granted Critical
Publication of CN115052160B publication Critical patent/CN115052160B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention relates to the technical field of cloud databases, in particular to an image coding method and device based on automatic downloading of cloud data and electronic equipment, comprising the following steps: and performing similar retrieval in a cloud data gallery by utilizing image characteristics to obtain a candidate similar image set, judging whether similar images exist or not, if not, encoding the image to be encoded by utilizing an intra-frame encoding algorithm to obtain an encoded image, and if so, preprocessing the similar images to obtain an encoded reference image and preprocessing parameters, performing inter-frame predictive encoding on the image to be encoded to obtain inter-frame encoding residual errors, constructing a compressed code stream by utilizing the preprocessing parameters, the inter-frame encoding residual errors and image indexes of the similar images, and storing the compressed code stream in the cloud data gallery. The invention also provides an image encoding device based on automatic downloading of cloud data, electronic equipment and a computer readable storage medium. The invention can solve the problems of low image data compression efficiency and low retrieval speed of the network cloud data gallery.

Description

Image coding method and device based on cloud data automatic downloading and electronic equipment
Technical Field
The present invention relates to the field of cloud databases, and in particular, to an image encoding method and apparatus based on automatic cloud data downloading, and an electronic device.
Background
With the popularization of image acquisition equipment, massive images are uploaded to the Internet for storage, so that the storage pressure of a network cloud data gallery is greatly increased.
At present, the network cloud data gallery can utilize a distributed storage mode to reduce the data storage pressure, but the storage of image data does not fully utilize the existing image data in the network cloud data gallery to improve the compression efficiency of the image data, and further reduce the storage pressure of the image data, so that the problems of low image data compression efficiency and low retrieval speed of the existing network cloud data gallery are caused.
Disclosure of Invention
The invention provides an image coding method and device based on automatic downloading of cloud data and electronic equipment, and mainly aims to solve the problems of low image data compression efficiency and low retrieval speed of a network cloud data gallery.
In order to achieve the above object, the present invention provides an image encoding method based on automatic downloading of cloud data, comprising:
Extracting image features of an image to be encoded, and performing similar retrieval in a pre-constructed cloud data gallery by utilizing the image features to obtain a candidate similar image set;
judging whether the candidate similar image set has a similar image of the image to be coded or not;
the judging whether the candidate similar image set has the similar image of the image to be coded or not comprises the following steps:
obtaining a similar image test set;
extracting local feature points of each image in the similar image test set;
the local feature points of each image in the similar image test set are aggregated to obtain a global descriptor of each image in the similar image test set;
sequentially extracting images to be compared in the similar image test set;
matching the global descriptors of the images to be compared with global descriptors of other images in the similar image test set to obtain a hamming distance set between the other images in the similar image test set and the images to be compared;
extracting the maximum hamming distance in the hamming distance set to obtain a similar hamming distance of the image to be compared;
extracting the maximum Hamming distance from the similar Hamming distances of all images in the similar image test set to obtain a similar Hamming distance threshold;
Judging whether a hamming distance smaller than the similar hamming distance threshold exists in the hamming distance sequence corresponding to the candidate similar image;
if the hamming distance sequence corresponding to the candidate similar images does not have hamming distances smaller than the similar hamming distance threshold, judging that the candidate similar images are concentrated and the similar images of the image to be coded do not exist;
and if the Hamming distance sequence corresponding to the candidate similar images has Hamming distance smaller than the similar Hamming distance threshold, judging that the candidate similar images are concentrated and the similar images of the image to be coded exist.
If the candidate similar images do not exist in the candidate similar image set, encoding the image to be encoded by utilizing a pre-built intra-frame encoding algorithm to obtain an encoded image, and storing the encoded image in the cloud data gallery to complete encoding of the image to be encoded;
if the candidate similar images are similar images of the image to be coded in the set, preprocessing the similar images to obtain a coding reference image and preprocessing parameters;
performing inter-frame predictive coding on the image to be coded by using the coding reference image to obtain inter-frame coding residual errors;
And extracting image indexes of the similar images in the cloud data gallery, constructing a compressed code stream by utilizing the preprocessing parameters, the inter-frame coding residual errors and the image indexes, and storing the compressed code stream in the cloud data gallery to finish the coding of the images to be coded.
Optionally, the extracting the image features of the image to be encoded includes:
detecting interest points of the image to be coded to obtain initial local feature points;
removing noise points in the initial local descriptors to obtain target local feature points;
compressing the target local feature points to obtain local descriptors;
aggregating the target local feature points to obtain a global descriptor;
and constructing the image characteristics of the image to be encoded according to the local descriptors and the global descriptors.
Optionally, performing similar retrieval in a pre-constructed cloud data gallery by using the image features to obtain a candidate similar image set, including:
extracting a pre-constructed global descriptor index table in the cloud data gallery;
matching global descriptors in the image features with global descriptors of images in the cloud data gallery by using the global descriptor index table to obtain a hamming distance sequence of the images in the cloud data gallery and the images to be coded;
Extracting a preset number of hamming distances and images corresponding to the preset number of hamming distances from the hamming distance sequence according to the sequence from small to large;
and taking the images corresponding to the hamming distances of the preset number as the candidate similar image set.
Optionally, before performing the similarity search in the pre-constructed cloud data gallery by using the image features to obtain the candidate similar image set, the method further includes:
extracting local feature points of all images in the cloud data gallery;
aggregating each local feature point to obtain a global descriptor corresponding to each image;
and constructing a global descriptor index table by utilizing the global descriptor corresponding to each image according to the pre-constructed multi-block index structure.
Optionally, if the candidate similar images exist in the set of similar images, before preprocessing the similar images, the method further includes:
extracting all hamming distances smaller than the similar hamming distance threshold value from the preset number of hamming distances to obtain a candidate hamming distance set;
sequentially extracting images corresponding to each candidate hamming distance in the candidate hamming distance set to obtain a global similar image set;
Extracting local feature points of each image in the global similar image set;
compressing local feature points of each image in the global similar image set to obtain local descriptors of each image in the global similar image set;
utilizing the local descriptors of each image in the global similar image set to carry out Hamming distance matching with the local descriptors of the images to be coded to obtain a local Hamming distance sequence;
extracting the minimum Hamming distance in the local Hamming distance sequence and an image corresponding to the minimum Hamming distance;
and taking the image corresponding to the minimum Hamming distance as a similar image of the image to be encoded.
Optionally, the preprocessing the similar image to obtain an encoded reference image and preprocessing parameters includes:
dividing the similar image according to the feature points matched with the image to be coded to obtain a similar block set;
obtaining an optimal transformation matrix corresponding to each similar block in the similar block set by utilizing a pre-constructed characteristic point matching distance formula;
deforming the similar images by utilizing the optimal transformation matrix corresponding to each similar block to obtain deformed reference images;
According to the difference of pixel values of the same positions of the image to be encoded and the deformed reference image, carrying out illumination compensation on the deformed reference image to obtain the encoded reference image;
and constructing the preprocessing parameters according to the optimal transformation matrix corresponding to each similar block and the numerical value difference of illumination compensation.
Optionally, the feature point matching distance formula is as follows:
d i ＝∥p(f i )-p(f i ′ )×H∥
wherein d i Representing the distance value of the characteristic points of the i-th block similar block and the corresponding block in the image to be coded, and p (f) i ) Representing the corresponding position of the i-th block similar block in the image to be coded, p (f) i ′ ) Represents the position of the i-th block similar block in the similar image, H represents the transformation matrix, f i Representing corresponding characteristic points of the i-th block similar block in the image to be coded, f i ′ And representing the characteristic points of the i-th block similar block in the similar image.
Optionally, the performing inter-frame prediction encoding on the image to be encoded by using the encoded reference image to obtain an inter-frame encoded residual, including:
performing block segmentation on the image to be coded and the coding reference image to obtain a block image set to be coded and a reference block image set;
Sequentially extracting block images to be encoded from the block image set to be encoded, and calculating the mean square error of each block of reference block image in the block image set to be encoded and the reference block image set to obtain the minimum mean square error and similar reference block images corresponding to the block images to be encoded;
and integrating the differences between all the blocks to be coded in the image to be coded and the corresponding similar reference block images to obtain the inter-frame coding residual errors.
In order to solve the above problems, the present invention also provides an image encoding apparatus based on automatic download of cloud data, the apparatus comprising:
the candidate similar image set retrieval module is used for extracting image features of an image to be encoded, and performing similar retrieval in a pre-constructed cloud data gallery by utilizing the image features to obtain a candidate similar image set;
the similar image existence judging module is used for judging whether the similar image of the image to be coded exists in the candidate similar image set; the judging whether the candidate similar image set has the similar image of the image to be coded or not comprises the following steps: obtaining a similar image test set; extracting local feature points of each image in the similar image test set; the local feature points of each image in the similar image test set are aggregated to obtain a global descriptor of each image in the similar image test set; sequentially extracting images to be compared in the similar image test set; matching the global descriptors of the images to be compared with global descriptors of other images in the similar image test set to obtain a hamming distance set between the other images in the similar image test set and the images to be compared; extracting the maximum hamming distance in the hamming distance set to obtain a similar hamming distance of the image to be compared; extracting the maximum Hamming distance from the similar Hamming distances of all images in the similar image test set to obtain a similar Hamming distance threshold; judging whether a hamming distance smaller than the similar hamming distance threshold exists in the hamming distance sequence corresponding to the candidate similar image; if the hamming distance sequence corresponding to the candidate similar images does not have hamming distances smaller than the similar hamming distance threshold, judging that the candidate similar images are concentrated and the similar images of the image to be coded do not exist; if the Hamming distance sequence corresponding to the candidate similar images has Hamming distance smaller than the similar Hamming distance threshold, judging that the candidate similar images are concentrated and similar images of the image to be coded exist;
The intra-frame coding module is used for coding the image to be coded by utilizing a pre-built intra-frame coding algorithm if the candidate similar image set does not contain the similar image of the image to be coded, so as to obtain a coded image, and storing the coded image in the cloud data gallery to finish the coding of the image to be coded;
the similar image preprocessing module is used for preprocessing the similar images if the candidate similar images are in the set of similar images of the image to be encoded, so as to obtain an encoding reference image and preprocessing parameters;
and the inter-frame prediction coding module is used for carrying out inter-frame prediction coding on the image to be coded by utilizing the coding reference image to obtain inter-frame coding residual errors.
The invention also provides an electronic device, which comprises:
At least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein, the liquid crystal display device comprises a liquid crystal display device,
the memory stores instructions executable by the at least one processor to implement the cloud data automatic download-based image encoding method described above.
In order to solve the above-mentioned problems, the present invention also provides a computer-readable storage medium having stored therein at least one instruction that is executed by a processor in an electronic device to implement the above-mentioned image encoding method based on automatic download of cloud data.
Compared with the background art, the method comprises the following steps: the method comprises the steps of searching a candidate similar image set in a cloud data gallery by utilizing image characteristics of an image to be encoded, further judging whether the candidate similar image set contains a similar image of the image to be encoded or not, compressing the image to be encoded according to a general intra-frame encoding algorithm if the candidate similar image set does not contain the similar image, compressing the image to be encoded by utilizing the similar image if the candidate similar image does not contain the similar image, improving compression efficiency, preprocessing the similar image to obtain an encoding reference image and preprocessing parameters, performing inter-frame predictive encoding by utilizing the encoding reference image to obtain an inter-frame encoding residual, constructing a compressed code stream according to image indexes, preprocessing parameters and inter-frame encoding residual of the similar image, and storing the compressed code stream in the cloud data gallery to finish encoding operation of the image to be encoded. Therefore, the image coding method, the device and the electronic equipment based on the cloud data automatic downloading can solve the problems of low image data compression efficiency and low retrieval speed of the network cloud data gallery.
Drawings
Fig. 1 is a schematic flow chart of an image encoding method based on automatic downloading of cloud data according to an embodiment of the present invention;
FIG. 2 is a detailed flow chart of one of the steps shown in FIG. 1;
FIG. 3 is a detailed flow chart of another step of FIG. 1;
FIG. 4 is a functional block diagram of an image encoding device based on automatic downloading of cloud data according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device for implementing the image encoding method based on automatic cloud data downloading according to an embodiment of the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The embodiment of the application provides an image coding method based on automatic downloading of cloud data. The execution subject of the image encoding method based on automatic downloading of cloud data includes, but is not limited to, at least one of a server, a terminal, and the like, which can be configured to execute the method provided by the embodiment of the application. In other words, the image encoding method based on the automatic download of cloud data may be performed by software or hardware installed at a terminal device or a server device. The service end includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like.
Example 1:
referring to fig. 1, a flowchart of an image encoding method based on automatic cloud data downloading according to an embodiment of the present invention is shown. In this embodiment, the image encoding method based on automatic downloading of cloud data includes:
s1, extracting image features of an image to be encoded, and performing similar retrieval in a pre-constructed cloud data gallery by utilizing the image features to obtain a candidate similar image set.
The image features can be explained by extracting features of the image to be encoded by using a visual search compact descriptor (the MPEG standardized Compact Descriptor for Visual Search, CDVS for short) to obtain the features. The visual search compact descriptor is a standard specified by the MPEG organization for the descriptor format of image retrieval and the characteristic extraction and search process, and compared with the general image characteristics, the visual search compact descriptor occupies fewer bytes, has good retrieval and matching performance, and has higher characteristic extraction and matching speed.
As can be appreciated, the cloud data gallery refers to a cloud data gallery for storing images. The candidate similar image set refers to similar images which are extracted from the cloud data gallery and possibly the images to be encoded.
In an embodiment of the present invention, the extracting image features of an image to be encoded includes:
detecting interest points of the image to be coded to obtain initial local feature points;
removing noise points in the initial local descriptors to obtain target local feature points;
compressing the target local feature points to obtain local descriptors;
aggregating the target local feature points to obtain a global descriptor;
and constructing the image characteristics of the image to be encoded according to the local descriptors and the global descriptors.
It should be appreciated that the initial local feature points need to be filtered to remove noise points, leaving only important local feature points.
And performing dimensionality reduction, aggregation and binarization processing on the target local feature points to obtain the global descriptor. And transforming and scalar quantizing the target local feature points to obtain the local descriptors. The target local feature points are compressed, byte occupation can be reduced, feature matching time is shortened, and the target local feature points are aggregated, so that the image to be encoded has different layers of information description, and the retrieval accuracy is improved.
In detail, referring to fig. 2, the performing similar search in a pre-constructed cloud database by using the image features to obtain a candidate similar image set includes:
s11, extracting a pre-constructed global descriptor index table in the cloud data gallery;
s12, matching global descriptors in the image features with global descriptors of images in the cloud data gallery by using the global descriptor index table to obtain a hamming distance sequence of the images in the cloud data gallery and the images to be coded;
s13, extracting a preset number of Hamming distances and images corresponding to the preset number of Hamming distances from the Hamming distance sequence according to the sequence from small to large;
s14, taking the images corresponding to the hamming distances of the preset number as the candidate similar image set.
It should be appreciated that it is desirable to generate the index table of the global descriptor from all images in the cloud data gallery based on a multi-block index structure, thereby improving retrieval efficiency.
It is understood that the hamming distance refers to a measure of feature distance, which represents the similarity between two features.
In the embodiment of the invention, a predetermined number of images need to be selected by using the global descriptor, the range of candidate images is reduced by using the predetermined number of images, the predetermined number can be 300, and then similar images are selected from the predetermined number of images by using the local descriptor.
In detail, before performing similar retrieval in the pre-constructed cloud data gallery by using the image features to obtain the candidate similar image set, the method further includes:
extracting local feature points of all images in the cloud data gallery;
aggregating each local feature point to obtain a global descriptor corresponding to each image;
and constructing a global descriptor index table by utilizing the global descriptor corresponding to each image according to the pre-constructed multi-block index structure.
S2, judging whether the candidate similar image set has the similar image of the image to be coded or not.
It can be understood that it is necessary to determine whether the similar image exists in the candidate similar image set, if so, compress and encode the image to be encoded using inter-frame prediction encoding, and if not, compress using an intra-frame encoding algorithm.
In the embodiment of the present invention, the determining whether the candidate similar image set has the similar image of the image to be encoded includes:
obtaining a similar image test set;
extracting local feature points of each image in the similar image test set;
The local feature points of each image in the similar image test set are aggregated to obtain a global descriptor of each image in the similar image test set;
sequentially extracting images to be compared in the similar image test set;
matching the global descriptors of the images to be compared with global descriptors of other images in the similar image test set to obtain a hamming distance set between the other images in the similar image test set and the images to be compared;
extracting the maximum hamming distance in the hamming distance set to obtain a similar hamming distance of the image to be compared;
extracting the maximum Hamming distance from the similar Hamming distances of all images in the similar image test set to obtain a similar Hamming distance threshold;
judging whether a hamming distance smaller than the similar hamming distance threshold exists in the hamming distance sequence corresponding to the candidate similar image;
if the hamming distance sequence corresponding to the candidate similar images does not have hamming distances smaller than the similar hamming distance threshold, judging that the candidate similar images are concentrated and the similar images of the image to be coded do not exist;
And if the Hamming distance sequence corresponding to the candidate similar images has Hamming distance smaller than the similar Hamming distance threshold, judging that the candidate similar images are concentrated and the similar images of the image to be coded exist.
The set of similar image tests may be interpreted as a set of similar images. And judging whether the similar images exist in the candidate similar images or not by using the similar Hamming distance threshold value obtained by the similar image test set. And if the distance is smaller than the similar Hamming distance threshold, indicating that similar images exist, otherwise, the similar images do not exist.
And if the candidate similar images do not exist in the set of similar images, executing S3, encoding the image to be encoded by utilizing a pre-built intra-frame encoding algorithm to obtain an encoded image, and storing the encoded image in the cloud data gallery to complete encoding of the image to be encoded.
In the embodiment of the invention, if the candidate similar images do not exist in the candidate similar image set, the existing intra-frame coding algorithm is utilized for coding compression, and the images in the cloud data gallery are not utilized. Wherein the intra-frame coding algorithm is a prior art and will not be described in detail herein.
And if the candidate similar images are similar images of the image to be coded in the set, executing S4 to preprocess the similar images to obtain a coding reference image and preprocessing parameters.
It should be understood that, the preprocessing refers to adjusting the shape and pixel value of the similar image according to the image to be encoded, so that the similar image is close to the image to be encoded, and is convenient for subsequent processing.
In the embodiment of the present invention, if the candidate similar images collectively include similar images of the image to be encoded, before preprocessing the similar images, the method further includes:
extracting all hamming distances smaller than the similar hamming distance threshold value from the preset number of hamming distances to obtain a candidate hamming distance set;
sequentially extracting images corresponding to each candidate hamming distance in the candidate hamming distance set to obtain a global similar image set;
extracting local feature points of each image in the global similar image set;
compressing local feature points of each image in the global similar image set to obtain local descriptors of each image in the global similar image set;
utilizing the local descriptors of each image in the global similar image set to carry out Hamming distance matching with the local descriptors of the images to be coded to obtain a local Hamming distance sequence;
Extracting the minimum Hamming distance in the local Hamming distance sequence and an image corresponding to the minimum Hamming distance;
and taking the image corresponding to the minimum Hamming distance as a similar image of the image to be encoded.
It will be appreciated that a smaller hamming distance means that the image to be encoded is more similar to the similar image.
In the embodiment of the present invention, the preprocessing the similar image to obtain the encoded reference image and the preprocessing parameters includes:
dividing the similar image according to the feature points matched with the image to be coded to obtain a similar block set;
obtaining an optimal transformation matrix corresponding to each similar block in the similar block set by utilizing a pre-constructed characteristic point matching distance formula;
deforming the similar images by utilizing the optimal transformation matrix corresponding to each similar block to obtain deformed reference images;
according to the difference of pixel values of the same positions of the image to be encoded and the deformed reference image, carrying out illumination compensation on the deformed reference image to obtain the encoded reference image;
and constructing the preprocessing parameters according to the optimal transformation matrix corresponding to each similar block and the numerical value difference of illumination compensation.
In the embodiment of the invention, the formula of the matching distance of the characteristic points is as follows:
d i ＝∥p(f i )-p(f i ′ )×H∥
wherein d i Representing the distance value of the characteristic points of the i-th block similar block and the corresponding block in the image to be coded, and p (f) i ) Representing the corresponding position of the i-th block similar block in the image to be coded, p (f) i ′ ) Represents the position of the i-th block similar block in the similar image, H represents the transformation matrix, f i Representing corresponding characteristic points of the i-th block similar block in the image to be coded, f i ′ And representing the characteristic points of the i-th block similar block in the similar image.
It can be understood that, according to the perspective transformation principle, a transformation matrix can be calculated for every four pairs of matched feature points, and the matrix can represent deformation information such as rotation, translation, scaling and the like between images.
S5, carrying out inter-frame prediction coding on the image to be coded by utilizing the coding reference image to obtain inter-frame coding residual errors.
It is understood that the inter-frame predictive coding may utilize HEVC (High Efficiency Video Coding) for coding prediction and image compression.
In detail, referring to fig. 3, the performing inter-frame prediction encoding on the image to be encoded by using the encoded reference image to obtain an inter-frame encoded residual, includes:
S51, performing block segmentation on the image to be coded and the coding reference image to obtain a block image set to be coded and a reference block image set;
s52, extracting block images to be encoded in the block image set to be encoded in sequence, and calculating the mean square error of each reference block image in the block image set to be encoded and each reference block image in the reference block image set to obtain the minimum mean square error and similar reference block images corresponding to the block images to be encoded;
and S53, integrating the differences between all the blocks to be coded in the image to be coded and the corresponding similar reference block images to obtain the inter-frame coding residual errors.
It should be understood that in the inter-frame prediction encoding process, the image to be encoded needs to be segmented, and then a similar reference block image with the minimum mean square error with the segmented image to be encoded is found in the similar image. And finally, obtaining the inter-frame coding residual error according to the differences between all similar reference block images and the corresponding block images to be coded.
And S6, extracting image indexes of the similar images in the cloud data gallery, constructing a compressed code stream by utilizing the preprocessing parameters, the inter-frame coding residual errors and the image indexes, and storing the compressed code stream in the cloud data gallery to finish coding of the images to be coded.
In the embodiment of the invention, the compressed code stream can be constructed and stored in the cloud data gallery after the preprocessing parameters, the inter-frame coding residual errors and the image indexes are obtained. When decoding is needed, the similar images are extracted from the cloud data gallery only according to the image indexes, the similar images are preprocessed by utilizing the preprocessing parameters, and the images to be encoded are calculated by utilizing the inter-frame coding residual errors and the processed similar images. The storage space is saved, and the retrieval efficiency is improved.
Compared with the background art, the method comprises the following steps: the method comprises the steps of searching a candidate similar image set in a cloud data gallery by utilizing image characteristics of an image to be encoded, further judging whether the candidate similar image set contains a similar image of the image to be encoded or not, compressing the image to be encoded according to a general intra-frame encoding algorithm if the candidate similar image set does not contain the similar image, compressing the image to be encoded by utilizing the similar image if the candidate similar image does not contain the similar image, improving compression efficiency, preprocessing the similar image to obtain an encoding reference image and preprocessing parameters, performing inter-frame predictive encoding by utilizing the encoding reference image to obtain an inter-frame encoding residual, constructing a compressed code stream according to image indexes, preprocessing parameters and inter-frame encoding residual of the similar image, and storing the compressed code stream in the cloud data gallery to finish encoding operation of the image to be encoded. Therefore, the image coding method, the device and the electronic equipment based on the cloud data automatic downloading can solve the problems of low image data compression efficiency and low retrieval speed of the network cloud data gallery.
Example 2:
fig. 4 is a functional block diagram of an image encoding device based on automatic cloud data downloading according to an embodiment of the present invention.
The image encoding device 100 based on automatic downloading of cloud data according to the present invention may be installed in an electronic apparatus. Depending on the implementation function, the image encoding device 100 based on the automatic downloading of cloud data may include a candidate similar image set retrieving module 101, a similar image existence judging module 102, an intra-frame encoding module 103, a similar image preprocessing module 104, an inter-frame prediction encoding module 105, and a compressed code stream storing module 106. The module of the invention, which may also be referred to as a unit, refers to a series of computer program segments, which are stored in the memory of the electronic device, capable of being executed by the processor of the electronic device and of performing a fixed function.
The candidate similar image set retrieval module 101 is configured to extract image features of an image to be encoded, and perform similar retrieval in a pre-constructed cloud data gallery by using the image features to obtain a candidate similar image set;
the image features can be explained by extracting features of the image to be encoded by using a visual search compact descriptor (the MPEG standardized Compact Descriptor for Visual Search, CDVS for short) to obtain the features. The visual search compact descriptor is a standard specified by the MPEG organization for the descriptor format of image retrieval and the characteristic extraction and search process, and compared with the general image characteristics, the visual search compact descriptor occupies fewer bytes, has good retrieval and matching performance, and has higher characteristic extraction and matching speed.
As can be appreciated, the cloud data gallery refers to a cloud data gallery for storing images. The candidate similar image set refers to similar images which are extracted from the cloud data gallery and possibly the images to be encoded.
In an embodiment of the present invention, the extracting image features of an image to be encoded includes:
detecting interest points of the image to be coded to obtain initial local feature points;
removing noise points in the initial local descriptors to obtain target local feature points;
compressing the target local feature points to obtain local descriptors;
aggregating the target local feature points to obtain a global descriptor;
and constructing the image characteristics of the image to be encoded according to the local descriptors and the global descriptors.
It should be appreciated that the initial local feature points need to be filtered to remove noise points, leaving only important local feature points.
And performing dimensionality reduction, aggregation and binarization processing on the target local feature points to obtain the global descriptor. And transforming and scalar quantizing the target local feature points to obtain the local descriptors. The target local feature points are compressed, byte occupation can be reduced, feature matching time is shortened, and the target local feature points are aggregated, so that the image to be encoded has different layers of information description, and the retrieval accuracy is improved.
In the embodiment of the present invention, performing similar search in a pre-constructed cloud data gallery by using the image features to obtain a candidate similar image set includes:
extracting a pre-constructed global descriptor index table in the cloud data gallery;
matching global descriptors in the image features with global descriptors of images in the cloud data gallery by using the global descriptor index table to obtain a hamming distance sequence of the images in the cloud data gallery and the images to be coded;
extracting a preset number of hamming distances and images corresponding to the preset number of hamming distances from the hamming distance sequence according to the sequence from small to large;
and taking the images corresponding to the hamming distances of the preset number as the candidate similar image set.
It should be appreciated that it is desirable to generate the index table of the global descriptor from all images in the cloud data gallery based on a multi-block index structure, thereby improving retrieval efficiency.
It is understood that the hamming distance refers to a measure of feature distance, which represents the similarity between two features.
In the embodiment of the invention, a predetermined number of images need to be selected by using the global descriptor, the range of candidate images is reduced by using the predetermined number of images, the predetermined number can be 300, and then similar images are selected from the predetermined number of images by using the local descriptor.
In the embodiment of the present invention, before performing similar search in a pre-constructed cloud data gallery by using the image features to obtain a candidate similar image set, the method further includes:
extracting local feature points of all images in the cloud data gallery;
aggregating each local feature point to obtain a global descriptor corresponding to each image;
and constructing a global descriptor index table by utilizing the global descriptor corresponding to each image according to the pre-constructed multi-block index structure.
The similar image existence judging module 102 is configured to judge whether a similar image of the image to be encoded exists in the candidate similar image set; the judging whether the candidate similar image set has the similar image of the image to be coded or not comprises the following steps: obtaining a similar image test set; extracting local feature points of each image in the similar image test set; the local feature points of each image in the similar image test set are aggregated to obtain a global descriptor of each image in the similar image test set; sequentially extracting images to be compared in the similar image test set; matching the global descriptors of the images to be compared with global descriptors of other images in the similar image test set to obtain a hamming distance set between the other images in the similar image test set and the images to be compared; extracting the maximum hamming distance in the hamming distance set to obtain a similar hamming distance of the image to be compared; extracting the maximum Hamming distance from the similar Hamming distances of all images in the similar image test set to obtain a similar Hamming distance threshold; judging whether a hamming distance smaller than the similar hamming distance threshold exists in the hamming distance sequence corresponding to the candidate similar image; if the hamming distance sequence corresponding to the candidate similar images does not have hamming distances smaller than the similar hamming distance threshold, judging that the candidate similar images are concentrated and the similar images of the image to be coded do not exist; if the Hamming distance sequence corresponding to the candidate similar images has Hamming distance smaller than the similar Hamming distance threshold, judging that the candidate similar images are concentrated and similar images of the image to be coded exist;
It can be understood that it is necessary to determine whether the similar image exists in the candidate similar image set, if so, compress and encode the image to be encoded using inter-frame prediction encoding, and if not, compress using an intra-frame encoding algorithm.
In the embodiment of the present invention, the determining whether the candidate similar image set has the similar image of the image to be encoded includes:
obtaining a similar image test set;
extracting local feature points of each image in the similar image test set;
the local feature points of each image in the similar image test set are aggregated to obtain a global descriptor of each image in the similar image test set;
sequentially extracting images to be compared in the similar image test set;
matching the global descriptors of the images to be compared with global descriptors of other images in the similar image test set to obtain a hamming distance set between the other images in the similar image test set and the images to be compared;
extracting the maximum hamming distance in the hamming distance set to obtain a similar hamming distance of the image to be compared;
Extracting the maximum Hamming distance from the similar Hamming distances of all images in the similar image test set to obtain a similar Hamming distance threshold;
judging whether a hamming distance smaller than the similar hamming distance threshold exists in the hamming distance sequence corresponding to the candidate similar image;
if the hamming distance sequence corresponding to the candidate similar images does not have hamming distances smaller than the similar hamming distance threshold, judging that the candidate similar images are concentrated and the similar images of the image to be coded do not exist;
and if the Hamming distance sequence corresponding to the candidate similar images has Hamming distance smaller than the similar Hamming distance threshold, judging that the candidate similar images are concentrated and the similar images of the image to be coded exist.
The set of similar image tests may be interpreted as a set of similar images. And judging whether the similar images exist in the candidate similar images or not by using the similar Hamming distance threshold value obtained by the similar image test set. And if the distance is smaller than the similar Hamming distance threshold, indicating that similar images exist, otherwise, the similar images do not exist.
The intra-frame encoding module 103 is configured to encode the image to be encoded by using a pre-built intra-frame encoding algorithm if the candidate similar image set does not have a similar image of the image to be encoded, obtain an encoded image, and store the encoded image in the cloud database, so as to complete encoding of the image to be encoded;
In the embodiment of the invention, if the candidate similar images do not exist in the candidate similar image set, the existing intra-frame coding algorithm is utilized for coding compression, and the images in the cloud data gallery are not utilized. Wherein the intra-frame coding algorithm is a prior art and will not be described in detail herein.
The similar image preprocessing module 104 is configured to, if the candidate similar images collectively include similar images of the image to be encoded, perform preprocessing on the similar images to obtain an encoding reference image and preprocessing parameters;
it should be understood that, the preprocessing refers to adjusting the shape and pixel value of the similar image according to the image to be encoded, so that the similar image is close to the image to be encoded, and is convenient for subsequent processing.
In the embodiment of the present invention, if the candidate similar images collectively include similar images of the image to be encoded, before preprocessing the similar images, the method further includes:
extracting all hamming distances smaller than the similar hamming distance threshold value from the preset number of hamming distances to obtain a candidate hamming distance set;
sequentially extracting images corresponding to each candidate hamming distance in the candidate hamming distance set to obtain a global similar image set;
Extracting local feature points of each image in the global similar image set;
compressing local feature points of each image in the global similar image set to obtain local descriptors of each image in the global similar image set;
utilizing the local descriptors of each image in the global similar image set to carry out Hamming distance matching with the local descriptors of the images to be coded to obtain a local Hamming distance sequence;
extracting the minimum Hamming distance in the local Hamming distance sequence and an image corresponding to the minimum Hamming distance;
and taking the image corresponding to the minimum Hamming distance as a similar image of the image to be encoded.
It will be appreciated that a smaller hamming distance means that the image to be encoded is more similar to the similar image.
In the embodiment of the present invention, the preprocessing the similar image to obtain the encoded reference image and the preprocessing parameters includes:
dividing the similar image according to the feature points matched with the image to be coded to obtain a similar block set;
obtaining an optimal transformation matrix corresponding to each similar block in the similar block set by utilizing a pre-constructed characteristic point matching distance formula;
Deforming the similar images by utilizing the optimal transformation matrix corresponding to each similar block to obtain deformed reference images;
according to the difference of pixel values of the same positions of the image to be encoded and the deformed reference image, carrying out illumination compensation on the deformed reference image to obtain the encoded reference image;
and constructing the preprocessing parameters according to the optimal transformation matrix corresponding to each similar block and the numerical value difference of illumination compensation.
In the embodiment of the invention, the formula of the matching distance of the characteristic points is as follows:
d i ＝∥p(f i )-p(f i ′ )×H∥
wherein d i Representing the distance value of the characteristic points of the i-th block similar block and the corresponding block in the image to be coded, and p (f) i ) Representing the corresponding position of the i-th block similar block in the image to be coded, p (f) i ′ ) Represents the position of the i-th block similar block in the similar image, H represents the transformation matrix, f i Representing corresponding characteristic points of the i-th block similar block in the image to be coded, f i ′ Representing the i-th block phase in the similar imageBlock-like feature points.
It can be understood that, according to the perspective transformation principle, a transformation matrix can be calculated for every four pairs of matched feature points, and the matrix can represent deformation information such as rotation, translation, scaling and the like between images.
The inter-frame prediction encoding module 105 is configured to perform inter-frame prediction encoding on the image to be encoded by using the encoded reference image, so as to obtain an inter-frame encoded residual;
it is understood that the inter-frame predictive coding may utilize HEVC (High Efficiency Video Coding) for coding prediction and image compression.
In the embodiment of the present invention, the performing inter-frame prediction encoding on the image to be encoded by using the encoding reference image to obtain an inter-frame encoded residual error includes:
performing block segmentation on the image to be coded and the coding reference image to obtain a block image set to be coded and a reference block image set;
sequentially extracting block images to be encoded from the block image set to be encoded, and calculating the mean square error of each block of reference block image in the block image set to be encoded and the reference block image set to obtain the minimum mean square error and similar reference block images corresponding to the block images to be encoded;
and integrating the differences between all the blocks to be coded in the image to be coded and the corresponding similar reference block images to obtain the inter-frame coding residual errors.
It should be understood that in the inter-frame prediction encoding process, the image to be encoded needs to be segmented, and then a similar reference block image with the minimum mean square error with the segmented image to be encoded is found in the similar image. And finally, obtaining the inter-frame coding residual error according to the differences between all similar reference block images and the corresponding block images to be coded.
The compressed code stream storage module 106 is configured to extract an image index of the similar image in the cloud data gallery, construct a compressed code stream by using the preprocessing parameter, the inter-frame coding residual error and the image index, and store the compressed code stream in the cloud data gallery to complete the encoding of the image to be encoded.
In the embodiment of the invention, the compressed code stream can be constructed and stored in the cloud data gallery after the preprocessing parameters, the inter-frame coding residual errors and the image indexes are obtained. When decoding is needed, the similar images are extracted from the cloud data gallery only according to the image indexes, the similar images are preprocessed by utilizing the preprocessing parameters, and the images to be encoded are calculated by utilizing the inter-frame coding residual errors and the processed similar images. The storage space is saved, and the retrieval efficiency is improved.
In detail, the image encoding device 100 based on automatic downloading of cloud data according to the embodiment of the present invention can have the following technical effects:
compared with the background art, the method comprises the following steps: the method comprises the steps of searching a candidate similar image set in a cloud data gallery by utilizing image characteristics of an image to be encoded, further judging whether the candidate similar image set contains a similar image of the image to be encoded or not, compressing the image to be encoded according to a general intra-frame encoding algorithm if the candidate similar image set does not contain the similar image, compressing the image to be encoded by utilizing the similar image if the candidate similar image does not contain the similar image, improving compression efficiency, preprocessing the similar image to obtain an encoding reference image and preprocessing parameters, performing inter-frame predictive encoding by utilizing the encoding reference image to obtain an inter-frame encoding residual, constructing a compressed code stream according to image indexes, preprocessing parameters and inter-frame encoding residual of the similar image, and storing the compressed code stream in the cloud data gallery to finish encoding operation of the image to be encoded. Therefore, the image coding method, the device and the electronic equipment based on the cloud data automatic downloading can solve the problems of low image data compression efficiency and low retrieval speed of the network cloud data gallery.
Example 3:
fig. 5 is a schematic structural diagram of an electronic device for implementing an image encoding method based on automatic cloud data downloading according to an embodiment of the present invention.
The electronic device 1 may comprise a processor 10, a memory 11, a bus 12 and a communication interface 13, and may further comprise a computer program stored in the memory 11 and executable on the processor 10, such as an image encoding program automatically downloaded based on cloud data.
The memory 11 includes at least one type of readable storage medium, including flash memory, a mobile hard disk, a multimedia card, a card memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of the electronic device 1, such as a removable hard disk of the electronic device 1. The memory 11 may in other embodiments also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the electronic device 1. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device 1. The memory 11 may be used not only for storing application software installed in the electronic device 1 and various types of data, such as codes of an image encoding program automatically downloaded based on cloud data, but also for temporarily storing data that has been output or is to be output.
The processor 10 may be comprised of integrated circuits in some embodiments, for example, a single packaged integrated circuit, or may be comprised of multiple integrated circuits packaged with the same or different functions, including one or more central processing units (Central Processing unit, CPU), microprocessors, digital processing chips, graphics processors, combinations of various control chips, and the like. The processor 10 is a Control Unit (Control Unit) of the electronic device, connects respective components of the entire electronic device using various interfaces and lines, executes various functions of the electronic device 1 and processes data by running or executing programs or modules (e.g., an image encoding program or the like automatically downloaded based on cloud data) stored in the memory 11, and calling data stored in the memory 11.
The bus may be a peripheral component interconnect standard (peripheral component interconnect, PCI) bus or an extended industry standard architecture (extended industry standard architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. The bus is arranged to enable a connection communication between the memory 11 and at least one processor 10 etc.
Fig. 5 shows only an electronic device with components, it being understood by a person skilled in the art that the structure shown in fig. 5 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than shown, or may combine certain components, or may be arranged in different components.
For example, although not shown, the electronic device 1 may further include a power source (such as a battery) for supplying power to each component, and preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that functions of charge management, discharge management, power consumption management, and the like are implemented through the power management device. The power supply may also include one or more of any of a direct current or alternating current power supply, recharging device, power failure detection circuit, power converter or inverter, power status indicator, etc. The electronic device 1 may further include various sensors, bluetooth modules, wi-Fi modules, etc., which will not be described herein.
Further, the electronic device 1 may also comprise a network interface, optionally the network interface may comprise a wired interface and/or a wireless interface (e.g. WI-FI interface, bluetooth interface, etc.), typically used for establishing a communication connection between the electronic device 1 and other electronic devices.
The electronic device 1 may optionally further comprise a user interface, which may be a Display, an input unit, such as a Keyboard (Keyboard), or a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like. The display may also be referred to as a display screen or display unit, as appropriate, for displaying information processed in the electronic device 1 and for displaying a visual user interface.
It should be understood that the embodiments described are for illustrative purposes only and are not limited to this configuration in the scope of the patent application.
The image encoding program stored in the memory 11 of the electronic device 1 and automatically downloaded based on cloud data is a combination of a plurality of instructions, which when executed in the processor 10, can implement:
extracting image features of an image to be encoded, and performing similar retrieval in a pre-constructed cloud data gallery by utilizing the image features to obtain a candidate similar image set;
judging whether the candidate similar image set has a similar image of the image to be coded or not;
The judging whether the candidate similar image set has the similar image of the image to be coded or not comprises the following steps:
obtaining a similar image test set;
extracting local feature points of each image in the similar image test set;
the local feature points of each image in the similar image test set are aggregated to obtain a global descriptor of each image in the similar image test set;
sequentially extracting images to be compared in the similar image test set;
matching the global descriptors of the images to be compared with global descriptors of other images in the similar image test set to obtain a hamming distance set between the other images in the similar image test set and the images to be compared;
extracting the maximum hamming distance in the hamming distance set to obtain a similar hamming distance of the image to be compared;
extracting the maximum Hamming distance from the similar Hamming distances of all images in the similar image test set to obtain a similar Hamming distance threshold;
judging whether a hamming distance smaller than the similar hamming distance threshold exists in the hamming distance sequence corresponding to the candidate similar image;
If the hamming distance sequence corresponding to the candidate similar images does not have hamming distances smaller than the similar hamming distance threshold, judging that the candidate similar images are concentrated and the similar images of the image to be coded do not exist;
and if the Hamming distance sequence corresponding to the candidate similar images has Hamming distance smaller than the similar Hamming distance threshold, judging that the candidate similar images are concentrated and the similar images of the image to be coded exist.
If the candidate similar images do not exist in the candidate similar image set, encoding the image to be encoded by utilizing a pre-built intra-frame encoding algorithm to obtain an encoded image, and storing the encoded image in the cloud data gallery to complete encoding of the image to be encoded;
if the candidate similar images are similar images of the image to be coded in the set, preprocessing the similar images to obtain a coding reference image and preprocessing parameters;
performing inter-frame predictive coding on the image to be coded by using the coding reference image to obtain inter-frame coding residual errors;
and extracting image indexes of the similar images in the cloud data gallery, constructing a compressed code stream by utilizing the preprocessing parameters, the inter-frame coding residual errors and the image indexes, and storing the compressed code stream in the cloud data gallery to finish the coding of the images to be coded.
Specifically, the specific implementation method of the above instruction by the processor 10 may refer to descriptions of related steps in the corresponding embodiments of fig. 1 to 4, which are not repeated herein.
Further, the modules/units integrated in the electronic device 1 may be stored in a computer readable storage medium if implemented in the form of software functional units and sold or used as separate products. The computer readable storage medium may be volatile or nonvolatile. For example, the computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM).
The present invention also provides a computer readable storage medium storing a computer program which, when executed by a processor of an electronic device, can implement:
extracting image features of an image to be encoded, and performing similar retrieval in a pre-constructed cloud data gallery by utilizing the image features to obtain a candidate similar image set;
judging whether the candidate similar image set has a similar image of the image to be coded or not;
The judging whether the candidate similar image set has the similar image of the image to be coded or not comprises the following steps:
obtaining a similar image test set;
extracting local feature points of each image in the similar image test set;
the local feature points of each image in the similar image test set are aggregated to obtain a global descriptor of each image in the similar image test set;
sequentially extracting images to be compared in the similar image test set;
matching the global descriptors of the images to be compared with global descriptors of other images in the similar image test set to obtain a hamming distance set between the other images in the similar image test set and the images to be compared;
extracting the maximum hamming distance in the hamming distance set to obtain a similar hamming distance of the image to be compared;
extracting the maximum Hamming distance from the similar Hamming distances of all images in the similar image test set to obtain a similar Hamming distance threshold;
judging whether a hamming distance smaller than the similar hamming distance threshold exists in the hamming distance sequence corresponding to the candidate similar image;
If the hamming distance sequence corresponding to the candidate similar images does not have hamming distances smaller than the similar hamming distance threshold, judging that the candidate similar images are concentrated and the similar images of the image to be coded do not exist;
and if the Hamming distance sequence corresponding to the candidate similar images has Hamming distance smaller than the similar Hamming distance threshold, judging that the candidate similar images are concentrated and the similar images of the image to be coded exist.
If the candidate similar images do not exist in the candidate similar image set, encoding the image to be encoded by utilizing a pre-built intra-frame encoding algorithm to obtain an encoded image, and storing the encoded image in the cloud data gallery to complete encoding of the image to be encoded;
if the candidate similar images are similar images of the image to be coded in the set, preprocessing the similar images to obtain a coding reference image and preprocessing parameters;
performing inter-frame predictive coding on the image to be coded by using the coding reference image to obtain inter-frame coding residual errors;
and extracting image indexes of the similar images in the cloud data gallery, constructing a compressed code stream by utilizing the preprocessing parameters, the inter-frame coding residual errors and the image indexes, and storing the compressed code stream in the cloud data gallery to finish the coding of the images to be coded.
In the several embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be other manners of division when actually implemented.
The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional module in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units can be realized in a form of hardware or a form of hardware and a form of software functional modules.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.
Finally, it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention.

Claims (8)

1. An image coding method based on automatic downloading of cloud data, which is characterized by comprising the following steps:
extracting image features of an image to be encoded, and performing similar retrieval in a pre-constructed cloud data gallery by utilizing the image features to obtain a candidate similar image set;
judging whether the candidate similar image set has a similar image of the image to be coded or not;
the judging whether the candidate similar image set has the similar image of the image to be coded or not comprises the following steps:
obtaining a similar image test set;
extracting local feature points of each image in the similar image test set;
the local feature points of each image in the similar image test set are aggregated to obtain a global descriptor of each image in the similar image test set;
sequentially extracting images to be compared in the similar image test set;
Matching the global descriptors of the images to be compared with global descriptors of other images in the similar image test set to obtain a hamming distance set between the other images in the similar image test set and the images to be compared;
extracting the maximum hamming distance in the hamming distance set to obtain a similar hamming distance of the image to be compared;
extracting the maximum Hamming distance from the similar Hamming distances of all images in the similar image test set to obtain a similar Hamming distance threshold;
judging whether a hamming distance smaller than the similar hamming distance threshold exists in the hamming distance sequence corresponding to the candidate similar image;
if the hamming distance sequence corresponding to the candidate similar images does not have hamming distances smaller than the similar hamming distance threshold, judging that the candidate similar images are concentrated and the similar images of the image to be coded do not exist;
if the Hamming distance sequence corresponding to the candidate similar images has Hamming distance smaller than the similar Hamming distance threshold, judging that the candidate similar images are concentrated and similar images of the image to be coded exist;
if the candidate similar images do not exist in the candidate similar image set, encoding the image to be encoded by utilizing a pre-built intra-frame encoding algorithm to obtain an encoded image, and storing the encoded image in the cloud data gallery to complete encoding of the image to be encoded;
If the candidate similar images are similar images of the image to be coded in the set, preprocessing the similar images to obtain a coding reference image and preprocessing parameters;
performing inter-frame predictive coding on the image to be coded by using the coding reference image to obtain inter-frame coding residual errors;
extracting image indexes of the similar images in the cloud data gallery, constructing a compressed code stream by utilizing the preprocessing parameters, inter-frame coding residual errors and the image indexes, and storing the compressed code stream in the cloud data gallery to finish the coding of the images to be coded;
preprocessing the similar images to obtain coded reference images and preprocessing parameters, wherein the preprocessing comprises the following steps:
dividing the similar image according to the feature points matched with the image to be coded to obtain a similar block set;
obtaining an optimal transformation matrix corresponding to each similar block in the similar block set by utilizing a pre-constructed characteristic point matching distance formula;
deforming the similar images by utilizing the optimal transformation matrix corresponding to each similar block to obtain deformed reference images;
according to the difference of pixel values of the same positions of the image to be encoded and the deformed reference image, carrying out illumination compensation on the deformed reference image to obtain the encoded reference image;
And constructing the preprocessing parameters according to the optimal transformation matrix corresponding to each similar block and the numerical value difference of illumination compensation.
2. The method for encoding an image based on automatic downloading of cloud data as claimed in claim 1, wherein the extracting image features of the image to be encoded comprises:
detecting interest points of the image to be coded to obtain initial local feature points;
removing noise points in the initial local descriptors to obtain target local feature points;
compressing the target local feature points to obtain local descriptors;
aggregating the target local feature points to obtain a global descriptor;
and constructing the image characteristics of the image to be encoded according to the local descriptors and the global descriptors.
3. The method for encoding images based on automatic downloading of cloud data according to claim 2, wherein the performing similar search in a pre-constructed cloud database by using the image features to obtain a candidate similar image set comprises:
extracting a pre-constructed global descriptor index table in the cloud data gallery;
matching global descriptors in the image features with global descriptors of images in the cloud data gallery by using the global descriptor index table to obtain a hamming distance sequence of the images in the cloud data gallery and the images to be coded;
Extracting a preset number of hamming distances and images corresponding to the preset number of hamming distances from the hamming distance sequence according to the sequence from small to large;
and taking the images corresponding to the hamming distances of the preset number as the candidate similar image set.
4. The method for encoding an image based on automatic downloading of cloud data as claimed in claim 3, wherein before performing similar search in a pre-constructed cloud data gallery by using the image features to obtain a candidate similar image set, the method further comprises:
extracting local feature points of all images in the cloud data gallery;
aggregating each local feature point to obtain a global descriptor corresponding to each image;
and constructing a global descriptor index table by utilizing the global descriptor corresponding to each image according to the pre-constructed multi-block index structure.
5. The method for encoding an image based on automatic downloading of cloud data as claimed in claim 4, wherein if there is a similar image of the image to be encoded in the candidate similar image set, before preprocessing the similar image, the method further comprises:
extracting all hamming distances smaller than the similar hamming distance threshold value from the preset number of hamming distances to obtain a candidate hamming distance set;
Sequentially extracting images corresponding to each candidate hamming distance in the candidate hamming distance set to obtain a global similar image set;
extracting local feature points of each image in the global similar image set;
compressing local feature points of each image in the global similar image set to obtain local descriptors of each image in the global similar image set;
utilizing the local descriptors of each image in the global similar image set to carry out Hamming distance matching with the local descriptors of the images to be coded to obtain a local Hamming distance sequence;
extracting the minimum Hamming distance in the local Hamming distance sequence and an image corresponding to the minimum Hamming distance;
and taking the image corresponding to the minimum Hamming distance as a similar image of the image to be encoded.
6. The image encoding method based on automatic downloading of cloud data as claimed in claim 5, wherein the feature point matching distance formula is as follows:
d i ＝∥p(f i )-p(f i ′ )×H∥
wherein d i Representing the distance value of the characteristic points of the i-th block similar block and the corresponding block in the image to be coded, and p (f) i ) Representing the corresponding position of the i-th block similar block in the image to be coded, p (f) i ′ ) Represents the position of the i-th block similar block in the similar image, H represents the transformation matrix, f i Representing the corresponding bit of the i-th block similar block in the image to be codedCharacterization point, f i ′ And representing the characteristic points of the i-th block similar block in the similar image.
7. The method for encoding an image based on automatic downloading of cloud data as claimed in claim 6, wherein said performing inter-frame predictive encoding on said image to be encoded using said encoded reference image to obtain an inter-frame encoded residual, comprises:
performing block segmentation on the image to be coded and the coding reference image to obtain a block image set to be coded and a reference block image set;
sequentially extracting block images to be encoded from the block image set to be encoded, and calculating the mean square error of each block of reference block image in the block image set to be encoded and the reference block image set to obtain the minimum mean square error and similar reference block images corresponding to the block images to be encoded;
and integrating the differences between all the blocks to be coded in the image to be coded and the corresponding similar reference block images to obtain the inter-frame coding residual errors.
8. An image encoding device based on automatic downloading of cloud data, the device comprising:
The candidate similar image set retrieval module is used for extracting image features of an image to be encoded, and performing similar retrieval in a pre-constructed cloud data gallery by utilizing the image features to obtain a candidate similar image set;
the similar image existence judging module is used for judging whether the similar image of the image to be coded exists in the candidate similar image set; the judging whether the candidate similar image set has the similar image of the image to be coded or not comprises the following steps: obtaining a similar image test set; extracting local feature points of each image in the similar image test set; the local feature points of each image in the similar image test set are aggregated to obtain a global descriptor of each image in the similar image test set; sequentially extracting images to be compared in the similar image test set; matching the global descriptors of the images to be compared with global descriptors of other images in the similar image test set to obtain a hamming distance set between the other images in the similar image test set and the images to be compared; extracting the maximum hamming distance in the hamming distance set to obtain a similar hamming distance of the image to be compared; extracting the maximum Hamming distance from the similar Hamming distances of all images in the similar image test set to obtain a similar Hamming distance threshold; judging whether a hamming distance smaller than the similar hamming distance threshold exists in the hamming distance sequence corresponding to the candidate similar image; if the hamming distance sequence corresponding to the candidate similar images does not have hamming distances smaller than the similar hamming distance threshold, judging that the candidate similar images are concentrated and the similar images of the image to be coded do not exist; if the Hamming distance sequence corresponding to the candidate similar images has Hamming distance smaller than the similar Hamming distance threshold, judging that the candidate similar images are concentrated and similar images of the image to be coded exist;
The intra-frame coding module is used for coding the image to be coded by utilizing a pre-built intra-frame coding algorithm if the candidate similar image set does not contain the similar image of the image to be coded, so as to obtain a coded image, and storing the coded image in the cloud data gallery to finish the coding of the image to be coded;
the similar image preprocessing module is configured to, if the candidate similar images collectively include similar images of the image to be encoded, perform preprocessing on the similar images to obtain an encoding reference image and preprocessing parameters, where the preprocessing module includes:
dividing the similar image according to the feature points matched with the image to be coded to obtain a similar block set;
obtaining an optimal transformation matrix corresponding to each similar block in the similar block set by utilizing a pre-constructed characteristic point matching distance formula;
deforming the similar images by utilizing the optimal transformation matrix corresponding to each similar block to obtain deformed reference images;
according to the difference of pixel values of the same positions of the image to be encoded and the deformed reference image, carrying out illumination compensation on the deformed reference image to obtain the encoded reference image;
Constructing the preprocessing parameters according to the optimal transformation matrix corresponding to each similar block and the numerical value difference of illumination compensation;
the inter-frame prediction coding module is used for carrying out inter-frame prediction coding on the image to be coded by utilizing the coding reference image to obtain inter-frame coding residual errors;
and the compressed code stream storage module is used for extracting image indexes of the similar images in the cloud data gallery, constructing a compressed code stream by utilizing the preprocessing parameters, the inter-frame coding residual errors and the image indexes, and storing the compressed code stream in the cloud data gallery to finish the coding of the images to be coded.
CN202210425363.4A 2022-04-22 2022-04-22 Image coding method and device based on cloud data automatic downloading and electronic equipment Active CN115052160B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210425363.4A CN115052160B (en) 2022-04-22 2022-04-22 Image coding method and device based on cloud data automatic downloading and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210425363.4A CN115052160B (en) 2022-04-22 2022-04-22 Image coding method and device based on cloud data automatic downloading and electronic equipment

Publications (2)

Publication Number Publication Date
CN115052160A CN115052160A (en) 2022-09-13
CN115052160B true CN115052160B (en) 2023-07-07

Family

ID=83157126

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210425363.4A Active CN115052160B (en) 2022-04-22 2022-04-22 Image coding method and device based on cloud data automatic downloading and electronic equipment

Country Status (1)

Country Link
CN (1) CN115052160B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104768011A (en) * 2015-03-31 2015-07-08 浙江大学 Image encoding and decoding method and related device
CN111935487A (en) * 2020-08-12 2020-11-13 汪礼君 Image compression method and system based on video stream detection

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004064518A (en) * 2002-07-30 2004-02-26 Nec Corp Moving image encoding method and device and its computer program
JP4515378B2 (en) * 2005-11-14 2010-07-28 日本電信電話株式会社 Image decoding apparatus, program, and computer-readable recording medium
US8913851B1 (en) * 2011-04-29 2014-12-16 Google Inc. Fingerprinting image using points of interest for robust image identification
US9274818B2 (en) * 2013-02-06 2016-03-01 International Business Machines Corporation Reliable and scalable image transfer for data centers with low connectivity using redundancy detection
CN106682092A (en) * 2016-11-29 2017-05-17 深圳市华尊科技股份有限公司 Target retrieval method and terminal
CN110113600B (en) * 2018-02-01 2022-08-26 腾讯科技（深圳）有限公司 Video encoding method, video encoding device, computer-readable storage medium, and computer apparatus
CN108259914B (en) * 2018-03-20 2019-10-11 西安电子科技大学 Cloud image encoding method based on object library
CN109522436A (en) * 2018-11-29 2019-03-26 厦门美图之家科技有限公司 Similar image lookup method and device
CN110134804B (en) * 2019-05-20 2021-09-17 北京达佳互联信息技术有限公司 Image retrieval method, device and storage medium
CN112887732B (en) * 2021-04-30 2021-10-01 浙江智慧视频安防创新中心有限公司 Method and device for inter-frame and intra-frame joint prediction coding and decoding with configurable weight
CN113422959A (en) * 2021-05-31 2021-09-21 浙江智慧视频安防创新中心有限公司 Video encoding and decoding method and device, electronic equipment and storage medium
CN113343920A (en) * 2021-07-01 2021-09-03 中诚信征信有限公司 Method and device for classifying face recognition photos, electronic equipment and storage medium

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104768011A (en) * 2015-03-31 2015-07-08 浙江大学 Image encoding and decoding method and related device
CN111935487A (en) * 2020-08-12 2020-11-13 汪礼君 Image compression method and system based on video stream detection

Also Published As

Publication number Publication date
CN115052160A (en) 2022-09-13

Similar Documents

Publication Publication Date Title
CN113705462B (en) Face recognition method, device, electronic equipment and computer readable storage medium
CN111431537A (en) Data compression method and device and computer readable storage medium
CN112560845A (en) Character recognition method and device, intelligent meal taking cabinet, electronic equipment and storage medium
CN110442489B (en) Method of data processing and storage medium
CN115499670A (en) Video image compression processing equipment based on edge calculation
CN113157962B (en) Image retrieval method, electronic device, and storage medium
CN115052160B (en) Image coding method and device based on cloud data automatic downloading and electronic equipment
CN108764258B (en) Optimal image set selection method for group image insertion
CN115409041B (en) Unstructured data extraction method, device, equipment and storage medium
CN112329666A (en) Face recognition method and device, electronic equipment and storage medium
US8817875B2 (en) Methods and systems to encode and decode sequences of images
US10268912B2 (en) Offline, hybrid and hybrid with offline image recognition
CN115798004A (en) Face card punching method and device based on local area, electronic equipment and medium
CN110765291A (en) Retrieval method and device and electronic equipment
CN103139566A (en) Method for efficient decoding of variable length codes
CN112506931B (en) Data query method, device, electronic equipment and storage medium
CN112054805B (en) Model data compression method, system and related equipment
CN113434413A (en) Data testing method, device and equipment based on data difference and storage medium
CN114267064A (en) Face recognition method and device, electronic equipment and storage medium
CN111538914B (en) Address information processing method and device
CN113610934A (en) Image brightness adjusting method, device, equipment and storage medium
CN116257488B (en) Geotechnical engineering investigation big data archiving method, device, electronic equipment and medium
CN117076713B (en) Video fingerprint extraction and retrieval method
CN113672483B (en) Device log storage method and device, electronic device and medium
CN116366790B (en) Network video storage method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant