CN112258394B - Data processing method, ship tracking method, device, equipment and storage medium - Google Patents
Data processing method, ship tracking method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112258394B CN112258394B CN202011257995.1A CN202011257995A CN112258394B CN 112258394 B CN112258394 B CN 112258394B CN 202011257995 A CN202011257995 A CN 202011257995A CN 112258394 B CN112258394 B CN 112258394B
- Authority
- CN
- China
- Prior art keywords
- image
- resolution
- image data
- video
- characteristic information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000003672 processing method Methods 0.000 title claims abstract description 29
- 238000005070 sampling Methods 0.000 claims abstract description 56
- 238000012545 processing Methods 0.000 claims abstract description 39
- 238000001514 detection method Methods 0.000 claims description 22
- 230000015654 memory Effects 0.000 claims description 21
- 230000000694 effects Effects 0.000 claims description 16
- 238000003062 neural network model Methods 0.000 claims description 15
- 238000012549 training Methods 0.000 claims description 12
- 230000003068 static effect Effects 0.000 claims description 11
- 238000007781 pre-processing Methods 0.000 claims description 10
- 230000006835 compression Effects 0.000 claims description 4
- 238000007906 compression Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 3
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 7
- 239000000284 extract Substances 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 238000003384 imaging method Methods 0.000 description 4
- 238000004804 winding Methods 0.000 description 4
- 238000006731 degradation reaction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000015556 catabolic process Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
- G06T3/4076—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution using the original low-resolution images to iteratively correct the high-resolution images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Processing (AREA)
Abstract
The invention relates to the field of artificial intelligence, in particular to a data processing method, a ship tracking device, equipment and a storage medium. The method comprises the following steps: acquiring image data to be reconstructed; extracting characteristic information of image data to be reconstructed; and performing iterative error feedback processing on the characteristic information by using the plurality of up-sampling units and the plurality of down-sampling units, and outputting a high-resolution image. A plurality of up-sampling units and a plurality of down-sampling units in the dense depth feedback network perform iterative error feedback processing on the characteristic information, and repeatedly and alternately calculate a high-resolution image and a low-resolution image.
Description
Technical Field
The invention relates to the field of artificial intelligence, in particular to a data processing method, a ship tracking device, equipment and a storage medium.
Background
In the field of remote sensing, a video satellite is a novel earth observation satellite and is mainly realized by using a low-orbit video imaging satellite or an agile imaging satellite.
The video satellite is different from the traditional earth optical remote sensing satellite in that the video satellite has higher time resolution, can continuously observe a certain area, obtains more motion change information of a target in a video recording mode, is particularly suitable for high-resolution observation of the moving target, and is used for obtaining the motion speed and direction of the target, and the important information is difficult to obtain by the traditional earth optical remote sensing satellite. Satellite video images are becoming important dynamic remote sensing big data and widely applied to the aspects of natural disaster forecast, surface three-dimensional modeling, earth resource general survey, dynamic target tracking and the like.
Compared with the traditional optical static remote sensing satellite, the video satellite improves the time resolution and sacrifices the space resolution. Meanwhile, the data volume of the video is far larger than that of a single remote sensing image, the data volume of the video collected by the video satellite rapidly rises, and the video collected by the satellite has to be compressed in order to adapt to the transmission capacity of a channel, so that the spatial resolution of the video transmitted back to the ground is greatly reduced, and the definition of the video is seriously reduced.
In the depth network of the super-resolution reconstruction proposed at present, the features of the input are mostly learned by adopting a forward structure and are mapped into high-resolution output through nonlinearity. However, these methods do not consider the interdependence between the high resolution image and the low resolution image, resulting in poor super-resolution reconstruction.
Disclosure of Invention
Therefore, the technical problem to be solved by the present invention is to overcome the defect of poor super-resolution reconstruction effect in the prior art, so as to provide an image data processing method, which comprises the following steps:
acquiring image data to be reconstructed;
extracting characteristic information of the image data to be reconstructed;
and taking the characteristic information as the input of a pre-constructed dense depth feedback network, wherein the dense depth feedback network is a network architecture formed by densely connecting a plurality of up-sampling units and a plurality of down-sampling units, and performing iterative error feedback processing on the characteristic information by utilizing the plurality of up-sampling units and the plurality of down-sampling units to output a high-resolution image.
Preferably, each of the up-sampling units is connected with at least one of the down-sampling units, and the output ends of all the up-sampling units are connected in series.
Preferably, the up-sampling unit at least comprises a first anti-convolution layer, a first convolution layer and a second anti-convolution layer which are connected in sequence, and the down-sampling unit at least comprises a second convolution layer, a third anti-convolution layer and a third convolution layer which are connected in sequence; wherein the convolutional layer is used for reducing the image resolution; the deconvolution layer is used to magnify the image resolution.
Preferably, the process of the upsampling unit is as follows:
a first deconvolution layer:
a first winding layer:
residual error:
a second deconvolution layer:
and (3) outputting:
wherein, denotes a convolution operation, Lt-1Representing the input, p, of the first deconvolution layertRepresents the first deconvolution layer ↓ ℃ @sRepresents the deconvolution of s times, gtRepresents the first convolution layer, ↓sRepresenting the convolution by s, qtRepresenting a second deconvolution layer;
the process of the down-sampling unit is as follows:
a second convolution layer:
a third deconvolution layer:
residual error:
a third convolutional layer:
and (3) outputting:
wherein HtDenotes the input of the second buildup layer, g'tDenotes a second buildup layer, p'tDenotes a third deconvolution layer, g'tShowing a third convolutional layer.
Preferably, the extracting the feature information of the image data to be reconstructed includes:
extracting a large amount of characteristic information through a fourth convolution layer, and reducing the amount of the characteristic information through a fifth convolution layer, wherein the convolution kernel of the fourth convolution layer is larger than that of the fifth convolution layer.
The invention also provides a ship tracking method, which comprises the following steps:
acquiring a video image to be tracked of a shot ship;
preprocessing the video image to be tracked by using an image data processing method to obtain a processed video image;
and carrying out ship tracking detection according to the processed video image.
The present invention also provides an image data processing apparatus, comprising:
the first acquisition unit is used for acquiring image data to be reconstructed;
the extraction unit is used for extracting the characteristic information of the image data to be reconstructed;
and the output unit is used for taking the characteristic information as the input of a pre-constructed dense depth feedback network, the dense depth feedback network is a network architecture formed by densely connecting a plurality of up-sampling units and a plurality of down-sampling units, and the characteristic information is subjected to iterative error feedback processing by utilizing the plurality of up-sampling units and the plurality of down-sampling units to output a high-resolution image.
The present invention also provides a vessel tracking device, comprising:
the second acquisition unit is used for acquiring a video image to be tracked of the shot ship;
the processing unit is used for preprocessing the video image to be tracked by using an image data processing method to obtain a processed video image;
and the tracking unit is used for carrying out ship tracking detection according to the processed video image.
The invention also provides an electronic device, which comprises a memory and a processor, wherein the memory and the processor are mutually connected in a communication way, the memory stores computer instructions, and the processor executes the computer instructions so as to execute an image data processing method or a ship tracking method.
The present invention also provides a computer-readable storage medium storing computer instructions for causing a computer to execute an image data processing method or a ship tracking method.
The technical scheme of the invention has the following advantages:
1. the image data processing method provided by the invention has the advantages that the neural network model extracts the characteristic information of the image data to be reconstructed, the dense depth feedback network in the neural network model carries out iterative error feedback processing on the extracted characteristic information, namely, a plurality of up-sampling units and a plurality of down-sampling units in the dense depth feedback network carry out iterative error feedback processing on the characteristic information, and high-resolution images and low-resolution images are repeatedly and alternately calculated.
2. According to the image data processing device provided by the invention, the first acquisition module acquires an image with low resolution, the extraction module extracts characteristic information of the acquired image with low resolution, the dense depth feedback network in the output unit performs iterative error feedback processing on the extracted characteristic information, namely, a plurality of up-sampling units and a plurality of down-sampling units in the dense depth feedback network perform iterative error feedback processing on the characteristic information, and the high-resolution image and the image with low resolution are repeatedly and alternately calculated.
3. According to the ship tracking method provided by the invention, the obtained video image to be tracked of the shot ship is preprocessed by using the image data processing method to obtain the high-resolution video image with good effect, and the ship tracking detection is performed by using the processed video image, so that the ship can be more accurate, the ship tracking detection effect is better improved, and the error rate of the tracking detection is reduced.
4. According to the ship tracking device provided by the invention, the second acquisition unit acquires the video image to be tracked, which is shot with a ship, the processing unit preprocesses the video image to be tracked acquired by the second acquisition unit to obtain the high-resolution video image with good effect, and the tracking unit performs ship tracking detection by using the processed video image, so that the ship can be more accurate, the ship tracking detection effect is improved, and the error rate of tracking detection is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flowchart illustrating an image data processing method according to an embodiment 1 of the present invention;
FIG. 2 is a schematic block diagram of an image data processing apparatus according to embodiment 2 of the present invention;
FIG. 3 is a flowchart of a ship tracking method according to embodiment 3 of the present invention;
FIG. 4 is a schematic block diagram of a ship tracking method according to embodiment 4 of the present invention;
fig. 5 is a schematic structural diagram of an electronic device in embodiment 5 of the present invention.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the accompanying drawings, and it should be understood that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the description of the present invention, it should be noted that the terms "first", "second", "third", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In addition, the technical features involved in the different embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
Example 1
Fig. 1 is a flowchart illustrating super-resolution reconstruction of image data to be reconstructed according to some embodiments of the present invention. Although the processes described below include operations that occur in a particular order, it should be appreciated that the processes may include more or fewer operations, which may be performed sequentially or in parallel (e.g., using parallel processors or a multi-threaded environment).
The image data processing method provided in this embodiment, as shown in fig. 1, includes the following steps:
and S101, acquiring image data to be reconstructed.
In the above embodiments, the image data to be reconstructed includes, but is not limited to, images/videos captured by a monitoring camera, images/videos captured by an electronic device with an image capturing function such as a mobile phone or a video camera, images/videos captured by a video satellite, and images/videos captured by an astronomical telescope. The acquired image data to be reconstructed has low resolution due to some reasons, for example, the image/video is compressed, noise is artificially added, and the like, which causes the resolution of the shot high-resolution image/video to be greatly reduced, and thus the sharpness of the image/video is seriously reduced.
For example, the image data to be reconstructed may be an image/video captured by a video satellite, and the video satellite has a higher time resolution, so that a certain area can be continuously observed, more motion change information of the target can be obtained in a video recording manner, and the method is particularly suitable for high-resolution observation of the moving target, for example, a ship is tracked by using the video captured by the video satellite. However, as the number of videos shot by a video satellite increases sharply, in order to adapt to channel transmission capacity, the shot satellite videos have to be compressed, which results in that the resolution of the satellite videos transmitted back to the ground is greatly reduced, the video definition is seriously reduced, and in order to better utilize the videos shot by the video satellite, it is very meaningful to perform super-resolution reconstruction on the satellite videos transmitted back to the ground.
And S102, extracting the characteristic information of the image data to be reconstructed.
In the above embodiment, the trained neural network model is used to extract the feature information of the image data to be reconstructed obtained in step S101, that is, the trained convolutional layer with large convolutional kernel in the neural network model is used to extract a large amount of feature information from the image data to be reconstructed, and the convolutional layer with small convolutional kernel is used to reduce the amount of the extracted feature information. For example, a large amount of feature information is extracted through a fourth convolutional layer in the trained neural network model, and the amount of feature information is reduced through a fifth convolutional layer, wherein the convolutional kernel of the fourth convolutional layer is larger than that of the fifth convolutional layer. In this embodiment, the sizes of the convolution kernels of the fourth convolution layer and the fifth convolution layer are not limited herein, and those skilled in the art can reasonably determine the sizes according to specific situations.
Before training the neural network model, a large number of images or videos required for training are collected, the collected images or videos are preprocessed, for example, the collected high-resolution images or videos are subjected to degradation processing such as compression and noise addition, and the images or videos after the degradation processing are used as a pre-trained image data set.
For example, in order to perform super-resolution reconstruction on an image or video transmitted back to the ground by a video satellite, a large number of related images need to be collected for preprocessing, however, the imaging conditions of the video satellite in the over-the-horizon working environment are different from those of common natural images, and the pertinence of machine learning can be improved by adopting a high-resolution remote sensing image under similar imaging conditions as a pre-trained image data set. The resolution of a static image of a traditional remote sensing satellite can reach 0.1m, a video satellite can only provide about 1m of resolution at present, and the static satellite contains more high-frequency detail information than a dynamic satellite. Therefore, the static remote sensing image can be used as a training image data set for video satellite super-resolution reconstruction.
After collecting enough static remote sensing images, preprocessing the collected static remote sensing images, namely simulating the degradation process of the satellite video returned to the ground in an artificial mode. Firstly, extracting the collected static remote sensing images frame by frame (if the collected static remote sensing images are videos) to obtain a high-resolution image sequence, and thus preprocessing the high-resolution image sequence, wherein the preprocessing comprises the following steps:
a. convolving the width and the height of each high-resolution image by k times to obtain a low-resolution image sequence; wherein k is an integer of 2 to 8, and a person skilled in the art can reasonably take the value of k according to specific situations without limitation.
In the step a, the height and the width of each high-resolution image extracted from the static remote sensing image frame by frame are convoluted by 4 times, so that a low-resolution sequence is obtained.
b. And carrying out video coding on the low-resolution image sequence according to H.264, wherein the code rate of each pixel is not lower than 1.98bps, and further obtaining the compressed low-resolution image sequence.
c. And coding the compressed low-resolution image sequence according to H.264 to obtain a low-resolution training image which is decoded and restored but has a compression distortion effect, and taking the low-resolution training image as a training set of the neural network model.
And S103, taking the characteristic information as the input of a pre-constructed dense depth feedback network, wherein the dense depth feedback network is a network architecture formed by densely connecting a plurality of up-sampling units and a plurality of down-sampling units, and performing iterative error feedback processing on the characteristic information by using the plurality of up-sampling units and the plurality of down-sampling units to output a high-resolution image.
In the above embodiment, a pre-constructed dense depth feedback network is stored in the trained neural network model, the dense depth feedback network is a network architecture formed by densely connecting a plurality of upsampling units and a plurality of downsampling units, and a better reconstruction effect can be obtained by constructing the dense depth feedback network.
In this embodiment, the up-sampling unit includes a first deconvolution layer, a first convolution layer, and a second deconvolution layer, which are connected in sequence, and the down-sampling unit includes a second convolution layer, a third deconvolution layer, and a third convolution layer, which are connected in sequence, where the first deconvolution layer, the second deconvolution layer, and the third deconvolution layer are used to amplify the image resolution, and the first convolution layer, the second convolution layer, and the third convolution layer are used to reduce the image resolution. In some embodiments, the upsampling unit may also connect the sixth convolutional layer and the fourth deconvolution layer in sequence after the second deconvolution layer, or even more, only the convolutional layer and the deconvolution layer are connected in sequence, and the last one is the deconvolution layer; the down-sampling unit may also connect the fifth deconvolution layer and the seventh convolution layer in sequence after the third convolution layer, or even more, as long as the deconvolution layer and the convolution layer are connected in sequence and the last one is the convolution layer. Convolutional layers are cross-linked to deconvolution layers, i.e., convolutional layers are only linked to deconvolution layers, which are also only linked to convolution layers.
In the dense depth feedback network, each up-sampling unit is connected with at least one down-sampling unit, high-resolution images output by the output ends of all the up-sampling units are connected in series, and a high-resolution video image is reconstructed through the convolutional layer.
In this embodiment, the up-sampling unit includes a first anti-convolution layer, a first convolution layer, and a second anti-convolution layer, which are connected in sequence, and the down-sampling unit includes a second convolution layer, a third anti-convolution layer, and a third convolution layer, which are connected in sequence. The feature information extracted in step S102 is used as an input of an upsampling unit in a dense depth feedback network constructed in advance, and the feature information extracted in step S102 is subjected to iterative error feedback processing by using a plurality of upsampling units and a plurality of downsampling units, so that a high-resolution image is obtained and output.
The extracted feature information is processed in an upsampling unit as follows:
a first deconvolution layer:
a first winding layer:
residual error:
a second deconvolution layer:
and (3) outputting:
wherein, denotes a convolution operation, Lt-1Representing the input, p, of the first deconvolution layertRepresents the first deconvolution layer ↓ ℃ @sRepresents the deconvolution of s times, gtRepresents the first convolution layer, ↓sRepresenting the convolution by s, qtRepresenting the second deconvolution layer.
Extracting the characteristic information Lt-1As the first deconvolution layer p in the up-sampling unittThe first deconvolution layer ptFor the characteristic information Lt-1Deconvolution s times to obtain characteristic information
Feature information
Input to the first winding layer gtIn the first winding layer gtFor characteristic information
Obtaining characteristic information by convolution s times
Feature information
And characteristic information Lt-1Processing residual error to obtain characteristic information
Feature information
As a second deconvolution layer qtInput of (a) a second deconvolution layer qtFor characteristic information
Deconvolution s times to obtain characteristic information
For characteristic information
And characteristic information
Is used for processing and transportingOut of feature information Ht。
Outputting the characteristic information HtAs input to the down-sampling unit, feature information HtThe procedure in the down-sampling unit is as follows:
a second convolution layer:
a third deconvolution layer:
residual error:
a third convolutional layer:
and (3) outputting:
wherein, denotes a convolution operation, HtDenotes the input of the second buildup layer, g'tDenotes a second buildup layer, p'tDenotes a third deconvolution layer, g'tRepresents the third convolution layer ↓ ℃ @sRepresenting deconvolution s times ↓sRepresenting the convolution by a factor of s.
Second convolutional layer g'tFor characteristic information HtObtaining characteristic information by convolution s times
Feature information
To the third deconvolution layer p'tOf (d), a third deconvolution layer p'tFor characteristic information
Deconvolution s times to obtain characteristic information
The characteristic information HtAnd characteristic information
Processing residual error to obtain characteristic information
Feature information
To the third buildup layer g'tMiddle, third convolution layer g'tFeature information
Obtaining characteristic information by convolution s times
For characteristic information
And characteristic information
Do processing output characteristic information LtAnd the characteristic information L is combinedtAnd performing iterative error feedback processing as an input of the up-sampling unit until the required iteration number is reached.
For example, a satellite video transmitted back to the ground is extracted frame by frame, the extracted image is introduced into a neural network model, the neural network model extracts the characteristic information of the image, and an intensive depth feedback network in the neural network model performs iterative error feedback processing on the extracted characteristic information, so that a high-resolution image is output.
In this embodiment, a neural network model extracts feature information of image data to be reconstructed, an dense depth feedback network in the neural network model performs iterative error feedback processing on the extracted feature information, that is, a plurality of upsampling units and a plurality of downsampling units in the dense depth feedback network perform iterative error feedback processing on the feature information, and a high-resolution image and a low-resolution image are repeatedly and alternately calculated.
Example 2
The present embodiment provides an image data processing apparatus for processing a low-resolution image to obtain an effective high-resolution image, as shown in fig. 2, including:
a first obtaining unit 201, configured to obtain image data to be reconstructed. For details, please refer to the related description of step S101 corresponding to method embodiment 1, which is not repeated herein.
An extracting unit 202, configured to extract feature information of the image data to be reconstructed. For details, please refer to the related description of step S102 corresponding to method embodiment 1, which is not repeated herein.
And the output unit 203 is configured to use the feature information as an input of a pre-constructed dense depth feedback network, where the dense depth feedback network is a network architecture formed by densely connecting a plurality of upsampling units and a plurality of downsampling units, and performs iterative error feedback processing on the feature information by using the plurality of upsampling units and the plurality of downsampling units to output a high-resolution image. For details, please refer to the related description of step S103 corresponding to method embodiment 1, which is not repeated herein.
In this embodiment, the first obtaining module 201 obtains an image with a low resolution, the extracting module 202 extracts feature information of the obtained image with the low resolution, the dense depth feedback network in the output unit 203 performs iterative error feedback processing on the extracted feature information, that is, a plurality of up-sampling units and a plurality of down-sampling units in the dense depth feedback network perform iterative error feedback processing on the feature information, and repeatedly and alternately calculate the image with the high resolution and the image with the low resolution.
Example 3
The motion condition of the ship is used as an important target of the perception information of the coastal area, and the monitoring and tracking of the offshore ship are beneficial to maintaining the national ocean safety. Fig. 3 is a flowchart illustrating how to track a ship by performing super-resolution reconstruction on a captured ship video image to obtain an effective high-resolution video according to some embodiments of the present invention. Although the processes described below include operations that occur in a particular order, it should be appreciated that the processes may include more or fewer operations, which may be performed sequentially or in parallel (e.g., using parallel processors or a multi-threaded environment).
S301, acquiring a to-be-tracked video image of the shot ship.
In the above embodiment, the video image to be tracked may be a video captured by a monitoring camera, a video captured by a satellite, or the like. However, since the ordinary video surveillance can only monitor ships near shore and can not monitor ships far away from shore, the video satellite can not be limited by distance, and the real-time performance and the wide breadth of the video satellite provide effective data for the tracking of the marine ships. Therefore, in the embodiment, the ship video images shot by the video satellite are used for super-resolution reconstruction and ship tracking. In some embodiments, video captured by a surveillance camera, or some other satellite may also be used. The method can be reasonably selected by a person skilled in the art according to specific situations, and is not limited herein.
S302, preprocessing the video image to be tracked by using the image data processing method in the embodiment 1 to obtain a processed video image.
The video image super-resolution reconstruction is based on image super-resolution, and video inter-frame correlation is utilized to carry out video super-resolution reconstruction. In the above embodiment, since the acquired video images are obtained, the acquired video images need to be extracted frame by frame to form a corresponding image sequence. The image data processing method described in embodiment 1 is used to process an image sequence to obtain a high-resolution image with a good effect, and the obtained high-resolution image is converted into a high-resolution video image.
And S303, carrying out ship tracking detection according to the processed video image.
In the above embodiment, step S302 has performed super-resolution reconstruction on the low-resolution satellite video, so as to obtain an effective high-resolution video image. The ship tracking is carried out by utilizing the obtained high-resolution video image with good effect, so that the ship tracking detection effect is better, and the error rate of tracking detection is reduced. In the present embodiment, the method for tracking the ship may be implemented by using the existing technology, and will not be described herein too much.
In this embodiment, the image data processing method described in embodiment 1 is used to preprocess the obtained video image to be tracked, which is shot of the ship, to obtain an effective high-resolution video image, and the processed video image is used to perform ship tracking detection, so that the ship can be more accurate, the ship tracking detection effect is improved, and the error rate of tracking detection is reduced.
Example 4
The embodiment provides a ship tracking device, which is used for performing super-resolution reconstruction and ship tracking on a shot ship video image, and as shown in fig. 4, the ship tracking device comprises:
and a second acquiring unit 401, configured to acquire a video image to be tracked of the ship. For details, please refer to the related description of step S301 corresponding to embodiment 3 of the method, which is not repeated herein.
A processing unit 402, configured to perform preprocessing on the video image to be tracked by using the image data processing method described in embodiment 1, so as to obtain a processed video image. For details, please refer to the related description of step S302 corresponding to embodiment 3 of the method, which is not repeated herein.
And a tracking unit 403, configured to perform ship tracking detection according to the processed video image. For details, please refer to the related description of step S303 corresponding to method embodiment 3, which is not repeated herein.
In this embodiment, the second acquiring unit 401 acquires a to-be-tracked video image with a ship, the processing unit 402 preprocesses the to-be-tracked video image acquired by the second acquiring unit 401 to obtain an effective high-resolution video image, and the tracking unit 403 performs ship tracking detection by using the processed video image, so that the ship can be more accurate, the ship tracking detection effect is improved, and the error rate of tracking detection is reduced.
Example 5
The present embodiment provides an electronic device, as shown in fig. 5, the electronic device includes a processor 501 and a memory 502, where the processor 501 and the memory 502 may be connected by a bus or by other means, and fig. 5 takes the connection by the bus as an example.
The memory 501, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions/modules corresponding to the image data processing method in embodiment 1 or the ship tracking method in embodiment 3 of the present invention (e.g., the first acquiring unit 201, the extracting unit 202, and the output unit 203 shown in fig. 2, or the second acquiring unit 401, the processing unit 402, and the tracking unit 403 shown in fig. 4). The processor 501 executes various functional applications and data processing of the processor 501 by running the non-transitory software programs, instructions and modules stored in the memory 502, that is, the image data processing method in embodiment 1 or the ship tracking method in embodiment 3 of the above method is implemented.
The memory 502 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by the processor 501, and the like. Further, the memory 502 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 502 optionally includes memory located remotely from processor 501, which may be connected to processor 501 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more modules are stored in the memory 502 and, when executed by the processor 501, perform the image data processing method shown in fig. 1 or the ship tracking method shown in fig. 3.
In this embodiment, the processor 501 executes the non-transitory software program, instructions and modules stored in the memory 502 to execute the image data processing method of embodiment 1 or the ship tracking method of embodiment 3, so as to obtain an effective high-resolution image, or make the ship follow more accurately, improve the ship tracking detection effect, and reduce the error rate of tracking detection.
The details of the computer device can be understood by referring to the corresponding related descriptions and effects in the embodiments shown in fig. 1 or fig. 3, and are not described herein again. For details of the technology that are not described in detail in this embodiment, reference may be made to the related description in the embodiment shown in fig. 1 or fig. 3.
The embodiment of the invention also provides a non-transitory computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions can execute the image data processing method or the ship tracking method in any method embodiment. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, abbreviated as HDD), a Solid State Drive (SSD), or the like; the storage medium may also comprise a combination of memories of the kind described above.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. And obvious variations or modifications therefrom are within the scope of the invention.
Claims (8)
1. An image data processing method, comprising the steps of:
acquiring image data to be reconstructed, wherein the image data is image data shot by a video satellite;
extracting characteristic information of the image data to be reconstructed;
taking the characteristic information as the input of a pre-constructed dense depth feedback network, wherein the dense depth feedback network is a network architecture formed by densely connecting a plurality of up-sampling units and a plurality of down-sampling units, and performing iterative error feedback processing on the characteristic information by utilizing the plurality of up-sampling units and the plurality of down-sampling units to output a high-resolution image;
each up-sampling unit is connected with at least one down-sampling unit, and the output ends of all the up-sampling units are connected in series;
wherein the method further comprises:
a. convolving the width and the height of each high-resolution image by k times to obtain a low-resolution image sequence; wherein k is an integer of 2-8, and each high-resolution image is a static remote sensing image;
b. carrying out video coding on the low-resolution image sequence according to H.264, wherein the code rate of each pixel is not lower than 1.98bps, and obtaining a compressed low-resolution image sequence;
c. and coding the compressed low-resolution image sequence according to H.264 to obtain a low-resolution training image which is decoded and restored but has a compression distortion effect, and taking the low-resolution training image as a training set of a neural network model.
2. The image data processing method of claim 1, wherein the up-sampling unit comprises at least a first anti-convolution layer, a first convolution layer and a second anti-convolution layer connected in sequence, and the down-sampling unit comprises at least a second convolution layer, a third anti-convolution layer and a third convolution layer connected in sequence; wherein the convolutional layer is used for reducing the image resolution; the deconvolution layer is used to magnify the image resolution.
3. The image data processing method of claim 1 or 2, wherein extracting feature information of the image data to be reconstructed comprises:
extracting a large amount of characteristic information through a fourth convolution layer, and reducing the amount of the characteristic information through a fifth convolution layer, wherein the convolution kernel of the fourth convolution layer is larger than that of the fifth convolution layer.
4. A method of vessel tracking, comprising the steps of:
acquiring a video image to be tracked of a shot ship;
preprocessing the video image to be tracked by using the image data processing method of any one of claims 1 to 3 to obtain a processed video image;
and carrying out ship tracking detection according to the processed video image.
5. An image data processing apparatus, comprising:
the device comprises a first acquisition unit, a second acquisition unit and a reconstruction unit, wherein the first acquisition unit is used for acquiring image data to be reconstructed, and the image data is image data shot by a video satellite;
the extraction unit is used for extracting the characteristic information of the image data to be reconstructed;
the output unit is used for taking the characteristic information as the input of a pre-constructed dense depth feedback network, the dense depth feedback network is a network architecture formed by densely connecting a plurality of up-sampling units and a plurality of down-sampling units, the characteristic information is subjected to iterative error feedback processing by utilizing the plurality of up-sampling units and the plurality of down-sampling units, and a high-resolution image is output; each up-sampling unit is connected with at least one down-sampling unit, and the output ends of all the up-sampling units are connected in series;
the device further comprises:
the convolution module is used for convolving the width and the height of each high-resolution image by k times to obtain a low-resolution image sequence; wherein k is an integer of 2-8, and each high-resolution image is a static remote sensing image;
the first coding module is used for carrying out video coding on the low-resolution image sequence according to H.264, wherein the code rate of each pixel is not lower than 1.98bps, and a compressed low-resolution image sequence is obtained;
and the second coding module is used for coding the compressed low-resolution image sequence according to H.264 to obtain a low-resolution training image which is decoded and restored but has a compression distortion effect, and taking the low-resolution training image as a training set of the neural network model.
6. A vessel tracking device, comprising:
the second acquisition unit is used for acquiring a video image to be tracked of the shot ship;
a processing unit, configured to perform preprocessing on the video image to be tracked by using the image data processing method according to any one of claims 1 to 3, so as to obtain a processed video image;
and the tracking unit is used for carrying out ship tracking detection according to the processed video image.
7. An electronic device, comprising a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the image data processing method according to any one of claims 1 to 3 or the ship tracking method according to claim 4.
8. A computer-readable storage medium storing computer instructions for causing a computer to execute the image data processing method according to any one of claims 1 to 3 or the ship tracking method according to claim 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011257995.1A CN112258394B (en) | 2020-11-11 | 2020-11-11 | Data processing method, ship tracking method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011257995.1A CN112258394B (en) | 2020-11-11 | 2020-11-11 | Data processing method, ship tracking method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112258394A CN112258394A (en) | 2021-01-22 |
| CN112258394B true CN112258394B (en) | 2022-04-29 |
Family
ID=74265504
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011257995.1A Active CN112258394B (en) | 2020-11-11 | 2020-11-11 | Data processing method, ship tracking method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112258394B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115810016B (en) * | 2023-02-13 | 2023-04-28 | 四川大学 | Automatic identification method, system, storage medium and terminal for CXR (Lung infection) image |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109740665A (en) * | 2018-12-29 | 2019-05-10 | 珠海大横琴科技发展有限公司 | Shielded image ship object detection method and system based on expertise constraint |
| CN109741260A (en) * | 2018-12-29 | 2019-05-10 | 天津大学 | A kind of efficient super-resolution method based on depth back projection network |
| US10410322B2 (en) * | 2017-04-05 | 2019-09-10 | Here Global B.V. | Deep convolutional image up-sampling |
| CN111353940A (en) * | 2020-03-31 | 2020-06-30 | 成都信息工程大学 | Image super-resolution reconstruction method based on deep learning iterative up-down sampling |
| CN111445388A (en) * | 2019-12-27 | 2020-07-24 | 珠海大横琴科技发展有限公司 | Image super-resolution reconstruction model training method, ship tracking method and ship tracking device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110087092B (en) * | 2019-03-11 | 2020-06-05 | 西安电子科技大学 | Low-bit-rate video coding and decoding method based on image reconstruction convolutional neural network |
-
2020
- 2020-11-11 CN CN202011257995.1A patent/CN112258394B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10410322B2 (en) * | 2017-04-05 | 2019-09-10 | Here Global B.V. | Deep convolutional image up-sampling |
| CN109740665A (en) * | 2018-12-29 | 2019-05-10 | 珠海大横琴科技发展有限公司 | Shielded image ship object detection method and system based on expertise constraint |
| CN109741260A (en) * | 2018-12-29 | 2019-05-10 | 天津大学 | A kind of efficient super-resolution method based on depth back projection network |
| CN111445388A (en) * | 2019-12-27 | 2020-07-24 | 珠海大横琴科技发展有限公司 | Image super-resolution reconstruction model training method, ship tracking method and ship tracking device |
| CN111353940A (en) * | 2020-03-31 | 2020-06-30 | 成都信息工程大学 | Image super-resolution reconstruction method based on deep learning iterative up-down sampling |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112258394A (en) | 2021-01-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111709895A (en) | Image blind deblurring method and system based on attention mechanism | |
| CN111598778B (en) | Super-resolution reconstruction method for insulator image | |
| CN111369440B (en) | Model training and image super-resolution processing method, device, terminal and storage medium | |
| Zhang et al. | One-two-one networks for compression artifacts reduction in remote sensing | |
| CN110599401A (en) | Remote sensing image super-resolution reconstruction method, processing device and readable storage medium | |
| CN111105352A (en) | Super-resolution image reconstruction method, system, computer device and storage medium | |
| CN111127325B (en) | Satellite video super-resolution reconstruction method and system based on cyclic neural network | |
| CN111091503A (en) | Image out-of-focus blur removing method based on deep learning | |
| CN109146813B (en) | Multitask image reconstruction method, device, equipment and medium | |
| CN109191411B (en) | Multitask image reconstruction method, device, equipment and medium | |
| CN111951195A (en) | Image enhancement method and device | |
| Dong et al. | Real-world remote sensing image super-resolution via a practical degradation model and a kernel-aware network | |
| CN113362338B (en) | Rail segmentation method, device, computer equipment and rail segmentation processing system | |
| CN114140346A (en) | Image processing method and device | |
| Chen et al. | Image denoising via deep network based on edge enhancement | |
| CN116485741A (en) | No-reference image quality evaluation method, system, electronic equipment and storage medium | |
| CN112258394B (en) | Data processing method, ship tracking method, device, equipment and storage medium | |
| Wang et al. | Afdn: Attention-based feedback dehazing network for UAV remote sensing image haze removal | |
| Ren et al. | A lightweight object detection network in low-light conditions based on depthwise separable pyramid network and attention mechanism on embedded platforms | |
| CN117314750A (en) | Image super-resolution reconstruction method based on residual error generation network | |
| Shin et al. | LoGSRN: Deep super resolution network for digital elevation model | |
| Cai et al. | Real-time super-resolution for real-world images on mobile devices | |
| Chen et al. | Nested error map generation network for no-reference image quality assessment | |
| CN116266336A (en) | Video super-resolution reconstruction method, device, computing equipment and storage medium | |
| CN114219738A (en) | Single-image multi-scale super-resolution reconstruction network structure and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |