CN111985475A - Ship board identification method, computing device and storage medium - Google Patents
Ship board identification method, computing device and storage medium Download PDFInfo
- Publication number
- CN111985475A CN111985475A CN202010874286.1A CN202010874286A CN111985475A CN 111985475 A CN111985475 A CN 111985475A CN 202010874286 A CN202010874286 A CN 202010874286A CN 111985475 A CN111985475 A CN 111985475A
- Authority
- CN
- China
- Prior art keywords
- ship
- image
- sequence
- region
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000003702 image correction Methods 0.000 claims abstract description 12
- 230000002457 bidirectional effect Effects 0.000 claims description 22
- 230000015654 memory Effects 0.000 claims description 16
- 238000012544 monitoring process Methods 0.000 claims description 14
- 239000013598 vector Substances 0.000 claims description 13
- 238000000605 extraction Methods 0.000 claims description 12
- 238000012937 correction Methods 0.000 claims description 10
- 238000004891 communication Methods 0.000 claims description 9
- 238000012795 verification Methods 0.000 claims description 8
- 238000005070 sampling Methods 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 230000007246 mechanism Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 230000009466 transformation Effects 0.000 description 5
- 238000001514 detection method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000007787 long-term memory Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000006403 short-term memory Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 239000003086 colorant Substances 0.000 description 3
- 230000001788 irregular Effects 0.000 description 3
- 238000011176 pooling Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000013524 data verification Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000010191 image analysis Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012896 Statistical algorithm Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000005452 bending Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/63—Scene text, e.g. street names
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
- G06V10/247—Aligning, centring, orientation detection or correction of the image by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Character Discrimination (AREA)
Abstract
The invention discloses a ship license plate identification method, a computing device and a storage medium, relates to the technical field of automation, and aims to solve the problem that ship license plate information cannot be identified remotely. The method of the invention comprises the following steps: capturing a ship image through a binocular camera; carrying out region identification on the ship image to obtain a ship plate region image; carrying out image correction on the ship board area image; and performing character recognition on the corrected ship plate area image to obtain ship plate information. The method can realize remote accurate identification of the ship cards in the channel.
Description
Technical Field
The invention relates to the technical field of automation, in particular to a ship board identification method, a computing device and a storage medium.
Background
With the rapid development of maritime trade shipping industry and inland river shipping industry, the identity of a ship, particularly a ship plate, is effectively identified, and the method has important significance for waterway traffic supervision, waterway transportation safety, coastal environment pollution control and the like.
The artificial intelligence correlation techniques such as image analysis and deep learning have made a lot of important progress in the land intelligent transportation systems such as license plate recognition and automatic driving, however, the application of the correlation techniques in the field of the land intelligent transportation is still lacking. In recent years, many advanced methods and systems are focused on solving the problem of ship identity authentication, and mainly perform ship detection and character positioning classification and identification based on images or videos, however, few technologies can solve the problem of ship and board identification in consideration of the complex environment of a navigation channel, a long identification distance, uncertain factors such as the positions of ship numbers, character sizes and arrangement modes of different types of ships in different periods of time and weather.
For example, the detection distance of a channel, particularly a port channel, is often long, a common camera can only capture a rough contour of a ship, and the cut ship picture has insufficient resolution, so that the ship number is difficult to effectively identify.
For another example, the biggest difference between a ship license plate and a license plate is that the ship license plate may be at any position of a ship body, the sizes, colors and shapes of fonts are different, and the arrangement mode of the ship number also has various situations such as horizontal bodies, vertical bodies and even arcs.
Disclosure of Invention
In view of the above problems, the present invention has been made to provide a ship board recognition method, a computing device, and a storage medium that overcome or at least partially solve the above problems.
In a first aspect, the present invention provides a ship board identification method, including:
capturing a ship image through a binocular camera;
carrying out region identification on the ship image to obtain a ship plate region image;
carrying out image correction on the ship board area image;
and performing character recognition on the corrected ship plate area image to obtain ship plate information.
In a second aspect, the invention provides a computing device comprising: the camera, the processor, the memory, the communication interface, the communication bus and the output unit are communicated with each other through the communication bus;
the memory is configured to store at least one executable instruction for controlling the computing device to:
capturing a ship image through a binocular camera;
carrying out region identification on the ship image to obtain a ship plate region image;
carrying out image correction on the ship board area image;
and performing character recognition on the corrected ship plate area image to obtain ship plate information.
In a third aspect, the present invention provides a computer storage medium having at least one executable instruction stored therein, the executable instruction being configured to:
capturing a ship image through a binocular camera;
carrying out region identification on the ship image to obtain a ship plate region image;
carrying out image correction on the ship board area image;
and performing character recognition on the corrected ship plate area image to obtain ship plate information.
According to the ship license plate identification method, the computing equipment and the storage medium, the binocular cameras are adopted to carry out image monitoring on the navigation channel, one camera is responsible for monitoring and searching a target ship in a full-state mode, the other camera is responsible for zooming and shooting a clear ship image, aiming at the problems that the navigation channel monitoring distance is long and the detection target is small, the detection distance can be effectively increased, the clear ship image can be captured, video streams and pictures can be processed in a compatible mode, the application range is wider, the single-point deployment can effectively cover the width of a conventional navigation channel, and the maximum range can cover 4 kilometers; in the aspect of character recognition, the ship plate region is comprehensively recognized through character segmentation and character association, the accuracy of ship plate region recognition can be effectively improved, image correction steps are added, the universality of recognition means is improved aiming at the problem that ship plate information lacks of style normalization, and ship plate information of different fonts, sizes, colors, shapes, arrangement modes (horizontal and vertical, irregular bending and the like) and the like can be effectively processed; in the aspect of ship plate information identification, a bidirectional identification mechanism is adopted, the front-back relevance of characters is considered, an attention mechanism is introduced, and the accuracy of character identification can be effectively improved; in addition, for the identified ship plate information, the AIS information is further combined to perform data fusion, the accuracy is further improved through a multi-dimensional data verification means, and meanwhile, the illegal ship state is screened out.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 is a flowchart illustrating a ship board recognition method according to an embodiment of the present invention;
FIG. 2 is a flow chart illustrating another vessel identification method provided by an embodiment of the present invention;
fig. 3 is a schematic diagram illustrating a processing mechanism of a binocular camera according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a VGG network provided by an embodiment of the present invention;
FIG. 5 is a flow chart illustrating an image rectification method according to an embodiment of the present invention;
FIG. 6 is a diagram illustrating a BLSTM + ATTENTION network according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a computing device according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Example one
The embodiment of the invention provides a ship board identification method, which is characterized in that a binocular camera is used for monitoring a channel image, a server is used for processing image processing, character identification, data storage and the like, and man-machine interaction equipment is used for outputting the identified ship board information. The binocular camera can be deployed at a fixed position around a navigation channel and can also be deployed on a ship in a moving scene; the server is usually deployed in a computer room remotely or can also be in the cloud on a third party; the human-computer interaction device can be a fixed device or a portable device. The three are in data transmission through mobile data networks such as GSM, 3G, 4G, 5G and the like, wireless hotspots or wired networks. It should be noted that, the embodiment of the present invention is described by taking a channel scene as an example, but the embodiment of the present invention is not only applicable to the channel scene, and in practical applications, when the binocular camera is deployed on a ship, the ship identification can be performed for any specific sea area.
As shown in fig. 1, the method of the present invention comprises:
s101, capturing a ship image through a binocular camera.
One camera is used for carrying out image monitoring on the channel, identifying ships appearing in the channel, and obtaining an amplified high-definition image of the ships through shooting of the other camera after the ships are identified. In practical application, in order to expand the monitoring range, a wide-angle camera can be used for channel monitoring, in order to obtain an amplified high-definition image, a zoom camera is preferably used for capturing a ship image, and the embodiment does not limit brands, parameters and specific performances of the binocular camera.
S102, carrying out area identification on the ship image to obtain a ship plate area image.
After the ship image taking the overall ship outline as the image main body is obtained, the ship plate area is further identified through the step, and a data basis is provided for a subsequent character identification link. In practical application, the ship license plate information is usually printed on the outer wall of a ship body, but due to lack of relevant specifications, the ship license plate information may appear at any position of the ship body, so that image region extraction cannot be performed by adopting a simple statistical algorithm.
And S103, carrying out image correction on the ship plate area image.
The ship board information of different ships has differences in the aspects of fonts, sizes, colors, arrangement modes and the like, and in order to improve the universality of the scheme of the embodiment, an image correction link is added in the step, so that the character recognition accuracy is improved particularly for the ship board with the special-shaped pattern. The image rectification also uses a deep learning network, and in one implementation mode, the image rectification can be realized by adopting a structure of cascade connection of a positioning network and a rectification network.
And S104, performing character recognition on the corrected ship plate area image to obtain ship plate information.
And performing character recognition on the corrected character area to obtain the ship plate information. The present embodiment adopts a sequence-to-sequence model, which is composed of an encoder and a decoder, and obtains the character recognition result by a prediction means, and the present embodiment does not limit the specific network structure of the encoder and the decoder.
Example two
Further, as a refinement and improvement of the solution of the foregoing embodiment, the present invention also provides another ship board recognition method, as shown in fig. 2, including:
s201, monitoring the image of the navigation channel through the wide-angle camera, and identifying the target ship.
The wide-angle camera monitors images of a navigation channel, transmits a shot video stream to the server, positions a ship in the images through an object tracking recognition algorithm by the server, and obtains coordinate information of the target ship and the minimum circumscribed rectangle of a ship body through an image analysis technology. And then converting the coordinate information and the minimum circumscribed rectangle into shooting parameters of the zoom camera, such as zoom multiple, lens shooting angle and the like, and sending the parameters to the zoom camera in a PTZ command form.
S202, when the target ship is monitored, shooting through a zoom camera to obtain a ship image of the target ship.
The zooming camera changes the angle of the camera according to the PTZ instruction, zooms and amplifies the ship body in the image, so that the ship body is located at the main body position in the image, and a high-definition ship image is shot after focusing is finished.
The foregoing steps S201 and S202 are as shown in fig. 3.
The embodiment adopts the binocular camera to carry out channel panoramic monitoring and specific ship high definition catch respectively, can effectively improve the resolution ratio of ship image, provides data basis for the accurate identification of follow-up ship tablet information, especially adapts to the remote image capture characteristics under the marine environment.
S203, carrying out area identification on the ship image through a VGG network to obtain a ship plate area image.
In the embodiment, the VGG network is used for identifying the ship plate area image, and has the characteristics of small convolution kernels (1 x1 to 3x3 can be set) and small pooling kernels (all pooling kernels of 2x 2), and meanwhile, the feature map with the deeper layers is wider. Because the convolution kernel is focused on expanding the number of channels and pooling is focused on reducing the width and the height, the model architecture is deeper and wider, and meanwhile, the increase of the calculation amount is slowed down. In addition, the VGG network also has the characteristics of full connection to convolution, three full connections in the training stage are replaced by three convolutions in the network testing stage, and parameters in training are reused in testing, so that the full convolution network obtained through testing can receive input with any width or height because the full convolution network does not have the limitation of full connection.
Specifically, as shown in fig. 4, the entire network is composed of a Conv layer, a maxporoling layer and an FC layer, the Conv layer is used to extract image features, and if a convolution kernel is 6433, it has 64 channels, that is, 64 features are extracted. The Maxpooling layer is used for carrying out feature screening according to a corresponding max or average mode, and meanwhile, oversampling can reduce overfitting, and deepening and widening of a network are facilitated. The FC layer has the maximum parameter number of the whole network, and in the VGG network, the influence of the node number of the FC on the network prediction effect is small.
The ship image is used as network input and is subjected to feature extraction through a Conv layer, then cross-connection up-sampling operation is adopted on a Maxpooling layer to decode the picture, more bottom layer features can be utilized similar to a U-net structure, and finally two scores, namely a region score and an association score, are output on an FC layer. The region score is the region position of the divided character, and the association score is the region position of the association between the character and the character. The ship plate area image is obtained by combining the area fraction and the association fraction for verification, the character area which is randomly arranged at any position can be accurately identified, the ship plate which is printed at any position and in any arrangement mode can be effectively identified in the practical application, and the universality and the practical value of the scheme are improved.
In this embodiment, a VGG network is taken as an example for explanation, and in practical application, a *** lenet or a conventional CNN network may be used instead of the VGG network.
And S204, correcting the ship plate area image by using a cascade network of the positioning network and the correction network.
The correction link is used for carrying out space change on the irregular bent ship number and the transverse and vertical ship number, converting the bent and irregular characters into the horizontal characters and reducing invalid areas. In this embodiment, a two-stage cascade structure of a positioning network and a correction network is adopted to perform 2D transformation on the ship plate area image, as shown in fig. 5, the ship plate area image is firstly input into the positioning network, regression fitting is performed on the key point positions of characters in the ship plate area image, and then 2D transformation is performed on the key point positions of the characters through the correction network. Specifically, the method comprises the following steps:
the positioning network is used for regression fitting of the positions C ', C' ═ C 'of the key points of the characters in the image'1,...,c′K]∈R2×KDenotes K control points, ck=[xk,yk]TThe abscissa and ordinate of the K-th point are shown.
The correction network adopts a Thin Plate Spline function (TPS for short) to carry out 2D transformation on real C and C'.
The specific principle of TPS is as follows:
knowing K control points C i1.. K }, the original coordinates can be transformed to another coordinate with a radial basis function for a given x-point:
where σ (r) ═ r2log r is the radial basis function kernel. Each new value is affected by all other non-one-to-one control points. Written as a vector is in the form:
yk=Φ(xk)
where the interpolation function can be written as:
Φ1(x)=c+aTx+wTs(x)
where c is a scalar quantity, a ∈ R2×1Represents 2 dimensions, w ∈ RN×1The function vector is:
s(x)=(σ(||x-x1||),σ(||x-x2||),...,σ(||x-xN||))T
by solving, one can get:
and further obtaining:
the purpose of 2D transformation can be realized by mapping points on the original data to the new data through the transformation matrix.
And S205, performing character recognition by adopting a sequence-to-sequence model to obtain the ship plate information.
In this embodiment, a sequence-to-sequence model is used for character recognition, and in one implementation, a bidirectional lstm + attention mode may be used. LSTM is a time-recursive neural network, known as a long-short term memory network, suitable for processing and predicting significant events of relatively long intervals and delays in a time series. Sometimes the prediction may need to be determined by the first and the last inputs, which is more accurate, so that a bidirectional long-short term memory network, i.e. BLSTM, is proposed. Compared with single character recognition, the bidirectional lstm + attention mechanism provided by the embodiment can consider the front and back relevance of characters, introduce an attention mechanism and effectively improve the accuracy of character recognition.
The sequence-to-sequence model comprises an encoder and a decoder, wherein the encoder comprises a convolution network (ConvNet) and a bidirectional long-short term memory network (BLSTM), the characteristics of the ship plate region image are extracted through the convolution network, the bidirectional sequence relation of the characteristics is analyzed through the bidirectional long-short term memory network, a plurality of characteristic sequences with equal length are output, and the characteristic sequences are recorded as H [ [ H ] ]1,...,hn],wheren=ωconv。
The decoder is an attention sequence to sequence model and is used for converting the characteristic sequence into a character sequence, and the output characters are predicted through multi-step iterative calculation to obtain a recognition result. Specifically, the method comprises the following steps: suppose the attention sequence model iterates through T steps, resulting in a character of length T, denoted as (y)1,...,yT) Then, in step t, the decoder bases on the output H of the encoder, the intermediate state st-1And y predicted in the previous stept-1Predicting the current output ytAn end-of-sequence symbol (EOS) and outputs the final prediction result as a character recognition result.
Specifically, the method comprises the following steps:
1. the decoder first calculates the attention weight vector alphatThe attention mechanism is expressed as:
wherein W, V are training weights.
2. The attention weight indicates the importance of each feature output by the decoder, and adding the attention weight to H can result in:
which is used to represent a portion of the entire encoding character H.
3. RNN based on gtIntermediate state st-1And y predicted in the previous stept-1Performing operation to obtain new state xt,st:
(xt,st)=mn(st-1,(gt,f(yt-1)))
And finally, predicting the output of the current character by using xt:
p(yt)=soft max(Woxt+bo)
yt~p(yt)
the character recognition process described above is illustrated in fig. 6.
S206, acquiring AIS information of all ships in the preset sea area range, and verifying the identified ship plate information according to the acquired AIS information.
The camera is connected with the host computer, and the host computer acquires AIS information of nearby ships through the AIS interface so as to perform reinforced verification on the ship plate information identified in S205. If the identified ship plate information exists in the acquired AIS information, the identification is successful, and meanwhile, the state information of the ship can be acquired from the AIS information of the ship through the incidence relation of the ship plate information.
In practical applications, the acquired AIS information includes, but is not limited to, ship identity information (type, ship name, call sign, MMSI, IMO), location, heading, speed, destination, and other information. In practical application, the preset sea area range can be set according to an empirical value (for example, a 5-sea range near the position of the camera), the larger the range is, the more the number of ships is involved, the higher the verification success rate is, and meanwhile, the larger the calculation amount is. In one implementation, the preset sea area range may be set in a linkage manner according to the coverage range of the binocular camera, for example, the preset sea area range may be calculated according to the angle of the wide-angle camera and the zoom factor of the zoom camera. When the ship corresponding to the AIS information is not matched in the preset sea area range, the preset sea area range may be gradually expanded according to a preset step length (for example, 1 nautical mile) until the ship is successfully matched or a maximum tolerance range threshold is reached.
The step can further combine AIS information to perform data fusion on the basis of accurately identifying the ship plate information, further improve the accuracy rate by means of multidimensional data verification, and simultaneously screen out illegal ship states.
EXAMPLE III
As shown in fig. 7, the present invention also provides a computing device comprising: the camera, the processor, the memory, the communication interface and the output unit complete mutual communication through the communication bus;
the memory is configured to store at least one executable instruction for controlling the computing device to:
capturing a ship image through a binocular camera;
carrying out region identification on the ship image to obtain a ship plate region image;
carrying out image correction on the ship board area image;
and performing character recognition on the corrected ship plate area image to obtain ship plate information.
Further, the binocular camera is composed of a wide-angle camera and a zoom camera, and the computing device is configured to:
monitoring images of a navigation channel through the wide-angle camera, and identifying a target ship;
transmitting the coordinate information of the target ship and the minimum external rectangle of the ship body to the zooming camera;
and controlling the zoom camera to amplify and focus the ship body based on the coordinate information and the minimum external rectangle, and shooting to obtain the ship image.
Further, the computing device is to:
performing feature extraction on the ship image through a VGG network;
performing up-sampling decoding on the image;
calculating and outputting a region score and an association score, wherein the region score is the position of the segmented ship board region, and the association score is the position of the association between the characters in the ship board region;
obtaining the ship board area image by using the area score and the association score in combination with verification.
Further, the computing device is to:
inputting the ship plate area image into a positioning network, and performing regression fitting on the key point positions of characters in the ship plate area image;
and 2D conversion is carried out on the positions of the key points of the characters through the correction network.
Further, the computing device is to:
character recognition is performed using a sequence-to-sequence model, which consists of an encoder and a decoder.
Furthermore, the encoder comprises a convolution network and a bidirectional long and short term memory network, the feature extraction is carried out on the ship plate area image through the convolution network, the bidirectional sequence relation of the features is analyzed through the bidirectional long and short term memory network, and a plurality of feature sequences with equal length are output.
Furthermore, the decoder is an attention sequence-to-sequence model and is used for converting the characteristic sequence into a character sequence, and predicting the output characters through multi-step iterative computation to obtain a recognition result.
Further, the computing device is to:
the decoder calculates a weight vector of attention;
distributing different weight vectors to each feature in the feature sequence;
and performing the latest state of the current round based on the weighting characteristics, the intermediate state and the prediction result of the previous step, and outputting the final prediction result as a character recognition result after completing the iteration of preset times.
Further, the computing device is to, after obtaining the ship placard information:
acquiring AIS information of all ships within a preset sea area range;
and verifying the identified ship plate information according to the acquired AIS information.
Example four
The present invention also provides a non-volatile computer storage medium having stored thereon at least one executable instruction that may perform the method of any of the above method embodiments.
The executable instructions may specifically perform the following operations:
capturing a ship image through a binocular camera;
carrying out region identification on the ship image to obtain a ship plate region image;
carrying out image correction on the ship board area image;
and performing character recognition on the corrected ship plate area image to obtain ship plate information.
Further, the binocular camera comprises a wide-angle camera and a zoom camera, and the ship image is captured through the binocular camera, including:
monitoring images of a navigation channel through the wide-angle camera, and identifying a target ship;
transmitting the coordinate information of the target ship and the minimum external rectangle of the ship body to the zooming camera;
and controlling the zoom camera to amplify and focus the ship body based on the coordinate information and the minimum external rectangle, and shooting to obtain the ship image.
Further, the performing region identification on the ship image to obtain a ship plate region image includes:
performing feature extraction on the ship image through a VGG network;
performing up-sampling decoding on the image;
calculating and outputting a region score and an association score, wherein the region score is the position of the segmented ship board region, and the association score is the position of the association between the characters in the ship board region;
obtaining the ship board area image by using the area score and the association score in combination with verification.
Further, the image rectification of the image of the ship board area comprises:
inputting the ship plate area image into a positioning network, and performing regression fitting on the key point positions of characters in the ship plate area image;
and 2D conversion is carried out on the positions of the key points of the characters through the correction network.
Further, the character recognition of the corrected ship plate area image to obtain the ship plate information includes:
character recognition is performed using a sequence-to-sequence model, which consists of an encoder and a decoder.
Furthermore, the encoder comprises a convolution network and a bidirectional long and short term memory network, the feature extraction is carried out on the ship plate area image through the convolution network, the bidirectional sequence relation of the features is analyzed through the bidirectional long and short term memory network, and a plurality of feature sequences with equal length are output.
Furthermore, the decoder is an attention sequence-to-sequence model and is used for converting the characteristic sequence into a character sequence, and predicting the output characters through multi-step iterative computation to obtain a recognition result.
Further, the predicting the output character through the multi-step iterative computation to obtain the recognition result includes:
the decoder calculates a weight vector of attention;
distributing different weight vectors to each feature in the feature sequence;
and performing the latest state of the current round based on the weighting characteristics, the intermediate state and the prediction result of the previous step, and outputting the final prediction result as a character recognition result after completing the iteration of preset times.
Further, after obtaining the ship plate information:
acquiring AIS information of all ships within a preset sea area range;
and verifying the identified ship plate information according to the acquired AIS information.
The algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system will be apparent from the description above. Moreover, the present invention is not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
Claims (9)
1. A ship board identification method, characterized in that the method comprises:
capturing a ship image through a binocular camera;
carrying out region identification on the ship image to obtain a ship plate region image;
carrying out image correction on the ship board area image;
and performing character recognition on the corrected ship plate area image to obtain ship plate information.
2. The method of claim 1, wherein the binocular camera is comprised of a wide angle camera and a zoom camera, the capturing of the ship image by the binocular camera comprises:
monitoring images of a navigation channel through the wide-angle camera, and identifying a target ship;
transmitting the coordinate information of the target ship and the minimum external rectangle of the ship body to the zooming camera;
and controlling the zoom camera to amplify and focus the ship body based on the coordinate information and the minimum external rectangle, and shooting to obtain the ship image.
Preferably, the performing region identification on the ship image to obtain a ship plate region image includes:
performing feature extraction on the ship image through a VGG network;
performing up-sampling decoding on the image;
calculating and outputting a region score and an association score, wherein the region score is the position of the segmented ship board region, and the association score is the position of the association between the characters in the ship board region;
obtaining the ship board area image by using the area score and the association score in combination with verification.
Preferably, the image rectification of the image of the ship board area includes:
inputting the ship plate area image into a positioning network, and performing regression fitting on the key point positions of characters in the ship plate area image;
and 2D conversion is carried out on the positions of the key points of the characters through the correction network.
Preferably, the character recognition of the corrected ship plate area image to obtain the ship plate information includes:
character recognition is performed using a sequence-to-sequence model, which consists of an encoder and a decoder.
Preferably, the encoder includes a convolution network and a bidirectional long-short term memory network, the feature extraction is performed on the ship plate area image through the convolution network, the bidirectional sequence relation of the features is analyzed through the bidirectional long-short term memory network, and a plurality of feature sequences with equal length are output.
Preferably, the decoder is an attention sequence-to-sequence model, and is configured to convert the feature sequence into a character sequence, and predict output characters through multi-step iterative computation to obtain a recognition result.
Preferably, the predicting the output character through the multi-step iterative computation to obtain the recognition result includes:
the decoder calculates a weight vector of attention;
distributing different weight vectors to each feature in the feature sequence;
and performing the latest state of the current round based on the weighting characteristics, the intermediate state and the prediction result of the previous step, and outputting the final prediction result as a character recognition result after completing the iteration of preset times.
3. The method of claim 1 or 2, wherein after obtaining the ship placard information, the method further comprises:
acquiring AIS information of all ships within a preset sea area range;
and verifying the identified ship plate information according to the acquired AIS information.
4. A computing device, wherein the computing device comprises: the camera, the processor, the memory, the communication interface and the output unit complete mutual communication through the communication bus;
the memory is configured to store at least one executable instruction for controlling the computing device to:
capturing a ship image through a binocular camera;
carrying out region identification on the ship image to obtain a ship plate region image;
carrying out image correction on the ship board area image;
and performing character recognition on the corrected ship plate area image to obtain ship plate information.
5. The computing device of claim 4, wherein the binocular camera is comprised of a wide angle camera and a zoom camera, the computing device configured to:
monitoring images of a navigation channel through the wide-angle camera, and identifying a target ship;
transmitting the coordinate information of the target ship and the minimum external rectangle of the ship body to the zooming camera;
and controlling the zoom camera to amplify and focus the ship body based on the coordinate information and the minimum external rectangle, and shooting to obtain the ship image.
Preferably, the computing device is configured to:
performing feature extraction on the ship image through a VGG network;
performing up-sampling decoding on the image;
calculating and outputting a region score and an association score, wherein the region score is the position of the segmented ship board region, and the association score is the position of the association between the characters in the ship board region;
obtaining the ship board area image by using the area score and the association score in combination with verification.
Preferably, the computing device is configured to:
inputting the ship plate area image into a positioning network, and performing regression fitting on the key point positions of characters in the ship plate area image;
and 2D conversion is carried out on the positions of the key points of the characters through the correction network.
Preferably, the computing device is configured to:
character recognition is performed using a sequence-to-sequence model, which consists of an encoder and a decoder.
Preferably, the encoder includes a convolution network and a bidirectional long-short term memory network, the feature extraction is performed on the ship plate area image through the convolution network, the bidirectional sequence relation of the features is analyzed through the bidirectional long-short term memory network, and a plurality of feature sequences with equal length are output.
Preferably, the decoder is an attention sequence-to-sequence model, and is configured to convert the feature sequence into a character sequence, and predict output characters through multi-step iterative computation to obtain a recognition result.
Preferably, the computing device is configured to:
the decoder calculates a weight vector of attention;
distributing different weight vectors to each feature in the feature sequence;
and performing the latest state of the current round based on the weighting characteristics, the intermediate state and the prediction result of the previous step, and outputting the final prediction result as a character recognition result after completing the iteration of preset times.
6. The computing device of claim 4 or 5, wherein the computing device is configured to, after obtaining the ship placard information:
acquiring AIS information of all ships within a preset sea area range;
and verifying the identified ship plate information according to the acquired AIS information.
7. A computer storage medium having at least one executable instruction stored therein, the executable instruction configured to perform the following operations:
capturing a ship image through a binocular camera;
carrying out region identification on the ship image to obtain a ship plate region image;
carrying out image correction on the ship board area image;
and performing character recognition on the corrected ship plate area image to obtain ship plate information.
8. The computer storage medium of claim 7, wherein the binocular camera is comprised of a wide angle camera and a zoom camera, the capturing of the ship image by the binocular camera comprises:
monitoring images of a navigation channel through the wide-angle camera, and identifying a target ship;
transmitting the coordinate information of the target ship and the minimum external rectangle of the ship body to the zooming camera;
and controlling the zoom camera to amplify and focus the ship body based on the coordinate information and the minimum external rectangle, and shooting to obtain the ship image.
Preferably, the performing region identification on the ship image to obtain a ship plate region image includes:
performing feature extraction on the ship image through a VGG network;
performing up-sampling decoding on the image;
calculating and outputting a region score and an association score, wherein the region score is the position of the segmented ship board region, and the association score is the position of the association between the characters in the ship board region;
obtaining the ship board area image by using the area score and the association score in combination with verification.
Preferably, the image rectification of the image of the ship board area includes:
inputting the ship plate area image into a positioning network, and performing regression fitting on the key point positions of characters in the ship plate area image;
and 2D conversion is carried out on the positions of the key points of the characters through the correction network.
Preferably, the character recognition of the corrected ship plate area image to obtain the ship plate information includes:
character recognition is performed using a sequence-to-sequence model, which consists of an encoder and a decoder.
Preferably, the encoder includes a convolution network and a bidirectional long-short term memory network, the feature extraction is performed on the ship plate area image through the convolution network, the bidirectional sequence relation of the features is analyzed through the bidirectional long-short term memory network, and a plurality of feature sequences with equal length are output.
Preferably, the decoder is an attention sequence-to-sequence model, and is configured to convert the feature sequence into a character sequence, and predict output characters through multi-step iterative computation to obtain a recognition result.
Preferably, the predicting the output character through the multi-step iterative computation to obtain the recognition result includes:
the decoder calculates a weight vector of attention;
distributing different weight vectors to each feature in the feature sequence;
and performing the latest state of the current round based on the weighting characteristics, the intermediate state and the prediction result of the previous step, and outputting the final prediction result as a character recognition result after completing the iteration of preset times.
9. The computer storage medium of claim 7 or 8, wherein after obtaining the ship placard information:
acquiring AIS information of all ships within a preset sea area range;
and verifying the identified ship plate information according to the acquired AIS information.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010874286.1A CN111985475A (en) | 2020-08-26 | 2020-08-26 | Ship board identification method, computing device and storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010874286.1A CN111985475A (en) | 2020-08-26 | 2020-08-26 | Ship board identification method, computing device and storage medium |
Publications (1)
Publication Number | Publication Date |
---|---|
CN111985475A true CN111985475A (en) | 2020-11-24 |
Family
ID=73441456
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010874286.1A Pending CN111985475A (en) | 2020-08-26 | 2020-08-26 | Ship board identification method, computing device and storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111985475A (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113239854A (en) * | 2021-05-27 | 2021-08-10 | 北京环境特性研究所 | Ship identity recognition method and system based on deep learning |
CN113486895A (en) * | 2021-07-16 | 2021-10-08 | 浙江华是科技股份有限公司 | Ship board identification method and device and computer readable storage medium |
CN114007050A (en) * | 2021-10-14 | 2022-02-01 | 桂林电子科技大学 | Target identification image transmission method based on Beidou communication |
CN114037909A (en) * | 2021-11-16 | 2022-02-11 | 中国电子科技集团公司第二十八研究所 | Automatic video labeling method and system for ship name identification characters |
CN117557785A (en) * | 2024-01-11 | 2024-02-13 | 宁波海上鲜信息技术股份有限公司 | Image processing-based long-distance fishing boat plate recognition method |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101145200A (en) * | 2007-10-26 | 2008-03-19 | 浙江工业大学 | Inner river ship automatic identification system of multiple vision sensor information fusion |
CN107886051A (en) * | 2017-10-19 | 2018-04-06 | 中国电子科技集团公司第二十八研究所 | Watercraft identification recognition methods based on image |
CN110287960A (en) * | 2019-07-02 | 2019-09-27 | 中国科学院信息工程研究所 | The detection recognition method of curve text in natural scene image |
CN111461116A (en) * | 2020-03-25 | 2020-07-28 | 深圳市云恩科技有限公司 | Ship board text recognition model, modeling method and training method thereof |
-
2020
- 2020-08-26 CN CN202010874286.1A patent/CN111985475A/en active Pending
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101145200A (en) * | 2007-10-26 | 2008-03-19 | 浙江工业大学 | Inner river ship automatic identification system of multiple vision sensor information fusion |
CN107886051A (en) * | 2017-10-19 | 2018-04-06 | 中国电子科技集团公司第二十八研究所 | Watercraft identification recognition methods based on image |
CN110287960A (en) * | 2019-07-02 | 2019-09-27 | 中国科学院信息工程研究所 | The detection recognition method of curve text in natural scene image |
CN111461116A (en) * | 2020-03-25 | 2020-07-28 | 深圳市云恩科技有限公司 | Ship board text recognition model, modeling method and training method thereof |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113239854A (en) * | 2021-05-27 | 2021-08-10 | 北京环境特性研究所 | Ship identity recognition method and system based on deep learning |
CN113239854B (en) * | 2021-05-27 | 2023-12-19 | 北京环境特性研究所 | Ship identity recognition method and system based on deep learning |
CN113486895A (en) * | 2021-07-16 | 2021-10-08 | 浙江华是科技股份有限公司 | Ship board identification method and device and computer readable storage medium |
CN113486895B (en) * | 2021-07-16 | 2022-04-19 | 浙江华是科技股份有限公司 | Ship board identification method and device and computer readable storage medium |
CN114007050A (en) * | 2021-10-14 | 2022-02-01 | 桂林电子科技大学 | Target identification image transmission method based on Beidou communication |
CN114037909A (en) * | 2021-11-16 | 2022-02-11 | 中国电子科技集团公司第二十八研究所 | Automatic video labeling method and system for ship name identification characters |
CN117557785A (en) * | 2024-01-11 | 2024-02-13 | 宁波海上鲜信息技术股份有限公司 | Image processing-based long-distance fishing boat plate recognition method |
CN117557785B (en) * | 2024-01-11 | 2024-04-02 | 宁波海上鲜信息技术股份有限公司 | Image processing-based long-distance fishing boat plate recognition method |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111985475A (en) | Ship board identification method, computing device and storage medium | |
Mahaur et al. | Small-object detection based on YOLOv5 in autonomous driving systems | |
US10672131B2 (en) | Control method, non-transitory computer-readable storage medium, and control apparatus | |
CN109145747B (en) | Semantic segmentation method for water surface panoramic image | |
CN114842028A (en) | Cross-video target tracking method, system, electronic equipment and storage medium | |
CN113614730B (en) | CNN classification of multi-frame semantic signals | |
CN111079739B (en) | Multi-scale attention feature detection method | |
CN109712071B (en) | Unmanned aerial vehicle image splicing and positioning method based on track constraint | |
CN110378837B (en) | Target detection method and device based on fish-eye camera and storage medium | |
CN111931764A (en) | Target detection method, target detection framework and related equipment | |
CN113807361B (en) | Neural network, target detection method, neural network training method and related products | |
KR102157610B1 (en) | System and method for automatically detecting structural damage by generating super resolution digital images | |
JP2023059794A (en) | Semantic graph embedding lifted for all azimuth direction location recognition | |
CN111738071B (en) | Inverse perspective transformation method based on motion change of monocular camera | |
CN110866475A (en) | Hand-off steering wheel and image segmentation model training method, device, terminal and medium | |
Dai et al. | GCD-YOLOv5: An armored target recognition algorithm in complex environments based on array lidar | |
Zhao et al. | Deep learning-based object detection in maritime unmanned aerial vehicle imagery: Review and experimental comparisons | |
CN114220087A (en) | License plate detection method, license plate detector and related equipment | |
CN112101113B (en) | Lightweight unmanned aerial vehicle image small target detection method | |
US20210174079A1 (en) | Method and apparatus for object recognition | |
CN115690770A (en) | License plate recognition method based on space attention characteristics in non-limited scene | |
CN115861922A (en) | Sparse smoke and fire detection method and device, computer equipment and storage medium | |
CN113436256B (en) | Shooting device state identification method, shooting device state identification device, computer equipment and storage medium | |
US20210012106A1 (en) | Image processing device, image processing method and object recognition system | |
CN114049377A (en) | Method and system for detecting high-dynamic small target in air |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |