CN117928567A - Ship auxiliary driving enhancing method - Google Patents
Ship auxiliary driving enhancing method Download PDFInfo
- Publication number
- CN117928567A CN117928567A CN202410324180.2A CN202410324180A CN117928567A CN 117928567 A CN117928567 A CN 117928567A CN 202410324180 A CN202410324180 A CN 202410324180A CN 117928567 A CN117928567 A CN 117928567A
- Authority
- CN
- China
- Prior art keywords
- ship
- digital twin
- information
- twin system
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 230000002708 enhancing effect Effects 0.000 title claims description 6
- 230000002452 interceptive effect Effects 0.000 claims description 18
- 230000005540 biological transmission Effects 0.000 claims description 15
- 238000012856 packing Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 7
- 230000001133 acceleration Effects 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000009434 installation Methods 0.000 claims description 6
- 239000005433 ionosphere Substances 0.000 claims description 4
- 230000003068 static effect Effects 0.000 claims description 4
- 239000005436 troposphere Substances 0.000 claims description 4
- 238000013461 design Methods 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- 239000000463 material Substances 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000012549 training Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 7
- 238000001514 detection method Methods 0.000 abstract description 2
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000012946 outsourcing Methods 0.000 description 3
- 230000008054 signal transmission Effects 0.000 description 2
- 241000098700 Sarcocheilichthys parvus Species 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Landscapes
- Traffic Control Systems (AREA)
Abstract
The invention discloses a ship auxiliary driving enhancement method, which is used for improving a ship auxiliary enhancement system, combining a high-precision Global Positioning System (GPS) and a real-time dynamic positioning technology (RTK), obtaining additionally arranged camera and laser radar data, constructing a virtual model of a ship, a sea surface and other physical entities or processes by using a data twin technology through a digital technology, and realizing accurate detection and prediction of a ship running state at sea by continuously updating the obtained real-time positioning data, the ship Euler angle and the pose data, so that the collision of the ship can be better avoided, the berthing safety is ensured, and meanwhile, the ship navigation path planning and the automatic navigation are provided.
Description
Technical Field
The invention belongs to the technical field of ship navigation control, and particularly relates to a ship auxiliary steering enhancement system method.
Background
At present, a great risk exists in marine vessel navigation, namely, the problem of collision between vessels, or between vessels and offshore platforms or sea foreign bodies. Due to the fact that sea surface airlines are numerous, a submerged reef, other ships or platforms can be possibly blocked on the airlines, but visual fields for observation and control in a captain cab are limited, and the captain cannot effectively acquire collision early warning and accurate distances between the ships and other objects, so that collision accidents occur.
In recent years, digital twinning has been widely studied and applied as a simulation process combining multiple disciplines, multiple physical quantities and multiple dimensions. The primary task of digital twinning applications is to create a digital twinning model of the application object. In the marine industry, digital twins can be used to design, simulate, optimize and manage the various stages and links of a ship.
The digital twin system for ship aided driving requires real-time mapping of physical information of real space in virtual space, and in order to restore the real scene to the maximum extent, the data measurement accuracy of complex offshore environment and the state of the ship itself must be improved. However, the current information sensing technology in ship driving mainly uses sensing equipment, a sensing network and information processing equipment to intelligently sense and acquire the state and surrounding environment information of the ship, so that the interactive acquisition of ship-shore information is realized, the acquired data volume is large, the variety of signals is various, and the information precision is not enough, so that further improvement and perfection are required.
Disclosure of Invention
The invention aims at the problems in the prior art and provides a ship auxiliary driving enhancing method.
The invention provides a ship auxiliary driving enhancement method, which comprises the following steps:
Step S100: 3D modeling is carried out on the ship by utilizing a CAD drawing of the ship, and a digital twin system is imported;
Step S200: installing a GPS receiver and an RTK receiver on the ship, receiving signals sent by a base station, calculating accurate position coordinates, and sending the position coordinates to a digital twin system;
step S300: acquiring real-time Euler angle and pose information of a ship through a compass, and adjusting the course and the pose of the ship in a digital twin system;
Step S400: installing a laser radar and a camera on the compass deck of the ship, acquiring point clouds and image data of other ships or offshore platforms acquired by the laser radar and the camera, converting a sensor coordinate system of the point clouds and the image data into a world coordinate system, and modeling in a digital twin system;
Step S500: detecting other moving targets based on point clouds and image data acquired by the laser radar and the camera during long voyage of the ship, and modeling in a data twin system; detecting a static target based on the point cloud and the image data when the ship is berthed, and modeling in a digital twin system; establishing an axial bounding box for a ship and an object which possibly collides in a digital twin system, calculating the nearest distance between the bounding box where the ship is located and other bounding boxes, and sending out report early warning and recording all data of current navigation when the distance is smaller than a safe distance;
Step S600: the method comprises the steps of obtaining marine hydrologic information of a current position of a ship, adding the marine hydrologic information into a digital twin system, and planning a path for next walking of the ship by the digital twin system according to the current ship pose information, the navigation speed, the acceleration and the position information of interference objects around the ship and combining the obtained marine hydrologic information.
Preferably, modeling the ship shape according to the CAD drawing of the ship design in the step S100 includes:
step S110: acquiring a CAD drawing of a ship, converting an AutoCAD graph, introducing the CAD graph into SolidWorks, acquiring a front view, a top view and a left view of the CAD drawing, and defining an auxiliary view; aligning the views, generating a 3D model using SolidWork;
Step S120: a model file in the format of. Obj or. Stl is derived using SolidWork, which contains vertex-related data for aggregate objects, the model file is loaded using WebGL, the material objects are custom-defined using the three. Js engine, and a three-dimensional model of the marine vessel is presented on a web page.
Preferably, the step S200 further includes:
step S210: the GPS uses the three-star positioning principle to determine the coordinate position of a ground object, and calculates the sea ship coordinate through the position of three satellites and the time for signals to reach the satellites;
；
Wherein x, y and z are the coordinate information of three satellites respectively for the ground object coordinates ,(x1,y1,z1),(x2,y2,z2),(x3,y3,z3) to be solved, T 1,T2,T3 is the time of the GPS signal reaching the three satellites respectively, and v is the signal propagation speed;
Step S220: the GPS uses the three-star positioning principle to determine the coordinate position of the ground object, adds a fourth satellite and sets a standard time difference because of clock errors in the initial time of the three satellites, and establishes 4 equation sets to calculate sea surface ship coordinates so as to reduce the clock errors assuming that the standard time difference of the ground object is t;
；
Wherein x, y and z are the coordinate information of four satellites respectively for the ground object coordinates ,(x1,y1,z1),(x2,y2,z2),(x3,y3,z3),(x4,y4,z4) to be solved, T 1,T2,T3, T4 is the time of the GPS signal reaching the four satellites respectively, deltat 1,Δt2,Δt3,Δt4 is the standard time difference of the four satellites respectively, and v is the signal propagation speed;
Step S230: the method comprises the steps that transmission errors exist in an ionosphere and a troposphere of signals, the transmission errors between satellites and signal towers are calculated based on RTK signal towers with determined positions, and the errors between the satellites and a ship are corrected according to the transmission errors, so that accurate coordinate information of the ship is obtained;
Step S240: in the digital twin system, a coordinate system is established by taking the sea level as the xoy plane and taking the axis perpendicular to the xoy plane as the z axis, and ship coordinates are marked as accurate coordinate information measured by using GPS and RTK.
Preferably, the step S300 further includes:
Step S310: determining the course of the ship and observing the target object azimuth according to the compass, acquiring three-axis angular velocity data based on the compass sensor, and calculating the Euler angle of the ship;
step S320: transmitting the Euler angle data to a digital twin system;
Step S330: and (3) determining the rotation center of the ship by combining the ship coordinate information and modeling of the digital twin system in the step (S240), calculating a rotation matrix of the current pose of the ship relative to the previous pose by utilizing the Euler angle obtained in the step (S320), and carrying out rotation change on the ship.
Preferably, the step S400 further includes:
Step S410: simultaneously installing a plurality of cameras and laser radars on a compass deck of the ship, and recording the installation position information of the cameras and the laser radars;
Step S420: the image information acquired by the camera is aligned and fused with the point cloud information acquired by the laser radar, and the point cloud is colored by using corresponding pixel points;
Step S430: according to the position information of the installation of the camera and the laser radar, the ship coordinates and the acquired point cloud coordinate information, carrying out coordinate conversion on the point cloud coordinate information to a world coordinate system;
step S440: and adding the point cloud to the corresponding position of the digital twin system according to the relative position information of the point cloud and the ship.
Preferably, step S500 further includes:
Step S510: detecting other moving targets by using inter-frame difference operation according to point cloud and image information acquired by a laser radar and a camera lens during long voyage of the ship;
Step S520, when the ship is berthed, performing model training on the image and the point cloud subjected to coordinate conversion by using a mmdetetion d frame, and detecting an interference object in the data;
Step S530: reasoning is carried out according to the trained model to obtain coordinate information of an interfering object, and the coordinate information of the interfering object is transmitted to a digital twin system;
Step S540: and drawing out an axial outer packing box of the ship and the interfering objects, calculating the nearest distance between the outer packing box of the ship and other outer packing boxes, and sending out report early warning and recording all data of the current sailing when the distance is smaller than the safe distance.
Preferably, step S510 further includes:
Step S511, performing graying processing on the image, and obtaining a difference image by taking the absolute value of the difference value of the gray values of the corresponding pixels of two adjacent frames of images, wherein (X, Y) is the coordinates of the pixels in the image, the gray value of the coordinates of the pixels in the previous frame of two adjacent frames of images is p n (X, Y), the gray value of the pixels in the next frame of images is p n+1 (X, Y), and the gray value of the pixels in the difference image is p d (X, Y), and the following formula is given:
；
Step S512: performing differential operation on pixel points in the image to obtain a gray value difference value of each pixel point, obtaining a complete differential image, performing threshold processing on the differential image, setting a gray threshold G, and setting the pixel value of each pixel point to 255 when the gray value in the differential image is larger than G, wherein the motion target point is regarded as a foreground; when the gray value in the differential image is smaller than G, the pixel value of the pixel point is set to 0, the background point is regarded as the background point, and the image after threshold processing is called as the target image The pixel value of each position point in the target image is p t (X, Y), where the pixel value of 255 is the detected target moving object:
；
step S513: and according to the position of the target moving object in the target image, finding the corresponding target object position in the point cloud in the image and point cloud fusion information, recording the coordinate information of the target object in the point cloud, and simultaneously labeling in a digital twin system.
Preferably, step S600 further includes:
Step S610, accessing a digital twin system into a network to obtain a real-time marine hydrologic system, inquiring and storing marine hydrologic information near the current ship and information of surrounding ships in a digital twin system background, and when the background detects that the marine climate is remarkably changed, carrying out early warning in a digital twin system foreground and displaying the climate state on an interface in real time; when the background detects that other ships exist in the safe distance, the digital twin system foreground is subjected to early warning and display;
Step S620: when the ship is at a distance and no obvious obstacle exists nearby the ship, calculating the optimal N routes according to the target moving object information obtained by differential operation in the step S510 and the marine hydrologic system information inquired by the digital twin system background and combining the current course and the destination of the ship, and drawing the optimal N routes in the digital twin system as navigation advice;
step S630: when the ship is berthed, judging the heading of the next step of the ship according to the current ship pose, running speed and acceleration in the digital twin system;
Step S640: according to the current position of the ship and the position of the interfering object, calculating M routes for safe navigation of the current ship course;
Step S650: calculating the nearest distance between each route and each outer box according to the outer boxes of the ship and the interfering object, and accumulating the distances;
step S660: deleting the current planning route if the accumulated distance is smaller than the safety distance in the path planning;
Step S670: and selecting the route with the largest accumulated distance, and marking by scribing in the digital twin system for suggesting the next driving route.
Compared with the prior art, the invention has the following beneficial effects:
1) According to the method, a high-precision Global Positioning System (GPS) and a real-time dynamic positioning technology (RTK) are combined on a ship auxiliary enhancement system, navigation instrument data and additionally installed camera and laser radar data on a ship are combined, a data twin technology is utilized to construct a virtual model of a ship sea surface and other physical entities or processes, real-time positioning data, ship Euler angles and pose data are obtained through continuous updating, and accurate detection and prediction of the running state of the ship on the sea are realized. Meanwhile, a digital model is built for other sea ships, sea platforms and the like shot by a laser radar and a camera, the digital model is updated to a digital twin system, corresponding distances are marked in the digital twin system, when the distances between the objects and the current ship are smaller than the set safe distance, the system gives out collision early warning, and the collision early warning is displayed in red flash on an interface and sounds to alert to cause the captain warning.
2) The method may also provide vessel navigation path planning and automated navigation advice to avoid potential collision risks. The system can also save abnormal data and alarm data existing in the ship navigation process so as to reproduce the running state of the ship and the driving condition of the captain in actual running.
Drawings
FIG. 1 is a flow chart of a method for assisting in steering of a vessel.
Detailed Description
The techniques described below are susceptible to various modifications and alternative embodiments, and are described in detail herein with reference to the accompanying drawings. However, this is not meant to limit the techniques described below to particular embodiments. It should be understood that the invention includes all similar modifications, equivalents and alternatives falling within the spirit and scope of the techniques described below.
Example 1
As shown in fig. 1, the present invention provides a method for enhancing driving assistance of a ship, comprising:
step S100: and 3D modeling is carried out on the ship by utilizing a CAD drawing of the ship, and a digital twin system is imported.
Preferably, the step S100 further includes:
Step S110: acquiring a CAD drawing of a ship, converting an AutoCAD graph, introducing the CAD graph into SolidWorks, acquiring a front view, a top view and a left view of the CAD drawing, and defining an auxiliary view; the views are aligned and a 3D model is generated using SolidWork.
Step S120: a model file in the format of. Obj or. Stl is derived using SolidWork, which contains vertex-related data for aggregate objects, the model file is loaded using WebGL, the material objects are custom-defined using the three. Js engine, and a three-dimensional model of the marine vessel is presented on a web page. Specifically, the vertex-related data includes a fixed point position, a fixed point normal vector.
Step S200: and installing a GPS receiver and an RTK receiver on the ship, receiving signals transmitted by a base station, calculating accurate position coordinates, and transmitting the position coordinates to a digital twin system.
Preferably, the step S200 further includes:
step S210: the GPS uses the three-star positioning principle to determine the coordinate position of a ground object, and calculates the sea ship coordinate through the position of three satellites and the time for signals to reach the satellites;
；
Wherein x, y and z are the coordinate information of three satellites respectively for the ground object coordinates ,(x1,y1,z1),(x2,y2,z2),(x3,y3,z3) to be solved, T 1,T2,T3 is the time of the GPS signal reaching the three satellites respectively, and v is the signal propagation speed.
Step S220: the GPS uses the three-star positioning principle to determine the coordinate position of the ground object, adds a fourth satellite and sets a standard time difference because of clock errors in the initial time of the three satellites, and establishes 4 equation sets to calculate sea surface ship coordinates so as to reduce the clock errors assuming that the standard time difference of the ground object is t;
；
wherein x, y and z are the coordinate information of four satellites respectively for the ground object coordinates ,(x1,y1,z1),(x2,y2,z2),(x3,y3,z3),(x4,y4,z4) to be solved, T 1,T2,T3, T4 is the time of arrival of the GPS signal at the four satellites respectively, deltat 1,Δt2,Δt3,Δt4 is the standard time difference of the four satellites, and v is the signal propagation speed.
Step S230: the signals have transmission errors in the ionosphere and the troposphere, the transmission errors between the satellites and the signal towers are calculated based on the RTK signal towers with the determined positions, and the errors between the satellites and the ship are corrected according to the transmission errors, so that accurate coordinate information of the ship is obtained. Specifically, in the actual transmission process of the signals, transmission errors exist in signal transmission due to the existence of an ionosphere, a troposphere and the like, and the transmission errors between the satellites and the ship are corrected through the transmission errors between the satellites and the signal towers, so that more accurate coordinate information of the ship is further obtained.
Step S240: in the digital twin system, a coordinate system is established by taking the sea level as the xoy plane and taking the axis perpendicular to the xoy plane as the z axis, and ship coordinates are marked as accurate coordinate information measured by using GPS and RTK.
In GPS positioning, on one hand, clock errors caused by non-uniform initial time of three satellites are considered, on the other hand, signal transmission errors are also considered, and through setting standard time differences and using transmission error correction signals to transmit between the satellites and a signal tower, the obtained ship coordinates are more accurate, the modeling of a twin system is more accurate, and the precision and accuracy of subsequent early warning and path planning are improved.
Step S300: and acquiring real-time Euler angle and pose information of the ship through a compass, and adjusting the course and the pose of the ship in a digital twin system.
Preferably, the step S300 further includes:
Step S310: and determining the course of the ship and observing the target object azimuth according to the compass, acquiring the three-axis angular velocity data based on the compass sensor, and calculating the Euler angle of the ship. The compass is an instrument for providing a direction reference and can be used for determining the heading of the ship and observing the target azimuth.
Step S320: and sending the Euler angle data to a digital twin system. Specifically, a communication program of the compass and the digital twin system is compiled, and data communication between Euler angle data and the twin system is completed through the communication program.
Step S330: and (3) determining the rotation center of the ship by combining the ship coordinate information and modeling of the digital twin system in the step (S240), calculating a rotation matrix of the current pose of the ship relative to the previous pose by utilizing the Euler angle obtained in the step (S320), and carrying out rotation change on the ship.
Step S400: and installing a laser radar and a camera on the compass deck of the ship, acquiring point clouds and image data of other ships or offshore platforms acquired by the laser radar and the camera, converting a sensor coordinate system of the point clouds and the image data into a world coordinate system, and modeling in a digital twin system.
Preferably, step S400 further comprises:
Step S410: and simultaneously installing a plurality of cameras and laser radars on the compass deck of the ship, and recording the installation position information of the cameras and the laser radars.
Step S420: and (3) aligning and fusing the image information acquired by the camera with the point cloud information acquired by the laser radar, and performing color adding on the point cloud by using the corresponding pixel points.
Step S430: and according to the position information of the installation of the camera and the laser radar, the ship coordinates and the acquired point cloud coordinate information, carrying out coordinate conversion on the point cloud coordinate information to a world coordinate system.
Step S440: and adding the point cloud to the corresponding position of the digital twin system according to the relative position information of the point cloud and the ship.
Step S500: detecting other moving targets based on point clouds and image data acquired by the laser radar and the camera during long voyage of the ship, and modeling in a data twin system; detecting a static target based on the point cloud and the image data when the ship is berthed, and modeling in a digital twin system; and establishing an axial bounding box for the ship and an object which possibly collides in the digital twin system, calculating the nearest distance between the bounding box where the ship is located and other bounding boxes, and sending out report early warning and recording all data of current navigation when the distance is smaller than the safe distance. In particular, the other moving objects comprise other vessels and the stationary objects comprise offshore platform FPSOs, shore side other vessels.
Preferably, step S500 further includes:
Step S510: and detecting other moving targets by using inter-frame difference operation according to the point cloud and the image information acquired by the laser radar and the camera lens when the ship is at a long distance.
Preferably, step S510 further includes:
Step S511, performing graying processing on the image, and obtaining a difference image by taking the absolute value of the difference value of the gray values of the corresponding pixels of two adjacent frames of images, wherein (X, Y) is the coordinates of the pixels in the image, the gray value of the coordinates of the pixels in the previous frame of two adjacent frames of images is p n (X, Y), the gray value of the pixels in the next frame of images is p n+1 (X, Y), and the gray value of the positions in the difference image is p d (X, Y), and the following formula is adopted:
；
Step S512: performing differential operation on pixel points in the image to obtain a gray value difference value of each pixel point, obtaining a complete differential image, performing threshold processing on the differential image, setting a gray threshold G, and setting the pixel value of each pixel point to 255 when the gray value in the differential image is larger than G, wherein the motion target point is regarded as a foreground; when the gray value in the differential image is smaller than G, the pixel value of the pixel point is set to 0, and the image after the thresholding is called a target image p t, the pixel value of each position point in the target image is p t (X, Y) as shown in the following formula, wherein the pixel value of 255 is the detected target moving object:
；
step S513: and according to the position of the target moving object in the target image, finding the corresponding target object position in the point cloud in the image and point cloud fusion information, recording the coordinate information of the target object in the point cloud, and simultaneously labeling in a digital twin system.
Step S520, when the ship is berthed, performing model training on the image and the point cloud subjected to coordinate conversion by using a mmdetetion d frame, and detecting an interference object in the data; specifically, the interfering objects comprise other ships on the shore, offshore platforms and submerged reefs;
Step S530: reasoning is carried out according to the trained model to obtain coordinate information of an interfering object, and the coordinate information of the interfering object is transmitted to a digital twin system; specifically, tcp is used to realize the transmission of coordinate information to a digital twin system;
step S540: and determining an axial outer packing box of the ship and the interfering objects, calculating the nearest distance between the outer packing box of the ship and other outer packing boxes, and sending out report early warning and recording all data of current sailing when the distance is smaller than the safe distance.
Step S600: the method comprises the steps of obtaining marine hydrologic information of a current position of a ship, adding the marine hydrologic information into a digital twin system, and planning a path for next walking of the ship by the digital twin system according to the current ship pose information, the navigation speed, the acceleration and the position information of interference objects around the ship and combining the obtained marine hydrologic information. Specifically, the marine hydrologic information includes a storm class, ocean currents, and the like.
On the basis of accurately positioning the ship, the method also considers the influence of the marine hydrologic information of the ship position on the ship, and alarms and reminds when the marine hydrologic information changes greatly, so that accurate planning is performed for the running of the subsequent ship, and the safety is improved.
Preferably, step S600 further includes:
Step S610, accessing a digital twin system into a network to obtain a real-time marine hydrologic system, inquiring and storing marine hydrologic information near the current ship and information of surrounding ships in a digital twin system background, and when the background detects that the marine climate is remarkably changed, carrying out early warning in a digital twin system foreground and displaying the climate state on an interface in real time; when the background detects that other ships exist in the safe distance, the digital twin system foreground is pre-warned and displayed. Specifically, significant changes in marine climate include storms while the background detects and warns of other dangerous situations that may occur, such as changes in the ocean water level near the periphery of the ship, etc.
Step S620: when the ship is at a distance and no obvious obstacle exists nearby the ship, according to the target moving object information obtained by differential operation in the step S510 and the marine hydrologic system information queried by the digital twin system background, the current course and the destination of the ship are combined, the movable route of the ship is calculated to obtain the optimal N routes, and the optimal N routes are drawn in the digital twin system to serve as navigation suggestions.
Step S630: when the ship is berthed, the heading of the next step of the ship is judged according to the current ship pose, running speed and acceleration in the digital twin system.
Step S640: and calculating M routes for safe navigation of the current ship course according to the current position of the ship and the position of the interfering object.
Step S650: and calculating the nearest distance between each route and each outer box according to the outer boxes of the ship and the interfering object, and accumulating the distances.
Further, the accumulated distance is a reference distance. In actual calculation, each outsourcing box is given a weight, and the reference distance is obtained by multiplying the nearest distance of the corresponding outsourcing box by the weight of the nearest distance.
Specifically, the weight of each outsourcing box is affected by two factors. First, the distance of the outer packing box from the berthing position of the ship. The distance between the outer packing box and the ship berthing position is in inverse proportion to the weight. And secondly, collision radius. The position of any outer box is taken as the center, the circular area covered by the collision radius is taken as the reference area, and the more the number of other outer boxes contained in the area is, the larger the weight is. The collision radius is dynamically determined according to the volume, pose and current hydrologic conditions of the ship.
Step S660: and deleting the current planned route if the accumulated distance is smaller than the safety distance in the path planning.
Step S670: and selecting the route with the largest accumulated distance, and marking in a digital twin system for suggesting the next driving route.
In summary, the invention obtains more accurate real-time ship coordinates by adopting a high-precision Global Positioning System (GPS) and a real-time dynamic positioning technology (RTK), models the real-time accurate ship coordinates in a digital twin system by combining camera images and point cloud information of a laser radar, obtains the distance relation between a ship and a moving target or a static target in the surrounding environment, and judges the safety of a route based on the accumulated distance between the ship and an interfering object; meanwhile, the method combines marine hydrologic information of the ship at the position, performs ship risk early warning and route planning, and avoids potential collision risks.
While the invention has been described in detail in connection with the general description and the specific embodiments thereof, modifications and improvements may be made thereto. The above description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the invention, but other variations and modifications can be made by those skilled in the art without departing from the spirit and scope of the invention.
Claims (10)
1. The auxiliary driving enhancing method for the ship is characterized by comprising the following steps of:
Step S100: 3D modeling is carried out on the ship by utilizing a CAD drawing of the ship, and a digital twin system is imported;
Step S200: installing a GPS receiver and an RTK receiver on the ship, receiving signals sent by a base station, calculating accurate position coordinates, and sending the position coordinates to a digital twin system;
step S300: acquiring real-time Euler angle and pose information of a ship through a compass, and adjusting the course and the pose of the ship in a digital twin system;
Step S400: installing a laser radar and a camera on the compass deck of the ship, acquiring point clouds and image data of other ships or offshore platforms acquired by the laser radar and the camera, converting a sensor coordinate system of the point clouds and the image data into a world coordinate system, and modeling in a digital twin system;
Step S500: detecting other moving targets based on point clouds and image data acquired by the laser radar and the camera during long voyage of the ship, and modeling in a data twin system; detecting a static target based on the point cloud and the image data when the ship is berthed, and modeling in a digital twin system; establishing an axial bounding box for a ship and an object which possibly collides in a digital twin system, calculating the nearest distance between the bounding box where the ship is located and other bounding boxes, and sending out report early warning and recording all data of current navigation when the distance is smaller than a safe distance;
Step S600: the method comprises the steps of obtaining marine hydrologic information of a current position of a ship, adding the marine hydrologic information into a digital twin system, and planning a path for next walking of the ship by the digital twin system according to the current ship pose information, the navigation speed, the acceleration and the position information of interference objects around the ship and combining the obtained marine hydrologic information.
2. The method for assisting pilot enhancement of a ship according to claim 1, wherein modeling the ship shape according to the CAD drawing of the ship design in step S100 comprises:
Step S110: acquiring a CAD drawing of a ship, converting an AutoCAD graph, introducing the CAD graph into SolidWorks, acquiring a front view, a top view and a left view of the CAD drawing, and defining an auxiliary view; aligning the views, generating a 3D model using SolidWork;
Step S120: a model file in the format of. Obj or. Stl is derived using SolidWork, which contains vertex-related data for aggregate objects, the model file is loaded using WebGL, the material objects are custom-defined using the three. Js engine, and a three-dimensional model of the marine vessel is presented on a web page.
3. The method for assisting the driving of the ship according to claim 1, wherein the step S200 of installing the GPS receiver and the RTK receiver on the ship, receiving the signals transmitted from the building station, calculating the accurate position coordinates and transmitting the accurate position coordinates to the digital twin system comprises:
step S210: the GPS uses the three-star positioning principle to determine the coordinate position of a ground object, and calculates the sea ship coordinate through the position of three satellites and the time for signals to reach the satellites;
；
Wherein x, y and z are the coordinate information of three satellites respectively for the ground object coordinates ,(x1,y1,z1),(x2,y2,z2),(x3,y3,z3) to be solved, T 1,T2,T3 is the time of the GPS signal reaching the three satellites respectively, and v is the signal propagation speed.
4. A method for enhancing the assisted steering of a ship according to claim 3, wherein the step S200 of installing a GPS receiver and an RTK receiver on the ship, receiving the signals transmitted from the building station, calculating the accurate position coordinates and transmitting the accurate position coordinates to the digital twin system comprises:
Step S220: the GPS uses the three-star positioning principle to determine the coordinate position of the ground object, adds a fourth satellite and sets a standard time difference because of clock errors in the initial time of the three satellites, and establishes 4 equation sets to calculate sea surface ship coordinates so as to reduce the clock errors assuming that the standard time difference of the ground object is t;
；
Wherein x, y and z are the coordinate information of four satellites respectively for the ground object coordinates ,(x1,y1,z1),(x2,y2,z2),(x3,y3,z3),(x4,y4,z4) to be solved, T 1,T2,T3, T4 is the time of arrival of the GPS signals at the four satellites respectively, deltat 1,Δt2,Δt3,Δt4 is the standard time difference of the four satellites respectively, and v is the signal propagation speed.
5. The method for assisting in driving a ship according to claim 4, wherein the step S200 of installing a GPS receiver and an RTK receiver on the ship, receiving the signals transmitted from the building station, calculating the accurate position coordinates, and transmitting the accurate position coordinates to the digital twin system further comprises:
Step S230: the method comprises the steps that transmission errors exist in an ionosphere and a troposphere of signals, the transmission errors between satellites and signal towers are calculated based on RTK signal towers with determined positions, and the errors between the satellites and a ship are corrected according to the transmission errors, so that accurate coordinate information of the ship is obtained;
Step S240: in the digital twin system, a coordinate system is established by taking the sea level as the xoy plane and taking the axis perpendicular to the xoy plane as the z axis, and ship coordinates are marked as accurate coordinate information measured by using GPS and RTK.
6. The method for assisting pilot enhancement of a vessel according to claim 5, wherein said step S300: acquiring real-time Euler angle and pose information of a ship through a compass, and adjusting the ship heading and pose in a digital twin system comprises the following steps:
Step S310: determining the course of the ship and observing the target object azimuth according to the compass, acquiring three-axis angular velocity data based on the compass sensor, and calculating the Euler angle of the ship;
step S320: transmitting the Euler angle data to a digital twin system;
Step S330: and (3) determining the rotation center of the ship by combining the ship coordinate information and modeling of the digital twin system in the step (S240), calculating a rotation matrix of the current pose of the ship relative to the previous pose by utilizing the Euler angle obtained in the step (S320), and carrying out rotation change on the ship.
7. The method for assisting pilot enhancement of a ship according to claim 1, wherein the step S400 further comprises:
Step S410: simultaneously installing a plurality of cameras and laser radars on a compass deck of the ship, and recording the installation position information of the cameras and the laser radars;
Step S420: the image information acquired by the camera is aligned and fused with the point cloud information acquired by the laser radar, and the point cloud is colored by using corresponding pixel points;
Step S430: according to the position information of the installation of the camera and the laser radar, the ship coordinates and the acquired point cloud coordinate information, carrying out coordinate conversion on the point cloud coordinate information to a world coordinate system;
step S440: and adding the point cloud to the corresponding position of the digital twin system according to the relative position information of the point cloud and the ship.
8. The method for assisting pilot enhancement of a vessel according to claim 7, wherein the step S500 further comprises:
Step S510: detecting other moving targets by using inter-frame difference operation according to point cloud and image information acquired by a laser radar and a camera lens during long voyage of the ship;
Step S520, when the ship is berthed, performing model training on the image and the point cloud subjected to coordinate conversion by using a mmdetetion d frame, and detecting an interference object in the data;
Step S530: reasoning is carried out according to the trained model to obtain coordinate information of an interfering object, and the coordinate information of the interfering object is transmitted to a digital twin system;
step S540: and determining an axial outer packing box of the ship and the interfering objects, calculating the nearest distance between the outer packing box of the ship and other outer packing boxes, and sending out report early warning and recording all data of current sailing when the distance is smaller than the safe distance.
9. The method for assisting ship driving according to claim 8, wherein the inter-frame difference operation in step S510 comprises:
Step S511, performing graying processing on the image, and obtaining a difference image by taking the absolute value of the difference value of the gray values of the corresponding pixels of two adjacent frames of images, wherein (X, Y) is the coordinates of the pixels in the image, the gray value of the coordinates of the pixels in the previous frame of two adjacent frames of images is p n (X, Y), the gray value of the pixels in the next frame of images is p n+1 (X, Y), and the gray value of the pixels in the difference image is p d (X, Y), and the following formula is given:
；
Step S512: performing differential operation on pixel points in the image to obtain a gray value difference value of each pixel point, obtaining a complete differential image, performing threshold processing on the differential image, setting a gray threshold G, and setting the pixel value of each pixel point to 255 when the gray value in the differential image is larger than G, wherein the motion target point is regarded as a foreground; when the gray value in the differential image is smaller than G, the pixel value of the pixel point is set to 0, and the image after the thresholding is called a target image p t, the pixel value of each position point in the target image is p t (X, Y) as shown in the following formula, wherein the pixel value of 255 is the detected target moving object:
；
step S513: and according to the position of the target moving object in the target image, finding the corresponding target object position in the point cloud in the image and point cloud fusion information, recording the coordinate information of the target object in the point cloud, and simultaneously labeling in a digital twin system.
10. The method for assisting pilot enhancement of a vessel according to claim 9, wherein the step S600 further comprises:
Step S610, accessing a digital twin system into a network to obtain a real-time marine hydrologic system, inquiring and storing marine hydrologic information near the current ship and information of surrounding ships in a digital twin system background, and when the background detects that the marine climate is remarkably changed, carrying out early warning in a digital twin system foreground and displaying the climate state on an interface in real time; when the background detects that other ships exist in the safe distance, the digital twin system foreground is subjected to early warning and display;
Step S620: when the ship is at a distance and no obvious obstacle exists nearby the ship, calculating the optimal N routes according to the target moving object information obtained by differential operation in the step S510 and the marine hydrologic system information inquired by the digital twin system background and combining the current course and the destination of the ship, and drawing the optimal N routes in the digital twin system as navigation advice;
step S630: when the ship is berthed, judging the heading of the next step of the ship according to the current ship pose, running speed and acceleration in the digital twin system;
Step S640: according to the current position of the ship and the position of the interfering object, calculating M routes for safe navigation of the current ship course;
Step S650: calculating the nearest distance between each route and each outer box according to the outer boxes of the ship and the interfering object, and accumulating the distances;
step S660: deleting the current planning route if the accumulated distance is smaller than the safety distance in the path planning;
Step S670: and selecting the route with the largest accumulated distance, and marking by scribing in the digital twin system for suggesting the next driving route.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410324180.2A CN117928567B (en) | 2024-03-21 | 2024-03-21 | Ship auxiliary driving enhancing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410324180.2A CN117928567B (en) | 2024-03-21 | 2024-03-21 | Ship auxiliary driving enhancing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117928567A true CN117928567A (en) | 2024-04-26 |
| CN117928567B CN117928567B (en) | 2024-06-04 |
Family
ID=90766801
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410324180.2A Active CN117928567B (en) | 2024-03-21 | 2024-03-21 | Ship auxiliary driving enhancing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117928567B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021152205A1 (en) * | 2020-01-28 | 2021-08-05 | Awake.Ai Oy | System for guiding vessel to port and method therefor |
| KR20210134132A (en) * | 2020-04-29 | 2021-11-09 | 한국해양대학교 산학협력단 | Constructing method of digital twin ship |
| CN114217616A (en) * | 2021-12-14 | 2022-03-22 | 武汉理工大学 | Ship-shore cooperative inland river ship remote control method and system |
| US20230077570A1 (en) * | 2021-09-16 | 2023-03-16 | International Business Machines Corporation | Digital twin simulation for transportation |
| CN116343551A (en) * | 2023-03-29 | 2023-06-27 | 武汉理工大学 | Intelligent ship control simulation system and method |
| CN116630907A (en) * | 2023-06-01 | 2023-08-22 | 数字太空(北京)科技股份公司 | Ship navigation digital twin method, system, terminal and storage medium based on remote sensing |
| CN116824912A (en) * | 2023-06-08 | 2023-09-29 | 华设设计集团股份有限公司 | Digital twin inland river ship navigation management and control system |
| CN117073676A (en) * | 2023-07-06 | 2023-11-17 | 中海电信有限公司 | Berthing auxiliary system based on combination of difference Beidou information and vision |
| CN117470249A (en) * | 2023-12-27 | 2024-01-30 | 湖南睿图智能科技有限公司 | Ship anti-collision method and system based on laser point cloud and video image fusion perception |
| CN117664120A (en) * | 2022-08-29 | 2024-03-08 | 厦门和丰互动科技有限公司 | Unmanned ship collision prevention method based on digital twin system |
| CN117688349A (en) * | 2023-12-08 | 2024-03-12 | 南通思诺船舶科技有限公司 | Maritime search and rescue method and system based on digital twinning |
-
2024
- 2024-03-21 CN CN202410324180.2A patent/CN117928567B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021152205A1 (en) * | 2020-01-28 | 2021-08-05 | Awake.Ai Oy | System for guiding vessel to port and method therefor |
| KR20210134132A (en) * | 2020-04-29 | 2021-11-09 | 한국해양대학교 산학협력단 | Constructing method of digital twin ship |
| US20230077570A1 (en) * | 2021-09-16 | 2023-03-16 | International Business Machines Corporation | Digital twin simulation for transportation |
| CN114217616A (en) * | 2021-12-14 | 2022-03-22 | 武汉理工大学 | Ship-shore cooperative inland river ship remote control method and system |
| CN117664120A (en) * | 2022-08-29 | 2024-03-08 | 厦门和丰互动科技有限公司 | Unmanned ship collision prevention method based on digital twin system |
| CN116343551A (en) * | 2023-03-29 | 2023-06-27 | 武汉理工大学 | Intelligent ship control simulation system and method |
| CN116630907A (en) * | 2023-06-01 | 2023-08-22 | 数字太空(北京)科技股份公司 | Ship navigation digital twin method, system, terminal and storage medium based on remote sensing |
| CN116824912A (en) * | 2023-06-08 | 2023-09-29 | 华设设计集团股份有限公司 | Digital twin inland river ship navigation management and control system |
| CN117073676A (en) * | 2023-07-06 | 2023-11-17 | 中海电信有限公司 | Berthing auxiliary system based on combination of difference Beidou information and vision |
| CN117688349A (en) * | 2023-12-08 | 2024-03-12 | 南通思诺船舶科技有限公司 | Maritime search and rescue method and system based on digital twinning |
| CN117470249A (en) * | 2023-12-27 | 2024-01-30 | 湖南睿图智能科技有限公司 | Ship anti-collision method and system based on laser point cloud and video image fusion perception |
Non-Patent Citations (3)
| Title |
|---|
| SUHYEON HEO ET AL: "A Preliminary Study for Development of Digital Twin Ship Technology Using Autonomous Surface Vehicle", 《2023 20TH INTERNATIONAL CONFERENCE ON UBIQUITOUS ROBOTS (UR)》, 24 August 2023 (2023-08-24), pages 255 - 260 * |
| 严新平;刘佳仑;范爱龙;马枫;李晨;: "智能船舶技术发展与趋势简述", 船舶工程, no. 03, 25 March 2020 (2020-03-25), pages 15 - 20 * |
| 胡一鹏等: "智能船艇虚实融合测试验证技术现状与展望", 《船舶工程》, vol. 44, no. 4, 25 April 2022 (2022-04-25), pages 4 - 11 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117928567B (en) | 2024-06-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10989537B2 (en) | Sonar sensor fusion and model based virtual and augmented reality systems and methods | |
| CN113340295B (en) | Unmanned ship near-shore real-time positioning and mapping method with multiple ranging sensors | |
| EP3388327B1 (en) | Route setting method for underwater vehicle, underwater vehicle optimum control method using same, and underwater vehicle | |
| CA2097822C (en) | Display system | |
| US11892298B2 (en) | Navigational danger identification and feedback systems and methods | |
| RU2483280C1 (en) | Navigation system | |
| JPH06301897A (en) | Navigation supporting device for vessel | |
| CN108939488B (en) | Sailing boat auxiliary training path planning method based on augmented reality | |
| KR102466804B1 (en) | Autonomous navigation method using image segmentation | |
| RU2467914C1 (en) | Method of ship navigability control and device to this end | |
| US11548598B2 (en) | Image generating device and method of generating image | |
| JP7255794B2 (en) | Disturbance zone determination method, mobile system and disturbance zone display method | |
| CN114820997A (en) | Unmanned ship virtual-real fusion multi-twin intelligent perception enhancement method | |
| CN117470249B (en) | Ship anti-collision method and system based on laser point cloud and video image fusion perception | |
| KR20240080189A (en) | Distance measurement method and distance measurement device using the same | |
| KR102339500B1 (en) | Method for providing recommended sea route based on electronic nautical chart by measuring topography and depth of water | |
| CN117928567B (en) | Ship auxiliary driving enhancing method | |
| CN116164747B (en) | Positioning and navigation method and system for underwater robot | |
| CN114092372B (en) | Method and system for superposing ship electronic channel chart and panoramic video image | |
| CN116486252A (en) | Intelligent unmanned search and rescue system and search and rescue method based on improved PV-RCNN target detection algorithm | |
| Douguet et al. | Multimodal perception for obstacle detection for flying boats-Unmanned Surface Vehicle (USV) | |
| JPS6130000A (en) | Automatic collision preventor for ship | |
| JPH0431439B2 (en) | ||
| KR102670260B1 (en) | Distance measurement method and distance measurement device using the same | |
| WO2023037500A1 (en) | Information processing device, determining method, program, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |