CN114020535A - Backtracking analysis system based on shipping data snapshot - Google Patents
Backtracking analysis system based on shipping data snapshot Download PDFInfo
- Publication number
- CN114020535A CN114020535A CN202111303459.5A CN202111303459A CN114020535A CN 114020535 A CN114020535 A CN 114020535A CN 202111303459 A CN202111303459 A CN 202111303459A CN 114020535 A CN114020535 A CN 114020535A
- Authority
- CN
- China
- Prior art keywords
- data
- information
- snapshot
- ais
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 99
- 238000000034 method Methods 0.000 claims abstract description 30
- 230000004931 aggregating effect Effects 0.000 claims abstract description 4
- 238000012216 screening Methods 0.000 claims description 24
- 230000008569 process Effects 0.000 claims description 18
- 238000012545 processing Methods 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 11
- 230000004044 response Effects 0.000 claims description 8
- 230000003203 everyday effect Effects 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 5
- 238000007405 data analysis Methods 0.000 claims description 4
- 238000004873 anchoring Methods 0.000 claims description 3
- 238000004140 cleaning Methods 0.000 claims description 3
- 235000019580 granularity Nutrition 0.000 description 26
- 238000010586 diagram Methods 0.000 description 20
- 238000007726 management method Methods 0.000 description 8
- 238000013500 data storage Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 238000013075 data extraction Methods 0.000 description 3
- PMHURSZHKKJGBM-UHFFFAOYSA-N isoxaben Chemical compound O1N=C(C(C)(CC)CC)C=C1NC(=O)C1=C(OC)C=CC=C1OC PMHURSZHKKJGBM-UHFFFAOYSA-N 0.000 description 3
- XEEYBQQBJWHFJM-UHFFFAOYSA-N Iron Chemical compound [Fe] XEEYBQQBJWHFJM-UHFFFAOYSA-N 0.000 description 2
- 238000005034 decoration Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 230000001172 regenerating effect Effects 0.000 description 2
- 230000008929 regeneration Effects 0.000 description 2
- 238000011069 regeneration method Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000007175 bidirectional communication Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000003245 coal Substances 0.000 description 1
- 230000006854 communication Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000002354 daily effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 229910052742 iron Inorganic materials 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/128—Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/148—File search processing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention relates to the field of shipping informatization, in particular to a backtracking analysis system based on shipping data snapshot, which comprises the following components: the storage center is used for storing data such as data snapshots; the snapshot generating module is used for generating data snapshots according to the snapshot time granularity aiming at the selected snapshot samples; the snapshot searching module is used for directly correlating ship data snapshots to quickly retrieve the latest state data of the specified slicing time; and the service analysis module is used for forming trend data taking the time slices as a time axis based on the data snapshots, defining the dynamically updated characteristic data snapshots into attribute groups according to categories, counting the characteristic data of each time slice, aggregating the time slices, namely forming cross-cycle trend statistics, and further carrying out next-step demand analysis. The method combines the characteristics of shipping data, carefully selects snapshot samples, self-defines the time granularity of the snapshots, and constructs a more flexible and fine-grained backtracking analysis system based on the data snapshots generated by full historical data.
Description
Technical Field
The invention relates to the technical field of shipping informatization, in particular to a backtracking analysis system based on shipping data snapshots.
Background
With the wide application and rapid development of technologies such as big data and cloud computing in various industries, informatization in the shipping field is gradually developing and growing, and particularly, the increase of shipping information data is gradual, so that the storage, analysis and utilization of shipping big data are particularly important. The shipping history data snapshot is mainly used for backtracking the history state or analyzing the history trend, and is an extremely important technology.
The definition of snapshots by the storage network industry association is: with respect to a fully available copy of a given data set, which includes an image of the corresponding data at some point in time (copy start point in time), the snapshot may be a copy of the data it represents, or a replica of the data. The snapshot has the main functions of online data backup and recovery, when application failure or file damage occurs to the storage device, rapid data recovery can be performed, the data is rolled back to a state of a qualified available time point, the other function of the snapshot is to provide another data access channel for a storage user, when metadata is subjected to online application processing, the user can access snapshot data, and the snapshot can be used for testing and other work.
At present, existing trace-back analysis systems or application systems mainly rely on data obtained by a traditional data snapshot generating method, such as patent CN111552437A as a representative, and such snapshots are mainly used in the field of secure operation and maintenance, such as fast restoration or repair of system environments. The other part is a snapshot technology for data, represented by patent CN102096613B, which also focuses on the data operation and maintenance level to recover data as the main purpose and function. The generation of the snapshots needs to be predefined, such as data ranges, time granularity, execution plans and the like of the snapshots, and once the snapshots are missed, the snapshots are difficult to make up, so that a backtracking analysis system based on the data snapshots is difficult to perform more accurate analysis and historical backtracking, especially under the condition of meeting the requirement of continuously updating and iterating a shipping data system.
In addition, in the field of shipping, the existing analysis systems mainly focus on the management, search and supervision of ships; for example, CN110135776A, a method, a system, and a computer storage medium for managing shipping information, which enable the shipping information to be updated in time and conveniently searched, and facilitate the supervision of supervision departments such as port and navigation. Or CN205508317U, can effectively display the port shipping information. Or for example, CN104112172A, can better manage and schedule shipping without being affected by severe weather. Further, for example, CN102103802A discloses a passenger ship shipping management system and a control method thereof, which utilize AIS (Automatic identification system) for bidirectional communication. That is, the analysis system in the prior art is mainly based on the analysis application of the data periodically reported by AIS, and is mainly applied to the management, positioning, navigation, supervision and the like of the ship, and an accurate analysis backtracking system is lacked to perform further analysis backtracking of the ship big data, so as to provide more analysis application functions for the shipping informatization field or other users.
In the aspect of the important characteristic that data applied to a shipping platform or a related analysis system is different from data in other fields, most of the data is intermediate calculation results, when calculation logic is changed, the data needs to be recalculated, and data snapshots also need to be updated based on the latest data.
Therefore, how to meet the background requirement of regenerating the snapshot of the data based on the latest calculation result in the technical field, and the shipping snapshot data based on the change can be quickly responded, accurately analyzed and traced back to help meet the specific analysis and application requirements of shipping companies or related companies and users, which is also a problem to be solved urgently in the shipping informatization technical field.
Disclosure of Invention
The invention provides a backtracking analysis system based on shipping data snapshot, which aims to meet the background requirement of regenerating data snapshot based on the latest calculation result in the technical field and can carry out quick response, accurate analysis and backtracking on the shipping snapshot data based on the change so as to help meet the specific analysis and application requirements of shipping companies or related companies and users.
The present invention provides the following scheme:
the invention provides a backtracking analysis system based on shipping data snapshots, which is characterized by comprising the following steps:
the storage center is used for storing data snapshots formed based on snapshot sample data and data generated in the running process of the storage system;
the snapshot generating module is used for generating shipping data snapshots according to certain snapshot time granularity aiming at different snapshot samples such as the latest AIS information, the AIS full history table and the flight information full history table which are fixedly reported, and storing the shipping data snapshots to the storage center in a storage table mode; the selection of the snapshot time granularity is determined according to the shipping cost and the refinement satisfaction degree of the shipping service;
the snapshot searching module is used for providing a query searching function based on the latest AIS information in a time dimension; the system comprises a data snapshot table, a ship basic information snapshot table and a data snapshot table, wherein the data snapshot table is used for storing ship basic information of a ship, and the data snapshot table is used for switching the association relation between the latest flight segment information table and the ship basic information into the association between the latest flight segment information snapshot table and the ship basic information based on the data snapshot, directly associating the data snapshot of the ship when different ship states in different periods are inquired and searched, and quickly searching the latest state data of the ship in a specified slicing time;
and the business analysis module is used for data analysis service of the ship, rapidly forming trend data taking the corresponding time slices as a time axis based on the data snapshots, defining the continuously and dynamically updated required characteristic data into attribute groups according to categories, counting the number of the ships of each corresponding time slice by utilizing the classified data snapshots based on the corresponding time slices and the attribute groups, aggregating the corresponding time slices, namely forming cross-cycle trend statistics, and assisting a user in carrying out next-step demand analysis based on the trend statistics.
Further, the snapshot generating module further includes a snapshot sample processing unit, including:
the snapshot sample data acquisition and fixed reporting submodule is used for acquiring the latest AIS information of fixed reporting, wherein the latest AIS information defines key information including AIS reporting time, ship position longitude and latitude, ground speed, ship heading, AIS reporting time and the like; the collection of all the AIS information table historical data forms an AIS full-scale historical table;
the dynamic update latest flight segment information submodule is used for updating latest flight segment information based on the latest AIS information of the obtained ship, and the latest flight segment information consists of the latest AIS information which is fixedly reported and flight segment information which dynamically changes along with the calculation logic; all the latest leg information historical data are collected to form a leg information total historical table;
and the sub-module for obtaining other important classification information is used for obtaining historical data based on the flight segment information, analyzing and summarizing data by combining with additional classification attributes such as trade, operation and commercial value of the ship, and obtaining internal and external trade operation states of the ship.
Further, the navigation section information dynamically changing along with the calculation logic refers to dynamic information of the ship calculated based on AIS historical data and a ship navigation dynamic judgment algorithm in the actual calculation process, wherein the dynamic information of the ship comprises dynamic navigation berthing, anchoring, starting port and destination port information and pre-support information; the dynamic judgment algorithm for the ship navigation is a dynamic algorithm which is continuously optimized in an alternating mode, the dynamic information of the ship is recalculated along with each optimization of the dynamic algorithm, and the latest flight section information is generated and updated based on the recalculated dynamic information of the ship.
Further, the snapshot generating module further includes a generating unit, including:
and (3) generating a latest AIS information data submodule: for generating the latest AIS information data based on the AIS data in the AIS data source extracted at the timing while copying into the AIS information table;
the direct snapshot generation submodule is used for configuring a timing task by utilizing an ETL tool, regularly copying a set of latest flight segment information every day, and directly generating the direct snapshot after the slicing time is associated;
the indirect snapshot generation submodule is used for selecting snapshot time granularity based on the AIS full-scale historical table, circularly extracting sample data near a specified time point according to a slice time sequence corresponding to the selected snapshot time granularity, and indirectly generating a data snapshot related to the latest AIS information; and selecting a snapshot time granularity based on the flight segment information full-scale historical table, circularly extracting sample data near a specified time point according to a slice time sequence corresponding to the selected snapshot time granularity, and indirectly generating a data snapshot related to the latest flight segment information.
Further, in the sub-module for generating the latest AIS information data, the method further includes:
an external data acquisition module: the AIS data is used for regularly extracting the AIS data with the reported updating time within the interval period from the external AIS data source;
a determine data increment module: the incremental data processing device is used for cleaning and converting the incremental data in the interval period and storing the incremental data as a temporary table;
an information generation module: the system is used for constructing PRE AIS information for the newly added AIS data, updating the latest AIS information containing PRE AIS information into a storage table of the latest AIS information, namely a set of the latest AIS information, and simultaneously copying the latest AIS information into the AIS full-scale history table.
Furthermore, the data snapshot in the storage center is stored in a storage table, each piece of data in the storage table includes two parts of data, the first part is a slice time, which refers to a slice time of one point on the snapshot time granularity, and includes a slice time primary key for uniquely defining one slice; the second part is a slice service, which is service data corresponding to the slice, wherein each service data includes ship unique identification, longitude, latitude, navigational speed, reporting time and the like.
Further, the snapshot search module is configured to switch the association relationship between the latest flight segment information table and the ship basic information to the association relationship between the latest flight segment information snapshot table and the ship basic information based on the data snapshot; the data snapshot comprises a generated latest flight segment information data snapshot, namely a directly generated snapshot.
Further, in the service analysis module, the method further includes:
the query information acquisition submodule is used for acquiring information input by a user, wherein the query information comprises information of regional analysis dimensions such as basic tag information, port name information and geographical position information of a ship, and time dimension information such as analysis time slice and reference process time;
and the quick response and analysis submodule is used for quickly responding and calling the corresponding data snapshot according to the acquired query information, analyzing, returning the return value of the analysis result to the display interface, and generating a corresponding analysis trend chart for the user to refer.
Further, the indirect snapshot generation submodule further includes a loop module, and the loop module is configured to generate a slice time sequence corresponding to a snapshot based on a corresponding snapshot time range; judging whether the screened slicing time is in the corresponding slicing time sequence or not, and further judging whether the screening cycle is finished or not; if yes, directly quitting the circulation and stopping screening; if not, generating a corresponding data snapshot for the corresponding slicing time through the AIS full-scale history table or the flight segment information full-scale history table; and finally, ending all the circulation until all the judgment processes stop screening, and ending the indirect snapshot generation.
Further, the indirectly generating the snapshot submodule further includes:
the data snapshot generating module of the latest AIS information comprises the following modules: the AIS data is used for screening all the AIS data of the ships with the reporting time before the corresponding slicing time in the AIS full-scale historical table and is defined as a temporary result corresponding to the AIS data; based on the corresponding temporary result, according to the ship identification group, screening the latest reporting time of each ship identification, forming a reporting time data set closest to the corresponding slicing time, and defining a corresponding temporary data set; associating the corresponding temporary result with the corresponding temporary data set through a Cartesian product, and screening out sample data (ship identification and latest reporting time) of a combination value in the corresponding temporary data set from the corresponding temporary result, namely indirectly generating a data snapshot associated with the latest AIS information;
the data snapshot generating module of the latest flight segment information: the system is used for screening the section information data of the ship before the corresponding slicing time in all time periods based on the section information total history table, and defining the section information data as a temporary result corresponding to the section information data; and grouping according to the ship identification based on the corresponding temporary result, acquiring corresponding latest flight segment information data when the slicing time falls in the corresponding flight segment period, and screening out a (ship identification, flight segment information) combination value, namely indirectly generating a data snapshot related to the latest flight segment information.
Compared with the prior art, the invention generates snapshots according to different time granularities, generates data snapshots based on the latest flight segment information data, and generates the latest state data snapshots based on massive historical data (AIS full history and flight segment information full history), thereby constructing a system convenient for backtracking and history analysis by utilizing snapshot data. The backtracking analysis system disclosed by the invention combines the characteristics of shipping data, carefully selects a sample set of snapshots, and by self-defining the time granularity of the snapshots and based on a snapshot generation method of the latest data and the historical data, under the condition that source data is changed, snapshot data based on the full-scale history are generated again in a key mode, so that the traditional snapshot characteristics are changed, namely the snapshot data must be captured based on the currently occurring data, namely a more flexible and fine-grained analysis backtracking system is constructed based on the historical data, and the shipping data analysis which is more complex than the prior art is realized.
The specific flexibility is embodied in the following aspects:
regional analysis latitude: based on the flight segment information snapshot data, the transportation capacity trend conditions of the port to port, port to country, country to port and country to region can be easily analyzed, and the analysis latitude supported by the flight segment information can be flexibly supported.
Time analysis latitude: based on the snapshot data with day as a unit, the running state of the ship can be recorded more accurately, the data storage granularity is adjusted from a minute level to a day level, the storage cost is reduced by nearly thousand times, the retrieval efficiency response is improved by nearly hundred times, and the multi-table association and complex business scene query can be supported more easily.
The snapshot data regeneration technology is convenient for generating more accurate snapshots based on the latest calculation results quickly after algorithm optimization, and further more accurate analysis results are obtained.
Drawings
Fig. 1 is a program module diagram of a backtracking analysis system based on a shipping data snapshot according to the present invention.
Fig. 2 is a diagram illustrating an example of storing a data snapshot generated in a backtracking analysis system based on a shipping data snapshot according to the present invention.
Fig. 3 is a program block diagram of a snapshot generating module in the system according to the present invention.
Fig. 4 is a program module diagram of a snapshot sample processing unit in a snapshot generating module in the system according to the present invention.
Fig. 5 is a program module diagram of a generating unit in a snapshot generating module in the system according to the present invention.
Fig. 6 is a program module diagram of a sub-module for generating the latest AIS information data in the system provided by the invention.
FIG. 7 is a block diagram of a program for indirectly generating a snapshot submodule in the system according to the present invention.
Fig. 8 is a program module diagram of a business analysis module in the system provided by the invention.
Fig. 9 is a trend analysis result diagram for both internal and external trade and camp vessels of the backtracking analysis system based on shipping data snapshot provided by the present invention.
Fig. 10 is a system deployment diagram of another traceability analysis system based on a shipping data snapshot provided by the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
First, some terms or terms appearing in the description of the embodiments of the present application are applicable to the following explanations:
an Automatic Identification System (AIS) for ships is a navigation aid System originally applied to marine safety and communication between ships and shore, and plays an important role in the field of marine service such as reducing ship collision accidents. The massive data information accumulated by the AIS reflects the real-time dynamics of global marine ships, and is a valuable big data resource which can be used for analyzing port and navigation management problems. Besides being used for ship collision avoidance, the massive data can also be applied to analysis and decision in the field of port and navigation management through technical methods such as data processing and mining, and has huge utilization potential, such as: the system comprises a port operation management system, a ship operation management system, a fleet dynamic monitoring system, a maritime management system and the like.
The international maritime organization stipulates that ships of international airlines of more than 300 total tons, cargo ships of non-international airlines of more than 500 total tons and all passenger ships need to be provided with AIS equipment; the AIS message contains the current dynamic information (such as position and speed), static information (such as ship length and width) and relevant information (such as destination and predicted arrival time) of the ship; the AIS messages are transmitted through a VHF channel with the broadcasting frequency of 2s-30 s; AIS incorporates standardized VHF transceivers, positioning systems (e.g., GPS receivers), and other electronic navigation sensors (e.g., gyrocompass, tachometer). Through the special VHF transmitter, the information data about the ship itself will be broadcast and transmitted to other receivers; the transmission distance limits for the inter-ship and inter-shore AIS signals are approximately 20nmi and 40nmi, respectively.
The ship-borne AIS equipment data is generally reported through a satellite, a coastal base station, ship-borne signal equipment and the like, an AIS data provider integrates collected AIS data to form AIS data service, and the AIS data service is synchronously provided for business partners in batches according to certain time frequency. In the present invention, as an AIS data consumer, the latest AIS data is purchased and periodically extracted from the AIS server.
The ETL data extraction transformation and loading technique is an abbreviation used in english Extract-Transform-Load and is used to describe the process of extracting (Extract), transforming (Transform), and loading (Load) data from a source to a destination. The main use of the ETL in the invention comprises configuring a timing execution task and calling a database storage process according to a plan; configuring a remote execution task, and remotely executing a command according to a plan and notifying an execution result by an email; and configuring a data checking task, checking the data according to the execution plan and notifying the result.
KETTLE data Extraction transfer, KettlE was originally an open source ETL tool, and is generally called KDE Extraction, transfer, Transformation and Loading Environment.
Besides, the AIS full-scale history table and the latest AIS information table have the same structure, except that each ship only has one latest record in the latest AIS information, but the AIS records of all histories, namely the time and the latitude are different in the full-scale history.
In order to make the advantages of the technical solutions of the present invention clearer, the present invention is described in detail below with reference to the accompanying drawings and embodiments.
The invention provides a backtracking analysis system based on shipping data snapshots, as shown in fig. 1, the invention provides a program module diagram of the backtracking analysis system based on shipping data snapshots, which comprises:
the storage center 101 is configured to store a data snapshot formed based on snapshot sample data and data generated during the operation of the storage system. In the embodiment, snapshot data formed based on the latest ship leg information and the important classification information of the ship has smaller storage pressure than the full history, can be stored in a single name space under the same database, can be directly associated and accessed with the service data, and forms a set of complete analysis and backtracking system by combining the snapshot data and the latest data. The hardware device of the storage center 101 is built using a virtual server environment of the cloud data center, and in practical applications, the hardware device is not limited to the above environment, and includes a local or remote, virtual or actual physical device that can be implemented by a person skilled in the art.
A snapshot generating module 102, configured to generate a shipping data snapshot according to a certain snapshot time granularity for different snapshot samples, such as the latest AIS information, the AIS total history table, and the leg information total history table that are fixedly reported, and store the shipping data snapshot in the storage center 101 in the form of a storage table; the selection of the snapshot time granularity is determined according to the shipping cost and the fine satisfaction degree of the shipping service. Based on the fact that different snapshot samples including the latest AIS information, the AIS full-scale historical table and the voyage information full-scale historical table which are fixedly reported can be selected in a fine mode, a foundation is provided for the system to provide a more complex scene and fine-grained analysis backtracking.
A snapshot search module 103, configured to provide a query search function based on the latest AIS information in a time dimension; and the system is used for switching the association relation between the latest flight segment information table and the ship basic information into the association between the latest flight segment information snapshot table and the ship basic information based on the data snapshot, directly associating the data snapshot of the ship when inquiring and searching different ship states in different periods, and quickly searching the latest state data of the ship in the appointed slice time. That is, for example, on the basis of the original search depending on a single piece (or one point) of data in the mass data, a search basis with a larger granularity is provided, for example, data of a latest flight segment data slice is searched in the mass shipping data, so that the search time is shortened, the rapid search is realized, and a basis is also provided for the following analysis.
And the business analysis module 104 is used for data analysis service of the ship, rapidly forming trend data taking the corresponding time slice as a time axis based on the data snapshot, defining the required feature data which is continuously and dynamically updated into attribute groups according to categories, counting the number of the ships of each corresponding time slice by using the classified data snapshot based on the corresponding time slice and the attribute groups, aggregating the corresponding time slices, namely forming cross-cycle trend statistics, and assisting a user in carrying out next-step demand analysis based on the trend statistics. The business analysis module 104 of the present invention relies on the generation module and the search module to perform further analysis backtracking of ship big data (including the latest AIS data extracted from AIS service providers periodically and the latest voyage data, history data, other classification data, etc. generated and selected by sorting in particular) to provide more analysis application functions for the shipping informatization field or other users.
In the snapshot generating module 102, the selection of the snapshot time granularity is mainly selected according to an actual industrial situation, for example, in the field of shipping data, and is determined according to the shipping cost and the refinement satisfaction degree of shipping services; specifically, in the snapshot sample, the update frequency of the latest AIS information is in minutes, the navigation dynamic information is in hours or days, and the important classification information can be in weeks or even months.
Preferably, in view of a plurality of factors such as the design speed, the actual speed, the voyage, the comprehensive cost, the efficiency and the service acceptance degree of the mainstream commercial ship, the selection of the snapshot time granularity takes day as a unit, namely, one snapshot is extracted every day according to the latest flight segment information and other important classification information of the ship; further refining the snapshot time granularity, and selecting a specific time point of the current day based on the local service actual operation time of a specific region time zone, wherein the specific time point comprises uniformly selecting UTC time 00:00, namely 08:00 in a GMT +8 time zone as a sampling time point.
In the actual process, based on the snapshot samples, the update frequency of the latest AIS information is in minutes, the navigation dynamic information is in hours or days, and the important classification information is in weeks or even months.
The AIS data is characterized in that the density is unequal and the magnitude is large, the AIS data is reported once in 10 minutes on average by a single ship, the AIS fault time is removed, the average reporting time in the whole year reaches 300, and the AIS data volume of one ship in one year reaches 43200. According to the data of the Lao's classification society, the number of global registered ships reaches 300, a part of small fishing boats, short barge ships and small special operation ships are removed, and the number of AIS data generated by global commercial ships in one year exceeds 800 hundred million according to the calculation of 200 million commercial ships. In the business system related to this embodiment, about 10 million ships are intensively tracked and analyzed, and about 80 million AIS historical data are obtained in 2 years. When the business system carries out history analysis and directly retrieves the full AIS historical data, the quick response is difficult, the user experience is poor, and the data with larger granularity needs to be separately stored, so that the selection of the data snapshot storage with large granularity and the selection of the appropriate slice time granularity of the data snapshot are very important.
Considering that the designed speed of the mainstream commercial ship is basically below 25 knots, namely 25 nautical miles per hour, and the actual speed is lower in the world, taking an Asia-Meixi route as an example, the voyage is usually more than 10 days, more than 5000 nautical miles, small state change in one day, comprehensive cost, efficiency, service acceptance degree and other factors, the scheme is unified to the day according to the snapshot time granularity of the shipping data samples, namely a snapshot is extracted every day according to the latest voyage section information and other important classification information of the ship. The specific time point of the daily snapshot may be any time point in the day in principle, and for convenience of operation, the UTC time 00:00 point, i.e., 08:00 point in the GMT +8 time zone, may be uniformly selected as the sampling time point.
In this embodiment, the data snapshots in the storage center 101 are stored in the form of a storage table, each piece of data in the storage table includes two parts of data, the first part is a slice time, which is a slice time at a point on the snapshot time granularity, and includes a slice time primary key for uniquely defining a slice; the second part is a slice service, which is service data corresponding to the slice, wherein each service data includes ship unique identification, longitude, latitude, navigational speed, reporting time and the like. For convenience of explaining the implementation process of the invention, corresponding symbols are defined for some terms, such as unique ship identification (MMSI), longitude LON, latitude LAT, speed of flight (SOG) and reporting time POSTIME; it is noted, however, that the method of the present invention is not limited to the definitions set forth herein, but includes all similar nomenclature used in the practice of the invention.
Preferably, as shown in fig. 2, in the example of storage of a data snapshot generated in the system provided by the present invention, taking the latest AIS information as an example in fig. 2, it is assumed that the object name of the storage table of the data is AIS _ NEW, which is different from the AIS history table that includes all AIS history records of every day ship, and in the AIS _ NEW table, each ship only stores the latest AIS record, and each line of data at least includes a ship unique identifier MMSI, a longitude on, a latitude LAT, a speed SOG, and a reporting time post. The AIS _ NEW SNAPSHOT table is stored separately from the AIS _ NEW table, for example, named as snap _ AIS _ NEW, and each snap _ AIS _ NEW needs to include at least two parts of data, where the first part is a TIME primary key SLICE _ TIME of a SLICE to uniquely define one SLICE, and the second part is service data corresponding to the SLICE, as shown in fig. 2, data in the same SLICE can be locked by the SLICE _ TIME, and different SLICEs are distinguished by different SLICE _ TIME.
Fig. 3 is a program block diagram of a snapshot generating module in the system provided by the present invention. As shown in fig. 3, the snapshot generating module further includes a snapshot sample processing unit 1021 and a generating unit 1022. The snapshot sample processing unit 1021 is mainly used for providing different snapshot samples; the generating unit 1022 is configured to generate a shipping data snapshot by one-touch fast according to a certain snapshot time granularity according to different snapshot samples, and store the shipping data snapshot in the storage center 101 in the form of a storage table.
As shown in fig. 4, a block diagram of a snapshot sample processing unit 1021 in the snapshot generating module 102 in the system provided by the present invention is shown.
Wherein the snapshot sample processing unit 1021 includes:
the snapshot sample data obtaining module 10211 is configured to obtain latest AIS information of the fixed report, where the latest AIS information defines key information including AIS report time, ship position longitude and latitude, speed to ground, ship heading, AIS report time, and the like.
A dynamic update latest flight segment information submodule 10212, configured to update latest flight segment information based on the latest AIS information of the acquired ship, where the latest flight segment information is composed of the latest AIS information that is fixedly reported and flight segment information that dynamically changes along with computation logic; all the latest leg information historical data are collected to form a leg information total historical table; in this embodiment, the flight segment information that dynamically changes along with the computation logic refers to dynamic information of a ship calculated based on AIS historical data and a ship navigation dynamic judgment algorithm in an actual computation process, where the dynamic information of the ship includes dynamic navigation berthing, anchoring, origin port and destination port information, and pre-support information; the dynamic judgment algorithm for the ship navigation is a dynamic algorithm which is continuously optimized in an alternating mode, the dynamic information of the ship is recalculated along with each optimization of the dynamic algorithm, and the latest flight section information is generated and updated based on the recalculated dynamic information of the ship.
The obtain other important classification information sub-module 10213 is configured to obtain historical data based on the leg information, and obtain data obtained by analyzing and summarizing the historical data in combination with additional classification attributes such as trade, operation, and commercial value of the ship, so as to obtain internal and external trade operation states of the ship. In this embodiment, the other important classification information is data obtained by analyzing and summarizing additional classification attributes, such as trade, operation, and commercial value of the ship, based on the historical data in the full-scale historical table of the leg information, and includes internal and external trade operation state data of the ship, and the like. The other important classification information, such as internal and external trade operation state information of the ship, is dynamically counted and summarized based on the ship history; preferably, for example, the internal and external trade attributes of the ship are determined based on the data snapshot of the internal and external trade operation states of the ship, that is, according to whether the port attachment record of a past period of time contains an overseas port, and the section data is formed, so as to record the internal and external trade characteristics of the section time node, that is, according to whether the port attachment record of a past period of time contains an overseas port, the internal and external trade attributes of the port are determined, and the section data is formed, so as to record the internal and external trade characteristics of the section time node. Preferably, as shown in fig. 7, the data of internal and external trade operation states and the like may be used to provide services such as foreign trade analysis, consultation, pricing, cost reference based on the external trade analysis, the data of internal and external trade operation states and the like to the user or the shipping enterprise.
As shown in fig. 5, the present invention provides a program module diagram of the generating unit 1022 in the snapshot generating module 102 in the system.
Wherein the generating unit 1022 includes:
generate up-to-date AIS information data sub-module 10221: the system is used for generating the latest AIS information data based on the AIS data in the timing extraction AIS data source and simultaneously copying the latest AIS information data into an AIS full-scale history table.
The snapshot directly generating sub-module 10222 is configured to utilize an ETL tool, in this embodiment, a key is utilized to perform data extraction and conversion, configure a timing task, periodically copy the generated set of the latest flight segment information every day, and directly generate the direct snapshot after associating the slicing time.
The indirect snapshot generation submodule 10223 is configured to cyclically extract sample data near a specified time point according to a slicing time sequence based on the AIS full history table, and indirectly associate a data snapshot of the latest AIS information; and the data snapshot system is used for selecting snapshot time granularity based on the flight segment information full-scale historical table, circularly extracting sample data near a specified time point according to a slice time sequence corresponding to the selected snapshot time granularity, and indirectly generating the data snapshot related to the latest flight segment information.
Preferably, in the module 10221 for generating the latest AIS information data, as shown in fig. 6, the program module diagram of the module for generating the latest AIS information data in the system provided by the present invention includes:
get external data module 1022101: the AIS data is used for regularly extracting the AIS data with the reported updating time within the interval period from the external AIS data source;
determine data increment module 1022102: the incremental data processing device is used for cleaning and converting the incremental data in the interval period and storing the incremental data as a temporary table;
the generate information module 1022103: the system is used for constructing PRE AIS information for the newly added AIS data, updating the latest AIS information containing PRE AIS information into a storage table of the latest AIS information, namely a set of the latest AIS information, and simultaneously copying the latest AIS information into the AIS full-scale history table.
In the indirect snapshot generating submodule 10223, as shown in fig. 7, a program module diagram of the indirect snapshot generating submodule in the system provided by the present invention includes:
a loop module 1022301 for generating a slice time sequence corresponding to a snapshot based on a corresponding snapshot time range; judging whether the screened slicing time is in the corresponding slicing time sequence or not, and further judging whether the screening cycle is finished or not; if yes, directly quitting the circulation and stopping screening; if not, generating a corresponding data snapshot for the corresponding slicing time through the AIS full-scale history table or the flight segment information full-scale history table; and finally, ending all the circulation until all the judgment processes stop screening, and ending the indirect snapshot generation.
Data snapshot generating module 1022302 of the latest AIS information: the AIS data is used for screening all the AIS data of the ships with the reporting time before the corresponding slicing time in the AIS full-scale historical table and is defined as a temporary result corresponding to the AIS data; based on the corresponding temporary result, according to the ship identification group, screening the latest reporting time of each ship identification, forming a reporting time data set closest to the corresponding slicing time, and defining a corresponding temporary data set; and associating the corresponding temporary result with the corresponding temporary data set through a Cartesian product, and screening out sample data (ship identification and latest reporting time) of a combination value in the corresponding temporary data set from the corresponding temporary result, namely indirectly generating a data snapshot associated with the latest AIS information.
Data snapshot generating module 1022303 of the latest leg information: the system is used for screening the section information data of the ship before the corresponding slicing time in all time periods based on the section information total history table, and defining the section information data as a temporary result corresponding to the section information data; and grouping according to the ship identification based on the corresponding temporary result, acquiring corresponding latest flight segment information data when the slicing time falls in the corresponding flight segment period, and screening out a (ship identification, flight segment information) combination value, namely indirectly generating a data snapshot related to the latest flight segment information.
In this embodiment, it is preferable that the snapshot search module 103 is configured to switch, based on the data snapshot, an association relationship between the latest flight segment information table and the ship basic information to an association relationship between the latest flight segment information snapshot table and the ship basic information; wherein the data snapshot comprises the latest generated AIS information data snapshot, namely the directly generated snapshot. Therefore, by the snapshot data regeneration technology, after algorithm optimization, more accurate snapshots can be generated quickly based on the latest calculation results, and more accurate search query results can be obtained during search query.
In this embodiment, as shown in fig. 8, a program module diagram of a business analysis module 104 in the system provided by the present invention, where the business analysis module 104 further includes:
the query information obtaining sub-module 1041 is configured to obtain information input by a user, where the query information includes information of regional analysis dimensions such as basic tag information of a ship, port name information, and geographic location information, and time dimension information such as an analysis time slice and a reference process time;
the quick response and analysis sub-module 1042 is configured to quickly respond to and call the corresponding data snapshot according to the obtained query information, perform analysis, return a return value of an analysis result to a display interface, and generate a corresponding analysis trend chart for a user to refer to.
Wherein, the regional analysis latitude is: based on the flight segment information snapshot data, the transportation capacity trend conditions of the port to port, port to country, country to port and country to region can be easily analyzed, and the analysis latitude supported by the flight segment information can be flexibly supported. Further, the latitude is analyzed in time: based on the snapshot data with day as a unit, the running state of the ship can be recorded more accurately, the data storage granularity is adjusted from a minute level to a day level, the storage cost is reduced by nearly thousand times, the retrieval efficiency response is improved by nearly hundred times, and the multi-table association and complex business scene query can be supported more easily.
Preferably, the business analysis module 104 can easily form trend data with SLICE _ TIME as a TIME axis based on the snapshot data, and the description will be given by taking a snapshot of ship classification state data as an example. As shown in fig. 9, a trend analysis result diagram for an internal and external trade-concurrently-operating ship of the backtracking analysis system based on shipping data snapshots provided by the present invention; internal and external trade characteristics of the ship are analyzed based on the ship attachment history, characteristic data are defined into VESSEL _ CATALOG according to categories, and the data are dynamic data and are updated once a day. By using the classified SNAPSHOT data SNAPSHOT _ VESSEL _ CATALOG, the number of internal and external trade ships of each SLICE _ TIME can be counted based on SLICE _ TIME and internal and external trade attribute grouping, and the aggregated SLICE _ TIME can further form cross-period trend statistics.
In order to further illustrate the present invention, a system deployment diagram of another traceability analysis system based on shipping data snapshots is shown here, as shown in fig. 10, where the system deployment diagram mainly includes deployment of storage of data, deployment of snapshot generation, and deployment of traceability analysis based on data snapshots, where a generated data snapshot is stored in a data storage system, a process of generating data snapshots again calls snapshot sample data in the data storage system, and the traceability analysis system calls the data storage system and stores traceability analysis results in the data storage system. The data storage deployment comprises the deployment of AIS data, flight segment data and pre-support data; the deployment of snapshot generation mainly comprises deployment of direct snapshot generation and deployment of indirect snapshot generation; deployment of backtracking analysis includes, but is not limited to, vessel history retrieval systems, cargo capacity analysis systems, inside and outside trade trend analysis systems, and the like.
Preferably, the freight condition between designated ports in a past period of time is statistically analyzed based on the flight section information data snapshot, for example, within 3 months, the transportation volume of dry bulk goods from black drac australia to the Qingdao china is analyzed, the snapshot data within 3 months is selected according to time slices, the flight section information in the statistical slice data conforms to the ship load ton of the dry bulk goods from black drac to the Qingdao china, the transportation capacity trend is generated, and the transportation capacity condition of coal or iron ore is tracked and analyzed.
Preferably, for example, based on the snapshot of the navigation section information data, statistics and analysis are performed on internal and external trade attribute conditions of different ships within a period of time in the past, namely, the destination ports of the ship are internal trades in the domestic ports, and any port of the destination ports or the ship is an external trade in the overseas ports.
Preferably, the query snapshot analysis service has a service accuracy similar to the AIS full-scale history of direct search, because the dynamic change period of the ship is long and is usually in units of days; the response time of inquiring snapshot analysis service is one tenth of that of directly searching AIS full-scale history, because the AIS full-scale history data of 10 ten thousand commercial ships in 3 years can reach 100 hundred million levels, the retrieval time is very long, and more data tables are difficult to associate to realize complex inquiry.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A backtracking analysis system based on a snapshot of shipping data, comprising:
the storage center is used for storing data snapshots formed based on snapshot sample data and data generated in the running process of the storage system;
the snapshot generating module is used for generating shipping data snapshots according to certain snapshot time granularity aiming at different snapshot samples such as the latest AIS information, the AIS full history table and the flight information full history table which are fixedly reported, and storing the shipping data snapshots to the storage center in a storage table mode; the selection of the snapshot time granularity is determined according to the shipping cost and the refinement satisfaction degree of the shipping service;
the snapshot searching module is used for providing a query searching function based on the latest AIS information in a time dimension; the system comprises a data snapshot table, a ship basic information snapshot table and a data snapshot table, wherein the data snapshot table is used for storing ship basic information of a ship, and the data snapshot table is used for switching the association relation between the latest flight segment information table and the ship basic information into the association between the latest flight segment information snapshot table and the ship basic information based on the data snapshot, directly associating the data snapshot of the ship when different ship states in different periods are inquired and searched, and quickly searching the latest state data of the ship in a specified slicing time;
and the business analysis module is used for data analysis service of the ship, rapidly forming trend data taking the corresponding time slices as a time axis based on the data snapshots, defining the continuously and dynamically updated required characteristic data into attribute groups according to categories, counting the number of the ships of each corresponding time slice by utilizing the classified data snapshots based on the corresponding time slices and the attribute groups, aggregating the corresponding time slices, namely forming cross-cycle trend statistics, and assisting a user in carrying out next-step demand analysis based on the trend statistics.
2. The retrospective analysis system of claim 1, further characterized in that the snapshot generating module further comprises a snapshot sample processing unit comprising:
the snapshot sample data acquisition and fixed reporting submodule is used for acquiring the latest AIS information of fixed reporting, wherein the latest AIS information defines key information including AIS reporting time, ship position longitude and latitude, ground speed, ship heading, AIS reporting time and the like; the collection of all the AIS information table historical data forms an AIS full-scale historical table;
the dynamic update latest flight segment information submodule is used for updating latest flight segment information based on the latest AIS information of the obtained ship, and the latest flight segment information consists of the latest AIS information which is fixedly reported and flight segment information which dynamically changes along with the calculation logic; all the latest leg information historical data are collected to form a leg information total historical table;
and the sub-module for obtaining other important classification information is used for obtaining historical data based on the flight segment information, analyzing and summarizing data by combining with additional classification attributes such as trade, operation and commercial value of the ship, and obtaining internal and external trade operation states of the ship.
3. The backtracking analysis system according to claim 2, wherein the leg information dynamically changing along with the computation logic is ship dynamic information calculated based on AIS historical data and a ship navigation dynamic judgment algorithm in an actual computation process, and the ship dynamic information includes navigation dynamic berthing, anchoring, origin port and destination port information, and pre-support information; the dynamic judgment algorithm for the ship navigation is a dynamic algorithm which is continuously optimized in an alternating mode, the dynamic information of the ship is recalculated along with each optimization of the dynamic algorithm, and the latest flight section information is generated and updated based on the recalculated dynamic information of the ship.
4. The backtracking analysis system according to claim 1 or 2, further characterized in that the snapshot generating module further comprises a generating unit comprising:
and (3) generating a latest AIS information data submodule: for generating the latest AIS information data based on the AIS data in the AIS data source extracted at the timing while copying into the AIS information table;
the direct snapshot generation submodule is used for configuring a timing task by utilizing an ETL tool, regularly copying a set of latest flight segment information every day, and directly generating the direct snapshot after the slicing time is associated;
the indirect snapshot generation submodule is used for selecting snapshot time granularity based on the AIS full-scale historical table, circularly extracting sample data near a specified time point according to a slice time sequence corresponding to the selected snapshot time granularity, and indirectly generating a data snapshot related to the latest AIS information; and selecting a snapshot time granularity based on the flight segment information full-scale historical table, circularly extracting sample data near a specified time point according to a slice time sequence corresponding to the selected snapshot time granularity, and indirectly generating a data snapshot related to the latest flight segment information.
5. The retrospective analysis system of claim 4, further characterized in that, in the generate-latest AIS information data submodule, further comprising:
an external data acquisition module: the AIS data is used for regularly extracting the AIS data with the reported updating time within the interval period from the external AIS data source;
a determine data increment module: the incremental data processing device is used for cleaning and converting the incremental data in the interval period and storing the incremental data as a temporary table;
an information generation module: the system is used for constructing PRE AIS information for the newly added AIS data, updating the latest AIS information containing PRE AIS information into a storage table of the latest AIS information, namely a set of the latest AIS information, and simultaneously copying the latest AIS information into the AIS full-scale history table.
6. The retrospective analysis system of claim 1, further characterized in that the data snapshots in the storage center are stored in the form of a storage table, each piece of data in the storage table includes two parts, the first part is a slice time, which is a slice time at a point on a snapshot time granularity, and includes a slice time primary key for uniquely defining a slice; the second part is a slice service, which is service data corresponding to the slice, wherein each service data includes ship unique identification, longitude, latitude, navigational speed, reporting time and the like.
7. The backtracking analysis system according to claim 1 or 4, further characterized in that the snapshot search module is configured to switch the association relationship between the latest flight segment information table and the ship basic information to the association relationship between the latest flight segment information snapshot table and the ship basic information based on the data snapshot; the data snapshot comprises a generated latest flight segment information data snapshot, namely a directly generated snapshot.
8. The retrospective analysis system of claim 1, further characterized in that, in the traffic analysis module, it further comprises:
the query information acquisition submodule is used for acquiring information input by a user, wherein the query information comprises information of regional analysis dimensions such as basic tag information, port name information and geographical position information of a ship, and time dimension information such as analysis time slice and reference process time;
and the quick response and analysis submodule is used for quickly responding and calling the corresponding data snapshot according to the acquired query information, analyzing, returning the return value of the analysis result to the display interface, and generating a corresponding analysis trend chart for the user to refer.
9. The retrospective analysis system of claim 1 or 4,
the indirect snapshot generation submodule further comprises a circulation module, and the circulation module is used for generating a slice time sequence corresponding to the snapshot based on the corresponding snapshot time range; judging whether the screened slicing time is in the corresponding slicing time sequence or not, and further judging whether the screening cycle is finished or not; if yes, directly quitting the circulation and stopping screening; if not, generating a corresponding data snapshot for the corresponding slicing time through the AIS full-scale history table or the flight segment information full-scale history table; and finally, ending all the circulation until all the judgment processes stop screening, and ending the indirect snapshot generation.
10. The backtracking analysis system according to claim 1 or 9, wherein the indirect snapshot generation submodule further comprises:
the data snapshot generating module of the latest AIS information comprises the following modules: the AIS data is used for screening all the AIS data of the ships with the reporting time before the corresponding slicing time in the AIS full-scale historical table and is defined as a temporary result corresponding to the AIS data; based on the corresponding temporary result, according to the ship identification group, screening the latest reporting time of each ship identification, forming a reporting time data set closest to the corresponding slicing time, and defining a corresponding temporary data set; associating the corresponding temporary result with the corresponding temporary data set through a Cartesian product, and screening out sample data (ship identification and latest reporting time) of a combination value in the corresponding temporary data set from the corresponding temporary result, namely indirectly generating a data snapshot associated with the latest AIS information;
the data snapshot generating module of the latest flight segment information: the system is used for screening the section information data of the ship before the corresponding slicing time in all time periods based on the section information total history table, and defining the section information data as a temporary result corresponding to the section information data; and grouping according to the ship identification based on the corresponding temporary result, acquiring corresponding latest flight segment information data when the slicing time falls in the corresponding flight segment period, and screening out a (ship identification, flight segment information) combination value, namely indirectly generating a data snapshot related to the latest flight segment information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111303459.5A CN114020535B (en) | 2021-11-05 | 2021-11-05 | Backtracking analysis system based on shipping data snapshot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111303459.5A CN114020535B (en) | 2021-11-05 | 2021-11-05 | Backtracking analysis system based on shipping data snapshot |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114020535A true CN114020535A (en) | 2022-02-08 |
| CN114020535B CN114020535B (en) | 2024-04-09 |
Family
ID=80061482
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111303459.5A Active CN114020535B (en) | 2021-11-05 | 2021-11-05 | Backtracking analysis system based on shipping data snapshot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114020535B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116319836A (en) * | 2023-05-22 | 2023-06-23 | 江苏物润船联网络股份有限公司 | Service flow data synchronous processing method and system, electronic equipment and storage medium |
| CN117520313A (en) * | 2024-01-02 | 2024-02-06 | 北京淇瑀信息科技有限公司 | Data backtracking method and device based on multidimensional associated data warehouse slice table |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110067030A1 (en) * | 2009-09-16 | 2011-03-17 | Microsoft Corporation | Flow based scheduling |
| CN112862321A (en) * | 2021-02-08 | 2021-05-28 | 国家海洋信息中心 | Ocean transportation statistical system based on AIS big data and statistical method thereof |
-
2021
- 2021-11-05 CN CN202111303459.5A patent/CN114020535B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110067030A1 (en) * | 2009-09-16 | 2011-03-17 | Microsoft Corporation | Flow based scheduling |
| CN112862321A (en) * | 2021-02-08 | 2021-05-28 | 国家海洋信息中心 | Ocean transportation statistical system based on AIS big data and statistical method thereof |
Non-Patent Citations (2)
| Title |
|---|
| 严勇杰;曹罡;: "下一代空管***运行概念及其关键技术", 指挥信息***与技术, no. 03, 2 July 2018 (2018-07-02) * |
| 何建春;: "试述基于多传感信息融合的内河水运统计应用研究", 中国水运, no. 10, 15 October 2017 (2017-10-15) * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116319836A (en) * | 2023-05-22 | 2023-06-23 | 江苏物润船联网络股份有限公司 | Service flow data synchronous processing method and system, electronic equipment and storage medium |
| CN117520313A (en) * | 2024-01-02 | 2024-02-06 | 北京淇瑀信息科技有限公司 | Data backtracking method and device based on multidimensional associated data warehouse slice table |
| CN117520313B (en) * | 2024-01-02 | 2024-03-26 | 北京淇瑀信息科技有限公司 | Data backtracking method and device based on multidimensional associated data warehouse slice table |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114020535B (en) | 2024-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210109235A1 (en) | System and method for tracking and forecasting the positions of marine vessels | |
| CN112862321B (en) | Ocean transportation statistical system based on AIS big data and statistical method thereof | |
| Patroumpas et al. | Online event recognition from moving vessel trajectories | |
| CN114020535B (en) | Backtracking analysis system based on shipping data snapshot | |
| EP3050005B1 (en) | Computer-implemented methods of analyzing spatial, temporal and contextual data for predictive decision-making | |
| US9158035B2 (en) | System and method of automated acquisition, correlation and display of power distribution grid operational parameters and weather events | |
| US10073724B2 (en) | Method of intuition generation | |
| US7933693B2 (en) | System and method for harvesting business intelligence from maritime communications | |
| CN105917625A (en) | Classification of detected network anomalies using additional data | |
| Štepec et al. | Machine learning based system for vessel turnaround time prediction | |
| CN114037230B (en) | Method and system for rapidly and accurately identifying predicted empty ship information | |
| CN110555452A (en) | network problem processing method and device based on intelligent clustering | |
| Alqudah et al. | Automated power system fault prediction and precursor discovery using multi-modal data | |
| Askari et al. | Taxi demand prediction using an LSTM-based deep sequence model and points of interest | |
| Cazzanti et al. | A document-based data model for large scale computational maritime situational awareness | |
| Zhang et al. | How liner shipping heals schedule disruption: A data-driven framework to uncover the strategic behavior of port-skipping | |
| CN114020534B (en) | Shipping data snapshot generating method | |
| Bonham et al. | Analysing port and shipping operations using big data | |
| CN116596435A (en) | Global ship cargo quantity statistical analysis method and system | |
| Viazilov et al. | On the Development of a Pipeline for Processing Hydrometeorological Data. | |
| Alessandrini et al. | Vessel tracking data usage to map Mediterranean flows | |
| Scully | Tidal analysis and arrival process mining using automatic identification system (AIS) data | |
| Ratnasari et al. | A Preliminary Study of Supply Chain Management for Logistics in Indonesia | |
| Stróżyna et al. | Architecture of maritime awareness system supplied with external information | |
| Wang et al. | A functional model of AIS data fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |