Disclosure of Invention
The invention aims to provide a method for integrally processing comprehensive risk census data of natural disasters and application thereof, which can realize synchronous updating after multi-source heterogeneous data fusion processing and field check data quality inspection by using a data linkage updating platform device and finally realize the establishment of an regional natural disaster index system and a natural disaster risk assessment model.
In order to achieve the above object, in a first aspect, the present invention provides a method for integrally processing natural disaster comprehensive risk census data, comprising the steps of:
acquiring multi-source heterogeneous data and constructing a standardized vector database;
acquiring a national platform vector database for natural disaster comprehensive risk screening, and constructing a mutual mapping relation between a standardized vector database and the national platform vector database;
obtaining field investigation data by field investigation;
performing quality inspection on the field investigation data;
and updating the field investigation data after quality inspection and the national platform vector database in a linkage way to obtain investigation result data.
In an alternative embodiment, the step of obtaining multi-source heterogeneous data and constructing a standardized vector database includes:
the method comprises the steps of obtaining multi-source heterogeneous data in different formats of different industry departments, classifying the obtained multi-source heterogeneous data into spatial data and non-spatial data, building a multi-source heterogeneous data directory tree, building a data cleaning rule and a model aiming at application requirements of natural disaster census data, cleaning the data, automatically converting the multi-mode modular data by adopting four modes of address matching, data association, entity fusion and position conversion to obtain correct data, and building a standardized database according to national house building and municipal facility investigation house building data database building standards.
In an alternative embodiment, constructing the normalized vector database and the country platform vector database with respect to each other includes:
and integrating the national platform vector database and the standardized vector database according to the data format and attribute requirements of national building and municipal facility investigation building database building standard, fusing and constructing a new building database, and synchronously updating the graph and the data to the national platform database.
In an alternative embodiment, the step of obtaining field survey data for a field survey includes:
the method comprises the steps of collecting data through a house building offline edition investigation system, based on Java language and SQL database by taking a computer and an Android mobile phone as hardware support, constructing a space database by using an ArcSDE database engine, and calling a space map base map image to input, edit, output, operate and photograph house general investigation data.
In an optional embodiment, the offline version investigation system is designed in a three-layer architecture mode and comprises an interface representation layer, a logic processing layer and a data operation layer, wherein the interface representation layer and the logic processing layer use interface clicking to perform data communication, when a user clicks an interface, the process from data processing to visualization can be completed through the interface clicking, and the data operation layer is used for packaging, reading and operating data so as to classify, manage and operate different data.
In an alternative embodiment, the quality inspection of the field investigation data is carried out through man-machine interaction inspection and software batch quality inspection respectively;
the man-machine interaction inspection comprises the steps of manually checking data through a house building interactive inspection system based on ArcGIS Add-In, wherein the house building interactive inspection system relies on ArcGIS as software, and a house census data quality inspection platform is formed by combining an SQL database by applying VB language, so that the house census data and photos can be loaded, visualized, edited, marked, modified and saved;
the software batch quality inspection comprises the steps of checking data software through a housing construction data quality control system based on Python, calling national platform data to a housing construction interactive inspection system through a database and a data interface exchange channel, and performing standard compliance inspection on the data according to established SQL rules, dictionary rules, regular rules, threshold rules, function dependent rules, inclusion dependent rules and standard rules.
In an alternative embodiment, after finishing quality inspection of the field investigation data, the inspected and modified data are updated to a national platform vector database according to the national standard to obtain investigation result data.
In a second aspect, the present invention provides a method for constructing a regional natural disaster index system and a natural disaster risk assessment model, according to investigation result data and other natural disaster data obtained by the integrated processing method for comprehensive risk census data of natural disasters in any one of the foregoing embodiments.
According to the method, an integrated flow of data processing is standardized by a method of fusing multi-source heterogeneous basic data, after field investigation and verification of an offline version investigation system are adopted, a quality control mechanism of man-machine interaction type inspection and program batch inspection is adopted, a one-key synchronous updating risk census platform of house building data is realized, and an area natural disaster index system and a natural disaster risk assessment model are established by combining investigation result data and other disaster data.
Additional features and advantages of the present application will be set forth in the detailed description which follows.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It will be apparent that the embodiments described are some, but not all, of the embodiments of the present application. The components of the embodiments of the present application, which are generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations.
In the description of the present application, it should be noted that, the azimuth or positional relationship indicated by the terms "inner", "outer", etc. are based on the azimuth or positional relationship shown in the drawings, or the azimuth or positional relationship that is commonly put when the product of the application is used, are merely for convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element to be referred to must have a specific azimuth, be configured and operated in a specific azimuth, and therefore should not be construed as limiting the present application. Furthermore, the terms "first," "second," and the like, are used merely to distinguish between descriptions and should not be construed as indicating or implying relative importance.
In the description of the present application, it should also be noted that, unless explicitly specified and limited otherwise, the terms "disposed," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the terms in this application will be understood by those of ordinary skill in the art in a specific context.
Referring to fig. 1, the integrated processing method for natural disaster comprehensive risk census data in the application performs house building site information acquisition through a house building offline version investigation system, performs house building achievement interactive inspection and batch inspection to form an effective integrated data processing mode, is applied to actual production of related investigation projects such as first national natural disaster census, realizes multi-source heterogeneous data fusion processing and synchronous updating after site inspection data quality inspection through data linkage updating, and finally realizes establishment of a regional natural disaster index system and a natural disaster risk assessment model, and solves the problem that the requirements of actual application requirements on normative basic data sources and site data are higher and higher to a certain extent.
The invention relates to a method for integrally processing natural disaster comprehensive risk census data and application thereof, which comprises the following steps: acquiring multi-source heterogeneous data in different formats of different industry departments, and constructing a standardized vector database; acquiring a national platform vector database for natural disaster comprehensive risk screening, and constructing a mutual mapping relation between a standardized vector database and the national platform vector database; the collection of on-site investigation data is realized through an autonomous development Java-based house construction off-line version investigation system, and after the on-site investigation data is subjected to quality inspection, the on-site investigation data is subjected to linkage update with a national platform vector database, so that investigation result data is obtained. And organically combining the investigation result data with a fuzzy comprehensive evaluation model and a natural disaster risk calculation model by using an analytic hierarchy process to perform county natural disaster comprehensive risk assessment.
The integrated data processing method can be used for solving the technical bottlenecks of App katon, irregular fusion processing of multi-source heterogeneous basic data, single data quality inspection rule, no linkage updating mechanism, imperfect natural disaster risk assessment model and the like in the existing natural disaster comprehensive risk screening online investigation, and an effective set of natural disaster comprehensive risk screening integrated data processing mode is formed by adopting a multi-source heterogeneous data standardized processing technology, a data linkage updating device, a custom quality inspection rule technology and a verified natural disaster risk assessment model.
Referring to fig. 2, the invention provides a multi-source heterogeneous data standardization processing method based on a preset analysis rule, and establishes a set of data sharing exchange engine. The step of obtaining multi-source heterogeneous data and constructing a standardized vector database comprises the following steps:
classifying the acquired multi-source heterogeneous data into spatial data and non-spatial data, constructing a multi-source heterogeneous data directory tree, establishing a data cleaning rule and model aiming at the application requirement of natural disaster census data, cleaning the data, developing multi-type components such as data processing, cleaning and converting by adopting 4 modes of address matching, data association, entity fusion and position conversion, constructing a multi-mode componentization data automatic conversion technology to obtain correct data, and constructing a standardized database according to national house building and municipal facility investigation house building database building standard.
Considering the conditions of inconsistent data formats and data standards in different industries, the invention provides a multi-source heterogeneous data fusion technical route covering the whole process of source data acquisition, data quality evaluation, data cleaning and data conversion. The multi-source heterogeneous data sharing exchange engine is established, and the sharing convergence capacity of multi-source heterogeneous data such as files, databases and service interfaces is provided, so that the online convergence, automatic analysis and warehousing, standardized data processing, online editing and auditing, resource release and the like of the data are realized.
The data demand party directly acquires the required data from the data provider in real time, and does not acquire the required data from the traditional data set any more, and the data demand response, the demand service and the demand acquisition are realized by taking the API data interface as a carrier. The shared exchange engine supports a main stream database such as Oracle, mySQL, SQLServer, postgreSQL; support Webservice, http, rest and other types of service interfaces; supporting various types of space data services such as OGC, arcGIS and the like; various forms, documents, pictures and other file formats need to be supported.
The construction of the mutual mapping relation between the standardized vector database and the national platform vector database comprises the following steps: and integrating the national platform vector database and the standardized vector database according to the data format and attribute requirements of national building and municipal facility investigation building database building standard, fusing and constructing a new building database, and synchronously updating the graph and the data to the national platform database.
In the field investigation process, the Java-based house construction offline version investigation system realizes rapid data acquisition, specifically, the offline version investigation system relies on a computer and an Android mobile phone as hardware, is based on Java language and is combined with an SQL database, a space database is built by using an ArcSDE database engine, and a space map base image is called to form a multifunctional house general investigation offline version mobile data acquisition platform, so that functions of inputting, editing, outputting, map operation, photographing, visualizing and the like of house general investigation data are realized, compared with the problems that the instantaneous access amount of a national platform online version App to a server is large and software is blocked during data acquisition, the system design applies for API service call quota of government agency level, the daily quota of the image base image, the vector base image, the image annotation, the vector annotation and the like reaches 500 ten thousand, and the stability of the investigation system is ensured. The off-line investigation system realizes house general investigation field property information investigation and photo collection work, and carries out data one-key updating on collected data to a national platform system according to national standards, thereby greatly improving the efficiency of house general investigation data collection.
The offline version investigation system adopts a three-layer architecture mode for design: (1) an interface representation layer, (2) a logic processing layer and (3) a data operation layer. And carrying out data communication between the interface and the logic layer by using interface clicking. When the user clicks the interface, the process from data processing to visualization can be completed through the interface clicking. The data layer is mainly responsible for packaging, reading and operating the data, and can effectively classify, manage and operate different data. The three-layer architecture is adopted, so that the system layers are favorable for completing part of specific subfunctions as independently as possible only by taking charge of a single task, so that the modules are characterized by high cohesion and low coupling, and the subsequent maintenance and modification are facilitated.
When the quality inspection is performed on the field investigation data, the data inspection is mainly performed In a software inspection and man-machine interaction inspection mode, and a house building data quality control system based on Python and a house building interactive inspection system based on ArcGIS Add-In are respectively provided. After the updated data is checked on site, besides the quality inspection of the basic quality inspection software issued by the country, the man-machine interaction inspection and the software batch quality inspection are finished according to the standard requirements in proportion.
The man-machine interaction inspection is mainly performed through an autonomously developed house building interactive inspection system based on ArcGIS Add-In (plug-In), the software is based on the principles of practicability and expandability, the ArcGIS is used as software, VB language is applied, and an SQL database is combined, so that a multifunctional house screening data quality inspection platform is created, functions of loading, visualizing, editing, marking, modifying, storing and the like of house screening data and photos are realized, and compared with the mode that quality inspection can only be performed at the web end of a national platform during conventional house screening, the work efficiency and the work quality of inspectors are greatly improved.
Software batch quality inspection is mainly carried out by carrying out data software inspection through an independently developed Python-based house building data quality control system, transferring national platform data to an interactive inspection system through a database and a data interface exchange channel, and carrying out standard compliance inspection on the data according to formulated SQL rules, dictionary rules, regular rules, threshold rules, function dependency rules, data quality evaluation rules including dependency rules, standard rules and the like.
After the man-machine interaction inspection and the software batch quality inspection are completed, the inspected and modified data are updated to a national platform vector database according to the national standard to obtain investigation result data, and the data integrated operation mode of 'one-time acquisition, multistage linkage and synchronous update' of house building disaster bearing body investigation is realized.
The invention also provides a construction method of the regional natural disaster index system and the natural disaster risk assessment model, which is carried out according to investigation result data and other natural disaster data obtained by the integrated processing method of the natural disaster comprehensive risk census data.
The natural disaster index system and the natural disaster risk assessment model adopt GIS space analysis and disaster risk assessment models to analyze and divide natural disasters on the county scale. The technical route for natural disaster comprehensive risk assessment in the application is formed and mainly comprises the following steps: (1) data collection and arrangement; (2) constructing an index system and determining weight; (3) index calculation and standardization treatment; (4) and constructing a natural disaster risk assessment model.
Aiming at the topographical features of county, combining the factors such as the vulnerability level, the sensitivity level and the risk level, integrating and fragment processing the pattern spots on the basis of the basic unit patterns of the natural disasters of county, and respectively endowing the pattern spots with the vulnerability level (V D ) Weight 0.3, sensitivity level (S D ) Weight 0.3, risk rating (P D ) Weight 0.4, obtaining a model calculation formula:
W=V D ×0.3+P D ×0.4+S D ×0.3;
wherein:
in the above formula: s is S D To sensitivity, k i Is the sensitive weight of the ith topography and topography, s i Is the area of the ith topography, s j Is the area of the area division. The sensitivity values of different landform areas are obtained through calculation, and the natural environment sensitivity is classified into five grades according to the arrangement of scores from high to low: very low sensitivity region, medium sensitivity region, high sensitivity region, and very high sensitivity region.
V D =P×0.6+Q×0.4;
In the above formula: p is the density grade value of the mouth, and Q is the economic density grade value.
P D =X/Y;
In the above formula: x is the total number of natural disasters counted in the gold altar area for about 30 years, and Y is the number of years participating in counting.
And finally, comparing and verifying the spatial distribution of the natural disasters in the county and the distribution result of the disaster risks by adopting a correlation analysis method, and completing disaster risk division work of the county by calculating the natural disaster risk degree value and each factor weight value.
In addition, the invention also provides a house construction investigation result management and application GIS system, which relies on an on-line geographic information data service of an on-line map, adopts a B/S architecture mode, is developed based on Web language, is assisted with RIA (rich client) technology, AJAX (dynamic webpage) technology and HTML5 technology, is integrated with house construction thematic information, integrates and distributes the space information and the house construction thematic information in a form of a house construction electronic map service website by utilizing map service technologies such as platform geographic information sharing and the like, and provides a house construction thematic information service integrated and flexibly interacted by vector map, image map and house construction thematic data for the public. The system is mainly divided into two modules, namely a front-end map website for information display, browsing and editing and a background information maintenance module for information updating and management. Based on the management system, the house general investigation result data of each county can be rapidly and intuitively searched through a graphic interaction mode and analyzed and utilized.
The following description will explain the technical solution of the present invention by way of specific examples:
referring to fig. 3, a basic workflow of the data integrated processing method and device is described by taking a first natural disaster comprehensive risk census house building disaster-bearing body investigation project of the tin-free city Jiangyin city as an example.
1) The example projects collect multi-source heterogeneous basic data such as third full national land investigation data, high-resolution remote sensing images, rural area and house integrated evidence data, large-scale topographic map data, building completion drawings and construction drawings, rural dangerous old house reconstruction data, public security three standard one real data, cultural relic protection unit directory, campus security engineering directory and the like, and the standardization of the multi-source heterogeneous data is realized by adopting the technologies such as standing account data vectorization processing, vector data layering extraction, remote sensing image data change detection, database standardization processing and the like.
2) Integrating the national platform database and the database processed in the step 1) according to the data format and attribute requirements of national building and municipal facility investigation building database building standard, fusing and constructing a new building database, and synchronously updating the graph and the data to the national platform database by a platform device.
3) And constructing a house building database by fusing the collected basic data, adopting a plurality of data processing methods such as remote sensing image change detection, big data mining technology, data linkage updating technology, standing account data vectorization operation and the like to manufacture a working base map, and synchronously importing the working base map into an offline edition investigation system.
4) The data collector performs on-site investigation work such as data collection, data editing, data checking, data storage, on-site photographing and the like through an off-line investigation system. The off-line investigation system is developed based on Java language, combines an SQL (structured query language) database, builds a space database by using an ArcSDE (space data) database engine, builds a multifunctional house general investigation off-line mobile terminal data acquisition platform, introduces basic base map data for investigation of a first national natural disaster comprehensive risk general investigation house disaster-bearing body, and realizes functions of input, editing, output, map operation, photographing, visualization and the like of house general investigation data by calling a daily map base map image and superposing the imported vector graph and basic attribute information.
5) The data acquired on site are synchronized to a national platform through a platform device, a data quality inspector firstly uses basic quality inspection software of the national platform to conduct quality inspection, and the data which is inspected by the quality inspection software of the national platform and modified enters batch quality inspection and man-machine interaction software inspection of self-research software through the platform device. The self-developed batched quality inspection software is a house building data quality control system developed based on Python language, and a data quality inspector can self-define house quality inspection rule tables of three layers in the area through SQL sentence rules to realize batched inspection of house building disaster-bearing body data. The man-machine interaction software is mainly used for finishing checking of house building attribute data, graphic data and site photos through a developed house building interaction checking system based on ArcGIS Add-In (plug-In), the interaction software is based on ArcGIS Add-In (plug-In), house building disaster-bearing body investigation result data and photos In the step 4) are imported into an ArcGIS platform, and a data quality inspector is used for finishing correctness checking of the attribute information and the site photos. After the software batch quality inspection and man-machine interaction software inspection and modification are completed, the result data is synchronously updated to the national platform system again through the platform device.
6) The method comprises the steps that a data analyzer acquires disaster-bearing body data of a building from a platform device, combines natural geographic elements such as topography, climate and the like, social manmade elements such as population, economy and the like, and natural disaster statistics data of the past year and the like, takes disaster-bearing bodies, disaster-tolerant environments and disaster-causing factors as system selection indexes, adopts a basic unit division method to construct a natural disaster index system of a river-yin city, adopts a weight assignment method to organically combine a fuzzy comprehensive evaluation model of an improved analytic hierarchy process with a natural disaster risk evaluation model, and completes construction work of the natural disaster evaluation model on a county scale by calculating a natural disaster risk value and each factor weight value.
7) Users with different roles complete the data integrated processing through the platform device, and the synchronous update mechanisms of house building attribute update data, house building graph update data, field shot photo update data and the like on a national platform are realized by combining the vector data linkage update technology independently developed by the unit, so that the organic combination of the platform data and a natural disaster risk assessment model is synchronously realized, and finally, the data integrated operation mode of house building disaster-bearing body investigation of one-time acquisition, multistage linkage and synchronous update is realized.
In the invention, the principles of real-time acquisition and data processing linkage updating in field photographing are adopted, the national platform base map data is taken as basic data, the whole collected multi-source heterogeneous basic data resource is taken as the basis of field investigation of an offline edition investigation system, the data quality control mechanism is optimized, and the integrated data operation flow is perfected. Meanwhile, authoritative data of different industry departments are integrated through the integrated data processing method and the platform device, data fusion work of collecting data and site information is completed rapidly and accurately, working efficiency of investigation of disaster-bearing bodies of general investigation buildings of houses is improved, and construction of an regional natural disaster index system and a risk assessment model is finally achieved.
Compared with the technical bottlenecks such as App katon, irregular fusion processing of multi-source heterogeneous basic data, single data quality inspection rule, no linkage updating mechanism, imperfect natural disaster risk assessment model and the like in the conventional natural disaster comprehensive risk screening online investigation, an effective natural disaster comprehensive risk screening integrated data processing mode is formed by adopting a multi-source heterogeneous data standardized processing technology, a data linkage updating device, a custom quality inspection rule technology and a verified natural disaster risk assessment model, and the method has higher popularization value and application range, avoids repeated acquisition and saves labor and fund cost.
The provided software such as the investigation system and the interactive inspection system of the android version provides a borrowable integrated processing scheme for investigation of the disaster-bearing body of the building construction for first national natural disaster comprehensive risk census, and the completeness of the scheme is verified through production practice, so that the investigation period is shortened, and the investigation quality is improved. The method has the advantages that a set of method for constructing a natural disaster index system by adopting basic unit division is innovatively provided, and a natural disaster risk assessment model is established on the county scale, so that remarkable social benefit and economic benefit are generated.
It is important to point out that the multi-source heterogeneous data processing method of the building construction is based on key technologies such as preset analysis rules and standardized processing, and realizes the standardized processing of the multi-source heterogeneous data through technical means such as standing book data vectorization, remote sensing image data change detection, spatial data matching and attribute data association.
Identifying the changed house building by adopting a remote sensing image automatic interpretation method; the house building offline version investigation system based on Java is developed to collect data; house building interactive checking system based on ArcGIS Add-In and house building data quality control system based on Python are developed to control quality; and finishing the integrated processing of the data through the linkage updating device.
And combining natural geographic elements such as topography and topography, climate and the like, social human elements such as population, economy and the like and natural disaster statistics data of past years, selecting indexes from a plurality of subsystems such as disaster factors, disaster bearing bodies, disaster pregnancy environments and the like, constructing a natural disaster index system by adopting a basic unit division method, and performing space analysis on county and domain scales to complete the establishment of a natural disaster risk assessment model.
It should be noted that, without conflict, features in the embodiments of the present application may be combined with each other.
The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the same, but rather, various modifications and variations may be made by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.