CN113792098B - Big data visualization method, system and medium based on database SQL (structured query language) imaging - Google Patents
Big data visualization method, system and medium based on database SQL (structured query language) imaging Download PDFInfo
- Publication number
- CN113792098B CN113792098B CN202110883103.7A CN202110883103A CN113792098B CN 113792098 B CN113792098 B CN 113792098B CN 202110883103 A CN202110883103 A CN 202110883103A CN 113792098 B CN113792098 B CN 113792098B
- Authority
- CN
- China
- Prior art keywords
- data
- big data
- screening
- database
- sql
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/26—Visual data mining; Browsing structured data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
- G06F16/2448—Query languages for particular applications; for extensibility, e.g. user defined types
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Remote Sensing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a large data visualization method, a large data visualization system and a large data visualization medium based on database SQL (structured query language) imaging, which are used for acquiring at least one type of planning large data; setting screening rules of planning big data, screening the planning big data through the screening rules, and obtaining a table structure database after the planning big data are sorted through a data table structure; the setting of the screening rules of the planning big data comprises the following steps: setting at least one coordinate position and at least one screening condition; setting screening conditions through a graphical interactive interface, and compiling by combining the screening conditions to form a complete SQL sentence; the table structure database executes SQL sentences to inquire so as to obtain corresponding summarized data; and after the summarized data are subjected to coordinate transformation and visualization, different graphical visualized data are obtained. According to the method and the device, sentences of the database are converted into graphic expressions according to common business logic, so that the threshold of big data analysis is greatly reduced, and the analysis result is quickly converted into the required graphic expressions by combining a visual platform.
Description
Technical Field
The application belongs to the technical field of databases, and particularly relates to a large data visualization method, a large data visualization system and a large data visualization medium based on database SQL (structured query language) imaging.
Background
With the rise of big data in recent years, planning big data is more visual and more accurate to reflect the real situation of the city due to the convenience, and becomes an important analysis dimension for planners in analyzing city problems and compiling planning schemes. The planning big data mainly comprises LBS or mobile phone signaling data reflecting population or crowd characteristics, enterprise data provided by enterprise information service providers, map data such as POIs provided by map service providers and the like.
When a planner analyzes the big data, the planner needs to master the visualization technologies such as GIS or SQL, python and the like, so a certain threshold is formed. Moreover, most of the existing technical services surround the visual display of fixed data, the analysis function is insufficient, and a full-flow line product from big data to visual pictures, which meets the low threshold of planners, is not formed yet.
The structured query language (Structured Query Language) is abbreviated as SQL, is a special purpose programming language, and is a database query and programming language for accessing data and querying, updating and managing relational database systems. The location-based service LBS (Location Based Services) is to obtain the current location of the positioning device by using various positioning technologies, and provide information resources and basic services for the positioning device through the mobile internet. POI is an abbreviation of "Point of Interest", chinese can be translated into "points of interest", and in a geographic information system, a POI can be a house, a business, a mailbox, a bus stop, etc.
Disclosure of Invention
According to the large data visualization method, system and medium based on the database SQL graphics, provided by the invention, the sentences of the database are converted into the graphic expressions according to the common business logic, so that the threshold of large data analysis is greatly reduced, and the analysis result is quickly converted into the required graphic expressions by combining with a visualization platform.
According to a first aspect of an embodiment of the present application, there is provided a big data visualization method based on database SQL imaging, including the steps of:
acquiring at least one type of planning big data, wherein the planning big data comprises a user geographic position, user characteristics and user behaviors;
setting screening rules of planning big data, screening at least one type of planning big data through the screening rules, and obtaining a table structure database after data table structure arrangement; the setting of the screening rules of the planning big data comprises the following steps: setting at least one coordinate position and at least one screening condition;
setting screening conditions through a graphical interactive interface, and compiling by combining the screening conditions to form a complete SQL sentence;
the table structure database executes SQL sentences to inquire so as to obtain corresponding summarized data;
and after the summarized data are subjected to coordinate transformation and visualization, different graphical visualized data are obtained.
In some implementations of the application, the planning big data includes location-based service data, cell phone signaling data, enterprise data, and/or map data.
In some embodiments of the present application, the table structure database includes a data table of one coordinate location and a data table of two coordinate locations; the screening conditions included: the conditions are filtered by different field attributes defined by the user's geographic location, user characteristics, and/or user behavior.
In some embodiments of the present application, a filtering condition is set through a graphical interactive interface, and a complete SQL statement is formed by compiling in combination with the filtering condition, which specifically includes:
setting screening conditions through a graphical interactive interface by a user;
compiling a select statement and a group by statement according to the coordinates of the screening conditions; compiling a sphere statement according to the attribute field of the screening rule;
the complete SQL statement is integrated from the where statement, the select statement, and the group by statement.
In some embodiments of the present application, the table structure database queries according to SQL statements to obtain corresponding summarized data, which specifically includes:
and obtaining corresponding summarized data in one coordinate position and/or corresponding summarized data in two coordinate positions.
In some embodiments of the present application, after coordinate transformation and visualization of the summarized data, different graphical visualized data are obtained, which specifically includes:
selecting different coordinates of the summarized data according to different map service provider maps through coordinate conversion to obtain summarized data under different map coordinates;
and importing the summarized data under different map coordinates into a visualization tool to obtain different graphical visualized data.
In some embodiments of the present application, after obtaining different graphical visual data, the method further includes data filtering and/or data summarizing, so as to obtain the graphical visual data after data filtering and/or data summarizing;
wherein, the data filtering includes: the geographical position coordinate values and the summarized values of the summarized data are screened, or the geographical position coordinate values of the summarized data are selected according to the geographical space through a geographical space analysis library to obtain new geographical position coordinate values for screening;
wherein, the data summarization includes: and carrying out various kinds of summarization calculation on the geographic position coordinate values and the summarization values of the summarization data, or carrying out corresponding geographic space filtering on the geographic position coordinate values of the summarization data through a geographic space analysis library to obtain new coordinate values for screening.
According to a second aspect of the embodiments of the present application, a big data visualization system based on database SQL imaging specifically includes:
big data acquisition module: the method comprises the steps of acquiring planning big data of at least one type, wherein the planning big data comprise user geographic positions, user characteristics and user behaviors;
a table structure database module: the method comprises the steps of setting screening rules of planning big data, screening at least one type of planning big data through the screening rules, and obtaining a table structure database after data table structure arrangement; the setting of the screening rules of the planning big data comprises the following steps: setting at least one coordinate position and at least one screening condition;
SQL statement compiling module: the method comprises the steps of setting screening conditions through a graphical interactive interface, and compiling by combining the screening conditions to form a complete SQL sentence;
database query module: the method comprises the steps of executing SQL sentences for inquiring by a table structure database to obtain corresponding summarized data;
big data visualization module: and the method is used for obtaining different graphical visual data after the summarized data are subjected to coordinate transformation and visualization.
According to a third aspect of the embodiments of the present application, there is provided a big data visualization device based on database SQL imaging, including:
A memory: for storing executable instructions; and
and the processor is used for being connected with the memory to execute executable instructions so as to complete the large data visualization method based on the database SQL imaging.
According to a fourth aspect of embodiments of the present application, there is provided a computer-readable storage medium having a computer program stored thereon; the computer program is executed by the processor to implement a database SQL graphical based big data visualization method.
By adopting the large data visualization method, system and medium based on the database SQL imaging, at least one type of planning large data is obtained, wherein the planning large data comprises user geographic positions, user characteristics and user behaviors; setting screening rules of planning big data, screening at least one type of planning big data through the screening rules, and obtaining a table structure database after data table structure arrangement; the setting of the screening rules of the planning big data comprises the following steps: setting at least one coordinate position and at least one screening condition; setting screening conditions through a graphical interactive interface, and compiling by combining the screening conditions to form a complete SQL sentence; the table structure database executes SQL sentences to inquire so as to obtain corresponding summarized data; and after the summarized data are subjected to coordinate transformation and visualization, different graphical visualized data are obtained.
According to the method and the device, sentences of the database are converted into graphic expressions according to common business logic, so that the threshold of big data analysis is greatly reduced, and the analysis result is quickly converted into the required graphic expressions by combining a visual platform. The method solves the problem of flow lines from large data SQL use to visual display, constructs data transmission standards and conversion algorithms for opening the whole flow line, and provides a convenient, visual and efficient large data analysis platform solution for planners or crowds with large data analysis requirements.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. In the drawings:
a schematic step diagram of a big data visualization method based on database SQL imaging according to an embodiment of the present application is shown in fig. 1;
a platform development and user usage logic flow diagram for a database SQL based graphical big data visualization method according to the present application is shown in fig. 2;
a schematic structural diagram of a big data visualization system based on database SQL imaging according to an embodiment of the present application is shown in fig. 3;
A schematic structural diagram of a big data visualization device based on database SQL imaging according to an embodiment of the present application is shown in fig. 4.
Detailed Description
In the process of realizing the application, the inventor finds that when analyzing planning big data, the visualization technologies such as GIS or SQL and python are required to be mastered, so that a certain threshold is formed. Moreover, most of the existing technical services surround the visual display of fixed data, the analysis function is insufficient, and a full-flow line product from big data to visual pictures, which meets the low threshold of planners, is not formed yet.
Therefore, in order to face the large data analysis demands of the vast planners, the application integrates and forms a technical route and a flow chart of common analysis on the basis of the practical experience of a plurality of large data. The large data can be conveniently calculated and analyzed through UI interaction of the web end, and meanwhile, the analysis result can be conveniently and rapidly imported to a visual platform, and the expression can be adjusted as required to form a corresponding planning analysis drawing.
The technical core of the application is to strengthen the connection with the database on the basis of the current general visual platform. The sentences of the database are converted into graphic expressions according to the common business logic, so that the threshold of big data analysis is greatly reduced. Meanwhile, a customized adjustment space is provided for the visualization by combining with the visualization platform, so that a planner can conveniently and quickly convert an analysis result into a required graphic expression.
In particular, the method comprises the steps of,
acquiring at least one type of planning big data, wherein the planning big data comprises a user geographic position, user characteristics and user behaviors; setting screening rules of planning big data, screening at least one type of planning big data through the screening rules, and obtaining a table structure database after data table structure arrangement; the setting of the screening rules of the planning big data comprises the following steps: setting at least one coordinate position and at least one screening condition; setting screening conditions through a graphical interactive interface, and compiling by combining the screening conditions to form a complete SQL sentence; the table structure database executes SQL sentences to inquire so as to obtain corresponding summarized data; and after the summarized data are subjected to coordinate transformation and visualization, different graphical visualized data are obtained.
According to the method and the device, sentences of the database are converted into graphic expressions according to common business logic, so that the threshold of big data analysis is greatly reduced, and the analysis result is quickly converted into the required graphic expressions by combining a visual platform. The method solves the problem of flow lines from large data SQL use to visual display, constructs data transmission standards and conversion algorithms for opening the whole flow line, and provides a convenient, visual and efficient large data analysis platform solution for planners or crowds with large data analysis requirements.
In order to make the technical solutions and advantages of the embodiments of the present application more apparent, the following detailed description of exemplary embodiments of the present application is given with reference to the accompanying drawings, and it is apparent that the described embodiments are only some of the embodiments of the present application and not exhaustive of all the embodiments. It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other.
Example 1
A schematic step diagram of a big data visualization method based on database SQL imaging according to an embodiment of the present application is shown in fig. 1.
As shown in fig. 1, the big data visualization method based on database SQL imaging in the embodiment of the present application specifically includes the following steps:
according to a first aspect of the embodiments of the present application, a big data visualization method based on database SQL imaging is provided, which specifically includes the following steps:
s101: at least one type of planning big data is acquired, the planning big data comprising a user geographical location, user characteristics and user behavior.
Wherein the planning big data comprises location-based service data, cell phone signaling data, enterprise data, and/or map data.
Specific commonly used planning big data types: the map data comprises LBS or mobile phone signaling data represented by population, crowd and the like, enterprise data of enterprise information service providers and map data of points of interest POIs and the like provided by map service providers.
S102: setting screening rules of planning big data, screening at least one type of planning big data through the screening rules, and obtaining a table structure database after data table structure arrangement; the setting of the screening rules of the planning big data comprises the following steps: at least one coordinate position and at least one screening condition are set.
The table structure database comprises a single-coordinate data table and a double-coordinate data table; the screening conditions included: screening conditions by using different field attributes defined by the geographic position, the user characteristics and/or the user behaviors of the user.
In this embodiment, according to the general type and use idea of the big data, the overall design framework integrates the above data into two types of data structures: a single-coordinate data table mainly comprises a table structure of single coordinates of various different field attributes; the other is a double-coordinate data table, which mainly comprises two position coordinates of various field attributes and a table structure with corresponding relations.
S103: and setting screening conditions through a graphical interactive interface, and compiling by combining the screening conditions to form a complete SQL sentence.
Specifically, firstly, a user sets screening conditions through a graphical interactive interface; secondly, compiling a select statement and a group by statement according to the coordinates of the screening conditions; compiling a sphere statement according to the attribute field of the screening rule; finally, integrating the whole SQL sentence according to the where sentence, the select sentence and the group by sentence.
S104: and the table structure database executes SQL sentences to query so as to obtain corresponding summarized data.
In this embodiment, the method specifically includes: and after the table structure database executes the SQL sentence to query, obtaining the corresponding summarized data in one coordinate position and/or the summarized data in two coordinate positions.
S105: and after the summarized data are subjected to coordinate transformation and visualization, different graphical visualized data are obtained.
Specifically, the method comprises the following steps:
firstly, selecting different coordinates of the summarized data according to different map service provider maps through coordinate conversion to obtain summarized data under different map coordinates;
and then, importing the summarized data under different map coordinates into a visualization tool to obtain different graphical visualized data.
In some embodiments of the present application, after obtaining different graphical visual data, the method further includes data filtering and/or data summarizing, so as to obtain the graphical visual data after data filtering and/or data summarizing;
wherein, the data filtering includes: the geographical position coordinate values and the summarized values of the summarized data are screened, or the geographical position coordinate values of the summarized data are selected according to the geographical space through a geographical space analysis library to obtain new geographical position coordinate values for screening;
Wherein, the data summarization includes: and carrying out various kinds of summarization calculation on the geographic position coordinate values and the summarization values of the summarization data, or carrying out corresponding geographic space filtering on the geographic position coordinate values of the summarization data through a geographic space analysis library to obtain new coordinate values for screening.
Optionally, the application adds a user management function during the interaction process with the user, establishes a user white list and controls the access right. Meanwhile, the running condition of the platform and the service condition of the user are counted according to the user information, SQL sentences running during use, SQL query states and the like, so that the system is convenient to maintain and perfect in function.
A platform development and user usage logic flow diagram for a database SQL based graphical big data visualization method according to the present application is shown in fig. 2.
For further explanation of the technical solution of the present application, as shown in fig. 2, the following logic flow is mainly included.
1) First, a database structure is clarified, and a table structure database is obtained.
Type of common planning big data: the map data comprises LBS or mobile phone signaling data represented by population, crowd and the like, enterprise data of enterprise information service providers and map data of points of interest POIs and the like provided by map service providers.
According to the common type and the use thought of the large data, the whole design framework integrates the data into two types of data structures: a single-coordinate data table mainly comprises a table structure of single coordinates of various different field attributes; the other is a double-coordinate data table, which mainly comprises two coordinates of various different field attributes and a table structure with corresponding relations.
2) Platform based on web end is built
Based on web development, a data analysis platform is built, and the platform comprises a rear-end access database, such as an open-source common database MySQL, postgreSQL. And integrating the imported data according to the definite database structure of the first step, and simultaneously realizing the platform to call the database.
Tools for importing visualizations in conjunction with the front end, such as open source ECharts, L7, etc., and APIs for accessing map service providers. The platform construction based on the web end is integrally realized.
3) Customized interactive design of database
And independently designing a UI interface, realizing the visual operation of SQL sentences, and constructing a screening function interface for different data.
During specific operation, the interactive design step one: the library table corresponding to the data is explicitly analyzed as a field of the from of the SQL statement.
And step two, interactive design: corresponding screening conditions were constructed around the different data.
Based on the three items of 'field', 'condition', 'value', and 'or' as relation, constructing composite screening condition, such as 'age > 18 years' and 'living in Shanghai'.
Because field attributes of different types of data have a certain difference from the screening conditions, specific examples are as follows.
(1) Such as crowd data, can be screened according to different attributes such as data acquisition time, cities in which the crowd is located, crowd precision, crowd characteristics (age, sex, work and the like), the screening conditions for the population data may comprise "greater than", "greater than or equal to", "less than or equal to", "comprising", and compiling the statement into a screening condition of where in SQL (structured query language) according to "> and" < "".ltoreq "" = "" like "in the SQL statement respectively.
For example, "data time=2018", "crowd age. Gtoreq.18 years", "city where crowd is located=Shanghai, no tin", etc.
(2) For example, the enterprise data can be screened according to different attributes such as enterprise registration or operation time, enterprise registration county, enterprise industry type, enterprise registration address or operation address, enterprise operation state and the like, and the screening conditions of the enterprise data can be combined with relations such as greater than, equal to, less than, equal to, containing and the like to compile the window screening conditions in the corresponding SQL statement.
Such as "enterprise registration time is less than or equal to 2020", "enterprise industry type=manufacturing", "enterprise registration address like changning road", etc.
(3) For example, the map POI data can be filtered according to the type of the POI, the city or county where the POI is located and other attributes, and the filtering conditions of the map data can be combined with the relations of greater than, equal to, less than, equal to, containing and the like, and compiled into the wheree filtering conditions in the corresponding SQL sentences.
Such as "POI type = hotel", "POI type like scientific research institution", etc.
And step three, interactive design: after the window screening condition is determined, the data is summarized and calculated by adding the SQL summarizing function, namely the SQL group by statement is added with summarizing fields, and optionally, all the attributes in the library table are listed, and the summarizing condition can be freely customized by a user. The common use of the method is single-coordinate data, which can be summarized according to two values of single coordinates, and double-coordinate data, which is suggested to be summarized according to four values of double coordinates.
And step four, interactive design: the select statement in SQL is composed of two values in single coordinates or four values in double coordinates, and further adds fields for summary calculation, such as sum () function for summary calculation.
The four steps form a complete SQL sentence together, and after completion, execution commands can be issued to the database.
4) Visual platform for importing results
After executing the SQL sentence, obtaining single coordinates or double coordinates and corresponding summarized data, selecting proper coordinates according to a map service provider map through coordinate conversion, converting a hundred-degree map into hundred-degree coordinates, converting a Goldmap into Mars coordinates, and then importing the data into a visual front end.
The coordinate values of the single coordinate data define the spatial position and the summary value as display data, and can be rendered into different types such as thermodynamic diagrams, bubble diagrams, scatter diagrams or brightness diagrams, and the like, and the coordinate values can be specifically selected according to requirements. The two coordinate values of the dual coordinate data can be associated into a line, and are suitable for rendering into a fly line graph, and the summarized value is taken as an attribute value of the line.
Different graphics can be adjusted according to the summarized values, and the summarized values are rendered into the thickness of points or lines or different colors, so that different graphic expressions are realized according to different business scene needs and personal preferences.
5) Data filtering and data summarizing functions
After the data enter the front-end visualization platform, the data filtering and summarizing functions can be added according to requirements.
Data filtering includes two ways. The first filtering according to the value mainly carries out screening around coordinate values and summary values, for example, the numerical value with the too large or too small summary value is screened, and the numerical value is fed back to the visualization of the graph in real time.
And secondly, filtering according to space, and introducing a geospatial analysis library such as Turf. Js and the like to realize interaction of coordinate values and geospatial selection. Specifically, the area space is manually drawn, the space analysis is carried out on the drawn area space and the coordinate data through the geographic space analysis library, and whether the coordinate data is in the area space is judged, so that the space filtering function is realized. And finally, the filtered data can be fed back to the graphic visualization in real time.
Ii) a data summarizing function, namely performing various calculations on data, such as summary values, such as summarizing and summing to obtain the sum of the data.
And secondly, combining interaction in space, carrying out space analysis on the area space and the coordinate data through a geographic space analysis library, judging whether the coordinate data is in the area space, and realizing space filtering, so that data in the area range is drawn, and various calculations, such as summarizing and summing, can be carried out on the data.
6) Setting up management background according to database
The platform can introduce a user management function, establish a user white list and control access rights. Meanwhile, the operation condition and the use condition of the platform can be counted according to the user information, SQL sentences, SQL query states and the like which are operated during use, and the system maintenance and the function perfection are facilitated.
The big data visualization method based on the database SQL graphics is adopted to obtain at least one type of planning big data, wherein the planning big data comprises a user geographic position, user characteristics and user behaviors; setting screening rules of planning big data, screening at least one type of planning big data through the screening rules, and obtaining a table structure database after data table structure arrangement; the setting of the screening rules of the planning big data comprises the following steps: setting at least one coordinate position and at least one screening condition; setting screening conditions through a graphical interactive interface, and compiling by combining the screening conditions to form a complete SQL sentence; the table structure database executes SQL sentences to inquire so as to obtain corresponding summarized data; and after the summarized data are subjected to coordinate transformation and visualization, different graphical visualized data are obtained.
According to the method and the device, sentences of the database are converted into graphic expressions according to common business logic, so that the threshold of big data analysis is greatly reduced, and the analysis result is quickly converted into the required graphic expressions by combining a visual platform. The method solves the problem of flow lines from large data SQL use to visual display, constructs data transmission standards and conversion algorithms for opening the whole flow line, and provides a convenient, visual and efficient large data analysis platform solution for planners or crowds with large data analysis requirements.
Example 2
The embodiment provides a big data visualization system based on database SQL (structured query language) imaging, and for details which are not disclosed in the big data visualization system based on database SQL imaging in the embodiment, please refer to the specific implementation content of the big data visualization method based on database SQL imaging in other embodiments.
A schematic structural diagram of a big data visualization system based on database SQL imaging according to an embodiment of the present application is shown in fig. 3.
As shown in fig. 3, the big data visualization system based on the database SQL imaging in the embodiment of the present application specifically includes a big data acquisition module 10, a table structure database module 20, an SQL statement compiling module 30, a database query module 40, and a big data visualization module 50.
In particular, the method comprises the steps of,
big data acquisition module 10: for obtaining at least one type of planning big data comprising a user geographical location, user characteristics and user behavior.
Wherein the planning big data comprises location-based service data, cell phone signaling data, enterprise data, and/or map data.
Specific commonly used planning big data types: the map data comprises LBS or mobile phone signaling data represented by population, crowd and the like, enterprise data of enterprise information service providers and map data of points of interest POIs and the like provided by map service providers.
Table structure database module 20: the method comprises the steps of setting screening rules for planning big data, screening at least one type of planning big data through the screening rules, and obtaining a table structure database after data table structure arrangement; the setting of the screening rules of the planning big data comprises the following steps: at least one coordinate position and at least one screening condition are set.
The table structure database comprises a single-coordinate data table and a double-coordinate data table; the screening conditions included: screening conditions by using different field attributes defined by the geographic position, the user characteristics and/or the user behaviors of the user.
In this embodiment, according to the general type and use idea of the big data, the overall design framework integrates the above data into two types of data structures: a single-coordinate data table mainly comprises various different types
A table structure of single coordinates of the field attributes; the other is a double-coordinate data table, which mainly comprises two position coordinates of various field attributes and a table structure with corresponding relations.
SQL statement compilation module 30: the method is used for setting screening conditions through a graphical interactive interface and compiling the screening conditions to form a complete SQL sentence.
Specifically, firstly, a user sets screening conditions through a graphical interactive interface; secondly, compiling a select statement and a group by statement according to the coordinates of the screening conditions; compiling a sphere statement according to the attribute field of the screening rule; finally, integrating the whole SQL sentence according to the where sentence, the select sentence and the group by sentence.
Database query module 40: and executing SQL sentences by the table structure database to query so as to obtain corresponding summarized data.
In this embodiment, the method specifically includes: and after the table structure database executes the SQL sentence to query, obtaining the corresponding summarized data in one coordinate position and/or the summarized data in two coordinate positions.
Big data visualization module 50: and the method is used for obtaining different graphical visual data after the summarized data are subjected to coordinate transformation and visualization.
Specifically, the method comprises the following steps:
firstly, selecting different coordinates of the summarized data according to different map service provider maps through coordinate conversion to obtain summarized data under different map coordinates;
and then, importing the summarized data under different map coordinates into a visualization tool to obtain different graphical visualized data.
In some embodiments of the present application, after obtaining different graphical visual data, the method further includes data filtering and/or data summarizing, so as to obtain the graphical visual data after data filtering and/or data summarizing;
Wherein, the data filtering includes: the geographical position coordinate values and the summarized values of the summarized data are screened, or the geographical position coordinate values of the summarized data are selected according to the geographical space through a geographical space analysis library to obtain new geographical position coordinate values for screening;
wherein, the data summarization includes: and carrying out various kinds of summarization calculation on the geographic position coordinate values and the summarization values of the summarization data, or carrying out corresponding geographic space filtering on the geographic position coordinate values of the summarization data through a geographic space analysis library to obtain new coordinate values for screening.
Optionally, the application adds a user management function during the interaction process with the user, establishes a user white list and controls the access right. Meanwhile, the running condition of the platform and the service condition of the user are counted according to the user information, SQL sentences running during use, SQL query states and the like, so that the system is convenient to maintain and perfect in function.
By adopting the big data visualization system based on the database SQL graphics of the embodiment, the big data acquisition module 10 acquires at least one type of planning big data, wherein the planning big data comprises a user geographic position, user characteristics and user behaviors; the table structure database module 20 sets screening rules of the planning big data, and at least one type of planning big data is screened by the screening rules and is arranged through a data table structure to obtain a table structure database; the SQL statement compiling module 30 sets a screening rule for planning big data, which comprises: setting at least one coordinate position and at least one screening condition; setting screening conditions through a graphical interactive interface, and compiling by combining the screening conditions to form a complete SQL sentence; the database query module 40 performs SQL statement query on the table structure database to obtain corresponding summarized data; the big data visualization module 50 obtains different graphical visualized data after transforming and visualizing the summarized data through coordinates.
According to the method and the device, sentences of the database are converted into graphic expressions according to common business logic, so that the threshold of big data analysis is greatly reduced, and the analysis result is quickly converted into the required graphic expressions by combining a visual platform. The method solves the problem of flow lines from large data SQL use to visual display, constructs data transmission standards and conversion algorithms for opening the whole flow line, and provides a convenient, visual and efficient large data analysis platform solution for planners or crowds with large data analysis requirements.
Example 3
The embodiment provides a big data visualization device based on database SQL (structured query language) imaging, and for details which are not disclosed in the big data visualization device based on database SQL imaging in the embodiment, please refer to specific implementation contents of the big data visualization method or system based on database SQL imaging in other embodiments.
A schematic structural diagram of a big data visualization device 400 according to an embodiment of the present application is shown in fig. 4.
As shown in fig. 4, the big data visualization apparatus 400 includes:
memory 402: for storing executable instructions; and
It will be appreciated by those skilled in the art that the schematic diagram 4 is merely an example of the big data visualization device 400 and does not constitute a limitation of the big data visualization device 400, and may include more or less components than illustrated, or may combine certain components, or different components, e.g. the big data visualization device 400 may further include input and output devices, network access devices, buses, etc.
The processor 401 (Central Processing Unit, CPU) may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. The general purpose processor may be a microprocessor or the processor 401 may be any conventional processor or the like, and the processor 401 is a control center of the big data visualization device 400, and various interfaces and lines are used to connect the various parts of the whole big data visualization device 400.
The memory 402 may be used to store computer readable instructions, and the processor 401 may implement various functions of the big data visualization device 400 by executing or executing the computer readable instructions or modules stored in the memory 402 and invoking data stored in the memory 402. The memory 402 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data created according to the use of the big data visualization apparatus 400, or the like. In addition, the Memory 402 may include a hard disk, memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card), at least one disk storage device, a Flash Memory device, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), or other non-volatile/volatile storage device.
The modules integrated with big data visualization device 400 may be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as a stand alone product. Based on such understanding, the present invention may implement all or part of the flow of the method of the above-described embodiments, or may be implemented by means of computer readable instructions to instruct related hardware, where the computer readable instructions may be stored in a computer readable storage medium, where the computer readable instructions, when executed by a processor, implement the steps of the method embodiments described above.
Example 4
The present embodiment provides a computer-readable storage medium having a computer program stored thereon; the computer program is executed by the processor to implement the database SQL graphical based big data visualization method in other embodiments.
According to the large data visualization equipment and the storage medium based on the database SQL imaging, at least one type of planning large data is obtained, and the planning large data comprises a user geographic position, user characteristics and user behaviors; setting screening rules of planning big data, screening at least one type of planning big data through the screening rules, and obtaining a table structure database after data table structure arrangement; the setting of the screening rules of the planning big data comprises the following steps: setting at least one coordinate position and at least one screening condition; setting screening conditions through a graphical interactive interface, and compiling by combining the screening conditions to form a complete SQL sentence; the table structure database executes SQL sentences to inquire so as to obtain corresponding summarized data; and after the summarized data are subjected to coordinate transformation and visualization, different graphical visualized data are obtained.
According to the method and the device, sentences of the database are converted into graphic expressions according to common business logic, so that the threshold of big data analysis is greatly reduced, and the analysis result is quickly converted into the required graphic expressions by combining a visual platform. The method solves the problem of flow lines from large data SQL use to visual display, constructs data transmission standards and conversion algorithms for opening the whole flow line, and provides a convenient, visual and efficient large data analysis platform solution for planners or crowds with large data analysis requirements.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the invention. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.
While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various modifications and variations can be made in the present application without departing from the spirit or scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims and the equivalents thereof, the present application is intended to cover such modifications and variations.
Claims (9)
1. The large data visualization method based on the database SQL imaging is characterized by comprising the following steps of:
acquiring at least one type of planning big data, wherein the planning big data comprises a user geographic position, user characteristics and user behaviors;
setting a screening rule of planning big data, wherein at least one type of planning big data is screened by the screening rule and is sorted by a data table structure to obtain a table structure database, and the table structure database comprises a data table of one coordinate position and a data table of two coordinate positions; the setting of the screening rules of the planning big data comprises the following steps: setting at least one coordinate position and at least one screening condition;
setting screening conditions through a graphical interactive interface by a user; compiling a select statement and a group by statement according to the coordinates of the screening conditions; compiling a sphere statement according to the attribute field of the screening rule; integrating the window statement, the select statement and the group by statement into a complete SQL statement;
the table structure database executes the SQL sentence to query so as to obtain corresponding summarized data;
and after the summarized data are subjected to coordinate transformation and visualization, different graphical visualized data are obtained.
2. The big data visualization method of claim 1, wherein the planning big data comprises location-based service data, cell phone signaling data, enterprise data, and/or map data.
3. The big data visualization method of claim 1, wherein the screening conditions include: screening conditions by using different field attributes defined by the geographic position, the user characteristics and/or the user behaviors of the user.
4. The big data visualization method according to claim 1, wherein the table structure database queries according to the SQL statement to obtain corresponding summarized data, and specifically comprises:
and obtaining corresponding summarized data in one coordinate position and/or corresponding summarized data in two coordinate positions.
5. The big data visualization method according to claim 4, wherein the obtaining different graphical visualized data after the summarized data is transformed and visualized by coordinates specifically includes:
selecting different coordinates of the summarized data according to different map service provider maps through coordinate conversion to obtain summarized data under different map coordinates;
and importing the summarized data under different map coordinates into a visualization tool to obtain different graphical visualization data.
6. The big data visualization method according to claim 1, wherein after the different graphical visualized data are obtained, further comprising data filtering and/or data summarization to obtain the graphical visualized data after the data filtering and/or data summarization;
wherein, the data filtering includes: the geographical position coordinate values and the summarized values of the summarized data are screened, or the geographical position coordinate values of the summarized data are selected according to the geographical space through a geographical space analysis library to obtain new geographical position coordinate values for screening;
wherein, the data summarization includes: and carrying out various kinds of summarization calculation on the geographic position coordinate values and the summarization values of the summarization data, or carrying out corresponding geographic space filtering on the geographic position coordinate values of the summarization data through a geographic space analysis library to obtain new coordinate values for screening.
7. The big data visualization system based on the database SQL imaging is characterized by comprising the following specific components:
big data acquisition module: the method comprises the steps of acquiring at least one type of planning big data, wherein the planning big data comprise user geographic positions, user characteristics and user behaviors;
a table structure database module: the method comprises the steps of setting a screening rule of planning big data, screening at least one type of planning big data through the screening rule, and obtaining a table structure database after finishing through a data table structure, wherein the table structure database comprises a data table of one coordinate position and a data table of two coordinate positions; the setting of the screening rules of the planning big data comprises the following steps: setting at least one coordinate position and at least one screening condition;
SQL statement compiling module: the method comprises the steps that the method is used for setting screening conditions through a graphical interactive interface by a user; compiling a select statement and a group by statement according to the coordinates of the screening conditions; compiling a sphere statement according to the attribute field of the screening rule; integrating the window statement, the select statement and the group by statement into a complete SQL statement;
database query module: the SQL sentence is executed by the table structure database to query, so that corresponding summarized data are obtained;
big data visualization module: and the method is used for obtaining different graphical visual data after the summarized data are subjected to coordinate transformation and visualization.
8. A database SQL patterning-based big data visualization device, comprising:
a memory: for storing executable instructions; and
a processor for interfacing with a memory to execute executable instructions to perform the database SQL based graphical big data visualization method of any of claims 1-6.
9. A computer-readable storage medium, characterized in that a computer program is stored thereon; computer program to be executed by a processor to implement a database SQL based graphical big data visualization method according to any of the claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110883103.7A CN113792098B (en) | 2021-08-02 | 2021-08-02 | Big data visualization method, system and medium based on database SQL (structured query language) imaging |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110883103.7A CN113792098B (en) | 2021-08-02 | 2021-08-02 | Big data visualization method, system and medium based on database SQL (structured query language) imaging |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113792098A CN113792098A (en) | 2021-12-14 |
| CN113792098B true CN113792098B (en) | 2023-06-20 |
Family
ID=79181295
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110883103.7A Active CN113792098B (en) | 2021-08-02 | 2021-08-02 | Big data visualization method, system and medium based on database SQL (structured query language) imaging |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113792098B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104915341A (en) * | 2014-03-10 | 2015-09-16 | 中国科学院沈阳自动化研究所 | Visual multi-database ETL integration method and system |
| CN108376176A (en) * | 2018-03-14 | 2018-08-07 | 深圳日彤大数据有限公司 | It can towed big data visualization analysis tools system |
| CN109271428A (en) * | 2018-09-11 | 2019-01-25 | 北京市计算中心 | Data pick-up method and method for exhibiting data based on geography information |
| CN111914135A (en) * | 2020-07-24 | 2020-11-10 | 平安证券股份有限公司 | Data query method and device, electronic equipment and storage medium |
| CN112559576A (en) * | 2019-09-26 | 2021-03-26 | 北京国双科技有限公司 | Data display method, system, device, storage medium and electronic equipment |
| CN113076336A (en) * | 2021-04-27 | 2021-07-06 | 刘文平 | GIS macro-micro decision support system for site selection of water plant in remote area |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110264461A1 (en) * | 1998-11-13 | 2011-10-27 | Anuthep Benja-Athon | Brains-server synapses-empowered networking |
-
2021
- 2021-08-02 CN CN202110883103.7A patent/CN113792098B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104915341A (en) * | 2014-03-10 | 2015-09-16 | 中国科学院沈阳自动化研究所 | Visual multi-database ETL integration method and system |
| CN108376176A (en) * | 2018-03-14 | 2018-08-07 | 深圳日彤大数据有限公司 | It can towed big data visualization analysis tools system |
| CN109271428A (en) * | 2018-09-11 | 2019-01-25 | 北京市计算中心 | Data pick-up method and method for exhibiting data based on geography information |
| CN112559576A (en) * | 2019-09-26 | 2021-03-26 | 北京国双科技有限公司 | Data display method, system, device, storage medium and electronic equipment |
| CN111914135A (en) * | 2020-07-24 | 2020-11-10 | 平安证券股份有限公司 | Data query method and device, electronic equipment and storage medium |
| CN113076336A (en) * | 2021-04-27 | 2021-07-06 | 刘文平 | GIS macro-micro decision support system for site selection of water plant in remote area |
Non-Patent Citations (2)
| Title |
|---|
| G-SQL: support for graph generation;K. Nayyar 等;《Proceedings of the IEEE INDICON 2004. First India Annual Conference, 2004.》;521-524 * |
| 智慧城市大数据可视化云平台的设计与实现;张帅;《中国优秀硕士学位论文全文数据库信息科技辑》(第6期);I138-249 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113792098A (en) | 2021-12-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8688680B2 (en) | System and method for preferred services in nomadic environments | |
| CN112650766B (en) | Database data operation method, system and server | |
| CN110855482A (en) | Three-dimensional dynamic communication network analog simulation method, system and storage medium | |
| CN110580189A (en) | method and device for generating front-end page, computer equipment and storage medium | |
| US20160034272A1 (en) | Managing a catalog of scripts | |
| CN113177058A (en) | Geographic position information retrieval method and system based on composite condition | |
| CN114510611A (en) | Method and device for constructing metadata blood relationship atlas and related equipment | |
| CN110888974A (en) | Data visualization method and device | |
| CN110322029A (en) | Resource recommendation method, device and electronic equipment | |
| CN114356962A (en) | Data query method and device, electronic equipment and storage medium | |
| CN113792098B (en) | Big data visualization method, system and medium based on database SQL (structured query language) imaging | |
| CN114064601B (en) | Storage process conversion method, device, equipment and storage medium | |
| JP6781819B2 (en) | Task processing method and distributed computing framework system | |
| CN113760969A (en) | Data query method and device based on elastic search | |
| EP3690675A1 (en) | Method, apparatus, and storage medium for providing visual representation of set of objects | |
| CN113760242A (en) | Data processing method, device, server and medium | |
| CN110019342A (en) | Partition table access method, device and equipment, computer readable storage medium | |
| CN105574019B (en) | Query parameter processing method and device | |
| CN114020852A (en) | Knowledge graph display method and device | |
| CN114064655A (en) | Configurable data query and automatic discovery method of data relationship | |
| CN113990068A (en) | Traffic data processing method, device, equipment and storage medium | |
| US11222033B2 (en) | Dynamic data retrieval and analytical chart rendering for data sets | |
| CN110909191A (en) | Graph data processing method and device, storage medium and electronic equipment | |
| CN115329029B (en) | Mobile terminal-oriented complex condition geographic information query method, device and medium | |
| CN112463146B (en) | Health risk financial preparation gold output method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |