CN115544010A - Mapping relation determining method and device, electronic equipment and storage medium - Google Patents
Mapping relation determining method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115544010A CN115544010A CN202211229715.5A CN202211229715A CN115544010A CN 115544010 A CN115544010 A CN 115544010A CN 202211229715 A CN202211229715 A CN 202211229715A CN 115544010 A CN115544010 A CN 115544010A
- Authority
- CN
- China
- Prior art keywords
- table structure
- structure information
- current
- determining
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a method and a device for determining a mapping relation, electronic equipment and a storage medium. The method comprises the following steps: acquiring current data to be acquired, and identifying current table structure information of the current data; determining historical table structure information corresponding to the current table structure information in a pre-established knowledge base, and determining the current structure attribute information of the current table structure information based on the corresponding relation between the historical table structure information and the structure attribute information stored in the knowledge base; based on the current structure attribute information, a mapping relationship between the current table structure information and the target table structure information is determined. According to the technical scheme of the embodiment of the invention, the current data can be stored according to the target table structure through the mapping relation, so that the efficiency and the accuracy of determining the mapping relation are improved, the operation is convenient and fast, the time and the labor are saved, and the follow-up maintenance is convenient.
Description
Technical Field
The embodiment of the invention relates to the technical field of data processing, in particular to a mapping relation determining method and device, electronic equipment and a storage medium.
Background
Currently, in a big data platform, the data table structure of the data is various due to different sources of the data to be collected. In order to convert multi-source data collected in a big data platform into a uniform target data table structure for processing, a manual analysis mode is generally adopted in the prior art, data analysis is performed on the data to be collected before the data are collected, and mapping relationships between table information and field information in the current data table structure of the data and table information and field information in the target data table structure are determined so as to store the data in the big data platform according to the target data table structure.
However, in the process of implementing the present invention, it is found that the prior art has at least the following technical problems: the large data platform has large data volume, and the manual analysis mode is time-consuming and labor-consuming and has high error rate; and after the target data table structure is updated every time, the manual analysis mode needs to be adopted again, and the acquired data is stored again according to the new target data table structure, so that the process is complex and difficult to maintain.
Disclosure of Invention
The embodiment of the invention provides a mapping relation determining method and device, electronic equipment and a storage medium, and aims to improve the efficiency and accuracy of determining the mapping relation, enable the operation to be more convenient and fast and facilitate subsequent maintenance.
According to an aspect of the present invention, there is provided a mapping relationship determining method, including:
acquiring current data to be acquired, and identifying current table structure information of the current data;
determining historical table structure information corresponding to the current table structure information in a pre-established knowledge base, and determining current structure attribute information of the current table structure information based on the corresponding relation between the historical table structure information and the structure attribute information stored in the knowledge base;
and determining the mapping relation between the current table structure information and the target table structure information based on the current structure attribute information.
According to another aspect of the present invention, there is provided a mapping relationship determination apparatus including:
the current data acquisition module is used for acquiring current data to be acquired and identifying current list structure information of the current data;
a current structure attribute information determination module, configured to determine, in a pre-established knowledge base, historical table structure information corresponding to the current table structure information, and determine, based on a correspondence between the historical table structure information and structure attribute information stored in the knowledge base, current structure attribute information of the current table structure information;
and the mapping relation determining module is used for determining the mapping relation between the current table structure information and the target table structure information based on the current structure attribute information.
According to another aspect of the present invention, there is provided an electronic apparatus including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the mapping relationship determining method according to any of the embodiments of the present invention.
According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing a processor to implement the mapping relation determining method according to any embodiment of the present invention when the computer instructions are executed.
According to the technical scheme of the embodiment of the invention, the current table structure information of the current data is identified by acquiring the current data to be acquired; determining historical table structure information corresponding to the current table structure information in a pre-established knowledge base, and determining the current structure attribute information of the current table structure information based on the corresponding relation between the historical table structure information and the structure attribute information stored in the knowledge base; based on the current structure attribute information, the mapping relation between the current table structure information and the target table structure information is determined, so that the current data is stored according to the target table structure through the mapping relation, the efficiency and the accuracy of determining the mapping relation are improved, the operation is convenient, the time and the labor are saved, and the follow-up maintenance is facilitated.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present invention, nor do they necessarily limit the scope of the invention. Other features of the present invention will become apparent from the following description.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a flowchart of a mapping relationship determining method according to an embodiment of the present invention;
fig. 2 is a flowchart of another mapping relationship determining method provided in an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a mapping relationship determining apparatus according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device implementing the mapping relationship determining method according to the embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Moreover, the terms "comprises," "comprising," and any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Fig. 1 is a flowchart of a mapping relationship determining method according to an embodiment of the present invention. The method may be performed by a mapping relation determining apparatus, which may be implemented in the form of hardware and/or software.
As shown in fig. 1, the method of the embodiment may specifically include:
and S110, acquiring current data to be acquired, and identifying current table structure information of the current data.
Specifically, the current data to be acquired can be acquired before the current data to be acquired is stored in the big data platform. Wherein, the current data comprises at least one data table. For example, the current data may be student data including a student performance data sheet, a basic information data sheet, and a curriculum schedule. The current table structure information includes information such as the current table file name, the current field name of the field in the table, the current field type, and the like.
In this embodiment, a specific implementation manner of identifying the current table structure information of the current data may be to identify the table file name of each data table in the current data, the field name of a field in the table, and the like by a Natural Language Processing (NLP) method, so as to determine the current table structure information.
In this embodiment, before identifying the current table structure information of the current data, the method further includes: clustering current data based on an unsupervised learning mode to obtain a classification data table; and determining characteristic fields in the classification data tables for each classification data table.
Specifically, unsupervised learning processing may be performed on data metadata in the metadata of the current data, so as to perform clustering processing on the data tables in the current data to obtain at least one type of data table, and each type of data table obtained may be referred to as a classified data table. When determining the characteristic fields in the classification data table for each type of classification data table, the field names of the fields in the classification data table can be identified based on a natural language processing mode, and the fields corresponding to the residual field names except invalid field names in the field names are determined as the characteristic fields. The invalid field names may include remarks, supplements, other field names, etc.
In a specific implementation, the manner of identifying the current table structure information of the current data may be: and identifying the current table structure information of the current data based on the characteristic fields determined in each type of classified data table. Specifically, a classification data table can be determined from the information of each classification data table, the current table structure information of the classification data table is determined according to the table file name of the classification data table and the feature field name of the feature field in each table, and the current table structure information is determined as the current table structure information of each classification data table in the classification data table. According to the embodiment, the data tables are classified through unsupervised learning processing, so that the efficiency of determining the structural information of the current table can be improved; and the current table structure information is identified based on the characteristic field, so that the interference of invalid fields can be filtered, and the current table structure information can be identified more accurately and efficiently.
In this embodiment, identifying the current table structure information of the current data based on the feature fields determined in each classification data table includes: identifying field attributes of the characteristic fields based on a natural language processing mode; and determining the current table structure information of the current data based on the field attribute of each characteristic field. The field attribute comprises at least one of field name, field identification, field type and field length. Through a natural language processing mode, at least one of a field name, a field length, a field type and a field identification of the characteristic field can be identified; furthermore, the table attributes such as the table name and the like of each classified data table can be determined through a natural language processing mode, so that the current table structure information of each classified data table in the current data is determined based on the field attributes and the table attributes.
S120, determining historical table structure information corresponding to the current table structure information in a pre-established knowledge base, and determining the current structure attribute information of the current table structure information based on the corresponding relation between the historical table structure information and the structure attribute information stored in the knowledge base.
The history table structure information comprises information such as a history table file name, a history field name of a field in the table, a history field type and the like; the knowledge base stores at least one historical table structure information; for example, the knowledge base may be a medical database, and the historical table structure information includes table structure information of each historical data collected by the big data platform at a historical time, such as table structure information corresponding to historical data such as user data and doctor data. Further, the knowledge base stores structure attribute information corresponding to the structure information of each history table. The structure attribute information comprises table attributes and/or field attributes, the table attributes comprise table names, table identifiers, character strings corresponding to the table names and the like, and the field attributes comprise field names, field identifiers, character strings corresponding to the fields and the like.
For example, if the table structure information is a table file name, and the history table structure information corresponding to the current table structure information is determined, the history table structure information of the history table file name identical to the current table file name may be used as the history table structure information corresponding to the current table structure information, and the structure attribute information corresponding to the history table structure information may be used as the current structure attribute information. For example, the current table structure information is "table a", and the historical table structure information stored in the knowledge base includes "table a", "table B", "table C", and "table D"; the corresponding relation between the history table structure information and the structure attribute information is as follows: "table a" and "table B" correspond to the structure attribute information "table 1", table C "corresponds to the structure attribute information" table 2", and table D" corresponds to the structure attribute information "table 3", so that the history table structure information corresponding to the current table structure information is "table a", and since the structure attribute information corresponding to "table a" is "table 1", the current structure attribute information is "table 1".
S130, determining the mapping relation between the current table structure information and the target table structure information based on the current structure attribute information.
The target table structure information comprises information such as a target table file name, a target field name of a field in the table, a target field type and the like; the mapping relationship comprises at least one of a first corresponding relationship between the file name of the current table and the file name of the target table, a second corresponding relationship between the name of the current field and the name of the target field, and a third corresponding relationship between the type of the current field and the type of the target field.
In a specific implementation, the corresponding relationship between the target table structure information and the structure attribute information may be stored in the knowledge base in advance. Based on the corresponding relation between the historical table structure information and the structure attribute information and the corresponding relation between the target table structure information and the structure attribute information, on the premise that the historical table structure information corresponding to the current table structure information is determined, the mapping relation between the current table structure information and the target table structure information can be determined. Exemplarily, when the table structure information is a table file name, the current table structure information is set as "table a", and the structure attribute information corresponding to "table a" is set as "table 1"; and the knowledge base stores the target table structure information corresponding to the structure attribute information of 'table 1' as 'table a', so that the mapping relation between 'table a' and 'table a' can be determined, namely the mapping relation between the current table structure information and the target table structure information is determined.
According to the technical scheme of the embodiment of the invention, the current table structure information of the current data is identified by acquiring the current data to be acquired; determining historical table structure information corresponding to the current table structure information in a pre-established knowledge base, and determining the current structure attribute information of the current table structure information based on the corresponding relation between the historical table structure information and the structure attribute information stored in the knowledge base; based on the current structure attribute information, the mapping relation between the current table structure information and the target table structure information is determined, so that the current data is stored according to the target table structure through the mapping relation, the efficiency and the accuracy of determining the mapping relation are improved, the operation is convenient, the time and the labor are saved, and the follow-up maintenance is facilitated.
Fig. 2 is a flowchart of another mapping relationship determining method according to an embodiment of the present invention. Optionally, determining, in a pre-established knowledge base, historical table structure information corresponding to the current table structure information, includes: determining similarity values between the current table structure information and each historical table structure information in a knowledge base; and determining the historical table structure information with the similarity value larger than a first preset threshold value as the historical table structure information corresponding to the current table structure information. The same or corresponding terms as those in the above embodiments are not explained in detail herein. As shown in fig. 2, the method includes:
s210, obtaining current data to be collected, and identifying current table structure information of the current data.
S220, determining similarity values between the current table structure information and the historical table structure information in the knowledge base; and determining the historical table structure information with the similarity value larger than a first preset threshold value as the historical table structure information corresponding to the current table structure information.
In a specific implementation, the similarity value between the current table structure information and the historical table structure information is determined, and specifically, the similarity value between the current character string corresponding to the current table structure information and the historical character string corresponding to the historical table structure information is determined. For example, the similarity value between the current table structure information and the historical table structure information may be determined by means of synonym search in combination with a similarity algorithm. The similarity algorithm may include at least one of a cosine similarity algorithm, a matrix similarity algorithm, and a string edit distance algorithm.
Specifically, a first preset threshold may be preset, and the history table structure information having a similarity value with the current table structure information greater than the first preset threshold may be used as the history structure information corresponding to the current table structure information. Illustratively, the first preset threshold is 0.8. Those skilled in the art can determine the specific value of the first preset threshold according to the actual application, which is not limited in the embodiment of the present invention.
Further, when the number of the history table structure information with the similarity value larger than the first preset threshold is larger than 1, the history table structure information with the maximum similarity value can be determined from the history table structure information with the similarity value larger than the first preset threshold, and the history table structure information is used as the history table structure information corresponding to the current table structure information. And when the number of the historical table structure information with the similarity value larger than the first preset threshold value is 1, determining the unique historical table structure information as the historical table structure information corresponding to the current table structure information.
In this embodiment, after determining similarity values between the current table structure information and the historical table structure information in the knowledge base, the method further includes: if the historical table structure information with the similarity value larger than the first preset threshold does not exist, determining the maximum value in the current similarity values; sending the historical table structure information corresponding to the maximum value to an operation and maintenance terminal, and acquiring feedback information sent by the operation and maintenance terminal; historical table structure information corresponding to the current table structure information is determined based on the feedback information.
Specifically, when the historical table structure information with the similarity value larger than the first preset threshold does not exist in the knowledge base, it is indicated that the historical table structure information in the knowledge base is not matched with the current table structure information. Determining that the historical table structure information corresponding to the maximum value in the current similarity value is sent to the operation and maintenance terminal, so that the operation and maintenance personnel can confirm and receive feedback information sent by the operation and maintenance terminal. The feedback information may include confirmation information, that is, information for confirming that the historical table structure information received by the operation and maintenance terminal matches with the current table structure information; when the two are matched, the history table structure information corresponding to the maximum value in the similarity values can be used as the history table structure information corresponding to the current table structure information.
Or when the historical table structure information with the similarity value larger than the first preset threshold value does not exist in the knowledge base, determining the historical table structure information with the similarity value larger than the second preset threshold value in the knowledge base; and the second preset threshold is smaller than the first preset threshold. And sending the historical table structure information with the similarity value larger than a second preset threshold value to the operation and maintenance terminal, and determining the historical table structure information corresponding to the current table structure information based on the obtained feedback information sent by the operation and maintenance terminal.
In the embodiment, the historical table structure information with the largest similarity value is sent to the operation and maintenance terminal, so that the operation and maintenance personnel can determine the historical table structure information corresponding to the current table structure information under the condition that the historical table structure information stored in the knowledge base is incomplete, and the accuracy of the determination result is improved.
Optionally, the feedback information includes structure information of a designated table; wherein determining historical table structure information corresponding to the current table structure information based on the feedback information comprises: the specified table structure information is determined as history table structure information corresponding to the current table structure information. Specifically, when the feedback information is the specified table structure information, the explanation is sent to
The feedback information may include designated table structure information for designating the history table structure information corresponding to the current table structure information when the history table structure information received by the operation and maintenance terminal does not match with the current table structure information. Specifically, the specified table structure information may be determined as the history table structure information corresponding to the current table structure information.
Optionally, after determining the specified table structure information as the history table structure information corresponding to the current table structure information, the method further includes: the specified table structure information is stored as history table structure information in a knowledge base.
Specifically, when the operation and maintenance terminal feeds back the structural information of the designated table, it is indicated that the structural information of each historical table stored in the knowledge base is not comprehensive and is not matched with the structural information of the current table, and the structural information of the designated table can be stored into the knowledge base as the structural information of the historical table, so that the information stored in the knowledge base is perfected, and the comprehensiveness of the knowledge base is improved in a man-machine cooperation manner.
S230, determining the current structure attribute information of the current table structure information based on the corresponding relation between the historical table structure information and the structure attribute information stored in the knowledge base.
S240, determining the mapping relation between the current table structure information and the target table structure information based on the current structure attribute information.
According to the embodiment of the invention, the historical table structure information corresponding to the current table structure information is determined according to the similarity value between the current table structure information and the historical table structure information, so that the efficiency and the accuracy of the determination process are improved; and moreover, the information stored in the knowledge base is perfected in a man-machine cooperation mode, and the comprehensiveness of the knowledge base is improved.
Fig. 3 is a schematic structural diagram of a mapping relationship determining apparatus according to an embodiment of the present invention, which is configured to execute the mapping relationship determining method according to any of the embodiments. The apparatus and the method for determining a mapping relationship in the embodiments belong to the same inventive concept, and details that are not described in detail in the embodiments of the apparatus for determining a mapping relationship may refer to the embodiments of the method for determining a mapping relationship. As shown in fig. 3, the apparatus includes:
a current data obtaining module 10, configured to obtain current data to be collected, and identify current table structure information of the current data;
a current structure attribute information determining module 11, configured to determine, in a pre-established knowledge base, historical table structure information corresponding to current table structure information, and determine, based on a correspondence between the historical table structure information and structure attribute information stored in the knowledge base, current structure attribute information of the current table structure information;
and a mapping relation determining module 12, configured to determine, based on the current structure attribute information, a mapping relation between the current table structure information and the target table structure information.
On the basis of any optional technical solution in the embodiment of the present invention, optionally, the current structure attribute information determining module 11 includes:
the similarity value unit is used for determining similarity values between the current table structure information and the historical table structure information in the knowledge base;
and the first history table structure information determining unit is used for determining the history table structure information with the similarity value larger than a first preset threshold value as the history table structure information corresponding to the current table structure information.
On the basis of any optional technical solution in the embodiment of the present invention, optionally, the current structure attribute information determining module 11 further includes:
the maximum value determining unit is used for determining the maximum value in the current similarity value if the historical table structure information with the similarity value larger than the first preset threshold value does not exist after the similarity value between the current table structure information and each historical table structure information in the knowledge base is determined;
the feedback information acquisition unit is used for sending the historical table structure information corresponding to the maximum value to the operation and maintenance terminal and acquiring the feedback information sent by the operation and maintenance terminal;
a second history table structure information determining unit for determining history table structure information corresponding to the current table structure information based on the feedback information.
On the basis of any optional technical scheme in the embodiment of the present invention, optionally, the feedback information includes specified table structure information; wherein, the second history table structure information determining unit includes:
a history table structure information determining subunit operable to determine the specified table structure information as history table structure information corresponding to the current table structure information.
On the basis of any optional technical solution in the embodiment of the present invention, optionally, the second history table structure information determining unit further includes:
a history table structure information storing subunit operable to store the specified table structure information as history table structure information into the knowledge base after the specified table structure information is determined as history table structure information corresponding to the current table structure information.
On the basis of any optional technical solution in the embodiment of the present invention, optionally, the apparatus further includes:
the cluster processing module is used for carrying out cluster processing on the current data based on an unsupervised learning mode before identifying the current table structure information of the current data to obtain a classified data table;
the characteristic field determining module is used for determining the characteristic fields in the classification data tables aiming at each classification data table;
the current data acquisition module 10 includes:
and the current table structure information identification unit is used for identifying the current table structure information of the current data based on the characteristic fields determined in each type of classified data table.
On the basis of any optional technical solution in the embodiment of the present invention, optionally, the current table structure information identifying unit includes:
a field attribute identification subunit, configured to identify a field attribute of the feature field based on a natural language processing manner;
and the current table structure information determining subunit is used for determining the current table structure information of the current data based on the field attribute of each characteristic field.
According to the technical scheme of the embodiment of the invention, the current table structure information of the current data is identified by acquiring the current data to be acquired; determining historical table structure information corresponding to the current table structure information in a pre-established knowledge base, and determining the current structure attribute information of the current table structure information based on the corresponding relation between the historical table structure information and the structure attribute information stored in the knowledge base; based on the current structure attribute information, the mapping relation between the current table structure information and the target table structure information is determined, so that the current data is stored according to the target table structure through the mapping relation, the efficiency and the accuracy of determining the mapping relation are improved, the operation is convenient, the time and the labor are saved, and the follow-up maintenance is facilitated.
It should be noted that, in the embodiment of the mapping relationship determining apparatus, each included unit and each included module are only divided according to functional logic, but are not limited to the above division as long as the corresponding function can be implemented; in addition, the specific names of the functional units are only for the convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
Fig. 4 is a schematic structural diagram of an electronic device implementing the mapping relationship determining method according to the embodiment of the present invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 4, the electronic device 20 includes at least one processor 21, and a memory communicatively connected to the at least one processor 21, such as a Read Only Memory (ROM) 22, a Random Access Memory (RAM) 23, and the like, wherein the memory stores a computer program executable by the at least one processor, and the processor 21 may perform various suitable actions and processes according to the computer program stored in the Read Only Memory (ROM) 22 or the computer program loaded from the storage unit 28 into the Random Access Memory (RAM) 23. In the RAM23, various programs and data necessary for the operation of the electronic apparatus 20 can also be stored. The processor 21, the ROM22, and the RAM23 are connected to each other via a bus 24. An input/output (I/O) interface 25 is also connected to bus 24.
A number of components in the electronic device 20 are connected to the I/O interface 25, including: an input unit 26 such as a keyboard, a mouse, etc.; an output unit 27 such as various types of displays, speakers, and the like; a storage unit 28 such as a magnetic disk, optical disk, or the like; and a communication unit 29 such as a network card, modem, wireless communication transceiver, etc. The communication unit 29 allows the electronic device 20 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The processor 21 may be any of various general purpose and/or special purpose processing components having processing and computing capabilities. Some examples of the processor 21 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, or the like. The processor 21 performs the various methods and processes described above, such as the mapping relation determination method.
In some embodiments, the mapping relationship determination method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 28. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 20 via the ROM22 and/or the communication unit 29. When the computer program is loaded into the RAM23 and executed by the processor 21, one or more steps of the mapping relationship determination method described above may be performed. Alternatively, in other embodiments, the processor 21 may be configured to perform the mapping relation determination method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on a machine, as a stand-alone software package partly on a machine and partly on a remote machine or entirely on a remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.
It should be understood that various forms of the flows shown above, reordering, adding or deleting steps, may be used. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.
The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for determining a mapping relationship, comprising:
acquiring current data to be acquired, and identifying current table structure information of the current data;
determining historical table structure information corresponding to the current table structure information in a pre-established knowledge base, and determining current structure attribute information of the current table structure information based on the corresponding relation between the historical table structure information and the structure attribute information stored in the knowledge base;
and determining the mapping relation between the current table structure information and the target table structure information based on the current structure attribute information.
2. The method of claim 1, wherein determining historical table structure information corresponding to the current table structure information in a pre-established knowledge base comprises:
determining similarity values between the current table structure information and the historical table structure information in the knowledge base;
and determining the historical table structure information with the similarity value larger than a first preset threshold value as the historical table structure information corresponding to the current table structure information.
3. The method of claim 2, further comprising, after said determining a similarity value between said current table structure information and each historical table structure information in said knowledge base:
if the historical table structure information with the similarity value larger than a first preset threshold does not exist, determining the maximum value in the current similarity values;
sending the history table structure information corresponding to the maximum value to an operation and maintenance terminal, and acquiring feedback information sent by the operation and maintenance terminal;
determining historical table structure information corresponding to the current table structure information based on the feedback information.
4. The method of claim 3, wherein the feedback information comprises specific table structure information; wherein the content of the first and second substances,
the determining historical table structure information corresponding to the current table structure information based on the feedback information includes:
determining the designated table structure information as history table structure information corresponding to the current table structure information.
5. The method according to claim 4, further comprising, after said determining said specified table structure information as historical table structure information corresponding to said current table structure information:
and storing the specified table structure information as historical table structure information into the knowledge base.
6. The method of claim 1, further comprising, prior to said identifying current table structure information for said current data:
clustering the current data based on an unsupervised learning mode to obtain a classification data table;
for each type of the classified data table, determining a characteristic field in the classified data table;
the current table structure information identifying the current data includes:
and identifying current table structure information of the current data based on the characteristic fields determined in each type of the classified data table.
7. The method of claim 6, wherein said identifying current table structure information of the current data based on the characteristic field determined in each of the classified data tables comprises:
identifying field attributes of the characteristic fields based on a natural language processing mode;
and determining current table structure information of the current data based on the field attribute of each characteristic field.
8. A mapping relationship determining apparatus, characterized by comprising:
the current data acquisition module is used for acquiring current data to be acquired and identifying current table structure information of the current data;
a current structure attribute information determination module, configured to determine, in a pre-established knowledge base, historical table structure information corresponding to the current table structure information, and determine, based on a correspondence between the historical table structure information and structure attribute information stored in the knowledge base, current structure attribute information of the current table structure information;
and the mapping relation determining module is used for determining the mapping relation between the current table structure information and the target table structure information based on the current structure attribute information.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the mapping determination method of any of claims 1-7.
10. A computer-readable storage medium storing computer instructions for causing a processor to implement the mapping relation determining method according to any one of claims 1 to 7 when executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211229715.5A CN115544010A (en) | 2022-10-08 | 2022-10-08 | Mapping relation determining method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211229715.5A CN115544010A (en) | 2022-10-08 | 2022-10-08 | Mapping relation determining method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115544010A true CN115544010A (en) | 2022-12-30 |
Family
ID=84733047
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211229715.5A Pending CN115544010A (en) | 2022-10-08 | 2022-10-08 | Mapping relation determining method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115544010A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115834654A (en) * | 2023-02-22 | 2023-03-21 | 广东广宇科技发展有限公司 | Data efficient transmission method based on multiple mappings |
-
2022
- 2022-10-08 CN CN202211229715.5A patent/CN115544010A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115834654A (en) * | 2023-02-22 | 2023-03-21 | 广东广宇科技发展有限公司 | Data efficient transmission method based on multiple mappings |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112559631B (en) | Data processing method and device of distributed graph database and electronic equipment | |
| CN113836314B (en) | Knowledge graph construction method, device, equipment and storage medium | |
| CN113792154A (en) | Method and device for determining fault association relationship, electronic equipment and storage medium | |
| CN114021156A (en) | Method, device and equipment for organizing vulnerability automatic aggregation and storage medium | |
| CN114461644A (en) | Data acquisition method and device, electronic equipment and storage medium | |
| CN115168562A (en) | Method, device, equipment and medium for constructing intelligent question-answering system | |
| CN115145924A (en) | Data processing method, device, equipment and storage medium | |
| CN115544010A (en) | Mapping relation determining method and device, electronic equipment and storage medium | |
| CN115048352B (en) | Log field extraction method, device, equipment and storage medium | |
| CN117171296A (en) | Information acquisition method and device and electronic equipment | |
| CN116414814A (en) | Data checking method, device, equipment, storage medium and program product | |
| US20220129418A1 (en) | Method for determining blood relationship of data, electronic device and storage medium | |
| CN115455091A (en) | Data generation method and device, electronic equipment and storage medium | |
| CN115422275A (en) | Data processing method, device, equipment and storage medium | |
| CN115454971A (en) | Data migration method and device, electronic equipment and storage medium | |
| CN114969444A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN114661751B (en) | Data production method, device, system, equipment and medium based on SQL (structured query language) knowledge base | |
| CN116431698B (en) | Data extraction method, device, equipment and storage medium | |
| CN116401269A (en) | Data query method and device, electronic equipment and storage medium | |
| CN115794830A (en) | Data value determination method and device, electronic equipment and storage medium | |
| CN116521866A (en) | Training sample construction method and device, electronic equipment and medium | |
| CN115730000A (en) | Medical data integration method, device, equipment and medium based on data lake | |
| CN117763060A (en) | Data processing method, device, equipment and storage medium based on user behavior | |
| CN117669570A (en) | Information matching method and device, electronic equipment and storage medium | |
| CN116089459A (en) | Data retrieval method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |