CN116069997A - Metadata analysis writing method, device, electronic equipment and storage medium - Google Patents

Metadata analysis writing method, device, electronic equipment and storage medium Download PDF

Info

Publication number
CN116069997A
CN116069997A CN202211182402.9A CN202211182402A CN116069997A CN 116069997 A CN116069997 A CN 116069997A CN 202211182402 A CN202211182402 A CN 202211182402A CN 116069997 A CN116069997 A CN 116069997A
Authority
CN
China
Prior art keywords
metadata
data
bibliographic
preset
analysis
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211182402.9A
Other languages
Chinese (zh)
Inventor
王晓光
林冠强
李惠松
陈洁洪
谢炜俊
叶晓君
江飞达
余旭飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Power Grid Co Ltd
Huizhou Power Supply Bureau of Guangdong Power Grid Co Ltd
Original Assignee
Guangdong Power Grid Co Ltd
Huizhou Power Supply Bureau of Guangdong Power Grid Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Power Grid Co Ltd, Huizhou Power Supply Bureau of Guangdong Power Grid Co Ltd filed Critical Guangdong Power Grid Co Ltd
Priority to CN202211182402.9A priority Critical patent/CN116069997A/en
Publication of CN116069997A publication Critical patent/CN116069997A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/906Clustering; Classification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/901Indexing; Data structures therefor; Storage structures
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a metadata analysis writing method, a metadata analysis writing device, electronic equipment and a storage medium. The method is characterized by comprising the following steps: extracting metadata to be analyzed; performing data analysis on the metadata according to a preset metadata analysis method, and determining metadata types corresponding to the metadata; classifying the metadata according to the metadata category, and generating bibliographic data of the metadata; and carrying out data inspection on the bibliographic data, and storing the bibliographic data into a bibliographic database if the bibliographic data meets the preset inspection condition. The method realizes the accurate classification and the writing of the metadata, improves the efficiency of the writing of the metadata, and reduces the error rate of the writing.

Description

Metadata analysis writing method, device, electronic equipment and storage medium
Technical Field
The present invention relates to the field of metadata processing, and in particular, to a metadata analysis writing method, apparatus, electronic device, and storage medium.
Background
In the current background of rapid development of social internet information, data is taken as a representation form and a carrier of the internet information, the importance of the data is self-evident, metadata is taken as data for describing the data, the processing and the management of the metadata are also very important components in the field of the data, and the analysis bibliography of the metadata is indispensable in the processing and the management of the metadata.
In the prior art, metadata is often required to be recorded manually one by one, a large number of manual teams are required to be recorded to meet the requirement of the recording, resources are very consumed, the recording efficiency is low, and data errors are easy to occur.
Disclosure of Invention
The invention provides a metadata analysis writing method, a metadata analysis writing device, electronic equipment and a storage medium, which are used for solving the problems of low efficiency and low accuracy of metadata analysis writing.
According to an aspect of the present invention, there is provided a metadata analysis authoring method including:
extracting metadata to be analyzed; performing data analysis on the metadata according to a preset metadata analysis method, and determining metadata types corresponding to the metadata;
classifying the metadata according to the metadata category, and generating bibliographic data of the metadata;
and carrying out data inspection on the bibliographic data, and storing the bibliographic data into a bibliographic database if the bibliographic data meets the preset inspection condition.
According to another aspect of the present invention, there is provided a metadata analysis writing apparatus comprising:
the data acquisition module is used for extracting metadata to be analyzed;
The data analysis module is used for carrying out data analysis on the metadata according to a preset metadata analysis method and determining metadata types corresponding to the metadata;
the data recording module is used for classifying the metadata according to the metadata category and generating recording data of the metadata;
and the data detection module is used for carrying out data inspection on the copyrighted data, and storing the copyrighted data into a copyrighted database if the copyrighted data meets the preset inspection condition.
According to another aspect of the present invention, there is provided an electronic apparatus including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the metadata analysis authoring method of any one of the embodiments of the present invention.
According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to implement the metadata analysis bibliographic method according to any embodiment of the present invention when executed.
According to the technical scheme, metadata to be analyzed are extracted; according to a preset metadata analysis method, data analysis is carried out on the metadata, metadata types corresponding to the metadata are determined, analysis and classification processing is carried out on a large amount of data, and metadata processing efficiency is improved; classifying and recording the metadata according to the metadata category, generating the recording data of the metadata, classifying and recording the metadata with different classifications, further realizing high-efficiency recording of the metadata with different types, and further improving the recording efficiency; and (3) carrying out data inspection on the bibliographic data, if the bibliographic data meets the inspection preset inspection requirement, storing the bibliographic data into a bibliographic database, reducing errors of the bibliographic data by utilizing data inspection, improving the accuracy of bibliographic, solving the technical problems of slower bibliographic efficiency and high error rate in the prior art, realizing accurate classification and bibliographic of metadata, improving the bibliographic efficiency of the metadata, and reducing the error rate of bibliographic.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a metadata analysis authoring method according to a first embodiment of the present invention;
FIG. 2 is a flowchart of another metadata analysis writing method according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of a metadata analysis writing device according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device implementing a metadata analysis writing method according to an embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the above figures are used for distinguishing between similar users and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
Fig. 1 is a flowchart of a metadata analysis writing method according to an embodiment of the present invention, where the method may be performed by a metadata analysis writing device, and the metadata analysis writing device may be implemented in hardware and/or software, and the metadata analysis writing device may be configured in an electronic device. As shown in fig. 1, the method includes:
S110, extracting metadata to be analyzed.
Wherein the metadata is information for describing data attributes; by way of example, the metadata may be storage locations for indicating data, historical data, resource lookups, file records.
In the embodiment of the invention, the metadata to be analyzed is obtained in a computer network or a data center.
And S120, carrying out data analysis on the metadata according to a preset metadata analysis method, and determining a metadata category corresponding to the metadata.
Wherein the data analysis may be for summarizing and categorizing metadata. The metadata analysis method may be a method previously set for data analysis of metadata; by way of example, the predetermined data method may be a list method and a mapping method. Taking a mapping method as an example, the acquired metadata to be analyzed is arranged, and then a chart is generated according to the metadata to be analyzed.
Wherein, the metadata category may be a data category describing different attributes of the data; for example, qualitative data for representing the nature of things and quantitative data for describing the number of runs may be used.
In the embodiment of the invention, after the metadata to be analyzed is obtained, the metadata to be analyzed is subjected to data analysis by using a preset metadata analysis method, the metadata to be analyzed is subjected to summarization classification, and the metadata category of the metadata to be analyzed is determined.
Optionally, in another optional embodiment of the present invention, the data analysis is performed on the metadata according to a preset metadata analysis method, and determining a metadata category of the metadata includes:
extracting metadata keywords of the metadata according to a preset metadata analysis method;
and determining the metadata category corresponding to the metadata according to the metadata keyword.
The metadata keyword may be a keyword for describing an attribute to which the metadata belongs. It should be noted that, the form of the metadata keyword is not fixed to the form of a word, and the metadata keyword may have various forms, which is not limited in the embodiment of the present invention. By way of example, metadata keywords may be in the form of images, data body words, data trending words, tables, numbers, and the like.
In the embodiment of the invention, metadata keywords of metadata to be analyzed are extracted through a preset metadata analysis method, metadata categories to which the metadata keywords belong are determined, and the metadata categories to which the metadata keywords belong are determined as metadata categories of metadata.
Optionally, in another optional embodiment of the present invention, after extracting the metadata keyword of the metadata according to a preset metadata analysis method, the method further includes:
Determining a preset security level corresponding to the metadata keywords according to the metadata keywords;
establishing an access mechanism of the metadata according to the preset security level; wherein the access mechanism includes at least one of secured account number access, digital certificates, and physical interruptions.
The preset security level may be preset to reflect the influence degree caused by metadata destruction or leakage; the preset security level can be set according to the influence degree caused by metadata destruction or leakage. For example, if the metadata is corrupted or compromised, which can pose a serious financial or legal risk, the preset security level of the current metadata should be set to be high.
Wherein the access mechanism may be a restriction on rights or capabilities of the object to access the metadata; an access mechanism may be used to prevent unauthorized access and unauthorized use of metadata.
Specifically, metadata keywords of metadata are obtained, preset security levels corresponding to the metadata keywords are searched in a preset security level list according to the metadata keywords, the searched preset security levels are determined to be the preset security levels of the current metadata keywords, the preset security levels of the metadata keywords are determined to be the security levels of the current metadata, and then a corresponding access mechanism is determined according to the security levels, so that an access mechanism is established for the current metadata. Wherein the access mechanism includes at least one of secured account number access, digital certificates, and physical interruptions. For example, the secured account access may be to set corresponding access rights to an account accessing metadata, and set data accessible to the account; the digital certificate can be identity information for establishing access data, when a user accesses metadata, the digital certificate is contacted with a database stored by the metadata, public key encryption information and private key encryption information are used for contact, and after verification is successful, data access is performed;
The physical partition may be to physically isolate an intranet from an extranet that stores metadata.
S130, classifying the metadata according to the metadata category, and generating the bibliographic data of the metadata.
Wherein, the classification bibliography can be to bibliographically record the metadata according to metadata category; the bibliographic data may be data in which metadata is bibliographically recorded by type, and the bibliographic data may include at least one of model metadata, evaluation type metadata, and scientific type metadata.
Specifically, after the metadata category of the metadata is obtained, the metadata is classified and recorded according to the metadata category, the metadata of the same metadata category is recorded in the same recorded data, and the recorded data corresponding to each metadata category is generated. Wherein, each bibliographic data contains metadata of the same metadata category.
And S140, carrying out data inspection on the bibliographic data, and storing the bibliographic data into a bibliographic database if the bibliographic data meets the preset inspection condition.
Wherein, the data can be used for writing whether the data has data errors or not; the bibliographic database may be a database for storing bibliographic data. The preset requirement may be preset for whether the bibliographic data satisfies the requirement of storing in the bibliographic database.
Specifically, after the bibliographic data is generated, data inspection is performed on the bibliographic data, whether the bibliographic data meets the inspection preset inspection requirement is judged, and if the bibliographic data meets the inspection preset inspection requirement, the bibliographic data is stored in a bibliographic database.
Optionally, in another optional embodiment of the present invention, the performing data on the bibliographic data includes:
and carrying out data inspection on the copyrighted data through a preset automatic auditing tool and a preset inspection condition. The automatic auditor can be an auditor for judging whether the copyrighted data meets the preset inspection requirement.
Optionally, the copyrighted data can be input into an automatic auditing tool, the automatic auditing tool examines the copyrighted data according to the checking preset checking requirement, the copyrighted data is matched with the checking preset checking requirement, if the automatic auditing tool matches the checking preset checking requirement, the copyrighted data is considered to have data errors, and the output copyrighted data does not meet the checking preset checking requirement; if the automatic auditing tool does not match the preset checking requirement, the copyrighted data is considered to have no data error, and the copyrighted data is output to meet the preset checking requirement.
Specifically, the automatic auditing tool is used for carrying out data inspection on the copyrighted data, judging whether the copyrighted data meets the inspection preset inspection requirement, if the automatic auditing tool does not match the inspection preset inspection requirement, considering that the copyrighted data has no data error, outputting the copyrighted data to meet the inspection preset inspection requirement, and storing the copyrighted data meeting the inspection preset inspection requirement into the copyrighted database.
According to the technical scheme, metadata to be analyzed are extracted; according to a preset metadata analysis method, data analysis is carried out on the metadata, metadata types corresponding to the metadata are determined, analysis and classification processing is carried out on a large amount of data, and metadata processing efficiency is improved; classifying and recording the metadata according to the metadata category, generating the recording data of the metadata, classifying and recording the metadata with different classifications, further realizing high-efficiency recording of the metadata with different types, and further improving the recording efficiency; and (3) carrying out data inspection on the bibliographic data, if the bibliographic data meets the inspection preset inspection requirement, storing the bibliographic data into a bibliographic database, reducing errors of the bibliographic data by utilizing data inspection, improving the accuracy of bibliographic, solving the technical problems of slower bibliographic efficiency and high error rate in the prior art, realizing accurate classification and bibliographic of metadata, improving the bibliographic efficiency of the metadata, and reducing the error rate of bibliographic.
Example two
Fig. 2 is a flowchart of a metadata analysis writing method according to a second embodiment of the present invention, and the method for performing data verification on writing data in this embodiment is further described. As shown in fig. 2, the method includes:
s210, extracting metadata to be analyzed.
In an embodiment of the present disclosure, metadata to be analyzed is obtained in a computer network or data center.
S220, carrying out data analysis on the metadata according to a preset metadata analysis method, and generating metadata categories of the metadata.
S230, classifying the metadata according to the metadata category, and generating the bibliographic data of the metadata.
S240, carrying out data inspection on the bibliographic data through a preset automatic auditing tool and a preset inspection condition.
The automatic auditor can be an auditor for judging whether the copyrighted data meets the preset check condition. The preset check condition may be a condition preset for checking the copyrighted data
Optionally, the copyrighted data can be input into an automatic auditing tool, the automatic auditing tool examines the copyrighted data according to a preset checking requirement, the copyrighted data is matched with a preset checking condition, if the automatic auditing tool is matched with the preset checking condition, the copyrighted data is considered to have data errors, and the output copyrighted data does not meet the preset checking condition; if the automatic auditing tool does not match the preset checking condition, the copyrighted data is considered to have no data error, and the output copyrighted data meets the preset checking condition.
Specifically, the automatic auditing tool is used for carrying out data inspection on the bibliographic data, judging whether the bibliographic data meets the preset inspection conditions, if the automatic auditing tool does not match the preset inspection conditions, considering that the bibliographic data has no data error, outputting the bibliographic data meeting the preset inspection conditions, and storing the bibliographic data meeting the preset inspection conditions into a bibliographic database.
Optionally, in another optional embodiment of the present invention, the preset check condition includes a field integrity check; checking the data of the bibliographic data through a preset automatic auditing tool and a preset checking condition, wherein the checking comprises the following steps:
performing field integrity check on the copyrighted data through a preset automatic auditing tool;
and if the copyrighted data passes the field integrity check, determining that the copyrighted data meets a preset check condition.
The field integrity may be used to determine whether the bibliographic data satisfies the field integrity.
Specifically, the automatic auditing tool is used for carrying out field integrity on the copyrighted data, judging whether the copyrighted data has field integrity errors, if the automatic auditing tool does not check the field integrity errors, considering that the copyrighted data passes the field integrity check, and outputting whether the copyrighted data meets the check preset check requirement.
Optionally, in another optional embodiment of the present invention, the field integrity check includes at least one of a field check, a type check, a code check, and a character code check; the field integrity check of the bibliographic data by a preset automatic auditor comprises the following steps:
checking whether the copyrighted data has missing fields, type errors, coding errors and unrecognizable character codes or not through a preset automatic auditing tool;
if the bibliographic data does not have missing fields, type errors, coding errors and unrecognizable character codes, determining that the bibliographic data passes the field integrity check;
and if the copyrighted data has missing fields, type errors, coding errors or unrecognizable character codes, sending the copyrighted data to a preset manual data auditing platform so as to carry out manual data auditing on the copyrighted data through the manual data auditing platform.
The field check may be to detect whether there is a null field in the bibliographic data, and if there is a null field in the bibliographic data, consider that there is a missing field in the bibliographic data currently; if the blank field does not exist in the copyrighted data, the current copyrighted data is considered to meet the field check;
The type check may be to detect whether metadata of other metadata categories exists in the bibliographic data, and if metadata of other metadata categories exists in the bibliographic data, consider that the current bibliographic data type is wrong; if metadata of other metadata categories does not exist in the bibliographic data, the current bibliographic is considered to meet the type test;
the coding test can be to detect whether there is coding error in the copyrighted data, if there is coding error in the copyrighted data, the current copyrighted data is considered not to meet the coding test, if there is no coding error in the copyrighted data, the current copyrighted data is considered to meet the coding test;
the character encoding test may be to detect whether an unrecognizable character encoding exists in the bibliographic data, consider the current bibliographic data to not satisfy the character encoding test if the unrecognizable character encoding exists in the bibliographic data, and consider the current bibliographic data to satisfy the character encoding test if the unrecognizable character encoding does not exist in the bibliographic data.
Specifically, field integrity inspection is carried out on the copyrighted data through an automatic auditing tool, and if the copyrighted data does not have missing fields, type errors, coding errors and unrecognizable character codes, the copyrighted data is considered to pass the field integrity inspection; if the copyrighted data has one of missing fields, type errors, coding errors and unrecognizable character codes, the copyrighted data is considered to pass the field integrity check, and the copyrighted data is sent to a preset manual data auditing platform so as to carry out manual data auditing on the copyrighted data through the manual data auditing platform.
Optionally, in another optional embodiment of the present invention, the method includes:
and if the feedback information of the bibliographic data passing the manual data audit is received, storing the bibliographic data into a bibliographic database.
The manual data auditing platform can be an auditing platform which is preset to provide a manual auditing window.
Optionally, after receiving the transmitted bibliographic data, the manual data auditing platform generates auditing tasks and displays the auditing tasks in the manual auditing window, an operator selects the auditing tasks to be audited in the manual auditing window provided by the manual data auditing platform, the bibliographic data corresponding to the selected auditing tasks is displayed in the manual auditing window, the operator carries out manual data auditing on the bibliographic data, and after the bibliographic data passes through the manual data auditing, the bibliographic data passing through the manual auditing is transmitted to a bibliographic database.
Specifically, if the bibliographic data has one of a missing field, a type error, a coding error and unrecognizable character codes, the bibliographic data is sent to a preset manual data auditing platform, the manual data auditing platform carries out manual auditing on the bibliographic data after receiving the sent bibliographic data, after the bibliographic data passes the manual data auditing, the feedback bibliographic data passes the manual data auditing, and if the feedback information that the bibliographic data passes the manual data auditing is received, the bibliographic data is stored in a bibliographic database.
S250, if the bibliographic data meets the preset checking requirement, storing the bibliographic data into a bibliographic database.
Specifically, when the automatic auditing tool is used for carrying out data inspection on the bibliographic data, if the output bibliographic data meets the inspection preset inspection requirement, the bibliographic data meeting the inspection preset inspection requirement is stored in a bibliographic database.
According to the technical scheme, metadata to be analyzed are extracted; according to a preset metadata analysis method, data analysis is carried out on the metadata, metadata types corresponding to the metadata are determined, analysis and classification processing is carried out on a large amount of data, and metadata processing efficiency is improved; classifying and recording the metadata according to the metadata category, generating the recording data of the metadata, classifying and recording the metadata with different classifications, further realizing high-efficiency recording of the metadata with different types, and further improving the recording efficiency; and if the writing data meets the preset checking requirement, the writing data is stored in a writing database, the data checking speed is improved through the automatic checking tool, the writing efficiency is further improved, the technical problems of slower writing efficiency and high error rate in the prior art are solved, the accurate classification and writing of metadata are realized, the writing efficiency of the metadata is further improved, and the writing error rate is reduced.
Optionally, the embodiment of the invention provides another metadata analysis writing method, and the management method includes the following steps:
s1, extracting data information: and acquiring data to be analyzed.
S2, data information analysis: analyzing a large amount of acquired data, summarizing, understanding and digesting the acquired data, extracting data keywords, dividing the data processed in the data analysis into qualitative data and quantitative data, wherein the analysis method comprises a list method and a drawing method, the extracted keywords comprise contents such as graphs, main words, hot words, tables, numbers and the like, and comprehensively analyzing the data.
S3, sorting and archiving: the method comprises the steps of sorting data to be analyzed, establishing a corresponding access mechanism according to a security level, restricting security of the data to be shared, sorting the extracted data to be analyzed, which is involved in sorting and archiving, establishing the corresponding access mechanism according to the security level, restricting security of the metadata to be shared, wherein the restriction means can be a user name/password, a digital certificate and a physical partition, and establishing the security mechanism.
S4, extracting metadata of a database: and (2) extracting the required data from the source database to a basic service library of the central database, and then performing metadata recording on the extracted data by using a metadata recording tool, wherein the content is classified and recorded according to the analysis of the data in the step (S2) during recording, and the recording can be divided into model metadata, evaluation metadata and scientific research metadata.
S5, data auditing: automatically auditing the recorded data, when the automatic auditing identifies that one of the field, the type error, the code missing or the unrecognizable character code is included, prompting to audit the error data if the error is found by the inspection, and warehousing the data if the problem is considered to be absent after the audit; if the test does not find an error, the data is put into a warehouse.
S6, metadata warehouse entry: and uploading the metadata to a metadata base for storage after checking.
The metadata analysis bibliographic method provided by the embodiment of the invention comprehensively analyzes the data, saves the time of manual analysis, improves the accuracy of comprehensive analysis of the data and improves the comprehensive analysis efficiency of the data.
Example III
Fig. 3 is a schematic structural diagram of a metadata analysis writing device according to a third embodiment of the present invention. As shown in fig. 3, the apparatus includes: a data acquisition module 310, a data analysis module 320, a data bibliography module 330, and a data detection module 340. Wherein,,
a data acquisition module 310, configured to extract metadata to be analyzed;
the data analysis module 320 is configured to perform data analysis on the metadata according to a preset metadata analysis method, and determine a metadata category corresponding to the metadata;
A data writing module 330, configured to sort the metadata according to the metadata category, and generate writing data of the metadata;
the data detection module 340 is configured to perform data verification on the bibliographic data, and store the bibliographic data in a bibliographic database if the bibliographic data meets a preset verification condition.
According to the technical scheme, metadata to be analyzed are extracted; according to a preset metadata analysis method, data analysis is carried out on the metadata, metadata types corresponding to the metadata are determined, analysis and classification processing is carried out on a large amount of data, and metadata processing efficiency is improved; classifying and recording the metadata according to the metadata category, generating the recording data of the metadata, classifying and recording the metadata with different classifications, further realizing high-efficiency recording of the metadata with different types, and further improving the recording efficiency; and (3) carrying out data inspection on the bibliographic data, if the bibliographic data meets the inspection preset inspection requirement, storing the bibliographic data into a bibliographic database, reducing errors of the bibliographic data by utilizing data inspection, improving the accuracy of bibliographic, solving the technical problems of slower bibliographic efficiency and high error rate in the prior art, realizing accurate classification and bibliographic of metadata, improving the bibliographic efficiency of the metadata, and reducing the error rate of bibliographic.
Optionally, the data analysis module 320 is specifically configured to:
extracting metadata keywords of the metadata according to a preset metadata analysis method;
and determining the metadata category corresponding to the metadata according to the metadata keyword. Optionally, the data analysis module 320 is specifically further configured to:
determining a preset security level corresponding to the metadata keywords according to the metadata keywords;
establishing an access mechanism of the metadata according to the preset security level; wherein the access mechanism includes at least one of secured account number access, digital certificates, and physical interruptions.
Optionally, the data detection module 340 is specifically configured to:
and carrying out data inspection on the copyrighted data through a preset automatic auditing tool and a preset inspection condition.
Optionally, the data detection module 340 is specifically further configured to:
performing field integrity check on the copyrighted data through a preset automatic auditing tool;
and if the copyrighted data passes the field integrity check, determining that the copyrighted data meets a preset check condition.
Optionally, the data detection module 340 is specifically further configured to:
checking whether the copyrighted data has missing fields, type errors, coding errors and unrecognizable character codes or not through a preset automatic auditing tool;
If the bibliographic data does not have missing fields, type errors, coding errors and unrecognizable character codes, determining that the bibliographic data passes the field integrity check;
and if the copyrighted data has missing fields, type errors, coding errors or unrecognizable character codes, sending the copyrighted data to a preset manual data auditing platform so as to carry out manual data auditing on the copyrighted data through the manual data auditing platform.
Optionally, the data detection module 340 is specifically further configured to:
and if the feedback information of the bibliographic data passing the manual data audit is received, storing the bibliographic data into a bibliographic database.
The metadata analysis writing device provided by the embodiment of the invention can execute the metadata analysis writing method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example IV
Fig. 4 shows a schematic diagram of the structure of an electronic device 10 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 4, the electronic device 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
Various components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as metadata analysis authoring methods.
In some embodiments, the metadata analysis bibliographic method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more of the steps of the metadata analysis authoring method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the metadata analysis authoring method in any other suitable manner (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
Example five
The present embodiment provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the metadata analysis bibliographic method steps as provided by any embodiment of the present invention, the method comprising:
extracting metadata to be analyzed; performing data analysis on the metadata according to a preset metadata analysis method, and determining metadata types corresponding to the metadata;
classifying the metadata according to the metadata category, and generating bibliographic data of the metadata;
and carrying out data inspection on the bibliographic data, and storing the bibliographic data into a bibliographic database if the bibliographic data meets the preset inspection condition.
The computer storage media of embodiments of the invention may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations of the present invention may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
It will be appreciated by those of ordinary skill in the art that the modules or steps of the invention described above may be implemented in a general purpose computing device, they may be centralized on a single computing device, or distributed over a network of computing devices, or they may alternatively be implemented in program code executable by a computer device, such that they are stored in a memory device and executed by the computing device, or they may be separately fabricated as individual integrated circuit modules, or multiple modules or steps within them may be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims (10)

1. A method of metadata analysis authoring, comprising:
extracting metadata to be analyzed; performing data analysis on the metadata according to a preset metadata analysis method, and determining metadata types corresponding to the metadata;
classifying the metadata according to the metadata category, and generating bibliographic data of the metadata;
and carrying out data inspection on the bibliographic data, and storing the bibliographic data into a bibliographic database if the bibliographic data meets the preset inspection condition.
2. The method of claim 1, wherein the data analysis of the metadata according to a predetermined metadata analysis method, determining a metadata category of the metadata, comprises:
extracting metadata keywords of the metadata according to a preset metadata analysis method;
and determining the metadata category corresponding to the metadata according to the metadata keyword.
3. The method according to claim 2, wherein after extracting the metadata keywords of the metadata according to a preset metadata analysis method, further comprising:
determining a preset security level corresponding to the metadata keywords according to the metadata keywords;
Establishing an access mechanism of the metadata according to the preset security level; wherein the access mechanism includes at least one of secured account number access, digital certificates, and physical interruptions.
4. The method of claim 1, wherein said performing a data check on said bibliographic data comprises:
and carrying out data inspection on the copyrighted data through a preset automatic auditing tool and a preset inspection condition.
5. The method of claim 4, wherein the predetermined verification conditions include a field integrity verification;
the data inspection of the bibliographic data through a preset automatic auditing tool and a preset inspection condition comprises the following steps:
performing field integrity check on the copyrighted data through a preset automatic auditing tool;
and if the copyrighted data passes the field integrity check, determining that the copyrighted data meets a preset check condition.
6. The method of claim 5, wherein the field integrity check comprises at least one of a field check, a type check, a code check, and a character code check;
the field integrity check of the bibliographic data by a preset automatic auditing tool comprises the following steps:
Checking whether the copyrighted data has missing fields, type errors, coding errors and unrecognizable character codes or not through a preset automatic auditing tool;
if the bibliographic data does not have missing fields, type errors, coding errors and unrecognizable character codes, determining that the bibliographic data passes the field integrity check;
and if the copyrighted data has missing fields, type errors, coding errors or unrecognizable character codes, sending the copyrighted data to a preset manual data auditing platform so as to carry out manual data auditing on the copyrighted data through the manual data auditing platform.
7. The method as recited in claim 6, further comprising:
and if the feedback information of the bibliographic data passing the manual data audit is received, storing the bibliographic data into a bibliographic database.
8. A metadata analysis writing apparatus, comprising:
the data acquisition module is used for extracting metadata to be analyzed;
the data analysis module is used for carrying out data analysis on the metadata according to a preset metadata analysis method and determining metadata types corresponding to the metadata;
The data recording module is used for classifying the metadata according to the metadata category and generating recording data of the metadata;
and the data detection module is used for carrying out data inspection on the copyrighted data, and storing the copyrighted data into a copyrighted database if the copyrighted data meets the preset inspection condition.
9. An electronic device, the electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the metadata analysis authoring method of any one of claims 1-7.
10. A computer readable storage medium storing computer instructions for causing a processor to perform the metadata analysis bibliographic method of any one of claims 1-7 when executed.
CN202211182402.9A 2022-09-27 2022-09-27 Metadata analysis writing method, device, electronic equipment and storage medium Pending CN116069997A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211182402.9A CN116069997A (en) 2022-09-27 2022-09-27 Metadata analysis writing method, device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211182402.9A CN116069997A (en) 2022-09-27 2022-09-27 Metadata analysis writing method, device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116069997A true CN116069997A (en) 2023-05-05

Family

ID=86175798

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211182402.9A Pending CN116069997A (en) 2022-09-27 2022-09-27 Metadata analysis writing method, device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116069997A (en)

Similar Documents

Publication Publication Date Title
CN111343161B (en) Abnormal information processing node analysis method, abnormal information processing node analysis device, abnormal information processing node analysis medium and electronic equipment
US11113317B2 (en) Generating parsing rules for log messages
CN110647523B (en) Data quality analysis method and device, storage medium and electronic equipment
CN113326247B (en) Cloud data migration method and device and electronic equipment
CN112148766A (en) Method and system for sampling data using artificial neural network model
CN115204733A (en) Data auditing method and device, electronic equipment and storage medium
CN116841779A (en) Abnormality log detection method, abnormality log detection device, electronic device and readable storage medium
CN112231696B (en) Malicious sample identification method, device, computing equipment and medium
CN112148841B (en) Object classification and classification model construction method and device
CN117609992A (en) Data disclosure detection method, device and storage medium
CN116244146A (en) Log abnormality detection method, training method and device of log abnormality detection model
CN116089985A (en) Encryption storage method, device, equipment and medium for distributed log
CN113037555B (en) Risk event marking method, risk event marking device and electronic equipment
CN116150394A (en) Knowledge extraction method, device, storage medium and equipment for knowledge graph
CN115408236A (en) Log data auditing system, method, equipment and medium
CN114444087A (en) Unauthorized vulnerability detection method and device, electronic equipment and storage medium
CN116069997A (en) Metadata analysis writing method, device, electronic equipment and storage medium
CN111782967B (en) Information processing method, apparatus, electronic device, and computer-readable storage medium
CN114722401A (en) Equipment safety testing method, device, equipment and storage medium
CN114741291A (en) Method, device, equipment and medium for automatically submitting vulnerability information
CN114443493A (en) Test case generation method and device, electronic equipment and storage medium
CN114301713A (en) Risk access detection model training method, risk access detection method and risk access detection device
CN114066513A (en) User classification method and device
CN113052509A (en) Model evaluation method, model evaluation apparatus, electronic device, and storage medium
CN113837278B (en) Method and device for detecting dirty data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination