CN110704404B - 一种数据质量校验方法、装置、*** - Google Patents
一种数据质量校验方法、装置、*** Download PDFInfo
- Publication number
- CN110704404B CN110704404B CN201910808716.7A CN201910808716A CN110704404B CN 110704404 B CN110704404 B CN 110704404B CN 201910808716 A CN201910808716 A CN 201910808716A CN 110704404 B CN110704404 B CN 110704404B
- Authority
- CN
- China
- Prior art keywords
- data
- check
- loading
- field data
- checked
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000012795 verification Methods 0.000 title claims description 39
- 239000012634 fragment Substances 0.000 claims description 25
- 230000004048 modification Effects 0.000 claims description 14
- 238000012986 modification Methods 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 5
- 238000004806 packaging method and process Methods 0.000 claims description 5
- 230000004888 barrier function Effects 0.000 abstract description 3
- 238000005086 pumping Methods 0.000 abstract description 2
- 238000004891 communication Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000004075 alteration Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013524 data verification Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0751—Error or fault detection not based on redundancy
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Claims (8)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910808716.7A CN110704404B (zh) | 2019-08-29 | 2019-08-29 | 一种数据质量校验方法、装置、*** |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910808716.7A CN110704404B (zh) | 2019-08-29 | 2019-08-29 | 一种数据质量校验方法、装置、*** |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110704404A CN110704404A (zh) | 2020-01-17 |
CN110704404B true CN110704404B (zh) | 2023-04-28 |
Family
ID=69194222
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910808716.7A Active CN110704404B (zh) | 2019-08-29 | 2019-08-29 | 一种数据质量校验方法、装置、*** |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110704404B (zh) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110929493B (zh) * | 2020-02-16 | 2020-08-04 | 广州信安数据有限公司 | 数据管理方法、冗余数据检测方法、存储介质及数据*** |
CN111506578B (zh) * | 2020-04-17 | 2024-07-09 | 深圳前海微众银行股份有限公司 | 业务数据的校验方法、装置、设备及存储介质 |
CN112148721B (zh) * | 2020-09-25 | 2022-08-19 | 新华三大数据技术有限公司 | 数据检核方法、装置、电子设备及存储介质 |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104391903A (zh) * | 2014-11-14 | 2015-03-04 | 广州科腾信息技术有限公司 | 一种基于分布存储和并行计算的电网数据质量检测方法 |
CN107491381A (zh) * | 2017-07-04 | 2017-12-19 | 广西电网有限责任公司电力科学研究院 | 一种设备状态监测数据质量评测*** |
CN107679146A (zh) * | 2017-09-25 | 2018-02-09 | 南方电网科学研究院有限责任公司 | 电网数据质量的校验方法和*** |
CN107908672A (zh) * | 2017-10-24 | 2018-04-13 | 深圳前海微众银行股份有限公司 | 基于Hadoop平台的应用报表实现方法、设备及存储介质 |
CN108776856A (zh) * | 2018-04-20 | 2018-11-09 | 国家电网有限公司 | 一种基于溯源关系的电力台账数据校验方法及装置 |
CN110059078A (zh) * | 2019-04-19 | 2019-07-26 | 中国航空无线电电子研究所 | 一种导航数据库定制数据的动态配置和校验方法 |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2505184A (en) * | 2012-08-21 | 2014-02-26 | Ibm | Checking data quality of an application program by monitoring runtime behaviour |
CN103699638B (zh) * | 2013-12-23 | 2017-02-15 | 国云科技股份有限公司 | 一种基于配置参数实现跨数据库类型同步数据的方法 |
US10713587B2 (en) * | 2015-11-09 | 2020-07-14 | Xerox Corporation | Method and system using machine learning techniques for checking data integrity in a data warehouse feed |
US10416931B2 (en) * | 2016-09-30 | 2019-09-17 | Hewlett Packard Enterprise Development Lp | Fault tolerance in shared memory |
CN107247799A (zh) * | 2017-06-27 | 2017-10-13 | 北京天机数测数据科技有限公司 | 兼容多种大数据存储的数据处理方法、***及其建模方法 |
CN107423436A (zh) * | 2017-08-04 | 2017-12-01 | 郑州云海信息技术有限公司 | 一种用于不同类型数据库间在线数据迁移的方法 |
CN109947833A (zh) * | 2019-02-27 | 2019-06-28 | 浪潮软件集团有限公司 | 一种基于b/s架构的数据质量管理方法 |
-
2019
- 2019-08-29 CN CN201910808716.7A patent/CN110704404B/zh active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104391903A (zh) * | 2014-11-14 | 2015-03-04 | 广州科腾信息技术有限公司 | 一种基于分布存储和并行计算的电网数据质量检测方法 |
CN107491381A (zh) * | 2017-07-04 | 2017-12-19 | 广西电网有限责任公司电力科学研究院 | 一种设备状态监测数据质量评测*** |
CN107679146A (zh) * | 2017-09-25 | 2018-02-09 | 南方电网科学研究院有限责任公司 | 电网数据质量的校验方法和*** |
CN107908672A (zh) * | 2017-10-24 | 2018-04-13 | 深圳前海微众银行股份有限公司 | 基于Hadoop平台的应用报表实现方法、设备及存储介质 |
CN108776856A (zh) * | 2018-04-20 | 2018-11-09 | 国家电网有限公司 | 一种基于溯源关系的电力台账数据校验方法及装置 |
CN110059078A (zh) * | 2019-04-19 | 2019-07-26 | 中国航空无线电电子研究所 | 一种导航数据库定制数据的动态配置和校验方法 |
Non-Patent Citations (4)
Title |
---|
Fariz Darari 等.Completeness Management for RDF Data Sources.ACM.2018,第1–53页. * |
张岳 ; 马卫峰 ; 张小波 ; 朱寿东 ; 郭兰萍 ; 王星星 ; .中药资源普查数据校验的方法与探讨.中国中药杂志.2013,(13),第2060-2062页. * |
杨博 ; 王聚全 ; 杜渂 ; .城市开放区域的客流数据质量管理研究与运用.电信快报.2018,(02),第19-23页. * |
林霞 ; 申端明 ; 时迎 ; 乔德新 ; .可配置组合式数据校验方法.计算机***应用.2015,(05),第161-166页. * |
Also Published As
Publication number | Publication date |
---|---|
CN110704404A (zh) | 2020-01-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110704404B (zh) | 一种数据质量校验方法、装置、*** | |
US10534773B2 (en) | Intelligent query parameterization of database workloads | |
US11249969B2 (en) | Data storage method and apparatus, and storage medium | |
US9477974B2 (en) | Method and systems for flexible and scalable databases | |
CN106802932B (zh) | 一种数据库的路由方法、装置及数据库*** | |
CN105630683A (zh) | 一种云测试体系架构 | |
US20190258736A1 (en) | Dynamic Execution of ETL Jobs Without Metadata Repository | |
CN113516506B (zh) | 一种数据处理方法、装置及电子设备 | |
CN103034730B (zh) | 一种基于安卓平台的数据处理方法和*** | |
CN112148705A (zh) | 数据迁移的方法和装置 | |
CN113419896B (zh) | 数据恢复方法、装置、电子设备及计算机可读介质 | |
CN116628042A (zh) | 数据处理方法、装置、设备及介质 | |
CN114064693A (zh) | 处理账户数据的方法、装置、电子设备和计算机可读介质 | |
CN114328700A (zh) | 医疗数据etl任务中的数据核查方法及装置 | |
CN113609152A (zh) | 数据处理方法、装置及计算设备 | |
CN109669668B (zh) | 一种***性能测试中实现模拟交易执行的方法和装置 | |
CN113779117A (zh) | 一种数据监控方法、装置、存储介质和电子设备 | |
CN111158994A (zh) | 一种压测性能测试方法及装置 | |
CN113760600A (zh) | 一种数据库备份方法、数据库还原方法和相关装置 | |
CN112235332A (zh) | 一种集群的读写切换方法和装置 | |
CN112182510B (zh) | 产品覆盖程度的度量方法、装置及设备 | |
CN113392010B (zh) | 公共组件测试方法、装置、电子设备及存储介质 | |
CN111352795A (zh) | 一种移动业务的监控方法及装置 | |
US20230185549A1 (en) | Automatic Workflow Generation | |
CN110309038B (zh) | 性能测试方法、装置、电子设备及计算机可读存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
CP03 | Change of name, title or address |
Address after: No.1-1 Suning Avenue, Xuzhuang Software Park, Xuanwu District, Nanjing, Jiangsu Province, 210000 Patentee after: Jiangsu Suning cloud computing Co.,Ltd. Country or region after: China Address before: No.1-1 Suning Avenue, Xuzhuang Software Park, Xuanwu District, Nanjing, Jiangsu Province, 210000 Patentee before: Suning Cloud Computing Co.,Ltd. Country or region before: China |
|
CP03 | Change of name, title or address | ||
TR01 | Transfer of patent right |
Effective date of registration: 20240123 Address after: 210000, 1-5 story, Jinshan building, 8 Shanxi Road, Nanjing, Jiangsu. Patentee after: SUNING.COM Co.,Ltd. Country or region after: China Address before: No.1-1 Suning Avenue, Xuzhuang Software Park, Xuanwu District, Nanjing, Jiangsu Province, 210000 Patentee before: Jiangsu Suning cloud computing Co.,Ltd. Country or region before: China |
|
TR01 | Transfer of patent right |