CN111461253A - Automatic feature extraction system and method - Google Patents
Automatic feature extraction system and method Download PDFInfo
- Publication number
- CN111461253A CN111461253A CN202010302592.8A CN202010302592A CN111461253A CN 111461253 A CN111461253 A CN 111461253A CN 202010302592 A CN202010302592 A CN 202010302592A CN 111461253 A CN111461253 A CN 111461253A
- Authority
- CN
- China
- Prior art keywords
- feature extraction
- module
- features
- characteristic
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Stored Programmes (AREA)
Abstract
The invention relates to an automatic feature extraction framework system and a method, comprising the following steps: a signal input module: for inputting field information; a configuration reading module: the method is used for defining field types and corresponding mapping services, and configuring corresponding fields and characteristic methods; a preprocessing module: the system is used for performing characteristic preprocessing on the input field; a feature extraction module: the characteristic extraction module is used for extracting the characteristics of the preprocessed data; a signal output module: the features are used for being jointly extracted and are uniformly output to the model; the invention provides an automatic feature extraction method based on a factory model, and provides a method for extracting features through multi-person cooperation co-construction and mutual interactive multiplexing in the whole implementation process; the invention abstracts each module and method of the feature extraction by adopting the reflection mechanism of the factory mode and the related feature class, fully automates the feature extraction, and can conveniently select and iterate the features through configuration.
Description
Technical Field
The invention relates to the technical field of internet, in particular to an automatic feature extraction framework system and method.
Background
With the rapid development of big data and machine computing power, machine learning and deep learning are increasingly applied to various aspects of solving practical problems.
The invention discloses a robot loop detection method and device based on depth measurement learning combined with a bag-of-words tree model, and belongs to the patent publication number CN 108986168. The method comprises the following steps: 1) Inputting a scene video stream with long-time environment appearance change; 2) Training and learning by adopting a depth measurement learning frame to obtain a feature extraction network; 3) Extracting features from the training video stream images by using a feature extraction network; 4) Performing iterative clustering on the obtained features, and establishing a bag-of-words tree model; 5) Inputting a current key frame in a video stream acquired by the robot in real time in the actual robot positioning and navigation process; 6) Extracting the characteristics of the current key frame by using a characteristic extraction network; 7) Adding the characteristics of the current key frame to the bag-of-words tree model; 8) And searching image frames with similar characteristics with the matched image by using the bag-of-words tree model, measuring the similarity, and judging whether the robot encounters a loop. The invention can realize the high-efficiency loop detection of the robot in the long-time positioning and navigation process in the dynamic environment.
In the prior art, a fully-automatic real-time feature extraction framework is not provided, more or less people are required to participate in each link basically, the fully-natural accumulated feature extraction class of a plurality of people cannot be met, a related implementation method is provided, and each iteration cannot be reused on a feature extraction method and specific parameters.
Disclosure of Invention
In order to solve the problems, the invention provides an automatic feature extraction framework system and method, which adopt a factory mode and a reflection mechanism of related feature classes to abstract each module and method for feature extraction, fully automate feature extraction, can conveniently select and iterate features through configuration, can be completed through configuration modification by one key when an algorithm iteratively selects a feature training test model for multiple times, and avoids the situation of insufficient reuse of multiple selected features.
The technical scheme of the invention is as follows:
an automated feature extraction framework system, comprising:
a signal input module: for inputting field information;
a configuration reading module: the method is used for defining field types and corresponding mapping services, and configuring corresponding fields and characteristic methods;
a preprocessing module: the system is used for performing characteristic preprocessing on the input field;
a feature extraction module: the characteristic extraction module is used for extracting the characteristics of the preprocessed data;
a signal output module: and the features are jointly extracted and uniformly output to the model.
Preferably, the configuration files abstracted from the configuration reading module are feallib.
Preferably, the configuration file feallib. conf is used for defining related field types and corresponding mapping services in an upstream log; conf is used to configure the corresponding fields and feature methods.
Preferably, the features processed by the preprocessing module are defined into a specific feature engineering class in a java reflection mode.
Preferably, the slot position corresponding to the good feature is defined in the feature extraction module, and the same field can be supported to extract multiple features or generate the same feature jointly by multiple fields.
Preferably, the features extracted by the feature extraction module are combined into a uniform sample instance through an output interface and output to the model.
The invention also provides an automatic feature extraction framework method, which comprises the following steps:
s1: inputting field information through a signal input interface;
s2: finding out a corresponding characteristic preprocessing method through configuration;
s3: then extracting corresponding characteristics through specific characteristic extraction classes, and carrying out related barrel encryption;
s4: and the plurality of encrypted characteristics are combined into a uniform sample instance through a signal output interface and output to the model.
More preferably, the field information can be processed by a plurality of feature preprocessing methods at the same time.
The invention has the beneficial effects that: the invention provides an automatic feature extraction method based on a factory model, and provides a method for extracting features through multi-person cooperation co-construction and mutual interactive multiplexing in the whole implementation process; according to the invention, each module and method for extracting the characteristics are abstracted by adopting a reflection mechanism of a factory mode and related characteristic classes, the characteristics are fully automatically extracted, and the characteristics can be conveniently selected and iterated through configuration; the invention abstracts the factory of the feature extraction class, can add the feature extraction class of a plurality of persons to the factory when in operation, and can also conveniently use the existing feature class in the factory; in the invention, when the algorithm iteratively selects the feature training test model for multiple times, the feature training test model can be completed by configuring and modifying a key, so that the condition of insufficient reuse of the selected features for multiple times is avoided.
Drawings
Fig. 1 is a flowchart of feature extraction in an embodiment of the present invention.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
As shown in fig. 1, an automated feature extraction framework method includes the following steps:
s1: inputting field information through a signal input interface;
s2: finding out a corresponding feature preprocessing method through format matching and configuration files; (support for simultaneous multi-feature preprocessing method)
S3: then extracting corresponding characteristics through specific characteristic extraction classes and configuration files, and carrying out related barrel encryption;
s4: and the plurality of encrypted characteristics are combined into a uniform sample instance through a signal output interface and output to the model.
In order to realize the method, the invention provides an automatic feature extraction framework system, which comprises a signal input module for inputting field information; a configuration reading module for defining field type and corresponding mapping service, and configuring corresponding field and characteristic method; the preprocessing module is used for performing characteristic preprocessing on the input field; the characteristic extraction module is used for extracting the characteristics of the preprocessed data; and the signal output module is used for jointly extracting the characteristics and uniformly outputting the characteristics to the model.
The configuration files abstracted from the configuration reading module are feallib.conf and feature _ list.conf, and the feallib.conf is used for defining related field types and corresponding mapping services in the upstream log; conf is used for configuring corresponding fields and feature methods, and the features processed in the preprocessing module are defined to specific feature engineering classes in a java reflection mode.
The slot position corresponding to the good characteristic is defined in the characteristic extraction module, and the same field can be supported to extract a plurality of characteristics or a plurality of fields are combined to generate the same characteristic.
The features extracted by the feature extraction module are combined into a uniform sample instance through an output interface and output to the model.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the present invention in its spirit and scope. Are intended to be covered by the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (8)
1. An automated feature extraction framework system, comprising:
a signal input module: for inputting field information;
a configuration reading module: the method is used for defining field types and corresponding mapping services, and configuring corresponding fields and characteristic methods;
a preprocessing module: the system is used for performing characteristic preprocessing on the input field;
a feature extraction module: the characteristic extraction module is used for extracting the characteristics of the preprocessed data;
a signal output module: and the features are jointly extracted and uniformly output to the model.
2. The automated feature extraction framework system of claim 1, wherein the configuration files abstracted from the configuration reading module are feallib.conf and feature _ list.conf.
3. The automated feature extraction framework system of claim 2, wherein the configuration file feallib. conf is used to define relevant field types and corresponding mapping traffic in upstream logs; conf is used to configure the corresponding fields and feature methods.
4. The automated feature extraction framework system of claim 3, wherein the features processed in the pre-processing module are defined to a specific feature engineering class by way of java reflection.
5. The automated feature extraction framework system of claim 1, wherein slot slots corresponding to good features are defined in the feature extraction module, and can support the same field to extract multiple features or combine multiple fields to generate the same feature.
6. The automated feature extraction framework system according to claim 1, wherein the features extracted by the feature extraction module are combined into a unified sample instance through an output interface and output to a model.
7. An automated feature extraction framework method, comprising the steps of:
s1: inputting field information through a signal input interface;
s2: finding out a corresponding characteristic preprocessing method through configuration;
s3: then extracting corresponding characteristics through specific characteristic extraction classes, and carrying out related barrel encryption;
s4: and the plurality of encrypted characteristics are combined into a uniform sample instance through a signal output interface and output to the model.
8. The automated feature extraction framework method of claim 7, wherein the field information can be processed by multiple feature pre-processing methods simultaneously.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010302592.8A CN111461253A (en) | 2020-04-17 | 2020-04-17 | Automatic feature extraction system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010302592.8A CN111461253A (en) | 2020-04-17 | 2020-04-17 | Automatic feature extraction system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111461253A true CN111461253A (en) | 2020-07-28 |
Family
ID=71685944
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010302592.8A Pending CN111461253A (en) | 2020-04-17 | 2020-04-17 | Automatic feature extraction system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111461253A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111967611A (en) * | 2020-08-20 | 2020-11-20 | 贝壳技术有限公司 | Feature generation method and apparatus, electronic device, and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107025233A (en) * | 2016-01-29 | 2017-08-08 | 苏宁云商集团股份有限公司 | A kind of processing method and processing device of data characteristics |
| CN108986168A (en) * | 2018-06-13 | 2018-12-11 | 深圳市感动智能科技有限公司 | A kind of robot winding detection method and device combining bag of words tree-model based on depth measure study |
-
2020
- 2020-04-17 CN CN202010302592.8A patent/CN111461253A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107025233A (en) * | 2016-01-29 | 2017-08-08 | 苏宁云商集团股份有限公司 | A kind of processing method and processing device of data characteristics |
| CN108986168A (en) * | 2018-06-13 | 2018-12-11 | 深圳市感动智能科技有限公司 | A kind of robot winding detection method and device combining bag of words tree-model based on depth measure study |
Non-Patent Citations (1)
| Title |
|---|
| 王磊: "《移动机器人混合视觉伺服控制方法研究》", 《中国硕士论文全文库》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111967611A (en) * | 2020-08-20 | 2020-11-20 | 贝壳技术有限公司 | Feature generation method and apparatus, electronic device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110581898B (en) | Internet of things data terminal system based on 5G and edge calculation | |
| CN109165573B (en) | Method and device for extracting video feature vector | |
| CN109740018B (en) | Method and device for generating video label model | |
| CN110443222B (en) | Method and device for training face key point detection model | |
| CN113365147B (en) | Video editing method, device, equipment and storage medium based on music card point | |
| CN110363220B (en) | Behavior class detection method and device, electronic equipment and computer readable medium | |
| CN109189544B (en) | Method and device for generating dial plate | |
| CN109389096B (en) | Detection method and device | |
| CN111275784A (en) | Method and device for generating image | |
| CN107729928A (en) | Information acquisition method and device | |
| CN114898466A (en) | Video motion recognition method and system for smart factory | |
| CN114419363A (en) | Target classification model training method and device based on label-free sample data | |
| CN107633058B (en) | Deep learning-based data dynamic filtering system and method | |
| CN111461253A (en) | Automatic feature extraction system and method | |
| CN114065720A (en) | Conference summary generation method and device, storage medium and electronic equipment | |
| CN111597361B (en) | Multimedia data processing method, device, storage medium and equipment | |
| CN107402878A (en) | Method of testing and device | |
| CN117611272A (en) | Commodity recommendation method and device and electronic equipment | |
| CN112633100A (en) | Behavior recognition method and device, electronic equipment and storage medium | |
| US11762939B2 (en) | Measure GUI response time | |
| CN114693554B (en) | Big data image processing method and system | |
| CN111459774A (en) | Method, device and equipment for acquiring flow of application program and storage medium | |
| CN113033282B (en) | Image recognition method, device and medium based on small object detection | |
| CN111176624B (en) | Method and device for generating stream type calculation index | |
| CN114676705A (en) | Dialogue relation processing method, computer and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |