CN112287273B - Method, system and storage medium for classifying website list pages - Google Patents
Method, system and storage medium for classifying website list pages Download PDFInfo
- Publication number
- CN112287273B CN112287273B CN202011161426.7A CN202011161426A CN112287273B CN 112287273 B CN112287273 B CN 112287273B CN 202011161426 A CN202011161426 A CN 202011161426A CN 112287273 B CN112287273 B CN 112287273B
- Authority
- CN
- China
- Prior art keywords
- page
- text
- cluster
- website
- dom tree
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/957—Browsing optimisation, e.g. caching or content distillation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a method for classifying web site list pages, wherein a web site is based on a hypertext markup language (HTML), and the method comprises the following steps: step 100, acquiring a website page set, wherein the pages belong to the same website; step 200, extracting tree structure characteristics and page text characteristics of a Document Object Model (DOM) of each website page to respectively form a DOM tree structure characteristic space and a page text characteristic space; step 300, clustering the DOM tree structure characteristics and the page text characteristics in the DOM tree structure characteristic space and the page text characteristic space respectively to obtain a structure cluster and a text cluster respectively; step 400, mapping between the structure type cluster and the text type cluster according to a website address link (URL) of a website page, when the mapping is many-to-one, selecting the structure type cluster or the text type cluster which is intersected at the maximum, and finding the nearest public father node of the structure type cluster or the text type cluster which is intersected at the maximum in the website, wherein the public father node is the list page.
Description
Technical Field
The invention relates to the technical field of webpage classification, in particular to an automatic classification method and system of a Board page based on heterogeneous space association mapping.
Background
With the gradual development of the internet in recent years, networks have become the largest data source. People have been focusing on the internet data collection task for a long time. One common acquisition mode is customized acquisition, i.e., customized development is performed on a certain or a certain specific website, website link conditions are analyzed, and then a data extraction method is constructed according to the page and network characteristics of the website.
Data in the internet can be divided into different information sources such as news, forums, blogs and the like according to the publishing and interaction forms of the data, each information source has a specific format, such as a news data source, the data comprises data such as news text, news authors, news topics, news comments and the like, and each news page has a category to which the data belongs. The forum is also divided into blocks, and the data of the forum includes the main posts of the forum, the reply posts of the forum and the like. The customized development of collectors for each information source, and even each website, necessarily results in collectors that cannot be reused. This is a waste to develop. The research on a large number of websites with multiple information sources finds that the network data structures of different information sources have different forms, but have certain common characteristics. For example, a website in a news information source, whether classified according to content or a website top page, has a page similar to a list, and the page directly and explicitly lists related news article links according to a certain rule, and depending on the number of all articles under the relevant rule, the page also has related page turning links, which can help to obtain more articles. Similarly, there may be a similar structure for a website in a blog information source, often more prominently a personal home page, or personal timeline. Similar structures also exist for web sites in forum information sources, and for such structures, they can be generalized to a Board-atricle structure, where the list page is called Board page and the real data page to be collected is called articule page.
While Board pages are usually theme-dependent, i.e. all the articule page links on a Board page are often around a uniform theme or have uniform strong features. The characteristic of the Board page ensures that data under the requirement theme can be captured through one Board page, so that the collection of redundant data is avoided. The Board page is used as a portal page, and the Article page has a tree structure instead of an open graph structure, so that the perception of data change can be realized by scanning the Board page. By analyzing the Board page, the change of the data can be easily obtained, so that the data can be tracked more efficiently. Therefore, how to find the Board page from the website becomes a problem that the customized acquisition must solve.
At present, the following methods are mainly found on the Board page:
(1) based on manual work: i.e. manually screening out the Board pages from the web site. Due to the significant diversity of web pages, the cost of manually screening Board pages is quite expensive when faced with large-scale web sites, especially large-scale web sites. Meanwhile, frequent edition modification of the website also increases instability of the Board page, and the Board page needs to be re-screened at a further manual cost.
(2) Based on the rule: the experience of manually screening the Board page is converted into rules, and a simulator discovers the Board page from a website based on the rules. Similarly, web pages have significant diversity, so that the rule-based method has the inherent defect of weak generalization capability, and the recall rate and accuracy of the Board page cannot be guaranteed.
Therefore, the existing Board page discovery method mainly depends on intuitive cognition of people on the Board page, various features of the Board page, especially some hidden regular features, cannot be fully utilized, the generalization capability of the method is weak, the recall rate and accuracy of the Board page cannot be guaranteed, and the data quality of customized acquisition is influenced to a great extent.
Disclosure of Invention
In order to solve the above technical problems, an object of the present invention is to provide a method and a system for classifying Board pages based on heterogeneous spatial association mapping, wherein the Board pages can be expressed in different feature spaces according to different features of the pages, and then a connection is established between the different feature spaces, so as to fully utilize various features to identify the Board pages.
Specifically, the invention discloses a method for classifying website list pages, wherein the website is based on hypertext markup language (HTML), and the method comprises the following steps:
According to the classification method, the extraction step of the page text features comprises the following steps:
step 210, extracting the title and content information of the website page according to a hypertext markup language (HTML) rule;
step 230, encoding the information by a word embedding method (WordEmbelling) to obtain a vector representation of a word, and encoding a sentence by combining with a conditioned p-mean and an SIF method to obtain a text feature of the website page.
According to the classification method, the text characteristics of the website page further include text length, text type, text paragraph number, and sentence number.
According to the classification method, the extraction step of the DOM tree structure features comprises the following steps:
step 220, traversing the DOM tree layer by layer to obtain an HTML sequence of the DOM tree, the number of various nodes in the DOM tree, the number of external links in the DOM tree and the number of nodes of the DOM tree;
and step 240, filling (padding) the features and then combining (concat) the features to form the DOM tree structure features of the page.
According to the classification method, the page text feature clustering step in the page text feature space comprises the following steps:
step 310, clustering the page text features by using a k-means clustering algorithm (Kmeans) to obtain the text cluster;
and 330, linearly detecting different k values, and selecting an optimal k value according to the obtained dispersion inflection point of the text cluster.
According to the classification method, the DOM tree structure features are clustered by using a spectral clustering method in the DOM tree structure feature space.
To achieve another object of the present invention, there is also provided a system for classifying web site list pages, wherein the web sites are based on hypertext markup language (HTML), the system comprising:
the system comprises a webpage acquisition module, a webpage selection module and a webpage selection module, wherein the webpage acquisition module is used for acquiring a website page set, and the pages belong to the same website;
a feature extraction module, configured to extract tree structure features and page text features of a Document Object Model (DOM) of each website page, and respectively form a DOM tree structure feature space and a page text feature space;
the characteristic clustering module is used for respectively clustering the DOM tree structure characteristics and the page text characteristics in the DOM tree structure characteristic space and the page text characteristic space to respectively obtain a structure cluster and a text cluster;
and the mapping classification module is used for mapping between the structure class cluster and the text class cluster according to a website link (URL) of the website page, when the mapping is in a many-to-one condition, selecting the structure class cluster or the text class cluster which is intersected at the maximum, and finding the nearest public father node of the structure class cluster or the text class cluster which is intersected at the maximum in the website, wherein the public father node is the list page.
The classification system, wherein the feature extraction module further comprises:
the first feature extraction submodule is used for extracting the page text features; and
and the second feature extraction submodule is used for extracting the DOM tree structure features.
The classification system according to the above, wherein the feature clustering module further comprises:
the first feature clustering submodule is used for clustering the page text features in the page text feature space to obtain the text cluster; and
and the second feature clustering submodule is used for clustering the DOM tree structure features in the DOM tree structure feature space to respectively obtain the structure clusters.
To achieve another object of the present invention, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores an implementation program of information transmission, and the program, when executed by a processor, implements the steps of the classification method according to any one of the above.
In order to make the aforementioned features and effects of the present invention more comprehensible, embodiments accompanied with figures are described in detail below.
Drawings
FIG. 1 is a flowchart illustrating a method for classifying web site list pages according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a DOM feature space and a text feature space formed in an original website page data space in the method for classifying website list pages according to an embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating respective clustering in DOM feature spaces and text feature spaces in the method for classifying web site list pages according to an embodiment of the present invention;
fig. 4 is a schematic diagram illustrating mapping of cluster classes collected by different feature spaces in the classification method for a website list page according to an embodiment of the present invention;
fig. 5 is a schematic diagram illustrating a search of a website list page in the method for classifying a website list page according to an embodiment of the present invention;
fig. 6 is a frame diagram of a classification system for web site list pages according to an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. It is to be understood that the embodiments described below are only a few embodiments of the present invention, and not all embodiments.
In addition, the descriptions related to "first", "second", etc. in the present invention are for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between the embodiments may be combined with each other, but must be based on the realization of the technical solutions by a person skilled in the art, and when the technical solutions are contradictory to each other or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.
The invention is used for solving the problems that the existing Board page discovery method in the prior art cannot fully utilize various characteristics of the Board page and has weak generalization capability, and provides an automatic Board page discovery method and system based on heterogeneous space association mapping.
In the present invention, the following assumptions exist for the website, the page and their relationship to each other:
1. a web site is made up of pages that have one or more identities, i.e., URLs. Each URL uniquely corresponds to a page.
2. The page itself is composed of HTML, and the HTML contains information of nodes, node attributes, node contents (text), and node styles, wherein URLs of other pages may exist in the node attributes.
3. A page can be considered to point to a page by the URLs of other pages in the current page.
Based on the above assumptions, the websites themselves form a directed network, and the nodes in the network have respective characteristics. In the invention, the characteristics of the Board page are abstracted, namely the definition of the Board page: the Board page is naturally owned by the website and can be regarded as a node in the network formed by the website. The Board page is an aggregation page of the Article page pointed by the Board page, so that a node corresponding to the Board page has a specific network structure characteristic in the network. The Board page points to similar content of the Article page, so that the similar characteristic of the Article page content can be utilized to analyze by using text characteristics.
The Board page points to the same Article page structure, so the structural characteristics of the Article page itself can be used for analysis by utilizing the same characteristics of the Article page structure.
The existing Board page discovery methods are mainly based on manual work and rules, the methods mainly depend on intuitive cognition of people on the Board page, various features of the Board page cannot be fully utilized, particularly some hidden regular features, the generalization capability of the methods is weak, the recall rate and accuracy of the Board page cannot be further guaranteed, and the quality of data acquired in a customized mode can be influenced to a great extent.
Aiming at the problems, the invention can express the page in different feature spaces according to different features of the page, and then establishes a connection between the different feature spaces, thereby fully utilizing various features to identify the Board page.
According to the invention, the association is established through mapping of the page between the clustering results of the text space and the structural space, so that the Board page is identified. The details are as follows:
clustering in a text space: the content of the Article pages pointed by the Board page is similar, so that the similar characteristic of the content of the Article pages can be utilized to perform clustering according to text characteristics, and text clusters can be found.
Clustering in a structural space: the Board page points to the same arrow page structure, so that the arrow page structure can be utilized to perform clustering by using the DOM (Document Object Model) based structural features of the arrow page itself, and a structural class cluster can be found.
And mapping between the cluster classes gathered in the two spaces through a page URL, wherein the mapped cluster is an Article page pointed by the defined Board page according to the definition of the Board page, and if the mapping is many-to-one, the largest cluster is selected.
The nearest common parent node of the Article page set is selected as the Board page.
The invention provides a method and a system for automatically discovering a Board page based on heterogeneous space association mapping. The method comprises the steps of respectively extracting the characteristics of a page in a text space and a structure space, then respectively clustering in the two spaces, establishing the characteristic relation of the page between the text space and the structure space through mapping between clustering results, and further identifying the Board page. The method solves the problems that the existing Board page discovery method cannot fully utilize various characteristics of the Board page and has weak generalization capability.
Referring to fig. 1, fig. 1 is a flowchart illustrating a method for classifying a website list page (Board page) according to an embodiment of the present invention. The method comprises the steps of expressing each original page in two independent feature spaces, namely a DOM feature space and a text feature space, clustering the DOM feature space and the text feature space respectively, and mapping through a page URL according to the clustered clusters. By doing so, the text features and the structural features of the page can be fully utilized, and more implicit relationship features can be captured by establishing a mapping between the text space and the structural space. The cluster with the largest intersection in the two spaces is selected. According to the definition of the Board page, the clusters mapped by the clustering results of the DOM feature space and the text feature space should be the arrow pages pointed by the defined Board page, if the mapping is many-to-one, the largest cluster is selected, and then the nearest common father node of the clusters in the network is found, wherein the father node is the Board page. By the method, the learning capacity of the Board page discovery method can be improved, the Board page discovery method is not dependent on manual marking data, and the influence of the quality of the marking data on the method is avoided. Specifically, as shown in fig. 1, the classification method includes the following steps:
Based on the same inventive concept, the present invention further provides a system 500 for classifying web site list pages, where the web site is based on hypertext markup language (HTML), as shown in fig. 6, and fig. 6 shows a frame diagram of a system for classifying web site list pages according to an embodiment of the present invention, where the system includes:
a web page obtaining module 510, configured to obtain a website page set, where the pages belong to the same website;
a feature extraction module 520, configured to extract tree structure features and page text features of a Document Object Model (DOM) of each website page, so as to respectively form a DOM tree structure feature space and a page text feature space; accordingly, the feature extraction module 520 further comprises: the first feature extraction submodule 521 is configured to extract the page text feature; and a second feature extraction submodule 522 for extracting the DOM tree structure features.
A feature clustering module 530, configured to cluster the DOM tree structure features and the page text features in the DOM tree structure feature space and the page text feature space, respectively, so as to obtain structure clusters and text clusters, respectively; accordingly, the feature clustering module further comprises: a first feature clustering sub-module 531, configured to cluster the page text features in the page text feature space to obtain the text cluster; and a second feature clustering submodule 532, configured to cluster the DOM tree structure features in the DOM tree structure feature space, so as to obtain the structure cluster respectively.
And a mapping classification module 540, configured to perform mapping between the structure class cluster and the text class cluster according to a website link (URL) of the website page, and when the mapping is performed in a many-to-one manner, select the structure class cluster or the text class cluster that is intersected maximally, and find a nearest common parent node of the structure class cluster or the text class cluster that is intersected maximally in the website, where the common parent node is a list page.
Based on the same inventive concept, the present invention further provides a computer-readable storage medium, on which an implementation program of information transfer is stored, and when the program is executed by a processor, the method implements the steps of any one of the classification methods.
The invention fully utilizes the text characteristic and the structural characteristic of the page and captures more implicit relational characteristics by establishing mapping between the text space and the structural space. In addition, the unsupervised machine learning method such as clustering is used in the invention, so that the learning capacity of the Board page discovery method can be improved, and the Board page discovery method does not depend on artificial labeling data. The manual labeling data has the problems of certain label missing, label error and a large amount of expired data, so the method avoids the influence of the quality of the labeling data on the method. Therefore, compared with the existing Board page discovery method, the Board page discovery method better utilizes various characteristics of the Board page such as text, structure and the like, and utilizes an unsupervised cluster analysis method, so that the method not only has better generalization capability, but also does not need to pay labor cost.
The present invention is capable of other embodiments, and various changes and modifications can be made by one skilled in the art without departing from the spirit and scope of the invention.
Claims (10)
1. A method for classifying web site listing pages, said web sites being based on hypertext markup language (HTML), said method comprising:
step 100, acquiring a website page set, wherein the pages belong to the same website;
step 200, extracting tree structure characteristics and page text characteristics of a Document Object Model (DOM) of each website page to respectively form a DOM tree structure characteristic space and a page text characteristic space;
step 300, clustering is respectively carried out on the DOM tree structure characteristic space and the page text characteristic space aiming at the DOM tree structure characteristic and the page text characteristic, and a structure cluster and a text cluster are respectively obtained;
step 400, mapping between the structure cluster and the text cluster according to a website link (URL) of the website page, and when many-to-one mapping occurs, selecting the structure cluster or the text cluster that is most intersected, and finding a nearest common parent node of the structure cluster or the text cluster that is most intersected in the website, where the common parent node is a list page.
2. The classification method according to claim 1, wherein the extraction step of the page text features comprises:
step 210, extracting the title and content information of the website page according to a hypertext markup language (HTML) rule;
step 230, encoding the title and the content information by a word embedding method (WordEmbelling) to obtain a vector representation of a word, and encoding a sentence by combining a constrained p-mean method and a smooth inverse word frequency method (SIF) to obtain the text characteristics of the website page.
3. The classification method according to claim 2, wherein the text features of the website page further include text length, text category, number of text paragraphs, and number of sentences.
4. A classification method according to any one of claims 1 to 3, characterised in that said extraction step of DOM tree structural features comprises:
step 220, traversing the DOM tree layer by layer to obtain an HTML sequence of the DOM tree, the number of various nodes in the DOM tree, the number of external links in the DOM tree and the number of nodes of the DOM tree;
and step 240, filling (padding) the features and then merging (concat) the features to form the DOM tree structure features of the page.
5. The method of classifying according to claim 4, wherein the step of clustering the page text features of the page text feature space comprises:
step 310, clustering the page text features by using a k-means clustering algorithm (Kmeans) to obtain the text cluster;
and 330, linearly detecting different k values, and selecting an optimal k value according to the obtained dispersion inflection point of the text cluster.
6. The classification method according to claim 5, wherein the DOM tree structure features are clustered using a spectral clustering method in the DOM tree structure feature space.
7. A system for classifying web site listing pages, said web sites being based on hypertext markup language (HTML), said system comprising:
the system comprises a webpage acquisition module, a webpage selection module and a webpage selection module, wherein the webpage acquisition module is used for acquiring a website page set, and the pages belong to the same website;
a feature extraction module, configured to extract tree structure features and page text features of a Document Object Model (DOM) of each website page, and respectively form a DOM tree structure feature space and a page text feature space;
the characteristic clustering module is used for respectively clustering the DOM tree structure characteristics and the page text characteristics in the DOM tree structure characteristic space and the page text characteristic space to respectively obtain a structure cluster and a text cluster;
and the mapping classification module is used for mapping between the structure class cluster and the text class cluster according to a website link (URL) of the website page, when the mapping is in a many-to-one condition, selecting the structure class cluster or the text class cluster which is intersected at the maximum, and finding the nearest public father node of the structure class cluster or the text class cluster which is intersected at the maximum in the website, wherein the public father node is the list page.
8. The classification system of claim 7, wherein the feature extraction module further comprises:
the first feature extraction submodule is used for extracting the page text features; and
and the second characteristic extraction submodule is used for extracting the DOM tree structure characteristics.
9. The classification system according to claim 7 or 8, wherein the feature clustering module further comprises:
the first feature clustering submodule is used for clustering the page text features in the page text feature space to obtain the text cluster; and
and the second feature clustering submodule is used for clustering the DOM tree structure features in the DOM tree structure feature space to respectively obtain the structure clusters.
10. A computer-readable storage medium, characterized in that it has stored thereon a program for implementing the transfer of information, which program, when being executed by a processor, implements the steps of the classification method according to any one of claims 1 to 6.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011161426.7A CN112287273B (en) | 2020-10-27 | 2020-10-27 | Method, system and storage medium for classifying website list pages |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011161426.7A CN112287273B (en) | 2020-10-27 | 2020-10-27 | Method, system and storage medium for classifying website list pages |
Publications (2)
Publication Number | Publication Date |
---|---|
CN112287273A CN112287273A (en) | 2021-01-29 |
CN112287273B true CN112287273B (en) | 2022-09-30 |
Family
ID=74373055
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202011161426.7A Active CN112287273B (en) | 2020-10-27 | 2020-10-27 | Method, system and storage medium for classifying website list pages |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN112287273B (en) |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11366645B2 (en) | 2019-11-11 | 2022-06-21 | Klarna Bank Ab | Dynamic identification of user interface elements through unsupervised exploration |
US11726752B2 (en) | 2019-11-11 | 2023-08-15 | Klarna Bank Ab | Unsupervised location and extraction of option elements in a user interface |
US11442749B2 (en) | 2019-11-11 | 2022-09-13 | Klarna Bank Ab | Location and extraction of item elements in a user interface |
US11379092B2 (en) | 2019-11-11 | 2022-07-05 | Klarna Bank Ab | Dynamic location and extraction of a user interface element state in a user interface that is dependent on an event occurrence in a different user interface |
US11409546B2 (en) | 2020-01-15 | 2022-08-09 | Klarna Bank Ab | Interface classification system |
US11386356B2 (en) * | 2020-01-15 | 2022-07-12 | Klama Bank AB | Method of training a learning system to classify interfaces |
US10846106B1 (en) | 2020-03-09 | 2020-11-24 | Klarna Bank Ab | Real-time interface classification in an application |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101470728A (en) * | 2007-12-25 | 2009-07-01 | 北京大学 | Method and device for automatically abstracting text of Chinese news web page |
CN105843844A (en) * | 2005-04-22 | 2016-08-10 | 谷歌公司 | Method for categorizing object, such as documents and/or clusters, with respect to a taxonomy and data structure derived from such categorization |
CN107590219A (en) * | 2017-09-04 | 2018-01-16 | 电子科技大学 | Webpage personage subject correlation message extracting method |
CN110110075A (en) * | 2017-12-25 | 2019-08-09 | 中国电信股份有限公司 | Web page classification method, device and computer readable storage medium |
CN110851606A (en) * | 2019-11-18 | 2020-02-28 | 杭州安恒信息技术股份有限公司 | Website clustering method and system based on webpage structure similarity |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11880414B2 (en) * | 2017-08-07 | 2024-01-23 | Criteo Technology Sas | Generating structured classification data of a website |
-
2020
- 2020-10-27 CN CN202011161426.7A patent/CN112287273B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105843844A (en) * | 2005-04-22 | 2016-08-10 | 谷歌公司 | Method for categorizing object, such as documents and/or clusters, with respect to a taxonomy and data structure derived from such categorization |
CN101470728A (en) * | 2007-12-25 | 2009-07-01 | 北京大学 | Method and device for automatically abstracting text of Chinese news web page |
CN107590219A (en) * | 2017-09-04 | 2018-01-16 | 电子科技大学 | Webpage personage subject correlation message extracting method |
CN110110075A (en) * | 2017-12-25 | 2019-08-09 | 中国电信股份有限公司 | Web page classification method, device and computer readable storage medium |
CN110851606A (en) * | 2019-11-18 | 2020-02-28 | 杭州安恒信息技术股份有限公司 | Website clustering method and system based on webpage structure similarity |
Also Published As
Publication number | Publication date |
---|---|
CN112287273A (en) | 2021-01-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN112287273B (en) | Method, system and storage medium for classifying website list pages | |
CN101615178B (en) | Method and system for building object hierarchy | |
CN102163187B (en) | Document marking method and device | |
CN101599089B (en) | Method and system for automatically searching and extracting update information on content of video service website | |
JP2009104591A (en) | Web document clustering method and system | |
CN102662969B (en) | Internet information object positioning method based on webpage structure semantic meaning | |
CN103443786A (en) | Machine learning method to identify independent tasks for parallel layout in web browsers | |
CA2517599A1 (en) | Web server for adapted web content | |
CN103559199B (en) | Method for abstracting web page information and device | |
CN103914478A (en) | Webpage training method and system and webpage prediction method and system | |
US20090019015A1 (en) | Mathematical expression structured language object search system and search method | |
CN108021598B (en) | Page extraction template matching method and device and server | |
CA2517189A1 (en) | Web content adaption process and system | |
CN106503211B (en) | Method for automatically generating mobile version facing information publishing website | |
CN112287272B (en) | Method, system and storage medium for classifying website list pages | |
CN104572934B (en) | A kind of webpage key content abstracting method based on DOM | |
US11263062B2 (en) | API mashup exploration and recommendation | |
WO2023155303A1 (en) | Webpage data extraction method and apparatus, computer device, and storage medium | |
CN105117434A (en) | Webpage classification method and webpage classification system | |
CN101894109A (en) | Database building method and device | |
CN111381809B (en) | Method and device for searching focus page | |
CN112287274B (en) | Method, system and storage medium for classifying website list pages | |
US20050188300A1 (en) | Determination of member pages for a hyperlinked document with link and document analysis | |
CN106897287B (en) | Webpage release time extraction method and device for webpage release time extraction | |
CN102567016B (en) | Method and device for extracting use example of application programming interface |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |