CN110956123B - Method, device, server and storage medium for auditing rich media content - Google Patents
Method, device, server and storage medium for auditing rich media content Download PDFInfo
- Publication number
- CN110956123B CN110956123B CN201911183179.8A CN201911183179A CN110956123B CN 110956123 B CN110956123 B CN 110956123B CN 201911183179 A CN201911183179 A CN 201911183179A CN 110956123 B CN110956123 B CN 110956123B
- Authority
- CN
- China
- Prior art keywords
- rich media
- media content
- content
- scene
- auditing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 230000001537 neural effect Effects 0.000 claims description 70
- 238000013528 artificial neural network Methods 0.000 claims description 62
- 238000004590 computer program Methods 0.000 claims description 5
- 238000012550 audit Methods 0.000 abstract description 27
- 238000007726 management method Methods 0.000 abstract description 4
- 238000012795 verification Methods 0.000 abstract description 4
- 238000007635 classification algorithm Methods 0.000 description 10
- 238000004422 calculation algorithm Methods 0.000 description 8
- 238000012015 optical character recognition Methods 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000007429 general method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001568 sexual effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000004383 yellowing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/40—Document-oriented image-based pattern recognition
- G06V30/41—Analysis of document content
- G06V30/413—Classification of content, e.g. text, photographs or tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Multimedia (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The embodiment of the invention relates to the field of information management, and discloses a method, a device, a server and a storage medium for auditing rich media contents, wherein the method for auditing the rich media contents comprises the following steps: performing scene classification on the rich media content to be audited to obtain scene types of the rich media content; checking the rich media content corresponding to the scene type according to the scene type; and if the rich media content contains illegal content, outputting a verification result of the content illegal. According to the method, the rich media content is subjected to scene classification, and the corresponding auditing method is adopted to audit the rich media content according to different scenes, so that the accuracy is ensured, the efficiency is greatly improved, and the labor cost is reduced.
Description
Technical Field
The embodiment of the invention relates to the field of information management, in particular to a method and a device for auditing rich media content, a server and a storage medium.
Background
With the vigorous development of internet technology, the display and sharing of rich media content are also becoming more and more abundant. And auditing the rich media content, filtering out junk content such as yellow, administrative, riot and advertisement information, and the like, and increasingly pressing the requirement of purifying the network environment. Because the deep learning has good application effect in the fields of images and texts, the deep learning is feasible for content auditing of rich media, and meanwhile, a great deal of labor cost can be saved, and corresponding concepts are also provided for how to train the neural network.
However, the inventors of the present invention found that: the existing method is only a general method for deep learning, identification and auditing, and how to perform neural network training, but the problems of high operation pressure and low auditing speed still exist in the auditing process.
Disclosure of Invention
The embodiment of the invention aims to provide a method, a device, a server and a storage medium for auditing rich media content, and the identification efficiency of the rich media content is improved.
In order to solve the technical problems, the embodiment of the invention provides an auditing method of rich media content, which comprises the following steps: performing scene classification on the rich media content to be audited to obtain the scene type of the rich media content; auditing the rich media content corresponding to the scene type according to the scene type; and if the rich media content contains illegal content, outputting a content illegal auditing result.
The embodiment of the invention also provides an auditing device of the rich media content, comprising: the scene classification unit is used for classifying scenes of the rich media content to be audited to obtain scene types of the rich media content; the auditing unit is used for auditing the rich media content corresponding to the scene type according to the scene type; and the output unit is used for outputting the auditing result of the content violation when the auditing unit judges that the rich media content contains the violation content.
The embodiment of the invention also provides a server, which comprises: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of auditing rich media content described above.
The embodiment of the invention also provides a computer readable storage medium which stores a computer program, wherein the computer program realizes the auditing method of the rich media content when being executed by a processor.
Compared with the prior art, the embodiment of the invention has the main differences and effects that in the auditing and identifying process of the rich media content, the scenes of the rich media content are firstly judged and classified, the scene types of the rich media content are determined, then the rich media content is identified and analyzed by combining different auditing methods corresponding to each scene, whether the problem of illegal exists in the rich media content is judged, and then the auditing result is output. Before content identification, the rich media content scenes are classified, and then identification analysis is performed by adopting a corresponding verification method according to different scene types, so that verification results can be obtained only after the rich media content is subjected to all identification algorithms in the identification process, the identification efficiency is greatly improved, and the operation pressure of a system in the identification process is reduced.
In addition, scene types of the rich media content include a general scene, a text scene, and a life scene; auditing the rich media content corresponding to the scene type according to the scene type comprises the following steps: if the scene type is a common scene, judging that the rich media content does not contain illegal content; if the scene type is a text scene, carrying out content identification on the rich media content by adopting a text neural network, and judging whether the rich media content contains illegal content or not; if the scene type is a living scene, the picture neural network is adopted to identify the content of the rich media content, whether the rich media content contains illegal content is judged, and when the system judges that the rich media content scene is a common scene, the result is directly output, so that the data of the rich media content to be identified is reduced, and the efficiency is improved; different neural networks are set for texts and pictures respectively, so that the recognition efficiency is improved.
In addition, the picture neural network comprises N neural sub-networks, wherein N is a natural number greater than 1, and each neural sub-network corresponds to a priority; when the picture neural network is adopted to carry out content identification on the rich media content: sequentially adopting each neural sub-network to identify the content of the rich media according to the order of the priorities of the neural sub-networks from large to small; and if the identification result of any neural subnetwork is that the illegal content is contained, the rich media content contains the illegal content. Considering that various violations are possible in the picture content, a plurality of neural networks are set for auditing, corresponding priorities are distributed for the neural networks, and auditing is performed according to the order of the priorities from high to low, and when any neural network auditing result is illegal, the auditing result is directly output, so that auditing quality is ensured, and auditing efficiency can be greatly improved.
In addition, the N neural subnetworks include: a political neural subnetwork, a Huang Shenjing subnetwork, an riot neural subnetwork, and an advertising neural subnetwork. A specific manner of constructing a neural network is presented herein.
In addition, when the content identification is performed on the rich media content by adopting the administration-related neural sub-network, the method comprises the following steps: inputting the rich media content into the political affair-related neural sub-network to obtain political affair-related probability output by the political affair-related neural sub-network; when the administrative probability is larger than a preset first preset threshold value, judging that the rich media content contains illegal content; when the administrative probability is smaller than a second preset threshold value, judging that the rich media content does not contain illegal content; wherein the first preset threshold is greater than the second preset threshold; and when the administrative probability is larger than a second preset threshold and smaller than the first preset threshold, performing manual auditing, and judging whether the rich media content contains illegal content according to the manual auditing result. The clear violation probability threshold is set for the judgment of the administrative violations, and the administrative violations are compared with the preset threshold, so that the recognition speed is improved while the judgment standard is clearer, the auditing and the dealing in the administrative probability interval where the machine is easy to cause erroneous judgment are manually processed, the erroneous judgment of the machine on the violation condition of the picture content is avoided, and the recognition accuracy is improved.
In addition, when the content identification is performed on the rich media content by adopting the Huang Shenjing sub-network, the method comprises the following steps: inputting the rich media content into a main wading Huang Shenjing sub-network to obtain a wading Huang Gailv output by the main wading Huang Shenjing sub-network; when Huang Gailv is smaller than a third preset threshold value, judging that the rich media content does not contain illegal content; when the yellow probability is larger than a fourth preset threshold value, adopting an auxiliary Huang Shenjing sub-network to identify the content of the rich media content; wherein the fourth preset threshold is greater than the third preset threshold; and when the yellowing probability is larger than the third preset threshold and smaller than the fourth preset threshold, performing manual auditing, and judging whether the rich media content contains illegal content according to the manual auditing result. And setting a clear threshold value for the yellow-related violations, comparing the yellow-related probability with a preset threshold value, improving the recognition speed while ensuring that the judging standard is clearer, manually processing the auditing and negotiating section Huang Gailv which is in a section where the machine is easy to cause misjudgment, directly judging the picture as the violations, and then carrying out the auditing of an auxiliary sub-neural network, thereby avoiding misjudgment of the machine on the violations of the picture content and improving the recognition accuracy.
In addition, when the advertising neural sub-network is adopted to carry out content identification on the rich media content, the method comprises the following steps: performing content identification on the rich media content by adopting a text neural sub-network; if the identification result is that the illegal content is contained, judging that the rich media content contains the illegal content; and if the identification result is that the illegal content is not contained, carrying out content identification on the rich media content by adopting the main advertising neural sub-network. The text sub-neural network is preset before the advertising neural network to audit the rich media content in advance, and the audit result is directly output when the judgment result is illegal, so that the advertising neural network is not audited, a large amount of time is saved, and the audit efficiency is improved.
In addition, in the scene classification process of the rich media content to be audited, a preset lightweight neural network is adopted to perform scene classification on the rich media content to be audited. The aim of the pre-classification algorithm is to quickly classify the picture scenes, so that the requirement on the operation speed of the algorithm is high, the lightweight neural network is adopted to classify the scenes of the rich media content, the time required by scene classification is reduced, and the overall time in the identification process is greatly reduced.
Drawings
One or more embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings.
FIG. 1 is a flowchart of a method for auditing rich media content according to a first embodiment of the invention;
FIG. 2 is a flowchart of a method for auditing rich media content in accordance with a second embodiment of the invention;
FIG. 3 is a flow chart of the identification of the neural networks involved in the rich media content auditing method according to a second embodiment of the present invention;
FIG. 4 is a flowchart of a Huang Shenjing sub-network identification involved in a rich media content auditing method according to a second embodiment of the present invention;
FIG. 5 is a flowchart of advertising neural subnetwork identification in a rich media content auditing method according to a second embodiment of the invention;
FIG. 6 is a schematic diagram of a rich media content auditing apparatus according to a third embodiment of the invention;
fig. 7 is a schematic diagram of a server apparatus according to a fourth embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the following detailed description of the embodiments of the present invention will be given with reference to the accompanying drawings. However, those of ordinary skill in the art will understand that in various embodiments of the present invention, numerous technical details have been set forth in order to provide a better understanding of the present application. However, the technical solutions claimed in the present application can be implemented without these technical details and with various changes and modifications based on the following embodiments.
In the embodiment, scene classification is performed on the rich media content to be audited, and then the corresponding auditing method is used for auditing the rich media content according to different scene types, and when illegal content is contained in the rich media content, the auditing result of the content illegal is output. The following describes the method for auditing rich media content in this embodiment, and the following is only implementation details provided for easy understanding, but is not necessary for implementing this embodiment, and a specific flowchart of this embodiment is shown in fig. 1, and includes:
and step 101, obtaining rich media content to be audited.
It should be noted that, the object of the audit may be a picture, a video frame or a text. The video data can extract video frame data and then audit; text data may also be processed directly as picture data.
Step 102, it is determined whether the rich media content is a normal scene, if it is determined that the rich media content is a normal scene, step 103 is entered, and if it is determined that the rich media content is not a normal scene, step 104 is entered.
Specifically, before auditing the rich media content, the obtained rich media content is input into a pre-scene classification algorithm, and the scene classification algorithm can classify the scene of the rich media content through a lightweight neural network, for example, a ShuffleNet is adopted as a basic network for pre-scene judgment. The classification algorithm is characterized by high running speed of the lightweight neural network, so that the efficiency of identifying the scenes of the rich media content is greatly improved, the scene classification algorithm identifies that the rich media content is an identity card photo, various certificate photos and the like, the photo is used for judging that the rich media content is a common scene, the rich media content is judged to be not the common scene when the content is not the content, and then the rich media content is correspondingly sent to the next auditing step according to the scene information of the obtained rich media content.
And step 103, judging that the rich media content has no illegal content.
Specifically, after the front scene classification algorithm determines that the rich media content is a common scene, the data belonging to the common scene, such as an identity card photo, various certificate photos and the like, does not relate to illegal content, so that the rich media content does not need to be sent to other auditing steps for further auditing analysis, and the result that the rich media content does not contain illegal content can be directly obtained.
Step 104, it is determined whether the scene of the rich media content is a text scene, if it is determined that the rich media content is a text scene, step 105 is entered, and if it is determined that the rich media content is not a text scene, step 106 is entered.
Specifically, when the front scene judges that the rich media content does not belong to a common scene, identifying whether the rich media content is a text scene or not through a front scene classification algorithm, namely, the rich media content is a document or a document screenshot, a contract shooting picture, a table and other contents containing a large amount of text and digital information, and if the rich media content is the text scene, adopting an algorithm corresponding to the text scene to carry out identification and verification; if not, the rich media content is considered to belong to life scene categories, and the picture content identification mode is correspondingly adopted for auditing.
Step 105, detecting whether the rich media content is illegal using the text neural network.
Specifically, after the pre-scene classification algorithm determines that the rich media content is a text scene, the text neural network is adopted to check the text information in the rich media content, and OCR (Optical Character Recognition ) can be adopted to check whether the text content of the rich media content is illegal or not. Outputting the rich media content violations when the illegal content is judged to be contained, and outputting the rich media content without violations when the illegal content is not recognized in the auditing.
And step 106, detecting whether the rich media content is illegal or not by using the picture neural network.
Specifically, when the front scene classification algorithm judges that the rich media content belongs to a living scene, the rich media content is sent into the picture neural network to audit the rich media content, the picture neural network analyzes and identifies pictures in the rich media content, whether the rich media content contains pictures with illegal conditions such as anti-society, pornography, violence and advertisement or not is audited, when the illegal pictures are detected, the rich media content is judged to be illegal, and when the illegal pictures are not detected, the rich media content is judged to be free from illegal.
And step 107, outputting an auditing result.
Therefore, according to the obtained rich media content, the front scene classification algorithm achieves efficient classification of scenes of the rich media content in advance by means of the advantage of high operation speed of the lightweight neural network, and then according to scene classification results of the rich media content, examination results are directly output or corresponding examination methods are adopted to examine the rich media content of different scenes.
The second embodiment of the present invention relates to a method for auditing rich media content, which is substantially similar to the first embodiment, except that the second embodiment further refines auditing the rich media content by using a photo neural network, so that the auditing mode is more accurate and effective.
As shown in fig. 2, the specific flowchart of the present embodiment includes:
step 201, obtaining rich media content to be audited.
Step 202, it is determined whether the rich media content is a normal scene, if it is determined that the rich media content is a normal scene, step 203 is entered, and if it is determined that the rich media content is not a normal scene, step 204 is entered.
Step 203, it is determined that the rich media content has no illegal content.
Step 204, it is determined whether the scene of the rich media content is a text scene, if it is determined that the rich media content is a text scene, step 205 is entered, and if it is determined that the rich media content is not a text scene, step 206 is entered.
At step 205, a text neural network is used to audit whether the rich media content is illicit.
Steps 201 to 205 in this embodiment are similar to steps 101 to 105 in the first embodiment, and are not repeated here, and specific differences are described below.
Step 206, checking whether the rich media content is illegal or not by using the administrative neural network, if yes, proceeding to step 211, and if no, proceeding to step 207.
In one example, the verifying whether the rich media content is illegal by the neural network may be performed by a flowchart as shown in fig. 3, including:
sub-step 2061 involves the neural subnetwork identifying a probability of violation of the rich media content.
Specifically, a convolutional neural network such as a ResNet algorithm is used as a political neural sub-network to analyze the content of the picture, whether the picture contains the content related to politics violations is detected, and the violations of the rich media content are calculated according to the quantity and the specific definitions of the related information expressions in the rich media content.
Substep 2062, determining whether the probability of violation is greater than a first preset threshold, if so, proceeding to substep 2063, and if not, proceeding to substep 2064.
Specifically, auditing of administrative related content is realized by adopting a convolutional neural network, a data tag can be set to be political related and non-political related, and a first preset threshold and a second preset threshold corresponding to the political related and non-political related scope are set. When the violation probability of the rich media content is larger than a first preset threshold, the rich media content can be directly judged to be illegal, and the next audit is not needed; and when the rule violation probability of the rich media content is not greater than a first preset threshold, further auditing is carried out.
Substep 2063, outputting a violation; and determining the specific type of the violation of the rich media content or the absence of the violation according to the obtained violation determination result.
Sub-step 2064, judging whether the violation probability is greater than a second preset threshold, if so, entering sub-step 2065, and if not, entering sub-step 2066.
Specifically, when the probability of violation of the rich media content is not greater than the second preset threshold, the rich media content can be directly determined to be not violated, and the auditing is not required to be continued.
Sub-step 2065, manually checking whether the rich media content is illegal, if yes, proceeding to sub-step 2063, and if no, proceeding to sub-step 2066.
Specifically, when the violation probability of the rich media content is not greater than the first threshold value and is greater than the second threshold value, the system can not accurately judge whether the content is violated, and then the rich media content is further audited manually, so that the violation condition of the rich media content is determined, and an audit result is output.
A substep 2066 of performing a next auditing step; when the administrative neural sub-network judges that no violation exists, other neural sub-networks are submitted for further auditing.
Step 207, checking whether the rich media content is illegal or not by using the yellow-related neural network, if yes, proceeding to step 211, and if no, proceeding to step 208.
Specifically, when the Huang Shenjing sub-network is used for auditing the rich media content, a third preset threshold value and a fourth preset threshold value are set, and the violation condition of the rich media content is judged according to the relationship between the violation probability and the two threshold values, in addition, when the main Huang Shenjing sub-network is used for directly judging the violation of the rich media content, the auxiliary yellow-related sub-neural network is used for auditing again, and when the violation probability is in a probability value interval where the violation condition cannot be accurately judged, the auditing is performed manually, and a specific flow chart is shown in fig. 4.
In one example, the checking whether the rich media content is illegal may be performed by the yellow-involved neural network, which is shown in a flowchart of fig. 4, and includes:
sub-step 2071, adopting convolutional neural network as main involved Huang Shenjing sub-network to identify the offending probability of rich media content; specifically, the ResNet algorithm can be adopted to analyze the content of the picture, detect whether the picture contains picture elements such as limb exposure, pornography, sexual implication and the like and text information, and calculate the violation probability of the rich media content according to the quantity and specific definition of related information expression in the rich media content.
Sub-step 2072, judging whether the rule breaking probability is greater than a fourth preset threshold, if the rule breaking probability is greater than the fourth preset threshold, entering sub-step 2073, and if the rule breaking probability is not greater than the fourth preset threshold, entering sub-step 2074.
Specifically, the fourth preset threshold is set according to the minimum value corresponding to the probability that the violation probability of the rich media content can be directly determined as the violation, and when the violation probability of the rich media content is greater than the fourth preset threshold, the rich media content can be directly determined as the violation, and the rich media content is submitted to the auxiliary Huang Shenjing subnetwork for further auditing; when the rule violation probability of the rich media content is not larger than the fourth preset threshold, the rule violation of the rich media content cannot be directly judged, and the auditing needs to be continued.
Sub-step 2073 assists Huang Shenjing in identifying whether the sub-network is illegal, if yes, then sub-step 2076 is entered, and if no, then sub-step 2077 is entered.
In particular, when the sub-network Huang Shenjing is mainly involved in judging the violation of the rich media content, in consideration of the actual situation in life, the rich media content in some special occasions may be misjudged, for example, the whole hand or other normal large-area body is exposed, and such image data is easier to be misidentified as a violation image. In order to avoid misjudgment, the auxiliary Huang Shenjing sub-network performs secondary identification on the rich media content, for example, a FasterRCNN algorithm is adopted to perform auditing, and whether the rich media content is misjudgment of pictures such as arms, lower legs, faces and the like is detected. If the auxiliary Huang Shenjing sub-network identification result is still illegal, the judgment result is illegal, otherwise, the judgment result is not illegal.
Sub-step 2074, judging whether the rule breaking probability is greater than a third preset threshold, if the rule breaking probability is greater than the third preset threshold, entering sub-step 2075, and if the rule breaking probability is not greater than the third preset threshold, entering sub-step 2077.
Specifically, the third preset threshold is set according to the maximum value corresponding to the probability of violation that the probability of violation can be directly determined to be not illegal, when the probability of violation of the rich media content is not greater than the third preset threshold, the rich media content can be directly determined to be not illegal without further auditing, and when the probability of violation of the rich media content is greater than the third preset threshold, the probability of violation of the rich media content cannot be directly determined to be not illegal, and further auditing is required.
Sub-step 2075, manually auditing if the rule is violated, if yes, proceeding to sub-step 2076, if no rule is violated, proceeding to sub-step 2077.
Specifically, when the violation probability of the rich media content is in a section where the violation situation cannot be directly judged, the rich media content is further audited manually, the violation situation of the rich media content is determined, when the violation of the rich media content is judged, the violation situation is output, and when the violation of the rich media content is judged, the next audit is carried out.
Sub-step 2076, outputting a violation; and determining the specific type of the violation of the rich media content or the absence of the violation according to the obtained violation determination result.
A substep 2077, performing the next auditing step; when Huang Shenjing subnetworks determine that no violation exists, other neural subnetworks are submitted for further auditing.
Step 208, checking whether the rich media content is illegal or not by using the riot neural network, if yes, proceeding to step 211, and if no, proceeding to step 209.
Specifically, the riot neural network analyzes and identifies the rich media content, detects whether picture elements of violence, bloody smell, explosion and the like are contained in the picture, judges that the rich media content is illegal when the rich media content is detected to contain relevant riot elements, outputs the illegal situation, judges that the rich media content is not illegal when the rich media content does not contain the riot elements, and carries out other types of auditing.
Step 209, detecting whether the rich media content is illegal or not by using the advertising neural sub-network, if yes, proceeding to step 211, and if no, proceeding to step 210.
Specifically, when the advertisement neural sub-network examines the rich media content, the front text neural sub-network examines the rich media content, when the front text neural sub-network judges that the rich media content is illegal, the front text neural sub-network directly outputs the illegal situation, when the front text neural sub-network judges that the advertisement neural sub-network is not illegal, the advertisement neural sub-network examines, when the front text neural sub-network judges that the rich media content is illegal, the advertisement neural sub-network judges that the rich media content is illegal, the illegal situation is output, when the rich media content is judged that the rich media content is not illegal, the illegal situation is not output. The specific flow chart is shown in fig. 5, and includes:
step 2091, the pre-text neural sub-network identifies whether the rule is violated, if yes, step 2092 is entered, and if no rule is entered, step 2093 is entered.
Specifically, OCR (Optical Character Recognition ) may be used to analyze the rich media content, detect whether the text in the picture contains obvious advertisement elements such as phone numbers and mailbox numbers, determine that the rich media content is illegal when detecting that the text elements contain violations, directly output the violations, and determine that the rich media content is not illegal when not detecting the violations, and perform the next audit.
Step 2092, outputting a violation; and determining the specific type of the violation of the rich media content or the absence of the violation according to the obtained violation determination result.
In step 2093, the advertising neural sub-network identifies whether the rule is violated, if yes, the process proceeds to step 2092, and if no rule is violated, the process proceeds to step 2094.
Specifically, the advertisement neural sub-network analyzes the rich media content, detects whether the picture contains picture elements such as improper popularization, hot line promotion, false promotion and the like or not, judges that the rich media content is illegal when detecting that the picture contains related elements which are illegal, directly outputs the illegal situation, judges that the rich media content is not illegal when not detecting the related elements which are illegal, and enters the next step.
Step 2094, the output rich media content is free of violations.
Step 210, it is determined that there is no offending content.
Step 211, determining that there is illegal content.
Specifically, when each sub-neural network is audited, any sub-neural network audits that the rich media content contains illegal content, the illegal result of the rich media content is directly output, and according to the specific sub-neural network of which the illegal result is output, the specific illegal situation of the rich media content is output according to the type of the illegal to which the picture belongs.
And step 212, outputting an audit result, and outputting a specific violation type or non-violation of the rich media content according to the obtained violation condition of the rich media content.
It should be noted that, a specific implementation method for auditing rich media content by using a photo neural network is provided herein, in practical application, the classification mode related to illegal content and the auditing sequence of each class can be changed according to actual needs, and the embodiment is not limited.
In the embodiment, all types of audit neural networks are ordered according to the corresponding priority and audit requirements, the rich media content is audited sequentially, in the audit of all the neural networks, different methods are adopted for improving all the neural networks, a threshold value is set to definitely give out a judging standard to improve the audit efficiency, meanwhile, manual audit and auxiliary neural networks are added in consideration of the limitation of a machine to ensure the audit accuracy, misjudgment is avoided, the audit efficiency of the advertising neural network is improved through simpler text neural network audit before audit, specific violation types are directly determined before audit results are output through detection of violation result sources on the premise of adopting a serial and parallel operation mode, the occurrence probability of each violation type is counted again in combination with specific audit requirements during output, the audit efficiency is greatly improved, meanwhile, the audit results are more definitely and more humanized due to the fact that the violation types can be directly output, and labor cost is saved.
It is not difficult to find that the above embodiments can be used in cooperation with each other, related technical details in the embodiments can be cited with each other, in the process of auditing the picture neural network, the priority of each neural network can be adjusted according to requirements, and the steps of the above methods are divided, only for clarity of description, the above embodiments can be combined into one step or split certain steps into a plurality of steps when implemented, and the steps are all within the protection scope of the patent as long as the steps comprise the same logic relationship; it is within the scope of this patent to add insignificant modifications to the algorithm or flow or introduce insignificant designs, but not to alter the core design of its algorithm and flow.
A third embodiment of the present invention relates to an auditing apparatus for rich media content, including:
601, scene classification unit: and acquiring the rich media content information to be identified, classifying the scenes of the rich media content, and transmitting the classification result to the auditing unit.
602, auditing unit: and obtaining classification results of the scene classification unit, auditing the rich media content by adopting a corresponding auditing method according to different scene types, obtaining auditing results, and transmitting the auditing results to the output unit.
603, an output unit: outputting the recognized type of violation or the result of no violation.
In one specific example, scene types include a general scene, a text scene, and a living scene. The auditing unit is specifically used for judging that the rich media content does not contain illegal content when the scene type is a common scene; and when the scene type is a text scene, carrying out content identification on the rich media content by adopting a text neural network, and judging whether the rich media content contains illegal content.
In a specific example, the image neural network includes N neural sub-networks, where N is a natural number greater than 1, and each neural sub-network corresponds to a priority level; when the picture neural network is adopted to identify the content of the rich media content, the method comprises the following steps: sequentially adopting each neural sub-network to identify the content of the rich media according to the order of the priorities of the neural sub-networks from large to small; and if the identification result of any neural subnetwork is that the illegal content is contained, judging that the rich media content contains the illegal content.
In a specific example, N neural subnetworks in the photo neural network include: a political neural subnetwork, a Huang Shenjing subnetwork, an riot neural subnetwork, and an advertising neural subnetwork.
In a specific example, the content identification of the rich media content using the political neural sub-network includes: inputting the rich media content into the political affair-related neural sub-network to obtain political affair-related probability output by the political affair-related neural sub-network; when the administrative probability is larger than a preset first preset threshold value, judging that the rich media content contains illegal content; when the administrative probability is smaller than a second preset threshold value, judging that the rich media content does not contain illegal content; wherein the first preset threshold is greater than the second preset threshold; and when the administrative probability is larger than a second preset threshold and smaller than the first preset threshold, performing manual auditing, and judging whether the rich media content contains illegal content according to the manual auditing result.
In a specific example, the sub-networks involved Huang Shenjing include a primary involved Huang Shenjing sub-network and a secondary involved Huang Shenjing sub-network; content identification of rich media content using a Huang Shenjing sub-network, comprising: inputting the rich media content into a main wading Huang Shenjing sub-network to obtain a wading Huang Gailv output by the main wading Huang Shenjing sub-network; when Huang Gailv is smaller than a third preset threshold value, judging that the rich media content does not contain illegal content; when the yellow-related probability is larger than a fourth preset threshold value, carrying out content identification on the rich media content by adopting the auxiliary Huang Shenjing sub-network; wherein the fourth preset threshold is greater than the third preset threshold; and when the administrative probability is larger than a third preset threshold and smaller than a fourth preset threshold, performing manual auditing, and judging whether the rich media content contains illegal content according to the manual auditing result.
In one specific example, the advertising neural subnetwork includes a text neural subnetwork and a primary advertising neural subnetwork; the method for identifying the content of the rich media content by adopting the advertising neural sub-network comprises the following steps: performing content identification on the rich media content by adopting a text neural sub-network; if the identification result is that the illegal content is contained, judging that the rich media content contains the illegal content; and if the identification result is that the illegal content is not contained, carrying out content identification on the rich media content by adopting a main advertising neural sub-network.
In a specific example, the scene classification of the rich media content to be audited specifically includes: and carrying out scene classification on the rich media content to be audited by adopting a preset lightweight neural network.
It should be understood that each module related to this embodiment is a logic module, and in practical application, one logic unit may be one physical unit, or may be a part of one physical unit, or may be implemented by a combination of multiple physical units. In addition, in order to highlight the innovative part of the present invention, units that are not so close to solving the technical problem presented by the present invention are not introduced in the present embodiment, but this does not indicate that other units are not present in the present embodiment.
The embodiment provides an auditing device of rich media content, which can realize the output of the auditing and identifying result from the acquisition of data information to the final completion of the whole identifying process.
A fourth embodiment of the present invention is directed to a server, as shown in fig. 7, comprising at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor to enable the at least one processor to perform an embodiment of the method of auditing rich media content.
Where the memory and the processor are connected by a bus, the bus may comprise any number of interconnected buses and bridges, the buses connecting the various circuits of the one or more processors and the memory together. The bus may also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art, and therefore, will not be described any further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or may be a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor is transmitted over the wireless medium via the antenna, which further receives the data and transmits the data to the processor.
The processor is responsible for managing the bus and general processing and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And memory may be used to store data used by the processor in performing operations.
A fifth embodiment of the present invention relates to a computer-readable storage medium storing a computer program which, when executed by a processor, implements the above-described method embodiments.
That is, it will be understood by those skilled in the art that all or part of the steps in implementing the methods of the embodiments described above may be implemented by a program stored in a storage medium, where the program includes several instructions for causing a device (which may be a single-chip microcomputer, a chip or the like) or a processor (processor) to perform all or part of the steps in the methods of the embodiments described herein. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
It will be understood by those of ordinary skill in the art that the foregoing embodiments are specific examples of carrying out the invention and that various changes in form and details may be made therein without departing from the spirit and scope of the invention.
Claims (9)
1. A method for auditing rich media content, comprising:
performing scene classification on the rich media content to be audited by adopting a preset lightweight neural network to obtain scene types of the rich media content, wherein the scene types comprise a common scene, a text scene and a living scene; the common scene is a scene comprising an identity card photo or various certificate photos; the text scene is a scene of content containing text or digital information;
checking the rich media content corresponding to the scene type according to the scene type, wherein if the scene type is a common scene, the rich media content is judged to not contain illegal content; if the scene type does not belong to a common scene, identifying whether the rich media content is a text scene through the lightweight neural network, if the scene type is a text scene, carrying out content identification on the rich media content by adopting the text neural network, and judging whether the rich media content contains illegal content; if the scene type is not a text scene, the rich media content is a live scene, and a picture neural network is adopted to identify the content of the rich media content, so as to judge whether the rich media content contains illegal content or not;
and if the rich media content contains illegal content, outputting a content illegal auditing result.
2. The method for auditing rich media content according to claim 1, wherein the picture neural network includes N neural sub-networks, where N is a natural number greater than 1, and each neural sub-network corresponds to a priority;
the content identification of the rich media content by adopting the picture neural network comprises the following steps:
sequentially adopting each neural sub-network to identify the content of the rich media according to the order of the priority of the neural sub-networks from big to small; and if the identification result of any neural subnetwork is that the illegal content is contained, judging that the rich media content contains the illegal content.
3. The method of auditing rich media content according to claim 2, wherein the N neural subnetworks comprise: a political neural subnetwork, a Huang Shenjing subnetwork, an riot neural subnetwork, and an advertising neural subnetwork.
4. The method of auditing rich media content according to claim 3, characterized in that using the political neural sub-network to perform content identification on the rich media content comprises:
inputting the rich media content into the political affairs-related neural sub-network to obtain political affairs-related probability output by the politics-related neural sub-network;
when the administrative probability is larger than a preset first preset threshold value, judging that the rich media content contains illegal content;
when the administrative probability is smaller than a second preset threshold value, judging that the rich media content does not contain illegal content; wherein the first preset threshold is greater than the second preset threshold;
and when the administrative probability is larger than the second preset threshold and smaller than the first preset threshold, performing manual auditing, and judging whether the rich media content contains illegal content according to the manual auditing result.
5. A method of auditing rich media content according to claim 3, in which the sub-networks involved Huang Shenjing include a primary involved Huang Shenjing sub-network and a secondary involved Huang Shenjing sub-network; performing content identification on the rich media content by adopting the Huang Shenjing sub-network, including:
inputting the rich media content into the main wading Huang Shenjing sub-network to obtain a wading Huang Gailv output by the main wading Huang Shenjing sub-network;
when the wading Huang Gailv is smaller than a third preset threshold value, judging that the rich media content does not contain illegal content;
when the yellow probability is larger than a fourth preset threshold value, adopting the auxiliary Huang Shenjing sub-network to identify the content of the rich media content; wherein the fourth preset threshold is greater than the third preset threshold;
and when the administrative probability is larger than the third preset threshold and smaller than the fourth preset threshold, performing manual auditing, and judging whether the rich media content contains illegal content according to the manual auditing result.
6. The method of auditing rich media content according to claim 3, wherein the advertising neural subnetwork comprises a text neural subnetwork and a primary advertising neural subnetwork; and carrying out content identification on the rich media content by adopting the advertising neural sub-network, wherein the content identification comprises the following steps:
performing content identification on the rich media content by adopting the text neural sub-network;
if the identification result is that the illegal content is contained, judging that the rich media content contains the illegal content; and if the identification result is that the illegal content is not contained, carrying out content identification on the rich media content by adopting the main advertising neural sub-network.
7. An auditing apparatus for rich media content, comprising:
the scene classification unit is used for classifying scenes of the rich media content to be audited by adopting a preset lightweight neural network to obtain scene types of the rich media content, wherein the scene types comprise a common scene, a text scene and a life scene; the common scene is a scene comprising an identity card photo or various certificate photos; the text scene is a scene of content containing text or digital information;
the auditing unit is used for auditing the rich media content corresponding to the scene type according to the scene type, wherein if the scene type is a common scene, the rich media content is judged to not contain illegal content; if the scene type does not belong to a common scene, identifying whether the rich media content is a text scene through the lightweight neural network, if the scene type is a text scene, carrying out content identification on the rich media content by adopting the text neural network, and judging whether the rich media content contains illegal content; if the scene type is not a text scene, the rich media content is a live scene, and a picture neural network is adopted to identify the content of the rich media content, so as to judge whether the rich media content contains illegal content or not;
and the output unit is used for outputting the auditing result of the content violation when the auditing unit judges that the rich media content contains the violation content.
8. A server, comprising:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of auditing rich media content according to any one of claims 1 to 6.
9. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the method of auditing rich media content according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911183179.8A CN110956123B (en) | 2019-11-27 | 2019-11-27 | Method, device, server and storage medium for auditing rich media content |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911183179.8A CN110956123B (en) | 2019-11-27 | 2019-11-27 | Method, device, server and storage medium for auditing rich media content |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110956123A CN110956123A (en) | 2020-04-03 |
| CN110956123B true CN110956123B (en) | 2024-02-27 |
Family
ID=69978604
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911183179.8A Active CN110956123B (en) | 2019-11-27 | 2019-11-27 | Method, device, server and storage medium for auditing rich media content |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110956123B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112380364A (en) * | 2020-11-17 | 2021-02-19 | 平安养老保险股份有限公司 | Method and system for file authentication |
| CN112818150B (en) * | 2021-01-22 | 2024-05-07 | 天翼视联科技有限公司 | Picture content auditing method, device, equipment and medium |
| CN113095178A (en) * | 2021-03-30 | 2021-07-09 | 北京大米科技有限公司 | Bad information detection method, system, electronic device and readable storage medium |
| CN114095882A (en) * | 2021-12-06 | 2022-02-25 | 联通在线信息科技有限公司 | 5G message content detection method |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20100084271A (en) * | 2009-01-16 | 2010-07-26 | 김휘진 | Movie contents protection apparatus and method by intermittent insertion of identification code, and computer-readable medium storing program for method thereof |
| CN105654057A (en) * | 2015-12-31 | 2016-06-08 | 中国建设银行股份有限公司 | Picture auditing system and picture auditing method based on picture contents |
| CN108124191A (en) * | 2017-12-22 | 2018-06-05 | 北京百度网讯科技有限公司 | A kind of video reviewing method, device and server |
| CN108170813A (en) * | 2017-12-29 | 2018-06-15 | 智搜天机（北京）信息技术有限公司 | A kind of method and its system of full media content intelligent checks |
| CN108419091A (en) * | 2018-03-02 | 2018-08-17 | 北京未来媒体科技股份有限公司 | A kind of verifying video content method and device based on machine learning |
| CN108960782A (en) * | 2018-07-10 | 2018-12-07 | 北京木瓜移动科技股份有限公司 | content auditing method and device |
| WO2019062080A1 (en) * | 2017-09-28 | 2019-04-04 | 平安科技（深圳）有限公司 | Identity recognition method, electronic device, and computer readable storage medium |
| CN109660823A (en) * | 2018-12-28 | 2019-04-19 | 广州华多网络科技有限公司 | Video distribution method, apparatus, electronic equipment and storage medium |
| CN109670055A (en) * | 2018-11-30 | 2019-04-23 | 广州市百果园信息技术有限公司 | A kind of multi-medium data checking method, device, equipment and storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW556426B (en) * | 2000-12-28 | 2003-10-01 | Trustview Inc | System and method for registration on multiple different devices using the same account |
| US20070124319A1 (en) * | 2005-11-28 | 2007-05-31 | Microsoft Corporation | Metadata generation for rich media |
| CN104683852B (en) * | 2013-11-29 | 2018-04-06 | 国际商业机器公司 | The method and apparatus for handling broadcast message |
-
2019
- 2019-11-27 CN CN201911183179.8A patent/CN110956123B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20100084271A (en) * | 2009-01-16 | 2010-07-26 | 김휘진 | Movie contents protection apparatus and method by intermittent insertion of identification code, and computer-readable medium storing program for method thereof |
| CN105654057A (en) * | 2015-12-31 | 2016-06-08 | 中国建设银行股份有限公司 | Picture auditing system and picture auditing method based on picture contents |
| WO2019062080A1 (en) * | 2017-09-28 | 2019-04-04 | 平安科技（深圳）有限公司 | Identity recognition method, electronic device, and computer readable storage medium |
| CN108124191A (en) * | 2017-12-22 | 2018-06-05 | 北京百度网讯科技有限公司 | A kind of video reviewing method, device and server |
| CN108170813A (en) * | 2017-12-29 | 2018-06-15 | 智搜天机（北京）信息技术有限公司 | A kind of method and its system of full media content intelligent checks |
| CN108419091A (en) * | 2018-03-02 | 2018-08-17 | 北京未来媒体科技股份有限公司 | A kind of verifying video content method and device based on machine learning |
| CN108960782A (en) * | 2018-07-10 | 2018-12-07 | 北京木瓜移动科技股份有限公司 | content auditing method and device |
| CN109670055A (en) * | 2018-11-30 | 2019-04-23 | 广州市百果园信息技术有限公司 | A kind of multi-medium data checking method, device, equipment and storage medium |
| CN109660823A (en) * | 2018-12-28 | 2019-04-19 | 广州华多网络科技有限公司 | Video distribution method, apparatus, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 郦荣.基于人工智能技术的富媒体信息管控研究.《电信工程技术与标准化,08》.2017,正文第1-4节以及图1-2. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110956123A (en) | 2020-04-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110956123B (en) | Method, device, server and storage medium for auditing rich media content | |
| CN106357618B (en) | Web anomaly detection method and device | |
| CN112861648B (en) | Character recognition method, character recognition device, electronic equipment and storage medium | |
| CN111382291B (en) | Machine auditing method and device and machine auditing server | |
| CN111090776A (en) | Video auditing method, device, auditing server and storage medium | |
| CN106376002B (en) | Management method and device and spam monitoring system | |
| CN111583180B (en) | Image tampering identification method and device, computer equipment and storage medium | |
| CN103064987A (en) | Bogus transaction information identification method | |
| CN113051500B (en) | Phishing website identification method and system fusing multi-source data | |
| US20230215125A1 (en) | Data identification method and apparatus | |
| CN110619535A (en) | Data processing method and device | |
| CN110020161B (en) | Data processing method, log processing method and terminal | |
| CN109670480B (en) | Image discrimination method, device, equipment and storage medium | |
| CN117392577A (en) | Behavior recognition method for judicial video scene, storage medium and electronic device | |
| CN113888760B (en) | Method, device, equipment and medium for monitoring violation information based on software application | |
| CN113569839B (en) | Certificate identification method, system, equipment and medium | |
| CN114241253A (en) | Model training method, system, server and storage medium for illegal content identification | |
| CN112989869B (en) | Optimization method, device, equipment and storage medium of face quality detection model | |
| CN113449506A (en) | Data detection method, device and equipment and readable storage medium | |
| CN113255758A (en) | Network information security detection method and device | |
| CN114362982A (en) | Flow subdivision identification method, system, electronic device and storage medium | |
| CN112257768A (en) | Method and device for identifying illegal financial picture and computer storage medium | |
| CN111291726A (en) | Medical bill sorting method, device, equipment and medium | |
| CN111753926A (en) | Data sharing method and system for smart city | |
| CN115361242B (en) | Web attack detection method based on multidimensional feature network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |