CN117037134A - Method and device for detecting application in barrier-free mode - Google Patents
Method and device for detecting application in barrier-free mode Download PDFInfo
- Publication number
- CN117037134A CN117037134A CN202311072616.5A CN202311072616A CN117037134A CN 117037134 A CN117037134 A CN 117037134A CN 202311072616 A CN202311072616 A CN 202311072616A CN 117037134 A CN117037134 A CN 117037134A
- Authority
- CN
- China
- Prior art keywords
- page
- detected
- detection frame
- detection
- framed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000001514 detection method Methods 0.000 claims abstract description 245
- 238000010586 diagram Methods 0.000 claims description 23
- 238000003860 storage Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 15
- 230000009191 jumping Effects 0.000 claims description 4
- 238000002360 preparation method Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 description 16
- 230000006870 function Effects 0.000 description 14
- 238000005516 engineering process Methods 0.000 description 9
- 230000006872 improvement Effects 0.000 description 9
- 238000012545 processing Methods 0.000 description 6
- 230000004888 barrier function Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 238000012015 optical character recognition Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 230000001771 impaired effect Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 229920001296 polysiloxane Polymers 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000010979 ruby Substances 0.000 description 1
- 229910001750 ruby Inorganic materials 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 208000029257 vision disease Diseases 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/19—Recognition using electronic means
- G06V30/191—Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
- G06V30/19173—Classification techniques
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
- G10L13/08—Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The specification discloses a method and a device for detecting an application in a barrier-free mode, wherein an application APP to be detected is operated, the barrier-free mode is started, a detection frame is displayed in a target page in the application APP to be detected, text recognition is carried out on page contents framed in the detection frame to obtain first text information, voice broadcast is carried out on page contents framed in the detection frame, and voice recognition is carried out on the broadcasted voice information to obtain second text information, so that the page contents framed in the detection frame are detected, after detection is completed, the detection frame is controlled to move in the target page, and the next page contents framed in the target page by the detection frame are detected until detection of all contents in the target page is completed. Compared with manual detection, the method not only greatly improves the detection efficiency, but also can effectively avoid subjective influence caused by manual detection, thereby obtaining accurate detection results.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method and an apparatus for detecting an application in an unobstructed mode.
Background
With the continuous development of science and technology, various emerging technologies are applied to daily business activities of people, for example, using artificial intelligence to protect private data of users, or recommending information to users through artificial intelligence, etc. In addition to enriching the needs of normal users, the actual needs and business experiences of specific people are more focused at present, so that an unobstructed mode is generated.
The barrier-free mode is a mode for providing users with visual barriers to use, and assists the users with visual barriers to broadcast page contents displayed in a screen in a voice mode when browsing the page contents by using terminal equipment such as a mobile phone, a tablet computer and the like, so that the users with visual barriers can know specific page contents.
There are many Applications (APP) currently that are not friendly to the viewing population. In the barrier-free mode, many APP functions can be inoperable or information broadcasting errors occur. In order to better improve the barrier-free mode, currently, an APP in the barrier-free mode is usually required to be detected in a manual mode, and the accuracy of broadcasting content is judged.
However, when the function of the APP is more and more complex, the APP is simply detected by means of manual mode, which is very difficult, not only is time-consuming and long, but also has stronger subjectivity when the comparison broadcast content is consistent with the original content on the APP, the condition of low accuracy of the detection result can occur, and the development of barrier-free mode is severely limited.
Based on this, how to improve the accuracy and the detection efficiency of the barrier-free mode detection is a problem to be solved urgently.
Disclosure of Invention
The present disclosure provides a method and apparatus for detecting an application in an unobstructed mode, so as to partially solve the above-mentioned problems in the prior art.
The technical scheme adopted in the specification is as follows:
the present specification provides a method of detecting an application in an unobstructed mode, comprising:
running an application APP to be detected, and starting an obstacle-free mode;
displaying a target page in the APP to be detected;
displaying a detection frame in the target page based on the barrier-free mode;
performing text recognition on the page content framed in the detection frame to obtain first text information, performing voice broadcasting on the page content framed in the detection frame, and performing voice recognition on the broadcasted voice information to obtain second text information;
Detecting page contents framed by the detection frame according to the first text information and the second text information, and after detection is completed, controlling the detection frame to move in the target page so as to detect next page contents framed by the detection frame in the target page until detection of all page contents contained in the target page is completed, and detecting other target pages in the APP to be detected.
Optionally, detecting other target pages in the APP to be detected specifically includes:
and according to a preset page relation diagram, jumping the target page to other target pages in the APP to be detected, and detecting the other target pages, wherein the page relation diagram is used for recording the jumping relation among the pages contained in the APP to be detected.
Optionally, text recognition is performed on the page content framed in the detection frame to obtain first text information, which specifically includes:
when the content framed in the detection frame contains picture information, identifying the text content framed in the detection frame to obtain first sub-text information;
Performing image recognition on the picture information, and determining second sub-text information aiming at the picture information according to the obtained recognition result and the context information of the picture information in the content framed by the detection frame;
and obtaining the first text information according to the first sub-text information and the second sub-text information.
Optionally, before performing text recognition on the page content framed in the detection frame to obtain the first text information, the method specifically includes:
determining a page area to be detected where the detection frame is located on the target page;
judging whether the page area to be detected is detected or not according to the stored detection record;
and judging whether the range framed by the detection frame is matched with the coverage range of the page area to be detected under the condition that the page area to be detected is not detected.
Optionally, text recognition is performed on the page content framed in the detection frame to obtain first text information, which specifically includes:
and under the condition that the range framed by the detection frame is matched with the coverage area of the page area to be detected, carrying out character recognition on the page content framed currently in the detection frame so as to obtain first character information.
Optionally, the method further comprises:
and under the condition that the range framed by the detection frame is not matched with the coverage area of the page area to be detected, adjusting the position of the detection frame in the target page, and judging whether the adjusted range framed by the detection frame is matched with the coverage area of the page area to be detected or not until the range framed by the detection frame is adjusted to be matched with the coverage area of the page area to be detected.
The present specification provides an apparatus for detecting an application in an unobstructed mode, comprising:
the starting module runs the application APP to be detected and starts an obstacle-free mode;
the display module displays the target page in the APP to be detected;
the preparation module is used for displaying a detection frame in the target page based on the barrier-free mode;
the recognition module is used for carrying out word recognition on the page content framed in the detection frame to obtain first word information, carrying out voice broadcasting on the page content framed in the detection frame, and carrying out voice recognition on the broadcasted voice information to obtain second word information;
the detection module is used for detecting page contents framed by the detection frame according to the first text information and the second text information, controlling the detection frame to move in the target page after detection is completed, detecting next page contents framed by the detection frame in the target page until detection of all page contents contained in the target page is completed, and detecting other target pages in the APP to be detected.
Optionally, the detection module is specifically configured to skip the target page to other target pages in the APP to be detected according to a preset page relationship diagram, and detect the other target pages, where the page relationship diagram is used to record a skip relationship between pages included in the APP to be detected.
Optionally, the identifying module is specifically configured to identify text content framed by the detecting frame when the content framed by the detecting frame includes picture information, so as to obtain first sub-text information; performing image recognition on the picture information, and determining second sub-text information aiming at the picture information according to the obtained recognition result and the context information of the picture information in the content framed by the detection frame; and obtaining the first text information according to the first sub-text information and the second sub-text information.
Optionally, before the identifying module identifies the text of the page content framed in the detecting frame to obtain the first text information, the detecting module is further configured to determine a to-be-detected page area where the detecting frame is located on the target page; judging whether the page area to be detected is detected or not according to the stored detection record; judging whether the range framed by the detection frame is matched with the coverage range of the page area to be detected under the condition that the page area to be detected is not detected;
The identification module is specifically configured to perform text identification on the currently framed page content in the detection frame to obtain first text information when the detection module determines that the range framed by the detection frame matches with the coverage area of the page area to be detected.
Optionally, when the detection module determines that the range defined by the detection frame does not match the coverage area of the to-be-detected page area, the detection module is specifically configured to adjust a position of the detection frame in the target page, and determine whether the adjusted range defined by the detection frame matches the coverage area of the to-be-detected page area, until the range defined by the detection frame is adjusted to match the coverage area of the to-be-detected page area.
The present specification provides a computer readable storage medium storing a computer program which when executed by a processor implements the above method of detecting an application in a barrier-free mode.
The present specification provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing any of the methods of detecting an application in an unobstructed mode described above when executing the program.
The above-mentioned at least one technical scheme that this specification adopted can reach following beneficial effect:
in the method for detecting the application in the barrier-free mode provided by the specification, the application APP to be detected is operated, the barrier-free mode is started, a detection frame is displayed in a target page in the application APP to be detected, text recognition is carried out on page contents framed in the detection frame to obtain first text information, voice broadcasting is carried out on page contents framed in the detection frame, voice recognition is carried out on the broadcasted voice information to obtain second text information, the page contents framed in the detection frame are detected, after detection is completed, the detection frame is controlled to move in the target page, the next page contents framed in the target page by the detection frame are detected until detection of all page contents contained in the target page is completed, and detection is carried out on other target pages in the APP to be detected.
According to the method, when the APP is detected in the barrier-free mode, the target page content in the APP is automatically detected in the barrier-free mode by adopting the mode of carrying out character recognition on the content in the detection frame and carrying out voice recognition on the broadcast content, so that compared with manual detection, the detection efficiency can be greatly improved, and moreover, the application in the barrier-free mode is detected by comparing the content defined by the detection frame with the broadcast content, the subjective influence caused by manual detection can be effectively avoided, and an accurate detection result can be obtained.
Drawings
The accompanying drawings, which are included to provide a further understanding of the specification, illustrate and explain the exemplary embodiments of the present specification and their description, are not intended to limit the specification unduly. In the drawings:
FIG. 1 is a flow chart of a method for detecting applications in barrier-free mode provided in the present specification;
fig. 2A and fig. 2B are schematic views of a process provided in the present specification for adjusting a range defined by a detection frame until the range matches with a coverage area of a to-be-detected page area;
FIG. 3 is a detailed flow chart of detecting an application in an unobstructed mode provided in this specification;
FIG. 4 is a schematic diagram of an apparatus for detecting applications in barrier-free mode provided herein;
fig. 5 is a schematic diagram of an electronic device corresponding to fig. 1 provided in the present specification.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the present specification more apparent, the technical solutions of the present specification will be clearly and completely described below with reference to specific embodiments of the present specification and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present specification. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are intended to be within the scope of the present disclosure.
The following describes in detail the technical solutions provided by the embodiments of the present specification with reference to the accompanying drawings.
Fig. 1 is a flow chart of a method for detecting an application in an unobstructed mode provided in this specification, including the following steps:
s101: and running the application APP to be detected, and starting an obstacle-free mode.
The barrier-free mode is mainly used for realizing a screen reading function and supporting users with vision disorder to read contents on a screen through a voice synthesis technology. When the function is started, any page content touched by the visually impaired person on the terminal equipment forms corresponding voice broadcasting, and the content of the voice broadcasting comprises all text and picture information on a screen. Therefore, the barrier-free mode can improve usability and accessibility of the product, enabling more users to enjoy the convenience of technology and services.
However, at present, in many APP's under the accessible mode can appear reporting the inaccurate condition, in order to guarantee the accurate operation of accessible mode, need detect accessible mode, promote accessible mode's improvement, but at present all detect accessible mode report the accuracy of content through the manual work, however, the mode of manual work detects not only inefficiency, and is influenced by the dominant fact moreover, its detection accuracy rate often is not high.
Based on the above, the present disclosure provides a method for detecting an application in a barrier-free mode, where barrier-free detection is performed on a target page, the content in a detection frame is identified, and then the content is converted into text information, and meanwhile, the information in the detection frame is broadcasted in a voice manner, and then the voice information is converted into text information, and further, the text information identified in the detection frame is compared with the voice broadcasted information. The detection method for the barrier-free mode does not need manual intervention, greatly improves the detection efficiency, and simultaneously eliminates the influence of subjective factors existing in manual detection, thereby improving the accuracy of the barrier-free mode detection and ensuring that the barrier-free mode can be smoothly improved.
In this specification, the execution body for implementing a method for detecting an application in an unobstructed mode may be a terminal device such as a mobile phone, a tablet computer, a notebook computer, or the like, and of course, may also be a client installed on the terminal device, where if it is a client, the APP to be detected and the client may be installed in the same terminal device. For convenience of description, the present specification will be described with reference to a client as an execution subject, and a method for detecting an application in an unobstructed mode provided in the present specification will be described.
In the process of barrier-free detection, the client needs to be started first, and after a detection instruction is acquired, a barrier-free mode in terminal equipment setting is started, and a specific APP to be detected is positioned at the same time and started. In this specification, there are various ways to receive the detection instruction, for example, the detection instruction is sent by the remote device or the server, so that the client obtains the detection instruction; for another example, in the page triggering detection, an APP to be detected is selected, and after confirmation operation is executed, the client obtains the detection instruction, and the APP to be detected is detected according to the detection instruction.
S102: and displaying the target page in the APP to be detected.
In the present specification, for an APP to be detected, the target page may refer to any page in the APP to be detected, that is, when performing barrier-free detection on an APP, barrier-free detection needs to be performed on all target pages.
When the client starts the APP to be detected, the target page is displayed, wherein the target page can be the first page displayed after the APP to be detected is started, and can also be a page jumped by other pages. That is, when a link to be jumped is encountered in the detection process, the jump link is jumped to another page, and the jumped page is the target page, so that the detection of the target page is continued, and when the detection is completed, the previous target page can be jumped back to continue to complete the detection.
S103: and displaying a detection frame in the target page based on the barrier-free mode.
When the barrier-free mode is opened, a detection frame appears in the target page, and the detection frame is used for framing the page content to be detected so as to compare the information in the frame with the broadcast content, and in the subsequent detection, all the content in the target page is detected by controlling the movement of the detection frame.
S104: and carrying out word recognition on the page content framed in the detection frame to obtain first word information, carrying out voice broadcasting on the page content framed in the detection frame, and carrying out voice recognition on the broadcasted voice information to obtain second word information.
In the present specification, identifying the content in the detection frame includes identifying the content such as text information, picture information, and icon information, and converting the identified content into a text form to obtain the first text information. If the content defined by the detection frame includes both text information and picture or icon information, the first text information mentioned herein may actually be obtained by two parts of text information, where the two parts of text information include: the first sub-text information and the second sub-text information.
Specifically, in the detection process, when the content framed in the detection frame contains both text information and picture or icon information, the text content framed in the detection frame is first identified to obtain first sub-text information, and then the picture or icon information is identified. Wherein, the character information can be extracted by utilizing an optical character recognition (Optical Character Recognition, OCR) mode to obtain the first sub-character information.
For the image recognition result of the picture or the icon information, the second sub-text information for the picture or the picture information can be determined according to the obtained recognition result and the context information of the picture or the icon information in the content framed by the detection frame, and then the first text information is obtained according to the obtained first sub-text information and the second sub-text information.
The client can splice the first sub-text information and the second sub-text information according to the sequence of the content framed by the detection frame to obtain the first text information. Of course, the client may also perform semantic analysis on the first sub-text information and the second sub-text information, and generate more complete first text information for representing text contents of the first sub-text information and the second sub-text information according to the semantic analysis result.
In addition, if the content framed in the current detection frame only contains text information, the text information framed in the detection frame can be directly identified to obtain the first text information.
In the barrier-free detection process, when the detection frame moves to one position, voice broadcasting can be automatically carried out on the content in the frame, the client can carry out voice recognition on the broadcasted content, and then the recognized content is converted into text information, namely second text information.
In the actual implementation of the present specification, it is also required to determine the area of the page to be detected where the detection frame is located on the target page, that is, determine the position of the detection frame in the target page, so as to determine whether the area of the page has been detected. Therefore, in this specification, there is a detection record updated in real time, and it is determined which areas in a page have been detected and which areas have not been detected by the stored detection record. And under the condition that the page area to be detected is not detected, continuously judging whether the range framed by the detection frame is matched with the coverage area of the page area to be detected, so as to ensure that the detection frame can frame a section of complete page content to ensure the accuracy of page detection.
Under the condition that the range framed by the detection frame is matched with the coverage area of the page area to be detected, the client performs text recognition on the currently framed page content in the detection frame to obtain first text information. And under the condition that the range framed by the detection frame is not matched with the coverage area of the page area to be detected, the client needs to adjust the position of the detection frame in the target page, judge whether the adjusted range framed by the detection frame is matched with the coverage area of the page area to be detected or not, and perform text recognition on the currently framed page content in the detection frame until the range framed by the detection frame is adjusted to be matched with the coverage area of the page area to be detected, so as to obtain first text information, as shown in fig. 2A and fig. 2B.
Fig. 2A and fig. 2B are schematic views of a process of adjusting a range defined by a detection frame until the range matches a coverage area of a page area to be detected.
Specifically, as can be seen in fig. 2A, the content in a target page is divided into several areas, the rectangular box represents a detection frame, and the content currently framed by the detection frame is only a part of the area of the page to be detected, and the area of the page is not completely framed, which may affect the detection result. Therefore, the client needs to adjust the detection frame down, as shown in fig. 2B, until the to-be-detected page area is completely framed, and at this time, the range framed by the detection frame is matched with the coverage area of the to-be-detected page area.
S105: detecting page contents framed by the detection frame according to the first text information and the second text information, and after detection is completed, controlling the detection frame to move in the target page so as to detect next page contents framed by the detection frame in the target page until detection of all page contents contained in the target page is completed, and detecting other target pages in the APP to be detected.
The client determines whether the content broadcasted in the barrier-free mode is consistent with the content in the actual detection frame by comparing the first text information with the second text information, and after the detection of the content in the detection frame is completed, the client continues to control the detection frame to move so as to detect the content in the next page area, so that all the contents in the target page are detected in the mode.
The method for determining whether the content broadcasted in the barrier-free mode is consistent with the content in the actual detection frame is as follows: the client can give a preset similarity threshold, and after obtaining the first text information and the second text information, the client can compare the information by calculating the text similarity of the first text information and the second text information. When the text similarity is not less than the similarity threshold, then the first text information and the second text information can be considered to be matched; when the text similarity is smaller than the similarity threshold, the first text information and the second text information can be considered to be different (i.e. not matched), and therefore the voice broadcasting of the page area is determined to be inaccurate in the barrier-free mode.
In the process of starting detection, the client side stores corresponding detection records, wherein the detection records record which pages are detected, which pages are not detected, and which partial areas in a target page are detected and which partial areas are not detected. Then, the page area which is still required to be detected in the target page can be determined through the detection record, and further the detection frame under the barrier-free model is controlled so as to detect the page area which is not detected yet. After one target page is detected, the detection of other target pages in the APP to be detected can be continued according to the detection conditions of the pages recorded in the detection record.
Further, when detecting other target pages in the APP to be detected, the client needs to determine the relationship between the pages and determine whether the page has been detected. Therefore, in the actual execution process, a page relation diagram needs to be preset, and the function of the page relation diagram is to describe the jump relation among the pages so as to detect the pages based on the jump relation among the pages recorded in the page relation diagram.
In the detection process, the content framed by the detection frame may include a jump link, so that, in addition to detecting the content in the detection frame, the client needs to trigger the jump link, and jump to another target page to continue detection based on the jump link. Meanwhile, the page relation graph records the jump relation between the two pages, so that after the detection of the jumped page is completed, the client can jump back to the last target page through the jump relation recorded by the page relation graph to continue the detection of the last target page.
To further describe a method for detecting an application in an unobstructed mode provided in this specification, the entire process is described below with reference to a detailed flowchart, as shown in fig. 3.
Fig. 3 is a detailed flow chart of detecting an application in an unobstructed mode provided in this specification.
In the specification, a client runs an application APP to be detected, starts a barrier-free mode, then displays a target page in the application APP to be detected, and displays a detection frame in the target page based on the barrier-free mode. The method can determine the page area to be detected where the detection frame is positioned on the target page by means of long screenshot or obtaining XML of the target page, and judge whether the page area to be detected is detected or not according to the stored detection record. And under the condition that the page area to be detected is not detected, judging whether the range framed by the detection frame is matched with the coverage area of the page area to be detected.
Under the condition that the range framed by the detection frame is matched with the coverage area of the page area to be detected, the client identifies the page content framed currently in the detection frame to obtain first text information; and under the condition that the range framed by the detection frame is not matched with the coverage area of the page area to be detected, adjusting the position of the detection frame in the target page, judging whether the adjusted range framed by the detection frame is matched with the coverage area of the page area to be detected or not, until the range framed by the detection frame is adjusted to be matched with the coverage area of the page area to be detected, and then identifying the page content currently framed in the detection frame to obtain the first text information.
In the barrier-free detection process, when the detection frame moves to one position, voice broadcasting can be automatically carried out on the content in the frame, the client can carry out voice recognition on the broadcasted content, and then the recognized content is converted into text information, namely second text information.
The client determines whether the content broadcasted in the barrier-free mode is consistent with the content in the actual detection frame by comparing the first text information with the second text information, and after the detection of the content in the detection frame is completed, the client continues to control the detection frame to move so as to detect the content in the next page area, so that all the contents in the target page are detected in the mode.
The foregoing describes one or more embodiments of the present disclosure as a method for detecting an application in an unobstructed mode, and based on the same concept, the present disclosure further provides a corresponding apparatus for detecting an application in an unobstructed mode, as shown in fig. 4.
Fig. 4 is a schematic diagram of an apparatus for detecting an application in an unobstructed mode provided in this specification, including:
the starting module 401 runs the application APP to be detected and starts an barrier-free mode;
the display module 402 displays a target page in the APP to be detected;
a preparation module 403, configured to display a detection frame in the target page based on the barrier-free mode;
the recognition module 404 performs text recognition on the page content framed in the detection frame to obtain first text information, performs voice broadcast on the page content framed in the detection frame, and performs voice recognition on the broadcast voice information to obtain second text information;
the detection module 405 detects the page content framed by the detection frame according to the first text information and the second text information, and after the detection is completed, controls the detection frame to move in the target page so as to detect the next page content framed by the detection frame in the target page until the detection of all page contents contained in the target page is completed, and detects other target pages in the APP to be detected.
Optionally, the detection module 405 is specifically configured to skip the target page to other target pages in the APP to be detected according to a preset page relationship diagram, and detect the other target pages, where the page relationship diagram is used to record a skip relationship between pages included in the APP to be detected.
Optionally, the identifying module 404 is specifically configured to identify text content framed by the detection frame to obtain first sub-text information when the content framed by the detection frame includes picture information; performing image recognition on the picture information, and determining second sub-text information aiming at the picture information according to the obtained recognition result and the context information of the picture information in the content framed by the detection frame; and obtaining the first text information according to the first sub-text information and the second sub-text information.
Optionally, before the identifying module 404 performs text identification on the page content framed in the detection frame to obtain the first text information, the detecting module 405 is further configured to determine a to-be-detected page area where the detection frame is located on the target page; judging whether the page area to be detected is detected or not according to the stored detection record; judging whether the range framed by the detection frame is matched with the coverage range of the page area to be detected under the condition that the page area to be detected is not detected;
The identifying module 404 is specifically configured to, when the detecting module determines that the range framed by the detecting frame matches the coverage area of the page area to be detected, perform text identification on the page content currently framed in the detecting frame, so as to obtain first text information.
Optionally, when the detecting module 405 determines that the range defined by the detecting frame does not match the coverage area of the to-be-detected page area, the detecting module 405 is specifically configured to adjust a position of the detecting frame in the target page, and determine whether the adjusted range defined by the detecting frame matches the coverage area of the to-be-detected page area, until the range defined by the detecting frame is adjusted to match the coverage area of the to-be-detected page area.
The present specification also provides a computer readable storage medium storing a computer program operable to perform a method of detecting an application in a barrier-free mode as provided in fig. 1 above.
The present specification also provides a schematic structural diagram of an electronic device corresponding to fig. 1 shown in fig. 5. At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile storage, as illustrated in fig. 5, although other hardware required by other services may be included. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs to implement the method for detecting applications in the barrier-free mode described above with reference to fig. 1. Of course, other implementations, such as logic devices or combinations of hardware and software, are not excluded from the present description, that is, the execution subject of the following processing flows is not limited to each logic unit, but may be hardware or logic devices.
Improvements to one technology can clearly distinguish between improvements in hardware (e.g., improvements to circuit structures such as diodes, transistors, switches, etc.) and software (improvements to the process flow). However, with the development of technology, many improvements of the current method flows can be regarded as direct improvements of hardware circuit structures. Designers almost always obtain corresponding hardware circuit structures by programming improved method flows into hardware circuits. Therefore, an improvement of a method flow cannot be said to be realized by a hardware entity module. For example, a programmable logic device (Programmable Logic Device, PLD) (e.g., field programmable gate array (Field Programmable Gate Array, FPGA)) is an integrated circuit whose logic function is determined by the programming of the device by a user. A designer programs to "integrate" a digital system onto a PLD without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, such programming is mostly implemented by using "logic compiler" software, which is similar to the software compiler used in program development and writing, and the original code before the compiling is also written in a specific programming language, which is called hardware description language (Hardware Description Language, HDL), but not just one of the hdds, but a plurality of kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), lava, lola, myHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog are currently most commonly used. It will also be apparent to those skilled in the art that a hardware circuit implementing the logic method flow can be readily obtained by merely slightly programming the method flow into an integrated circuit using several of the hardware description languages described above.
The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable logic controllers, and embedded microcontrollers, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, atmel AT91SAM, microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic of the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller in a pure computer readable program code, it is well possible to implement the same functionality by logically programming the method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Such a controller may thus be regarded as a kind of hardware component, and means for performing various functions included therein may also be regarded as structures within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in one or more software and/or hardware elements when implemented in the present specification.
It will be appreciated by those skilled in the art that embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, the present specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
The present description is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
It will be appreciated by those skilled in the art that embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, the present specification may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
The description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
The foregoing is merely exemplary of the present disclosure and is not intended to limit the disclosure. Various modifications and alterations to this specification will become apparent to those skilled in the art. Any modifications, equivalent substitutions, improvements, or the like, which are within the spirit and principles of the present description, are intended to be included within the scope of the claims of the present description.
Claims (12)
1. A method of detecting an application in an unobstructed mode, comprising:
running an application APP to be detected, and starting an obstacle-free mode;
displaying a target page in the APP to be detected;
displaying a detection frame in the target page based on the barrier-free mode;
performing text recognition on the page content framed in the detection frame to obtain first text information, performing voice broadcasting on the page content framed in the detection frame, and performing voice recognition on the broadcasted voice information to obtain second text information;
Detecting page contents framed by the detection frame according to the first text information and the second text information, and after detection is completed, controlling the detection frame to move in the target page so as to detect next page contents framed by the detection frame in the target page until detection of all page contents contained in the target page is completed, and detecting other target pages in the APP to be detected.
2. The method of claim 1, wherein the detecting other target pages in the APP to be detected specifically includes:
and according to a preset page relation diagram, jumping the target page to other target pages in the APP to be detected, and detecting the other target pages, wherein the page relation diagram is used for recording the jumping relation among the pages contained in the APP to be detected.
3. The method of claim 1, wherein the text recognition is performed on the page content framed in the detection frame to obtain the first text information, and specifically includes:
when the content framed in the detection frame contains picture information, identifying the text content framed in the detection frame to obtain first sub-text information;
Performing image recognition on the picture information, and determining second sub-text information aiming at the picture information according to the obtained recognition result and the context information of the picture information in the content framed by the detection frame;
and obtaining the first text information according to the first sub-text information and the second sub-text information.
4. The method of claim 1, further comprising, prior to text recognition of the page content framed in the detection box to obtain the first text information:
determining a page area to be detected where the detection frame is located on the target page;
judging whether the page area to be detected is detected or not according to the stored detection record;
judging whether the range framed by the detection frame is matched with the coverage range of the page area to be detected under the condition that the page area to be detected is not detected;
performing text recognition on the page content framed in the detection frame to obtain first text information, wherein the text recognition specifically comprises the following steps:
and under the condition that the range framed by the detection frame is matched with the coverage area of the page area to be detected, carrying out character recognition on the page content framed currently in the detection frame so as to obtain first character information.
5. The method of claim 4, the method further comprising:
and under the condition that the range framed by the detection frame is not matched with the coverage area of the page area to be detected, adjusting the position of the detection frame in the target page, and judging whether the adjusted range framed by the detection frame is matched with the coverage area of the page area to be detected or not until the range framed by the detection frame is adjusted to be matched with the coverage area of the page area to be detected.
6. An apparatus of a method of detecting an application in an unobstructed mode, comprising:
the starting module runs the application APP to be detected and starts an obstacle-free mode;
the display module displays the target page in the APP to be detected;
the preparation module is used for displaying a detection frame in the target page based on the barrier-free mode;
the recognition module is used for carrying out word recognition on the page content framed in the detection frame to obtain first word information, carrying out voice broadcasting on the page content framed in the detection frame, and carrying out voice recognition on the broadcasted voice information to obtain second word information;
The detection module is used for detecting page contents framed by the detection frame according to the first text information and the second text information, controlling the detection frame to move in the target page after detection is completed, detecting next page contents framed by the detection frame in the target page until detection of all page contents contained in the target page is completed, and detecting other target pages in the APP to be detected.
7. The device of claim 6, wherein the detection module is specifically configured to skip the target page to other target pages in the APP to be detected according to a preset page relationship diagram, and detect the other target pages, where the page relationship diagram is used to record a skip relationship between pages included in the APP to be detected.
8. The apparatus of claim 6, wherein the identification module is specifically configured to identify text content framed by the detection frame to obtain first sub-text information when the text content framed by the detection frame includes picture information; performing image recognition on the picture information, and determining second sub-text information aiming at the picture information according to the obtained recognition result and the context information of the picture information in the content framed by the detection frame; and obtaining the first text information according to the first sub-text information and the second sub-text information.
9. The apparatus of claim 6, wherein before the recognition module performs text recognition on the page content framed in the detection frame to obtain the first text information, the detection module is further configured to determine a to-be-detected page area where the detection frame is located on the target page; judging whether the page area to be detected is detected or not according to the stored detection record; judging whether the range framed by the detection frame is matched with the coverage range of the page area to be detected under the condition that the page area to be detected is not detected;
the identification module is specifically configured to perform text identification on the currently framed page content in the detection frame to obtain first text information when the detection module determines that the range framed by the detection frame matches with the coverage area of the page area to be detected.
10. The apparatus of claim 9, wherein when the detection module determines that the range defined by the detection frame does not match the coverage area of the to-be-detected page area, the detection module is specifically configured to adjust a position of the detection frame in the target page, and determine whether the adjusted range defined by the detection frame matches the coverage area of the to-be-detected page area, until the range defined by the detection frame is adjusted to match the coverage area of the to-be-detected page area.
11. A computer readable storage medium storing a computer program which, when executed by a processor, implements the method of any one of the preceding claims 1-5.
12. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any of the preceding claims 1-5 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311072616.5A CN117037134A (en) | 2023-08-23 | 2023-08-23 | Method and device for detecting application in barrier-free mode |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311072616.5A CN117037134A (en) | 2023-08-23 | 2023-08-23 | Method and device for detecting application in barrier-free mode |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117037134A true CN117037134A (en) | 2023-11-10 |
Family
ID=88602217
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311072616.5A Pending CN117037134A (en) | 2023-08-23 | 2023-08-23 | Method and device for detecting application in barrier-free mode |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117037134A (en) |
-
2023
- 2023-08-23 CN CN202311072616.5A patent/CN117037134A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20180232438A1 (en) | Title display method and apparatus | |
| CN110457578B (en) | Customer service demand identification method and device | |
| CN111523289B (en) | Text format generation method, device, equipment and readable medium | |
| EP3015997A1 (en) | Method and device for facilitating selection of blocks of information | |
| US11397597B2 (en) | Application processing method and apparatus | |
| CN112347512A (en) | Image processing method, device, equipment and storage medium | |
| CN112000329B (en) | Data display method, device, equipment and medium | |
| WO2024041316A1 (en) | Payment processing method and apparatus | |
| CN112966577A (en) | Method and device for model training and information providing | |
| US10824306B2 (en) | Presenting captured data | |
| CN111242106A (en) | Facial image acquisition method, device and equipment and facial recognition equipment | |
| CN117037134A (en) | Method and device for detecting application in barrier-free mode | |
| US20110014944A1 (en) | Text processing method for a digital camera | |
| CN116188895A (en) | Model training method and device, storage medium and electronic equipment | |
| CN115017905A (en) | Model training and information recommendation method and device | |
| CN113344590A (en) | Method and device for model training and complaint rate estimation | |
| CN113467858B (en) | Program starting method, device, equipment and medium | |
| CN117807961B (en) | Training method and device of text generation model, medium and electronic equipment | |
| CN111523866B (en) | Method and device for providing system operation enabling for user | |
| CN117520627B (en) | Project retrieval data processing method and device | |
| CN113821437B (en) | Page test method, device, equipment and medium | |
| CN116127220A (en) | Page jump processing method and device | |
| CN116303051A (en) | Test case generation method and device of user interface | |
| CN114296846A (en) | Page guide configuration method, system and device | |
| CN111142773A (en) | Operation control method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |