CN117806764A - Data collection method, device, equipment and storage medium of user interface - Google Patents
Data collection method, device, equipment and storage medium of user interface Download PDFInfo
- Publication number
- CN117806764A CN117806764A CN202410070697.3A CN202410070697A CN117806764A CN 117806764 A CN117806764 A CN 117806764A CN 202410070697 A CN202410070697 A CN 202410070697A CN 117806764 A CN117806764 A CN 117806764A
- Authority
- CN
- China
- Prior art keywords
- target
- user interface
- information
- image
- interface
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000013480 data collection Methods 0.000 title claims abstract description 35
- 238000004590 computer program Methods 0.000 claims description 16
- 238000013075 data extraction Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 6
- 238000011161 development Methods 0.000 abstract description 5
- 238000004891 communication Methods 0.000 description 10
- 238000012545 processing Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 5
- 230000018109 developmental process Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000032683 aging Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- General Engineering & Computer Science (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a data collection method, a device, equipment and a storage medium of a user interface, wherein the method comprises the following steps: acquiring preset automatic operation information, and executing target operation on a target user interface according to the automatic operation information; extracting a target interface image of the target user interface after the target user interface executes the target operation; and determining object information to be identified of the target interface image, carrying out image identification on the target interface image according to the object information to be identified, and extracting target data according to an image identification result. The data collection method of the user interface disclosed by the invention utilizes the image recognition mode to collect the data of the user interface of the third party, can realize the automatic collection of information, unify the information acquisition flow of the third party, avoid repeated development work aiming at different interfaces, and also avoid the limitation of the interface of the third party.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a storage medium for data collection of a user interface.
Background
When the message communication is performed with the third party application, the message communication is often realized by calling interfaces such as Web Service and RESTful API provided by the third party, but in reality, the use of the interfaces often has various problems such as opening or not of the interfaces, limitation of authority, whether the transmitted message can meet the requirement, calling frequency of the interfaces, aging performance of message content and the like. In addition, the interfaces or services provided by third party applications often have a five-in-eight door design, require different system modifications to adapt to different services or interfaces, and are often repetitive and redundant development efforts.
Therefore, the existing method for communicating with the third party based on the interface mainly has the following problems: 1. interfaces for third party applications tend to set usage restrictions; 2. the interface use matters need to be coordinated with an interface provider of a third party; 3. the data to be obtained cannot be flexibly obtained, and application modification is required for different interfaces.
Disclosure of Invention
The invention provides a data collection method, device, equipment and storage medium of a user interface, which are used for realizing information acquisition of a third party without using the interface.
According to an aspect of the present invention, there is provided a data collection method of a user interface, including:
acquiring preset automatic operation information, and executing target operation on a target user interface according to the automatic operation information;
extracting a target interface image of the target user interface after the target operation is executed;
and determining object information to be identified of the target interface image, carrying out image identification on the target interface image according to the object information to be identified, and extracting target data according to an image identification result.
Further, before acquiring the preset automation operation information, the method further includes:
setting the automatic operation information; the automatic operation information comprises clicking operation, information input operation, operation interval time and operation cycle times.
Further, extracting a target interface image of the target user interface after the target operation is performed, including:
after the target operation is executed on the target user interface, screen capturing operation is executed on the target user interface;
and taking the image obtained by screen capturing as the target interface image.
Further, the object information to be identified comprises the position of the object to be identified and the label of the object to be identified.
Further, performing image recognition on the target interface image according to the object information to be recognized, including:
determining a target identification object in the target interface image according to the position of the object to be identified;
and carrying out image recognition on the target recognition object according to the object label to be recognized.
Further, before acquiring the preset automation operation information, the method further includes:
the resolution of the target user interface is set to a target resolution.
According to another aspect of the present invention, there is provided a data collection apparatus of a user interface, comprising:
the target operation execution module is used for acquiring preset automatic operation information and executing target operation on a target user interface according to the automatic operation information;
the target interface image extraction module is used for extracting a target interface image of the target user interface after the target user interface executes the target operation;
the target data extraction module is used for determining the object information to be identified of the target interface image, carrying out image identification on the target interface image according to the object information to be identified, and extracting target data according to an image identification result.
Optionally, the device further comprises an automation operation information setting module, configured to set the automation operation information; the automatic operation information comprises clicking operation, information input operation, operation interval time and operation cycle times.
Optionally, the target interface image extraction module is further configured to:
after the target operation is executed on the target user interface, screen capturing operation is executed on the target user interface;
and taking the image obtained by screen capturing as the target interface image.
Optionally, the object information to be identified includes an object position to be identified and an object tag to be identified.
Optionally, the target data extraction module is further configured to:
determining a target identification object in the target interface image according to the position of the object to be identified;
and carrying out image recognition on the target recognition object according to the object label to be recognized.
Optionally, the apparatus further comprises a resolution setting module for setting a resolution of the target user interface to a target resolution.
According to another aspect of the present invention, there is provided an electronic apparatus including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the data collection method of the user interface of any one of the embodiments of the present invention.
According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to execute a data collection method of a user interface according to any embodiment of the present invention.
The invention discloses a data collection method of a user interface, which comprises the steps of firstly acquiring preset automatic operation information, and executing target operation on a target user interface according to the automatic operation information; then extracting a target interface image of the target user interface after the target user interface executes the target operation; and finally, determining the object information to be identified of the target interface image, carrying out image identification on the target interface image according to the object information to be identified, and extracting target data according to an image identification result. The data collection method of the user interface disclosed by the invention utilizes the image recognition mode to collect the data of the user interface of the third party, can realize the automatic collection of information, unify the information acquisition flow of the third party, avoid repeated development work aiming at different interfaces, and also avoid the limitation of the interface of the third party.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for data collection of a user interface provided in accordance with a first embodiment of the present invention;
FIG. 2 is a flow chart of a method for data collection of a user interface according to a second embodiment of the present invention;
FIG. 3 is a schematic diagram of a data collection device of a user interface according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device implementing a data collection method of a user interface according to a fourth embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
Fig. 1 is a flowchart of a data collection method of a user interface according to a first embodiment of the present invention, where the method may be performed by a data collection device of a user interface, the data collection device of the user interface may be implemented in hardware and/or software, and the data collection device of the user interface may be configured in an electronic device. As shown in fig. 1, the method includes:
s110, acquiring preset automatic operation information, and executing target operation on a target user interface according to the automatic operation information.
The automatic operation information may be preset operations that need to be performed on the target user interface, such as clicking, information input, etc.; the target user interface is a third party user interface which can be operated based on a local operation platform, software or system; the target operation is an actual operation performed on the target user interface, which is determined according to preset automation operation information.
In this embodiment, the automation operation information may be preset according to actual requirements, and when data is collected, it is determined, according to the preset automation operation information, what operation needs to be performed on the target user interface, and the operation is performed.
Optionally, the method for executing the target operation on the target user interface according to the automation operation information may be that determining the type of operation to be executed on the target user interface according to the automation operation information. For example, if the automation operation information includes a click event, it is determined that a target operation to be performed on the target user interface is a click operation. Meanwhile, the automation operation information also comprises a specific click object and an operation sequence of the click event, and then the click operation is executed on the click object in the target user interface according to the preset operation sequence in the automation operation information.
S120, extracting a target interface image of the target user interface after the target user interface executes the target operation.
The target interface image is an image corresponding to the display state on the target user interface after the target operation is executed.
In this embodiment, when the target operation is performed on the target user interface, a new display state of the target user interface may be obtained, and the new display state may be extracted in the form of an image, so as to obtain a target interface image.
Preferably, the target interface image of the target user interface may be extracted in a screen capturing manner after the target operation is performed on the target user interface.
S130, determining object information to be identified of the target interface image, carrying out image identification on the target interface image according to the object information to be identified, and extracting target data according to an image identification result.
The object information to be identified is an object to be identified in the target interface image and the position of each object. The target data is specific information in the image recognition result, including a numerical value, an object ID, and the like.
Preferably, the object information to be identified may further include tag information of an object to be identified, which is used to indicate a content attribute that needs to be extracted from the object to be identified when performing image identification, for example, the tag information may be a device ID, and when performing image identification on the target interface image according to the object information to be identified, the object to be identified needs to be identified at a location of the object to be identified, so as to extract the device ID information contained therein.
In the present embodiment, image recognition is a technique of processing, analyzing, and understanding an image with a computer to recognize targets and objects of various modes. By utilizing the image recognition technology, the needed information can be extracted from the target interface image to obtain target data.
Further, before the preset automation operation information is obtained, the method may further include: setting automatic operation information; the automatic operation information comprises clicking operation, information input operation, operation interval time and operation cycle times.
In this embodiment, the automation operation information may be preset, and what operation needs to be performed when the data is collected on the target user interface may be set, including whether to click, whether to input information in an information input field of the target user interface, an interval time between each operation, and whether to cycle the preset operation and the number of cycles.
Further, before the preset automation operation information is obtained, the method may further include: the resolution of the target user interface is set to the target resolution.
In this embodiment, in order to facilitate image recognition of the target interface image, the resolution of the target user interface may be preset.
Preferably, the resolution of the interface of the local operating platform, software or system may be taken as the target resolution, so that the resolution of the target user interface is consistent with the resolution of the local operating platform, software or system.
The invention discloses a data collection method of a user interface, which comprises the steps of firstly acquiring preset automatic operation information, and executing target operation on a target user interface according to the automatic operation information; then extracting a target interface image of the target user interface after the target user interface executes the target operation; and finally, determining the object information to be identified of the target interface image, carrying out image identification on the target interface image according to the object information to be identified, and extracting target data according to an image identification result. The data collection method of the user interface disclosed by the invention utilizes the image recognition mode to collect the data of the user interface of the third party, can realize the automatic collection of information, unify the information acquisition flow of the third party, avoid repeated development work aiming at different interfaces, and also avoid the limitation of the interface of the third party.
Example two
Fig. 2 is a flowchart of a data collection method of a user interface according to a second embodiment of the present invention, and this embodiment is a refinement of the foregoing embodiment. As shown in fig. 2, the method includes:
s210, acquiring preset automatic operation information, and executing target operation on a target user interface according to the automatic operation information.
In this embodiment, the automation operation information may be preset according to actual requirements, and when data is collected, it is determined, according to the preset automation operation information, what operation needs to be performed on the target user interface, and the operation is performed.
S220, after target operation is carried out on the target user interface, screen capturing operation is carried out on the target user interface, and an image obtained through screen capturing is used as a target interface image.
In this embodiment, when the target operation is performed on the target user interface, a new display state of the target user interface may be obtained, and then the new display state may be extracted in the form of an image through the screen capturing operation, so as to obtain the target interface image.
S230, determining object information to be identified of the target interface image.
The object information to be identified comprises the position of the object to be identified and the object label to be identified.
In this embodiment, the object information to be identified is the object to be identified and the position of each object in the target interface image, and may further include tag information of the object to be identified, where the tag information is used to indicate the content attribute that needs to be extracted from the object to be identified when the image identification is performed.
For example, if the target user interface is a traffic information query interface, the ID of the vehicle and the measured vehicle speed that are entered in the background can be queried through the interface, and the automated operation information can be preset as the input vehicle ID 001, and the query key is clicked. And executing corresponding operation on the target user interface according to the automatic operation information, so that the query result display of the target user interface can be obtained, and a target interface image containing the query result can be obtained through screen capturing operation. Further, the information to be identified preset for the target interface image contains an object to be identified, namely a query result displayed in the target interface image, the information of the object to be identified is a label attached to the position of the query result, and the content of the label is vehicle speed information.
S240, determining a target identification object in the target interface image according to the position of the object to be identified, and carrying out image identification on the target identification object according to the label of the object to be identified.
In this embodiment, the position of the target recognition object may be determined according to the position of the object to be recognized in the object information to be recognized, and then the target of image recognition may be determined according to the object tag to be recognized.
For example, according to the information of the object to be identified, i.e. the label attached to the position of the query result, which is contained in the information to be identified, the position of the label, i.e. the position of the object to be identified, the object to be identified and the specific target for image identification can be identified according to the position and the content of the label of the object to be identified.
S250, extracting target data according to the image recognition result.
In this embodiment, an image recognition technology may be used, and then, required information is extracted according to an image recognition result, so as to obtain target data obtained by collecting data from a target user interface.
The data collection method of the user interface disclosed by the embodiment of the invention utilizes the image recognition mode to collect the data of the user interface of the third party, can realize the automatic collection of information, unify the information acquisition flow of the third party, avoid repeated development work aiming at different interfaces, and also avoid the limitation of the interface of the third party.
Example III
Fig. 3 is a schematic structural diagram of a data collection device of a user interface according to a third embodiment of the present invention. As shown in fig. 3, the apparatus includes: a target operation execution module 310, a target interface image extraction module 320, and a target data extraction module 330.
The target operation executing module 310 is configured to obtain preset automation operation information, and execute a target operation on the target user interface according to the automation operation information.
The target interface image extraction module 320 is configured to extract a target interface image of the target user interface after performing the target operation.
The target data extraction module 330 is configured to determine object information to be identified of the target interface image, perform image identification on the target interface image according to the object information to be identified, and extract target data according to an image identification result.
Optionally, the apparatus further includes an automation operation information setting module 340 for setting automation operation information; the automatic operation information comprises clicking operation, information input operation, operation interval time and operation cycle times.
Optionally, the target interface image extraction module 320 is further configured to:
after the target user interface is subjected to target operation, performing screen capturing operation on the target user interface; taking the image obtained by screen capturing as a target interface image.
Optionally, the object information to be identified includes an object position to be identified and an object tag to be identified.
Optionally, the target data extraction module 330 is further configured to:
determining a target identification object in the target interface image according to the position of the object to be identified; and carrying out image recognition on the target recognition object according to the object label to be recognized.
Optionally, the apparatus further comprises a resolution setting module 350 for setting the resolution of the target user interface to the target resolution.
The data collection device of the user interface provided by the embodiment of the invention can execute the data collection method of the user interface provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example IV
Fig. 4 shows a schematic diagram of the structure of an electronic device 10 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 4, the electronic device 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
Various components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as the data collection method of the user interface.
In some embodiments, the data collection method of the user interface may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more of the steps of data collection of the user interface described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the data collection method of the user interface in any other suitable manner (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (9)
1. A method of data collection for a user interface, comprising:
acquiring preset automatic operation information, and executing target operation on a target user interface according to the automatic operation information;
extracting a target interface image of the target user interface after the target operation is executed;
and determining object information to be identified of the target interface image, carrying out image identification on the target interface image according to the object information to be identified, and extracting target data according to an image identification result.
2. The method of claim 1, further comprising, prior to obtaining the preset automation information:
setting the automatic operation information; the automatic operation information comprises clicking operation, information input operation, operation interval time and operation cycle times.
3. The method of claim 1, wherein extracting a target interface image of the target user interface after performing the target operation comprises:
after the target operation is executed on the target user interface, screen capturing operation is executed on the target user interface;
and taking the image obtained by screen capturing as the target interface image.
4. The method of claim 1, wherein the object information to be identified comprises an object location to be identified and an object tag to be identified.
5. The method of claim 4, wherein performing image recognition on the target interface image according to the object information to be recognized comprises:
determining a target identification object in the target interface image according to the position of the object to be identified;
and carrying out image recognition on the target recognition object according to the object label to be recognized.
6. The method of claim 1, further comprising, prior to obtaining the preset automation information:
the resolution of the target user interface is set to a target resolution.
7. A data collection device for a user interface, comprising:
the target operation execution module is used for acquiring preset automatic operation information and executing target operation on a target user interface according to the automatic operation information;
the target interface image extraction module is used for extracting a target interface image of the target user interface after the target user interface executes the target operation;
the target data extraction module is used for determining the object information to be identified of the target interface image, carrying out image identification on the target interface image according to the object information to be identified, and extracting target data according to an image identification result.
8. An electronic device, the electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the data collection method of the user interface of any one of claims 1-6.
9. A computer readable storage medium storing computer instructions for causing a processor to perform the data collection method of the user interface of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410070697.3A CN117806764A (en) | 2024-01-17 | 2024-01-17 | Data collection method, device, equipment and storage medium of user interface |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410070697.3A CN117806764A (en) | 2024-01-17 | 2024-01-17 | Data collection method, device, equipment and storage medium of user interface |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117806764A true CN117806764A (en) | 2024-04-02 |
Family
ID=90429924
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410070697.3A Pending CN117806764A (en) | 2024-01-17 | 2024-01-17 | Data collection method, device, equipment and storage medium of user interface |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117806764A (en) |
-
2024
- 2024-01-17 CN CN202410070697.3A patent/CN117806764A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115687406B (en) | Sampling method, device, equipment and storage medium for call chain data | |
| CN117033058A (en) | Analysis method, device, equipment and medium for software crash data | |
| CN116545905A (en) | Service health detection method and device, electronic equipment and storage medium | |
| CN116451619A (en) | Component information processing method, device, medium and equipment | |
| CN116450176A (en) | Version updating method and device, electronic equipment and storage medium | |
| CN115794744A (en) | Log display method, device, equipment and storage medium | |
| CN117806764A (en) | Data collection method, device, equipment and storage medium of user interface | |
| CN117539840B (en) | Log acquisition method, device, equipment and medium | |
| CN117742900B (en) | Method, device, equipment and storage medium for constructing service call graph | |
| CN118350631A (en) | Equipment risk identification method and device, electronic equipment and storage medium | |
| CN117271289A (en) | Webpage monitoring method, device, equipment and storage medium | |
| CN117312095A (en) | Java virtual machine heap memory set object monitoring method and device | |
| CN118226120A (en) | Power consumption determination method and device, electronic equipment and storage medium | |
| CN117453746A (en) | Method, device, equipment and medium for data cycle screening | |
| CN118012936A (en) | Data extraction method, device, equipment and storage medium | |
| CN116028228A (en) | Bitmap memory occupation detection method, device, equipment and storage medium | |
| CN117573491A (en) | Positioning method, device, equipment and storage medium for performance bottleneck | |
| CN116382831A (en) | Page switching method, device, equipment and medium | |
| CN115983222A (en) | EasyExcel-based file data reading method, device, equipment and medium | |
| CN116860557A (en) | Method, device, equipment and medium for collecting service stack information | |
| CN118277143A (en) | Port abnormality repairing method, device, equipment and medium of input equipment | |
| CN117251419A (en) | File monitoring method and device, electronic equipment and storage medium | |
| CN115904899A (en) | Operation record generation method, operation record acquisition method, operation record generation device, operation record acquisition device and operation record acquisition medium | |
| CN118132533A (en) | Data processing method, device, equipment and storage medium | |
| CN117033463A (en) | Optimization strategy generation method, device, equipment and medium based on video log |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |