CN112633270B - Skeleton screen generation method, device and equipment based on picture identification and storage medium - Google Patents
Skeleton screen generation method, device and equipment based on picture identification and storage medium Download PDFInfo
- Publication number
- CN112633270B CN112633270B CN202011594807.4A CN202011594807A CN112633270B CN 112633270 B CN112633270 B CN 112633270B CN 202011594807 A CN202011594807 A CN 202011594807A CN 112633270 B CN112633270 B CN 112633270B
- Authority
- CN
- China
- Prior art keywords
- skeleton screen
- page
- coordinate information
- region
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000012545 processing Methods 0.000 claims description 15
- 238000007781 pre-processing Methods 0.000 claims description 13
- 238000009877 rendering Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 8
- 238000012937 correction Methods 0.000 claims description 6
- 230000009467 reduction Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000002347 injection Methods 0.000 claims description 4
- 239000007924 injection Substances 0.000 claims description 4
- 238000011161 development Methods 0.000 abstract description 29
- 238000012423 maintenance Methods 0.000 abstract description 9
- 238000005516 engineering process Methods 0.000 abstract description 8
- 238000004458 analytical method Methods 0.000 abstract description 3
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 238000012015 optical character recognition Methods 0.000 description 28
- 230000006870 function Effects 0.000 description 11
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 239000000243 solution Substances 0.000 description 5
- 238000013461 design Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- VYZAMTAEIAYCRO-UHFFFAOYSA-N Chromium Chemical compound [Cr] VYZAMTAEIAYCRO-UHFFFAOYSA-N 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000002860 competitive effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000003631 expected effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000007935 neutral effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
- G06V10/243—Aligning, centring, orientation detection or correction of the image by compensating for image skew or non-uniform image deformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- User Interface Of Digital Computer (AREA)
- Processing Or Creating Images (AREA)
Abstract
The application relates to the technical field of artificial intelligence, and provides a skeleton screen generation method, device and equipment based on picture identification and a storage medium. The method comprises the following steps: acquiring a screenshot corresponding to a page loaded with real data; OCR recognition is carried out on the screenshot to obtain region coordinate information corresponding to the page element; and automatically generating a skeleton screen according to the region coordinate information. According to the application, by combining the characteristics of simple layout and single style of the skeleton screen, a picture identification technology is adopted to identify the page loaded with real data, so that the region coordinate information corresponding to page elements is obtained, and then the skeleton screen is automatically generated, so that the consistency of the skeleton screen structure and the page structure is ensured, the skeleton screen development flow is simplified, and the development and maintenance cost is reduced; the skeleton screen code does not need to be written manually, and the code structure analysis and configuration are not needed; the skeleton screen code structure is clear and reasonable, and the size is small; the white screen time is greatly reduced, the page loading experience is optimized, and the user experience is improved.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a skeleton screen generation method, device and equipment based on picture identification and a storage medium.
Background
The App market is competitive nowadays, and the user experience of an App product is crucial to the success and failure of the product; the optimization of the page loading experience is important to improve the details of the user experience. The loading of the App page data is gradually converted from Loading and the progress bar into a skeleton screen loading mode, so that the waiting experience of a user is improved.
The skeleton screen displays the general structure of the page to the user before the real data of the page is not loaded, and generally adopts a light gray occupied bitmap, so that the user can have general expectation on the loaded content; the attention of a user to the loading process can be lightened, the attention is transferred to the content being loaded, the user can feel that the content is gradually loaded and is about to be presented, the scorching emotion is reduced, the loading process is subjectively smooth, the waiting time of the user perception is relatively short, the page is rendered after the request data is returned, and the data content to be displayed is supplemented; the effect is more pronounced in case of poor network environment or large amount of loaded data.
The skeleton screen is characterized in that: the layout is simple, the style is single, and blocks with light gray or neutral filling colors are used for describing the rough framework of the page; but the page layout looks simple, and the implementation process is not simple; the related links are more, for example, a designer and a developer are needed to participate in the implementation process to finish the implementation, the designer gives out a skeleton screen design draft, and a development engineer writes the skeleton screen code manually; meanwhile, the skeleton screen can be changed frequently due to different page interaction requirements, so that the skeleton screen is a great problem for maintenance, and a part of manpower investment is needed.
The main skeleton screen generation schemes at present are as follows:
1. Firstly, a designer provides a skeleton screen design draft, a developer finishes writing related codes of the skeleton screen, and then the skeleton screen is manually placed in an html file. The scheme needs to be participated by a designer and a developer to write skeleton screen codes manually, and the skeleton screen codes need to be injected into html files manually, so that the flow is long, the flexibility is not realized, and the changing cost is high; meanwhile, due to the independent design, the situation that the actual content of the skeleton screen and the page is inconsistent is easy to occur.
2. Firstly, a designer provides a skeleton screen design draft, a developer completes writing of a skeleton screen component, and the skeleton screen component is analyzed through SSR RENDER plug-ins and automatically injected into an html file. According to the scheme, although the skeleton screen codes are automatically injected into the html file, the conditions of long period, poor flexibility and inconsistent actual content of the skeleton screen and the page still exist.
3. Without designing manuscripts, a page needing to generate a skeleton screen is opened by controlling a header Chrome at a server through Puppeteer, and the existing elements are subjected to style stacking by deleting or adding the elements in the page, so that pictures and characters are hidden under the condition that the page layout is not changed, and are displayed as gray blocks through style coverage; the modified code is then extracted for use as a skeleton screen. Although the scheme can automatically generate the skeleton screen without participation of a designer, certain requirements are imposed on the page structure, and if the expected effect is to be achieved, a plurality of configurations are needed, such as: the DOM attribute is used for carrying out framework screen fine granularity configuration, designated node configuration and self-adaptive unit requirements, and the use threshold is high; meanwhile, the generated skeleton screen code is complex and has large volume; puppeteer has about 80M, is easy to install and fails, and has the use limitation in the use under the intranet office environment.
In summary, how to simplify the development flow of the skeleton screen, ensure automatic generation, ensure the consistency with the real page structure, make the skeleton screen have concise skeleton screen codes, simplify the use mode, have no invasiveness to the page structure, and reduce the development and maintenance costs is a main technical problem to be solved in the field at present.
Disclosure of Invention
The invention aims to provide a skeleton screen generation method, device, equipment and storage medium based on picture identification, which are used for solving the problems that the structure of a skeleton screen is inconsistent with the structure of a page at present, the generation flow of the skeleton screen is complex and the like.
The above purpose is achieved by the following technical scheme:
according to one aspect of the invention, the invention provides a skeleton screen generation method based on picture identification, which comprises the following steps:
Acquiring a screenshot corresponding to a page loaded with real data;
OCR recognition is carried out on the screenshot, and region coordinate information corresponding to the page element is obtained;
and automatically generating a skeleton screen according to the region coordinate information.
Optionally, the step of performing OCR recognition on the screenshot to obtain region coordinate information corresponding to a page element includes:
preprocessing the screenshot;
Dividing logic areas of the preprocessed screenshot to obtain logic areas; the logic area comprises a title bar, a left bar, a right bar, an upper bar and a lower bar;
finely dividing the logic area to obtain page elements;
And judging the shape outline of the page element according to the pixel characteristics to obtain the region coordinate information corresponding to the page element.
Optionally, the step of automatically generating the skeleton screen according to the region coordinate information includes: according to the point coordinates in the area coordinate information of each page element, calculating to obtain the width and the height of the area block corresponding to each page element; and automatically generating a skeleton screen according to the region coordinate information and the width and the height of the region blocks.
Optionally, the step of automatically generating the skeleton screen according to the region coordinate information and the width and the height of the region block includes: positioning the left side and the top of the area block according to the coordinates of the upper left corner point; positioning the right side and the bottom of the area block according to the coordinates of the lower right corner point; and giving out a region block pattern according to the width and the height of the region block, and automatically generating a skeleton screen.
Optionally, the step of giving the region block style and automatically generating the skeleton screen includes: giving a region block pattern and converting the region block pattern into a skeleton screen code; automatically injecting the skeleton screen code into an html file; and rendering the page to obtain the skeleton screen.
Optionally, the skeleton screen code is automatically injected into the html file by adopting a mode of externally referencing the Js file.
Optionally, the real data is real page data presented to the user; the page element comprises text and pictures; the region coordinate information includes a page element type and a point coordinate of the page element.
According to another aspect of the present invention, there is provided a skeleton screen generating device based on picture recognition, the skeleton screen generating device including:
The acquisition unit is used for acquiring a screenshot corresponding to the page loaded with the real data;
The recognition unit is used for performing OCR (optical character recognition) on the screenshot to obtain region coordinate information corresponding to the page element;
and the generating unit is used for automatically generating a skeleton screen according to the region coordinate information.
According to still another aspect of the present invention, there is provided an electronic apparatus including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform steps in the picture-recognition-based skeleton screen generation method.
According to still another aspect of the present invention, there is further provided a computer readable storage medium having stored therein a computer program which, when executed by a processor, implements the steps of the skeleton screen generating method based on picture recognition.
The beneficial effects are that: according to the skeleton screen generation method, device, equipment and storage medium based on picture identification, an OCR picture identification technology is adopted to identify pictures of pages loaded with real data, so that regional coordinate information of page elements such as pictures, texts and the like is obtained, and then the skeleton screen is automatically generated according to the regional coordinate information, so that the consistency of a page structure and a skeleton screen structure is ensured, the development flow of the skeleton screen is simplified, and the development and maintenance cost is reduced; the white screen time is greatly reduced, the page loading experience is optimized, and the user experience is improved.
Drawings
Other objects and attainments together with a more complete understanding of the invention will become apparent and appreciated by referring to the following description taken in conjunction with the accompanying drawings. In the accompanying drawings:
FIG. 1 is a flow chart of a skeleton screen generation method based on picture recognition according to an embodiment of the invention;
FIG. 2 is a flow chart of obtaining region coordinate information according to an embodiment of the present invention;
FIG. 3 is a flow diagram of automatically generating a skeleton screen in accordance with one embodiment of the present invention;
FIG. 4 is a flow chart of an automatic generation of a skeleton screen in accordance with another embodiment of the present invention;
FIG. 5 is a diagram of an integrated information list page loaded with real data according to an embodiment of the present invention;
FIG. 6 is a logical block diagram of a skeleton screen generation method based on picture recognition in an embodiment of the invention;
FIG. 7 is a schematic diagram of a skeleton screen of comprehensive information obtained in an embodiment of the present invention;
FIG. 8 is a block diagram of a skeleton screen generating device based on picture recognition provided by the invention;
Fig. 9 is a schematic structural view of the electronic device of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention:
Fig. 1 schematically shows a flowchart of a skeleton screen generating method based on picture recognition according to an embodiment of the present invention. As shown in fig. 1, the invention provides a skeleton screen generation method based on picture recognition, which comprises the following steps:
step S100, obtaining a screenshot corresponding to a page loaded with real data; wherein the real data is the real page data that is ultimately presented to the user for viewing.
Step S200, performing OCR (optical character recognition) on the screenshot to obtain region coordinate information corresponding to the page element; the page elements can comprise elements such as pictures, texts, videos, buttons and the like; the region coordinate information may include page element type and point coordinates of page elements, etc.
And step S300, automatically generating a skeleton screen according to the region coordinate information.
According to the invention, by combining the characteristics of simple layout and single style of the skeleton screen, the OCR picture recognition technology is adopted to recognize the page loaded with real data to obtain the region coordinate information of page elements such as texts, pictures and the like, and then the region block style is given out according to the corresponding region coordinate information to automatically generate the skeleton screen, so that the consistency of the structure of the skeleton screen and the page structure is ensured, the development flow of the skeleton screen is simplified, and the development and maintenance cost is reduced; and the development efficiency of the skeleton screen is improved. The skeleton screen code structure generated by the skeleton screen generation method based on picture identification is very clear, reasonable and small in size. By adopting the skeleton screen generation method based on picture identification, the white screen time is greatly reduced, the page loading experience is optimized, and the user experience is improved.
In step S100, in the step of obtaining the screenshot corresponding to the page loaded with the real data, the page loaded with the real data is obtained by rendering the real data through page loading after the developer completes the development of the page function, where the real data is the real page data that is finally presented to the user. According to the invention, the skeleton screen is generated by the real data page, namely, the page structure and the skeleton screen structure are consistent, and when the page interaction is changed, the updating of the skeleton screen can be completed with lower cost.
Fig. 5 schematically shows a comprehensive information list page loaded with real data. Illustratively, at the front end, a developer completes the function development of the comprehensive information list page, and then loads real data to render the page, so that the comprehensive information list page loaded with real data as shown in the figure can be obtained. Based on the page, the following operations such as screenshot, OCR processing and the like can be performed.
"Functional development of list pages" refers to complete page development, including, for example, page layout, interface interfacing, and the like. The page elements in a page may include: pictures, texts, videos, buttons and the like, and the following embodiments of the present invention are exemplified by only texts and pictures as common page elements; taking the comprehensive information list page as an example, as shown in the figure, the upper part of the page is provided with a title, each title comprises two columns of page elements, one column at the left side can be a text element, one column at the right side can be a picture element, and the text elements and the picture elements can be in one-to-one correspondence.
In step S200, in the step of performing OCR recognition on the screenshot to obtain the region coordinate information corresponding to the page element, the screenshot may be recognized by using an OCR image recognition technology through an OCR platform. FIG. 6 schematically illustrates a logical block diagram of a skeleton screen generation method based on picture recognition according to an embodiment of the present application, as shown in FIG. 6, including a front end and an OCR platform, where the front end completes the development of page functions, renders a page using real data and captures a screenshot; uploading the screenshot to an OCR platform connected with a front-end interface, identifying characters and pictures in the screenshot by the OCR platform, generating coordinate information of the character and picture areas, and returning the coordinate information of each area to the front-end; and generating each region style by the front end according to the coordinate information of each region, such as the left upper corner and the right lower corner, namely automatically generating the skeleton screen, and performing subsequent operations such as rendering of the skeleton screen page. The OCR platform can comprise an image preprocessing module, a layout module, a character recognition module, a character editing module and the like. The application has short whole flow and flexible generation process; the characteristics of simple layout and single style of the skeleton screen are perfectly matched with the region identification characteristics of the picture identification technology, and the skeleton screen is suitable for generating the approximate outline of the skeleton screen; moreover, the skeleton screen finally obtained by adopting OCR picture recognition has a concise skeleton screen code, the method has no invasiveness to the page structure, and the generated skeleton screen code structure is very clear, reasonable and small in size.
The invention adopts OCR picture recognition technology to recognize the screenshot of the page loaded with real data, can rapidly and accurately recognize the corresponding region coordinate information of page elements such as pictures, texts and the like from the screenshot, simplifies the whole development process, reduces the development and maintenance cost, greatly improves the development efficiency of the skeleton screen, and can be applied, popularized and implemented.
Fig. 2 schematically illustrates a flow of obtaining region coordinate information in an embodiment of the present invention, and as shown in fig. 2, the step of performing OCR recognition on the screenshot to obtain region coordinate information corresponding to a page element may include:
Step 210, preprocessing the screenshot. The preprocessing may include steps such as graying processing, noise reduction processing, inclination correction, image cropping, and the like. The graying treatment is to carry out gray treatment on the color screenshot to obtain a black-white picture, for example, only gray foreground information and white background information are displayed after treatment. The noise reduction processing is to perform the noise reduction processing according to the noise characteristics. The tilt correction is a correction process for a screenshot of tilt. Image cropping is to crop the screenshot into the truly desired area to reduce unnecessary running costs to increase OCR recognition speed.
Step 220, carrying out logic region division on the preprocessed screenshot to obtain a logic region, and carrying out fine division on the logic region to obtain page elements; the logic area comprises a title bar, a left bar, a right bar, an upper bar and a lower bar; the page element may include: text, title, picture, form, line, etc.;
And 230, judging the shape outline of the page element according to the pixel characteristics, and obtaining the region coordinate information corresponding to the page element. Wherein the region coordinate information may include: page element type, point coordinate data of page elements, and the like. Wherein the page element type is mainly indicative of whether the page element is a picture or text or other page element. The point coordinate data of the page element may be coordinates of each vertex of a rectangular frame, for example, a rectangular frame. It should be noted that, in the present invention, the area corresponding to the page element is illustrated as a rectangular frame, and of course, the area corresponding to the page element may be other shapes, for example, a circle.
Preferably, the region coordinate information may further include region position information of the page element, for example, it may be indicated whether the page element belongs to a title at the top, or the like, such as "comprehensive information" or the like.
Taking a page element outline as a rectangular frame as an example, the area coordinate information corresponding to the page element is obtained as follows: the text information text is "product week notice", and the coordinate information of four vertexes of the rectangular frame area:
x：498.525，y：1401.702；
x：492.745，y：1180.304；
x：562.105，y：1178.838；
x：567.885，y：1398.770。
FIG. 3 schematically illustrates a flow for automatically generating a skeletal screen in one embodiment of the present invention. As shown in fig. 3, in the step of automatically generating the skeleton screen according to the region coordinate information, the method specifically may include:
step S310, according to the point coordinates in the area coordinate information of each page element, the width and the height of the area block corresponding to each page element are calculated. The point coordinates may be, for example, point coordinates of two end points on a certain diagonal line. When the area of the page element is circular, the outline of the block area can also be determined according to the circle center, the radius and the like.
Step S320, automatically generating a skeleton screen according to the region coordinate information and the width and the height of the region blocks. Considering page performance and rationality, the width and the height of the region can be calculated according to the region coordinate information, and then the skeleton screen is regenerated.
FIG. 4 schematically illustrates a flow of one embodiment for automatically generating a skeletal screen. As shown in fig. 4, taking a page element outline as a rectangular frame as an example, the step S320 may include the following steps:
step S321, positioning the left side and the top of the area block according to the coordinates of the upper left corner;
step S322, positioning the right side and the bottom of the area block according to the coordinates of the lower right corner;
Step S323, giving a region block pattern according to the width and the height of the region block, and automatically generating a skeleton screen. Specifically, according to the specification requirement of the H5 static page designed in the row, a rem unit is used, and a layout scheme in the row is compatible to generate a skeleton screen. The layout scheme refers to a scheme of multi-terminal adaptation of the H5 page, namely, the layout of the page completed when the developer completes the development of the page function. In the embodiment, the rectangular frame is taken as an example for explanation, the rectangular frame belongs to area positioning, and not specific positioning of a certain point, the width and the height of an area are calculated according to the area coordinate information in consideration of the performance and the rationality of a page, then the area block style is finally given according to the width and the height of an area block and the area coordinate information, and then a skeleton screen is automatically generated. The region block pattern may include a position, a size, a background color, etc. of a color block, for example, a gray block is described as an example in the present invention, but the present invention is not limited thereto, and may be other color blocks.
In an alternative embodiment, the step of giving the gray area block pattern and automatically generating the skeleton screen may include: giving gray area block patterns and converting the gray area block patterns into skeleton screen codes; automatically injecting the skeleton screen code into an html file; and (5) performing page rendering, and obtaining the skeleton screen when the codes are analyzed. The skeleton screen code, that is, the color block code, is the color block described in the present invention, for example, the gray area block code. Optionally, in the step of automatically injecting the skeleton screen code into the html file, the skeleton screen code may be specifically automatically injected into the html file by adopting a mode of externally referring to the Js file.
FIG. 7 schematically illustrates the structure of a skeleton screen for integrating information in the practice of the present application; that is, the skeleton screen finally obtained by adopting the comprehensive information list page loaded with real data as shown in fig. 5, screenshot and OCR recognition can be seen, and in this embodiment of the present application, the skeleton screen corresponding to the comprehensive information list page is automatically generated by adopting the comprehensive information list page loaded with real data, so that the consistency of the skeleton screen structure and the page structure can be ensured, and the development flow of the skeleton screen can be simplified. As shown in fig. 7, the left side is a text gray area block corresponding to fig. 5, and the right side is a picture gray area block corresponding to fig. 5. According to the embodiment of the application, the automatic generation of the skeleton screen is dynamically realized through the js code at the front end, so that the development flow of the skeleton screen is simplified, the use mode of the skeleton screen is simplified, and the development and maintenance cost is reduced.
In the embodiment of the invention, the skeleton screen generation method based on picture recognition combines the characteristics of simple skeleton screen layout and single style, adopts OCR picture recognition technology to recognize the page loaded with real data to obtain the region coordinate information of the text and the picture, then obtains the width and the height of the region block according to the region coordinate information of the text and the picture, and further automatically generates the skeleton screen, thereby ensuring the consistency of the skeleton screen structure and the page structure, simplifying the development flow of the skeleton screen and reducing the development and maintenance cost.
In the embodiment of the invention, the skeleton screen generating method based on picture identification greatly reduces the white screen time, optimizes the page loading experience and improves the user experience. The "white screen" time of the skeleton screen refers to the time before the real content (dynamically loaded) of the page reaches the browser, and is the time of network loading. When the page is loaded in the same network environment, the white screen time of the prior art is about 990 milliseconds, but the white screen time of the invention can be reduced to 30 milliseconds, and the skeleton screen is displayed for 970 milliseconds, so that the invention greatly reduces the white screen time and improves the user experience; in addition, under the environment of poor network, the technical effect of short white screen time of the invention is more obvious compared with the prior art. The invention is suitable for most single-page application projects at present, and html files only need to contain the simple skeleton screen codes, so that the content is less and the loading is quick.
In the embodiment of the invention, the skeleton screen generation method based on picture identification does not need to be participated by a designer, does not need to develop a manual skeleton screen code, and shortens the development flow and period. In the embodiment of the invention, the skeleton screen generation method based on picture identification does not involve analysis of the page code structure, does not need configuration, has simple and flexible use mode and wide application range.
Fig. 8 schematically shows the structure of a skeleton screen generating device based on picture recognition according to an embodiment of the present application. As shown in fig. 8, the application provides a skeleton screen generating device based on picture recognition, which includes: an acquisition unit 100, an identification unit 200, and a generation unit 300.
The acquiring unit 100 is configured to acquire a screenshot corresponding to a page loaded with real data.
The recognition unit 200 is configured to perform OCR recognition on the screenshot to obtain region coordinate information corresponding to a page element.
The generating unit 300 is configured to automatically generate a skeleton screen according to the region coordinate information.
In an alternative embodiment, the identifying unit 200 may include: the device comprises a preprocessing module, a dividing module and a region coordinate information acquisition module. The preprocessing module is used for executing the step of preprocessing the screenshot. Optionally, the preprocessing module may further include sub-modules such as graying processing, noise reduction processing, tilt correction, and image cropping. The dividing module is used for executing the steps of carrying out layout division on the preprocessed screenshot to obtain the area of each page element. The region coordinate information acquisition module is used for executing the step of acquiring region coordinate information corresponding to the page element according to the page element region.
In an alternative embodiment, the generating unit 300 may include: and the regional block calculation module and the skeleton screen generation module. The area block calculation module is used for executing the step of calculating the width and the height of the area block according to the point coordinates in the area coordinate information of each page element. The skeleton screen generating module is used for executing the step of automatically generating the skeleton screen according to the region coordinate information and the width and the height of the region blocks.
Optionally, the skeleton screen generating module may include: a conversion sub-module, an injection sub-module, and a rendering sub-module. The conversion sub-module is used for executing the step of converting the given regional block style into skeleton screen codes. The injection sub-module is used for executing the step of automatically injecting the skeleton screen code into an html file. And the rendering sub-module is used for executing page rendering to obtain a skeleton screen.
In an alternative embodiment, the generating unit 300 may further include a positioning sub-module. The positioning sub-module may be configured to perform the step of positioning the left and top of the region block according to the upper left corner coordinates; and a step for performing positioning of the right and bottom of the region block according to the lower right corner coordinates.
In the embodiment of the application, the skeleton screen generating device based on picture recognition adopts OCR picture recognition technology to recognize the page loaded with real data by combining the characteristics of simple layout and single style of the skeleton screen to obtain page elements, and then automatically generates the skeleton screen according to the corresponding region coordinate information, thereby ensuring the consistency of the skeleton screen structure and the page structure, simplifying the development flow of the skeleton screen and reducing the development and maintenance cost; the generated skeleton screen code structure is very clear, reasonable and small in size; thereby greatly reducing the white screen time, optimizing the page loading experience and improving the user experience; moreover, no designer participates, and manual skeleton screen code development is not needed, so that the development flow and period are shortened; the method does not involve analysis of the page code structure, does not need configuration, has simple and flexible use mode and wide application range, is suitable for most single-page application projects at present, and html files only need to contain the simple skeleton screen codes, so that the content is less and the loading is quick.
Fig. 9 schematically shows the structure of an electronic device in the present invention. As shown in fig. 9, in the present invention, the electronic device 1 includes: a memory 11, a processor 10, a bus, and a computer program stored in the memory 11 and executable on the processor 10, such as a skeleton screen generating program 12. The steps of the skeleton screen generating method based on picture recognition of the present invention are realized when the processor 10 executes the computer program. For example, obtaining a screenshot corresponding to a page loaded with real data is realized; OCR recognition is carried out on the screenshot, and region coordinate information corresponding to the page element is obtained; and automatically generating a skeleton screen according to the region coordinate information.
The memory 11 includes at least one type of readable storage medium including flash memory, a removable hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. The memory may in some embodiments be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. The memory 11 may in other embodiments also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart memory card (SMART MEDIA CARD, SMC), a Secure Digital (SD) card, a flash memory card (FLASH CARD) or the like, which are provided on the electronic device 1. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device 1. The memory 11 may be used not only for storing application software installed in the electronic device 1 and various types of data, such as codes of the skeleton screen generating program 12, but also for temporarily storing data that has been output or is to be output.
The processor 10 may be comprised of integrated circuits in some embodiments, for example, a single packaged integrated circuit, or may be comprised of multiple integrated circuits packaged with the same or different functions, including one or more central processing units (Central Processing unit, CPU), microprocessors, digital processing chips, graphics processors, combinations of various control chips, and the like. The processor 10 is a Control Unit (Control Unit) of the electronic device 1, connects respective components of the entire electronic device 1 using various interfaces and lines, executes or executes programs or modules stored in the memory 11, and invokes data stored in the memory 11 to perform various functions of the electronic device 1 and process data.
The bus may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, PCI) bus, or an extended industry standard architecture (extended industry standard architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. The bus is arranged to enable a connection communication between the memory 11 and at least one processor 10 etc.
It will be appreciated by a person skilled in the art that the electronic device 1 with the above-described components is not limiting of the electronic device 1, and may comprise fewer or more components, or may be combined with certain components, or may be arranged in different components. For example, although not shown, the electronic device 1 may further include a power source (such as a battery) for supplying power to each component, and preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that functions of charge management, discharge management, power consumption management, and the like are implemented through the power management device. The power supply may also include one or more of any of a direct current or alternating current power supply, recharging device, power failure detection circuit, power converter or inverter, power status indicator, etc. The electronic device 1 may further include various sensors, bluetooth modules, wi-Fi modules, etc., which will not be described herein.
Further, the electronic device 1 may also comprise a network interface, optionally the network interface may comprise a wired interface and/or a wireless interface (e.g. WI-FI interface, bluetooth interface, etc.), typically used for establishing a communication connection between the electronic device 1 and other electronic devices.
Optionally, the electronic device 1 may further comprise a user interface, which may be a Display, an input unit, such as a Keyboard (Keyboard), or a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like. The display may also be referred to as a display screen or display unit, as appropriate, for displaying information processed in the electronic device 1 and for displaying a visual user interface.
It should be understood that the embodiments described are for illustrative purposes only and are not limited to this configuration in the scope of the patent application.
The skeleton screen generating program 12 stored in the memory 11 of the electronic device 1 is a combination of instructions, which when executed in the processor 10, may implement the steps of the method, for example: obtaining a screenshot corresponding to a page loaded with real data; OCR recognition is carried out on the screenshot, and region coordinate information corresponding to the page element is obtained; automatically generating a skeleton screen according to the region coordinate information; or may implement the functions of each unit and/or module, for example, implement the function of the acquiring unit 100, for acquiring the screenshot corresponding to the page loaded with the real data; the recognition unit 200 is configured to perform OCR recognition on the screenshot to obtain area coordinate information corresponding to the page element. The generating unit 300 is configured to automatically generate a function of the skeleton screen according to the region coordinate information.
In the present invention, the modules/units integrated in the electronic device 1 may be stored in a computer readable storage medium if implemented in the form of software functional units and sold or used as a stand alone product. The computer readable storage medium may be any tangible medium containing or storing a program or instructions for executing a computer program stored thereon, where the computer program is executed by the processor 10, where the functions of each step in the skeleton screen generating method based on picture recognition or each unit/module in the system are implemented by hardware related to the stored program instructions. For example, obtaining a screenshot corresponding to a page loaded with real data is realized; OCR recognition is carried out on the screenshot, and region coordinate information corresponding to the page element is obtained; and automatically generating a skeleton screen according to the region coordinate information. The computer readable medium may include, for example: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM).
In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus and device may be implemented in other manners. The system embodiments described above are merely illustrative, e.g., the division of the modules is merely a logical functional division, and other manners of division are possible in actual practice.
The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional module in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units can be realized in a form of hardware or a form of hardware and a form of software functional modules.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.
The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. A plurality of units or modules recited in the apparatus claims can also be implemented by means of one unit or module in software or hardware. The terms second, etc. are used to denote a name, but not any particular order.
Finally, it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention.
Claims (6)
1. A skeleton screen generation method based on picture recognition, the method comprising:
Acquiring a screenshot corresponding to a page loaded with real data;
OCR recognition is carried out on the screenshot through an OCR platform, and region coordinate information corresponding to the page element is obtained, wherein the region coordinate information comprises the page element type and the point coordinates of the page element; comprising the following steps: preprocessing the screenshot, wherein the preprocessing comprises graying processing, noise reduction processing, inclination correction and image clipping; dividing a logic area of the preprocessed screenshot to obtain a logic area, wherein the logic area comprises a title bar, a left column, a right column, an upper column and a lower column; finely dividing the logic area to obtain page elements, wherein the page elements comprise texts and pictures; judging the shape outline of the page element according to the pixel characteristics to obtain the region coordinate information corresponding to the page element;
According to the region coordinate information, automatically generating a skeleton screen; comprising the following steps: according to the point coordinates in the area coordinate information of each page element, calculating to obtain the width and the height of the area block corresponding to each page element; automatically generating a skeleton screen according to the region coordinate information and the width and the height of the region blocks;
Wherein, the step of automatically generating the skeleton screen according to the region coordinate information and the width and the height of the region blocks comprises the following steps: positioning the left side and the top of the area block according to the coordinates of the upper left corner point; positioning the right side and the bottom of the area block according to the coordinates of the lower right corner point; giving a region block pattern according to the width and the height of the region block, and automatically generating a skeleton screen;
The step of giving the region block style and automatically generating the skeleton screen comprises the following steps: giving a region block pattern and converting the region block pattern into a skeleton screen code; automatically injecting the skeleton screen code into an html file; and rendering the page to obtain the skeleton screen.
2. The skeleton screen generation method based on picture recognition according to claim 1, wherein the skeleton screen code is automatically injected into an html file by adopting a mode of externally referring to Js files.
3. The picture recognition-based skeleton screen generation method of claim 1, wherein the real data is real page data presented to a user.
4. The utility model provides a skeleton screen generating device based on picture discernment which characterized in that includes:
The acquisition unit is used for acquiring a screenshot corresponding to the page loaded with the real data; wherein the real data is real page data presented to a user;
The recognition unit is used for carrying out OCR recognition on the screenshot through an OCR platform to obtain region coordinate information corresponding to the page element; comprising the following steps: the system comprises a preprocessing module, a dividing module and a region coordinate information acquisition module, wherein the preprocessing module is used for carrying out grey processing, noise reduction processing, inclination correction and image cutting preprocessing on a screenshot; the division module is used for dividing a logic area of the preprocessed screenshot to obtain a logic area, and finely dividing the logic area to obtain page elements, wherein the logic area comprises a title bar, a left column, a right column, an upper column and a lower column; the region coordinate information acquisition module is used for judging the shape outline of the page element according to the pixel characteristics to obtain the region coordinate information corresponding to the page element; wherein the page element comprises text and pictures; the region coordinate information comprises a page element type and point coordinates of the page element;
The generating unit is used for automatically generating a skeleton screen according to the region coordinate information; the generation unit includes: the area block calculation module and the skeleton screen generation module; the area block calculation module is used for calculating the width and the height of the area block according to the point coordinates in the area coordinate information of each page element; the skeleton screen generation module is used for automatically generating a skeleton screen according to the region coordinate information and the width and the height of the region blocks;
The generating unit further comprises a positioning submodule, wherein the positioning submodule is used for positioning the left side and the top of the area block according to the coordinates of the upper left corner; positioning the right side and the bottom of the area block according to the coordinates of the lower right corner point; giving a region block pattern according to the width and the height of the region block;
The skeleton screen generation module comprises: a conversion sub-module, an injection sub-module, and a rendering sub-module; the conversion submodule is used for converting the given regional block style into a skeleton screen code; the injection submodule is used for automatically injecting the skeleton screen code into an html file; the rendering sub-module is used for page rendering to obtain a skeleton screen.
5. An electronic device, the electronic device comprising:
at least one processor; and
A memory communicatively coupled to the at least one processor; wherein,
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the picture-recognition-based skeleton screen generation method of any one of claims 1-3.
6. A computer-readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the steps of the picture-recognition-based skeleton screen generation method of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011594807.4A CN112633270B (en) | 2020-12-29 | 2020-12-29 | Skeleton screen generation method, device and equipment based on picture identification and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011594807.4A CN112633270B (en) | 2020-12-29 | 2020-12-29 | Skeleton screen generation method, device and equipment based on picture identification and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112633270A CN112633270A (en) | 2021-04-09 |
| CN112633270B true CN112633270B (en) | 2024-07-05 |
Family
ID=75286100
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011594807.4A Active CN112633270B (en) | 2020-12-29 | 2020-12-29 | Skeleton screen generation method, device and equipment based on picture identification and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112633270B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113553523A (en) * | 2021-06-10 | 2021-10-26 | 北京达佳互联信息技术有限公司 | Method and device for generating skeleton screen page, electronic equipment, medium and product |
| CN113569177A (en) * | 2021-07-16 | 2021-10-29 | 上海淇玥信息技术有限公司 | Method and device for optimizing skipped web pages and electronic equipment |
| CN116781771B (en) * | 2023-08-21 | 2023-11-17 | 南京粒聚智能科技有限公司 | Automatic screen capturing picture analysis method of station machine by using OCR technology |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110489682A (en) * | 2019-07-08 | 2019-11-22 | 北京三快在线科技有限公司 | A kind of page loading method and device |
| CN112015634A (en) * | 2019-05-31 | 2020-12-01 | 百度在线网络技术（北京）有限公司 | Page structure information generation method and device and electronic equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112100543B (en) * | 2020-08-31 | 2023-10-27 | 东软集团股份有限公司 | Webpage rendering method and device, storage medium, electronic equipment and server |
-
2020
- 2020-12-29 CN CN202011594807.4A patent/CN112633270B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112015634A (en) * | 2019-05-31 | 2020-12-01 | 百度在线网络技术（北京）有限公司 | Page structure information generation method and device and electronic equipment |
| CN110489682A (en) * | 2019-07-08 | 2019-11-22 | 北京三快在线科技有限公司 | A kind of page loading method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112633270A (en) | 2021-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112633270B (en) | Skeleton screen generation method, device and equipment based on picture identification and storage medium | |
| CN110069257B (en) | Interface processing method and device and terminal | |
| CN110377285B (en) | Method and device for generating page skeleton screen and computer equipment | |
| CN101740004B (en) | Picture processing using a hybrid system configuration | |
| CN109389661B (en) | Animation file conversion method and device | |
| CN106095437B (en) | The implementation method and device of the user interface layout type of RTL from right to left | |
| CN108010112A (en) | Animation processing method, device and storage medium | |
| CN113095995B (en) | Webpage watermark adding method and device, electronic equipment and storage medium | |
| CN112991377A (en) | Keying algorithm based on opencv and shader fusion technology | |
| CN113222874B (en) | Data enhancement method, device, equipment and storage medium applied to target detection | |
| CN112233125A (en) | Image segmentation method and device, electronic equipment and computer readable storage medium | |
| CN111476760B (en) | Medical image generation method and device, electronic equipment and medium | |
| CN110969004B (en) | Automatic typesetting method and system for graphics context, server and medium | |
| CN112528909A (en) | Living body detection method, living body detection device, electronic apparatus, and computer-readable storage medium | |
| US9153193B2 (en) | Primitive rendering using a single primitive type | |
| CN114004591A (en) | Automatic user-defined form auditing method and device, electronic equipment and storage medium | |
| CN111932563B (en) | Picture region segmentation method and device, electronic equipment and storage medium | |
| CN112486807A (en) | Pressure testing method and device, electronic equipment and readable storage medium | |
| CN112686939A (en) | Depth image rendering method, device and equipment and computer readable storage medium | |
| CN112233194B (en) | Medical picture optimization method, device, equipment and computer readable storage medium | |
| CN113741835B (en) | Trotting horse lamp effect realization method and trotting horse lamp effect realization device | |
| CN103488619B (en) | Method and device for processing document file | |
| CN115238667A (en) | Poster generation method, system, device and storage medium | |
| CN113010129A (en) | Virtual studio full-flow multi-terminal blackboard writing extraction method and device | |
| CN107704240A (en) | A kind of ink screen development system based on Raspberry Pi and Pygame platforms |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |