CN113110802B - Image block processing method and device, electronic equipment and readable storage medium - Google Patents

Image block processing method and device, electronic equipment and readable storage medium Download PDF

Info

Publication number
CN113110802B
CN113110802B CN202110413137.XA CN202110413137A CN113110802B CN 113110802 B CN113110802 B CN 113110802B CN 202110413137 A CN202110413137 A CN 202110413137A CN 113110802 B CN113110802 B CN 113110802B
Authority
CN
China
Prior art keywords
image block
image
module
target
coordinates
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110413137.XA
Other languages
Chinese (zh)
Other versions
CN113110802A (en
Inventor
张兴进
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Vivo Mobile Communication Co Ltd
Original Assignee
Vivo Mobile Communication Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vivo Mobile Communication Co Ltd filed Critical Vivo Mobile Communication Co Ltd
Priority to CN202110413137.XA priority Critical patent/CN113110802B/en
Publication of CN113110802A publication Critical patent/CN113110802A/en
Priority to PCT/CN2022/086096 priority patent/WO2022218253A1/en
Application granted granted Critical
Publication of CN113110802B publication Critical patent/CN113110802B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0625Power saving in storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/064Management of blocks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0656Data buffering arrangements
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Memory System Of A Hierarchy Structure (AREA)
  • Image Processing (AREA)
  • Image Input (AREA)

Abstract

The embodiment of the application provides a method and a device for processing an image block, electronic equipment and a readable storage medium. The processing method of the image block comprises the following steps: receiving a first read request for a target image, the first read request including acquiring coordinates of a first image block and coordinates of a second image block in the target image; obtaining an overlapping area of the first image block and the second image block according to the coordinates of the first image block and the coordinates of the second image block; and writing the pixel block data corresponding to the non-overlapping region in the first image block into the buffer memory module according to the overlapping region, or discarding writing the pixel block data corresponding to the first image block into the buffer memory module.

Description

Image block processing method and device, electronic equipment and readable storage medium
Technical Field
The present application relates to the field of data transmission technology, and in particular, to a method for processing an image block, a device for processing an image block, an electronic device, a readable storage medium, and a chip.
Background
On handheld devices provided with ultra-wide angle cameras, there is often a problem of image distortion. In the related art, however, the image distortion correction algorithm often needs to process the image block unit to obtain a corrected image after correction. The Cache memory (Cache) can only write the image into DDR (double rate synchronous dynamic random access memory) first, then read the required image block from DDR, and send to the memory to make interpolation operation to obtain corrected image. Although the cache memory is capable of Direct Memory Access (DMA), there is a possibility of overlapping between image blocks read from the DDR due to image distortion, which results in direct memory access of repeated image data, which puts a certain pressure on the bandwidth of the DDR bus, while there is a waste of power consumption.
Disclosure of Invention
The embodiment of the application provides a processing method, a device, electronic equipment and a readable storage medium of an image block, which can effectively reduce the bandwidth and power consumption required during reading and reduce the requirement of a memory on operation capability.
In a first aspect, an embodiment of the present application provides a method for processing an image block, including:
Receiving a first read request for a target image, the first read request including acquiring coordinates of a first image block and coordinates of a second image block in the target image;
Obtaining an overlapping area of the first image block and the second image block according to the coordinates of the first image block and the coordinates of the second image block;
And writing the pixel block data corresponding to the non-overlapping region in the first image block into the buffer memory module according to the overlapping region, or discarding writing the pixel block data corresponding to the first image block into the buffer memory module.
In a second aspect, an embodiment of the present application provides an apparatus for processing an image block, including:
A first request module for receiving a first read request for a target image, the first read request including coordinates of a first image block and coordinates of a second image block in the target image;
The computing module is used for obtaining an overlapping area of the first image block and the second image block according to the coordinates of the first image block and the coordinates of the second image block;
And the writing module is used for writing the pixel block data corresponding to the non-overlapping area in the first image block into the buffer memory module according to the overlapping area, or discarding writing the pixel block data corresponding to the first image block into the buffer memory module.
In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, and a program or instructions stored on the memory and executable on the processor, the program or instructions implementing the steps of the method for processing image blocks as provided in the first aspect when executed by the processor.
In a fourth aspect, embodiments of the present application provide a readable storage medium having stored thereon a program or instructions which, when executed by a processor, implement the steps of the method of processing image blocks as provided in the first aspect.
In a fifth aspect, an embodiment of the present application provides a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being configured to execute programs or instructions to implement the steps of the method for processing image blocks as provided in the first aspect.
In the embodiment of the application, when image data processing is required, pixel block data is read from a memory module (DDR) according to the coordinates of each image block of a target image to be processed in a received first reading request, and the pixel block data is written into a cache module so as to be convenient for subsequent processing. In addition, a second image block adjacent to the first image block is determined using the coordinates, and an overlapping area between the adjacent image blocks is determined using the coordinates of the first image block and the coordinates of the second image block. If the overlapping area exists, the problem of repeated writing of partial pixel block data is solved, and at the moment, the pixel block data corresponding to the non-overlapping area in the first image block is written into the buffer module through the overlapping area. If the overlapping area does not exist, the pixel block data corresponding to the first image block and the pixel block data corresponding to the second image block are different, and writing the pixel block data corresponding to the first image block into the buffer memory module is abandoned. On one hand, the overlapping area of repeated reading and writing is avoided, the bandwidth and the power consumption required during reading are effectively reduced, the requirement of the cache module on the computing capacity is reduced, and on the other hand, the coordinates of the image blocks are used as positioning basis, so that the processing efficiency of the image blocks is improved when the image blocks are in butt joint with an image processor.
Drawings
FIG. 1 illustrates one of the flowcharts of a method of processing an image block according to one embodiment of the application;
FIG. 2 shows a second flowchart of a method of processing an image block according to one embodiment of the application;
FIG. 3 illustrates a third flowchart of a method of processing an image block according to one embodiment of the application;
FIG. 4 shows a fourth flowchart of a method of processing an image block according to one embodiment of the application;
FIG. 5 shows a fifth flow chart of a method of processing an image block according to one embodiment of the application;
FIG. 6 illustrates a schematic diagram of a cache module according to one embodiment of the application;
FIG. 7 shows a block diagram of an image block processing apparatus according to one embodiment of the application;
FIG. 8 shows a logical block diagram of an image block processing apparatus according to one embodiment of the application;
FIG. 9 shows one of the block diagrams of the electronic device according to one embodiment of the application;
FIG. 10 shows a second block diagram of an electronic device according to one embodiment of the application;
fig. 11 shows a hardware configuration diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order that the above-recited objects, features and advantages of the present application will be more clearly understood, a more particular description of the application will be rendered by reference to the appended drawings and appended detailed description. It should be noted that, without conflict, the embodiments of the present application and features in the embodiments may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application, but the present application may be practiced in other ways than those described herein, and therefore the scope of the present application is not limited to the specific embodiments disclosed below.
A processing method of an image block, a processing apparatus of an image block, an electronic device, a readable storage medium, and a chip according to some embodiments of the present application are described below with reference to fig. 1 to 11.
As shown in fig. 1, in one embodiment of the present application, a method for processing an image block is provided, including:
Step 102, receiving a first reading request of a target image;
wherein the first read request includes acquiring coordinates of a first image block and coordinates of a second image block in the target image.
In this embodiment, the lens module of the electronic device is generally configured with default lens distortion parameters, and when performing image processing, for example, correction processing, the image processor can calculate coordinates corresponding to an image block in a target image to be processed of each image block in the processed corrected image according to a distortion algorithm and the lens distortion parameters, and the image processor generates a first reading request according to the coordinates so as to read pixel block data required in the memory module.
It is understood that the coordinates of the image block are the coordinates of any pixel point in the image block in the whole target image. For the convenience of statistics, all image blocks in the target image adopt pixel points at the same position to determine the coordinates of the image blocks. For example, the coordinates of the image block are set as the coordinates of the pixel point in the upper left corner of the image block in the target image. The coordinate dimensions of the image block are the width and height of the image block.
104, Obtaining an overlapping area of the first image block and the second image block according to the coordinates of the first image block and the coordinates of the second image block;
and step 106, writing the pixel block data corresponding to the non-overlapping region in the first image block into the buffer memory module according to the overlapping region, or discarding writing the pixel block data corresponding to the first image block into the buffer memory module.
In this embodiment, when image data processing is required, pixel block data is read from a memory module (DDR) according to coordinates of each image block of a target image to be processed in a received first read request, and the pixel block data is written into a buffer module for use in subsequent processing. In addition, a second image block adjacent to the first image block is determined using the coordinates, and an overlapping area between the adjacent image blocks is determined using the coordinates of the first image block and the coordinates of the second image block. If the overlapping area exists, the problem of repeated writing of partial pixel block data is solved, and at the moment, the pixel block data corresponding to the non-overlapping area in the first image block is written into the buffer module through the overlapping area. If the overlapping area does not exist, the pixel block data corresponding to the first image block and the pixel block data corresponding to the second image block are different, and writing the pixel block data corresponding to the first image block into the buffer memory module is abandoned. On one hand, the overlapping area of repeated reading and writing is avoided, the bandwidth and the power consumption required during reading are effectively reduced, the requirement of the cache module on the computing capacity is reduced, and on the other hand, the coordinates of the image blocks are used as positioning basis, so that the processing efficiency of the image blocks is improved when the image blocks are in butt joint with an image processor.
The memory module (DDR) is used for storing data required by the operation of the electronic equipment. The buffer module may be a Random Access Memory (RAM), such as a Static Random Access Memory (SRAM), which includes a plurality of banks (banks) therein, so that a plurality of pixel block data can be read out from the banks at the same time in one clock cycle.
The target image to be corrected, i.e., the original image, is processed by the distortion correction algorithm to obtain a corrected image, where the corrected image is composed of a plurality of image blocks, i.e., each image block in the corrected image is obtained from a corresponding region in the target image by the algorithm. Since the image is distorted such that the size and coordinates of the corresponding region in the target image are different from those of the image block of the target image, overlapping may occur in the adjacent regions in the target image corresponding to the image blocks of the adjacent corrected images.
As shown in fig. 2, step 106, writing pixel block data corresponding to a non-overlapping region in the first image block into the buffer module according to the overlapping region includes:
Step 202, determining a non-overlapping region in the first image block according to the overlapping region when the size of the overlapping region is smaller than the size of the first image block;
step 204, reading pixel block data corresponding to the non-overlapping area from the memory module;
and 206, writing the pixel block data corresponding to the non-overlapping area into the buffer module.
In this embodiment, it is possible to determine whether or not there is an overlap region between the front and rear image blocks, and determine the size of the overlap region, based on the positional relationship (coordinates) of the first image block and the second image block. If the size of the overlapping area is smaller than the size of the first image block, the second image block only contains partial pixel block data of the first image block or does not contain pixel block data of the first image block at all, and at this time, reading and writing of the pixel block data of the first image block are required to be normally carried out, a non-overlapping area in the first image block is determined through the overlapping area, and the pixel block data corresponding to the non-overlapping area in the memory module is written into the buffer module according to the size of the non-overlapping area. Therefore, when the target image is cached, even if the problem of overlapping between the image blocks occurs, pixel block data of an overlapping area cannot be read from the memory module, so that the bandwidth waste caused by displacement of the image blocks in the y direction can be reduced to the greatest extent, and the power consumption of the memory module is reduced.
Specifically, the required data may be read from the memory module via the memory bus (cacheline).
It will be appreciated that the second image block may be either the preceding image block or the following image block of the first image block, in order of reading.
For example, if the first image block is not hit, i.e. the size of the overlapping area is smaller than the size of the first image block, the second image block does not cover the first image block completely, which means that the first image block needs to be fetched from the DDR and then switched to the next image block in the RAM.
As shown in fig. 3, step 106, discarding writing pixel block data corresponding to the first image block into the buffer module according to the overlapping area includes:
In step 302, when the size of the overlapping area is equal to the size of the first image block, the reading of the pixel block data corresponding to the first image block from the memory module is abandoned, and the writing of the pixel block data corresponding to the first image block into the buffer module is abandoned.
In this embodiment, it is possible to determine whether or not there is an overlap region between the front and rear image blocks, and determine the size of the overlap region, based on the positional relationship (coordinates) of the first image block and the second image block. If the size of the overlapping area is not smaller than the size of the first image block, that is, the size of the overlapping area is equal to the size of the first image block, it is indicated that the second image block includes all pixel block data of the first image block, that is, when the second image block is written, all pixel block data of the first image block can be written into the buffer memory module, after receiving the first reading request, all pixel block data corresponding to the first image block is abandoned to be read from the memory module, and all pixel block data corresponding to the first image block cannot be written into the buffer memory module. Therefore, when the target image is cached, the same pixel block data is prevented from being repeatedly read from the memory module, and further, the bandwidth waste caused by displacement of the image block in the y direction can be reduced to the greatest extent, and meanwhile, the power consumption of the memory module is reduced.
For example, if a first tile is hit, i.e. the size of the overlap area is equal to the size of the first tile, i.e. the second tile completely covers the first tile. If the pixel block data corresponding to the second image block is already written into the RAM, the first image block data is already completely present in the RAM. At this time, it is not necessary to read the pixel block data corresponding to the first image block from the DDR and write it into the RAM. But still requires tag information when reading the pixel block data corresponding to the image block from RAM to indicate the fetch from the second image block when reading the first image block from RAM.
As shown in fig. 4, step 204 of reading pixel block data corresponding to the non-overlapping area from the memory module according to the non-overlapping area includes:
Step 402, determining a target row of the first image block and a target row coordinate corresponding to the target row in the memory module according to the size of the non-overlapping area, the preset read size and the preset row coordinate threshold;
The preset reading size is used for aligning the data coordinates so as to avoid incomplete data reading.
In this embodiment, in the case where the second image block does not completely cover the first image block, the reading coordinates of the pixel block data to be read in the image block are determined according to the sum of the size of the non-overlapping area and the preset reading size. When the pixel block data to be read is determined, calculation is performed according to the sum of the size of the non-overlapping area and the preset reading size, so that the pixel block data actually read is more than the pixel block data contained in the size range of the non-overlapping area, the problem that the subsequent data writing is affected due to the data loss caused by the occurrence of a problem in the reading process is avoided, the incomplete buffering of a target image is prevented, and the reading fault tolerance is enhanced.
For example, assuming that the coordinates and width of the second image block are (x 1, y1, w1, h 1) and the coordinates and width of the first image block are (x 2, y2, w2, h 2), the width of the overlapping region is MAX (0, (x1+w1+x_ext-x 2)), the height of the overlapping is MAX (0, (y1+h1+y_ext-y 2)), and if the width of the overlapping region is equal to the width of the first image block, the first image block is hit. The x_ext is a pre-fetch portion of the buffer module, that is, a preset read size, and is calculated according to the preset read size, so that the actually read pixel block data is more than the pixel block data included in the size range of the non-overlapping area. Therefore, the problem that the subsequent data writing is affected due to the data loss caused by the occurrence of problems in the reading process is avoided, the incomplete caching of the target image is prevented, and the reading fault tolerance is enhanced.
Further, since the coordinates of the pixel block data to be read are dense and discontinuous, if each discontinuous pixel block data generates a second read request (AXIburst), more AXIburst will be generated, more times of reading from the memory module are required, and the service life of the memory module is easily reduced. Therefore, a row coordinate threshold is pre-configured, and the read coordinates and the row coordinate threshold are used to take a plurality of adjacent pixel block data with smaller space between coordinates as a target row, that is, when the distance between the coordinates of a plurality of pixel block data in a row of the image block is smaller than the row coordinate threshold, the coordinates of a plurality of pixel block data can be generated into one AXIburst. And then calculating the coordinates of the target row corresponding to the target row in the memory module so as to conveniently position the pixel block data which needs to be read from the memory module. Thereby reducing the number of times of reading data from the memory module and prolonging the service life of the memory module. Specifically, the row coordinate threshold may also be sized according to the configuration of the memory. For example, if the first image block does not hit, it is calculated which lines in the first image block need to read DDR and the start coordinates and aligned end coordinates of each line read, one line corresponding to one AXIburst command, that is, the size of the memory bus (cacheline) is dynamically adaptive, since too small AXIburst is not DDR friendly, here cacheline gives a minimum limit.
For example, as shown in fig. 6, a maximum of 4×4 pixel blocks are taken as an example. SRAMARRAY (cache module) is divided into 8 banks (banks of the cache module), 2 banks in the horizontal direction and 4 banks in the vertical direction, and the storage bit width of each bank is 4 pixels in bit width. According to the low 2 bits of the Y-axis coordinate in the read-write coordinates, 4 rows of banks in the vertical direction are selected, so that 8 pixels can be written in each clock cycle, the coordinates of the 8 pixels in the target image are aligned, wherein the upper 4 pixels are written in the left row of banks, the lower 4 pixels are written in the right row of banks, and the writing address of each bank is formed by splicing the X-axis coordinate and the Y-axis coordinate in the tag information. Thereby enabling the reduction of the area size (the size of the 4 largest image blocks) required for the SRAM.
Step 404, generating a second read request according to the target line and the target line coordinates;
in step 406, the pixel block data corresponding to the non-overlapping region is read from the memory module according to the second read request.
In this embodiment, a second read request is generated according to the target row and the target row coordinates to read the pixel block data of the required non-overlapping area from the memory module. And finally, writing the read pixel block data corresponding to the non-overlapping region into a buffer memory module so as to be convenient for obtaining the required pixel block data from the buffer memory module at any time when processing the target image.
As shown in fig. 5, in one embodiment of the present application, a method for processing an image block further includes:
Step 502, generating label information of a first image block according to the overlapping area;
Wherein the tag information includes: the size of the overlapped area, the size of the first image block and the read-write coordinates of the first image block in the buffer module.
In this embodiment, after the overlapping area between the first image block and the second image block is obtained, tag information of the first image block is generated according to the overlapping area so as to provide read-write information such as the size of the overlapping area, the size of the first image block, the read-write coordinates of the buffer module in which pixel block data is written into the buffer module, and the like for writing and reading of the buffer module, so as to provide reliable positioning data for read-write control of the memory. Therefore, when the pixel block data is required to be read from the buffer module, the required pixel block data can be quickly and accurately positioned through the read-write coordinates, so that the writing and reading speeds are improved, and the image block processing efficiency is improved.
It should be noted that the buffer module may include a plurality of banks, and the target bank for writing the pixel block data may be determined according to the width and height of the pixel block data read from the memory module. For example, a maximum of 4×4 pixel blocks are read. SRAMARRAY (composed of a plurality of SRAMs) are divided into 8 banks, 2 banks in the horizontal direction and 4 banks in the vertical direction, and the SRAM bit width of each bank is 4 pixels bit width. Wherein 4 rows of banks in the vertical direction are a first row of banks 00 and 01, a second row of banks 10 and 11, a third row of banks 20 and 21, and a fourth row of banks 30 and 31. P0, p1, p2, p3 in fig. 6 refer to the width of the memory bit width of 4 pixels.
Specifically, for example, the read-write information is calculated according to the target row coordinates of the DDR to be read by the first image block, and each first read request generates a set of tag information including whether or not hit, that is, whether or not the overlapping size is 0, the width and height of the first image block, the width and height of the overlapping area of the first image block and the second image block, and the read-write coordinates in the SRAM (cache module).
Step 504, receiving a third read request for target pixel block data of the first image block in the buffer module;
wherein the third read request includes a pixel bit width indicating the target pixel block data and the target coordinates.
Step 506, outputting the target pixel block data from the buffer module according to the pixel bit width, the target coordinates and the tag information in the third read request.
In this embodiment, after the pixel block data of the first image block of the target image has been written to the buffer module, a third read request of the target pixel block data of the first image block in the buffer module is received. And reading and outputting the target pixel block data from the cache module by using the storage bit width indicated by the third reading request, the target coordinates and the label information generated by caching the target image. For a subsequent image processor to process the target pixel block data according to a specified algorithm. For example, the image is corrected, the image processor corrects the target pixel block data according to a correction algorithm to obtain a corrected image block, and the corrected image can be output through the corrected image block.
For example, as shown in fig. 6, when the SRAM is read, a maximum of 4×4 pixel blocks need to be read from 8 banks in one clock cycle. Taking the example of reading a4×4 pixel block, the coordinates of the pixel block may be random, four rows of the 4×4 pixel block are respectively read from 4 different banks, each row of 4 pixels of the 4×4 pixel block may need to read only one column of banks, or two banks in each bank row may be read, then the required 4 pixel blocks are taken out from the read 8 pixels, and the read 4×4 pixel blocks return to the image processor in the same clock cycle.
In one embodiment of the present application, as shown in fig. 7, there is provided an image block processing apparatus 700 including: a first request module 702, the first request module 702 being configured to receive a first read request for a target image, the first read request including obtaining coordinates of a first image block and coordinates of a second image block in the target image; the computing module 704, the computing module 704 is configured to obtain an overlapping area of the first image block and the second image block according to the coordinates of the first image block and the coordinates of the second image block; the writing module 706, the writing module 706 is configured to write the pixel block data corresponding to the non-overlapping region in the first image block into the buffer module according to the overlapping region, or discard writing the pixel block data corresponding to the first image block into the buffer module.
Optionally, as shown in fig. 8, the calculating module 704 is further configured to determine, in a case where the size of the overlapping area is smaller than the size of the first image block, a non-overlapping area in the first image block according to the overlapping area; the image block processing apparatus 700 further includes: the access module 708 is configured to read pixel block data corresponding to the non-overlapping area from the memory module; the writing module 706 is further configured to write pixel block data corresponding to the non-overlapping area into the buffer module.
Optionally, the writing module 706 is further configured to discard the reading of the pixel block data corresponding to the first image block from the memory module and discard the writing of the pixel block data corresponding to the first image block into the buffer module if the size of the overlapping area is equal to the size of the first image block.
Optionally, as shown in fig. 8, the calculating module 704 is further configured to determine, according to the size of the non-overlapping area, the preset read size, and the preset row coordinate threshold, a target row of the first image block and a target row coordinate corresponding to the target row in the memory module; the image block processing apparatus 700 further includes: the first generating module 710, the first generating module 710 is configured to generate a second read request according to the target line and the target line coordinate; the accessing module 708 is further configured to read pixel block data corresponding to the non-overlapping area from the memory module according to the second read request.
Optionally, the image block processing device further includes: the second generating module is used for generating label information of the first image block according to the overlapping area, wherein the label information comprises: the size of the overlapped area, the size of the first image block and the read-write coordinates of the first image block in the cache module;
The second request module is used for receiving a third read request aiming at the target pixel block data of the first image block in the cache module, wherein the third read request comprises the storage bit width and the target coordinates of the target pixel block data; and the output module is used for outputting the target pixel block data from the buffer memory module according to the storage bit width, the target coordinates and the tag information.
In this embodiment, the steps of the image block processing method in any of the embodiments are implemented when the modules of the image block processing apparatus 700 execute the respective functions, so that the image block processing apparatus 700 also includes all the advantages of the image block processing apparatus method in any of the embodiments, which are not described herein.
The processing device of the image block in the embodiment of the application can be a device, and also can be a component, an integrated circuit or a chip in a terminal. The device may be a mobile electronic device or a non-mobile electronic device. By way of example, the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a Personal Digital Assistant (PDA), etc., and the non-mobile electronic device may be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a Television (TV), a teller machine, a self-service machine, etc., and the embodiments of the present application are not limited in particular.
The image block processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an ios operating system, or other possible operating systems, and the embodiment of the present application is not limited specifically.
In one embodiment of the present application, as shown in fig. 9, there is provided an electronic device 1000 including: the steps of the image block processing method provided in any of the above embodiments are implemented by the processor 1004, the memory 1002, and a program or an instruction stored in the memory 1002 and capable of running on the processor 1004, and therefore, the electronic device 1000 includes all the advantages of the image block processing method provided in any of the above embodiments, and are not described herein.
In one embodiment of the application, as shown in FIG. 10, the memory 1002 further comprises: a memory module 1006 coupled to the processor 1004; a cache module 1008 coupled to the processor 1004; the electronic device 1000 further includes: an image processor 1010, coupled to the memory 1002 and the processor 1004, the image processor 1010 is configured to process the target image.
Further, in some embodiments, the cache module 1008 may include a first cache module and a second cache module. The first buffer module is used for storing data. The second cache module is used for caching the tag information. If the first image block is hit, that is, the size of the overlapping area is equal to the size of the first image block, the corresponding pixel block data does not need to be read from the memory module 1006 and written into the first buffer module, but the tag information needs to be written into the second buffer module, so that when the first image block is read, the reading is performed according to the second image block. On one hand, information confusion during pixel data reading and writing can be avoided, further accuracy of data reading is improved, image processing efficiency is improved, processed image quality is guaranteed, on the other hand, memory occupation of the first cache module is reduced, storage requirements of products are reduced, and manufacturing cost of the products is reduced.
It should be noted that the second buffer module may use a first-in first-out (FIFO) principle.
The electronic equipment to be detected in the embodiment of the application can be mobile electronic equipment or non-mobile electronic equipment. By way of example, the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or Personal Digital Assistant (PDA), etc., and the non-mobile electronic device may be a server, a network attached cache module (Network Attached Storage, NAS), a personal computer (personal computer, PC), a Television (TV), a teller machine or a self-service machine, etc., and the embodiments of the present application are not limited in particular.
Fig. 11 is a schematic diagram of a hardware structure of an electronic device 1200 implementing an embodiment of the present application. The electronic device 1200 includes, but is not limited to: radio frequency unit 1202, network module 1204, audio output unit 1206, input unit 1208, sensor 1210, display unit 1212, user input unit 1214, interface unit 1216, memory 1218, processor 1220, and the like.
Those skilled in the art will appreciate that the electronic device 1200 may further include a power source (e.g., a battery) for powering the various components, and that the power source may be logically coupled to the processor 1220 via a power management system such that charge, discharge, and power consumption management functions are performed by the power management system. The electronic device structure shown in fig. 11 does not constitute a limitation of the electronic device, and the electronic device may include more or less components than illustrated, or may combine certain components, or may be arranged in different components. In an embodiment of the present application, the electronic device includes, but is not limited to, a mobile terminal, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a wearable device, a pedometer, and the like.
Wherein the processor 1220 is configured to receive a first read request for a target image, the first read request including obtaining coordinates of a first image block and coordinates of a second image block in the target image; obtaining an overlapping area of the first image block and the second image block according to the coordinates of the first image block and the coordinates of the second image block; and writing the pixel block data corresponding to the non-overlapping region in the first image block into the buffer memory module according to the overlapping region, or discarding writing the pixel block data corresponding to the first image block into the buffer memory module.
Further, the processor 1220 is further configured to determine a non-overlapping region in the first image block according to the overlapping region if the size of the overlapping region is smaller than the size of the first image block; reading pixel block data corresponding to the non-overlapping area from the memory module; and writing the pixel block data corresponding to the non-overlapping area into the buffer memory module.
Further, the processor 1220 is further configured to discard the reading of the pixel block data corresponding to the first image block from the memory module and discard the writing of the pixel block data corresponding to the first image block into the buffer module if the size of the overlapping area is equal to the size of the first image block.
Further, the processor 1220 is further configured to determine, according to the size of the non-overlapping area, the preset read size, and the preset row coordinate threshold, a target row of the first image block and a target row coordinate corresponding to the target row in the memory module; generating a second reading request according to the target row and the target row coordinates; and reading pixel block data corresponding to the non-overlapping area from the memory module according to the second reading request.
Further, the processor 1220 is further configured to generate tag information of the first image block according to the overlapping area, where the tag information includes: the size of the overlapped area, the size of the first image block and the read-write coordinates of the first image block in the cache module; receiving a third read request for target pixel block data of the first image block in the cache module, wherein the third read request comprises a storage bit width and target coordinates of the target pixel block data; and outputting the target pixel block data from the cache module according to the storage bit width, the target coordinates and the tag information.
It should be understood that, in the embodiment of the present application, the radio frequency unit 1202 may be configured to receive and transmit information or signals during a call, and specifically, receive downlink data of a base station or send uplink data to the base station. The radio frequency unit 1202 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like.
The network module 1204 provides wireless broadband internet access to users, such as helping users send and receive e-mail, browse web pages, and access streaming media, etc.
The audio output unit 1206 may convert audio data received by the radio frequency unit 1202 or the network module 1204 or stored in the memory 1218 into an audio signal and output as sound. Also, the audio output unit 1206 may also provide audio output related to a particular function performed by the electronic device 1200 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 1206 includes a speaker, a buzzer, a receiver, and the like.
The input unit 1208 is used to receive an audio or video signal. The input unit 1208 may include a graphics processor (Graphics Processing Unit, GPU) 5082 and a microphone 5084, the graphics processor 5082 processing pixel block data of still pictures or video obtained by an image capture device (e.g., a camera) in a video capture mode or an image capture mode. The processed image frames may be displayed on the display unit 1212, or stored in the memory 1218 (or other storage medium), or transmitted via the radio frequency unit 1202 or the network module 1204. The microphone 5084 may receive sound and may be capable of processing the sound into audio data, which may be converted into a format output that may be transmitted to a mobile communication base station via the radio frequency unit 1202 in case of a phone call mode.
The electronic device 1200 also includes at least one sensor 1210, such as a fingerprint sensor, pressure sensor, iris sensor, molecular sensor, gyroscope, barometer, hygrometer, thermometer, infrared sensor, light sensor, motion sensor, and other sensors.
The display unit 1212 is used to display information input by a user or information provided to the user. The display unit 1212 may include a display panel 5122, and the display panel 5122 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like.
The user input unit 1214 may be used to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the electronic device. In particular, the user input unit 1214 includes a touch panel 5142 and other input devices 5144. The touch panel 5142, also referred to as a touch screen, can collect touch operations thereon or thereabout by a user. The touch panel 5142 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch azimuth of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends the touch point coordinates to the processor 1220, and receives and executes commands sent from the processor 1220. Other input devices 5144 can include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and so forth, which are not described in detail herein.
Further, the touch panel 5142 can be overlaid on the display panel 5122, and when the touch panel 5142 detects a touch operation thereon or thereabout, the touch panel is transferred to the processor 1220 to determine the type of touch event, and then the processor 1220 provides a corresponding visual output on the display panel 5122 according to the type of touch event. The touch panel 5142 and the display panel 5122 may be two independent components or may be integrated into one component.
The interface unit 1216 is an interface for connecting an external device to the electronic apparatus 1200. For example, the external devices may include a wired or wireless headset port, an external power (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 1216 may be used to receive input (e.g., data information, power, etc.) from an external device and to transmit the received input to one or more elements within the electronic apparatus 1200 or may be used to transmit data between the electronic apparatus 1200 and an external device.
Memory 1218 may be used to store application programs as well as various data. The memory 1218 may include primarily a stored program area and a stored data area, wherein the stored program area may store an operating system, application programs required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, phonebooks, etc.) created according to the use of the mobile terminal, etc. In addition, memory 1218 may include a cache module, and may also include a nonvolatile cache module, such as at least one disk cache module, flash memory device, or other volatile solid state cache module.
Processor 1220 performs various functions of electronic device 1200 and processes data by running or executing application programs and/or modules stored in memory 1218, and invoking data stored in memory 1218, thereby performing overall monitoring of electronic device 1200. Processor 1220 may include one or more processing units; processor 1220 may integrate an application processor that primarily handles the operation of the operating system, user interfaces, applications, etc., with a modem processor that primarily communicates.
In one embodiment of the present application, there is provided a readable storage medium having stored thereon a program or instructions which, when executed by a processor, implement the steps of the image block processing method as provided in any of the embodiments described above.
In this embodiment, the readable storage medium can implement each process of the image block processing method provided in the embodiment of the present application, and achieve the same technical effects, and for avoiding repetition, a description is omitted herein.
Wherein the processor is a processor in the communication device in the above embodiment. Readable storage media include computer readable storage media such as a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic or optical disk, and the like.
The embodiment of the application also provides a chip, which comprises a processor and a communication interface, wherein the communication interface is coupled with the processor, and the processor is used for running programs or instructions to realize the processes of the embodiment of the processing method of the image block and achieve the same technical effects, so that repetition is avoided and no repeated description is given here.
It should be understood that the chips referred to in the embodiments of the present application may also be referred to as system-on-chip chips, chip systems, or system-on-chip chips, etc.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in an opposite order depending on the functions involved, e.g., the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising several instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method of the embodiments of the present application.
The embodiments of the present application have been described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present application and the scope of the claims, which are to be protected by the present application.
The above is only a preferred embodiment of the present application, and is not intended to limit the present application, but various modifications and variations can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims (9)

1. A method of processing an image block, comprising:
Receiving a first read request for a target image, the first read request including acquiring coordinates of a first image block and coordinates of a second image block in the target image;
Obtaining an overlapping area of the first image block and the second image block according to the coordinates of the first image block and the coordinates of the second image block;
Determining a non-overlapping region in the first image block according to the overlapping region, reading pixel block data corresponding to the non-overlapping region from the memory module according to the non-overlapping region, and writing the pixel block data corresponding to the non-overlapping region in the first image block into the cache module;
the reading the pixel block data corresponding to the non-overlapping area from the memory module according to the non-overlapping area includes:
Determining a target row of the first image block and a target row coordinate corresponding to the target row in a memory module according to the size of the non-overlapping area, a preset reading size and a preset row coordinate threshold;
generating a second reading request according to the target row and the target row coordinates;
And reading the pixel block data corresponding to the non-overlapping region from the memory module according to the second reading request.
2. The method for processing an image block according to claim 1, wherein writing pixel block data corresponding to a non-overlapping region in the first image block into the buffer module according to the overlapping region includes:
Determining a non-overlapping region in the first image block according to the overlapping region in the case that the size of the overlapping region is smaller than the size of the first image block;
Reading pixel block data corresponding to the non-overlapping region from a memory module;
And writing the pixel block data corresponding to the non-overlapping region into the buffer memory module.
3. The method for processing an image block according to claim 1, further comprising:
Generating label information of the first image block according to the overlapping area, wherein the label information comprises: the size of the overlapped area, the size of the first image block and the read-write coordinates of the first image block in the cache module;
Receiving a third read request for target pixel block data of the first image block in the cache module, wherein the third read request comprises a storage bit width and target coordinates of the target pixel block data;
And outputting the target pixel block data from the cache module according to the storage bit width, the target coordinates and the tag information.
4. An image block processing apparatus, comprising:
A first request module for receiving a first read request for a target image, the first read request including acquiring coordinates of a first image block and coordinates of a second image block in the target image;
The computing module is used for obtaining an overlapping area of the first image block and the second image block according to the coordinates of the first image block and the coordinates of the second image block;
The writing module is used for determining a non-overlapping area in the first image block according to the overlapping area and writing pixel block data corresponding to the non-overlapping area in the first image block into the caching module;
the access module is used for reading pixel block data corresponding to the non-overlapping area from the memory module according to the non-overlapping area;
The computing module is further configured to determine a target line of the first image block and a target line coordinate corresponding to the target line in the memory module according to the size of the non-overlapping area, a preset read size and a preset line coordinate threshold;
The first generation module is used for generating a second reading request according to the target row and the target row coordinates;
the access module is further configured to read pixel block data corresponding to the non-overlapping area from the memory module according to the second read request.
5. The apparatus for processing an image block according to claim 4, wherein,
The computing module is further configured to determine the non-overlapping region in the first image block according to the overlapping region if the size of the overlapping region is smaller than the size of the first image block;
The access module is used for reading the pixel block data corresponding to the non-overlapping area from the memory module;
the writing module is further configured to write pixel block data corresponding to the non-overlapping area into the caching module.
6. The apparatus for processing an image block according to claim 4, further comprising:
the second generating module is configured to generate tag information of the first image block according to the overlapping area, where the tag information includes: the size of the overlapped area, the size of the first image block and the read-write coordinates of the first image block in the cache module;
A second request module, configured to receive a third read request for target pixel block data of the first image block in the buffer module, where the third read request includes a storage bit width and a target coordinate of the target pixel block data;
and the output module is used for outputting the target pixel block data from the buffer memory module according to the storage bit width, the target coordinates and the tag information.
7. An electronic device comprising a processor, a memory and a program or instruction stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the method of processing image blocks as claimed in any one of claims 1 to 3.
8. A readable storage medium, characterized in that the readable storage medium has stored thereon a program or instructions which, when executed by a processor, implement the steps of the image block processing method according to any of claims 1 to 3.
9. A chip comprising a processor and a communication interface, the communication interface being coupled to the processor for running a program or instructions to implement the steps of the method of processing image blocks as claimed in any one of claims 1 to 3.
CN202110413137.XA 2021-04-16 2021-04-16 Image block processing method and device, electronic equipment and readable storage medium Active CN113110802B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202110413137.XA CN113110802B (en) 2021-04-16 2021-04-16 Image block processing method and device, electronic equipment and readable storage medium
PCT/CN2022/086096 WO2022218253A1 (en) 2021-04-16 2022-04-11 Image block processing method and apparatus, and electronic device and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110413137.XA CN113110802B (en) 2021-04-16 2021-04-16 Image block processing method and device, electronic equipment and readable storage medium

Publications (2)

Publication Number Publication Date
CN113110802A CN113110802A (en) 2021-07-13
CN113110802B true CN113110802B (en) 2024-06-04

Family

ID=76718089

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110413137.XA Active CN113110802B (en) 2021-04-16 2021-04-16 Image block processing method and device, electronic equipment and readable storage medium

Country Status (2)

Country Link
CN (1) CN113110802B (en)
WO (1) WO2022218253A1 (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113110802B (en) * 2021-04-16 2024-06-04 维沃移动通信有限公司 Image block processing method and device, electronic equipment and readable storage medium
CN113592844A (en) * 2021-08-10 2021-11-02 北京紫光展锐通信技术有限公司 Image distortion correction method and device, terminal and storage medium
CN115660941B (en) * 2022-12-27 2023-03-14 北京象帝先计算技术有限公司 Image moving method and device, electronic equipment and computer readable storage medium
CN116009792B (en) * 2023-03-24 2023-09-29 摩尔线程智能科技(北京)有限责任公司 Data reading and writing device and method in image processing and electronic equipment
CN116152043B (en) * 2023-03-24 2024-03-08 摩尔线程智能科技(北京)有限责任公司 Memory management method and device based on image processing and electronic equipment
CN116719483B (en) * 2023-08-09 2023-10-27 成都泛联智存科技有限公司 Data deduplication method, apparatus, storage device and computer readable storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0884890A1 (en) * 1997-06-12 1998-12-16 Hewlett-Packard Company Image processing method and device
CN107220930A (en) * 2017-08-02 2017-09-29 珠海全志科技股份有限公司 Fish eye images processing method, computer installation and computer-readable recording medium
CN109559270A (en) * 2018-11-06 2019-04-02 华为技术有限公司 A kind of image processing method and electronic equipment

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4882905B2 (en) * 2007-08-03 2012-02-22 セイコーエプソン株式会社 Image data processing apparatus and image data processing method
JP2014068085A (en) * 2012-09-24 2014-04-17 Kyocera Document Solutions Inc Image processing apparatus and image forming apparatus
CN111683249A (en) * 2020-06-24 2020-09-18 湖南国科微电子股份有限公司 Data reading method, device, decoder and storage medium
CN113110802B (en) * 2021-04-16 2024-06-04 维沃移动通信有限公司 Image block processing method and device, electronic equipment and readable storage medium

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0884890A1 (en) * 1997-06-12 1998-12-16 Hewlett-Packard Company Image processing method and device
CN107220930A (en) * 2017-08-02 2017-09-29 珠海全志科技股份有限公司 Fish eye images processing method, computer installation and computer-readable recording medium
CN109559270A (en) * 2018-11-06 2019-04-02 华为技术有限公司 A kind of image processing method and electronic equipment

Also Published As

Publication number Publication date
WO2022218253A1 (en) 2022-10-20
CN113110802A (en) 2021-07-13

Similar Documents

Publication Publication Date Title
CN113110802B (en) Image block processing method and device, electronic equipment and readable storage medium
CN108182043B (en) Information display method and mobile terminal
WO2021057267A1 (en) Image processing method and terminal device
CN110896451B (en) Preview picture display method, electronic device and computer readable storage medium
CN109885257B (en) Disk file reading and writing method and device
US20220086798A1 (en) Notification message processing method and terminal
CN108205398B (en) Method and device for adapting webpage animation to screen
CN111562896B (en) Screen projection method and electronic equipment
CN108257104B (en) Image processing method and mobile terminal
CN109348019B (en) Display method and device
CN111031234B (en) Image processing method and electronic equipment
CN108616772B (en) Bullet screen display method, terminal and server
CN112887608A (en) Image processing method and device, image processing chip and electronic equipment
US20200341623A1 (en) Image display method and mobile terminal
CN110830713A (en) Zooming method and electronic equipment
CN110602389A (en) Display method and electronic equipment
CN111158815B (en) Dynamic wallpaper blurring method, terminal and computer readable storage medium
CN109992192B (en) Interface display method and terminal equipment
CN109462732B (en) Image processing method, device and computer readable storage medium
US11418945B2 (en) Information processing method for RF device and mobile terminal
CN109597546B (en) Icon processing method and terminal equipment
CN111026674A (en) Data storage method and electronic equipment
CN109104573B (en) Method for determining focusing point and terminal equipment
CN108536513B (en) Picture display direction adjusting method and mobile terminal
CN111026263B (en) Audio playing method and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant