WO2014017003A1 - 更新領域検出装置 - Google Patents
更新領域検出装置 Download PDFInfo
- Publication number
- WO2014017003A1 WO2014017003A1 PCT/JP2013/002893 JP2013002893W WO2014017003A1 WO 2014017003 A1 WO2014017003 A1 WO 2014017003A1 JP 2013002893 W JP2013002893 W JP 2013002893W WO 2014017003 A1 WO2014017003 A1 WO 2014017003A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- update
- area
- update area
- division
- blocks
- Prior art date
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 84
- 238000012545 processing Methods 0.000 claims abstract description 37
- 238000000034 method Methods 0.000 claims description 85
- 230000008569 process Effects 0.000 claims description 58
- 238000004886 process control Methods 0.000 claims description 2
- 238000000638 solvent extraction Methods 0.000 abstract 7
- 238000005192 partition Methods 0.000 abstract 1
- 230000008859 change Effects 0.000 description 6
- 230000011218 segmentation Effects 0.000 description 5
- 238000013500 data storage Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000006866 deterioration Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1415—Digital output to display device ; Cooperation and interconnection of the display device with other functional units with means for detecting differences between the image stored in the host and the images displayed on the displays
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
- G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
- G09G5/39—Control of the bit-mapped memory
- G09G5/393—Arrangements for updating the contents of the bit-mapped memory
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2320/00—Control of display operating conditions
- G09G2320/10—Special adaptations of display systems for operation with variable images
- G09G2320/103—Detection of image changes, e.g. determination of an index representative of the image change
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2370/00—Aspects of data communication
- G09G2370/02—Networking aspects
- G09G2370/022—Centralised management of display operation, e.g. in a server instead of locally
Definitions
- the present invention relates to an update area detection apparatus, and more particularly to an update area detection apparatus, an update area detection method, and a program for detecting an update area from frames before and after drawing data such as a computer screen.
- drawing data of an application program executed on the server device is transferred to a terminal device on the client side through a network such as a LAN and displayed on the screen of the terminal device. For this reason, when the processing amount of drawing data increases or the number of terminal devices connected to the server device increases, the load on the server device for transferring drawing data to the terminal device increases, and the response time increases. Problems such as a drop and a significant limitation on the number of connected terminal devices occur.
- Patent Document 1 describes a method in which a screen is divided into blocks and an image is transmitted only for blocks including pixel differences. Further, for example, in the section “Background Art” of Patent Document 2, a method for transmitting an image included in a minimum rectangular area including all areas in which a difference value from the previous frame is detected is described.
- the following update region detection technique is described.
- the screen is divided into blocks, and for each block, a minimum rectangular area including all areas in which a difference value from the previous frame is detected is detected.
- both horizontally or vertically adjacent blocks have the rectangular area, and the rectangular areas are in contact with each other in the horizontal or vertical direction, the rectangular areas of the blocks adjacent in the horizontal or vertical direction are both The smallest rectangular area that is included is the update area.
- Patent Document 3 describes a method of dividing a right-angled polygonal area generated on the layout surface of an integrated circuit by a floor plan into a plurality of rectangular areas.
- Patent Document 4 describes a method of detecting a rectangle having the maximum area remaining in one area from a plurality of areas existing in an image.
- these Patent Documents 3 and 4 have no suggestion of application to processing for detecting an update region from the preceding and succeeding frames of drawing data.
- Patent Documents 1 and 2 described above when an update area is detected from the preceding and following frames of drawing data, the area is divided into a plurality of rectangular areas, and a rectangle having the maximum area remaining in one area is detected. There is no description that suggests.
- JP 2009-018414 A JP 2005-033763 A Japanese Patent Laid-Open No. 2000-020666 JP 2007-140593 A
- a minimum rectangular area including all areas in which a difference value from the previous frame is detected is detected for each block. If the blocks adjacent in the horizontal or vertical direction both have the rectangular area, and the rectangular areas are in contact in the horizontal or vertical direction, the rectangular areas of the blocks adjacent in the horizontal or vertical direction are both In the method in which the smallest rectangular area that is included is used as the update area, a large area in which the pixel value does not change may be incorporated into the update area. Specifically, when a pixel difference is generated in a shape like a square frame in the screen, the circumscribed rectangle of the frame is detected as an update region in the above-described method of Patent Document 2. As a result, an area in the frame where the pixel value is not changed is included in the update area.
- the object of the present invention is to increase the amount of data for indicating the coordinates of the update area and to change the pixel value when the update area is a set of blocks including pixels whose pixel values have changed.
- the size of the update area becomes uselessly large, the update area is detected for each block, and the update areas connected vertically or horizontally are integrated into those circumscribed rectangles.
- An object of the present invention is to provide an update region detection apparatus that solves the problem that a large region in which a pixel value does not change is incorporated into an update region.
- Update block detection that compares pixel values of pixels at the same position in the current frame and the previous frame of the input video, and detects blocks including pixels whose pixel values have changed among the blocks divided into blocks, as update blocks Means,
- Initial update area registration means for registering circumscribed rectangular areas of all update blocks detected by the update block detection means in the update area storage means as one update area;
- a removal area that detects a non-update area in which one or more blocks that do not include a pixel whose pixel value has changed continuously form a rectangle from among the one or more update areas stored in the update area storage means.
- Update area dividing means for dividing the update area having the non-update area detected by the removal area detection means into new update areas that are two or more rectangles not including the non-update area;
- An update area list rewriting means for deleting the update area before the division from the update area storage means, and additionally registering the update area after the division in the update area storage means; Control the repetition of the division process of the update area by the removal area detection means, the update area division means and the update area list rewriting means, and the removal area detection means no longer detects the new non-update area, Alternatively, the division that ends the repetition of the division process on the condition that the number of the update areas stored in the update area storage means after the additional registration exceeds a threshold value or which is established earlier Processing control means.
- an update area detection method includes: Update block detection that compares pixel values of pixels at the same position in the current frame and the previous frame of the input video, and detects blocks including pixels whose pixel values have changed among the blocks divided into blocks, as update blocks Process, An initial update area registration step of registering the circumscribed rectangular area of all the detected update blocks as an update area in the update area storage means; A removal area that detects a non-update area in which one or more blocks that do not include a pixel whose pixel value has changed continuously form a rectangle from among the one or more update areas stored in the update area storage means.
- a detection process An update area dividing step of dividing the update area having the detected no update area into new update areas that are two or more rectangles not including the no update area; An update area list rewriting step of deleting the update area before the division from the update area storage means, and additionally registering the update area after the division in the update area storage means;
- the repetition of the removal area detection process, the update area division process, and the update area list rewriting process is controlled, and a new non-update area is not detected in the removal area detection process, or after the additional registration
- a division process control step for terminating the repetition of the division process on condition that the number of the update areas stored in the update area storage means exceeds a threshold value or which is established earlier.
- the present invention has the above-described configuration, the following effects can be obtained.
- the amount of data for indicating the coordinates of the update area can be reduced.
- the size of the update area can be made smaller than when the circumscribed rectangle of all the blocks including the pixel whose pixel value has changed is used as the update area.
- the update area is detected for each block, and the update area connected vertically or horizontally is integrated into the circumscribed rectangle so that a large area in which the pixel value does not change is incorporated into the update area. Disappears.
- the embodiment of the present invention includes a signal input device 11, a data storage device 12, a data processing device 13 that operates under program control, and a region information output device 14.
- the signal input device 11 is a device for inputting a video signal.
- a color video signal on a display screen of a computer (not shown) is analog-captured or digitally captured and stored in the data storage device 12.
- the captured video signal for one screen is called a frame or screen data.
- the area information output device 14 is configured by a storage device such as a magnetic disk device, a communication device, or the like that stores or transmits the coordinate information of the update region output from the data processing device 13.
- the data storage device 12 includes a previous frame memory 121, a current frame memory 122, an update block memory 123, an update area list memory 124, and an update area number memory 125.
- the previous frame memory 121 holds one or more frames input in the past.
- a frame input in the past is hereinafter referred to as a previous frame.
- the current frame memory 122 holds one most recently input frame.
- the most recently input frame is called the current frame.
- the previous frame corresponds to, for example, a frame immediately before the current frame.
- the update block memory 123 divides the frame into fixed pixel sizes (for example, m ⁇ n pixels, where m and n are integers of 1 or more), and whether there is a difference between the previous frame and the current frame for each of the divided blocks. Holds data indicating that. The difference is that the pixel value of the pixel in the original frame is different from the pixel value of the pixel at the same position as the pixel in the previous frame.
- the update area list memory 124 holds a coordinate string indicating the update area.
- the update area is assumed to be composed of one or more rectangles, and the update area list memory 124 stores the coordinate values of the upper end, the lower end, the left end, and the right end indicating each rectangle.
- the XY coordinate system is defined on the frame, and the position on the frame is specified by the XY coordinate value.
- Each rectangle is specified by two XY coordinate values of its upper left end and lower right end.
- the update area number memory 125 holds the number of rectangles currently stored in the update area list memory 124.
- the data processing device 13 includes an update block detection unit 131, an initial update region registration unit 132, a removal region detection unit 133, an update region division unit 134, an update region list rewriting unit 135, a division processing control unit 136, An update area boundary adjusting unit 137 and an update area information output unit 138 are included.
- the update block detection means 131 reads the previous frame data from the previous frame memory 121 and the current frame data from the current frame memory 122, and compares the pixel values for each block. As a result of the comparison, data indicating whether or not there is an update is written in the update block memory 123, assuming that there is an update for a block including a pixel whose pixel value has changed and no update for a block that does not include a pixel.
- the initial update area registration unit 132 reads the update block information from the update block memory 123, and when one or more update blocks exist in the screen, obtains the smallest rectangle including all the update blocks, The coordinates are written in the update area list memory 124 as the update area. Further, 1 is written in the update area number memory 125 as the initial value of the update area number.
- the removal area detection means 133 detects a rectangle which is a block of non-update blocks existing in the update area for all the update areas registered in the update area list memory 124. In addition, a rectangle having the largest area is selected as a removal area among rectangles that are blocks of non-update blocks detected from all update areas. When a rectangle that is a block of non-update blocks is detected from any update area and a removal area is selected, the update area dividing unit 134 is notified of the coordinate information of the removal area and the update area including the removal area.
- the update area dividing unit 134 determines the update area rectangle including the removal area as a maximum excluding the removal area. Divide into 4 rectangles. Further, the update area list rewriting unit 135 is notified of information on the update area before the division and the update area after the division.
- the update area list rewriting means 135 deletes the update area before the division from the update area list memory 124, and adds the update area after the division to the update area list memory 124. Further, the number of update areas stored in the update area number memory 125 is updated based on the deletion and addition of the update area.
- the division processing control means 136 controls the repetition of the update area division processing by the removal area detection means 133, the update area division means 134, and the update area list rewriting means 135.
- the division processing control unit 136 determines whether the update region number 125 stored in the update region number 125 reaches a predetermined number or the removal region detection unit 133 does not detect the removal region, whichever comes first. Then, the repetition of the update area dividing process by the removal area detecting unit 133, the update area dividing unit 134, and the update area list rewriting unit 135 is ended.
- the update region boundary adjusting unit 137 stores the coordinate information of the update region from the update region list memory 124 after the update region dividing process by the removal region detection unit 133, the update region dividing unit 134, and the update region list rewriting unit 135 is completed. Extract one by one and finely adjust the coordinates of the top, bottom, left and right edges of each update area. Specifically, pixel values are read from the previous frame memory 121 and the current frame memory 122 for each of the upper, lower, left, and right boundary lines to check for differences, and the boundaries are reduced line by line until a difference pixel is found. This process is performed for each update area, and the coordinate values stored in the update area list memory 124 are updated.
- the update area information output means 138 outputs the coordinate information of the update area stored in the update area list memory 124 to the area information output device 14.
- the update block detection means 131 reads the previous frame data from the previous frame memory 121 and the current frame data from the current frame memory 122, and compares the pixel values for each block. As a result of the comparison, data indicating whether or not there is an update is written in the update block memory 123, assuming that there is an update for a block including a pixel whose pixel value has changed, and no update for a block that does not include a pixel (step S101).
- the initial update area registration unit 132 reads the update block information from the update block memory 123, determines whether one or more update blocks exist (step S102), and if there is no update block (NO in step S102). Then, the process shown in FIG. If it exists (YES in step S102), the process proceeds to step S103 and subsequent steps.
- step S103 the initial update area registration unit 132 obtains a minimum rectangle including all the update blocks, and uses the rectangle as the first update area, and writes the coordinates in the update area list memory 124 (step S103).
- step S103 1 is written in the update area count memory 125 as the initial value of the update area count.
- the removal area detection unit 133 initializes the scan position of the update area list memory 124 to the top of the list (step S104).
- the removal area detection means 133 detects a rectangle composed of a block of non-update blocks having the maximum area in the update area of the update area of the scan position (step S105). Further, in step S105, a rectangle having the largest area among rectangles made up of blocks of non-update blocks detected in the past including the updated area that has been scanned is stored as a removal area, and the removal area is further included. Store the update area.
- the removal area detection unit 133 moves the scan position of the update area by one (step S106), and determines whether the process has been completed for all the update areas (step S107). If all have not been completed (NO in step S107), the process returns to step S105 to process the remaining update area. If all have been completed (YES in step S107), the process proceeds to step S108 and subsequent steps.
- step S108 the division processing control unit 136 determines whether or not the removal region detection unit 133 has detected the removal region. If not detected (NO in step S108), the process proceeds to step S116 and subsequent steps. If it is detected (YES in step S108), the process proceeds to step S109 and subsequent steps.
- the update area dividing unit 134 determines the rectangle of the update area including the removal area based on the removal area detected and stored by the removal area detection unit 133 and the coordinate information of the update area including the removal area. Is divided into a maximum of four rectangles excluding the removal area as follows (steps S109 to S112).
- the update area dividing means 134 calculates the aspect ratio of the update area from the coordinate information of the update area including the removal area (step S109).
- the aspect ratio is determined (step S110), and if it is horizontally long (width> height) (YES in step S110), vertical line priority division is performed (step S111). If it is not horizontally long (width ⁇ height) (NO in step S110), horizontal line priority division is performed (step S112).
- the vertical line priority division in step S111 when there is a plurality of division line methods for the update area including the removal area, the division is performed using a division method that gives priority to the left and right divisions (vertical division lines). .
- the horizontal line priority division in step S112 when there are a plurality of division line methods for the update area including the removal area, the division is performed using a division method that gives priority to the vertical division (horizontal division line). .
- the update area list rewriting means 135 deletes the update area before the division from the update area list memory 124 (Step S113), and adds the update area after the division to the update area list memory 124 (Step S114).
- the number of update areas stored in the update area number memory 125 is updated in accordance with deletion and addition.
- the division processing control unit 136 compares the number of update areas (the number of registrations) stored in the update area number memory 125 with a predetermined threshold (step S115). If the registered number is less than the threshold value (NO in step S115), the process returns to step S104 and further attempts to divide the update area. If the number of registrations is equal to or greater than the threshold (YES in step S115), the process proceeds to step S116.
- the update region boundary adjusting unit 137 performs the update region boundary adjustment process as follows. First, the update area boundary adjusting unit 137 initializes the scan position of the update area list memory 124 to the head of the list (step S116).
- the update area boundary adjusting unit 137 reads the coordinates of the upper, lower, left, and right edges of the update area at the scan position from the update area list memory 124.
- pixel values are read from the previous frame memory 121 and the current frame memory 122 for each boundary line specified by the coordinates of the upper, lower, left, and right ends, the presence or absence of a difference is checked, and the boundary is reduced line by line until a difference pixel is found. (Step S117). Specifically, for example, the pixel value of the pixel on the upper end line specified by the upper end coordinate value stored in the update area list memory 124 is read from the previous frame memory 121 and the current frame memory 122 to determine whether there is a difference.
- the upper end coordinate value stored in the update area list memory 123 is reduced by one line toward the inner side of the update area, and the upper end specified by the new upper end coordinate value again. Repeat the process for the line as before. If there is a difference, the upper end line has been determined, so next, for example, the lower end line is processed in the same way as the upper end line to reduce the lower end boundary line by line toward the inside of the update area until a difference pixel is found. To go. When the lower end line is determined, the left end line is processed next, and finally the right end line is processed.
- the order of the upper end, the lower end, the left end, and the right end is an example, and this order may not be required.
- step S118 moves the scan position of the update area by one (step S118), and determines whether the processing has been completed for all the update areas (step S119). If all have not been completed (NO in step S119), the process returns to step S117 to process the remaining update areas. If all have been completed (YES in step S119), the process proceeds to step S120.
- the update area information output unit 138 outputs the coordinate information of the update area stored in the update area list memory 124 to the area information output device 14.
- the amount of additional data for indicating the coordinates of the update area can be reduced as compared with a case where a set of blocks including pixels whose pixel values have changed is used as the update area.
- the reason is that the removal area detection unit 133 can no longer detect a new non-updated area, or the number of update areas after additional registration by the update area list rewriting unit 135 exceeds the threshold, whichever comes first. This is because an increase in the number of update areas can be suppressed because the update area dividing process is terminated on the condition of the above.
- the size of the update area can be reduced as compared with the case where the circumscribed rectangle of all the blocks including the pixel whose pixel value has changed is used as the update area.
- the reason is that two or more rectangles that do not include the non-updated area are divided into update areas having non-updated areas in which one or more blocks that do not include pixels whose pixel values have changed continuously form a rectangle. This is to divide into update areas.
- an update area is detected for each block, and update areas connected vertically or horizontally are integrated into their circumscribed rectangles. It will not be incorporated. The reason is that the present embodiment does not consolidate a plurality of update areas into one rectangle, but conversely, one update area is replaced with two or more update areas that do not include a non-update area in the update area. This is to divide.
- the method of adjusting the boundary of each update area by the update area boundary adjusting unit 137 after obtaining the update area in block units has been described.
- setting the block size from 8 ⁇ 8 to 32 ⁇ 32 provides a good balance between processing speed and accuracy.
- the boundary of the update block obtained by the update area dividing unit 134 is substantially the boundary of the update area in units of pixels. Adjustment processing is not necessary. In this case, the update area boundary adjustment process may be omitted.
- the condition of the removal area detected by the removal area detection unit 133 is that the area of the detected non-update block block is the maximum, but other conditions may be used. For example, it may be adopted as a condition that the area is equal to or larger than the threshold, and the threshold may be lowered step by step each time the division processing control unit 136 repeats the processing. As a result, it is possible to find a plurality of removal areas only by scanning all the update areas once, so that the number of repetitions by the division processing control unit 136 can be reduced, and the processing becomes faster.
- condition of the removal area detected by the removal area detection unit 133 may be that the area of the detected non-update block block is the maximum and is equal to or larger than a predetermined lower limit area.
- the update area dividing unit 134 is operated so that the horizontally long update area has priority to the left and right divisions, and the update rectangle that has not been given priority to the upper and lower divisions. Either method may be taken at random.
- the division method described above when used, the update area as a result of the division does not become excessively flat, and thus there is an advantage that the compression rate when the two-dimensional image compression is applied to the update area is improved.
- the vertical line priority division is selected. May be.
- the method of detecting “no update block block” is not specifically described, but the designer can freely determine it according to conditions such as high speed and accuracy.
- an algorithm that searches a binary area having a maximum area among “a block of 0-value elements of squares (1 ⁇ 1, 2 ⁇ 2, 3 ⁇ 3,...)” At high speed is generally known. Utilizing this, the operation may be performed with a constraint that a block of square non-update blocks is detected.
- the first embodiment will be described more specifically.
- 3 to 6 show specific examples of the operation of the update region detection apparatus according to the present embodiment.
- the threshold value for the number of update areas is five.
- Z102 shows the state of the update block memory 123 after the update block detection means 131 is applied to the frame.
- a block including a difference pixel black in Z101
- is detected as an update block black in Z102.
- Z103 indicates the range of the initial update area registered by the initial update area registration means 132.
- the circumscribed rectangle of all the update blocks black in Z102 is registered as the initial update area (shaded area in Z103).
- the removal area detection unit 133 detects a block of non-update blocks from all the update areas (in this case, only one of the initial update areas) existing in the screen, and the removal area is determined as the maximum area. To detect.
- the lower left 4 ⁇ 3 block of the initial region is detected as a block of non-update blocks having the maximum area and is selected as a removal region.
- a 3 ⁇ 3 non-update block inside the frame-like update block is also detected, but is rejected because it is not the maximum area.
- Z105 indicates how the update area dividing means 134 divides the update area for the first time.
- the shaded area is the entire update area registered at the present time, and the bold line is the boundary line of the new update area generated by the division.
- the size of the update area to be divided is 9 blocks wide by 6 blocks long, which is horizontally long. Therefore, a dividing method is adopted in which the vertical dividing line is preferentially selected. As a result, the update area is divided into left and right so as not to include the removal area, and two new update areas indicated by Z105 are obtained.
- the removal area detection means 133 detects a block of non-update blocks from all the update areas (two in this case) existing in the screen, and detects the largest area as a removal area. As a result, a non-update block of 3 ⁇ 3 blocks inside the frame-like update block is detected as a removal area.
- Z107 indicates how the update area is divided by the update area dividing means 134 for the second time.
- the shaded area is the entire update area registered at the present time, and the bold line is the boundary line of the new update area generated by the division.
- the size of the update area to be divided is 5 horizontal blocks ⁇ 6 vertical blocks, and is not horizontally long. Therefore, a dividing method is adopted in which the horizontal dividing line is preferentially selected. As a result, the update area is divided into upper, middle, and lower so as not to include the removal area, and four new update areas indicated by Z107 are obtained.
- the black area indicates the difference pixel.
- Z117 shows a state after the boundary line of each update area is reduced by the update area boundary adjusting means 137.
- the update areas (thick frames) in Z116 the upper, lower, left, and right boundary lines are reduced to the position where the difference pixel exists, and the update area in Z117 is obtained.
- each of the 23 blocks painted black in Z102 is the update area.
- the area for a total of 54 blocks, which is shaded with Z103 becomes the update area.
- the size of a total of 25 blocks surrounded by a thick line in Z116 is the update area. Therefore, the size of the update area can be reduced. Note that the size of the update area can be further reduced by performing the update area boundary adjustment processing as in Z117.
- the circumscribed rectangle of the frame is An area that is detected as an update area and that has no change in pixel value inside the frame is included in the update area.
- an area where the pixel value inside the frame does not change is excluded from the update area.
- the first embodiment will be described more specifically.
- 3 to 5 and 7 to 10 show specific examples of the operation of the update region detection apparatus according to the present embodiment.
- the threshold value for the number of update areas is seven.
- the removal area detection unit 133 detects a block of non-update blocks from all the update areas (in this case, five) existing in the screen, and detects the largest area as a removal area. As a result, a 5 ⁇ 1 non-update block included in the update area at the top of the frame is detected as a removal area.
- Z109 indicates the state of the third update area division by the update area dividing means 134.
- the shaded area is the entire update area registered at the present time, and the bold line is the boundary line of the new update area generated by the division.
- the size of the update area to be divided is horizontal 5 blocks ⁇ vertical 2 blocks, which is horizontally long. Therefore, a dividing method is adopted in which the vertical dividing line is preferentially selected. In this case, since there is only one way of dividing the upper and lower parts, the upper and lower parts are divided vertically, and one new update area indicated by Z109 is obtained.
- the removal area detection unit 133 detects a block of non-update blocks from all the update areas (in this case, five) existing in the screen, and detects the largest area as a removal area. As a result, a 1 ⁇ 3 block non-update block included in the update area on the left side of the screen is detected as a removal area.
- Z111 indicates the state of the fourth update area division by the update area dividing means 134.
- the shaded area is the entire update area registered at the present time, and the bold line is the boundary line of the new update area generated by the division.
- the size of the update area to be divided is 4 horizontal blocks ⁇ 3 vertical blocks, which is horizontally long. Therefore, a dividing method is adopted in which the vertical dividing line is preferentially selected. In this case, since there is only one way of dividing the left and right, it is divided into left and right, and one new update area indicated by Z111 is obtained.
- the removal area detection unit 133 detects a block of non-update blocks from all the update areas (in this case, five) existing in the screen, and detects the largest area as a removal area. As a result, the 1 ⁇ 2 non-update block included in the update area on the left side of the screen is detected as the removal area.
- Z113 indicates the fifth update area division by the update area dividing means 134.
- the shaded area is the entire update area registered at the present time, and the bold line is the boundary line of the new update area generated by the division.
- the size of the update area to be divided is 3 horizontal blocks ⁇ 3 vertical blocks, and is not horizontally long. Therefore, a dividing method is adopted in which the horizontal dividing line is preferentially selected. As a result, the update area is divided into left and right so as not to include the removal area, and two new update areas indicated by Z113 are obtained.
- the shaded area indicates difference pixels.
- Z115 shows a state after the boundary line of each update region is reduced by the update region boundary adjusting means 137. With respect to all the update areas (thick frames) of Z114, the upper, lower, left, and right boundary lines are reduced to the places where the difference pixels exist, and the update area of Z115 is obtained.
- the update area division is repeated until no new update area as a division target is detected by the removal area detection unit 133, thereby covering update blocks without excess or deficiency, In addition, an update area that is not excessively divided can be obtained.
- the update area is obtained in units of blocks, and then the boundary of each update area is adjusted by the update area boundary adjusting unit 137.
- the update area enhancement adjusting unit 137 Processing may be omitted.
- the update unit information output unit 138 further includes an encoding unit that encodes an image in each update region. Instead of outputting the contents of the update region list memory 124, the update region information output unit 138 displays the code output from the encoding unit as region information. You may make it output to the output device 14. FIG.
- the present invention can be used for the purpose of reducing the amount of transmission in a screen transmission type thin client system. Further, it can be applied to other uses by utilizing the feature that the coordinates of the update area of the screen can be efficiently expressed with an appropriate granularity. For example, by accumulating the updated coordinates of the screen obtained by the present invention as a log, it can be used for purposes such as measuring the number of screen element rewrites and estimating the deterioration state.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Human Computer Interaction (AREA)
- General Engineering & Computer Science (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Digital Computer Display Output (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
Description
入力映像の現フレームと前フレームとにおける同じ位置の画素の画素値を比較し、フレームをブロック状に分割したブロックのうち、画素値が変化した画素を含むブロックを更新ブロックとして検出する更新ブロック検出手段と、
上記更新ブロック検出手段で検出された全ての更新ブロックの外接矩形領域を一つの更新領域として更新領域記憶手段に登録する初期更新領域登録手段と、
上記更新領域記憶手段に記憶されている1以上の上記更新領域の中から、画素値が変化した画素を含まないブロックが1以上連続して矩形を形成している無更新領域を検出する除去領域検出手段と、
上記除去領域検出手段で検出された上記無更新領域を有する上記更新領域を、上記無更新領域を含まない2以上の矩形である新たな上記更新領域に分割する更新領域分割手段と、
上記分割前の上記更新領域を上記更新領域記憶手段から削除し、分割後の上記更新領域を上記更新領域記憶手段に追加登録する更新領域リスト書換手段と、
上記除去領域検出手段と上記更新領域分割手段と上記更新領域リスト書換手段とによる上記更新領域の分割処理の繰り返しを制御し、上記除去領域検出手段で新たな上記無更新領域が検出されなくなるか、または、上記追加登録後の上記更新領域記憶手段に記憶されている上記更新領域の数が閾値を超えることになるか、何れか早く成立した方を条件として、上記分割処理の繰り返しを終了する分割処理制御手段と
を有する。
入力映像の現フレームと前フレームとにおける同じ位置の画素の画素値を比較し、フレームをブロック状に分割したブロックのうち、画素値が変化した画素を含むブロックを更新ブロックとして検出する更新ブロック検出工程と、
上記検出された全ての更新ブロックの外接矩形領域を一つの更新領域として更新領域記憶手段に登録する初期更新領域登録工程と、
上記更新領域記憶手段に記憶されている1以上の上記更新領域の中から、画素値が変化した画素を含まないブロックが1以上連続して矩形を形成している無更新領域を検出する除去領域検出工程と、
上記検出された上記無更新領域を有する上記更新領域を、上記無更新領域を含まない2以上の矩形である新たな上記更新領域に分割する更新領域分割工程と、
上記分割前の上記更新領域を上記更新領域記憶手段から削除し、分割後の上記更新領域を上記更新領域記憶手段に追加登録する更新領域リスト書換工程と、
上記除去領域検出工程と上記更新領域分割工程と上記更新領域リスト書換工程との繰り返しを制御し、上記除去領域検出工程で新たな上記無更新領域が検出されなくなるか、または、上記追加登録後の上記更新領域記憶手段に記憶されている上記更新領域の数が閾値を超えることになるか、何れか早く成立した方を条件として、上記分割処理の繰り返しを終了する分割処理制御工程と
を有する。
[第1の実施形態]
図1を参照すると、本発明の実施の形態は、信号入力装置11と、データ記憶装置12と、プログラム制御により動作するデータ処理装置13と、領域情報出力装置14とから構成されている。
信号入力装置11は、映像信号を入力するための装置であり、例えば図示しないコンピュータのディスプレイ画面のカラー映像信号をアナログキャプチャあるいはディジタルキャプチャし、データ記憶装置12に記憶する。キャプチャされた1画面分の映像信号は、フレームあるいは画面データと呼ぶ。
次に、図2のフローチャートを参照して、本発明の実施の形態の全体の動作について詳細に説明する。
本実施形態では、第1の実施形態をより具体化して説明する。図3乃至図6に、本実施形態による更新領域検出装置の動作の具体例を示す。本実施形態では、更新領域数の閾値は5である。
本実施形態では、第1の実施形態をより具体化して説明する。図3~図5、および図7~図10に、本実施形態による更新領域検出装置の動作の具体例を示す。本実施形態では、更新領域数の閾値は7である。
上記の各実施形態では、ブロック単位で更新領域を求めてから、更新領域境界調整手段137によって、個々の更新領域の境界を調整したが、ブロックのサイズにかかわらず、更新領域強化調整手段137による処理を省略しても良い。
12…データ記憶装置
13…データ処理装置
14…領域情報出力装置
121…前フレームメモリ
122…現フレームメモリ
123…更新ブロックメモリ
124…更新領域リストメモリ
125…更新領域数メモリ
131…更新ブロック検出手段
132…初期更新領域登録手段
133…除去領域検出手段
134…更新領域分割手段
135…更新領域リスト書換手段
136…分割処理制御手段
137…更新領域境界調整手段
138…更新領域情報出力手段
Claims (14)
- 入力映像の現フレームと前フレームとにおける同じ位置の画素の画素値を比較し、フレームをブロック状に分割したブロックのうち、画素値が変化した画素を含むブロックを更新ブロックとして検出する更新ブロック検出手段と、
前記更新ブロック検出手段で検出された全ての更新ブロックの外接矩形領域を一つの更新領域として更新領域記憶手段に登録する初期更新領域登録手段と、
前記更新領域記憶手段に記憶されている1以上の前記更新領域の中から、画素値が変化した画素を含まないブロックが1以上連続して矩形を形成している無更新領域を検出する除去領域検出手段と、
前記除去領域検出手段で検出された前記無更新領域を有する前記更新領域を、前記無更新領域を含まない2以上の矩形である新たな前記更新領域に分割する更新領域分割手段と、
前記分割前の前記更新領域を前記更新領域記憶手段から削除し、分割後の前記更新領域を前記更新領域記憶手段に追加登録する更新領域リスト書換手段と、
前記除去領域検出手段と前記更新領域分割手段と前記更新領域リスト書換手段とによる前記更新領域の分割処理の繰り返しを制御し、前記除去領域検出手段で新たな前記無更新領域が検出されなくなるか、または、前記追加登録後の前記更新領域記憶手段に記憶されている前記更新領域の数が閾値を超えることになるか、何れか早く成立した方を条件として、前記分割処理の繰り返しを終了する分割処理制御手段と
を有する更新領域検出装置。 - 前記分割処理の繰り返しの終了後、前記更新領域記憶手段に記憶されている全ての更新領域に対して、当該更新領域の上下左右の各境界ラインについて、前フレームと現フレームとの間で画素値を比較し、ライン内に差分が見つかるまで境界を1ラインずつ当該更新領域の内部方向に縮小する更新領域境界調整手段を更に備える
請求項1に記載の更新領域検出装置。 - 前記更新領域分割手段は、分割対象となる前記更新領域の幅が高さよりも大きければ左右への分割を優先し、幅が高さよりも小さければ上下への分割を優先するよう、前記更新領域の分割を行う
請求項1または2に記載の更新領域検出装置。 - 前記除去領域検出手段は、面積が最大の前記無更新領域を検出する
請求項1乃至3の何れかに記載の更新領域検出装置。 - 前記除去領域検出手段は、面積が下限値を超える前記無更新領域を検出する
請求項1乃至4の何れかに記載の更新領域検出装置。 - 前記除去領域検出手段は、面積が下限値を超える前記無更新領域を検出し、前記下限値は、前記分割処理の繰り返しの進行に応じて、より小さな値に変更する
請求項1乃至5の何れかに記載の更新領域検出装置。 - 前記除去領域検出手段は、前記更新領域毎に、当該更新領域に含まれるN×Nブロック(Nは1以上の整数)の前記無更新領域のうちNが最大となるものを検出し、最終的に、全更新領域でNが最大となる前記無更新領域を検出する
請求項1乃至6の何れかに記載の更新領域検出装置。 - 前記除去領域検出手段は、前記更新領域毎に、当該更新領域に含まれるN×Nブロック(Nは1以上の整数)の前記無更新領域のうちNが最大となるものを検出し、該検出した前記無更新領域の上下左右を1ブロック幅ずつ拡大できるかどうかを試行してM×Lブロック(M≧N,L≧N)の前記無更新領域を検出し、最終的に、全更新領域でM×Lが最大となる前記無更新領域を検出する
請求項1乃至7の何れかに記載の更新領域検出装置。 - 前記分割処理の繰り返しの終了後、前記更新領域記憶手段に記憶されている前記更新領域に関する情報を出力する更新領域情報出力手段を更に有する
請求項1に記載の更新領域検出装置。 - 前記更新領域境界調整手段による処理の終了後、前記更新領域記憶手段に記憶されている前記更新領域に関する情報を出力する更新領域情報出力手段を更に有する
請求項2に記載の更新領域検出装置。 - 前記分割処理の繰り返しの終了後、前記現フレームの画像のうち、前記更新領域記憶手段に記憶されている前記更新領域の画像を符号化して出力する符号化手段を更に有する
請求項1に記載の更新領域検出装置。 - 前記更新領域境界調整手段による処理の終了後、前記現フレームの画像のうち、前記更新領域記憶手段に記憶されている前記更新領域の画像を符号化して出力する符号化手段を更に有する
請求項2に記載の更新領域検出装置。 - 入力映像の現フレームと前フレームとにおける同じ位置の画素の画素値を比較し、フレームをブロック状に分割したブロックのうち、画素値が変化した画素を含むブロックを更新ブロックとして検出する更新ブロック検出工程と、
前記検出された全ての更新ブロックの外接矩形領域を一つの更新領域として更新領域記憶手段に登録する初期更新領域登録工程と、
前記更新領域記憶手段に記憶されている1以上の前記更新領域の中から、画素値が変化した画素を含まないブロックが1以上連続して矩形を形成している無更新領域を検出する除去領域検出工程と、
前記検出された前記無更新領域を有する前記更新領域を、前記無更新領域を含まない2以上の矩形である新たな前記更新領域に分割する更新領域分割工程と、
前記分割前の前記更新領域を前記更新領域記憶手段から削除し、分割後の前記更新領域を前記更新領域記憶手段に追加登録する更新領域リスト書換工程と、
前記除去領域検出工程と前記更新領域分割工程と前記更新領域リスト書換工程との繰り返しを制御し、前記除去領域検出工程で新たな前記無更新領域が検出されなくなるか、または、前記追加登録後の前記更新領域記憶手段に記憶されている前記更新領域の数が閾値を超えることになるか、何れか早く成立した方を条件として、前記分割処理の繰り返しを終了する分割処理制御工程と
を有する更新領域検出方法。 - 記憶装置を有するコンピュータを、
入力映像の現フレームと前フレームとにおける同じ位置の画素の画素値を比較し、フレームをブロック状に分割したブロックのうち、画素値が変化した画素を含むブロックを更新ブロックとして検出する更新ブロック検出手段と、
前記更新ブロック検出手段で検出された全ての更新ブロックの外接矩形領域を一つの更新領域として前記記憶装置に登録する初期更新領域登録手段と、
前記記憶装置に記憶されている1以上の前記更新領域の中から、画素値が変化した画素を含まないブロックが1以上連続して矩形を形成している無更新領域を検出する除去領域検出手段と、
前記除去領域検出手段で検出された前記無更新領域を有する前記更新領域を、前記無更新領域を含まない2以上の矩形である新たな前記更新領域に分割する更新領域分割手段と、
前記分割前の前記更新領域を前記記憶装置から削除し、分割後の前記更新領域を前記記憶装置に追加登録する更新領域リスト書換手段と、
前記除去領域検出手段と前記更新領域分割手段と前記更新領域リスト書換手段とによる前記更新領域の分割処理の繰り返しを制御し、前記除去領域検出手段で新たな前記無更新領域が検出されなくなるか、または、前記追加登録後の前記記憶装置に記憶されている前記更新領域の数が閾値を超えることになるか、何れか早く成立した方を条件として、前記分割処理の繰り返しを終了する分割処理制御手段と
して機能させるためのプログラム。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/416,363 US9633414B2 (en) | 2012-07-25 | 2013-04-30 | Update region detection device |
JP2014526731A JPWO2014017003A1 (ja) | 2012-07-25 | 2013-04-30 | 更新領域検出装置 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2012164739 | 2012-07-25 | ||
JP2012-164739 | 2012-07-25 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2014017003A1 true WO2014017003A1 (ja) | 2014-01-30 |
Family
ID=49996831
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2013/002893 WO2014017003A1 (ja) | 2012-07-25 | 2013-04-30 | 更新領域検出装置 |
Country Status (3)
Country | Link |
---|---|
US (1) | US9633414B2 (ja) |
JP (1) | JPWO2014017003A1 (ja) |
WO (1) | WO2014017003A1 (ja) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110134409A (zh) * | 2019-04-12 | 2019-08-16 | 宁波麦度智联科技股份有限公司 | 一种程序的烧录优化方法 |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6248671B2 (ja) * | 2014-02-10 | 2017-12-20 | 富士通株式会社 | 情報処理装置、方法、プログラム、および情報処理システム |
JP6444835B2 (ja) * | 2015-09-07 | 2018-12-26 | 株式会社東芝 | 画像処理装置、画像処理プログラムおよび画像処理システム |
US10902656B2 (en) * | 2016-02-29 | 2021-01-26 | Fujifilm North America Corporation | System and method for generating a digital image collage |
KR102612768B1 (ko) * | 2020-01-08 | 2023-12-13 | 스냅 인코포레이티드 | 디스플레이 디바이스 상에 디스플레이되는 이미지를 업데이트하는 시스템들 및 방법들 |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2011040808A (ja) * | 2009-08-06 | 2011-02-24 | Nec Personal Products Co Ltd | 情報処理装置、制御方法及びプログラム |
JP2012074999A (ja) * | 2010-09-29 | 2012-04-12 | Nec Corp | 画像データ伝送システム、サーバ装置、クライアント端末、画像データ伝送方法、及び制御プログラム |
JP2012120066A (ja) * | 2010-12-03 | 2012-06-21 | Fujitsu Ltd | 情報処理装置、情報処理方法および情報処理プログラム |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4161418B2 (ja) | 1998-06-30 | 2008-10-08 | 富士通株式会社 | 直角多角形領域分割方法及び直角多角形領域分割プログラムを記録した記録媒体並びに直角多角形領域分割装置 |
JP2005033763A (ja) | 2003-06-19 | 2005-02-03 | Matsushita Electric Ind Co Ltd | 送信装置、画像処理システム、画像処理方法、プログラム、及び記録媒体 |
JP4594848B2 (ja) | 2005-11-14 | 2010-12-08 | 日本電信電話株式会社 | 内接矩形検出装置および内接矩形検出プログラム |
JP2008131569A (ja) * | 2006-11-24 | 2008-06-05 | Sony Corp | 画像情報伝送システム、画像情報送信装置、画像情報受信装置、画像情報伝送方法、画像情報送信方法、画像情報受信方法 |
JP2009017414A (ja) | 2007-07-08 | 2009-01-22 | System Integrate Co Ltd | 情報漏洩監視システム |
US8625910B2 (en) * | 2011-02-25 | 2014-01-07 | Adobe Systems Incorporated | Compression of image data |
US8565540B2 (en) * | 2011-03-08 | 2013-10-22 | Neal Solomon | Digital image and video compression and decompression methods |
JP2013126185A (ja) * | 2011-12-15 | 2013-06-24 | Fujitsu Ltd | 情報処理装置、画像送信方法及び画像送信プログラム |
JP6046948B2 (ja) * | 2012-08-22 | 2016-12-21 | キヤノン株式会社 | 物体検知装置及びその制御方法、プログラム、並びに記憶媒体 |
-
2013
- 2013-04-30 JP JP2014526731A patent/JPWO2014017003A1/ja active Pending
- 2013-04-30 WO PCT/JP2013/002893 patent/WO2014017003A1/ja active Application Filing
- 2013-04-30 US US14/416,363 patent/US9633414B2/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2011040808A (ja) * | 2009-08-06 | 2011-02-24 | Nec Personal Products Co Ltd | 情報処理装置、制御方法及びプログラム |
JP2012074999A (ja) * | 2010-09-29 | 2012-04-12 | Nec Corp | 画像データ伝送システム、サーバ装置、クライアント端末、画像データ伝送方法、及び制御プログラム |
JP2012120066A (ja) * | 2010-12-03 | 2012-06-21 | Fujitsu Ltd | 情報処理装置、情報処理方法および情報処理プログラム |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110134409A (zh) * | 2019-04-12 | 2019-08-16 | 宁波麦度智联科技股份有限公司 | 一种程序的烧录优化方法 |
Also Published As
Publication number | Publication date |
---|---|
US9633414B2 (en) | 2017-04-25 |
US20150206281A1 (en) | 2015-07-23 |
JPWO2014017003A1 (ja) | 2016-07-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2014017003A1 (ja) | 更新領域検出装置 | |
US9959670B2 (en) | Method for rendering terrain | |
CN106412720B (zh) | 视频水印去除的方法和装置 | |
US20170208345A1 (en) | Method and apparatus for false contour detection and removal for video coding | |
CN105631913B (zh) | 用于图像的基于云的内容认知填充 | |
JP2016505186A (ja) | エッジ保存・ノイズ抑制機能を有するイメージプロセッサ | |
US9241091B2 (en) | Image processing device, image processing method, and computer program | |
KR20170057362A (ko) | 훈련 품질 척도를 이용한 비감독 이미지 세그먼트 분할을 위한 방법 및 시스템 | |
US20160232420A1 (en) | Method and apparatus for processing signal data | |
CN109272526B (zh) | 图像处理方法、***及电子设备 | |
EP3343446A1 (en) | Method and apparatus for encoding and decoding lists of pixels | |
CN116756836B (zh) | 一种隧道超欠挖体积计算方法、电子设备及存储介质 | |
Patel et al. | Accelerated seam carving for image retargeting | |
US9307245B2 (en) | Moving image encoding device | |
US20100014716A1 (en) | Method for determining ground line | |
CN115880362B (zh) | 码区定位方法、装置、计算机设备及计算机可读存储介质 | |
US8805062B2 (en) | Image processing apparatus, computer readable medium for processing image and method for processing image | |
CN109685861B (zh) | 图片压缩方法、装置、设备及计算机可读存储介质 | |
US20220284633A1 (en) | Method and apparatus for constructing a 3d geometry | |
KR101667306B1 (ko) | Topological Derivative 기반 저 복잡도 영상 세그먼테이션 방법 및 시스템 | |
CN110516680B (zh) | 图像处理方法及装置 | |
CN113362438A (zh) | 全景渲染的方法、装置、电子设备、介质及程序 | |
CN113191210A (zh) | 一种图像处理方法、装置及设备 | |
WO2020140244A1 (zh) | 一种视频图像处理方法、设备及存储介质 | |
EP3565253A1 (en) | A method and an apparatus for reducing an amount of data representative of a multi-view plus depth content |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 13822761 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2014526731 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 14416363 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 13822761 Country of ref document: EP Kind code of ref document: A1 |