CN113727009A - Tracking display method, device and storage medium - Google Patents
Tracking display method, device and storage medium Download PDFInfo
- Publication number
- CN113727009A CN113727009A CN202010448183.9A CN202010448183A CN113727009A CN 113727009 A CN113727009 A CN 113727009A CN 202010448183 A CN202010448183 A CN 202010448183A CN 113727009 A CN113727009 A CN 113727009A
- Authority
- CN
- China
- Prior art keywords
- target
- frame
- size
- preview
- period
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/64—Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/63—Control of cameras or camera modules by using electronic viewfinders
- H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Studio Devices (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application provides a tracking display method, a tracking display device and a storage medium, relates to the technical field of video image processing, and can realize the target tracking function of an image in a software mode without hardware support, thereby reducing the cost. The tracking display method comprises the following steps: periodically executing a target preview box generation process, wherein the target preview box generation process comprises the following steps: acquiring an original image; performing target detection; carrying out region merging; judging whether the combined target frame falls into a complementary set area or not; when the merged target frame is determined to fall into the complementary set area, the target preview frame generated in the previous period is used as the target preview frame generated in the current period; when the merged target frame is determined not to fall into the complementary set area, the merged target frame or the merged target frame is expanded outwards by a first preset size to serve as a target preview frame generated in the current period; and according to the target preview frame generated in the current period, performing cutting or scaling processing on the original image.
Description
Technical Field
The present application relates to the field of video image processing technologies, and in particular, to a tracking display method, apparatus, and storage medium.
Background
In technologies such as video monitoring, in order to achieve the purpose of automatic monitoring, a function of locking a picture on a target object and tracking the target object by a physical zooming manner appears to achieve target tracking of an image, however, the current target tracking function needs support of related hardware, and the cost is high.
Disclosure of Invention
The technical scheme of the application provides a tracking display method, a device and a storage medium, which can realize the target tracking function of an image in a software mode without hardware support, thereby reducing the cost.
In a first aspect, the present application provides a tracking display method, where the method is used to only show image content including an object of interest in an image, and the method includes:
periodically executing a target preview box generation process, wherein in the current period, the target preview box generation process comprises the following steps:
acquiring an original image;
executing target detection to obtain a target frame of an interested target in the image;
carrying out region merging on the target frames of all the interested targets so that the merged target frames cover all the interested targets;
judging whether the merged target frame falls into a complementary set area of a first preset target frame and a second preset target frame, wherein the first preset target frame and the second preset target frame are determined based on a target preview frame generated in a previous period, the size of the first preset target frame is larger than that of the target preview frame generated in the previous period, and the size of the target preview frame generated in the previous period is larger than that of the second preset target frame;
when the merged target frame is determined to fall into a complementary set area, the target preview frame generated in the previous period is used as the target preview frame generated in the current period;
when the merged target frame is determined not to fall into the complementary set area, the merged target frame or the merged target frame is expanded outwards by a first preset size to serve as a target preview frame generated in the current period; and
and according to the target preview frame generated in the current period, performing cutting or scaling processing on the original image to generate a corresponding new image so that the new image only shows the image content of the target of interest.
Optionally, the first preset target frame is formed by outwardly expanding the merged target frame in the previous period by a second preset size, and the second preset target frame is formed by inwardly reducing the merged target frame in the previous period by a third preset size.
Optionally, the width-to-length ratio of the first preset size is the same as the width-to-length ratio of the original image size, and the width-to-length ratio of the second preset size is the same as the width-to-length ratio of the original image size.
Optionally, wherein the second predetermined size is the same as the first predetermined size.
Optionally, the acquiring the original image is: acquiring an original image of a last frame in a current period;
the target preview frame generated in the previous period is the target preview frame of the last frame generated in the previous period;
the target preview frame generated in the current period is a target preview frame of the last frame generated in the current period;
the target preview frame of the last frame generated in the last period has a first size and a first position, the target preview frame of the last frame generated in the current period has a second size and a second position, and the target preview frame generation process further includes:
determining the size and the position of a target preview frame of each other frame in the current period, so that in each frame of original images from the last frame of the previous period to the last frame of the current period, the target preview frame is gradually changed from the first size and the first position to the second size and the second position, and in the current period, the change step length of the position of the target preview frame between any two adjacent frames is consistent;
and according to the determined target preview frame of each other frame in the current period, cutting or scaling the original image of each other frame in the current period to generate a new image corresponding to the original image of each frame, so that the new image only shows the image content of the target of interest.
Optionally, in the determining the size and the position of the target preview frame of each other frame in the current period, so that in the process of gradually changing the target preview frame from the first size and the first position to the second size and the second position in each frame of the original image from the last frame of the previous period to the last frame of the current period, in the current period, the change step size of the target preview frame between any two adjacent frames is consistent.
Optionally, the difference between the first size and the second size in each period is the size difference of the preview frame corresponding to the period, and in different periods, the size change step of the target preview frame is positively correlated with the size difference of the corresponding preview frame;
and/or the difference between the first position and the second position in each period is the position difference of the preview frame corresponding to the period, and in different periods, the position change step length of the target preview frame is positively correlated with the position difference of the corresponding preview frame.
In a second aspect, the present technical solution also provides a tracking display apparatus, where the apparatus is configured to periodically execute a target preview box generation process, and the apparatus includes:
the image acquisition module is used for acquiring an original image;
the target detection module is used for executing target detection and acquiring a target frame of an interested target in the image;
the merging module is used for performing region merging on the target frames of all the interested targets so that the merged target frames cover all the interested targets;
a judging module, configured to judge whether the merged target frame falls into a complementary set region of a first preset target frame and a second preset target frame, where the first preset target frame and the second preset target frame are determined based on a target preview frame generated in a previous period, a size of the first preset target frame is larger than a size of the target preview frame generated in the previous period, and a size of the target preview frame generated in the previous period is larger than a size of the second preset target frame;
a target preview frame generating module, configured to, when it is determined that the merged target frame falls into the complementary set region, use the target preview frame generated in the previous period as the target preview frame generated in the current period, and when it is determined that the merged target frame does not fall into the complementary set region, use the merged target frame or the merged target frame with an outwardly expanded first preset size as the target preview frame generated in the current period; and
and the image generation module is used for cutting or scaling the original image according to the target preview frame generated in the current period to generate a corresponding new image so that the new image only shows the image content of the target of interest.
In a third aspect, the present technical solution further provides a tracking display device, including:
a processor and a memory for storing at least one instruction which is loaded and executed by the processor to implement the method described above.
In a fourth aspect, the present invention also provides a computer-readable storage medium, in which a computer program is stored, and when the computer program runs on a computer, the computer is caused to execute the above method.
According to the tracking display method, the tracking display device and the storage medium in the embodiment of the application, all targets are covered by the target frame in the target tracking process through regional combination, the combined target frame is judged whether to be located in the shaking range or not, the shaking range is used as the basis for generating the target preview frame, the shaking of the tracking frame is reduced, the image data is finally processed according to the target preview frame, and the image is cut and zoomed, namely, the target tracking function of the image is realized in a software mode, hardware support is not needed, the cost is reduced, and the compatibility of the target tracking function is improved.
Drawings
FIG. 1 is a flowchart of a tracking display method according to an embodiment of the present application;
FIG. 2 is a diagram illustrating a jitter range according to an embodiment of the present application;
FIG. 3 is a flow chart of another tracking display method in the embodiment of the present application;
FIG. 4a is a timing diagram illustrating another tracking display method according to an embodiment of the present application;
FIG. 4b is a flowchart illustrating a process of generating a target preview box in a period corresponding to FIG. 4 a;
FIG. 4c is a schematic diagram of an original image and a target preview box corresponding to f6 frames in an embodiment of the present application;
FIG. 4d is a schematic diagram of an original image and a target preview box corresponding to f12 frames in an embodiment of the present application;
FIG. 4e is a schematic diagram of original images and target preview boxes corresponding to frames f6 to f12 in the embodiment of the present application;
fig. 5 is a block diagram of a tracking display device according to an embodiment of the present application.
Detailed Description
The terminology used in the description of the embodiments section of the present application is for the purpose of describing particular embodiments of the present application only and is not intended to be limiting of the present application.
As shown in fig. 1, fig. 1 is a flowchart of a tracking display method in an embodiment of the present application, and the embodiment of the present application provides a tracking display method for displaying only image contents including an object of interest in an image, where the method includes:
the target preview box generation process is performed periodically.
The target preview box generation process is performed, for example, every 100 ms.
In the current period, the target preview box generation process comprises the following steps:
102, executing target detection, and acquiring a target frame of an interested target in an image;
for example, based on a target tracking algorithm (HMS) or other algorithms, an original image to be displayed is processed to obtain a corresponding area of a target to be tracked in a displayed image, and the corresponding area of each target in the image is referred to as a target frame.
103, carrying out region combination on the target frames of all the interested targets so that the combined target frames cover all the interested targets to ensure that all the targets are covered in the target tracking process;
105, taking the target preview frame generated in the previous period as the target preview frame generated in the current period;
specifically, in the process of target tracking, the target frame has a certain jitter, for example, a person as a target is likely to swing left and right in a small amplitude during walking, and if the left and right swings during walking of the person are also tracked, frequent swinging of a final picture is caused, so that a poor experience is caused to a process of observing the picture by a user, and the final presentation of target tracking is not facilitated. As shown in fig. 2, fig. 2 is a schematic diagram of a jitter range in the embodiment of the present application, in fig. 2, T' is a target preview frame generated in a previous cycle, R1 is a first preset target preview frame as a jitter upper limit range, R2 is a second preset target preview frame as a jitter lower limit range, and T is a target frame combined in a current cycle. In step 1014, it is determined whether the merged target frame T falls into the complementary set area, if the merged target frame T falls into the complementary set area, it indicates that the target frame has a small change and belongs to a shaking range, and the target preview frame generated in the previous period is still used to reduce shaking of the finally presented image without changing the screen.
And step 107, according to the target preview frame generated in the current period, performing cutting or scaling processing on the original image to generate a corresponding new image, so that the new image only shows the image content of the target of interest.
Specifically, after steps 105 and 106, a target preview frame generated in the current period is determined, in step 107, the raw image data in YUV encoding format, for example, is processed according to the determined target preview frame, so that the portion of the whole image outside the target preview frame is clipped, and at the same time, the portion of the target preview frame is scaled to realize the highlight display, and the display screen is gradually changed from the target preview frame generated in the previous period to the target preview frame generated in the current period, and the processed image data is encoded to realize the display of the preview tracking screen.
According to the tracking display method in the embodiment of the application, all targets are covered by the image in the target tracking process by carrying out region merging on the target frame, the jitter of the tracking image is reduced by judging whether the merged target frame is located in the jitter range or not and taking the jitter as the basis for generating the target preview frame, the image data is processed according to the target preview frame, and the image is cut and zoomed, namely, the target tracking function of the image is realized in a software mode without hardware support, so that the cost is reduced, and the compatibility of the target tracking function is improved.
Optionally, as shown in fig. 3, fig. 3 is a flowchart of another tracking display method in this embodiment of the application, and in a process of executing generation of a target preview frame each time, the process of executing target detection and acquiring a target frame of an object of interest in an image specifically includes:
and step 1022, taking the target frame which appears in the pre-target frame set for n times continuously as the target frame acquired this time.
For example, if the target frame acquired in the current period has been used as a pre-target frame in 3 consecutive periods, the pre-target frame is used as a target frame in the target frame set acquired in the current period, that is, the target frames to be tracked acquired through the algorithm are screened, and the target frames that continuously appear in the pre-target frame are used as target frames that need to be tracked in the current period, so as to achieve more reliable target tracking and improve the accuracy of target tracking.
Alternatively, as shown in fig. 2, the first preset target frame R1 is formed by outwardly expanding the target frame T 'merged in the previous period by a second preset size, and the second preset target frame R2 is formed by inwardly contracting the target frame T' merged in the previous period by a third preset size, where the second preset size and the third preset size may be set as needed.
Alternatively, as shown in fig. 2, the width-to-length ratio of the first preset size is the same as the width-to-length ratio of the original image size, and the width-to-length ratio of the second preset size is the same as the width-to-length ratio of the original image size.
Specifically, for example, the center of the first preset target frame R1 is located at the same position as the center of the target frame T ' merged in the previous period, the difference between the width of the first preset target frame R1 and the width of the target frame T ' merged in the previous period is equal to q times the width of the original image, and the difference between the height of the first preset target frame R1 and the height of the target frame T ' merged in the previous period is equal to q times the height of the original image; the center of the second preset target frame R2 is located at the same position as the center of the target frame T ' merged in the previous period, the difference between the width of the second preset target frame R2 and the width of the target frame T ' merged in the previous period is equal to p times the width of the original image, and the difference between the height of the second preset target frame R2 and the height of the target frame T ' merged in the previous period is equal to p times the height of the original image. For example, if the resolution of the original image is 1920 × 1080, the width of the first preset target frame R1 is increased by 192 pixels based on the width of the target frame T 'merged in the previous period, and the height of the first preset target frame R1 is increased by 108 pixels based on the height of the target frame T' merged in the previous period, that is, q is 10; similarly, p may be 10 or other setting.
Optionally, the second preset size is the same as the first preset size, that is, in step 106, the first preset target frame R1 is used as the target preview frame generated in the current period.
Optionally, in the above embodiment, the acquiring the original image is: acquiring an original image of a last frame in a current period; the target preview frame generated in the previous period is the target preview frame of the last frame generated in the previous period; the target preview frame generated in the current period is the target preview frame of the last frame generated in the current period; as shown in fig. 4a and 4b, fig. 4a is a timing diagram of another tracking display method in the embodiment of the present application, and fig. 4b is a flowchart of a process of generating a target preview frame in a period corresponding to fig. 4a, for example, as shown in fig. 4a, fig. 4a illustrates 15 frame times from f1 to f15, in an image display process, each frame time is used for displaying a picture, for example, every 6 frames is used as a period T, and the target preview frame generation process is performed once every period T, and includes:
specifically, after obtaining a target preview frame of an original image of f12 frames and performing cropping and scaling processing to obtain a new image corresponding to the frame, determining target preview frames of other frames in the current period, that is, determining target preview frames of frames f7, f8, f9, f10 and f11, where the determination method of the target preview frames corresponding to these frames is different from the determination method of the target preview frame of the f12 frame, but is determined according to the target not determined according to the target as in step 202, but according to the target preview frame of the last frame (that is, f6 frame) generated in the previous period and the target preview frame of the last frame (that is, f12 frame) generated in the current period, for example, as shown in fig. 4c and 4d, fig. 4c is a schematic diagram of the original image and the target preview frame corresponding to f6 frame in the embodiment of the present application, and fig. 4d is a schematic diagram of fig. 4dFor the schematic diagram of the original image and the target preview frame corresponding to f12 frames in the embodiment of the present application, it can be seen that the size and the position of the target preview frame are changed from f6 frames to f12 frames. The position of the target preview frame may be represented by a reference coordinate, and the reference coordinate may be defined as needed, for example, the geometric center coordinate of the target preview frame may be defined as its reference coordinate, and then only the geometric center coordinate of the target preview frame, that is, the reference coordinate of the target preview frame, needs to be determined, or the upper left corner coordinate of the target preview frame may be defined as its reference coordinate, and then only the upper left corner coordinate of the target preview frame, that is, the reference coordinate of the target preview frame, needs to be determined, and a change in the reference coordinate represents a change in the position of the target preview frame. For example, f6 frame corresponds to target preview box S1Has a reference coordinate of (x)1，y1)，(x1，y1) For the first position, f12 frame corresponds to the target preview frame S2Has a reference coordinate of (x)2，y2)，(x2，y2) Is the second position. The size of the target preview box can be represented by its length and width, for example, f6 frame corresponding to the target preview box S1Has a width of W1Height of H1F12 frame corresponding to the target preview frame S2Has a width of W2Height of H2，W1And H1Is a first size, W2And H2Is the second size. As shown in fig. 4e, fig. 4e is a schematic diagram of each original image and target preview frame corresponding to frames f6 to f12 in the embodiment of the present application, in each frame original image from frames f6 to f12, the target preview frame gradually changes from the first size and the first position to the second size and the second position, the position change steps of the target preview frame are consistent, that is, the change step of the abscissa in the reference coordinate of the target preview frame between any two adjacent frames is Sx, and the change step of the ordinate between any two adjacent frames is Sy, wherein,
in the continuous N frames, the reference coordinate is changed for N times, wherein the x coordinate is changed by Sx every time, the y coordinate is changed by Sy every time, the frame number corresponding to each period is the same, so N is a fixed value, and in the frames from f6 to f12, the position change step length of the target preview frame between any two adjacent frames is consistent, so that the change process of the target preview frame is more gradual and stable in the target preview frame determination process in the same period.
Step 209, according to the determined target preview frame of each other frame in the current period, clipping or scaling the original image of each other frame in the current period to generate a new image corresponding to the original image of each frame, so that the new image only shows the image content of the target of interest.
Specifically, in step 207, the original image of the f12 frame is cropped and scaled according to the target preview frame to obtain a corresponding new image, and in step 209, the original images of the frames f7, f8, f9, f10, and f11 are cropped and scaled according to the corresponding target preview frame to obtain a new image corresponding to each frame, so that all frames in the whole screen realize displaying only the image content of the target of interest, that is, realize the target tracking of the video image.
Optionally, in step 208, the size and position of the target preview frame of each other frame in the current period are determined, so that in each frame of original images from the last frame of the previous period to the last frame of the current period, the target preview frame is gradually changed from the first size and the first position to the second size and the second position, and in the current period, the change step size of the target preview frame is consistent between any two adjacent frames.
Specifically, as shown in fig. 4e, in each original image of frames f6 to f12, the target preview frame gradually changes from a first size and a first position to a second size and a second position, the size change steps of the target preview frame are identical, that is, the change step of the preview frame width between two adjacent frames is SW, the change step of the preview frame height between two adjacent frames is SH,
in the continuous N frames, the width is changed N times, each time the change SW is changed, the height is changed N times, each time the change SH is changed, the frame number corresponding to each period is the same, so N is a fixed value, and in the frames from f6 to f12, the size change step length of the target preview frame between any two adjacent frames is consistent, so that the size process of the target preview frame is more gradually and stably determined in the target preview frame determination process of the same period.
Optionally, the difference between the first size and the second size in each period is the size difference of the preview frame corresponding to the period, and in different periods, the size change step of the target preview frame is positively correlated with the size difference of the corresponding preview frame; and/or the difference between the first position and the second position in each period is the position difference of the preview frame corresponding to the period, and in different periods, the position change step length of the target preview frame is positively correlated with the corresponding position difference of the preview frame.
Specifically, as shown in fig. 4e, in the process of gradually changing from the target preview frame of f6 frame to the target preview frame of f12 frame, if the moving speed of the preview frame is slow and the distance between the target preview frame of f6 frame and the target preview frame of f12 frame is long, the tracking screen may be caused to be out of time, and conversely, if the moving speed of the preview frame is fast, the tracking screen may be caused to jump greatly; therefore, in the embodiment of the application, the moving speed and the zooming speed of the preview frame are dynamically adjusted according to the relative position relationship and the size relationship of the preview frame corresponding to the last frame of the two periods, so that the moving and zooming processes of the preview frame are smooth and efficient. The moving speed of the preview frame can be adjusted through the position change step of the preview frame, in the process of clipping the image, the position change step can represent the pixel granularity of the clipping position interval in two adjacent frames, the larger the step is, the larger the pixel granularity of the clipping position interval is, the faster the moving speed is, but the fluency is poor; the preview frame zooming speed can be adjusted through the size change step of the preview frame, in the process of cropping the image, the size change step can represent the granularity of pixels cropped in two adjacent frames, the larger the step is, the faster the zooming speed is, but the fluency is poor, wherein the sizes respectively comprise the width and the height. In the embodiment of the application, if the difference between the first position and the second position is larger, the preview frame has a faster displacement speed, that is, a larger position change step length; if the difference between the first position and the second position is smaller, the preview frame is made to have a slower displacement speed, namely, a smaller position change step size; if the difference between the width in the first size and the width in the second size is larger, enabling the preview frame to have a faster width scaling speed, namely a larger width change step size; if the difference between the width in the first dimension and the width in the second dimension is small, the preview frame is made to have a slower width scaling speed, i.e., a smaller width change step size; if the difference value between the height in the first size and the height in the second size is larger, enabling the preview frame to have a faster height scaling speed, namely a larger height change step length; if the difference between the height in the first dimension and the height in the second dimension is small, the preview box is made to have a slower height scaling speed, i.e. a smaller height change step size.
As shown in fig. 5, fig. 5 is a block diagram of a tracking display apparatus in an embodiment of the present application, and the embodiment of the present application provides a tracking display apparatus, where the apparatus is configured to periodically perform a target preview box generation process, and the apparatus includes: the image acquisition module 1 is used for acquiring an original image; the target detection module 2 is used for executing target detection and acquiring a target frame of an interested target in the image; the merging module 3 is configured to perform region merging on the target frames of all the interested targets, so that the merged target frames cover all the interested targets; the judging module 4 is configured to judge whether the merged target frame falls into a complementary set region of a first preset target frame and a second preset target frame, where the first preset target frame and the second preset target frame are determined based on a target preview frame generated in a previous period, a size of the first preset target frame is larger than a size of the target preview frame generated in the previous period, and a size of the target preview frame generated in the previous period is larger than a size of the second preset target frame; the target preview frame generating module 5 is configured to, when it is determined that the merged target frame falls into the complementary set region, use the target preview frame generated in the previous period as the target preview frame generated in the current period, and when it is determined that the merged target frame does not fall into the complementary set region, use the merged target frame or the merged target frame with an outwardly expanded first preset size as the target preview frame generated in the current period; and the image generation module 6 is used for performing cutting or scaling processing on the original image according to the target preview frame generated in the current period to generate a corresponding new image so that the new image only shows the image content of the target of interest.
Specifically, the tracking display device may apply the tracking display method in the foregoing embodiments, and specific processes and principles are not described herein again.
It should be understood that the division of the modules of the apparatus shown in fig. 5 is merely a logical division, and the actual implementation may be wholly or partially integrated into one physical entity or may be physically separated. And these modules can be realized in the form of software called by processing element; or may be implemented entirely in hardware; and part of the modules can be realized in the form of calling by the processing element in software, and part of the modules can be realized in the form of hardware. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in the form of software.
For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. As another example, when one of the above modules is implemented in the form of a Processing element scheduler, the Processing element may be a general purpose processor, such as a Central Processing Unit (CPU) or other processor capable of invoking programs.
An embodiment of the present application further provides a tracking display device, including: a processor and a memory for storing at least one instruction which is loaded and executed by the processor to implement the tracking display method in the above embodiments.
The number of processors may be one or more, and the processors and the memory may be connected by a bus or other means.
The memory, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the transmission methods in the embodiments of the present application. The processor executes various functional applications and data processing by executing non-transitory software programs, instructions and modules stored in the memory, that is, the tracking display method in any of the above-described method embodiments is implemented.
The memory may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; and necessary data, etc. Further, the memory may include high speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid state storage device.
An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program runs on a computer, the computer is caused to execute the tracking display method in the foregoing embodiment.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the procedures or functions described in accordance with the present application are generated, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber optic, digital subscriber line) or wirelessly (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk), among others.
In the embodiments of the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, and means that there may be three relationships, for example, a and/or B, and may mean that a exists alone, a and B exist simultaneously, and B exists alone. Wherein A and B can be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or multiple.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
1. A tracking display method for presenting only image content including an object of interest in an image, the method comprising:
periodically executing a target preview box generation process, wherein in the current period, the target preview box generation process comprises the following steps:
acquiring an original image;
executing target detection to obtain a target frame of an interested target in the image;
carrying out region merging on the target frames of all the interested targets so that the merged target frames cover all the interested targets;
judging whether the merged target frame falls into a complementary set area of a first preset target frame and a second preset target frame, wherein the first preset target frame and the second preset target frame are determined based on a target preview frame generated in a previous period, the size of the first preset target frame is larger than that of the target preview frame generated in the previous period, and the size of the target preview frame generated in the previous period is larger than that of the second preset target frame;
when the merged target frame is determined to fall into a complementary set area, the target preview frame generated in the previous period is used as the target preview frame generated in the current period;
when the merged target frame is determined not to fall into the complementary set area, the merged target frame or the merged target frame is expanded outwards by a first preset size to serve as a target preview frame generated in the current period; and
and according to the target preview frame generated in the current period, performing cutting or scaling processing on the original image to generate a corresponding new image so that the new image only shows the image content of the target of interest.
2. The method of claim 1, wherein the first predetermined target frame is formed by the merged target frame in the previous period being expanded outward by a second predetermined size, and the second predetermined target frame is formed by the merged target frame in the previous period being reduced inward by a third predetermined size.
3. The method of claim 2, wherein the width-to-length ratio of the first preset size is the same as the width-to-length ratio of the original image size, and the width-to-length ratio of the second preset size is the same as the width-to-length ratio of the original image size.
4. The method of claim 3, wherein the second predetermined size is the same as the first predetermined size.
5. The method of claim 1,
the acquiring of the original image is as follows: acquiring an original image of a last frame in a current period;
the target preview frame generated in the previous period is the target preview frame of the last frame generated in the previous period;
the target preview frame generated in the current period is a target preview frame of the last frame generated in the current period;
the target preview frame of the last frame generated in the last period has a first size and a first position, the target preview frame of the last frame generated in the current period has a second size and a second position, and the target preview frame generation process further includes:
determining the size and the position of a target preview frame of each other frame in the current period, so that in each frame of original images from the last frame of the previous period to the last frame of the current period, the target preview frame is gradually changed from the first size and the first position to the second size and the second position, and in the current period, the change step length of the position of the target preview frame between any two adjacent frames is consistent;
and according to the determined target preview frame of each other frame in the current period, cutting or scaling the original image of each other frame in the current period to generate a new image corresponding to the original image of each frame, so that the new image only shows the image content of the target of interest.
6. The method of claim 5,
in the process of determining the size and the position of the target preview frame of each other frame in the current period so that the target preview frame gradually changes from the first size and the first position to the second size and the second position in each frame of the original image from the last frame of the previous period to the last frame of the current period, the change step length of the size of the target preview frame between any two adjacent frames in the current period is consistent.
7. The method of claim 5,
the difference between the first size and the second size in each period is the size difference of the preview frame corresponding to the period, and in different periods, the size change step length of the target preview frame is positively correlated with the size difference of the corresponding preview frame;
and/or the difference between the first position and the second position in each period is the position difference of the preview frame corresponding to the period, and in different periods, the position change step length of the target preview frame is positively correlated with the position difference of the corresponding preview frame.
8. A tracking display apparatus for periodically performing a target preview box generation process, the apparatus comprising:
the image acquisition module is used for acquiring an original image;
the target detection module is used for executing target detection and acquiring a target frame of an interested target in the image;
the merging module is used for performing region merging on the target frames of all the interested targets so that the merged target frames cover all the interested targets;
a judging module, configured to judge whether the merged target frame falls into a complementary set region of a first preset target frame and a second preset target frame, where the first preset target frame and the second preset target frame are determined based on a target preview frame generated in a previous period, a size of the first preset target frame is larger than a size of the target preview frame generated in the previous period, and a size of the target preview frame generated in the previous period is larger than a size of the second preset target frame;
a target preview frame generating module, configured to, when it is determined that the merged target frame falls into the complementary set region, use the target preview frame generated in the previous period as the target preview frame generated in the current period, and when it is determined that the merged target frame does not fall into the complementary set region, use the merged target frame or the merged target frame with an outwardly expanded first preset size as the target preview frame generated in the current period; and
and the image generation module is used for cutting or scaling the original image according to the target preview frame generated in the current period to generate a corresponding new image so that the new image only shows the image content of the target of interest.
9. A tracking display device, comprising:
a processor and a memory for storing at least one instruction which is loaded and executed by the processor to implement the method of any one of claims 1 to 7.
10. A computer-readable storage medium, in which a computer program is stored which, when run on a computer, causes the computer to carry out the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010448183.9A CN113727009B (en) | 2020-05-25 | 2020-05-25 | Tracking display method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010448183.9A CN113727009B (en) | 2020-05-25 | 2020-05-25 | Tracking display method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113727009A true CN113727009A (en) | 2021-11-30 |
| CN113727009B CN113727009B (en) | 2022-11-08 |
Family
ID=78671533
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010448183.9A Active CN113727009B (en) | 2020-05-25 | 2020-05-25 | Tracking display method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113727009B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114222065A (en) * | 2021-12-20 | 2022-03-22 | 北京奕斯伟计算技术有限公司 | Image processing method, image processing apparatus, electronic device, storage medium, and program product |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120093407A1 (en) * | 2010-09-21 | 2012-04-19 | Sony Corporation | Object detection and classification method and apparatus |
| US20140205141A1 (en) * | 2013-01-22 | 2014-07-24 | Qualcomm Incorporated | Systems and methods for tracking and detecting a target object |
| US20150178943A1 (en) * | 2013-12-21 | 2015-06-25 | Qualcomm Incorporated | System and method to stabilize display of an object tracking box |
| JP2015228164A (en) * | 2014-06-02 | 2015-12-17 | 大日本印刷株式会社 | Object detection system and object detection method |
| US20170302719A1 (en) * | 2016-04-18 | 2017-10-19 | Qualcomm Incorporated | Methods and systems for auto-zoom based adaptive video streaming |
| US20180254065A1 (en) * | 2017-03-03 | 2018-09-06 | Qualcomm Incorporated | Methods and systems for splitting non-rigid objects for video analytics |
| US20180268563A1 (en) * | 2017-03-14 | 2018-09-20 | Qualcomm Incorporated | Methods and systems for performing sleeping object detection in video analytics |
| US20180286199A1 (en) * | 2017-03-31 | 2018-10-04 | Qualcomm Incorporated | Methods and systems for shape adaptation for merged objects in video analytics |
| CN108629354A (en) * | 2017-03-17 | 2018-10-09 | 杭州海康威视数字技术股份有限公司 | Object detection method and device |
| US20190130580A1 (en) * | 2017-10-26 | 2019-05-02 | Qualcomm Incorporated | Methods and systems for applying complex object detection in a video analytics system |
| CN109712167A (en) * | 2017-10-25 | 2019-05-03 | 财团法人资讯工业策进会 | Target detecting and method for tracing and system |
-
2020
- 2020-05-25 CN CN202010448183.9A patent/CN113727009B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120093407A1 (en) * | 2010-09-21 | 2012-04-19 | Sony Corporation | Object detection and classification method and apparatus |
| US20140205141A1 (en) * | 2013-01-22 | 2014-07-24 | Qualcomm Incorporated | Systems and methods for tracking and detecting a target object |
| US20150178943A1 (en) * | 2013-12-21 | 2015-06-25 | Qualcomm Incorporated | System and method to stabilize display of an object tracking box |
| JP2015228164A (en) * | 2014-06-02 | 2015-12-17 | 大日本印刷株式会社 | Object detection system and object detection method |
| US20170302719A1 (en) * | 2016-04-18 | 2017-10-19 | Qualcomm Incorporated | Methods and systems for auto-zoom based adaptive video streaming |
| US20180254065A1 (en) * | 2017-03-03 | 2018-09-06 | Qualcomm Incorporated | Methods and systems for splitting non-rigid objects for video analytics |
| US20180268563A1 (en) * | 2017-03-14 | 2018-09-20 | Qualcomm Incorporated | Methods and systems for performing sleeping object detection in video analytics |
| CN108629354A (en) * | 2017-03-17 | 2018-10-09 | 杭州海康威视数字技术股份有限公司 | Object detection method and device |
| US20180286199A1 (en) * | 2017-03-31 | 2018-10-04 | Qualcomm Incorporated | Methods and systems for shape adaptation for merged objects in video analytics |
| CN109712167A (en) * | 2017-10-25 | 2019-05-03 | 财团法人资讯工业策进会 | Target detecting and method for tracing and system |
| US20190130580A1 (en) * | 2017-10-26 | 2019-05-02 | Qualcomm Incorporated | Methods and systems for applying complex object detection in a video analytics system |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114222065A (en) * | 2021-12-20 | 2022-03-22 | 北京奕斯伟计算技术有限公司 | Image processing method, image processing apparatus, electronic device, storage medium, and program product |
| CN114222065B (en) * | 2021-12-20 | 2024-03-08 | 北京奕斯伟计算技术股份有限公司 | Image processing method, image processing apparatus, electronic device, storage medium, and program product |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113727009B (en) | 2022-11-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3511903A1 (en) | Three-dimensional model generating device and three-dimensional model generating method | |
| US10412462B2 (en) | Video frame rate conversion using streamed metadata | |
| US11141557B2 (en) | Information processing apparatus, information processing method, and storage medium | |
| WO2019210546A1 (en) | Data processing method and computing device | |
| CN109242944B (en) | Display method and device | |
| KR102248799B1 (en) | Target object display method and apparatus, and electronic device | |
| CN110796664B (en) | Image processing method, device, electronic equipment and computer readable storage medium | |
| US10154228B1 (en) | Smoothing video panning | |
| KR101831516B1 (en) | Method and apparatus for generating image using multi-stiker | |
| CN107959798B (en) | Video data real-time processing method and device and computing equipment | |
| CN107087121A (en) | A kind of Autoconducting method and device based on motion detection | |
| CN113727009B (en) | Tracking display method, device and storage medium | |
| CN111340101B (en) | Stability evaluation method, apparatus, electronic device, and computer-readable storage medium | |
| CN110689014B (en) | Method and device for detecting region of interest, electronic equipment and readable storage medium | |
| JP2008046608A (en) | Video window detector | |
| US9582893B2 (en) | Motion vector detector, distance detector, motion vector detection method and computer program product | |
| CN111833459B (en) | Image processing method and device, electronic equipment and storage medium | |
| CN112788337A (en) | Video automatic motion compensation method, device, equipment and storage medium | |
| CN108198232B (en) | Method and equipment for drawing track box | |
| JP2022529414A (en) | Methods and systems for motion detection without malfunction | |
| CN115049572A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN115937291A (en) | Binocular image generation method and device, electronic equipment and storage medium | |
| CN111221486B (en) | Information display system and information display method | |
| JP2018147241A (en) | Image processing device, image processing method, and image processing program | |
| CN107977644B (en) | Image data processing method and device based on image acquisition equipment and computing equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |