WO2019041661A1 - Video abstract generating method and device - Google Patents

Video abstract generating method and device Download PDF

Info

Publication number
WO2019041661A1
WO2019041661A1 PCT/CN2017/117167 CN2017117167W WO2019041661A1 WO 2019041661 A1 WO2019041661 A1 WO 2019041661A1 CN 2017117167 W CN2017117167 W CN 2017117167W WO 2019041661 A1 WO2019041661 A1 WO 2019041661A1
Authority
WO
WIPO (PCT)
Prior art keywords
slices
video
slice
target
moving
Prior art date
Application number
PCT/CN2017/117167
Other languages
French (fr)
Chinese (zh)
Inventor
许芳
丁美玉
赵波
晋兆龙
陈卫东
Original Assignee
苏州科达科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州科达科技股份有限公司 filed Critical 苏州科达科技股份有限公司
Publication of WO2019041661A1 publication Critical patent/WO2019041661A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/835—Generation of protective data, e.g. certificates
    • H04N21/8352—Generation of protective data, e.g. certificates involving content or source identification data, e.g. Unique Material Identifier [UMID]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/858—Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
    • H04N21/8586—Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot by using a URL

Definitions

  • the present invention relates to the field of video processing technologies, and in particular, to a video summary generation method and apparatus.
  • Video as a major type of multimedia, has played an increasingly prominent role in enriching people's lives, education, and entertainment. How to effectively organize, manage, query, retrieve and browse video data becomes an urgent problem to be solved.
  • Video summary is an effective way to solve the above problems.
  • Video summary also known as video enrichment, refers to a simple summary of video content.
  • video enrichment In an automatic or semi-automatic way, first through moving target analysis, extracting moving targets, and then targeting each target. The motion trajectory is analyzed, and different targets are spliced into a common background scene, and they are combined in some way.
  • Video summaries play an important role in video analytics and content-based video retrieval. Usually a few hours of video, you can condense into a few minutes of concentrated video, greatly speeding up the video viewing speed.
  • the inventor In the method for generating the video summary of the inventor, the inventor firstly analyzes the video content from the going to the rear, extracts the moving target, and then analyzes the motion trajectory of each target.
  • the method for generating a video digest has different time for generating video digests for video content of different durations.
  • the digest generation time is relatively long and inefficient.
  • the embodiments of the present invention provide a method and a device for generating a video digest to solve the problem of long time and low efficiency in generating video digests for video content with a long duration.
  • a first aspect of the present invention provides a video summary generating method, including the following steps:
  • the analysis result of the moving target is concentrated and displayed by using the sequence of the plurality of slices to generate a video summary of the to-be-processed video.
  • the method further includes:
  • a unique identifier is generated for each of the slices, and the identifier corresponding to each slice also corresponds to its location in the to-be-processed video.
  • the performing a target analysis task for each of the slices separately, and obtaining an analysis result of the moving target in each of the slices including:
  • the identifiers corresponding to the moving targets are merged into one.
  • the first thread is started to perform a condensed calculation on the analysis result of the moving target by using the sequence of the multiple slices to generate a video summary of the to-be-processed video
  • the second thread is started to Concentrate the calculated video summary for display.
  • the concentrating calculation of the analysis result of the moving target by using the sequence of the plurality of slices comprises:
  • the video image is extracted in turn based on the sorted moving objects in the slice.
  • a second aspect of the present invention provides a video summary generating apparatus, including:
  • a first acquiring unit configured to acquire a video to be processed
  • a slice dividing unit configured to divide the to-be-processed video into multiple slices
  • a processing unit configured to open a plurality of threads corresponding to the plurality of slices one by one, perform target analysis tasks for each of the slices separately, and obtain an analysis result of the moving targets in each of the slices;
  • a concentrating unit configured to perform a condensed calculation on the analysis result of the moving target by using a sequence of the plurality of slices to generate a video summary of the to-be-processed video.
  • the video summary generating apparatus further includes:
  • a recording unit for recording the starting and ending position of each slice
  • a marking unit configured to generate a unique identifier for each of the slices, and the identifier corresponding to each slice also corresponds to a location in the to-be-processed video.
  • the processing unit includes:
  • a first obtaining subunit configured to acquire a moving target in each of the slices
  • a marking subunit configured to generate an identifier of a moving target of each of the slices by using a sequence of the plurality of slices
  • a second acquiring subunit configured to acquire a moving target that appears in an adjacent slice at an adjacent moment
  • a determining subunit configured to determine, according to the location feature of the moving target, whether the moving targets of the adjacent slices appearing at the adjacent moments are the same;
  • the merging subunit is configured to combine the identifiers corresponding to the moving objects into one when it is determined that the moving targets of the adjacent slices appearing at the same time are the same.
  • the concentration unit includes:
  • a sorting subunit configured to sort the slices according to the identifier of the slice
  • extracting a subunit configured to sequentially extract a video image according to the sorted moving target in the slice.
  • a third aspect of the present invention provides an image processing apparatus including at least one processor; and a memory communicably coupled to the at least one processor; wherein the memory stores instructions executable by the one processor, The instructions are executed by the at least one processor to cause the at least one processor to perform the video summary generation method of any of the first or first aspect of the invention.
  • a fourth aspect of the invention provides a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the first or first aspect of the invention
  • the video summary generation method according to any one of the preceding claims.
  • a fifth aspect of the invention provides a computer program product comprising a computing program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer
  • the computer is configured to perform the video summary generation method described in the first aspect or any one of the optional aspects of the first aspect.
  • the video summary generating method includes the following steps: acquiring a to-be-processed video; dividing the to-be-processed video into multiple slices; and opening multiple threads corresponding to the multiple slices one by one, respectively Performing a target analysis task for each of the slices, obtaining an analysis result of the moving target in each of the slices; performing a concentration calculation on the analysis result of the moving target by using the sequence of the plurality of slices to generate the A video summary of the pending video.
  • the video summary generation method provided by the embodiment of the present invention uses a multi-thread to process a target analysis task of a video in parallel, and then comprehensively condenses the distributed analysis result to generate a video summary, so as to achieve the purpose of browsing all the contents of the video in a short time. , which greatly reduces the waiting time.
  • the video summary generating method provided by the embodiment of the present invention, after dividing the to-be-processed video into a plurality of slices, further comprising: recording a start and end position of each slice; generating a unique identifier for each of the slices
  • the identifier corresponding to each slice also corresponds to its location in the to-be-processed video.
  • by assigning a unique identifier to each slice it is convenient to save the target analysis result in different slices according to the identifier when performing target analysis on all the slices at the same time, thereby avoiding the storage result between the storage results. conflict.
  • the video summary generating method provided by the embodiment of the present invention, wherein the first thread is started to perform a condensed calculation on the analysis result of the moving target by using the sequence of the plurality of slices to generate the to-be-processed video.
  • the video summary is simultaneously opened by the second thread to display the video summary obtained by the concentration calculation.
  • the segmentation and concentration calculation of the moving target in the slice is performed, and the concentration calculation and the concentration display are performed in two threads concurrently, thereby achieving the purpose of displaying the edge while processing, thereby solving the target in the to-be-processed video. More, need It takes a lot of time to wait for the target to be concentrated.
  • the method for generating a video summary wherein the concentrating calculation of the analysis result of the moving target by using the sequence of the plurality of slices comprises: performing the slice according to the identifier of the slice Sorting; sequentially extracting video images from the moving targets in the sorted slices.
  • the target analysis is performed on all the slices at the same time, and the analysis result can be stored in different places. Therefore, the position of each slice in the video can be restored by the identifier of the slice, thereby avoiding confusion of the slice sequence, thereby The conditions for extracting video images from moving targets are provided.
  • the video summary generating apparatus includes: a first acquiring unit, configured to acquire a video to be processed; a slice dividing unit, configured to divide the to-be-processed video into multiple slices; and a processing unit, configured to: Opening a plurality of threads corresponding to the plurality of slices one by one, performing a target analysis task for each of the slices separately, obtaining an analysis result of the moving target in each slice; and a concentrating unit, configured to utilize the plurality of The sequence of the slices is used to perform a condensed calculation on the analysis result of the moving target to generate a video summary of the to-be-processed video.
  • the video summary generating apparatus processes a target analysis task of a video by using multiple threads in parallel, and then comprehensively condenses the distributed analysis results to generate a video summary, so as to achieve the purpose of browsing all the contents of the video in a short time. , which greatly reduces the waiting time.
  • FIG. 1 is a flowchart showing a specific schematic diagram of a video digest generating method in Embodiment 1 of the present invention
  • FIG. 2 is a flowchart showing a specific schematic diagram of a video summary generating method in Embodiment 2 of the present invention
  • FIG. 3 is a flowchart showing a specific schematic diagram of a video digest generating method in Embodiment 3 of the present invention.
  • Figure 4 is a diagram showing a specific schematic of the video summary generating apparatus in Embodiment 4 of the present invention.
  • FIG. 5 is a block diagram showing a specific schematic diagram of a processing unit in Embodiment 4 of the present invention.
  • Figure 6 is a block diagram showing a specific schematic diagram of a concentration unit in Embodiment 4 of the present invention.
  • Fig. 7 is a block diagram showing a specific schematic diagram of an image processing apparatus in Embodiment 5 of the present invention.
  • This embodiment provides a video summary generation method for use in a video summary generating device. As shown in FIG. 1, the video summary generating method includes the following steps:
  • step S11 a video to be processed is obtained.
  • the video to be processed in this embodiment may be a video that is previously stored in the video summary generating device, or that is acquired by the video summary generating device from the outside world in real time, or may be a video segment extracted by the video summary generating device from a video.
  • step S12 the video to be processed is divided into a plurality of slices.
  • the number of slices can be divided according to the processing capability of the video summary generating device. For example, according to the ability of the hardware circuit to process data in the video summary device, the maximum amount of data that can be concurrently processed can be determined, and the division of the to-be-processed video can be performed according to the amount of data.
  • all slices constitute a complete pending video
  • several video frames may be included in each slice, and each frame includes several targets.
  • Step S13 Open a plurality of threads corresponding to the plurality of slices one by one, perform target analysis tasks for each slice separately, and obtain an analysis result of the moving target in each slice.
  • each of the threads is separately executed in the video summary generating device.
  • the number of threads to be opened is equal to the number of slices divided by the video to be processed, that is, the target analysis task of performing all slices simultaneously in the video summary generating device can be achieved.
  • the analysis results of the moving targets in each slice can be saved in different paths, or can be distinguished by different tags in the same path.
  • step S14 the analysis result of the moving target is concentrated and calculated by using the sequence of the plurality of slices to generate a video summary of the video to be processed.
  • step S13 all the slices are simultaneously subjected to the target analysis task, and the analysis results of the moving targets in each slice are not necessarily maintained in the original sequential relationship. Therefore, when performing the condensed calculation of the analysis result of the moving target, it is necessary to follow the order of the slices, so that the disorder of the slicing order can be avoided.
  • the video summary generation method provided in this embodiment uses a multi-thread to process a target analysis task of a video in parallel, and then comprehensively condenses the distributed analysis results to generate a video summary, so as to achieve the purpose of browsing all the contents of the video in a short time. This greatly reduces the waiting time, and the effect is better for video content with a longer duration.
  • This embodiment provides a video summary generation method for use in a video summary generating device. As shown in FIG. 2, the video summary generation method includes the following steps:
  • Step S21 Acquire a video to be processed. The same as step S11 of Embodiment 1, and details are not described herein again.
  • step S22 the video to be processed is divided into a plurality of slices.
  • the video to be processed is divided into a plurality of slice files by previewing the total duration of the video to be processed in the video summary generating device and according to the capability of processing data of the hardware circuit in the video summary generating device.
  • the start and end positions of each slice are recorded for subsequent decoding operations on the slices before performing the target analysis task for each slice.
  • a unique identifier is generated for each slice, the size of the identifier corresponding to the position of the slice in the video to be processed; that is, the size of the identifier and the time sequence of the slice in the video to be processed remain Consistent, so that the adjacent front and back slices can be quickly found by the logo.
  • the identifier in this embodiment may be represented by a numerical value or a character. Among them, the value or character is set in order according to the time of the slice in the video to be processed.
  • the identifier in this embodiment is represented by a numerical value. For example, use 1, 2, 3, ..., in turn as the identifier of the slice.
  • Step S23 opening a plurality of threads corresponding to the plurality of slices one by one, respectively performing target analysis tasks for each slice, and obtaining an analysis result of the moving targets in each slice.
  • the target analysis task for each slice is separately performed by turning on a plurality of threads in the video summary generating device.
  • the number of threads to be opened is equal to the number of slices divided by the video to be processed, that is, the target analysis task of performing all slices simultaneously in the video summary generating device can be achieved.
  • each slice includes several targets, so in the present embodiment, by assigning a unique identifier to each target in each slice, the target is distinguished, so that the target in the pending video is unique. logo.
  • performing a target analysis task for each slice separately, and obtaining an analysis result of the moving target in each slice including the following steps:
  • Step S231 acquiring a moving target in each slice.
  • a small segment of each slice is quickly analyzed to obtain a static background of a stable slice; then, according to the static background, the target in the slice that does not belong to the static background is extracted. , that is, the goal of sports.
  • the static background is not static for some scenes in the slice, and any moving to a stationary target will become part of the background. Therefore, it is necessary to constantly update the background and save each changed background. And an image of the target track in each of the to-be-processed videos, specifically, a time period in which the background image disappears, and a position in the video to be processed.
  • Step S232 generating an identifier of the moving target of each slice by using the sequence of the plurality of slices.
  • each slice includes a plurality of targets, and each target in each slice is assigned a unique identifier for distinguishing the target, thereby achieving the target in the to-be-processed video.
  • a unique identifier There is a unique identifier.
  • the identifier can be identified by a numerical value or by a character.
  • the number of targets appearing in each slice may be determined in advance, and then the start identifier of the target in each slice is sequentially set according to the order of the slices, and the difference between the start identifiers of the connected slices is greater than or equal to the pre-determined The number of targets in each slice.
  • the target identifier in each slice may be sequentially incremented according to the order in which the target appears, on the basis of the initial identifier.
  • each slice in the video to be processed and the identifier of the target in each slice are as follows:
  • the first column of each row represents each slice in the to-be-processed video and its corresponding identifier
  • the remaining columns in each row represent the target in the slice represented in the first column and its corresponding identifier.
  • slice (001) identifies the slice as 001
  • target (001001) represents the first target in slice (001)
  • the target's identity is 001001; the rest, and so on.
  • each slice includes a plurality of video frames, and in the process of performing target analysis in the slice, the same target is merged for the adjacent frames, thereby avoiding that the same target in the adjacent video frames is given different Logo.
  • the moving target extracted from each frame of the slice is matched before and after according to the position feature of the moving target, and the target with the matching match is given the same identifier, and the matching is successful.
  • the identity of the target remains the same.
  • the target information of the merged process is saved, and specifically, the information includes the target track, the appearance and disappearance time, and the like.
  • Step S233 Acquire a moving target that appears in an adjacent slice at an adjacent moment.
  • the adjacent slice is obtained by using the identifier of the current slice, and the current slice and the adjacent target slice are respectively acquired at the adjacent time, and the corresponding target is acquired at the same time.
  • the identification of the moving target is obtained by using the identifier of the current slice, and the current slice and the adjacent target slice are respectively acquired at the adjacent time, and the corresponding target is acquired at the same time.
  • Step S234 judging whether the moving targets appearing at the adjacent moments of the adjacent slices are the same according to the position feature of the moving target. If the result of the determination is yes, then step S235 is performed; otherwise, step S24 is performed.
  • the positional feature of the obtained moving target is calculated by the tracking algorithm. If the calculated positional images of the moving target overlap, it means that the adjacent moving objects appear at the same time are the same; if the calculated If the position pictures of the moving objects do not overlap, it means that the adjacent slices have different moving targets at the adjacent moments.
  • step S235 the identifiers corresponding to the moving targets are merged into one.
  • the same target is given the same identifier.
  • the merging of the same target is for the purpose of not considering the content of the puncturing point when slicing the video to be processed, which will result in the same target in the real video at the segmentation point (ie, the character or The vehicle is allocated in two slices, then the same target will have different identifiers in each slice; if the same target is not merged, then the same target will be different in the subsequent target concentration calculation. Time appears.
  • the adjacent slices are acquired according to the identifier of the slice, and whether the moving targets of the adjacent slices appearing at the adjacent time are the same, and if they are the same, the same identifier is given, thereby Achieve the same goal at the same time.
  • step S24 the analysis result of the moving target is concentrated and calculated by using the sequence of the plurality of slices to generate a video summary of the video to be processed.
  • step S14 of Embodiment 1 the same as step S14 of Embodiment 1, and details are not described herein again.
  • This embodiment provides a video summary generation method for use in a video summary generating device. As shown in FIG. 3, the video summary generating method includes the following steps:
  • Step S31 acquiring a video to be processed. The same as step S21 of Embodiment 2, and details are not described herein again.
  • step S32 the video to be processed is divided into a plurality of slices. The same as step S22 of Embodiment 2, and details are not described herein again.
  • Step S33 opening a plurality of threads corresponding to the plurality of slices one by one, respectively performing target analysis tasks for each slice, and obtaining an analysis result of the moving targets in each slice.
  • step S23 of Embodiment 2 The same as step S23 of Embodiment 2, and details are not described herein again.
  • step S34 the analysis result of the moving target is concentrated and calculated by using the sequence of the plurality of slices to generate a video summary of the video to be processed.
  • the first thread is started by the video summary generating device to perform a condensed calculation on the analysis result of the moving target by using the sequence of the plurality of slices to generate a video summary of the to-be-processed video, and simultaneously open the second thread to the concentrated calculation.
  • the resulting video summary is displayed.
  • the analysis result of the moving target is concentrated and calculated by using the sequence of the plurality of slices, and the following steps are included:
  • step S341 the slices are sorted according to the identifier of the slice.
  • the video summary generating device extracts the target analysis result of each slice from each storage path, and starts the first thread, and sorts the extracted slice according to the identifier of the slice.
  • the moving targets in each slice are sequentially read from the back to the rear, and when a certain number of moving targets is read, the reading operation is stopped first; the moving targets that have been read are arranged, and then Start the second thread and display the results of the layout calculation. Repeat the process of reading, calculating, and displaying until all targets have been processed.
  • the purpose of the edge processing and the display can be realized, thereby solving the target in the to-be-processed video.
  • the number is large, and it takes a lot of time to wait for the target to be concentrated.
  • the first obtaining unit 41 is configured to acquire a video to be processed.
  • the slice dividing unit 42 is configured to divide the to-be-processed video into multiple slices.
  • the concentrating unit 44 is configured to perform condensed calculation on the analysis result of the moving target by using the sequence of the plurality of slices to generate a video summary of the video to be processed.
  • the first obtaining subunit 431 is configured to acquire a moving target in each slice.
  • the marking sub-unit 432 is configured to generate an identification of the moving target of each slice by using the sequence of the plurality of slices.
  • the second obtaining sub-unit 433 is configured to acquire a moving target that appears in an adjacent slice at an adjacent moment.
  • the determining sub-unit 434 is configured to determine, according to the position feature of the moving target, whether the moving targets of the adjacent slices appearing at the adjacent moments are the same.
  • the merging sub-unit 435 is configured to determine the moving target phase of the adjacent slice at the adjacent moment At the same time, the logos corresponding to the moving targets are merged into one.
  • the concentrating unit 44 includes:
  • the sorting sub-unit 441 is configured to sort the slices according to the identifier of the slice.
  • the extracting sub-unit 442 is configured to sequentially extract a video image according to the moving target in the sorted slice.
  • FIG. 7 is a schematic diagram showing the hardware structure of an image processing apparatus according to an embodiment of the present invention. As shown in FIG. 7, the apparatus includes one or more processors 51 and a memory 52. One processor 51 is taken as an example in FIG.
  • the memory 52 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function; the storage data area may store data created according to use of the video summary generating device, and the like.
  • the memory 52 can include a high speed
  • the machine access memory may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid state storage device.
  • memory 52 can optionally include memory remotely located relative to processor 51, which can be connected to the video summary generating device over a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
  • the one or more modules are stored in the memory 52, and when executed by the one or more processors 51, the video summary generating method described in any one of Embodiments 1 to 3 is performed.
  • the above product can perform the method provided by the embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
  • the technical details that are not described in detail in this embodiment, refer to the related description in the embodiment shown in FIG.
  • the embodiment of the present invention further provides a non-transitory computer storage medium storing computer executable instructions, which can execute the video according to any one of Embodiments 1 to 3. Summary generation method.
  • the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk (Hard). Disk Drive, abbreviated as: HDD) or Solid-State Drive (SSD), etc.; the storage medium may also include a combination of the above types of memories.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

Disclosed in the invention are a video abstract generating method and device. The method comprises the following steps: obtaining a video to be processed; dividing the video to be processed into a plurality of slices; starting a plurality of threads in one-to-one correspondence to the plurality of slices, and respectively performing a target analysis task on each slice to obtain an analysis result of a moving target in each slice; and carrying out concentration calculation on the analysis results of the moving targets based on a sequential order of the plurality of slices to generate a video abstract of the video to be processed. According to the video abstract generating method provided by embodiments of the invention, by processing target analysis tasks of a video by using multiple threads in parallel and then carrying out integrated concentration on the dispersed analysis results to generate a video abstract, an objective of browsing all contents of the video within a short time is achieved and thus waiting time is significantly reduced.

Description

视频摘要生成方法及装置Video summary generation method and device 技术领域Technical field
本发明涉及视频处理技术领域，具体涉及一种视频摘要生成方法及装置。The present invention relates to the field of video processing technologies, and in particular, to a video summary generation method and apparatus.
背景技术Background technique
随着人们对多媒体信息需求的不断增加，多媒体数据大量涌现，视频作为一种主要的多媒体类型，在丰富人们的生活、教育、娱乐等方面起到越来越突出的作用。如何有效地组织、管理、查询、检索和浏览视频数据，成为迫切需要解决的问题。As people's demand for multimedia information continues to increase, multimedia data has emerged in large numbers. Video, as a major type of multimedia, has played an increasingly prominent role in enriching people's lives, education, and entertainment. How to effectively organize, manage, query, retrieve and browse video data becomes an urgent problem to be solved.
视频摘要是解决上述问题的一个有效途径，视频摘要又称视频浓缩，是指对视频内容的一个简单的概括，以自动或半自动的方式，先通过运动目标分析，提取运动目标，然后对各个目标的运动轨迹进行分析，将不同的目标拼接到一个共同的背景场景中，将它们以某种方式进行组合。视频摘要在视频分析和基于内容的视频检索中扮演着重要角色。通常几个小时的录像，可以浓缩成几分钟的浓缩视频，大大加快了录像查看的速度。Video summary is an effective way to solve the above problems. Video summary, also known as video enrichment, refers to a simple summary of video content. In an automatic or semi-automatic way, first through moving target analysis, extracting moving targets, and then targeting each target. The motion trajectory is analyzed, and different targets are spliced into a common background scene, and they are combined in some way. Video summaries play an important role in video analytics and content-based video retrieval. Usually a few hours of video, you can condense into a few minutes of concentrated video, greatly speeding up the video viewing speed.
发明人在研发视频摘要的生成方法中，先是采用对视频内容从前往后依次分析，提取运动目标，然后对各个目标的运动轨迹进行分析。但是，发明人发现这种视频摘要的生成方法，对于不同时长的视频内容，视频摘要生成的时间不同；此外，对于时长较长的视频内容，其摘要生成的时间相对较长、效率低。In the method for generating the video summary of the inventor, the inventor firstly analyzes the video content from the going to the rear, extracts the moving target, and then analyzes the motion trajectory of each target. However, the inventor has found that the method for generating a video digest has different time for generating video digests for video content of different durations. In addition, for video content with a long duration, the digest generation time is relatively long and inefficient.
发明内容Summary of the invention
有鉴于此，本发明实施例提供了一种视频摘要生成方法及装置，以解决时长较长的视频内容生成视频摘要的耗时长、效率低的问题。In view of this, the embodiments of the present invention provide a method and a device for generating a video digest to solve the problem of long time and low efficiency in generating video digests for video content with a long duration.
本发明第一方面提供了一种视频摘要生成方法，包括以下步骤： A first aspect of the present invention provides a video summary generating method, including the following steps:
获取待处理视频；Get the pending video;
将所述待处理视频划分为多个切片；Dividing the to-be-processed video into multiple slices;
开启与所述多个切片一一对应的多个线程，分别执行对每个所述切片的目标分析任务，得到每个所述切片中运动目标的分析结果；Opening a plurality of threads corresponding to the plurality of slices one by one, respectively performing target analysis tasks for each of the slices, and obtaining an analysis result of the moving targets in each of the slices;
利用所述多个切片的先后顺序对所述运动目标的分析结果进行浓缩展示，生成所述待处理视频的视频摘要。The analysis result of the moving target is concentrated and displayed by using the sequence of the plurality of slices to generate a video summary of the to-be-processed video.
可选地，在将所述待处理视频划分为多个切片之后，还包括：Optionally, after the video to be processed is divided into multiple slices, the method further includes:
记录每个切片的起止位置；Record the start and end position of each slice;
对每个所述切片生成唯一的标识，每个切片对应的标识还与其在所述待处理视频中的位置对应。A unique identifier is generated for each of the slices, and the identifier corresponding to each slice also corresponds to its location in the to-be-processed video.
可选地，所述分别执行对每个所述切片的目标分析任务，得到每个所述切片中运动目标的分析结果，包括：Optionally, the performing a target analysis task for each of the slices separately, and obtaining an analysis result of the moving target in each of the slices, including:
获取每个所述切片中的运动目标；Obtaining a moving target in each of the slices;
利用所述多个切片的先后顺序生成每个所述切片的运动目标的标识；Generating an identification of a moving target of each of the slices using a sequence of the plurality of slices;
获取相邻的切片在相邻时刻出现的运动目标；Obtaining moving targets of adjacent slices that appear at adjacent moments;
根据所述运动目标的位置特征判断所述相邻的切片在相邻时刻出现的运动目标是否相同；Determining, according to the position feature of the moving target, whether the moving targets of the adjacent slices appearing at the adjacent moments are the same;
当判断所述相邻的切片在相邻时刻出现的运动目标相同时，将所述运动目标对应的标识合并成一个。When it is determined that the moving targets of the adjacent slices appearing at the same time are the same, the identifiers corresponding to the moving targets are merged into one.
可选地，开启第一线程以执行利用所述多个切片的先后顺序对所述运动目标的分析结果进行浓缩计算，以生成所述待处理视频的视频摘要，同时开启第二线程对所述浓缩计算得到的视频摘要进行展示。Optionally, the first thread is started to perform a condensed calculation on the analysis result of the moving target by using the sequence of the multiple slices to generate a video summary of the to-be-processed video, and the second thread is started to Concentrate the calculated video summary for display.
可选地，所述利用所述多个切片的先后顺序对所述运动目标的分析结果进行浓缩计算，包括：Optionally, the concentrating calculation of the analysis result of the moving target by using the sequence of the plurality of slices comprises:
根据所述切片的标识对所述切片进行排序；Sorting the slices according to the identifier of the slice;
依次根据排序后的所述切片中运动目标提取视频图像。The video image is extracted in turn based on the sorted moving objects in the slice.
本发明第二方面提供了一种视频摘要生成装置，包括：A second aspect of the present invention provides a video summary generating apparatus, including:
第一获取单元，用于获取待处理视频； a first acquiring unit, configured to acquire a video to be processed;
切片划分单元，用于将所述待处理视频划分为多个切片；a slice dividing unit, configured to divide the to-be-processed video into multiple slices;
处理单元，用于开启与所述多个切片一一对应的多个线程，分别执行对每个所述切片的目标分析任务，得到每个所述切片中运动目标的分析结果；a processing unit, configured to open a plurality of threads corresponding to the plurality of slices one by one, perform target analysis tasks for each of the slices separately, and obtain an analysis result of the moving targets in each of the slices;
浓缩单元，用于利用所述多个切片的先后顺序对所述运动目标的分析结果进行浓缩计算，以生成所述待处理视频的视频摘要。And a concentrating unit, configured to perform a condensed calculation on the analysis result of the moving target by using a sequence of the plurality of slices to generate a video summary of the to-be-processed video.
可选地，所述视频摘要生成装置，还包括：Optionally, the video summary generating apparatus further includes:
记录单元，用于记录每个切片的起止位置；a recording unit for recording the starting and ending position of each slice;
标记单元，用于对每个所述切片生成唯一的标识，每个切片对应的标识还与其在所述待处理视频中的位置对应。And a marking unit, configured to generate a unique identifier for each of the slices, and the identifier corresponding to each slice also corresponds to a location in the to-be-processed video.
可选地，所述处理单元，包括：Optionally, the processing unit includes:
第一获取子单元，用于获取每个所述切片中的运动目标；a first obtaining subunit, configured to acquire a moving target in each of the slices;
标记子单元，用于利用所述多个切片的先后顺序生成每个所述切片的运动目标的标识；a marking subunit, configured to generate an identifier of a moving target of each of the slices by using a sequence of the plurality of slices;
第二获取子单元，用于获取相邻的切片在相邻时刻出现的运动目标；a second acquiring subunit, configured to acquire a moving target that appears in an adjacent slice at an adjacent moment;
判断子单元，用于根据所述运动目标的位置特征判断所述相邻的切片在相邻时刻出现的运动目标是否相同；a determining subunit, configured to determine, according to the location feature of the moving target, whether the moving targets of the adjacent slices appearing at the adjacent moments are the same;
合并子单元，用于当判断所述相邻的切片在相邻时刻出现的运动目标相同时，将所述运动目标对应的标识合并成一个。The merging subunit is configured to combine the identifiers corresponding to the moving objects into one when it is determined that the moving targets of the adjacent slices appearing at the same time are the same.
可选地，所述浓缩单元，包括：Optionally, the concentration unit includes:
排序子单元，用于根据所述切片的标识对所述切片进行排序；a sorting subunit, configured to sort the slices according to the identifier of the slice;
提取子单元，用于依次根据排序后的所述切片中运动目标提取视频图像。And extracting a subunit, configured to sequentially extract a video image according to the sorted moving target in the slice.
本发明第三方面提供了一种图像处理装置，包括至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器执行本发明第一方面或第一方面中任一项所述的视频摘要生成方法。 A third aspect of the present invention provides an image processing apparatus including at least one processor; and a memory communicably coupled to the at least one processor; wherein the memory stores instructions executable by the one processor, The instructions are executed by the at least one processor to cause the at least one processor to perform the video summary generation method of any of the first or first aspect of the invention.
本发明第四方面提供了一种非暂态计算机可读存储介质，所述非暂态计算机可读存储介质存储计算机指令，所述计算机指令用于使计算机执行本发明第一方面或第一方面中任一项所述的视频摘要生成方法。A fourth aspect of the invention provides a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the first or first aspect of the invention The video summary generation method according to any one of the preceding claims.
本发明第五方面提供了一种计算机程序产品，所述计算机程序产品包括存储在非暂态计算机可读存储介质上的计算程序，所述计算机程序包括程序指令，当所述程序指令被计算机执行时，使所述计算机执行第一方面或者第一方面的任意一种可选方式中所述的视频摘要生成方法。A fifth aspect of the invention provides a computer program product comprising a computing program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer The computer is configured to perform the video summary generation method described in the first aspect or any one of the optional aspects of the first aspect.
本发明提供的技术方案，具有如下优点：The technical solution provided by the invention has the following advantages:
1.本发明实施例提供的视频摘要生成方法，包括以下步骤：获取待处理视频；将所述待处理视频划分为多个切片；开启与所述多个切片一一对应的多个线程，分别执行对每个所述切片的目标分析任务，得到每个所述切片中运动目标的分析结果；利用所述多个切片的先后顺序对所述运动目标的分析结果进行浓缩计算，以生成所述待处理视频的视频摘要。本发明实施例提供的视频摘要生成方法，通过使用多线程并行处理一个视频的目标分析任务，然后对于分散的分析结果进行综合浓缩，生成视频摘要，以实现在短时间内浏览视频所有内容的目的，从而极大减少了等待时间。The video summary generating method provided by the embodiment of the present invention includes the following steps: acquiring a to-be-processed video; dividing the to-be-processed video into multiple slices; and opening multiple threads corresponding to the multiple slices one by one, respectively Performing a target analysis task for each of the slices, obtaining an analysis result of the moving target in each of the slices; performing a concentration calculation on the analysis result of the moving target by using the sequence of the plurality of slices to generate the A video summary of the pending video. The video summary generation method provided by the embodiment of the present invention uses a multi-thread to process a target analysis task of a video in parallel, and then comprehensively condenses the distributed analysis result to generate a video summary, so as to achieve the purpose of browsing all the contents of the video in a short time. , which greatly reduces the waiting time.
2.本发明实施例提供的视频摘要生成方法，其中，在将所述待处理视频划分为多个切片之后，还包括：记录每个切片的起止位置；对每个所述切片生成唯一的标识，每个切片对应的标识还与其在所述待处理视频中的位置对应。本发明实施例中，通过对每个切片分配唯一的标识，便于在同时对所有切片进行目标分析时，根据该标识对不同的切片中的目标分析结果进行保存，从而避免了存储结果之间的冲突。The video summary generating method provided by the embodiment of the present invention, after dividing the to-be-processed video into a plurality of slices, further comprising: recording a start and end position of each slice; generating a unique identifier for each of the slices The identifier corresponding to each slice also corresponds to its location in the to-be-processed video. In the embodiment of the present invention, by assigning a unique identifier to each slice, it is convenient to save the target analysis result in different slices according to the identifier when performing target analysis on all the slices at the same time, thereby avoiding the storage result between the storage results. conflict.
3.本发明实施例提供的视频摘要生成方法，其中，开启第一线程以执行利用所述多个切片的先后顺序对所述运动目标的分析结果进行浓缩计算，以生成所述待处理视频的视频摘要，同时开启第二线程对所述浓缩计算得到的视频摘要进行展示。本发明实施例通过对切片中的运动目标进行分段浓缩计算，并将浓缩计算和浓缩展示分两个线程并发执行，可以实现边处理边展示的目的，从而可以解决待处理视频中的目标个数较多，需要 耗费大量时间进行目标浓缩展示的等待时间。The video summary generating method provided by the embodiment of the present invention, wherein the first thread is started to perform a condensed calculation on the analysis result of the moving target by using the sequence of the plurality of slices to generate the to-be-processed video. The video summary is simultaneously opened by the second thread to display the video summary obtained by the concentration calculation. In the embodiment of the present invention, the segmentation and concentration calculation of the moving target in the slice is performed, and the concentration calculation and the concentration display are performed in two threads concurrently, thereby achieving the purpose of displaying the edge while processing, thereby solving the target in the to-be-processed video. More, need It takes a lot of time to wait for the target to be concentrated.
4.本发明实施例提供的视频摘要生成方法，其中，所述利用所述多个切片的先后顺序对所述运动目标的分析结果进行浓缩计算，包括：根据所述切片的标识对所述切片进行排序；依次根据排序后的所述切片中运动目标提取视频图像。本发明实施例中，采用同时对所有切片进行目标分析，分析结果可存储在不同的地方，因此，通过切片的标识可以还原各切片在视频中的位置，避免了切片顺序的混乱，从而为从运动目标中提取视频图像提供了条件。The method for generating a video summary according to the embodiment of the present invention, wherein the concentrating calculation of the analysis result of the moving target by using the sequence of the plurality of slices comprises: performing the slice according to the identifier of the slice Sorting; sequentially extracting video images from the moving targets in the sorted slices. In the embodiment of the present invention, the target analysis is performed on all the slices at the same time, and the analysis result can be stored in different places. Therefore, the position of each slice in the video can be restored by the identifier of the slice, thereby avoiding confusion of the slice sequence, thereby The conditions for extracting video images from moving targets are provided.
5.本发明实施例提供的视频摘要生成装置，包括，第一获取单元，用于获取待处理视频；切片划分单元，用于将所述待处理视频划分为多个切片；处理单元，用于开启与所述多个切片一一对应的多个线程，分别执行对每个所述切片的目标分析任务，得到每个所述切片中运动目标的分析结果；浓缩单元，用于利用所述多个切片的先后顺序对所述运动目标的分析结果进行浓缩计算，以生成所述待处理视频的视频摘要。本发明实施例提供的视频摘要生成装置，通过使用多线程并行处理一个视频的目标分析任务，然后对于分散的分析结果进行综合浓缩，生成视频摘要，以实现在短时间内浏览视频所有内容的目的，从而极大减少了等待时间。The video summary generating apparatus provided by the embodiment of the present invention includes: a first acquiring unit, configured to acquire a video to be processed; a slice dividing unit, configured to divide the to-be-processed video into multiple slices; and a processing unit, configured to: Opening a plurality of threads corresponding to the plurality of slices one by one, performing a target analysis task for each of the slices separately, obtaining an analysis result of the moving target in each slice; and a concentrating unit, configured to utilize the plurality of The sequence of the slices is used to perform a condensed calculation on the analysis result of the moving target to generate a video summary of the to-be-processed video. The video summary generating apparatus provided by the embodiment of the present invention processes a target analysis task of a video by using multiple threads in parallel, and then comprehensively condenses the distributed analysis results to generate a video summary, so as to achieve the purpose of browsing all the contents of the video in a short time. , which greatly reduces the waiting time.
附图说明DRAWINGS
通过参考附图会更加清楚的理解本发明的特征和优点，附图是示意性的而不应理解为对本发明进行任何限制，在附图中：The features and advantages of the present invention are more clearly understood from the following description of the drawings.
图1示出了本发明实施例1中视频摘要生成方法的一个具体示意的流程图；FIG. 1 is a flowchart showing a specific schematic diagram of a video digest generating method in Embodiment 1 of the present invention;
图2示出了本发明实施例2中视频摘要生成方法的一个具体示意的流程图；FIG. 2 is a flowchart showing a specific schematic diagram of a video summary generating method in Embodiment 2 of the present invention;
图3示出了本发明实施例3中视频摘要生成方法的一个具体示意的流程图；FIG. 3 is a flowchart showing a specific schematic diagram of a video digest generating method in Embodiment 3 of the present invention;
图4示出了本发明实施例4中视频摘要生成装置的一个具体示意的结 构图；Figure 4 is a diagram showing a specific schematic of the video summary generating apparatus in Embodiment 4 of the present invention. Composition
图5示出了本发明实施例4中处理单元的一个具体示意的结构图；Figure 5 is a block diagram showing a specific schematic diagram of a processing unit in Embodiment 4 of the present invention;
图6示出了本发明实施例4中浓缩单元的一个具体示意的结构图；Figure 6 is a block diagram showing a specific schematic diagram of a concentration unit in Embodiment 4 of the present invention;
图7示出了本发明实施例5中图像处理装置的一个具体示意的结构图。Fig. 7 is a block diagram showing a specific schematic diagram of an image processing apparatus in Embodiment 5 of the present invention.
具体实施方式Detailed ways
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. It is a partial embodiment of the invention, and not all of the embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.
实施例1Example 1
本施例提供一种视频摘要生成方法，用于视频摘要生成装置中。如图1所示，该视频摘要生成方法包括以下步骤：This embodiment provides a video summary generation method for use in a video summary generating device. As shown in FIG. 1, the video summary generating method includes the following steps:
步骤S11，获取待处理视频。In step S11, a video to be processed is obtained.
本实施例中的待处理视频可以是事先存储在视频摘要生成装置中的，或视频摘要生成装置实时从外界获取的视频，也可以是视频摘要生成装置从一段录像中提取的视频段。The video to be processed in this embodiment may be a video that is previously stored in the video summary generating device, or that is acquired by the video summary generating device from the outside world in real time, or may be a video segment extracted by the video summary generating device from a video.
步骤S12，将待处理视频划分为多个切片。In step S12, the video to be processed is divided into a plurality of slices.
本实施例中，通过在视频摘要生成装置中预览待处理视频的总时长，同时可以根据在视频摘要生成装置的处理能力，划分切片的个数。例如，可以根据视频摘要装置中，硬件电路的处理数据的能力，确定最大能够并发处理的数据量，根据该数据量进行待处理视频的划分。In this embodiment, by previewing the total duration of the video to be processed in the video summary generating device, the number of slices can be divided according to the processing capability of the video summary generating device. For example, according to the ability of the hardware circuit to process data in the video summary device, the maximum amount of data that can be concurrently processed can be determined, and the division of the to-be-processed video can be performed according to the amount of data.
其中，所有切片组成一个完整待处理视频，同时在每个切片中可以包括若干视频帧，每一帧中包括若干目标。Among them, all slices constitute a complete pending video, and several video frames may be included in each slice, and each frame includes several targets.
步骤S13，开启与多个切片一一对应的多个线程，分别执行对每个切片的目标分析任务，得到每个切片中运动目标的分析结果。Step S13: Open a plurality of threads corresponding to the plurality of slices one by one, perform target analysis tasks for each slice separately, and obtain an analysis result of the moving target in each slice.
本实施例中，通过在视频摘要生成装置中开启多个线程分别执行对每 个切片的目标分析任务。其中，所开启的线程的个数与待处理视频所划分出的切片的个数相等，即能够达到在视频摘要生成装置中同时进行所有切片的目标分析任务。In this embodiment, each of the threads is separately executed in the video summary generating device. A slice of the target analysis task. The number of threads to be opened is equal to the number of slices divided by the video to be processed, that is, the target analysis task of performing all slices simultaneously in the video summary generating device can be achieved.
其中，每个切片中运动目标的分析结果可以保存在不同的路径下，也可以在同一路径下通过不同的标记加以区分。The analysis results of the moving targets in each slice can be saved in different paths, or can be distinguished by different tags in the same path.
步骤S14，利用多个切片的先后顺序对运动目标的分析结果进行浓缩计算，以生成待处理视频的视频摘要。In step S14, the analysis result of the moving target is concentrated and calculated by using the sequence of the plurality of slices to generate a video summary of the video to be processed.
本实施例中，在上述步骤S13中是对所有切片同时进行目标分析任务，而每个切片中运动目标的分析结果的保存时不一定保持原来的先后顺序关系。因此，在进行运动目标的分析结果浓缩计算时，需要按照切片的先后顺序进行，从而才能够避免切片顺序的混乱。In this embodiment, in the above step S13, all the slices are simultaneously subjected to the target analysis task, and the analysis results of the moving targets in each slice are not necessarily maintained in the original sequential relationship. Therefore, when performing the condensed calculation of the analysis result of the moving target, it is necessary to follow the order of the slices, so that the disorder of the slicing order can be avoided.
本实施例提供的视频摘要生成方法，通过使用多线程并行处理一个视频的目标分析任务，然后对于分散的分析结果进行综合浓缩，生成视频摘要，以实现在短时间内浏览视频所有内容的目的，从而极大减少了等待时间，对于时长较长的视频内容，其效果更佳明显。The video summary generation method provided in this embodiment uses a multi-thread to process a target analysis task of a video in parallel, and then comprehensively condenses the distributed analysis results to generate a video summary, so as to achieve the purpose of browsing all the contents of the video in a short time. This greatly reduces the waiting time, and the effect is better for video content with a longer duration.
实施例2Example 2
本施例提供一种视频摘要生成方法，用于视频摘要生成装置中。如图2所示，该视频摘要生成方法包括以下步骤：This embodiment provides a video summary generation method for use in a video summary generating device. As shown in FIG. 2, the video summary generation method includes the following steps:
步骤S21，获取待处理视频。与实施例1步骤S11相同，在此不再赘述。Step S21: Acquire a video to be processed. The same as step S11 of Embodiment 1, and details are not described herein again.
步骤S22，将待处理视频划分为多个切片。In step S22, the video to be processed is divided into a plurality of slices.
本实施例中，通过在视频摘要生成装置中预览待处理视频的总时长，并根据视频摘要生成装置中硬件电路的处理数据的能力，将待处理视频划分为多个切片文件。In this embodiment, the video to be processed is divided into a plurality of slice files by previewing the total duration of the video to be processed in the video summary generating device and according to the capability of processing data of the hardware circuit in the video summary generating device.
在将待处理视频划分为多个切片之后，记录每个切片的起止位置，用于后续对每个切片进行目标分析任务之前的对切片的解码操作。After dividing the to-be-processed video into a plurality of slices, the start and end positions of each slice are recorded for subsequent decoding operations on the slices before performing the target analysis task for each slice.
此外，对每个切片生成唯一的标识，该标识的大小与切片在待处理视频中的位置对应；即该标识的大小与切片在待处理视频中的时间顺序保持 一致，从而达到通过该标识能够快速找到相邻的前后切片。In addition, a unique identifier is generated for each slice, the size of the identifier corresponding to the position of the slice in the video to be processed; that is, the size of the identifier and the time sequence of the slice in the video to be processed remain Consistent, so that the adjacent front and back slices can be quickly found by the logo.
本实施例中的标识可以采用数值表示，也可以采用字符表示。其中，数值或字符按照切片在待处理视频中的时间进行顺序设置。The identifier in this embodiment may be represented by a numerical value or a character. Among them, the value or character is set in order according to the time of the slice in the video to be processed.
作为本实施例的一种可选实施方式，本实施例中的标识采用数值表示。例如，采用1，2，3，…，依次作为切片的标识。As an optional implementation manner of this embodiment, the identifier in this embodiment is represented by a numerical value. For example, use 1, 2, 3, ..., in turn as the identifier of the slice.
步骤S23，开启与多个切片一一对应的多个线程，分别执行对每个切片的目标分析任务，得到每个切片中运动目标的分析结果。Step S23, opening a plurality of threads corresponding to the plurality of slices one by one, respectively performing target analysis tasks for each slice, and obtaining an analysis result of the moving targets in each slice.
本实施例中，通过在视频摘要生成装置中开启多个线程分别执行对每个切片的目标分析任务。其中，所开启的线程的个数与待处理视频所划分出的切片的个数相等，即能够达到在视频摘要生成装置中同时进行所有切片的目标分析任务。In this embodiment, the target analysis task for each slice is separately performed by turning on a plurality of threads in the video summary generating device. The number of threads to be opened is equal to the number of slices divided by the video to be processed, that is, the target analysis task of performing all slices simultaneously in the video summary generating device can be achieved.
此外，每个切片包括若干目标，因此在本实施例中，通过对每个切片中的每个目标分配唯一的标识，用于进行目标的区分，从而达到在待处理视频中的目标都有唯一的标识。In addition, each slice includes several targets, so in the present embodiment, by assigning a unique identifier to each target in each slice, the target is distinguished, so that the target in the pending video is unique. Logo.
具体地，分别执行对每个切片的目标分析任务，得到每个切片中运动目标的分析结果，包括以下步骤：Specifically, performing a target analysis task for each slice separately, and obtaining an analysis result of the moving target in each slice, including the following steps:
步骤S231，获取每个切片中的运动目标。Step S231, acquiring a moving target in each slice.
本实施例中，首先根据背景建模方法，快速分析每个切片的前面一小段，获取一个稳定的切片的静态背景；然后，根据该静态背景，将切片中不属于该静态背景中的目标提取，即为运动目标。In this embodiment, first, according to the background modeling method, a small segment of each slice is quickly analyzed to obtain a static background of a stable slice; then, according to the static background, the target in the slice that does not belong to the static background is extracted. , that is, the goal of sports.
在对切片进行目标分析过程中，对于切片中的某些场景而言，静态背景并非一成不变，任何运动到静止的目标都会成为背景的一部分，因此，需要不断更新背景，并保存每一个变化的背景图像；以及，每一个背景图像在待处理视频中的目标轨迹信息，具体地，包括背景图像出现消失的时间段、以及在待处理视频中的位置。During the target analysis of the slice, the static background is not static for some scenes in the slice, and any moving to a stationary target will become part of the background. Therefore, it is necessary to constantly update the background and save each changed background. And an image of the target track in each of the to-be-processed videos, specifically, a time period in which the background image disappears, and a position in the video to be processed.
步骤S232，利用多个切片的先后顺序生成每个切片的运动目标的标识。Step S232, generating an identifier of the moving target of each slice by using the sequence of the plurality of slices.
本实施例中，每个切片包括有若干目标，对每个切片中的每个目标分配唯一的标识，用于进行目标的区分，从而达到在待处理视频中的目标都 有唯一的标识。该标识可以用数值标识，也可以用字符标识。In this embodiment, each slice includes a plurality of targets, and each target in each slice is assigned a unique identifier for distinguishing the target, thereby achieving the target in the to-be-processed video. There is a unique identifier. The identifier can be identified by a numerical value or by a character.
其中，可以预先判定出每个切片中出现目标的个数，然后按照切片的先后顺序依次设置每个切片中目标的起始标识，相连切片的起始标识之间的差值大于等于预先判定的每个切片中出现目标的个数。在进行目标标识时，每个切片中目标标识可以按照目标出现的先后，在起始标识的基础上，顺序递增。Wherein, the number of targets appearing in each slice may be determined in advance, and then the start identifier of the target in each slice is sequentially set according to the order of the slices, and the difference between the start identifiers of the connected slices is greater than or equal to the pre-determined The number of targets in each slice. When the target identifier is performed, the target identifier in each slice may be sequentially incremented according to the order in which the target appears, on the basis of the initial identifier.
作为本实施例的一种可选实施方式，待处理视频中各个切片以及每个切片中目标的标识，如下表所示：As an optional implementation manner of this embodiment, each slice in the video to be processed and the identifier of the target in each slice are as follows:
表1 各个切片以及每个切片中目标的标识Table 1 Identification of each slice and the target in each slice
切片(001)Slice (001) 目标(001001)Target (001001) 目标(001002)Target (001002) …... 目标(001***)Target (001***)
切片(002)Slice (002) 目标(002001)Target (002001) 目标(002002)Target (002002) …... 目标(002***)Target (002***)
切片(003)Slice (003) 目标(003001)Target (003001) 目标(003002)Target (003002) …... 目标(003***)Target (003***)
…... …... …... …... …...
切片(***)slice(***) 目标(***001)Target (***001) 目标(***002)Target (***002) …... 目标(******)aims(******)
上表中，每行第一列的代表待处理视频中各个切片以及其对应的标识，每行中其余列代表第一列中表示的切片中的目标以及其对应的标识。例如，切片(001)标识该切片的标识为001，目标(001001)表示切片(001)中的第一个目标，该目标的标识为001001；其余，依次类推。In the above table, the first column of each row represents each slice in the to-be-processed video and its corresponding identifier, and the remaining columns in each row represent the target in the slice represented in the first column and its corresponding identifier. For example, slice (001) identifies the slice as 001, target (001001) represents the first target in slice (001), the target's identity is 001001; the rest, and so on.
本实施例中，每个切片中包括若干视频帧，在进行切片中目标分析过程中，对相邻帧进行相同目标的合并，从而避免了相邻的视频帧中的相同的目标被赋予了不同的标识。In this embodiment, each slice includes a plurality of video frames, and in the process of performing target analysis in the slice, the same target is merged for the adjacent frames, thereby avoiding that the same target in the adjacent video frames is given different Logo.
作为本实施例的一种可选实施方式，将切片的每帧图像中提取出的运动目标，根据该运动目标的位置特征进行前后匹配，将匹配成功的目标赋予相同的标识，未匹配成功的目标的标识保持不变。As an optional implementation manner of the embodiment, the moving target extracted from each frame of the slice is matched before and after according to the position feature of the moving target, and the target with the matching match is given the same identifier, and the matching is successful. The identity of the target remains the same.
本实施例中，在进行了切片中相邻帧的目标合并之后，保存合并处理胡的目标信息，具体地，该信息包括目标轨迹、出现和消失时间等等。In this embodiment, after the target combination of adjacent frames in the slice is performed, the target information of the merged process is saved, and specifically, the information includes the target track, the appearance and disappearance time, and the like.
步骤S233，获取相邻的切片在相邻时刻出现的运动目标。 Step S233: Acquire a moving target that appears in an adjacent slice at an adjacent moment.
本实施例中，在当前切片的目标分析任务完成之后，通过当前切片的标识获取相邻的切片，并分别获取当前切片以及与其相邻的切片在相邻时刻出现的运动目标，同时获取对应于运动目标的标识。In this embodiment, after the target analysis task of the current slice is completed, the adjacent slice is obtained by using the identifier of the current slice, and the current slice and the adjacent target slice are respectively acquired at the adjacent time, and the corresponding target is acquired at the same time. The identification of the moving target.
步骤S234，根据所述运动目标的位置特征判断所述相邻的切片在相邻时刻出现的运动目标是否相同。若判断结果为是，则执行步骤S235；否则，执行步骤S24。Step S234, judging whether the moving targets appearing at the adjacent moments of the adjacent slices are the same according to the position feature of the moving target. If the result of the determination is yes, then step S235 is performed; otherwise, step S24 is performed.
本实施例中，通过跟踪算法计算获取的运动目标的位置特征，如果计算出来的运动目标的位置图片重叠，则表示相邻的切片在相邻时刻出现的运动目标是相同的；若计算出来的运动目标的位置图片不重叠，则表示相邻的切片在相邻时刻出现的运动目标不同。In this embodiment, the positional feature of the obtained moving target is calculated by the tracking algorithm. If the calculated positional images of the moving target overlap, it means that the adjacent moving objects appear at the same time are the same; if the calculated If the position pictures of the moving objects do not overlap, it means that the adjacent slices have different moving targets at the adjacent moments.
步骤S235，将运动目标对应的标识合并成一个。In step S235, the identifiers corresponding to the moving targets are merged into one.
本实施例中，在判断出相邻的切片在相邻时刻出现的运动目标是相同的，则将相同的目标赋予相同的标识。进行相同目标的合并，是为了达到下述目的：由于在对待处理视频进行切片划分时，并没有考虑切分点的内容，会导致在切分点将真实视频中的同一个目标(即人物或者车辆)分配在两个切片中，那么在每个切片中同一个目标会拥有各自不同的标识；如果不进行相同目标的合并，那么在后续的目标浓缩计算时，就会出现同一个目标在不同时间出现。In this embodiment, when it is determined that the moving targets appearing at the adjacent moments of the adjacent slices are the same, the same target is given the same identifier. The merging of the same target is for the purpose of not considering the content of the puncturing point when slicing the video to be processed, which will result in the same target in the real video at the segmentation point (ie, the character or The vehicle is allocated in two slices, then the same target will have different identifiers in each slice; if the same target is not merged, then the same target will be different in the subsequent target concentration calculation. Time appears.
因此，在每个切片的目标分析任务完成之后，根据切片的标识获取相邻的切片，并分析相邻的切片在相邻时刻出现的运动目标是否相同，若相同，则赋予同一个标识，从而达到同一个目标在同一个时间出现的目的。Therefore, after the target analysis task of each slice is completed, the adjacent slices are acquired according to the identifier of the slice, and whether the moving targets of the adjacent slices appearing at the adjacent time are the same, and if they are the same, the same identifier is given, thereby Achieve the same goal at the same time.
步骤S24，利用多个切片的先后顺序对运动目标的分析结果进行浓缩计算，以生成待处理视频的视频摘要。与实施例1步骤S14相同，在此不再赘述。In step S24, the analysis result of the moving target is concentrated and calculated by using the sequence of the plurality of slices to generate a video summary of the video to be processed. The same as step S14 of Embodiment 1, and details are not described herein again.
本发明实施例中，通过对每个切片分配唯一的标识，便于在同时对所有切片进行目标分析时，根据该标识对不同的切片中的目标分析结果进行保存，从而避免了存储结果之间的冲突；此外，通过同一切片的相邻帧以及对相邻切片在相邻时刻出现的相同的运动目标进行合并，从而避免了生 成视频摘要之后，相同的目标出现在不同的时刻。In the embodiment of the present invention, by assigning a unique identifier to each slice, it is convenient to save the target analysis result in different slices according to the identifier when performing target analysis on all the slices at the same time, thereby avoiding the storage result between the storage results. Collision; in addition, avoiding the birth by merging adjacent frames of the same slice and the same moving targets that appear adjacent to each other at adjacent moments After the video summary, the same goal appears at different times.
实施例3Example 3
本施例提供一种视频摘要生成方法，用于视频摘要生成装置中。如图3所示，该视频摘要生成方法包括以下步骤：This embodiment provides a video summary generation method for use in a video summary generating device. As shown in FIG. 3, the video summary generating method includes the following steps:
步骤S31，获取待处理视频。与实施例2步骤S21相同，在此不再赘述。Step S31, acquiring a video to be processed. The same as step S21 of Embodiment 2, and details are not described herein again.
步骤S32，将待处理视频划分为多个切片。与实施例2步骤S22相同，在此不再赘述。In step S32, the video to be processed is divided into a plurality of slices. The same as step S22 of Embodiment 2, and details are not described herein again.
步骤S33，开启与多个切片一一对应的多个线程，分别执行对每个切片的目标分析任务，得到每个切片中运动目标的分析结果。与实施例2步骤S23相同，在此不再赘述。Step S33, opening a plurality of threads corresponding to the plurality of slices one by one, respectively performing target analysis tasks for each slice, and obtaining an analysis result of the moving targets in each slice. The same as step S23 of Embodiment 2, and details are not described herein again.
步骤S34，利用多个切片的先后顺序对运动目标的分析结果进行浓缩计算，以生成待处理视频的视频摘要。In step S34, the analysis result of the moving target is concentrated and calculated by using the sequence of the plurality of slices to generate a video summary of the video to be processed.
本实施例中，通过视频摘要生成装置开启第一线程以执行利用多个切片的先后顺序对运动目标的分析结果进行浓缩计算，以生成待处理视频的视频摘要，同时开启第二线程对浓缩计算得到的视频摘要进行展示。In this embodiment, the first thread is started by the video summary generating device to perform a condensed calculation on the analysis result of the moving target by using the sequence of the plurality of slices to generate a video summary of the to-be-processed video, and simultaneously open the second thread to the concentrated calculation. The resulting video summary is displayed.
具体地，本实施例中的利用多个切片的先后顺序对运动目标的分析结果进行浓缩计算，包括以下步骤：Specifically, in the embodiment, the analysis result of the moving target is concentrated and calculated by using the sequence of the plurality of slices, and the following steps are included:
步骤S341，根据切片的标识对切片进行排序。In step S341, the slices are sorted according to the identifier of the slice.
本实施例中，视频摘要生成装置从各个存储路径中提取各个切片的目标分析结果，并开启第一线程，按照切片的标识对提取的切片进行前后的排序。In this embodiment, the video summary generating device extracts the target analysis result of each slice from each storage path, and starts the first thread, and sorts the extracted slice according to the identifier of the slice.
步骤S342，依次根据排序后的切片中运动目标提取视频图像。Step S342, sequentially extracting video images according to the moving targets in the sorted slices.
本实施例中，从前往后依次读取各个切片中的运动目标，当读取到一定的运动目标个数时，先停止读取操作；对这些已经读取的运动目标进行排布计算，然后开启第二线程，对排布计算结果进行展示。通过重复读取，计算，展示的步骤，直到所有目标全都处理完毕。 In this embodiment, the moving targets in each slice are sequentially read from the back to the rear, and when a certain number of moving targets is read, the reading operation is stopped first; the moving targets that have been read are arranged, and then Start the second thread and display the results of the layout calculation. Repeat the process of reading, calculating, and displaying until all targets have been processed.
本发明实施例，通过对切片中的运动目标进行分段浓缩计算，并将浓缩计算和浓缩展示分两个线程并发执行，可以实现边处理边展示的目的，从而可以解决待处理视频中的目标个数较多，需要耗费大量时间进行目标浓缩展示的等待时间。In the embodiment of the present invention, by performing segmentation and concentration calculation on the moving target in the slice, and performing the concentration calculation and the concentrated display in two threads concurrently, the purpose of the edge processing and the display can be realized, thereby solving the target in the to-be-processed video. The number is large, and it takes a lot of time to wait for the target to be concentrated.
实施例4Example 4
本施例提供一种视频摘要生成装置，用于执行本发明实施例1至实施例3中的视频摘要生成方法。如图4所示，该视频摘要生成装置包括：The present embodiment provides a video summary generating apparatus for performing the video summary generating method in Embodiments 1 to 3 of the present invention. As shown in FIG. 4, the video summary generating apparatus includes:
第一获取单元41，用于获取待处理视频。The first obtaining unit 41 is configured to acquire a video to be processed.
切片划分单元42，用于将待处理视频划分为多个切片。The slice dividing unit 42 is configured to divide the to-be-processed video into multiple slices.
处理单元43，用于开启与多个切片一一对应的多个线程，分别执行对每个切片的目标分析任务，得到每个切片中运动目标的分析结果。The processing unit 43 is configured to open a plurality of threads corresponding to the plurality of slices one by one, perform target analysis tasks for each slice separately, and obtain an analysis result of the moving target in each slice.
浓缩单元44，用于利用多个切片的先后顺序对运动目标的分析结果进行浓缩计算，以生成待处理视频的视频摘要。The concentrating unit 44 is configured to perform condensed calculation on the analysis result of the moving target by using the sequence of the plurality of slices to generate a video summary of the video to be processed.
本发明实施例提供的视频摘要生成装置，通过使用多线程并行处理一个视频的目标分析任务，然后对于分散的分析结果进行综合浓缩，生成视频摘要，以实现在短时间内浏览视频所有内容的目的，从而极大减少了等待时间。The video summary generating apparatus provided by the embodiment of the present invention processes a target analysis task of a video by using multiple threads in parallel, and then comprehensively condenses the distributed analysis results to generate a video summary, so as to achieve the purpose of browsing all the contents of the video in a short time. , which greatly reduces the waiting time.
作为本实施例的一种可选实施方式，其中，如图5所示，处理单元43，包括：As an optional implementation manner of this embodiment, as shown in FIG. 5, the processing unit 43 includes:
第一获取子单元431，用于获取每个切片中的运动目标。The first obtaining subunit 431 is configured to acquire a moving target in each slice.
标记子单元432，用于利用多个切片的先后顺序生成每个切片的运动目标的标识。The marking sub-unit 432 is configured to generate an identification of the moving target of each slice by using the sequence of the plurality of slices.
第二获取子单元433，用于获取相邻的切片在相邻时刻出现的运动目标。The second obtaining sub-unit 433 is configured to acquire a moving target that appears in an adjacent slice at an adjacent moment.
判断子单元434，用于根据运动目标的位置特征判断相邻的切片在相邻时刻出现的运动目标是否相同。The determining sub-unit 434 is configured to determine, according to the position feature of the moving target, whether the moving targets of the adjacent slices appearing at the adjacent moments are the same.
合并子单元435，用于当判断相邻的切片在相邻时刻出现的运动目标相 同时，将运动目标对应的标识合并成一个。The merging sub-unit 435 is configured to determine the moving target phase of the adjacent slice at the adjacent moment At the same time, the logos corresponding to the moving targets are merged into one.
作为本实施例的一种可选实施方式，其中，如图6所示，浓缩单元44，包括：As an optional implementation manner of this embodiment, as shown in FIG. 6, the concentrating unit 44 includes:
排序子单元441，用于根据切片的标识对切片进行排序。The sorting sub-unit 441 is configured to sort the slices according to the identifier of the slice.
提取子单元442，用于依次根据排序后的切片中运动目标提取视频图像。The extracting sub-unit 442 is configured to sequentially extract a video image according to the moving target in the sorted slice.
实施例5Example 5
图7是本发明实施例提供的图像处理装置的硬件结构示意图，如图7所示，该装置包括一个或多个处理器51以及存储器52，图7中以一个处理器51为例。FIG. 7 is a schematic diagram showing the hardware structure of an image processing apparatus according to an embodiment of the present invention. As shown in FIG. 7, the apparatus includes one or more processors 51 and a memory 52. One processor 51 is taken as an example in FIG.
该图像处理装置还可以包括：图像显示器(未示出)，用于展示视频摘要。处理器51、存储器52和图像显示器可以通过总线或者其他方式连接，图7中以通过总线连接为例。The image processing apparatus may further include an image display (not shown) for displaying the video summary. The processor 51, the memory 52, and the image display may be connected by a bus or other means, as exemplified by a bus connection in FIG.
处理器51可以为中央处理器(Central Processing Unit，CPU)。处理器51还可以为其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等芯片，或者上述各类芯片的组合。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。The processor 51 can be a Central Processing Unit (CPU). The processor 51 can also be another general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or Other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., or a combination of the above various types of chips. The general purpose processor may be a microprocessor or the processor or any conventional processor or the like.
存储器52作为一种非暂态计算机可读存储介质，可用于存储非暂态软件程序、非暂态计算机可执行程序以及模块，如本发明实施例中的视频摘要生成方法对应的程序指令/模块。处理器51通过运行存储在存储器52中的非暂态软件程序、指令以及模块，从而执行服务器的各种功能应用以及数据处理，即实现上述实施例中的视频摘要生成方法。The memory 52 is used as a non-transitory computer readable storage medium, and can be used for storing a non-transitory software program, a non-transitory computer executable program, and a module, such as a program instruction/module corresponding to the video summary generation method in the embodiment of the present invention. . The processor 51 executes various function applications and data processing of the server by running non-transitory software programs, instructions, and modules stored in the memory 52, that is, implementing the video summary generating method in the above embodiments.
存储器52可以包括存储程序区和存储数据区，其中，存储程序区可存储操作***、至少一个功能所需要的应用程序；存储数据区可存储根据视频摘要生成装置的使用所创建的数据等。此外，存储器52可以包括高速随 机存取存储器，还可以包括非暂态存储器，例如至少一个磁盘存储器件、闪存器件、或其他非暂态固态存储器件。在一些实施例中，存储器52可选包括相对于处理器51远程设置的存储器，这些远程存储器可以通过网络连接至视频摘要生成装置。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。The memory 52 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function; the storage data area may store data created according to use of the video summary generating device, and the like. In addition, the memory 52 can include a high speed The machine access memory may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 52 can optionally include memory remotely located relative to processor 51, which can be connected to the video summary generating device over a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
所述一个或者多个模块存储在所述存储器52中，当被所述一个或者多个处理器51执行时，执行实施例1至实施例3中任一项所述的视频摘要生成方法。The one or more modules are stored in the memory 52, and when executed by the one or more processors 51, the video summary generating method described in any one of Embodiments 1 to 3 is performed.
上述产品可执行本发明实施例所提供的方法，具备执行方法相应的功能模块和有益效果。未在本实施例中详尽描述的技术细节，具体可参见如图1所示的实施例中的相关描述。The above product can perform the method provided by the embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. For details of the technical details that are not described in detail in this embodiment, refer to the related description in the embodiment shown in FIG.
实施例6Example 6
本发明实施例还提供了一种非暂态计算机存储介质，所述计算机存储介质存储有计算机可执行指令，该计算机可执行指令可执行实施例1至实施例3中任一项所述的视频摘要生成方法。其中，所述存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)、随机存储记忆体(Random Access Memory，RAM)、快闪存储器(Flash Memory)、硬盘(Hard Disk Drive，缩写：HDD)或固态硬盘(Solid-State Drive，SSD)等；所述存储介质还可以包括上述种类的存储器的组合。The embodiment of the present invention further provides a non-transitory computer storage medium storing computer executable instructions, which can execute the video according to any one of Embodiments 1 to 3. Summary generation method. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk (Hard). Disk Drive, abbreviated as: HDD) or Solid-State Drive (SSD), etc.; the storage medium may also include a combination of the above types of memories.
本领域技术人员可以理解，实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储记忆体(ROM)或随机存储记忆体(RAM)等。It can be understood by those skilled in the art that all or part of the processes in the foregoing embodiments may be implemented by a computer program to instruct related hardware, and the program may be stored in a computer readable storage medium. When executed, the flow of an embodiment of the methods as described above may be included. The storage medium may be a magnetic disk, an optical disk, a read only memory (ROM), or a random access memory (RAM).
虽然结合附图描述了本发明的实施例，但是本领域技术人员可以在不脱离本发明的精神和范围的情况下作出各种修改和变型，这样的修改和变型均落入由所附权利要求所限定的范围之内。 While the invention has been described with respect to the embodiments of the embodiments of the embodiments of the invention Within the limits defined.

Claims (10)

  1. 一种视频摘要生成方法，其特征在于，包括以下步骤：A video summary generation method, comprising the steps of:
    获取待处理视频；Get the pending video;
    将所述待处理视频划分为多个切片；Dividing the to-be-processed video into multiple slices;
    开启与所述多个切片一一对应的多个线程，分别执行对每个所述切片的目标分析任务，得到每个所述切片中运动目标的分析结果；Opening a plurality of threads corresponding to the plurality of slices one by one, respectively performing target analysis tasks for each of the slices, and obtaining an analysis result of the moving targets in each of the slices;
    利用所述多个切片的先后顺序对所述运动目标的分析结果进行浓缩计算，以生成所述待处理视频的视频摘要。Performing a concentration calculation on the analysis result of the moving target by using the sequence of the plurality of slices to generate a video summary of the to-be-processed video.
  2. 根据权利要求1所述的视频摘要生成方法，其特征在于，在将所述待处理视频划分为多个切片之后，还包括：The video digest generating method according to claim 1, further comprising: after dividing the to-be-processed video into a plurality of slices, further comprising:
    记录每个切片的起止位置；Record the start and end position of each slice;
    对每个所述切片生成唯一的标识，每个切片对应的标识还与其在所述待处理视频中的位置对应。A unique identifier is generated for each of the slices, and the identifier corresponding to each slice also corresponds to its location in the to-be-processed video.
  3. 根据权利要求1所述的视频摘要生成方法，其特征在于，The video summary generating method according to claim 1, wherein
    所述分别执行对每个所述切片的目标分析任务，得到每个所述切片中运动目标的分析结果，包括：Performing a target analysis task for each of the slices separately, and obtaining an analysis result of the moving target in each of the slices, including:
    获取每个所述切片中的运动目标；Obtaining a moving target in each of the slices;
    利用所述多个切片的先后顺序生成每个所述切片的运动目标的标识；Generating an identification of a moving target of each of the slices using a sequence of the plurality of slices;
    获取相邻的切片在相邻时刻出现的运动目标；Obtaining moving targets of adjacent slices that appear at adjacent moments;
    根据所述运动目标的位置特征判断所述相邻的切片在相邻时刻出现的运动目标是否相同；Determining, according to the position feature of the moving target, whether the moving targets of the adjacent slices appearing at the adjacent moments are the same;
    当判断所述相邻的切片在相邻时刻出现的运动目标相同时，将所述运动目标对应的标识合并成一个。When it is determined that the moving targets of the adjacent slices appearing at the same time are the same, the identifiers corresponding to the moving targets are merged into one.
  4. 根据权利要求1所述的视频摘要生成方法，其特征在于，The video summary generating method according to claim 1, wherein
    开启第一线程以执行利用所述多个切片的先后顺序对所述运动目标的分析结果进行浓缩计算，以生成所述待处理视频的视频摘要，同时开启第二线程对所述浓缩计算得到的视频摘要进行展示。Opening a first thread to perform a condensed calculation on the analysis result of the moving target by using a sequence of the plurality of slices to generate a video summary of the to-be-processed video, and simultaneously opening a second thread to calculate the concentration The video summary is displayed.
  5. 根据权利要求4所述的视频摘要生成方法，其特征在于，所述利用 所述多个切片的先后顺序对所述运动目标的分析结果进行浓缩计算，包括：The video summary generating method according to claim 4, wherein said utilizing The sequence of the plurality of slices is used to perform a concentration calculation on the analysis result of the moving target, including:
    根据所述切片的标识对所述切片进行排序；Sorting the slices according to the identifier of the slice;
    依次根据排序后的所述切片中运动目标提取视频图像。The video image is extracted in turn based on the sorted moving objects in the slice.
  6. 一种视频摘要生成装置，其特征在于，包括：A video summary generating device, comprising:
    第一获取单元，用于获取待处理视频；a first acquiring unit, configured to acquire a video to be processed;
    切片划分单元，用于将所述待处理视频划分为多个切片；a slice dividing unit, configured to divide the to-be-processed video into multiple slices;
    处理单元，用于开启与所述多个切片一一对应的多个线程，分别执行对每个所述切片的目标分析任务，得到每个所述切片中运动目标的分析结果；a processing unit, configured to open a plurality of threads corresponding to the plurality of slices one by one, perform target analysis tasks for each of the slices separately, and obtain an analysis result of the moving targets in each of the slices;
    浓缩单元，用于利用所述多个切片的先后顺序对所述运动目标的分析结果进行浓缩计算，以生成所述待处理视频的视频摘要。And a concentrating unit, configured to perform a condensed calculation on the analysis result of the moving target by using a sequence of the plurality of slices to generate a video summary of the to-be-processed video.
  7. 根据权利要求6所述的视频摘要生成装置，其特征在于，所述视频摘要生成装置，还包括：The video summary generating apparatus according to claim 6, wherein the video summary generating apparatus further comprises:
    记录单元，用于记录每个切片的起止位置；a recording unit for recording the starting and ending position of each slice;
    标记单元，用于对每个所述切片生成唯一的标识，每个切片对应的标识还与其在所述待处理视频中的位置对应。And a marking unit, configured to generate a unique identifier for each of the slices, and the identifier corresponding to each slice also corresponds to a location in the to-be-processed video.
  8. 根据权利要求6所述的视频摘要生成方法，其特征在于，所述处理单元，包括：The video summary generating method according to claim 6, wherein the processing unit comprises:
    第一获取子单元，用于获取每个所述切片中的运动目标；a first obtaining subunit, configured to acquire a moving target in each of the slices;
    标记子单元，用于利用所述多个切片的先后顺序生成每个所述切片的运动目标的标识；a marking subunit, configured to generate an identifier of a moving target of each of the slices by using a sequence of the plurality of slices;
    第二获取子单元，用于获取相邻的切片在相邻时刻出现的运动目标；a second acquiring subunit, configured to acquire a moving target that appears in an adjacent slice at an adjacent moment;
    判断子单元，用于根据所述运动目标的位置特征判断所述相邻的切片在相邻时刻出现的运动目标是否相同；a determining subunit, configured to determine, according to the location feature of the moving target, whether the moving targets of the adjacent slices appearing at the adjacent moments are the same;
    合并子单元，用于当判断所述相邻的切片在相邻时刻出现的运动目标相同时，将所述运动目标对应的标识合并成一个。The merging subunit is configured to combine the identifiers corresponding to the moving objects into one when it is determined that the moving targets of the adjacent slices appearing at the same time are the same.
  9. 根据权利要求6所述的视频摘要生成装置，其特征在于，所述浓缩单元，包括： The video summary generating apparatus according to claim 6, wherein the concentrating unit comprises:
    排序子单元，用于根据所述切片的标识对所述切片进行排序；a sorting subunit, configured to sort the slices according to the identifier of the slice;
    提取子单元，用于依次根据排序后的所述切片中运动目标提取视频图像。And extracting a subunit, configured to sequentially extract a video image according to the sorted moving target in the slice.
  10. 一种图像处理装置，其特征在于，包括至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器执行权利要求1至5中任一项所述的视频摘要生成方法。 An image processing apparatus comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the one processor, the instructions Executed by the at least one processor to cause the at least one processor to perform the video summary generation method of any one of claims 1 to 5.
PCT/CN2017/117167 2017-08-31 2017-12-19 Video abstract generating method and device WO2019041661A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710775389.0 2017-08-31
CN201710775389.0A CN107360476B (en) 2017-08-31 2017-08-31 Video abstraction generating method and device

Publications (1)

Publication Number Publication Date
WO2019041661A1 true WO2019041661A1 (en) 2019-03-07

Family

ID=60289779

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/117167 WO2019041661A1 (en) 2017-08-31 2017-12-19 Video abstract generating method and device

Country Status (2)

Country Link
CN (1) CN107360476B (en)
WO (1) WO2019041661A1 (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107360476B (en) * 2017-08-31 2019-09-20 苏州科达科技股份有限公司 Video abstraction generating method and device
CN109862396A (en) * 2019-01-11 2019-06-07 苏州科达科技股份有限公司 A kind of analysis method of video code flow, electronic equipment and readable storage medium storing program for executing
CN110087144A (en) * 2019-05-15 2019-08-02 深圳市商汤科技有限公司 Video file processing method, device, electronic equipment and computer storage medium
CN110336824B (en) * 2019-07-10 2021-12-03 中国民航信息网络股份有限公司 Weak password detection method, detection control equipment and weak password detection system
CN118075574A (en) * 2022-11-22 2024-05-24 荣耀终端有限公司 Strategy determination method for generating video and electronic equipment

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100289959A1 (en) * 2007-11-22 2010-11-18 Koninklijke Philips Electronics N.V. Method of generating a video summary
CN203799368U (en) * 2013-12-13 2014-08-27 广州必视谷信息技术有限公司 Distributed video analyzing system
CN104394353A (en) * 2014-10-14 2015-03-04 浙江宇视科技有限公司 Video compression method and device
CN105357594A (en) * 2015-11-19 2016-02-24 南京云创大数据科技股份有限公司 Massive video abstraction generation method based on cluster and H264 video concentration algorithm
CN106534344A (en) * 2016-12-07 2017-03-22 高新兴科技集团股份有限公司 Cloud platform video processing system and application method thereof
CN107360476A (en) * 2017-08-31 2017-11-17 苏州科达科技股份有限公司 Video abstraction generating method and device

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102740106B (en) * 2011-03-31 2014-12-03 富士通株式会社 Method and device for detecting movement type of camera in video
CN103092963A (en) * 2013-01-21 2013-05-08 信帧电子技术（北京）有限公司 Video abstract generating method and device
CN103227963A (en) * 2013-03-20 2013-07-31 西交利物浦大学 Static surveillance video abstraction method based on video moving target detection and tracing
CN104639994B (en) * 2013-11-08 2018-10-09 杭州海康威视数字技术股份有限公司 Method, system and the network storage equipment of video frequency abstract are generated based on moving target
CN105592369B (en) * 2014-10-21 2019-11-05 ***通信集团公司 A kind of video frequency abstract processing method and processing device
CN106713964A (en) * 2016-12-05 2017-05-24 乐视控股（北京）有限公司 Method of generating video abstract viewpoint graph and apparatus thereof

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100289959A1 (en) * 2007-11-22 2010-11-18 Koninklijke Philips Electronics N.V. Method of generating a video summary
CN203799368U (en) * 2013-12-13 2014-08-27 广州必视谷信息技术有限公司 Distributed video analyzing system
CN104394353A (en) * 2014-10-14 2015-03-04 浙江宇视科技有限公司 Video compression method and device
CN105357594A (en) * 2015-11-19 2016-02-24 南京云创大数据科技股份有限公司 Massive video abstraction generation method based on cluster and H264 video concentration algorithm
CN106534344A (en) * 2016-12-07 2017-03-22 高新兴科技集团股份有限公司 Cloud platform video processing system and application method thereof
CN107360476A (en) * 2017-08-31 2017-11-17 苏州科达科技股份有限公司 Video abstraction generating method and device

Also Published As

Publication number Publication date
CN107360476B (en) 2019-09-20
CN107360476A (en) 2017-11-17

Similar Documents

Publication Publication Date Title
WO2019041661A1 (en) Video abstract generating method and device
US11783585B2 (en) Detection of demarcating segments in video
CN106162223B (en) News video segmentation method and device
KR102187208B1 (en) Method and apparatus for highlight extraction of sports game
JP2020536455A5 (en)
US9098585B2 (en) Clustering multimedia search
US10853433B2 (en) Method and device for generating briefing
EP3473016B1 (en) Method and system for automatically producing video highlights
Saba et al. Analysis of vision based systems to detect real time goal events in soccer videos
US10104345B2 (en) Data-enhanced video viewing system and methods for computer vision processing
US20170272800A1 (en) Method for bullet screen pushing and electronic device
US11853357B2 (en) Method and system for dynamically analyzing, modifying, and distributing digital images and video
CN111586474A (en) Live video processing method and device
KR102248456B1 (en) Method and apparatus for highlight extraction of sports game
CN113613065A (en) Video editing method and device, electronic equipment and storage medium
US20200005048A1 (en) Transformation of video streams
Midoglu et al. MMSys' 22 Grand Challenge on AI-based Video Production for Soccer
US20160027050A1 (en) Method of providing advertisement service using cloud album
CN113992973A (en) Video abstract generation method and device, electronic equipment and storage medium
Midoglu et al. Ai-based sports highlight generation for social media
US20170040040A1 (en) Video information processing system
KR20200013145A (en) Apparatus of extracting highlight and method thereof
US20240062545A1 (en) Information processing device, information processing method, and recording medium
CN113409074A (en) Data processing method and device, electronic equipment and storage medium
US11979620B2 (en) System, non-transitory computer readable storage medium and method for automatically placing virtual advertisements in sports videos

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17923242

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17923242

Country of ref document: EP

Kind code of ref document: A1