US11894021B2 - Data processing method and system, storage medium, and computing device - Google Patents
Data processing method and system, storage medium, and computing device Download PDFInfo
- Publication number
- US11894021B2 US11894021B2 US17/202,751 US202117202751A US11894021B2 US 11894021 B2 US11894021 B2 US 11894021B2 US 202117202751 A US202117202751 A US 202117202751A US 11894021 B2 US11894021 B2 US 11894021B2
- Authority
- US
- United States
- Prior art keywords
- target
- video data
- image
- target object
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
- 238000003672 processing method Methods 0.000 title claims description 41
- 238000012545 processing Methods 0.000 claims abstract description 168
- 230000002194 synthesizing effect Effects 0.000 claims abstract description 82
- 238000000034 method Methods 0.000 claims abstract description 73
- 230000011218 segmentation Effects 0.000 claims description 133
- 230000015654 memory Effects 0.000 claims description 34
- 230000002452 interceptive effect Effects 0.000 claims description 5
- 230000003247 decreasing effect Effects 0.000 claims 3
- 238000012804 iterative process Methods 0.000 claims 3
- 238000001514 detection method Methods 0.000 description 199
- 230000015572 biosynthetic process Effects 0.000 description 32
- 238000003786 synthesis reaction Methods 0.000 description 32
- 230000008569 process Effects 0.000 description 26
- 238000010586 diagram Methods 0.000 description 12
- 230000003993 interaction Effects 0.000 description 12
- 230000005540 biological transmission Effects 0.000 description 10
- 230000000717 retained effect Effects 0.000 description 8
- 238000009833 condensation Methods 0.000 description 6
- 230000005494 condensation Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000006835 compression Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000003203 everyday effect Effects 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
- G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/48—Matching video sequences
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/49—Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G11B27/34—Indicating arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/272—Means for inserting a foreground image in a background image, i.e. inlay, outlay
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Definitions
- This specification relates to the field of processing video data, and in particular, to a data processing method and system, a storage medium, and a computing device for processing video data.
- Embodiments of this specification provide a data processing method and system, a storage medium, and a computing device, to at least resolve a technical problem that a target object cannot be identified quickly due to a large amount of stored video data and a large amount of redundant information in the related art.
- a data processing method including: displaying original video data and object information of a target object; and displaying target video data, where the target video data is obtained by synthesizing a target image sequence of the target object and a background image in the original video data, the target image sequence is obtained by processing the original video data based on the object information of the target object, and the background image is included in the original video data and does not include a detection object.
- a data processing method including: displaying object information of a target object; and displaying target video data, where the target video data is obtained by synthesizing a target image sequence of the target object and a background image in original video data, the target image sequence is obtained by matching the object information of the target object with a pre-stored image sequence of at least one detection object, and the background image is included in the original video data and does not include a detection object.
- a data processing method including: obtaining original video data and object information of a target object; processing the original video data based on the object information of the target object, to obtain a target image sequence of the target object; processing the original video data, to obtain a background image, where the background image is included in the original video data and does not include a detection object; and synthesizing the target image sequence and the background image, to obtain target video data.
- a data processing method including: obtaining object information of a target object; matching the object information of the target object with a pre-stored image sequence of at least one detection object, to obtain a target image sequence of the target object, where the image sequence of the at least one detection object is obtained by processing original video data; and synthesizing the target image sequence and a background image, to obtain target video data, where the background image is included in the original video data and does not include a detection object.
- a data processing method including: obtaining original video data; receiving a target object inputted by a client; obtaining a target image sequence corresponding to the target object from the original video data; and displaying the target image sequence in a fade-in and fade-out manner.
- a storage medium is further provided.
- the storage medium includes a stored program, where the program, when run by a processor, controls a device at which the storage medium is located to perform any of the foregoing data processing methods.
- a computing device including a memory and a processor, where the processor is configured to run a program stored in the memory, and the program, when run, performs any of the foregoing data processing methods.
- a data processing system including: a processor; and a memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: obtaining original video data and object information of a target object; processing the original video data based on the object information of the target object, to obtain a target image sequence of the target object; processing the original video data, to obtain a background image, where the background image is included in the original video data and does not include a detection object; and synthesizing the target image sequence and the background image, to obtain target video data.
- a data processing method includes: obtaining video data; receiving a target object; processing the video data to obtain one or more first images of the video data that contain an image of the target object and one or more second images of the video data that do not contain the image of the target object; discarding the one or more second images from the video data; generating a background image based on the one or more second images; synthesizing the one or more first images of the video data with the background image to generate one or more synthesized images; obtaining a target image sequence with the one or more synthesized images based on a time order of the one or more first images in the video data; and displaying each of the one or more synthesized images in the target image sequence in a fade-in and fade-out manner.
- the processing the video data to obtain one or more first images of the video data that contain an image of the target object comprises: obtaining a third image of the video data that includes the image of the target object; carving out the image of the target object from the third image to generate a segmentation mask; and using the segmentation mask to obtain the one or more first images of the video data that contain the image of the target object.
- the method further includes: periodically obtaining a segmentation mask by processing the video data to obtain a plurality of segmentation masks; and using a tracing algorithm to obtain identification information of the segmentation masks. Segmentation masks corresponding to same identification information are masks for the same object in the video.
- the processing the video data to obtain one or more first images of the video data that contain an image of the target object comprises: processing the video data based on a target attribute of the target object and a target time period, by: determining attribute information of an object in the one or more first images; matching the target attribute of the target object with the attribute information of the object in the one or more first images to determine an image sequence of the target object; and obtaining an image sequence in the target time period from the image sequence of the target object to obtain the target image sequence.
- each image of the video data is given a time stamp
- the synthesizing the one or more first images of the video data with the background image to generate one or more synthesized images comprises: synthesizing a plurality of first images onto the background image to generate one synthesized image.
- the synthesizing a plurality of first images onto the background image to generate one synthesized image comprises: including time stamps of the plurality of first images in the background image to generate the one synthesized image.
- the target object is a first target object
- the method further comprises: receiving a second target object different from the first target object; and processing the video data to obtain one or more fourth images of the video data that contain an image of the second target object.
- the one or more synthesized images are obtained by: synthesizing one of the one or more first images and one of the one or more fourth images onto the background image.
- the synthesizing one of the one or more first images and one of the one or more fourth images onto the background image comprises: synthesizing one of the one or more first images and one of the one or more fourth images onto the background image such that the image of the first target object does not overlap the image of the second target object.
- an apparatus includes one or more processors and a memory storing instructions executable by the one or more processors to cause the one or more processors to perform operations comprising: obtaining video data; receiving a target object; processing the video data to obtain one or more first images of the video data that contain an image of the target object and one or more second images of the video data that do not contain the image of the target object; discarding the one or more second images from the video data; generating a background image based on the one or more second images; synthesizing the one or more first images of the video data with the background image to generate one or more synthesized images; obtaining a target image sequence with the one or more synthesized images based on a time order of the one or more first images in the video data; and displaying each of the one or more synthesized images in the target image sequence in a fade-in and fade-out manner.
- one or more non-transitory computer-readable storage media are provided.
- the one or more non-transitory computer-readable storage media are configured with instructions executable by one or more processors to cause the one or more processors to perform operations comprising: obtaining video data; receiving a target object; processing the video data to obtain one or more first images of the video data that contain an image of the target object and one or more second images of the video data that do not contain the image of the target object; discarding the one or more second images from the video data; generating a background image based on the one or more second images; synthesizing the one or more first images of the video data with the background image to generate one or more synthesized images; obtaining a target image sequence with the one or more synthesized images based on a time order of the one or more first images in the video data; and displaying each of the one or more synthesized images in the target image sequence in a fade-in and fade-out manner.
- a target image sequence of the target object may be obtained by processing the original video data based on the object information of the target object; a background image is obtained by processing the original video data; and target video data is obtained by synthesizing the target image sequence and the background image, thereby implementing video condensation.
- FIG. 1 is a structural block diagram of hardware of a computer terminal configured to implement a data processing method, according to an embodiment of this specification;
- FIG. 2 is a flowchart of a first data processing method, according to an embodiment of this specification
- FIG. 3 is a schematic diagram of example original video data, according to an embodiment of this specification.
- FIG. 4 is a schematic diagram of example target video data, according to an embodiment of this specification.
- FIG. 5 is a schematic diagram of an example interaction interface, according to an embodiment of this specification.
- FIG. 6 A is a schematic diagram of obtaining an image by using a target box, according to an embodiment of this specification.
- FIG. 6 B is a schematic diagram of obtaining an image by using a mask, according to an embodiment of this specification.
- FIG. 7 is a flowchart of an example data processing method, according to an embodiment of this specification.
- FIG. 8 is a flowchart of a second data processing method, according to an embodiment of this specification.
- FIG. 9 is a flowchart of a third data processing method, according to an embodiment of this specification.
- FIG. 10 is a flowchart of a fourth data processing method, according to an embodiment of this specification.
- FIG. 11 is a schematic diagram of a first data processing apparatus, according to an embodiment of this specification.
- FIG. 12 is a schematic diagram of a second data processing apparatus, according to an embodiment of this specification.
- FIG. 13 is a schematic diagram of a third data processing apparatus, according to an embodiment of this specification.
- FIG. 14 is a flowchart of a fourth data processing method, according to an embodiment of this specification.
- FIG. 15 is a structural block diagram of a computer terminal, according to an embodiment of this specification.
- Video condensation redundant information in video data may be removed, and targets that users are interested in are condensed, to generate a synthesized video with a high density both in time and space.
- a processed image may be covered (wholly or partially) by using a selected image, graphic, or object, to control an image processing region or an image processing procedure.
- a mask is used for covering a particular image or object.
- the mask may be a binary image composed of 0 and 1.
- key frame screenshots may be saved for a single target, or blank frames in the long or short videos may be deleted.
- track information of the target may be lost in the process of saving the key frame screenshots, which results in a great information loss.
- To merely delete the blank frames results in a low video compression ratio.
- a data processing method is provided. It should be noted that steps shown in the flowchart of the accompanying drawings may be performed, for example, in a computer system storing a group of computer executable instructions. In addition, although a logic sequence is shown in the flowchart, in some cases, the shown or described steps may be performed in a sequence different from that described herein.
- FIG. 1 is a structural block diagram of hardware of a computer terminal (or a mobile device) configured to implement a data processing method.
- a computer terminal 10 (or a mobile device 10 ) may include one or more (shown as 102 a , 102 b . . .
- processors 102 may include, but is not limited to, a processing apparatus, for example, a microprocessor (MCU) or a field programmable gate array (FPGA)), a memory 104 configured to store data, and a transmission apparatus 106 for a communication function.
- I/O interface input/output interface
- USB universal serial bus
- the computer terminal 10 may also include more or fewer components than those shown in FIG. 1 , or have a configuration different from that shown in FIG. 1 .
- processors 102 and/or other data processing circuits in the context may be generally referred to as a “data processing circuit.”
- the data processing circuit may be entirely or partly embodied as software, hardware, firmware, or any combination thereof.
- the data processing circuit may be an independent processing module, or may be combined into any of other elements in the computer terminal 10 (or the mobile device) entirely or partly.
- the data processing circuit is used as a processor control (for example, a selection of a variable resistance terminal path connected to an interface).
- the memory 104 may be configured to store a software program and a module of application software, for example, a program instruction/data storage apparatus corresponding to the data processing method in the embodiments of this specification.
- the processor 102 runs the software program and the module stored in the memory 104 , to implement various functional applications and data processing, that is, implement the foregoing data processing method.
- the memory 104 may include a high-speed random memory, and a non-volatile memory such as one or more magnetic storage apparatuses, a flash memory, or another non-volatile solid-state memory.
- the memory 104 may further include memories remotely disposed relative to the processor 102 , and these remote memories may be connected to the computer terminal 10 through a network.
- An example of the foregoing network includes, but is not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.
- the transmission apparatus 106 is configured to receive or send data through a network.
- An example of the foregoing network may include a wireless network provided by a communication provider of the computer terminal 10 .
- the transmission apparatus 106 includes a network interface controller (NIC), which may be connected to another network device through a base station so as to communicate with the Internet.
- the transmission apparatus 106 may be a radio frequency (RF) module, which is configured to communicate with the Internet in a wireless manner.
- RF radio frequency
- the display may be a touch screen type liquid crystal display (LCD), and the LCD enables the user to interact with a user interface of the computer terminal 10 (or the mobile device).
- LCD liquid crystal display
- the computer device (or the mobile device) shown in FIG. 1 may include a hardware element (including a circuit), a software element (including computer code stored on a computer-readable medium), or a combination of the hardware element and the software element. It should be pointed out that, FIG. 1 is only one of concrete instances, and aims to show types of components existing in the foregoing computer device (or the mobile device).
- the computer device (or the mobile device) shown in FIG. 1 has a touch display (or referred to as a “touch screen” or a “touch display screen”).
- the computer device (or the mobile device) shown in FIG. 1 has a graphical user interface (GUI), and a user may perform a man-machine interaction with the GUI by touching a touch sensitive surface with a finger touch and/or by using a gesture.
- GUI graphical user interface
- the man-machine interaction function herein may include the following interactions such as webpage creating, drawing, word processing, electronic document making, games, video conference, instant messaging, email receiving and sending, call interface, digital video playing, digital music playing and/or network browsing.
- Executable instructions used for the foregoing man-machine interaction function are configured/stored in computer program products or readable storage media that can be executed by one or more processors.
- FIG. 2 is a flowchart of a first data processing method, according to an embodiment of this specification. As shown in FIG. 2 , the method includes steps S 202 -S 208 .
- step S 202 the method includes obtaining original video data and object information of a target object.
- the original video data in the foregoing step may be video data acquired by cameras installed in different application scenarios.
- the foregoing application scenarios include but are not limited to: a security surveillance scenario, a traffic surveillance scenario, an off-line shopping scenario, and the like.
- the video data is usually all-day 24-hour video data with a large amount of redundant information, which results in large occupied storage space and a long browsing time.
- the redundant information is mainly embodied in the following two aspects: there are more pure background frames with no targets, for example, the third, fourth, fifth, and sixth frames in FIG. 1 are all blank frames with no targets.
- Target density is low.
- there is only one target in the first and the second frames in FIG. 1 and the target is represented by using a triangle.
- the redundant information has low value, and if all the information is stored, storage resources are wasted and it is difficult for users to find effective information.
- the redundant information may be removed. If targets in the video are sparse, a one-day, 24-hour video may be reduced to several minutes after the redundant information is removed.
- the blank frames may be deleted, and targets of two different spaces are synthesized into the same frame for display.
- the processed video only includes 2 frames, the storage cost is reduced greatly, and the users may obtain all information of interest by browsing only two frames of the video.
- the target object in the foregoing step may be a target of interest to a user in the video data, for example, a person or a vehicle. That is, the target object may be a target that satisfies a particular condition and that is determined according to requirements of the user, for example, a male target, or a target of purchasing designated goods, but is not limited thereto.
- the object information may include: a target attribute and a target time period.
- the target attribute may include an attribute of the target object, for example, gender, age, height, or garment, or may alternatively include a particular operation performed by the target object, for example, purchasing designated goods in an off-line shopping scenario, but is not limited thereto.
- the target time period may be an acquisition time of video data that the user needs to browse, for example, 24 hours, but is not limited thereto.
- the object information of the target object needs to be determined according to requirements of the user. Therefore, an interaction interface may be provided for the users. As shown in FIG. 5 , the user may operate in the interaction interface, select a target time period of video data that the user needs to browse, input a target attribute of a target that the user is interested in, and confirm the foregoing information by clicking a “video condensation” button after the foregoing operations. Therefore, a server may compress, based on the information inputted by the user, original video data acquired by a camera.
- step S 204 the method includes processing the original video data based on the object information of the target object, to obtain a target image sequence of the target object.
- the target image sequence in the foregoing step may be images of the target object acquired in different frames of images in the video data in the target time period. All the images of the target object may be sorted according to the acquisition time. To facilitate the user to accurately learn a time point at which the image is acquired, a timestamp may be labeled on each image.
- the server may first perform target detection on each frame of image in the original video data based on the target attribute to recognize the target object, then extract an image sequence of the target object, and finally screens out an image sequence in the target time period, to obtain the target image sequence.
- the server may first obtain a corresponding part of video data from the original video data based on the target time period, then perform target detection on each frame of image in the part of video data based on the target attribute to recognize the target object, and finally extract images of the target object, to obtain the target image sequence.
- step S 206 the method includes processing the original video data, to obtain a background image, where the background image is included in the original video data and does not include a detection object.
- the background image in the foregoing step may be a background image in an original video image.
- the detection object may be a target such as a person or a vehicle, and the user may determine a target of interest in all the targets according to requirements, that is, the target object is an object satisfying the users' requirements in the detection objects.
- an image that is in the original video data and that only includes background information may be obtained as a background image by background modeling or selecting video frames without targets, or in other manners.
- the background image may alternatively be obtained through other manners, which is not limited in this specification.
- step S 208 the method includes synthesizing the target image sequence and the background image, to obtain target video data.
- images in the target image sequence may be superimposed into the background image in chronological order, to generate the target video data.
- images at different moments in the target image sequence may be superimposed on the background image separately.
- a length of the target video data is the same as a length of the target image sequence.
- images at different moments in the target image sequence may be superimposed on different locations in the background image. In this case, the length of the target video data is shorter than the length of the target image sequence.
- the video synthesis process destroys a time relationship between the images and the original video data. Therefore, to facilitate the user to obtain an occurrence time of the target object, the acquisition time of the image may also be superimposed on the background image.
- the target video data may be displayed in an interaction interface shown in FIG. 5 , and an occurrence time point of the target object in the original video may be displayed above the target object, so as to facilitate the user to find required information from the target video data.
- the foregoing method may further be applied to a tourism scenario, an exhibition hall visiting scenario, and the like.
- the user wants to generate video collections of travel or visit based on the captured original video data.
- the original video data may be captured by the user through a mobile terminal
- the target object may be the user, or may be all companions in the journey or the exhibition hall, but this specification is not limited thereto.
- the user may add different special effects into the target video data after the target video data is synthesized according to the preferences of the user, to obtain a final video collection to be watched by the relatives and friends.
- a target image sequence of the target object may be obtained by processing the original video data based on the object information of the target object; a background image is obtained by processing the original video data; and target video data is obtained by synthesizing the target image sequence and the background image, thereby implementing video condensation.
- the processing the original video data based on the object information of the target object, to obtain a target image sequence of the target object includes: processing the original video data, to obtain a segmentation mask of at least one detection object; processing the original video data by using the segmentation mask of the at least one detection object, to obtain an image sequence of the at least one detection object; and determining the target image sequence based on the object information of the target object.
- the target image sequence is extracted by using a target box in the existing solution.
- a target box introduces a large amount of interference information not belonging to the target object around the target object, which results in low visual effect and quality of the synthesized target video data.
- the target that the user is interested in is represented by using a triangle.
- another target represented by using a nearby rectangle and background information in the target box are retained in addition to the target represented by using the triangle.
- a mask of the target object may be predicted by using an instance segmentation technology, and the target box is replaced with the mask, so as to obtain more accurate information about the target object.
- the target that the user is interested in is represented by using a triangle, and only related information of the target represented by using the triangle is retained in the image obtained by using the mask.
- a segmentation mask of the detection object may be obtained by using deep learning segmentation. Then the detection object is extracted from the original video data according to the segmentation mask, to obtain an image sequence of the detection object, and a target image sequence of the target object is screened out finally.
- the segmentation mask of the detection object is obtained by using the instance segmentation technology, so as to improve the accuracy of obtaining effective information and the quality of the synthesized video.
- the processing the original video data to obtain a segmentation mask of at least one detection object includes: processing the original video data, to obtain at least one segmentation mask; processing the at least one segmentation mask by using a tracing algorithm, to obtain identification information of the at least one segmentation mask, where segmentation masks corresponding to the same identification information are masks of the same detection object; and obtaining the segmentation mask of the at least one detection object based on the identification information of the at least one segmentation mask.
- the foregoing identification information may be a unique ID number assigned to each detection object.
- the detection object in the video may be detected at regular intervals, to obtain a segmentation mask of a detection object in each frame of image.
- Segmentation masks in different image frames may be segmentation masks of the same detection object, or may be segmentation masks of different detection objects. Therefore, processing may be performed by using the tracing algorithm, and an ID number is assigned to each detection object.
- the same detection object has the same ID number, so that an ID number corresponding to each segmentation mask may be obtained. Further, all the obtained segmentation masks are classified according to the ID numbers, so as to obtain a series of segmentation masks of each detection object.
- the determining the target image sequence based on the object information of the target object includes: recognizing the image sequence of the at least one detection object, to determine attribute information of the at least one detection object; matching the target attribute of the target object with the attribute information of the at least one detection object, to determine an image sequence of the target object; and obtaining an image sequence in the target time period from the image sequence of the target object, to obtain the target image sequence.
- attributes such as age and gender of each detection object may be recognized from the image of the detection object by calling an attribute algorithm, or operations performed by each detection object may be recognized based on a movement track of the detection object, so as to obtain attribute information of each detection object. Further, the target attribute of the target object is matched with the attribute information of the detection object. The image sequence of the target object may be obtained when the matching is successful, and further, the target image sequence in the target time period may be extracted from the image sequence.
- image sequences of all detection objects in the time period may be determined, and then the target attribute is matched with the attribute information of the detection objects.
- the target image sequence may be obtained in a case of successful matching.
- the synthesizing the target image sequence and the background image, to obtain target video data includes: determining a target location of the target image sequence in the target video data based on a preset video synthesis rule; and synthesizing the target image sequence and the background image based on the target location, to obtain the target video data.
- the foregoing preset video synthesis rule may be a rule that ensures a high compression ratio of the target video data and ensures that the target object can be accurately recognized by the user.
- the preset video synthesis rule may include that: the target video data has relatively short duration, and all target sequences may be displayed within a limited time; target objects that are not associated with each other in the original video data do not block each other in the target video data; and target objects that have peer or interactive relationships in the original video data are still retained in the target video data.
- the foregoing target location may be a space and time location of the target image sequence in the target video data, and includes: a background image frame in the target video data and a location in the background image frame.
- images of target objects at different moments may be synthesized at different locations of the background image based on the preset video synthesis rule. As shown in FIG. 4 , images of different targets in the first frame and the seventh frame are synthesized at different locations of the same frame, and images of different targets in the second frame and the eighth frame are synthesized at different locations of the same frame.
- the determining a target location of the target image sequence in the target video data based on a preset video synthesis rule includes: determining a length of the target video data based on a length of the target image sequence, and determining a first location of the target image sequence in the target video data; obtaining a loss value of the first location based on the preset video synthesis rule; determining, when the loss value of the first location is greater than a preset value, a second location of the target image sequence in the target video data, and using the second location as the first location until the loss value of the first location is less than the preset value; and determining the first location as the target location when the loss value of the first location is less than the preset value.
- the foregoing loss value may represent a loss cost of the space and time location of the target image sequence in the target video data.
- the loss cost is low if the preset video synthesis rule is satisfied, and the loss cost is high if the preset video synthesis rule is not satisfied. For example, if target objects that are not associated with each other in the original video data are blocked in the target video data, the loss cost increases; otherwise, the loss cost decreases. In another example, if association relationships of target image sequences that have peer or interactive relationships in the original video data are destroyed in the target video data, the loss cost increases; otherwise, the loss cost decreases.
- the foregoing preset value may be a preset minimum loss cost, and may be determined according to actual video compression requirements, or may be set by the user according to the requirements. This is not limited in this specification.
- the length of the target video data may be estimated based on the length of the target image sequence, a location of the target image sequence in the target video data is randomly initialized, then the loss cost of the space and time location of the target image sequence in the target video data is calculated, and the space and time location of the target image sequence is further adjusted. Therefore, the loss cost is minimized through optimization and is used as a final target location for video synthesis.
- the method may be performed by a server, and the procedure does not involve a step of inputting a target attribute and a target time period by the user and steps performed by a client, for example, displaying the target video data.
- the method includes the following steps: reading an original video; generating a background image only including background information; detecting targets of interest to a user in the original video and obtaining segmentation masks of the targets; determining movement tracks of the targets by calling a tracing algorithm, and assigning an ID number to each target; classifying the masks based on the ID numbers, and extracting mask sequences of the targets from the original video; recognizing an attribute of each target by calling an attribute algorithm, and labeling the mask sequences; and screening out a particular target sequence from the mask sequences according to a label selected by the user, and superimposing the target sequence on the background image in chronological order.
- target sequences at different moments may occur at different locations at the same moment in the synthesized video, to enable each frame in the synthesized video to include more information.
- the user may screen out required labels to generate a condensed video, thereby improving user experience.
- the computer software product is stored in a storage medium (for example, a read-only memory (ROM)/random access memory (RAM), a magnetic disk or an optical disc), and includes several instructions for instructing a terminal device (which may be a mobile phone, a computer, a server, a network device or the like) to perform the method described in the embodiments of this specification.
- a storage medium for example, a read-only memory (ROM)/random access memory (RAM), a magnetic disk or an optical disc
- a terminal device which may be a mobile phone, a computer, a server, a network device or the like
- a server may process, after receiving object information of a target object inputted by a user, original video data based on the object information of the target object. A large amount of data needs to be processed and the whole processing takes a long time.
- a data processing method further is provided.
- FIG. 8 is a flowchart of a second data processing method according to an embodiment of this specification. As shown in FIG. 8 , the method includes steps S 802 -S 806 .
- step S 802 the method includes obtaining object information of a target object.
- the target object in the foregoing step may be a target of interest to a user in the video data, for example, a person or a vehicle. That is, the target object may be a target that satisfies a particular condition and that is determined according to requirements of the user, for example, a male target, or a target of purchasing designated goods, but is not limited thereto.
- the object information may include: a target attribute and a target time period.
- the target attribute may include an attribute of the target object, for example, gender, age, height, or garment, or may alternatively include a particular operation performed by the target object, for example, purchasing designated goods in an off-line shopping scenario, but is not limited thereto.
- the target time period may be an acquisition time of video data that the user needs to browse, for example, 24 hours, but is not limited thereto.
- step S 804 the method includes matching the object information of the target object with a pre-stored image sequence of at least one detection object, to obtain a target image sequence of the target object, where the image sequence of the at least one detection object is obtained by processing original video data.
- the original video data in the foregoing step may be video data acquired by cameras installed in different application scenarios.
- the foregoing application scenarios include but are not limited to: a security surveillance scenario, a traffic surveillance scenario, an off-line shopping scenario, and the like.
- the video data is usually all-day 24-hour video data with a large amount of redundant information, which results in large occupied storage space and a long browsing time.
- the detection object may be a target such as a person or a vehicle, and the user may determine a target of interest in all the targets according to requirements, that is, the target object is an object satisfying the users' requirements in the detection objects.
- the image sequence may be images of detection objects acquired in different frames of images in the original video data. All the images are sorted according to the acquisition time. To facilitate the user to accurately learn a time point at which the detection object is acquired, a timestamp may be labeled on each image.
- the server may directly perform, after obtaining the original video data acquired by a camera, target detection on the original video data, to recognize each detection object, extract image sequences of the detection objects, and finally store the extracted image sequences in a database. Based on this, after receiving the target attribute and the target time period, the server only needs to match the target attribute and the target time period with the stored image sequences of the detection objects, determine the image sequence of the target object when the matching is successful, and screen out the target image sequence in the target time period based on the timestamp of the image.
- step S 806 the method includes synthesizing the target image sequence and a background image, to obtain target video data, where the background image is included in the original video data and does not include a detection object.
- the background image in the foregoing step may be a background image in an original video image.
- an image that is in the original video data and that only includes background information may be directly obtained as a background image by background modeling or selecting video frames without targets, or in other manners, and the background image is stored. Therefore, after the target image sequence is obtained, the target video data may be generated directly. By storing the background image, the background image can be reused to save computing resources of the server.
- a target image sequence of the target object may be obtained by matching the object information of the target object with a pre-stored image sequence of at least one detection object; further, target video data is obtained by synthesizing the target image sequence and the background image, thereby implementing video condensation.
- the method further includes: processing the original video data, to obtain a segmentation mask of at least one detection object; processing the original video data by using the segmentation mask of the at least one detection object, to obtain an image sequence of the at least one detection object; recognizing the image sequence of the at least one detection object, to determine attribute information of the at least one detection object; and storing the image sequence and the attribute information of the at least one detection object.
- a mask of the target object may be predicted by using an instance segmentation technology, and the target box is replaced with the mask, so as to obtain more accurate information about the target object.
- a segmentation mask of the detection object may be obtained by using deep learning segmentation. Then the detection object is extracted from the original video data according to the segmentation mask, to obtain an image sequence of the detection object, and a target image sequence of the target object is screened out finally.
- the server may recognize the detection object, to obtain attribute information of the detection object and label the image sequence, so as to facilitate matching with the target attribute subsequently and obtaining the target image sequence of the target object quickly.
- the processing the original video data to obtain a segmentation mask of at least one detection object includes: processing the original video data, to obtain at least one segmentation mask; processing the at least one segmentation mask by using a tracing algorithm, to obtain identification information of the at least one segmentation mask, where segmentation masks corresponding to the same identification information are masks of the same detection object; and obtaining the segmentation mask of the at least one detection object based on the identification information of the at least one segmentation mask.
- the matching the object information of the target object with a pre-stored image sequence of at least one detection object, to obtain a target image sequence of the target object includes: matching the target attribute of the target object with the attribute information of the at least one detection object, to determine an image sequence of the target object; and obtaining an image sequence in the target time period from the image sequence of the target object, to obtain the target image sequence.
- the image sequences stored in the database are labeled in advance, and the attribute information of each detection object is annotated.
- the target object may be determined, and the image sequence of the target object is read. Further, the target image sequence is determined based on the timestamp.
- the synthesizing the target image sequence and the background image, to obtain target video data includes: determining a target location of the target image sequence in the target video data based on a preset video synthesis rule; and synthesizing the target image sequence and the background image based on the target location, to obtain the target video data.
- the determining a target location of the target image sequence in the target video data based on a preset video synthesis rule includes: determining a length of the target video data based on a length of the target image sequence, and determining a first location of the target image sequence in the target video data; obtaining a loss value of the first location based on the preset video synthesis rule; determining, when the loss value of the first location is greater than a preset value, a second location of the target image sequence in the target video data, and using the second location as the first location until the loss value of the first location is less than the preset value; and determining the first location as the target location when the loss value of the first location is less than the preset value.
- the exemplary implementation solution in the foregoing embodiments of this specification has the same as the solution, the application scenario, and the implementation process provided in Embodiment 1, but is not limited to the solution provided in Embodiment 1.
- a data processing method is further provided.
- FIG. 9 is a flowchart of a third data processing method according to an embodiment of this specification. As shown in FIG. 9 , the method includes steps S 902 and S 904 .
- step S 902 the method includes displaying original video data and object information of a target object.
- the original video data and the object information of the target object may be displayed in an interaction interface shown in FIG. 5 , where the original video data is displayed in a display region at the right side.
- step S 904 the method includes displaying target video data, where the target video data is obtained by synthesizing a target image sequence of the target object and a background image in the original video data, the target image sequence is obtained by processing the original video data based on the object information of the target object, and the background image is included in the original video data and does not include a detection object.
- the target video data may also be displayed in the interaction interface shown in FIG. 5 .
- the target video data may replace the original video data and be displayed in the display region at the right side.
- the processing the original video data based on the object information of the target object includes: processing the original video data, to obtain a segmentation mask of at least one detection object; processing the original video data by using the segmentation mask of the at least one detection object, to obtain an image sequence of the at least one detection object; and determining the target image sequence based on the object information of the target object.
- the processing the original video data to obtain a segmentation mask of at least one detection object includes: processing the original video data, to obtain at least one segmentation mask; processing the at least one segmentation mask by using a tracing algorithm, to obtain identification information of the at least one segmentation mask, where segmentation masks corresponding to the same identification information are masks of the same detection object; and obtaining the segmentation mask of the at least one detection object based on the identification information of the at least one segmentation mask.
- the determining the target image sequence based on the object information of the target object includes: recognizing the image sequence of the at least one detection object, to determine attribute information of the at least one detection object; matching the target attribute of the target object with the attribute information of the at least one detection object, to determine an image sequence of the target object; and obtaining an image sequence in the target time period from the image sequence of the target object, to obtain the target image sequence.
- the synthesizing a target image sequence of the target object and a background image in the original video data includes: determining a target location of the target image sequence in the target video data based on a preset video synthesis rule; synthesizing the target image sequence and the background image based on the target location, to obtain the target video data.
- the determining a target location of the target image sequence in the target video data based on a preset video synthesis rule includes: determining a length of the target video data based on a length of the target image sequence, and determining a first location of the target image sequence in the target video data; obtaining a loss value of the first location based on the preset video synthesis rule; determining, when the loss value of the first location is greater than a preset value, a second location of the target image sequence in the target video data, and using the second location as the first location until the loss value of the first location is less than the preset value; and determining the first location as the target location when the loss value of the first location is less than the preset value.
- the target image sequence and/or an acquisition time of the target image sequence is displayed in the target video data in a preset manner.
- a preset manner may be set in advance.
- the manner may be used as a display manner of the target image sequence and/or the acquisition time, to remind the user.
- the display manner may be adjusted by the user according to preferences.
- the preset manner being flicker is used as an example for description.
- a flickering target box may be displayed in the target video data, where the target object is located in the target box.
- an image of the target object may be directly controlled to flicker.
- the acquisition time may be directly controlled to flicker.
- the image of the target object and the acquisition time may be controlled to flicker at the same time.
- the foregoing preset manners may be other manners, which is not limited in this specification.
- the exemplary implementation solution in the foregoing embodiments of this specification has the same as the solution, the application scenario, and the implementation process provided in Embodiment 1, but is not limited to the solution provided in Embodiment 1.
- a data processing method is further provided.
- FIG. 10 is a flowchart of a fourth data processing method according to an embodiment of this specification. As shown in FIG. 10 , the method includes steps S 102 and S 104
- step S 102 the method includes displaying object information of a target object.
- the object information of the target object may be displayed in an interaction interface shown in FIG. 5 .
- step S 202 the method includes displaying target video data, where the target video data is obtained by synthesizing a target image sequence of the target object and a background image in original video data, the target image sequence is obtained by matching the object information of the target object with a pre-stored image sequence of at least one detection object, and the background image is included in the original video data and does not include a detection object.
- the target video data may also be displayed in the interaction interface shown in FIG. 5 , where the target video data is displayed in the display region at the right side.
- the method further includes: processing the original video data, to obtain a segmentation mask of at least one detection object; processing the original video data by using the segmentation mask of the at least one detection object, to obtain an image sequence of the at least one detection object; recognizing the image sequence of the at least one detection object, to determine attribute information of the at least one detection object; and storing the image sequence and the attribute information of the at least one detection object.
- the processing the original video data to obtain a segmentation mask of at least one detection object includes: processing the original video data, to obtain at least one segmentation mask; processing the at least one segmentation mask by using a tracing algorithm, to obtain identification information of the at least one segmentation mask, where segmentation masks corresponding to the same identification information are masks of the same detection object; and obtaining the segmentation mask of the at least one detection object based on the identification information of the at least one segmentation mask.
- the matching the object information of the target object with a pre-stored image sequence of at least one detection object includes: matching the target attribute of the target object with the attribute information of the at least one detection object, to determine an image sequence of the target object; obtaining an image sequence in the target time period from the image sequence of the target object, to obtain the target image sequence.
- the synthesizing a target image sequence of the target object and a background image in the original video data includes: determining a target location of the target image sequence in the target video data based on a preset video synthesis rule; synthesizing the target image sequence and the background image based on the target location, to obtain the target video data.
- the determining a target location of the target image sequence in the target video data based on a preset video synthesis rule includes: determining a length of the target video data based on a length of the target image sequence, and determining a first location of the target image sequence in the target video data; obtaining a loss value of the first location based on the preset video synthesis rule; determining, when the loss value of the first location is greater than a preset value, a second location of the target image sequence in the target video data, and using the second location as the first location until the loss value of the first location is less than the preset value; and determining the first location as the target location when the loss value of the first location is less than the preset value.
- the target image sequence and/or an acquisition time of the target image sequence is displayed in the target video data in a preset manner.
- the exemplary implementation solution in the foregoing embodiments of this specification has the same as the solution, the application scenario, and the implementation process provided in Embodiment 3, but is not limited to the solution provided in Embodiment 3.
- the apparatus 1100 includes: an obtaining module 1102 , a first processing module 1104 , a second processing module 1106 , and a synthesizing module 1108 .
- the obtaining module 1102 is configured to obtain original video data and object information of a target object.
- the first processing module 1104 is configured to process the original video data based on the object information of the target object, to obtain a target image sequence of the target object.
- the second processing module 1106 is configured to process the original video data, to obtain a background image, where the background image is included in the original video data and does not include a detection object.
- the synthesizing module 1108 is configured to synthesize the target image sequence and the background image, to obtain target video data.
- the obtaining module 1102 , the first processing module 1104 , the second processing module 1106 , and the synthesizing module 1108 correspond to step S 202 to step S 208 in Embodiment 1.
- An implementation instance and an application scenario of the four modules are the same as those of the corresponding steps, but are not limited to the content disclosed in Embodiment 1.
- the foregoing modules may be run on a computer terminal 10 provided in Embodiment 1 as a part of the apparatus.
- the first processing module includes: a first processing unit, a second processing unit, and a first determining unit.
- the first processing unit is configured to process the original video data, to obtain a segmentation mask of at least one detection object.
- the second processing unit is configured to process the original video data by using the segmentation mask of the at least one detection object, to obtain an image sequence of the at least one detection object.
- the first determining unit is configured to determine the target image sequence based on the object information of the target object.
- the first processing unit includes: a detection subunit, a first processing subunit, and a second processing subunit.
- the detection subunit is configured to detect the original video data, to obtain at least one segmentation mask.
- the first processing subunit is configured to process the at least one segmentation mask by using a tracing algorithm, to obtain identification information of the at least one segmentation mask, where segmentation masks corresponding to the same identification information are masks of the same detection object.
- the second processing subunit is configured to obtain the segmentation mask of the at least one detection object based on the identification information of the at least one segmentation mask.
- the first determining unit includes: a recognition subunit, a matching subunit, and an obtaining subunit.
- the recognition subunit is configured to recognize the image sequence of the at least one detection object, to determine attribute information of the at least one detection object.
- the matching subunit is configured to match the target attribute of the target object with the attribute information of the at least one detection object, to determine an image sequence of the target object.
- the obtaining subunit is configured to obtain an image sequence in the target time period from the image sequence of the target object, to obtain the target image sequence.
- the synthesizing module includes a second determining unit and a synthesizing unit.
- the second determining unit is configured to determine a target location of the target image sequence in the target video data based on a preset video synthesis rule.
- the synthesizing unit is configured to synthesize the target image sequence and the background image based on the target location, to obtain the target video data.
- the second determining unit includes: a first determining subunit, a third processing subunit, a second determining subunit, and a third determining subunit.
- the first determining subunit is configured to determine a length of the target video data based on a length of the target image sequence, and determine a first location of the target image sequence in the target video data.
- the third processing subunit is configured to obtain a loss value of the first location based on the preset video synthesis rule.
- the second determining subunit is configured to determine, when the loss value of the first location is greater than a preset value, a second location of the target image sequence in the target video data, and use the second location as the first location until the loss value of the first location is less than the preset value.
- the third determining subunit is configured to determine the first location as the target location when the loss value of the first location is less than the preset value.
- the exemplary implementation solution in the foregoing embodiments of this specification has the same as the solution, the application scenario, and the implementation process provided in Embodiment 1, but is not limited to the solution provided in Embodiment 1.
- the apparatus 1200 includes: an obtaining module 1202 , a matching module 1204 , and a synthesizing module 1206 .
- the obtaining module 1202 is configured to obtain object information of a target object.
- the matching module 1204 is configured to match the object information of the target object with a pre-stored image sequence of at least one detection object, to obtain a target image sequence of the target object, where the image sequence of the at least one detection object is obtained by processing original video data.
- the synthesizing module 1206 is configured to synthesize the target image sequence and a background image, to obtain target video data, where the background image is included the original video data and does not include a detection object.
- the obtaining module 1202 , the matching module 1204 , and the synthesizing module 1206 correspond to step S 802 to step S 806 in Embodiment 2.
- An implementation instance and an application scenario of the three modules are the same as those of the corresponding steps, but are not limited to the content disclosed in Embodiment 2. It should be noted that, the foregoing modules may be run on a computer terminal 10 provided in Embodiment 1 as a part of the apparatus.
- the apparatus further includes: a first processing module, a second processing module, a determining module, and a storage module.
- the first processing module is configured to process the original video data, to obtain a segmentation mask of at least one detection object.
- the second processing module is configured to process the original video data by using the segmentation mask of the at least one detection object, to obtain an image sequence of the at least one detection object.
- the determining module is configured to recognize the image sequence of the at least one detection object, to determine attribute information of the at least one detection object.
- the storage module is configured to store the image sequence and the attribute information of the at least one detection object.
- the first processing module includes: a detection unit, a first processing unit, and a second processing unit.
- the detection unit is configured to detect the original video data, to obtain at least one segmentation mask.
- the first processing unit is configured to process the at least one segmentation mask by using a tracing algorithm, to obtain identification information of the at least one segmentation mask, where segmentation masks corresponding to the same identification information are masks of the same detection object.
- the second processing unit is configured to obtain the segmentation mask of the at least one detection object based on the identification information of the at least one segmentation mask.
- the matching module includes a matching unit and an obtaining unit.
- the matching unit is configured to match the target attribute of the target object with the attribute information of the at least one detection object, to determine an image sequence of the target object.
- the obtaining unit is configured to obtain an image sequence in the target time period from the image sequence of the target object, to obtain the target image sequence.
- the synthesizing module includes a determining unit and a synthesizing unit.
- the determining unit is configured to determine a target location of the target image sequence in the target video data based on a preset video synthesis rule.
- the synthesizing unit is configured to synthesize the target image sequence and the background image based on the target location, to obtain the target video data.
- the determining unit includes: a first determining subunit, a processing subunit, a second determining subunit, and a third determining subunit.
- the first determining subunit is configured to determine a length of the target video data based on a length of the target image sequence, and determine a first location of the target image sequence in the target video data.
- the processing subunit is configured to obtain a loss value of the first location based on the preset video synthesis rule.
- the second determining subunit is configured to determine, when the loss value of the first location is greater than a preset value, a second location of the target image sequence in the target video data, and use the second location as the first location until the loss value of the first location is less than the preset value.
- the third determining subunit is configured to determine the first location as the target location when the loss value of the first location is less than the preset value.
- the exemplary implementation solution in the foregoing embodiments of this specification has the same as the solution, the application scenario, and the implementation process provided in Embodiment 1, but is not limited to the solution provided in Embodiment 1.
- the apparatus 1300 includes: a first display module 1302 and a second display module 1304 .
- the first display module 1302 is configured to display original video data and object information of a target object.
- the second display module 1304 is configured to display target video data, where the target video data is obtained by synthesizing a target image sequence of the target object and a background image in the original video data, the target image sequence is obtained by processing the original video data based on the object information of the target object, and the background image is included in the original video data and does not include a detection object.
- first display module 1302 and the second display module 1304 correspond to step S 902 to step S 904 in Embodiment 3.
- An implementation instance and an application scenario of the two modules are the same as those of the corresponding steps, but are not limited to the content disclosed in Embodiment 3.
- the foregoing modules may be run on a computer terminal 10 provided in Embodiment 1 as a part of the apparatus.
- the apparatus further includes: a first processing module, a second processing module, and a first determining module.
- the first processing module is configured to process the original video data, to obtain a segmentation mask of at least one detection object.
- the second processing module is configured to process the original video data by using the segmentation mask of the at least one detection object, to obtain an image sequence of the at least one detection object.
- the first determining module is configured to determine the target image sequence based on the object information of the target object.
- the first processing module includes: a detection unit, a first processing unit, and a second processing unit.
- the detection unit is configured to detect the original video data, to obtain at least one segmentation mask.
- the first processing unit is configured to process the at least one segmentation mask by using a tracing algorithm, to obtain identification information of the at least one segmentation mask, where segmentation masks corresponding to the same identification information are masks of the same detection object.
- the second processing unit is configured to obtain the segmentation mask of the at least one detection object based on the identification information of the at least one segmentation mask.
- the first determining module includes: a recognition unit, a matching unit, and an obtaining unit.
- the recognition unit is configured to recognize the image sequence of the at least one detection object, to determine attribute information of the at least one detection object.
- the matching unit is configured to match the target attribute of the target object with the attribute information of the at least one detection object, to determine an image sequence of the target object.
- the obtaining unit is configured to obtain an image sequence in the target time period from the image sequence of the target object, to obtain the target image sequence.
- the apparatus further includes: a second determining module and a synthesizing module.
- the second determining module is configured to determine a target location of the target image sequence in the target video data based on a preset video synthesis rule.
- the synthesizing module is configured to synthesize the target image sequence and the background image based on the target location, to obtain the target video data.
- the second determining module includes: a first determining unit, a third processing unit, a second determining unit, and a third determining unit.
- the first determining unit is configured to determine a length of the target video data based on a length of the target image sequence, and determine a first location of the target image sequence in the target video data.
- the third processing unit is configured to obtain a loss value of the first location based on the preset video synthesis rule.
- the second determining unit is configured to determine, when the loss value of the first location is greater than a preset value, a second location of the target image sequence in the target video data, and use the second location as the first location until the loss value of the first location is less than the preset value.
- the third determining unit is configured to determine the first location as the target location when the loss value of the first location is less than the preset value.
- the second display module is further configured to display the target image sequence and/or an acquisition time of the target image sequence in the target video data in a preset manner.
- the exemplary implementation solution in the foregoing embodiments of this specification has the same as the solution, the application scenario, and the implementation process provided in Embodiment 1, but is not limited to the solution provided in Embodiment 1.
- a data processing apparatus for performing the foregoing data processing method is further provided. As shown in FIG. 13 , the apparatus includes: a first display module 1302 and a second display module 1304 .
- the first display module 1302 is configured to display object information of a target object.
- the second display module 1304 is configured to display target video data, where the target video data is obtained by synthesizing a target image sequence of the target object and a background image in original video data, the target image sequence is obtained by matching the object information of the target object with a pre-stored image sequence of at least one detection object, and the background image is included in the original video data and does not include a detection object.
- first display module 1302 and the second display module 1304 correspond to step S 102 to step S 104 in Embodiment 4.
- An implementation instance and an application scenario of the two modules are the same as those of the corresponding steps, but are not limited to the content disclosed in Embodiment 4.
- the foregoing modules may be run on a computer terminal 10 provided in Embodiment 1 as a part of the apparatus.
- the apparatus further includes: a first processing module, a second processing module, a first determining module, and a storage module.
- the first processing module is configured to process the original video data, to obtain a segmentation mask of at least one detection object.
- the second processing module is configured to process the original video data by using the segmentation mask of the at least one detection object, to obtain an image sequence of the at least one detection object.
- the first determining module is configured to recognize the image sequence of the at least one detection object, to determine attribute information of the at least one detection object.
- the storage module is further configured to store the image sequence and the attribute information of the at least one detection object.
- the first processing module includes: a detection unit, a first processing unit, and a second processing unit.
- the detection unit is configured to detect the original video data, to obtain at least one segmentation mask.
- the first processing unit is configured to process the at least one segmentation mask by using a tracing algorithm, to obtain identification information of the at least one segmentation mask, where segmentation masks corresponding to the same identification information are masks of the same detection object.
- the second processing unit is configured to obtain the segmentation mask of the at least one detection object based on the identification information of the at least one segmentation mask.
- the apparatus further includes: a matching module and an obtaining module.
- the matching module is configured to match the target attribute of the target object with the attribute information of the at least one detection object, to determine an image sequence of the target object.
- the obtaining module is configured to obtain an image sequence in the target time period from the image sequence of the target object, to obtain the target image sequence.
- the apparatus further includes: a second determining module and a synthesizing module.
- the second determining module is configured to determine a target location of the target image sequence in the target video data based on a preset video synthesis rule.
- the synthesizing module is configured to synthesize the target image sequence and the background image based on the target location, to obtain the target video data.
- the second determining module includes: a first determining unit, a third processing unit, a second determining unit, and a third determining unit.
- the first determining unit is configured to determine a length of the target video data based on a length of the target image sequence, and determine a first location of the target image sequence in the target video data.
- the third processing unit is configured to obtain a loss value of the first location based on the preset video synthesis rule.
- the second determining unit is configured to determine, when the loss value of the first location is greater than a preset value, a second location of the target image sequence in the target video data, and use the second location as the first location until the loss value of the first location is less than the preset value.
- the third determining unit is configured to determine the first location as the target location when the loss value of the first location is less than the preset value.
- the second display module is further configured to display the target image sequence and/or an acquisition time of the target image sequence in the target video data in a preset manner.
- the exemplary implementation solution in the foregoing embodiments of this specification has the same as the solution, the application scenario, and the implementation process provided in Embodiment 1, but is not limited to the solution provided in Embodiment 1.
- a data processing method is further provided.
- FIG. 14 is a flowchart of a fifth data processing method according to an embodiment of this specification. As shown in FIG. 14 , the method includes steps S 1402 -S 1408 .
- Step S 1402 the method includes obtaining original video data.
- the original video data in the foregoing step may be video data acquired by cameras installed in different application scenarios.
- the foregoing application scenarios include but are not limited to: a security surveillance scenario, a traffic surveillance scenario, an off-line shopping scenario, and the like.
- the video data is usually all-day 24-hour video data with a large amount of redundant information, which results in large occupied storage space and a long browsing time.
- Step S 1402 the method includes receiving a target object inputted by a client.
- the client in the foregoing step may be a mobile terminal, such as a smartphone (including an Android mobile phone or an iOS mobile phone), a tablet computer, a palmtop computer, or a notebook computer of the user, but is not limited thereto.
- the target object may be a target of interest to a user in the video data, for example, a person or a vehicle. That is, the target object may be a target that satisfies a particular condition and that is determined according to requirements of the user, for example, a male target or a target of purchasing designated goods, but this is not limited thereto.
- the object information may include: a target attribute and a target time period.
- the target attribute may include an attribute of the target object, for example, gender, age, height, or garment, and may alternatively include a particular operation performed by the target object, for example, purchasing designated goods in an off-line shopping scenario, but this is not limited thereto.
- the target time period may be an acquisition time of video data that the user needs to browse, for example, 24 hours, but is not limited thereto.
- Step S 1406 the method includes obtaining a target image sequence corresponding to the target object from the original video data.
- the target image sequence in the foregoing step may be images of the target object acquired in different frames of images in the video data in the target time period. All the images of the target object may be sorted according to the acquisition time. To facilitate the user to accurately learn a time point at which the image is acquired, a timestamp may be labeled on each image.
- Step S 1408 the method includes displaying the target image sequence in a fade-in and fade-out manner.
- each image in the target image sequence may be displayed successively in chronological order, and each image is displayed in a fade-in and fade-out manner. Therefore, the user may see the target object moving on the screen and finally disappearing, making it more intuitive for the user to determine a movement track of the target object.
- the target image sequence corresponding to the target object may be obtained from the original video data, and the target image sequence is displayed in a fade-in and fade-out manner, thereby implementing video condensation.
- the processing the original video data based on the object information of the target object, to obtain a target image sequence of the target object includes: processing the original video data, to obtain a segmentation mask of at least one detection object; processing the original video data by using the segmentation mask of the at least one detection object, to obtain an image sequence of the at least one detection object; and determining the target image sequence based on the object information of the target object.
- the processing the original video data to obtain a segmentation mask of at least one detection object includes: processing the original video data, to obtain at least one segmentation mask; processing the at least one segmentation mask by using a tracing algorithm, to obtain identification information of the at least one segmentation mask, where segmentation masks corresponding to the same identification information are masks of the same detection object; and obtaining the segmentation mask of the at least one detection object based on the identification information of the at least one segmentation mask.
- the determining the target image sequence based on the object information of the target object includes: recognizing the image sequence of the at least one detection object, to determine attribute information of the at least one detection object; matching the target attribute of the target object with the attribute information of the at least one detection object, to determine an image sequence of the target object; and obtaining an image sequence in the target time period from the image sequence of the target object, to obtain the target image sequence.
- An embodiment of this specification may further provide a computer terminal, and the computer terminal may be any computer terminal device in a computer terminal cluster.
- the foregoing computer terminal may be replaced with a terminal device such as a mobile terminal.
- the foregoing computer terminal may be located in at least one of a plurality of network devices in a computer network.
- the computer terminal may execute program code of the following steps in the data processing method: obtaining original video data and object information of a target object; processing the original video data based on the object information of the target object, to obtain a target image sequence of the target object; processing the original video data, to obtain a background image, where the background image is included in the original video data and does not include a detection object; and synthesizing the target image sequence and the background image, to obtain target video data.
- FIG. 15 is a structural block diagram of a computer terminal according to an embodiment of this specification.
- the computer terminal A may include: one or more (only one is shown in the figure) processors 1502 and a memory 1504 .
- the memory may be configured to store a software program and a module, for example, a program instruction/module corresponding to the data processing method and apparatus in the embodiments of this specification.
- the processor runs the software program and the module stored in the memory, to implement various functional applications and data processing, that is, implement the foregoing data processing method.
- the memory may include a high-speed random access memory, and may also include a non-volatile memory, for example, one or more magnetic storage apparatuses, flash memories, or other non-volatile solid-state memories.
- the memory may further include memories remotely disposed relative to the processor, and these remote memories may be connected to the terminal A through a network.
- An example of the foregoing network includes, but is not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.
- the processor may call, by using the transmission apparatus, the information and the application program that are stored in the memory, to perform the following steps: obtaining original video data and object information of a target object; processing the original video data based on the object information of the target object, to obtain a target image sequence of the target object; processing the original video data, to obtain a background image, where the background image is included in the original video data and does not include a detection object; and synthesizing the target image sequence and the background image, to obtain target video data.
- the foregoing processor may further execute program code of the following steps: processing the original video data, to obtain a segmentation mask of at least one detection object; processing the original video data by using the segmentation mask of the at least one detection object, to obtain an image sequence of the at least one detection object; and determining the target image sequence based on the object information of the target object.
- the foregoing processor may further execute program code of the following steps: detecting the original video data, to obtain at least one segmentation mask; processing the at least one segmentation mask by using a tracing algorithm, to obtain identification information of the at least one segmentation mask, where segmentation masks corresponding to the same identification information are masks of the same detection object; and obtaining the segmentation mask of the at least one detection object based on the identification information of the at least one segmentation mask.
- the object information includes at least a target attribute and a target time period
- the foregoing processor may further execute program code of the following steps: recognizing the image sequence of the at least one detection object, to determine attribute information of the at least one detection object; matching the target attribute of the target object with the attribute information of the at least one detection object, to determine an image sequence of the target object; and obtaining an image sequence in the target time period from the image sequence of the target object, to obtain the target image sequence.
- the foregoing processor may further execute program code of the following steps: determining a target location of the target image sequence in the target video data based on a preset video synthesis rule; and synthesizing the target image sequence and the background image based on the target location, to obtain the target video data.
- the foregoing processor may further execute program code of the following steps: determining a length of the target video data based on a length of the target image sequence, and determining a first location of the target image sequence in the target video data; obtaining a loss value of the first location based on the preset video synthesis rule; determining, when the loss value of the first location is greater than a preset value, a second location of the target image sequence in the target video data, and using the second location as the first location until the loss value of the first location is less than the preset value; and determining the first location as the target location when the loss value of the first location is less than the preset value.
- the processor may call, by using the transmission apparatus, the information and the application program that are stored in the memory, to perform the following steps: obtaining object information of a target object; matching the object information of the target object with a pre-stored image sequence of at least one detection object, to obtain a target image sequence of the target object, where the image sequence of the at least one detection object is obtained by processing original video data; and synthesizing the target image sequence and a background image, to obtain target video data, where the background image is included in the original video data and does not include a detection object.
- the foregoing processor may further execute program code of the following steps: processing the original video data, to obtain a segmentation mask of at least one detection object; processing the original video data by using the segmentation mask of the at least one detection object, to obtain an image sequence of the at least one detection object; and recognizing the image sequence of the at least one detection object, to determine attribute information of the at least one detection object; and storing the image sequence and the attribute information of the at least one detection object.
- the foregoing processor may further execute program code of the following steps: matching the target attribute of the target object with the attribute information of the at least one detection object, to determine an image sequence of the target object; and obtaining an image sequence in the target time period from the image sequence of the target object, to obtain the target image sequence.
- the processor may call, by using the transmission apparatus, the information and the application program that are stored in the memory, to execute the following steps: displaying original video data and object information of a target object; and displaying target video data, where the target video data is obtained by synthesizing a target image sequence of the target object and a background image in the original video data, the target image sequence is obtained by processing the original video data based on the object information of the target object, and the background image is included in the original video data and does not include a detection object.
- the foregoing processor may further execute program code of the following steps: displaying the target image sequence and/or an acquisition time of the target image sequence in the target video data in a preset manner.
- the processor may call, by using the transmission apparatus, the information and the application program that are stored in the memory, to perform the following steps: displaying object information of a target object; and displaying target video data, where the target video data is obtained by synthesizing a target image sequence of the target object and a background image in original video data, the target image sequence is obtained by matching the object information of the target object with a pre-stored image sequence of at least one detection object, and the background image is included in the original video data and does not include a detection object.
- a data processing solution is provided.
- By extracting a target image sequence and synthesizing the target image sequence with a background image redundant information in an original video is removed, and only a target of interest to a user is retained in the video, thereby reducing a length of a long-duration video, reducing storage costs, and improving efficiency of obtaining effective information by the user, further resolving a technical problem that a target object cannot be identified quickly due to a large amount of stored video data and a large amount of redundant information in the related art.
- the processor may call, by using the transmission apparatus, the information and the application program that are stored in the memory, to perform the following steps: obtaining original video data; receiving a target object inputted by a client; obtaining a target image sequence corresponding to the target object from the original video data; and displaying the target image sequence in a fade-in and fade-out manner.
- the computer terminal may be a terminal device such as a smartphone (for example, an Android mobile phone or an iOS mobile phone), a tablet computer, a palmtop computer, a mobile Internet device (MID), and a PAD.
- FIG. 15 does not constitute a limitation on a structure of the foregoing electronic apparatus.
- the computer terminal A may further include more or fewer components (for example, a network interface and a display apparatus) than those shown in FIG. 15 , or have a configuration different with that shown in FIG. 15 .
- the program may be stored in a computer-readable storage medium.
- the storage medium may include a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disc, and the like.
- An embodiment of this specification further provides a storage medium.
- the storage medium may be configured to store program code executed in the data processing method provided in the foregoing embodiments.
- the storage medium may be located in any computer terminal in a computer terminal cluster in a computer network, or in any mobile terminal in a mobile terminal cluster.
- the storage medium is configured to store program code used for performing the following steps: obtaining original video data and object information of a target object; processing the original video data based on the object information of the target object, to obtain a target image sequence of the target object; processing the original video data, to obtain a background image, where the background image is included in the original video data and does not include a detection object; and synthesizing the target image sequence and the background image, to obtain target video data.
- the storage medium is further configured to store program code used for performing the following steps: processing the original video data, to obtain a segmentation mask of at least one detection object; processing the original video data by using the segmentation mask of the at least one detection object, to obtain an image sequence of the at least one detection object; and determining the target image sequence based on the object information of the target object.
- the storage medium is further configured to store program code used for performing the following steps: detecting the original video data, to obtain at least one segmentation mask; processing the at least one segmentation mask by using a tracing algorithm, to obtain identification information of the at least one segmentation mask, where segmentation masks corresponding to the same identification information are masks of the same detection object; and obtaining the segmentation mask of the at least one detection object based on the identification information of the at least one segmentation mask.
- the object information includes at least a target attribute and a target time period
- the storage medium is further configured to store program code used for performing the following steps: recognizing the image sequence of the at least one detection object, to determine attribute information of the at least one detection object; matching the target attribute of the target object with the attribute information of the at least one detection object, to determine an image sequence of the target object; and obtaining an image sequence in the target time period from the image sequence of the target object, to obtain the target image sequence.
- the storage medium is further configured to store program code used for performing the following steps: determining a target location of the target image sequence in the target video data based on a preset video synthesis rule; and synthesizing the target image sequence and the background image based on the target location, to obtain the target video data.
- the storage medium is further configured to store program code used for performing the following steps: determining a length of the target video data based on a length of the target image sequence, and determining a first location of the target image sequence in the target video data; obtaining a loss value of the first location based on the preset video synthesis rule; determining, when the loss value of the first location is greater than a preset value, a second location of the target image sequence in the target video data, and using the second location as the first location until the loss value of the first location is less than the preset value; and determining the first location as the target location when the loss value of the first location is less than the preset value.
- the processor may call, by using the transmission apparatus, the information and the application program that are stored in the memory, to perform the following steps: obtaining object information of a target object; matching the object information of the target object with a pre-stored image sequence of at least one detection object, to obtain a target image sequence of the target object, where the image sequence of the at least one detection object is obtained by processing original video data; and synthesizing the target image sequence and a background image, to obtain target video data, where the background image is included in the original video data and does not include a detection object.
- the storage medium is further configured to store program code used for performing the following steps: processing the original video data, to obtain a segmentation mask of at least one detection object; processing the original video data by using the segmentation mask of the at least one detection object, to obtain an image sequence of the at least one detection object; recognizing the image sequence of the at least one detection object, to determine attribute information of the at least one detection object; and storing the image sequence and the attribute information of the at least one detection object.
- the storage medium is further configured to store program code used for performing the following steps: matching the target attribute of the target object with the attribute information of the at least one detection object, to determine an image sequence of the target object; and obtaining an image sequence in the target time period from the image sequence of the target object, to obtain the target image sequence.
- the storage medium is configured to store program code used for performing the following steps: displaying original video data and object information of a target object; and displaying target video data, where the target video data is obtained by synthesizing a target image sequence of the target object and a background image in the original video data, the target image sequence is obtained by processing the original video data based on the object information of the target object, and the background image is included in the original video data and does not include a detection object.
- the storage medium is configured to store program code used for performing the following steps: displaying the target image sequence and/or an acquisition time of the target image sequence in the target video data in a preset manner.
- the storage medium is configured to store program code used for performing the following steps: displaying object information of a target object; and displaying target video data, where the target video data is obtained by synthesizing a target image sequence of the target object and a background image in original video data, the target image sequence is obtained by matching the object information of the target object with a pre-stored image sequence of at least one detection object, and the background image is included in the original video data and does not include a detection object.
- the storage medium is configured to store program code used for performing the following steps: obtaining original video data; receiving a target object inputted by a client; obtaining a target image sequence corresponding to the target object from the original video data; and displaying the target image sequence in a fade-in and fade-out manner.
- the disclosed technical content may be implemented in other manners.
- the foregoing described apparatus embodiments are merely examples.
- the unit division is merely logical function division and there may be other division manners in actual implementation.
- multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed.
- the coupling, or direct coupling, or communication connection between the displayed or discussed components may be the indirect coupling or communication connection through some interfaces, units, or modules, and can be in electrical or other forms.
- the units described as separate parts may or may not be physically separate. Parts displayed as units may or may not be physical units, and may be located in one position, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual requirements to achieve the objectives of the solutions in the embodiments.
- functional units in the embodiments of this specification may be integrated into one processing unit, or each of the units can exist alone physically, or two or more units are integrated into one unit.
- the integrated unit may be implemented in the form of hardware, or may be implemented in the form of a software function unit.
- the integrated unit When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, the integrated unit may be stored in a computer-readable storage medium.
- the computer software product is stored in a storage medium and includes several instructions for instructing one or more computer devices (which may be a personal computer, a server, a network device, or the like) to perform all or some of the steps of the methods described in the embodiments of this specification.
- the above storage medium includes any medium that can store program code, such as a USB flash disk, a ROM, a RAM, a removable hard disk, a magnetic disk, or an optical disc.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Processing (AREA)
Abstract
Description
Claims (20)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010237158.6 | 2020-03-30 | ||
CN202010237158.6A CN113469200A (en) | 2020-03-30 | 2020-03-30 | Data processing method and system, storage medium and computing device |
Publications (2)
Publication Number | Publication Date |
---|---|
US20210304796A1 US20210304796A1 (en) | 2021-09-30 |
US11894021B2 true US11894021B2 (en) | 2024-02-06 |
Family
ID=77856616
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/202,751 Active 2041-10-08 US11894021B2 (en) | 2020-03-30 | 2021-03-16 | Data processing method and system, storage medium, and computing device |
Country Status (3)
Country | Link |
---|---|
US (1) | US11894021B2 (en) |
CN (1) | CN113469200A (en) |
WO (1) | WO2021202096A1 (en) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20230023359A (en) * | 2021-08-10 | 2023-02-17 | 한화테크윈 주식회사 | surveillance camera system |
CN114394100B (en) * | 2022-01-12 | 2024-04-05 | 深圳力维智联技术有限公司 | Unmanned patrol car control system and unmanned car |
CN115457113B (en) * | 2022-09-03 | 2023-05-23 | 北京津发科技股份有限公司 | Man-machine interaction movement track detection method, device, equipment and readable storage medium |
CN116193193A (en) * | 2022-12-28 | 2023-05-30 | 天翼云科技有限公司 | Video processing method and device |
CN117528262B (en) * | 2023-12-29 | 2024-04-05 | 江西赛新医疗科技有限公司 | Control method and system for data transmission of medical equipment |
Citations (39)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5870101A (en) | 1992-08-26 | 1999-02-09 | Namco Ltd. | Image synthesizing system with texture mapping |
US6215505B1 (en) | 1997-06-20 | 2001-04-10 | Nippon Telegraph And Telephone Corporation | Scheme for interactive video manipulation and display of moving object on background image |
US20020044603A1 (en) | 1998-01-26 | 2002-04-18 | Rajesh Rajagopalan | Two-pass encoding method of digital motion video sequences for constant-or variable bit rate |
US6621524B1 (en) | 1997-01-10 | 2003-09-16 | Casio Computer Co., Ltd. | Image pickup apparatus and method for processing images obtained by means of same |
US6940526B2 (en) | 2000-06-19 | 2005-09-06 | Fuji Photo Film Co., Ltd. | Image synthesizing apparatus |
US6987535B1 (en) | 1998-11-09 | 2006-01-17 | Canon Kabushiki Kaisha | Image processing apparatus, image processing method, and storage medium |
US7432953B2 (en) | 2003-09-02 | 2008-10-07 | Canon Kabushiki Kaisha | Image-taking apparatus detecting vibration and correcting image blurring |
US20090044136A1 (en) | 2007-08-06 | 2009-02-12 | Apple Inc. | Background removal tool for a presentation application |
US7511730B2 (en) | 2001-06-27 | 2009-03-31 | Sony Corporation | Image processing apparatus and method and image pickup apparatus |
US7623733B2 (en) | 2002-08-09 | 2009-11-24 | Sharp Kabushiki Kaisha | Image combination device, image combination method, image combination program, and recording medium for combining images having at least partially same background |
US20100073475A1 (en) | 2006-11-09 | 2010-03-25 | Innovative Signal Analysis, Inc. | Moving object detection |
US7843511B2 (en) | 2004-11-26 | 2010-11-30 | Lg Electronics Inc. | Apparatus and method for combining images in a terminal device |
US20100328452A1 (en) | 2009-04-13 | 2010-12-30 | Sang-Hack Jung | Method for pose invariant vessel fingerprinting |
US8125521B2 (en) | 2002-06-14 | 2012-02-28 | Canon Kabushiki Kaisha | Multiple image processing and synthesis using background image extraction |
US8170752B2 (en) | 2007-07-31 | 2012-05-01 | Kabushiki Kaisha Toyota Jidoshokki | Parking assistance apparatus, vehicle-side apparatus of parking assistance apparatus, parking assist method, and parking assist program |
US8633994B2 (en) | 2008-12-18 | 2014-01-21 | Casio Computer Co., Ltd. | Image pickup device, flash image generating method and computer-readable memory medium |
US20140153900A1 (en) * | 2012-12-05 | 2014-06-05 | Samsung Electronics Co., Ltd. | Video processing apparatus and method |
US20140347439A1 (en) | 2013-05-22 | 2014-11-27 | Nvidia Corporation | Mobile device and system for generating panoramic video |
US8941749B2 (en) | 2011-05-09 | 2015-01-27 | Canon Kabushiki Kaisha | Image processing apparatus and method for image synthesis |
US9020241B2 (en) | 2011-03-03 | 2015-04-28 | Panasonic Intellectual Property Management Co., Ltd. | Image providing device, image providing method, and image providing program for providing past-experience images |
US20150127626A1 (en) * | 2013-11-07 | 2015-05-07 | Samsung Tachwin Co., Ltd. | Video search system and method |
US9055277B2 (en) | 2011-03-31 | 2015-06-09 | Panasonic Intellectual Property Management Co., Ltd. | Image rendering device, image rendering method, and image rendering program for rendering stereoscopic images |
US9129414B2 (en) | 2011-10-14 | 2015-09-08 | Morpho, Inc. | Image compositing apparatus, image compositing method, image compositing program, and recording medium |
US9241112B2 (en) | 2010-09-03 | 2016-01-19 | Olympus Corporation | Imaging apparatus, imaging method and computer-readable recording medium |
US9398251B2 (en) | 2011-04-19 | 2016-07-19 | Samsung Electronics Co., Ltd | Apparatus and method for compositing image in a portable terminal |
US20160210516A1 (en) * | 2015-01-15 | 2016-07-21 | Hanwha Techwin Co., Ltd. | Method and apparatus for providing multi-video summary |
US20160323658A1 (en) * | 2014-01-26 | 2016-11-03 | BriefCam Ltd. | Method and system for producing relevance sorted video summary |
US9560271B2 (en) | 2013-07-16 | 2017-01-31 | Samsung Electronics Co., Ltd. | Removing unwanted objects from photographed image |
US9578260B2 (en) | 2011-12-21 | 2017-02-21 | Samsung Electronics Co., Ltd. | Digital photographing apparatus and method of controlling the digital photographing apparatus |
US20170140541A1 (en) | 2015-11-18 | 2017-05-18 | Yi-Chih Lu | Method for Identifying a Target Object in a Video File |
US10043120B2 (en) | 2014-10-27 | 2018-08-07 | Moon Key Lee | Translucent mark, method for synthesis and detection of translucent mark, transparent mark, and method for synthesis and detection of transparent mark |
US10171728B2 (en) | 2013-02-20 | 2019-01-01 | Sony Corporation | Image processing device, photographing control method, and program |
US10319410B1 (en) * | 2015-12-21 | 2019-06-11 | Amazon Technologies, Inc. | Video story assembly |
US10334162B2 (en) | 2014-08-18 | 2019-06-25 | Samsung Electronics Co., Ltd. | Video processing apparatus for generating panoramic video and method thereof |
US20190251364A1 (en) * | 2016-06-30 | 2019-08-15 | Kt Corporation | Image summarization system and method |
US20190303683A1 (en) * | 2015-09-24 | 2019-10-03 | Amazon Technologies, Inc. | Video summarization using selected characteristics |
US10623664B2 (en) * | 2014-06-30 | 2020-04-14 | Nec Corporation | Image processing system, image processing method and program storage medium for protecting privacy |
US20200143838A1 (en) * | 2018-11-02 | 2020-05-07 | BriefCam Ltd. | Method and system for automatic object-aware video or audio redaction |
US10958854B2 (en) * | 2018-11-02 | 2021-03-23 | BriefCam Ltd. | Computer-implemented method for generating an output video from multiple video sources |
-
2020
- 2020-03-30 CN CN202010237158.6A patent/CN113469200A/en active Pending
-
2021
- 2021-03-16 US US17/202,751 patent/US11894021B2/en active Active
- 2021-03-16 WO PCT/US2021/022509 patent/WO2021202096A1/en active Application Filing
Patent Citations (41)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5870101A (en) | 1992-08-26 | 1999-02-09 | Namco Ltd. | Image synthesizing system with texture mapping |
US6621524B1 (en) | 1997-01-10 | 2003-09-16 | Casio Computer Co., Ltd. | Image pickup apparatus and method for processing images obtained by means of same |
US6215505B1 (en) | 1997-06-20 | 2001-04-10 | Nippon Telegraph And Telephone Corporation | Scheme for interactive video manipulation and display of moving object on background image |
US20020044603A1 (en) | 1998-01-26 | 2002-04-18 | Rajesh Rajagopalan | Two-pass encoding method of digital motion video sequences for constant-or variable bit rate |
US6987535B1 (en) | 1998-11-09 | 2006-01-17 | Canon Kabushiki Kaisha | Image processing apparatus, image processing method, and storage medium |
US6940526B2 (en) | 2000-06-19 | 2005-09-06 | Fuji Photo Film Co., Ltd. | Image synthesizing apparatus |
US7511730B2 (en) | 2001-06-27 | 2009-03-31 | Sony Corporation | Image processing apparatus and method and image pickup apparatus |
US8125521B2 (en) | 2002-06-14 | 2012-02-28 | Canon Kabushiki Kaisha | Multiple image processing and synthesis using background image extraction |
US7623733B2 (en) | 2002-08-09 | 2009-11-24 | Sharp Kabushiki Kaisha | Image combination device, image combination method, image combination program, and recording medium for combining images having at least partially same background |
US7432953B2 (en) | 2003-09-02 | 2008-10-07 | Canon Kabushiki Kaisha | Image-taking apparatus detecting vibration and correcting image blurring |
US7843511B2 (en) | 2004-11-26 | 2010-11-30 | Lg Electronics Inc. | Apparatus and method for combining images in a terminal device |
US20100073475A1 (en) | 2006-11-09 | 2010-03-25 | Innovative Signal Analysis, Inc. | Moving object detection |
US8170752B2 (en) | 2007-07-31 | 2012-05-01 | Kabushiki Kaisha Toyota Jidoshokki | Parking assistance apparatus, vehicle-side apparatus of parking assistance apparatus, parking assist method, and parking assist program |
US20090044136A1 (en) | 2007-08-06 | 2009-02-12 | Apple Inc. | Background removal tool for a presentation application |
US8633994B2 (en) | 2008-12-18 | 2014-01-21 | Casio Computer Co., Ltd. | Image pickup device, flash image generating method and computer-readable memory medium |
US20100328452A1 (en) | 2009-04-13 | 2010-12-30 | Sang-Hack Jung | Method for pose invariant vessel fingerprinting |
US9241112B2 (en) | 2010-09-03 | 2016-01-19 | Olympus Corporation | Imaging apparatus, imaging method and computer-readable recording medium |
US9020241B2 (en) | 2011-03-03 | 2015-04-28 | Panasonic Intellectual Property Management Co., Ltd. | Image providing device, image providing method, and image providing program for providing past-experience images |
US9055277B2 (en) | 2011-03-31 | 2015-06-09 | Panasonic Intellectual Property Management Co., Ltd. | Image rendering device, image rendering method, and image rendering program for rendering stereoscopic images |
US9398251B2 (en) | 2011-04-19 | 2016-07-19 | Samsung Electronics Co., Ltd | Apparatus and method for compositing image in a portable terminal |
US8941749B2 (en) | 2011-05-09 | 2015-01-27 | Canon Kabushiki Kaisha | Image processing apparatus and method for image synthesis |
US9129414B2 (en) | 2011-10-14 | 2015-09-08 | Morpho, Inc. | Image compositing apparatus, image compositing method, image compositing program, and recording medium |
US9578260B2 (en) | 2011-12-21 | 2017-02-21 | Samsung Electronics Co., Ltd. | Digital photographing apparatus and method of controlling the digital photographing apparatus |
US20140153900A1 (en) * | 2012-12-05 | 2014-06-05 | Samsung Electronics Co., Ltd. | Video processing apparatus and method |
US10171728B2 (en) | 2013-02-20 | 2019-01-01 | Sony Corporation | Image processing device, photographing control method, and program |
US20140347439A1 (en) | 2013-05-22 | 2014-11-27 | Nvidia Corporation | Mobile device and system for generating panoramic video |
US9560271B2 (en) | 2013-07-16 | 2017-01-31 | Samsung Electronics Co., Ltd. | Removing unwanted objects from photographed image |
US20150127626A1 (en) * | 2013-11-07 | 2015-05-07 | Samsung Tachwin Co., Ltd. | Video search system and method |
US20160323658A1 (en) * | 2014-01-26 | 2016-11-03 | BriefCam Ltd. | Method and system for producing relevance sorted video summary |
US9877086B2 (en) * | 2014-01-26 | 2018-01-23 | BriefCam Ltd. | Method and system for producing relevance sorted video summary |
US10623664B2 (en) * | 2014-06-30 | 2020-04-14 | Nec Corporation | Image processing system, image processing method and program storage medium for protecting privacy |
US10334162B2 (en) | 2014-08-18 | 2019-06-25 | Samsung Electronics Co., Ltd. | Video processing apparatus for generating panoramic video and method thereof |
US10043120B2 (en) | 2014-10-27 | 2018-08-07 | Moon Key Lee | Translucent mark, method for synthesis and detection of translucent mark, transparent mark, and method for synthesis and detection of transparent mark |
US20160210516A1 (en) * | 2015-01-15 | 2016-07-21 | Hanwha Techwin Co., Ltd. | Method and apparatus for providing multi-video summary |
US20190303683A1 (en) * | 2015-09-24 | 2019-10-03 | Amazon Technologies, Inc. | Video summarization using selected characteristics |
US20170140541A1 (en) | 2015-11-18 | 2017-05-18 | Yi-Chih Lu | Method for Identifying a Target Object in a Video File |
US10319410B1 (en) * | 2015-12-21 | 2019-06-11 | Amazon Technologies, Inc. | Video story assembly |
US20190251364A1 (en) * | 2016-06-30 | 2019-08-15 | Kt Corporation | Image summarization system and method |
US10949675B2 (en) * | 2016-06-30 | 2021-03-16 | Kt Corporation | Image summarization system and method |
US20200143838A1 (en) * | 2018-11-02 | 2020-05-07 | BriefCam Ltd. | Method and system for automatic object-aware video or audio redaction |
US10958854B2 (en) * | 2018-11-02 | 2021-03-23 | BriefCam Ltd. | Computer-implemented method for generating an output video from multiple video sources |
Non-Patent Citations (2)
Title |
---|
Tsai et al., "Independent component analysis-based background subtraction for indorr surveillance," IEEE Transaction on image processing 18.1, Nov. 18, 2008. |
Written Opinion and International Search Report for PCT Application No. PCT/US2021/022509 dated May 27, 2021. |
Also Published As
Publication number | Publication date |
---|---|
WO2021202096A1 (en) | 2021-10-07 |
US20210304796A1 (en) | 2021-09-30 |
CN113469200A (en) | 2021-10-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11894021B2 (en) | Data processing method and system, storage medium, and computing device | |
AU2018250370B2 (en) | Weakly supervised model for object detection | |
US11630861B2 (en) | Method and apparatus for video searching, terminal and storage medium | |
US20180213289A1 (en) | Method of authorizing video scene and metadata | |
JP2019531548A (en) | Video capture framework for visual search platform | |
US10147230B2 (en) | Dynamic video visualization | |
CN105631393A (en) | Information recognition method and device | |
US11768597B2 (en) | Method and system for editing video on basis of context obtained using artificial intelligence | |
US20220172476A1 (en) | Video similarity detection method, apparatus, and device | |
WO2021088422A1 (en) | Application message notification method and device | |
CN111814885A (en) | Method, system, device and medium for managing image frames | |
CN111683267A (en) | Method, system, device and storage medium for processing media information | |
CN109116718B (en) | Method and device for setting alarm clock | |
US20170013309A1 (en) | System and method for product placement | |
US20230377331A1 (en) | Media annotation with product source linking | |
US11989209B2 (en) | Electronic device, and computer-readable storage medium | |
CN115687692A (en) | Video processing method and device, computer storage medium and intelligent interactive panel | |
CN116452702B (en) | Information chart rapid design method, device, computer equipment and storage medium | |
CN115147856B (en) | Table information extraction method and electronic equipment | |
JP6410329B2 (en) | Video recording creation apparatus, video recording creation method and program | |
US20230119785A1 (en) | System and method for personalizing a product content | |
CN114973219A (en) | Text content extraction method and device | |
CN112732464A (en) | Pasting method and device and electronic equipment | |
CN116775924A (en) | Image display control method, device and equipment | |
CN114756710A (en) | Video tag obtaining method and device, electronic equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: ALIBABA GROUP HOLDING LIMITED, CAYMAN ISLANDS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:YANG, ZHE;REEL/FRAME:055605/0517 Effective date: 20210302 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |