CN109960731B - Data processing method, device and storage medium - Google Patents
Data processing method, device and storage medium Download PDFInfo
- Publication number
- CN109960731B CN109960731B CN201910244412.2A CN201910244412A CN109960731B CN 109960731 B CN109960731 B CN 109960731B CN 201910244412 A CN201910244412 A CN 201910244412A CN 109960731 B CN109960731 B CN 109960731B
- Authority
- CN
- China
- Prior art keywords
- position information
- data
- fragments
- sub
- fragment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/41—Indexing; Data structures therefor; Storage structures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a data processing method, a device and a storage medium, wherein the method comprises the following steps: acquiring a header file fragment aiming at target multimedia data from a fragment set; acquiring absolute position information of each frame of data in the target multimedia data according to the header file fragments, determining relative position information of sub-fragments corresponding to each frame of data in the target multimedia data according to the absolute position information, and taking the relative position information of the sub-fragments as information to be matched; matching the information to be matched with the standard relative position information contained in all the sub-fragments in the fragment set to obtain the address information of the sub-fragment to which the successfully matched standard relative position information belongs; and generating a multimedia fragment index corresponding to the target multimedia data according to the incidence relation between the address information and the target multimedia data. By adopting the invention, the possibility of data loss can be reduced, and the data security is improved.
Description
Technical Field
The present invention relates to the field of electronic technologies, and in particular, to a data processing method, device, and storage medium.
Background
With continuous development and improvement of information technology, a distributed storage system is more and more popular in use, and in order to increase storage and distribution efficiency, the distributed storage system generally decomposes a large file into a plurality of small files, then numbers the small files to form a file index to be stored in a database, for example, the large file (a video file of dozens of MB or hundreds of MB) is decomposed into N small files of 512KB in sequence to be stored and distributed, when the large file is accessed, only the file index of the large file needs to be known, each small file is found according to the index, then the small files are assembled into the large file, and then the large file is accessed. However, due to system failure or human operation error, the file index is sometimes lost, and the loss of the file index cannot cause that small files cannot be recombined into a large file, so that all stored data is lost, even huge economic loss is caused.
Disclosure of Invention
Embodiments of the present invention provide a data processing method, device, and storage medium, which can reduce the possibility of data loss and improve the security of data.
An aspect of an embodiment of the present invention provides a data processing method, which may include:
acquiring a header file fragment aiming at target multimedia data from a fragment set; the fragment set is used for storing a plurality of unit data fragments, and the unit data fragments comprise a plurality of header file fragments and sub-fragments of multimedia data;
acquiring absolute position information of each frame of data in the target multimedia data according to the header file fragments, determining the relative position information of the sub-fragments corresponding to each frame of data in the target multimedia data according to the absolute position information, and taking the relative position information of the sub-fragments as information to be matched;
matching the information to be matched with the standard relative position information contained in all the sub-fragments in the fragment set to obtain the address information of the sub-fragment to which the successfully matched standard relative position information belongs;
and generating a multimedia fragment index corresponding to the target multimedia data according to the association relationship between the address information and the target multimedia data.
Wherein, the acquiring header file fragments aiming at the target multimedia data from the fragment set comprises:
detecting the file header information of all unit data fragments in the fragment set, and determining the unit data fragments of which the file header information is a first target field as the head file fragments of the target multimedia data.
The acquiring absolute position information of each frame of data in the target multimedia data according to the header file fragment, determining the relative position information of the sub-fragment corresponding to each frame of data in the target multimedia data according to the absolute position information, and using the relative position information of the sub-fragment as information to be matched includes:
acquiring a second target field from the header file fragment; the second target field comprises absolute position information of each frame of data in the target multimedia data;
acquiring relative position information corresponding to each frame of data in the target multimedia data in the sub-fragments according to the absolute position information and the memory size information of the data fragments of the target multimedia data, and taking the relative position information as information to be matched;
and storing the absolute position information and the relative position information.
The acquiring absolute position information of each frame of data in the target multimedia data according to the header file fragment, determining the relative position information of the sub-fragment corresponding to each frame of data in the target multimedia data according to the absolute position information, and after taking the relative position information of the sub-fragment as information to be matched, further includes:
according to frame header synchronous word information of the data fragments, standard relative position information of each frame data in all the sub-fragments in the sub-fragments is obtained from the fragment set;
and acquiring address information of all sub-fragments according to the storage address of the fragment set, and storing the standard relative position information and the address information.
The matching the information to be matched with the standard relative position information contained in the sub-slice in the slice set to obtain the address information of the sub-slice to which the successfully matched standard relative position information belongs includes:
searching standard relative position information which is the same as the relative position information from standard relative position information contained in sub-fragments in the fragment set, determining the sub-fragments to which the standard relative position information which is the same as the relative position information belongs as target sub-fragments, and if the number of the target sub-fragments meets a preset number condition, acquiring the address information of the target sub-fragments.
Wherein, still include:
and if the number of the target sub-fragments does not meet the preset number condition, outputting prompt information of multimedia fragment index reconstruction failure.
Generating a multimedia fragment index corresponding to the target multimedia data according to the association relationship between the address information and the target multimedia data, including:
and arranging the address information according to the incidence relation between the address information and the data fragments of the target multimedia data and the playing sequence of the target sub-fragments, and generating a multimedia fragment index corresponding to the multimedia data according to the arranged address information.
An aspect of an embodiment of the present invention provides a data processing apparatus, which may include:
the fragment acquisition unit is used for acquiring header file fragments aiming at the target multimedia data from the fragment set; the fragment set is used for storing a plurality of unit data fragments, and the unit data fragments comprise a plurality of header file fragments and sub-fragments of multimedia data;
a position information obtaining unit, configured to obtain absolute position information of each frame of data in the target multimedia data according to the header file fragment, determine, according to the absolute position information, sub-fragment relative position information corresponding to each frame of data in the target multimedia data, and use the sub-fragment relative position information as information to be matched;
a first address information obtaining unit, configured to match the information to be matched with standard relative position information included in all sub-slices in the slice set, and obtain address information of a sub-slice to which the successfully matched standard relative position information belongs;
and the index generating unit is used for generating a multimedia slicing index corresponding to the target multimedia data according to the incidence relation between the address information and the target multimedia data.
Wherein, the fragment acquiring unit is specifically configured to:
and detecting the file header information of all unit data fragments in the fragment set, and determining the unit data fragments of which the file header information is a first target field as the header file fragments of the target multimedia data.
Wherein the position information acquiring unit includes:
an absolute position information obtaining subunit, configured to obtain a second target field from the header file fragment; the second destination field comprises absolute position information of each frame of data in the destination multimedia data;
a relative position information obtaining subunit, configured to obtain, according to the absolute position information and memory size information of a data fragment of the target multimedia data, relative position information corresponding to each frame of data in the target multimedia data in the sub-fragment, and use the relative position information as information to be matched;
a first storage subunit, configured to store the absolute position information and the relative position information.
Wherein, still include:
a standard position information obtaining unit, configured to obtain, from the fragmentation set, standard relative position information of each frame data in all the sub-fragments in the sub-fragment to which the frame header synchronization word information of the data fragment belongs;
and the second address information acquisition unit is used for acquiring the address information of all sub-fragments according to the storage address of the fragment set and storing the standard relative position information and the address information.
The first address information obtaining unit is specifically configured to:
searching standard relative position information which is the same as the relative position information from standard relative position information contained in sub-slices in the slice set, determining sub-slices to which the standard relative position information which is the same as the relative position information belongs as target sub-slices, and if the number of the target sub-slices meets a preset number condition, acquiring address information of the target sub-slices.
Wherein, still include:
and the output unit is used for outputting prompt information of multimedia fragment index reconstruction failure if the number of the target sub-fragments does not meet the preset number condition.
Wherein the index generation unit is specifically configured to:
and arranging the address information according to the incidence relation between the address information and the data fragments of the target multimedia data and the playing sequence of the target sub-fragments, and generating a multimedia fragment index corresponding to the multimedia data according to the arranged address information.
An aspect of the embodiments of the present invention provides a computer storage medium storing a plurality of instructions adapted to be loaded by a processor and to perform the above-mentioned method steps.
In one aspect, an embodiment of the present invention provides a data processing apparatus, including a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the steps of:
acquiring a header file fragment aiming at target multimedia data from a fragment set; the fragment set is used for storing a plurality of unit data fragments, and the unit data fragments comprise a plurality of header file fragments and sub-fragments of multimedia data;
acquiring absolute position information of each frame of data in the target multimedia data according to the header file fragments, determining relative position information of sub-fragments corresponding to each frame of data in the target multimedia data according to the absolute position information, and taking the relative position information of the sub-fragments as information to be matched;
matching the information to be matched with the standard relative position information contained in all the sub-fragments in the fragment set to obtain the address information of the sub-fragment to which the successfully matched standard relative position information belongs;
and generating a multimedia fragment index corresponding to the target multimedia data according to the incidence relation between the address information and the target multimedia data.
In the embodiment of the invention, header file fragments aiming at target multimedia data are obtained from the fragment set; acquiring absolute position information of each frame of data in the target multimedia data according to the header file fragments, determining relative position information of sub-fragments corresponding to each frame of data in the target multimedia data according to the absolute position information, and taking the relative position information of the sub-fragments as information to be matched; matching the information to be matched with the standard relative position information contained in all the sub-fragments in the fragment set to obtain the address information of the sub-fragment to which the successfully matched standard relative position information belongs; and generating a multimedia fragment index corresponding to the target multimedia data according to the incidence relation between the address information and the target multimedia data. The file index of the large file to which the small file belongs is reconstructed through the position information of each frame of data in the small file, so that the problem that the large file cannot be read due to the loss of the file index is avoided, the possibility of data loss is reduced, and the safety of data is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flowchart of a data processing method according to an embodiment of the present invention;
fig. 2 is a schematic diagram illustrating an example of a data processing method according to an embodiment of the present invention;
FIG. 3 is a flow chart of another data processing method provided by the embodiment of the invention;
fig. 4 is a schematic structural diagram of a data processing device according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of another data processing apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a location information obtaining unit according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of another data processing apparatus according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.
The data processing method provided by the embodiment of the invention can be applied to the reconstruction of the file index in the distributed storage system, and specifically, the data processing equipment acquires the header file fragment aiming at the target multimedia data from the fragment set; acquiring absolute position information of each frame of data in the target multimedia data according to the header file fragments, determining relative position information of sub-fragments corresponding to each frame of data in the target multimedia data according to the absolute position information, and taking the relative position information of the sub-fragments as information to be matched; matching the information to be matched with the standard relative position information contained in all the sub-fragments in the fragment set to obtain the address information of the sub-fragment to which the successfully matched standard relative position information belongs; and generating a multimedia fragment index corresponding to the target multimedia data according to the incidence relation between the address information and the target multimedia data. The file index of the large file to which the small file belongs is reconstructed through the position information of each frame of data in the small file, so that the problem that the large file cannot be read due to the loss of the file index is avoided, the possibility of data loss is reduced, and the safety of data is improved.
The data processing apparatus according to an embodiment of the present invention may include: terminal devices such as tablet computers, personal Computers (PCs), notebook computers, and Mobile Internet Devices (MIDs).
Referring to fig. 1, a flow chart of a data processing method according to an embodiment of the present invention is shown. As shown in fig. 1, the method of the embodiment of the present invention may include the following steps S101 to S104.
S101, acquiring header file fragments aiming at target multimedia data from a fragment set; the fragment set is used for storing a plurality of unit data fragments, and the unit data fragments comprise a plurality of header file fragments and sub-fragments of multimedia data;
specifically, the data processing device acquires a header file fragment for the target multimedia data from the fragment set; the slice set is configured to store a plurality of unit data slices, the plurality of unit data slices includes a plurality of header file slices and sub-slices of multimedia data, it is understood that the slice set stores a plurality of unit data slices, the slice set may collectively be a storage space in a distributed storage system, the unit data slices may be header file slices of the multimedia data or sub-slices of the multimedia data, the unit data slices are small files into which large files are decomposed by using a slice storage technique, the header file slices are unit data slices storing encoding encapsulation characteristics of the multimedia data and absolute position information of each frame data in the multimedia data, the sub-slices are unit data slices storing partial data frame information after the decomposition of the multimedia data, the multimedia data is a Video file or an audio file, for example, the multimedia data may be an H264 or Advanced Video Coding (mp4) encapsulated file in a Moving Picture Experts Group (Moving Picture Group) or an AVC Video file, when the multimedia data is stored in the distributed storage system, the header file slices and the sub-slices may be obtained from one multimedia data slice, and the multimedia data slices are reconstructed from one multimedia data slice set.
S102, acquiring absolute position information of each frame of data in the target multimedia data according to the header file fragments, determining relative position information of sub-fragments corresponding to each frame of data in the target multimedia data according to the absolute position information, and taking the relative position information of the sub-fragments as information to be matched;
specifically, the data processing device obtains absolute position information of each frame data in the target multimedia data according to the header fragment, determines, according to the absolute position information, relative position information of a sub-fragment corresponding to each frame data in the target multimedia data, and uses the relative position information of the sub-fragment as information to be matched, it can be understood that information related to the absolute position information of each frame data in the target multimedia data is stored in the header fragment, for example, the target multimedia data is an MP 4-encapsulated H264 or AVC video file, a "stco" field exists in the header fragment, an absolute position information of each frame in the video file is stored in the "stco" field, the absolute position information is a position of each frame data in the target multimedia file, the relative position information is a position of each frame data in the target multimedia data in the data fragment, please refer to fig. 2 together, and an exemplary schematic diagram of a data processing method is provided for an embodiment of the present invention. As shown in fig. 2, the target multimedia file is decomposed to generate a header file fragment and a plurality of sub-fragments, the header file fragment stores absolute position information of each frame of data in the target multimedia data, the position information of each frame of data in the sub-fragments is relative position information, the position information of each frame of data in the target multimedia data is different from the position information in the sub-fragments, the data processing device may calculate, according to the absolute position information, the relative position information of the sub-fragments corresponding to each frame of data in the target multimedia data by using a preset algorithm, and use the relative position information of the sub-fragments as information to be matched, which is also exemplified by the target multimedia data, the unit fragment length of the target multimedia data is a fixed value, a modulo operation is performed on a difference between the absolute position information and the unit fragment length, and a result after the modulo operation is the relative position information.
S103, matching the information to be matched with the standard relative position information contained in all the sub-fragments in the fragment set to obtain the address information of the sub-fragment to which the successfully matched standard relative position information belongs;
specifically, the data processing device matches the information to be matched with the standard relative position information included in all sub-slices in the slice set, and obtains the address information of the sub-slice to which the successfully matched standard relative position information belongs, it can be understood that the data processing device obtains the standard relative position information of all sub-slices in the slice set, the standard relative position information is the relative position information of each frame of data in all sub-slices in the slice set in the sub-slice to which the frame of data belongs, the information to be matched is matched with the standard relative position information, specifically, the standard relative position information identical to the relative position information is searched for from the standard relative position information, the data processing device obtains the address information of the sub-slice to which the successfully matched standard relative position information belongs, and the address information is the storage address of the sub-slice in the storage space.
And S104, generating a multimedia slicing index corresponding to the target multimedia data according to the incidence relation between the address information and the target multimedia data.
Specifically, the data processing device generates a multimedia fragment index corresponding to the target multimedia data according to an association relationship between the address information and the target multimedia data, where it can be understood that the multimedia data includes multiple unit data fragments, each unit data fragment corresponds to one address information, the association relationship is a correspondence relationship between the address information of a unit data fragment in the target multimedia data and the target multimedia data, and generates the multimedia fragment index corresponding to the target multimedia data according to the association relationship, where the multimedia fragment index is address information corresponding to all unit data fragments in the target multimedia data.
In the embodiment of the invention, header file fragments aiming at target multimedia data are obtained from the fragment set; acquiring absolute position information of each frame of data in the target multimedia data according to the header file fragments, determining relative position information of sub-fragments corresponding to each frame of data in the target multimedia data according to the absolute position information, and taking the relative position information of the sub-fragments as information to be matched; matching the information to be matched with the standard relative position information contained in all the sub-fragments in the fragment set to obtain the address information of the sub-fragment to which the successfully matched standard relative position information belongs; and generating a multimedia fragment index corresponding to the target multimedia data according to the incidence relation between the address information and the target multimedia data. The file index of the large file to which the small file belongs is reconstructed through the position information of each frame of data in the small file, so that the problem that the large file cannot be read due to the loss of the file index is avoided, the possibility of data loss is reduced, and the safety of data is improved.
Referring to fig. 3, a flow chart of a data processing method according to an embodiment of the invention is shown. As shown in fig. 3, the method of the embodiment of the present invention may include the following steps S201 to S209.
S201, detecting the file header information of all unit data fragments in a fragment set, and determining the unit data fragments of which the file header information is a first target field as the header file fragments of target multimedia data;
specifically, the data processing apparatus detects header information of all unit data slices in a slice set, and determines a unit data slice with the header information being a first target field as a header file slice of target multimedia data, where it is understood that the header information is a segment of a character string that determines a type of the unit data slice, and header information corresponding to multimedia data encoded and encapsulated in different ways is different, and the first target field is header information corresponding to the target multimedia data, and searches header information of all unit data slices in the slice set, and determines the header information of the unit data slice as the first target field as the header file slice of the target multimedia data, for example, the target multimedia data is an MP 264 or AVC video file encapsulated with 4 bytes, and 4 th to 8 th bytes of the header file slice of the target multimedia data are certainly an "ftyp" character string, and in the slice set, 4 th to 8 th bytes of the unit data slice are an "ftyp" character string as the header file of the target multimedia data.
S202, acquiring a second target field from the header file fragment; the second destination field comprises absolute position information of each frame of data in the destination multimedia data;
specifically, the data processing device acquires a second target field from the header file fragment; the second destination field includes absolute position information of each frame of data in the destination multimedia data, and it can be understood that the second destination field is a field in a header file fragment, and second destination fields corresponding to multimedia files encoded and encapsulated in different manners are different, as described above for the destination multimedia data, the second destination field is a "stco" field, and the "stco" field includes absolute position information of each frame of data in the destination multimedia data.
S203, acquiring corresponding relative position information of each frame of data in the target multimedia data in the sub-fragments according to the absolute position information and the memory size information of the data fragments of the target multimedia data, and taking the relative position information as information to be matched;
specifically, the data processing device obtains, according to the absolute position information and memory size information of a data fragment of the target multimedia data, relative position information corresponding to each frame of data in the target multimedia data in a sub-fragment, and uses the relative position information as information to be matched, it can be understood that the memory size information of the data fragment is a memory size of a unit data fragment, the multimedia data may include a plurality of unit data fragments, the memory size information of each unit data fragment is the same, the data processing device calculates, according to the absolute position information and the memory size information, the relative position information of the sub-fragment corresponding to each frame of data in the target multimedia data by using a preset algorithm, and uses the relative position information of the sub-fragment as information to be matched, which is also exemplified by the target multimedia data, the unit fragment length of the target multimedia data is 512k, a modulo operation is performed on a difference between the absolute position information and the unit fragment length, a result after the modulo operation is the relative position information, and the relative position information is used as information to be matched.
S204, storing the absolute position information and the relative position information;
in particular, the data processing device stores the absolute position information and the relative position information, it being understood that the absolute position information and the relative position information may be stored in a document or other storage space.
S205, according to frame header synchronous word information of the data fragments, standard relative position information of each frame data in all sub-fragments in the sub-fragments is obtained from the fragment set;
specifically, the data processing device obtains standard relative position information of each frame data in all sub-slices in the sub-slice from the slice set according to frame header synchronization word information of the data slices, and it can be understood that frame header synchronization word information corresponding to multimedia data encoded and encapsulated in different ways is different, the frame header synchronization word information corresponds to the multimedia data, for example, the target multimedia data is an H264 or AVC video file encapsulated by MP4, the frame header synchronization word is "0x000000", a byte with a character string as the frame header synchronization word is searched in the sub-slice according to the frame header synchronization word information of the data slice, the byte is determined as the frame header relative position information of one frame data, and the standard relative position information of each frame data in all sub-slices in the sub-slice set is obtained from the slice set according to the frame header synchronization word information.
S206, acquiring address information of all sub-fragments according to the storage address of the fragment set, and storing the standard relative position information and the address information;
specifically, the data processing device obtains address information of all sub-slices according to a storage address of the slice set, and stores the standard relative position information and the address information, it can be understood that the storage address of the slice set is address information of the slice set in a storage space, the storage address includes address information of each unit data slice in the slice set, the address information of all sub-slices is obtained according to the storage address of the slice set, the standard relative position information of each frame of data in each sub-slice and the address information corresponding to each sub-slice are stored, and the standard relative position information and the address information can be stored in a document or other storage spaces.
S207, searching standard relative position information which is the same as the relative position information from standard relative position information contained in sub-slices in the slice set, determining sub-slices to which the standard relative position information which is the same as the relative position information belongs as target sub-slices, and if the number of the target sub-slices meets a preset number condition, acquiring address information of the target sub-slices;
specifically, the data processing device searches standard relative position information that is the same as the relative position information from standard relative position information included in sub-slices in the slice set, determines a sub-slice to which the standard relative position information that is the same as the relative position information belongs as a target sub-slice, and acquires address information of the target sub-slice if the number of the target sub-slices satisfies a preset number condition, it may be understood that the data processing device searches standard relative position information that is the same as the relative position information from the standard relative position information included in all sub-slices in the slice set, determines a sub-slice to which the standard relative position information that is the same as the relative position information belongs as a target sub-slice, and detects the number of the target sub-slices, and acquires address information corresponding to the target sub-slice if the number of the target sub-slices satisfies a preset number condition, the preset number condition is preset, and may be adjusted according to actual circumstances, for example, if the preset number condition is 1, when it is detected that the number of the target sub-slices is 1, it means that only one sub-slice in the target sub-slice corresponds to the standard relative position information, and it may be determined that the target sub-slice corresponds to the target sub-slice, and the target sub-slice corresponds to the target sub-slice.
S208, if the number of the target sub-fragments does not meet the preset number condition, outputting prompt information of multimedia fragment index reconstruction failure;
specifically, if the number of the target sub-segments does not satisfy the preset number condition, the data processing device outputs a prompt message indicating that the reconstruction of the multimedia segment index fails, where it may be understood that the prompt message may be a voice prompt message or a text box, for example, if the preset number condition is that the number of the target sub-segments is 1, the data processing device outputs a prompt message indicating that the reconstruction of the multimedia segment index fails when the target sub-segments are not detected or a plurality of target sub-segments are detected.
S209, arranging the address information according to the playing sequence of the target sub-fragments according to the incidence relation between the address information and the data fragments of the target multimedia data, and generating a multimedia fragment index corresponding to the multimedia data according to the arranged address information.
Specifically, the data processing device arranges the address information according to the association relationship between the address information and the data fragments of the target multimedia data, and generates a multimedia fragment index corresponding to the multimedia data according to the arranged address information, as can be understood, the multimedia data includes multiple unit data fragments, each unit data fragment corresponds to one address information, the association relationship is the correspondence between the address information of a unit data fragment in the target multimedia data and the target multimedia data, the corresponding address information of the target sub-fragment is arranged according to the playing order of the target sub-fragment, the multimedia fragment index corresponding to the target multimedia data is generated according to the arranged address information, when the target multimedia file is accessed, only the multimedia fragment index corresponding to the target multimedia data needs to be acquired, all the unit data fragments corresponding to the target multimedia data are acquired according to the address information of each unit data fragment in the multimedia fragment index, and then the unit data fragments are assembled into the target multimedia file.
In the embodiment of the invention, header file fragments aiming at target multimedia data are obtained from the fragment set; acquiring absolute position information of each frame of data in the target multimedia data according to the header file fragments, determining relative position information of sub-fragments corresponding to each frame of data in the target multimedia data according to the absolute position information, and taking the relative position information of the sub-fragments as information to be matched; matching the information to be matched with the standard relative position information contained in all the sub-fragments in the fragment set to obtain the address information of the sub-fragment to which the successfully matched standard relative position information belongs; and generating a multimedia fragment index corresponding to the target multimedia data according to the association relationship between the address information and the target multimedia data. The address information of the small file in the storage space is obtained through the position information of each frame of data in the small file, the file index of the large file to which the small file belongs is reconstructed, and a user is informed of the result of the reconstruction of the file index, so that the problem that the large file cannot be read due to the loss of the file index is avoided, the possibility of data loss is reduced, and the safety of data is improved.
Fig. 4 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present invention. As shown in fig. 4, the data processing apparatus 1 according to the embodiment of the present invention may include: a slice acquiring unit 11, a position information acquiring unit 12, a first address information acquiring unit 13, and an index generating unit 14.
The fragment acquiring unit 11 is configured to acquire a header file fragment for target multimedia data from a fragment set; the fragment set is used for storing a plurality of unit data fragments, and the unit data fragments comprise a plurality of header file fragments and sub-fragments of multimedia data;
a position information obtaining unit 12, configured to obtain absolute position information of each frame of data in the target multimedia data according to the header file fragment, determine, according to the absolute position information, relative position information of a sub-fragment corresponding to each frame of data in the target multimedia data, and use the relative position information of the sub-fragment as information to be matched;
a first address information obtaining unit 13, configured to match the information to be matched with the standard relative position information included in all the sub-slices in the slice set, and obtain address information of a sub-slice to which the successfully matched standard relative position information belongs;
and an index generating unit 14, configured to generate a multimedia slicing index corresponding to the target multimedia data according to the association relationship between the address information and the target multimedia data.
In the embodiment of the invention, header file fragments aiming at target multimedia data are obtained from the fragment set; acquiring absolute position information of each frame of data in the target multimedia data according to the header file fragments, determining relative position information of sub-fragments corresponding to each frame of data in the target multimedia data according to the absolute position information, and taking the relative position information of the sub-fragments as information to be matched; matching the information to be matched with the standard relative position information contained in all the sub-fragments in the fragment set to obtain the address information of the sub-fragment to which the successfully matched standard relative position information belongs; and generating a multimedia fragment index corresponding to the target multimedia data according to the association relationship between the address information and the target multimedia data. The file index of the large file to which the small file belongs is reconstructed through the position information of each frame of data in the small file, so that the problem that the large file cannot be read due to the loss of the file index is avoided, the possibility of data loss is reduced, and the safety of data is improved.
Fig. 5 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present invention. As shown in fig. 5, the data processing apparatus 1 according to the embodiment of the present invention may include: a slice acquiring unit 11, a position information acquiring unit 12, a first address information acquiring unit 13, an index generating unit 14, a standard position information acquiring unit 15, a second address information acquiring unit 16, and an output unit 17.
The fragment acquisition unit 11 is configured to detect header information of all unit data fragments in a fragment set, and determine a unit data fragment whose header information is a first target field as a header file fragment of target multimedia data;
a position information obtaining unit 12, configured to obtain absolute position information of each frame of data in the target multimedia data according to the header file fragment, determine, according to the absolute position information, relative position information of a sub-fragment corresponding to each frame of data in the target multimedia data, and use the relative position information of the sub-fragment as information to be matched;
referring to fig. 6, a schematic structural diagram of the position information obtaining unit 12 is provided for the embodiment of the present invention. As shown in fig. 6, the location information acquiring unit 12 according to an embodiment of the present invention may include: an absolute position information acquisition subunit 121, a relative position information acquisition subunit 122, and a first storage subunit 123.
An absolute position information obtaining subunit 121, configured to obtain a second target field from the header file fragment; the second destination field comprises absolute position information of each frame of data in the destination multimedia data;
a relative position information obtaining subunit 122, configured to obtain, according to the absolute position information and the memory size information of the data segment of the target multimedia data, relative position information corresponding to each frame of data in the target multimedia data in the data segment, and use the relative position information as information to be matched;
a first storage subunit 123, configured to store the absolute position information and the relative position information;
a standard position information obtaining unit 15, configured to obtain, from the fragment set, standard relative position information of each frame data in all the sub-fragments in the sub-fragment to which the frame data belongs according to frame header synchronization word information of the data fragments;
a second address information obtaining unit 16, configured to obtain address information of all sub-slices according to the storage address of the slice set, and store the standard relative position information and the address information;
a first address information obtaining unit 13, configured to search standard relative position information that is the same as the relative position information from standard relative position information included in sub-slices in the slice set, determine a sub-slice to which the standard relative position information that is the same as the relative position information belongs as a target sub-slice, and if the number of the target sub-slices meets a preset number condition, obtain address information of the target sub-slice;
an output unit 17, configured to output a prompt message indicating that the multimedia fragment index reconstruction fails if the number of the target sub-fragments does not meet a preset number condition;
and an index generating unit 14, configured to arrange the address information according to a playing sequence of the target sub-segment according to an association relationship between the address information and the data segment of the target multimedia data, and generate a multimedia segment index corresponding to the multimedia data according to the arranged address information.
In the embodiment of the invention, header file fragments aiming at target multimedia data are obtained from the fragment set; acquiring absolute position information of each frame of data in the target multimedia data according to the header file fragments, determining relative position information of sub-fragments corresponding to each frame of data in the target multimedia data according to the absolute position information, and taking the relative position information of the sub-fragments as information to be matched; matching the information to be matched with the standard relative position information contained in all the sub-fragments in the fragment set to obtain the address information of the sub-fragment to which the successfully matched standard relative position information belongs; and generating a multimedia fragment index corresponding to the target multimedia data according to the incidence relation between the address information and the target multimedia data. The address information of the small file in the storage space is obtained through the position information of each frame of data in the small file, the file index of the large file to which the small file belongs is reconstructed, and a user is informed of the result of the reconstruction of the file index, so that the problem that the large file cannot be read due to the loss of the file index is avoided, the possibility of data loss is reduced, and the safety of data is improved.
An embodiment of the present invention further provides a computer storage medium, where the computer storage medium may store a plurality of instructions, where the instructions are suitable for being loaded by a processor and executing the method steps in the embodiments shown in fig. 1 to fig. 2, and a specific execution process may refer to specific descriptions of the embodiments shown in fig. 1 to fig. 2, which are not described herein again.
Fig. 7 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present invention. As shown in fig. 7, the data processing apparatus 1000 may include: at least one processor 1001, such as a CPU, at least one network interface 1004, a user interface 1003, memory 1005, at least one communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), and the optional user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 1005 may optionally be at least one memory device located remotely from the processor 1001. As shown in fig. 7, the memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a data processing application program.
In the terminal 1000 shown in fig. 7, the user interface 1003 is mainly used as an interface for providing input for a user, and acquiring data input by the user; and the processor 1001 may be configured to invoke a data processing application stored in the memory 1005 and specifically perform the following operations:
acquiring a header file fragment aiming at target multimedia data from a fragment set; the fragment set is used for storing a plurality of unit data fragments, and the unit data fragments comprise a plurality of header file fragments and sub-fragments of multimedia data;
acquiring absolute position information of each frame of data in the target multimedia data according to the header file fragments, determining relative position information of sub-fragments corresponding to each frame of data in the target multimedia data according to the absolute position information, and taking the relative position information of the sub-fragments as information to be matched;
matching the information to be matched with the standard relative position information contained in all the sub-fragments in the fragment set to obtain the address information of the sub-fragment to which the successfully matched standard relative position information belongs;
and generating a multimedia fragment index corresponding to the target multimedia data according to the incidence relation between the address information and the target multimedia data.
In the embodiment of the invention, header file fragments aiming at target multimedia data are obtained from the fragment set; acquiring absolute position information of each frame of data in the target multimedia data according to the header file fragments, determining relative position information of sub-fragments corresponding to each frame of data in the target multimedia data according to the absolute position information, and taking the relative position information of the sub-fragments as information to be matched; matching the information to be matched with the standard relative position information contained in all the sub-fragments in the fragment set to obtain the address information of the sub-fragment to which the successfully matched standard relative position information belongs; and generating a multimedia fragment index corresponding to the target multimedia data according to the incidence relation between the address information and the target multimedia data. The file index of the large file to which the small file belongs is reconstructed through the position information of each frame of data in the small file, so that the problem that the large file cannot be read due to the loss of the file index is avoided, the possibility of data loss is reduced, and the safety of data is improved.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention, and it is therefore to be understood that the invention is not limited by the scope of the appended claims.
Claims (10)
1. A data processing method, comprising:
acquiring a header file fragment aiming at target multimedia data from a fragment set; the fragment set is used for storing a plurality of unit data fragments, and the unit data fragments comprise a plurality of header file fragments and sub-fragments of multimedia data;
acquiring absolute position information of each frame of data in the target multimedia data according to the header file fragments, determining relative position information of sub-fragments corresponding to each frame of data in the target multimedia data according to the absolute position information, and taking the relative position information of the sub-fragments as information to be matched;
matching the information to be matched with the standard relative position information contained in all the sub-fragments in the fragment set to obtain the address information of the sub-fragment to which the successfully matched standard relative position information belongs;
and generating a multimedia fragment index corresponding to the target multimedia data according to the incidence relation between the address information and the target multimedia data.
2. The method according to claim 1, wherein the obtaining the header file slice for the target multimedia data from the slice set comprises:
and detecting the file header information of all unit data fragments in the fragment set, and determining the unit data fragments of which the file header information is a first target field as the header file fragments of the target multimedia data.
3. The method according to claim 1, wherein the obtaining absolute position information of each frame of data in the target multimedia data according to the header file fragment, determining relative position information of a sub-fragment corresponding to each frame of data in the target multimedia data according to the absolute position information, and using the relative position information of the sub-fragment as information to be matched comprises:
acquiring a second target field from the header file fragment; the second destination field comprises absolute position information of each frame of data in the destination multimedia data;
acquiring relative position information corresponding to each frame of data in the target multimedia data in the sub-fragments according to the absolute position information and the memory size information of the data fragments of the target multimedia data, and taking the relative position information as information to be matched;
and storing the absolute position information and the relative position information.
4. The method according to claim 1, wherein the obtaining absolute position information of each frame of data in the target multimedia data according to the header file fragment, determining relative position information of a sub-fragment corresponding to each frame of data in the target multimedia data according to the absolute position information, and after taking the relative position information of the sub-fragment as information to be matched, further comprises:
according to frame header synchronous word information of the data fragments, standard relative position information of each frame data in all sub-fragments in the sub-fragments is obtained from the fragment set;
and acquiring address information of all sub-fragments according to the storage address of the fragment set, and storing the standard relative position information and the address information.
5. The method according to claim 4, wherein the matching the information to be matched with the standard relative position information included in the sub-slices in the slice set to obtain the address information of the sub-slice to which the successfully matched standard relative position information belongs comprises:
searching standard relative position information which is the same as the relative position information from standard relative position information contained in sub-fragments in the fragment set, determining the sub-fragments to which the standard relative position information which is the same as the relative position information belongs as target sub-fragments, and if the number of the target sub-fragments meets a preset number condition, acquiring the address information of the target sub-fragments.
6. The method of claim 5, further comprising:
and if the number of the target sub-fragments does not meet the preset number condition, outputting prompt information of multimedia fragment index reconstruction failure.
7. The method according to claim 1, wherein the generating a multimedia slice index corresponding to the target multimedia data according to the association relationship between the address information and the target multimedia data comprises:
and arranging the address information according to the incidence relation between the address information and the data fragments of the target multimedia data and the playing sequence of the target sub-fragments, and generating a multimedia fragment index corresponding to the multimedia data according to the arranged address information.
8. A data processing apparatus, characterized by comprising:
the fragment acquisition unit is used for acquiring header file fragments aiming at the target multimedia data from the fragment set; the fragment set is used for storing a plurality of unit data fragments, and the unit data fragments comprise a plurality of header file fragments and sub-fragments of multimedia data;
a position information obtaining unit, configured to obtain absolute position information of each frame of data in the target multimedia data according to the header file fragment, determine, according to the absolute position information, relative position information of a sub-fragment corresponding to each frame of data in the target multimedia data, and use the relative position information of the sub-fragment as information to be matched;
a first address information obtaining unit, configured to match the information to be matched with standard relative position information included in all sub-slices in the slice set, and obtain address information of a sub-slice to which the successfully matched standard relative position information belongs;
and the index generating unit is used for generating a multimedia slicing index corresponding to the target multimedia data according to the incidence relation between the address information and the target multimedia data.
9. A computer storage medium, characterized in that it stores a plurality of instructions adapted to be loaded by a processor and to perform the method steps according to any one of claims 1 to 7.
10. A data processing apparatus characterized by comprising: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the steps of:
acquiring a header file fragment aiming at target multimedia data from a fragment set; the fragment set is used for storing a plurality of unit data fragments, and the unit data fragments comprise a plurality of header file fragments and sub-fragments of multimedia data;
acquiring absolute position information of each frame of data in the target multimedia data according to the header file fragments, determining relative position information of sub-fragments corresponding to each frame of data in the target multimedia data according to the absolute position information, and taking the relative position information of the sub-fragments as information to be matched;
matching the information to be matched with the standard relative position information contained in all the sub-fragments in the fragment set to obtain the address information of the sub-fragment to which the successfully matched standard relative position information belongs;
and generating a multimedia fragment index corresponding to the target multimedia data according to the incidence relation between the address information and the target multimedia data.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910244412.2A CN109960731B (en) | 2019-03-28 | 2019-03-28 | Data processing method, device and storage medium |
| PCT/CN2019/103348 WO2020192012A1 (en) | 2019-03-28 | 2019-08-29 | Data processing method and apparatus, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910244412.2A CN109960731B (en) | 2019-03-28 | 2019-03-28 | Data processing method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109960731A CN109960731A (en) | 2019-07-02 |
| CN109960731B true CN109960731B (en) | 2022-11-18 |
Family
ID=67025158
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910244412.2A Active CN109960731B (en) | 2019-03-28 | 2019-03-28 | Data processing method, device and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109960731B (en) |
| WO (1) | WO2020192012A1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109960731B (en) * | 2019-03-28 | 2022-11-18 | 腾讯音乐娱乐科技（深圳）有限公司 | Data processing method, device and storage medium |
| CN110704536A (en) * | 2019-09-27 | 2020-01-17 | 北京迈格威科技有限公司 | Distributed data processing method, device and system and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018014523A1 (en) * | 2016-07-18 | 2018-01-25 | 华为技术有限公司 | Media data acquisition method and apparatus |
| CN108391179A (en) * | 2018-04-26 | 2018-08-10 | 腾讯科技（深圳）有限公司 | live data processing method, device, server, terminal and storage medium |
| CN108737908A (en) * | 2018-05-21 | 2018-11-02 | 腾讯科技（深圳）有限公司 | A kind of media playing method, device and storage medium |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101198016A (en) * | 2007-12-05 | 2008-06-11 | 中兴通讯股份有限公司 | Contents issuance and storage method for interactive individual television media paying system |
| CN102845041B (en) * | 2011-04-21 | 2015-11-25 | 华为技术有限公司 | Based on multi code Rate of Chinese character positioning playing method, the equipment and system of HTML (Hypertext Markup Language) |
| US10645425B2 (en) * | 2014-06-13 | 2020-05-05 | Samsung Electronics Co., Ltd. | Method and device for managing multimedia data |
| CN104506966B (en) * | 2014-12-10 | 2018-01-30 | 曙光信息产业(北京)有限公司 | The coding/decoding method and device of audio, video data |
| CN108093299B (en) * | 2017-12-22 | 2020-08-04 | 厦门市美亚柏科信息股份有限公司 | Method for repairing damaged MP4 file and storage medium |
| CN109960731B (en) * | 2019-03-28 | 2022-11-18 | 腾讯音乐娱乐科技（深圳）有限公司 | Data processing method, device and storage medium |
-
2019
- 2019-03-28 CN CN201910244412.2A patent/CN109960731B/en active Active
- 2019-08-29 WO PCT/CN2019/103348 patent/WO2020192012A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018014523A1 (en) * | 2016-07-18 | 2018-01-25 | 华为技术有限公司 | Media data acquisition method and apparatus |
| CN108391179A (en) * | 2018-04-26 | 2018-08-10 | 腾讯科技（深圳）有限公司 | live data processing method, device, server, terminal and storage medium |
| CN108737908A (en) * | 2018-05-21 | 2018-11-02 | 腾讯科技（深圳）有限公司 | A kind of media playing method, device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109960731A (en) | 2019-07-02 |
| WO2020192012A1 (en) | 2020-10-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107395209B (en) | Data compression method, data decompression method and equipment thereof | |
| US10210064B2 (en) | Systems and methods for device compatibility testing and reporting | |
| US9009460B2 (en) | Node computing data encryption method | |
| CN106936441B (en) | Data compression method and device | |
| US20110066723A1 (en) | Generating statistics of popular content | |
| CN110716895A (en) | Target data archiving method and device, computer equipment and medium | |
| US10476928B2 (en) | Network video playback method and apparatus | |
| US9355250B2 (en) | Method and system for rapidly scanning files | |
| CN109960731B (en) | Data processing method, device and storage medium | |
| CN106874348A (en) | File is stored and the method for indexing means, device and reading file | |
| CN112866730B (en) | Streaming media data transmission method, device, system, electronic equipment and storage medium | |
| US20140164574A1 (en) | Method and System for Downloading a Font File | |
| WO2017075493A1 (en) | Video frame difference engine | |
| CN112839003A (en) | Data verification method and system | |
| CN110888918A (en) | Similar data detection method and device, computer equipment and storage medium | |
| CN112436943B (en) | Request deduplication method, device, equipment and storage medium based on big data | |
| CN111506747B (en) | File analysis method, device, electronic equipment and storage medium | |
| CN113704206B (en) | Metadata processing method and device, electronic equipment and storage medium | |
| CN113111200B (en) | Method, device, electronic equipment and storage medium for auditing picture files | |
| CN104637496A (en) | Computer system and audio comparison method | |
| CN105469433B (en) | Picture compression method and equipment thereof | |
| CN106961626A (en) | The method and apparatus that a kind of video metamessage auto-complete is arranged | |
| CN107291763B (en) | Electronic document management method and management device | |
| US20140344305A1 (en) | System and method for managing related information of audio content | |
| CN118277628B (en) | Data access control method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |