CN101720041A - Hierarchical coding-based data processing method - Google Patents
Hierarchical coding-based data processing method Download PDFInfo
- Publication number
- CN101720041A CN101720041A CN 200910223268 CN200910223268A CN101720041A CN 101720041 A CN101720041 A CN 101720041A CN 200910223268 CN200910223268 CN 200910223268 CN 200910223268 A CN200910223268 A CN 200910223268A CN 101720041 A CN101720041 A CN 101720041A
- Authority
- CN
- China
- Prior art keywords
- data
- storage
- layer
- server
- storage server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses a hierarchical coding-based data processing method, which comprises that: a coder acquires a storage plan and codes data hierarchy into a basic layer and at least one reinforcing layer according to data hierarchical information; and the coder stores the data of the basic layer and the at least one reinforcing layer to corresponding storage resources in a storage server respectively. In the method, the storage efficiency of the storage server is improved.
Description
Technical field
The present invention relates to communication technical field, particularly relate to a kind of data processing method based on hierarchical coding.
Background technology
In supervisory control system, in order to adapt to the heterogeneous networks situation of decoding end, encoder need adopt the hierarchical coding technology usually, terminal image is divided into basic layer and reinforced layer, decoding end then can be regulated the receiving ability of self according to network presence, thereby the information of real-time reception different aspects (for example, basic layer, reinforced layer etc.).Wherein, this hierarchical coding technology comprises SNR (Signal to Noise Ratio, signal to noise ratio) hierarchical coding technology, Temporal (time) hierarchical coding technology, Spatial (space) hierarchical coding technology etc.
The hierarchical coding schematic diagram of a kind of encoder as shown in Figure 1, this hierarchical coding technology provide a kind of and (for example, the environment of network congestion often occurred) in heterogeneous network environments, effectively realize the solution of dynamic coding and transmission.In addition, because the premium properties of hierarchical coding technology makes that the code stream behind the coding has higher code efficiency and good transmission reliability.
Further, as shown in Figure 2, be the storing process schematic diagram in a kind of supervisory control system, in this supervisory control system, set up index structure by video server, encoder and IPSAN (Storage AreaNetwork, storage area network) set up ISCSI (Internet Small Computer System Interface, the Internet small computer system interface) connects, and write data by encoder corresponding logical resource in IPSAN; And playback decoding client (being decoding end) connects with IPSAN under the control of video server, and from the corresponding logical resource of IPSAN reading of data.
To the video information of storage when backing up, carry out the efficient that independent backup can improve backup by using the information of hierarchical coding technology on will basic layer, making the hierarchical coding technology be applied in the storage of monitoring video and the playback becomes trend of the times.
In the prior art, under the control of video server, encoder and IPSAN connect, logical resource corresponding in IPSAN writes data, the decoding client connects with corresponding IPSAN, and from IPSAN reading of data in the corresponding logical resource, at this moment, the data of each layer (basic layer and reinforced layer etc.) are to be stored together.
In addition, this video server can be set up 2 grades of index structures, and data are to carry out index according to 2 grades of index structures setting up in the video server.Wherein, these 2 grades of index are to GOP (the Group Of Picture of each second, image sets) and the EGOP of each reinforced layer (enhancing image sets) carry out index simultaneously, and the base layer data in one second of every correspondence in these 2 grades of index and this second pairing reinforced layer data; Owing to the sign of end is arranged between GOP and EGOP, promptly can distinguish GOP and EGOP.The schematic diagram of every content structure in a kind of 2 grades of index as shown in Figure 3, in Fig. 3, EI is the data message of the I frame of reinforced layer, the data message of P frame that EP is reinforced layer.
But in the storing process of existing supervisory control system, when the logical resource of encoder correspondence in IPSAN write data, because the access bandwidth of encoder, the restriction of the performance bottleneck of IPSAN self and array status etc. can cause storing losing of video recording; And because the data of storage video recording need very high accuracy, it is bigger to cause storing code stream, makes that then the data of storage video recording are subject to the constraint of network throughput, storage system throughput when playback fast, and playback speed is affected.
Further, when the lack of memory capacity of IPSAN, if desired storage resources is put in order (for example, need the storage resources that the part reinforced layer takies being discharged), realize very complicated; When stored resource is backed up, also can't effectively back up the backup inefficiency at key message; In addition, when needs disperse to back up, can't be effectively at the backup that walks abreast of base layer data and key stratum data.
Summary of the invention
The invention provides a kind of data processing method, with the efficient of raising monitoring storage system and the adaptability of network based on hierarchical coding.
In order to achieve the above object, the present invention proposes a kind of data processing method, be applied to comprise in the system of encoder and storage server, said method comprising the steps of based on hierarchical coding:
Described encoder obtains storage plan, has carried the data hierarchy information of a basic layer and at least one reinforced layer in the described storage plan;
Described encoder is encoded into a basic layer and at least one reinforced layer according to described data hierarchy information with data hierarchy;
Described encoder stores the data of a described basic layer and at least one reinforced layer respectively on the storage resources corresponding in the described storage server.
Described encoder stores described basic layer of data with at least one reinforced layer respectively on the storage resources corresponding in the described storage server and specifically comprises:
Described encoder stores base layer data on the basic layer of corresponding storage resources of described storage server;
Described encoder with the reinforced layer storage to the storage resources of the reinforced layer correspondence of described storage server.
Described method also comprises:
When the memory property of described storage server was lower than predetermined threshold value, described encoder received the information of the preferential storage base layer data of described storage server notice;
Described encoder preferentially stores base layer data on the basic layer of corresponding storage resources of described storage server.
A kind of data processing method based on hierarchical coding is applied to comprise in the system of encoder and storage server, said method comprising the steps of:
Described storage server obtains storage policy, has carried the data hierarchy information of a basic layer and at least one reinforced layer in the described storage policy;
Described storage server is that the data of a basic layer and at least one reinforced layer are set up storage resources respectively according to described data hierarchy information;
Described storage server receives the data from described encoder, and the base layer data in the described data is stored on the corresponding storage resources of basic layer; With the reinforced layer storage to the storage resources of reinforced layer correspondence.
Also comprise:
Described storage server detects self memory property, and when described memory property is lower than preset first threshold value,
The base layer data that described storage server is preferentially stored described encoder by the corresponding storage resources of basic layer; And/or,
Described storage server notifies described encoder preferentially base layer data to be stored on the corresponding storage resources of described basic layer.
Specifically comprise on the storage resources that described storage server notifies described encoder preferentially base layer data to be stored into described basic layer correspondence:
Described storage server notifies described encoder preferentially base layer data to be stored on the corresponding storage resources of described basic layer by the asynchronous message of ISCSI.
Also comprise:
Described storage server detects self storage resources, and when described storage resources is lower than default second threshold value,
The preferential storage resources that discharges described reinforced layer correspondence of described storage server is removed the reinforced layer data in the storage resources of described reinforced layer correspondence.
A kind of data processing method based on hierarchical coding, be applied to comprise in the system of video server, encoder and storage server, storage plan and storage policy have been pre-established in the described video server, all carried the data hierarchy information of a basic layer and at least one reinforced layer in described storage plan and the storage policy, said method comprising the steps of:
Described video server sends to described encoder with described storage plan, according to described storage plan data hierarchy is encoded into a basic layer and at least one reinforced layer by described encoder;
And described storage policy sent to described storage server, be that the data of a basic layer and at least one reinforced layer are set up storage resources respectively by described storage server according to described storage policy.
Also comprise:
The indexed mode of described video server employing hierarchical coding is that the storage resources on the described storage server is set up index; Be specially:
Described video server is set up basic layer index for the pairing storage resources of basic layer, and sets up the reinforced layer index for the pairing storage resources of reinforced layer, and described basic layer index and described reinforced layer index have corresponding relation.
A kind of data processing method based on hierarchical coding is applied to comprise in the system of storage server and backup server, said method comprising the steps of:
Described backup server is that the data of a basic layer and at least one reinforced layer are set up storage resources respectively;
Described backup server backs up base layer data in the described storage server by using the corresponding storage resources of basic layer; Perhaps, described backup server independently backs up reinforced layer data in the described storage server by the storage resources that uses the corresponding storage resources of basic layer and independently back up the base layer data in the described storage server and pass through use reinforced layer correspondence.
Also comprise:
Described backup server detects self storage resources, and when described storage resources is lower than predetermined threshold value,
The preferential storage resources that discharges described reinforced layer correspondence of described backup server is removed the reinforced layer data in the storage resources of described reinforced layer correspondence.
A kind of data processing method based on hierarchical coding is applied to comprise in the system of storage server and client, said method comprising the steps of:
Described client is obtained the quality of data rank that self needs, and obtains corresponding data readback mode according to described quality of data rank;
Described client is obtained base layer data at least according to described data readback mode from described storage server.
Described client is obtained base layer data at least according to described data readback mode and is specifically comprised from described storage server:
When described data readback mode was minimum quality playback mode, described client was obtained base layer data according to basic layer index from described storage server; Perhaps,
When described data readback mode is first water playback mode, described client is obtained base layer data according to basic layer index from described storage server, and obtains all reinforced layer data according to pairing all the reinforced layer index of described basic layer index from described storage server; Perhaps,
When described data readback mode is other quality playback modes, described client is obtained base layer data according to basic layer index from described storage server, and obtains part reinforced layer data according to the pairing part reinforced layer of described basic layer index index from described storage server.
Compared with prior art, the present invention has the following advantages: by base layer data and reinforced layer data are stored into respectively on the corresponding stored resource of storage server, improved the storage efficiency of storage server, the equipment in the system that makes can be handled respectively base layer data and reinforced layer data, has improved the network-adaptive of supervisory control system.
Description of drawings
Fig. 1 is the hierarchical coding schematic diagram of encoder in the prior art;
Fig. 2 is the storing process schematic diagram in the supervisory control system in the prior art;
Fig. 3 is the content structure schematic diagram of each in a kind of 2 grades of index in the prior art;
A kind of data processing method flow chart that Fig. 4 proposes for the present invention based on hierarchical coding;
A kind of data processing method flow chart that Fig. 5 proposes down for application scenarios of the present invention based on hierarchical coding;
Fig. 6 connects the idiographic flow schematic diagram that writes data on the storage resources of correspondence for encoder under the application scenarios of the present invention by ISCSI;
The indexed mode schematic diagram that Fig. 7 separates the data of basic layer and reinforced layer down for application scenarios of the present invention;
Fig. 8 is the asynchronous message schematic diagram under the application scenarios of the present invention;
A kind of data handling system structure chart that Fig. 9 proposes for the present invention based on hierarchical coding;
A kind of coder structure figure that Figure 10 proposes for the present invention;
A kind of storage server structure chart that Figure 11 proposes for the present invention;
A kind of video server structure chart that Figure 12 proposes for the present invention;
A kind of backup server structure chart that Figure 13 proposes for the present invention;
A kind of client terminal structure figure that Figure 14 proposes for the present invention.
Embodiment
Basic thought of the present invention is when data are carried out hierarchical coding, result according to hierarchical coding sets up storage resources at storage server, be basic layer and at least one reinforced layer and set up storage resources respectively, and use the corresponding storage resources storage base layer data of basic layer, use the storage resources storage reinforced layer data of reinforced layer correspondence; Then the equipment in the system that makes can be handled respectively base layer data and reinforced layer data, thereby has improved the network-adaptive and the treatment effeciency of supervisory control system.
Concrete, when the storage server self performance is not enough, on storage server, can only store base layer data, thereby guarantee under all stored situation of base layer data the reinforced layer data optionally to be stored.
When storage server self insufficient space, can under the situation of not losing base layer data, preferentially discharge the storage resources of reinforced layer correspondence, promptly remove the reinforced layer data in the storage resources of reinforced layer correspondence, thereby save the space resources of storage server, improved the arrangement efficient in space.
When data are backed up, by on backup server, independently backing up base layer data and reinforced layer data, can under user's control, back up as required the data on the storage server, only back up base layer data, or backup base layer data and part reinforced layer data, improve the efficient of backup and the independence of backup, and reduced the use amount in space.
When data were carried out playback, the user can control the replayed section of client, selected different playback modes according to the quality of data of self needs, thereby improved the efficient of playback.Promptly, improve the speed and the video recording accuracy of playing back videos greatly by base layer data and reinforced layer data are separated.
As shown in Figure 4, a kind of data processing method based on hierarchical coding that the present invention proposes is applied to comprise in the system of encoder and storage server, said method comprising the steps of:
Below in conjunction with a kind of concrete application scenarios, the data processing method based on hierarchical coding that the present invention is proposed is elaborated, these data are at data such as video, audio frequency, for convenience of description, all describe with data instance among the present invention, this application scenarios is that example describes with the processing procedure in the supervisory control system shown in Figure 2; This data processing method is applied in the hierarchical coding technology, and this hierarchical coding is meant and encodes the data to a basic layer and one or more reinforced layer; A lower but acceptable quality should basic layer be provided, and each additional reinforced layer progressively improves the quality.As shown in Figure 5, should may further comprise the steps based on the data processing method of hierarchical coding:
Step 501 is formulated storage plan, storage policy and priority policy on video server.Wherein, this storage plan, storage policy and priority policy are to be provided with arbitrarily according to actual needs.
This storage plan is the plan of storing based on the data of hierarchical coding for this, for example, for the data of passage 1, stores between 9:00-10:00; Again for example, for the data of passage 1, data are divided into a basic layer and two reinforced layers (among the present invention be that example describe in this hierarchical coding mode).
This storage policy is the storage mode that is provided with for storage server (being IPSAN), for example, the storage plan that is divided into a basic layer and two reinforced layers for the data of passage 1, this storage policy can be at encoder passage 1, set up the storage resources of a basic layer and two reinforced layers, one of them storage resources is used to store basic layer information, and two storage resources are used to store reinforced layer information.
In addition, the video server storage policy that is handed down to storage server also comprises:
When disk array (being storage server) performance is not enough, require which layer data information of the preferential storage coder of storage server; For example, require the data message of the basic layer of the preferential storage of storage server.
When the disk array inadequate resource, require storage server preferentially which data resource to be put in order; For example, require the data message of the basic layer of the preferential arrangement of storage server.
Further, video server also needs to issue storage policy and gives backup server, and this backup server is among the IPSAN, corresponding to the backup server of above-mentioned storage server; This storage policy that is handed down to backup server comprises:
In the time need backing up to the data on the storage resources (the storage server correspondence), require backup server which passage is carried out the basic layer of only backup, which passage is carried out the backup of basic layer and a plurality of reinforced layers.
As can be seen, formulation by this storage policy, and in follow-up storage and backup procedure, use this storage policy, can guarantee that storage server and backup server can preferentially store base layer data, and on the complete basis of base layer data, according to real-time information such as disk array state, disk performance, network throughput and user configured priority, the reinforced layer data are stored and backed up, improved storage and backup performance significantly, guaranteed to handle at critical data.
This priority policy is to be the set precedence information of the pairing passage of data, and for example, passage 1 has the highest priority, and passage 2 when the storage data, needs the data of preferential memory channel 1 when having minimum priority.
In this step, video server informs that by storage plan encoder need be divided into data (corresponding to passage 1) a basic layer and two reinforced layers; Inform that by storage policy storage server is the storage resources that encoder passage 1 is set up a basic layer and two reinforced layers.
Concrete, encoder passage 1 is divided into a basic layer and two reinforced layers by hierarchical coding with data (storage flow), respectively 3 SANDISK resources on the corresponding stored resource (being positioned at storage server), one of them deposits basic layer information, deposits reinforced layer information for two.
Need to prove, encoder is not limited to know this storage plan among the present invention from video server, in the practical application, this storage plan can directly be set on encoder, also can storage plan be sent to this encoder by other equipment, promptly encoder knows that the mode of storage plan can select arbitrarily, knows that with encoder this storage plan is that example describes from video server among the present invention, for other obtain manner, give unnecessary details no longer in detail.
Same, storage server also is not limited to knows this storage policy and priority policy from video server, in the practical application, this storage policy and priority policy can directly be set on storage server, also can storage policy and priority policy be sent to storage server by other equipment, be that storage server knows that the mode of storage policy and priority policy can select arbitrarily, know that with storage server this storage policy and priority policy are that example describes from video server among the present invention, for other obtain manner, give unnecessary details no longer in detail.
In this step, the logical volume on the storage server is carried out the index initialization be specially the storage resources on the storage server is carried out index.Wherein, when the storage resources on the storage server is carried out index, video server will adopt the indexed mode of hierarchical coding, and the data that are about to basic layer and reinforced layer are separately carried out index, wherein, in the process of carrying out this step, just set up the storage resources on the storage server is carried out the corresponding relation of index, and in subsequent step, also need to handle accordingly according to the corresponding relation of this foundation, this process will describe in detail in follow-up step, repeat no more in this step.
In addition, the logical volume on the storage server is carried out after the index initialization, on video server, storage server and the encoder all with the logical volume corresponding relation of memory channel and resource, a kind of corresponding relation as shown in table 1; Certainly, according to the actual needs, the content in can also his-and-hers watches 1 is made amendment, and repeats no more among the present invention.
Table 1
| Channel number | The IP of storage server | Corresponding Target information | Basic layer logical volume | The first reinforced layer logical volume | The second reinforced layer logical volume |
| ??ECaChannel1 | ??IP | ??targetID | ??LUNA | ??LUNB | ??LUNC |
As can be seen, by table 1 as can be known, encoder can be by storage server the IP address with the storage among the ECaChannel1 of self to storage server, being about to base layer data stores among the logical volume LUNA, with the first reinforced layer storage in logical volume LUNB, with the second reinforced layer storage in logical volume LUNC.
In this step, on the storage resources of correspondence, write data by the ISCSI connection and comprise:
(1) encoder obtains the logical volume corresponding relation of memory channel and resource.
Concrete, video server is when formulating storage plan, need on storage server, create target, under this target, distribute the logical resource corresponding, and corresponding information (i.e. the logical volume corresponding relation of memory channel shown in the table 1 and resource) is sent to encoder with coding pass.
(2) encoder is inquired about the memory channel that this video server issues and the logical volume corresponding relation of resource, obtains the IP address and the targetID of passage institute corresponding stored server.
(3) encoder is initiated the ISCSIlogon connection request according to the IP address and the targetID of storage server to storage server, obtains logical volume information.
(4) encoder is initiated the ISCSI connection to storage server, and after connection is set up successfully, according to the basic layer logic reel number in the logical volume corresponding relation of memory channel and resource and the logic reel number of each reinforced layer, send to different logical resources respectively and write request of data, and base layer data stores in the basic layer logical volume the most at last, with the reinforced layer storage in the reinforced layer logical volume of correspondence.
Concrete, as shown in Figure 6, for above-mentioned encoder connects the idiographic flow schematic diagram that writes data on the storage resources of correspondence by ISCSI, this process is existing execution mode, gives unnecessary details no longer in detail in this step.
Need to prove that this step also needs to write the result according to this index is upgraded in the process that writes data into the corresponding stored resource, promptly the pairing index relative of setting up in the above-mentioned steps 502 of storage resources is upgraded, as shown in Figure 7.
When under master index, the pairing data of passage being carried out index, can only index the data of basic layer, and each basic layer is to have corresponding relation with self reinforced layer, thus can be according to index search of this basic layer index to reinforced layer.
Concrete index information comprises: master index carries out index to the data block of every X MB, data and the position thereof of expression a period of time; Secondary index carries out index to GOP, in each index entry of secondary index, increases the pairing reinforced layer EGOP of this index entry basic layer GOP information information, and is as shown in table 2.
| The time of this GOP | Skew | The piece number that this GOP uses | Strengthen |
Strengthen |
Concrete, carry out respective handling according to the storage policy of on storage server, storing and include but not limited to:
(1) when detecting performance of disk arrays and be lower than preset first threshold value (optional according to actual needs numerical value), show the performance of disk arrays deficiency, at this moment, according to storage policy, this storage server will preferentially be stored the data of basic layer.
Concrete, owing to also stored priority policy on the storage server, in this step, storage server can carry out the storage of basic layer and reinforced layer information to the data channel of high priority, data channel to low priority then abandons certain reinforced layer information or whole reinforced layer information, and the information of only storing basic layer.Certainly, in actual applications, if when performance of disk arrays is very low, can also be to all data channel, information of the basic layer of storage only all.
(2) be lower than default second threshold value and (be optional numerical value according to actual needs when detecting performance of disk arrays, can be identical with above-mentioned preset first threshold value, also can be different) time, show the performance of disk arrays deficiency, at this moment, according to storage policy, the data notification of which layer that this storage server needs self is given encoder, is only got final product to the data of storage server storage respective layer by encoder.For example, when storage server only needs basic layer of information of memory channel 1, notice encoder self only needs the basic layer information of memory channel 1, after encoder receives this message, only need will basic layer information stores to storage server, and no longer needing, thereby the network traffics on the minimizing network to storage server storage reinforced layer information.
Concrete, the basic layer information (is example with this situation) that storage server only need need self notifies the Initiator encoder of basic layer logical volume correspondence on this disk array to get final product, and this encoder only writes base layer data after receiving message in corresponding logical volume.
Need to prove that the basic layer information that this storage server can the asynchronous message by ISCSI needs self is notified to the Initiator encoder of basic layer logical volume correspondence on this disk array; A kind of asynchronous message schematic diagram as shown in Figure 8, the logic reel number that preceding 4 bit representations in the Reserved field of this asynchronous message go wrong, for example, preceding 4 of Reserved is 0000, expression only need write basic layer information, and 0001 expression need write the information of the basic layer and first reinforced layer.Certainly, this advice method is not limited to the Reserved field of asynchronous message, can also use other fields or other message informings of asynchronous message, explanation no longer in detail among the present invention.
Concrete, when storage server detects disk array resource (being the disk array space) when not enough, need put in order storage resources according to storage policy (requiring the data of the basic layer of the preferential arrangement of storage server), promptly guaranteeing under the situation that base layer data is not lost, remove the reinforced layer data of self storing, thereby saved the disk array resource of self.
In this step, when removing the reinforced layer data of self storing, storage server can directly remove the reinforced layer pointer content of secondary index, and the pairing SANDISK resource of reinforced layer is put in order, promptly can remove the data of reinforced layer.
Need to prove, when only backing up base layer data, can save the storage resources of backup server.And when backup base layer data and reinforced layer data, base layer data and reinforced layer data need be backuped to respectively independently on the logical resource, at this moment, base layer data need be backuped on the most reliable resource; Wherein, during backup reinforced layer data, will be in conjunction with the index information of basic layer, set up the index of each reinforced layer, be that video server has been set up index structure (as shown in Figure 7) on the logical resource of backup reinforced layer, according to this index structure of setting up, carry out the backup of reinforced layer data then, make when reading the reinforced layer data, write the index information of reinforced layer.
Need to prove, this backup server need detect in real time to self disk array resource equally, when backup server detects disk array resource (being the disk array space) when not enough, need put in order storage resources according to storage policy (requiring the data of the basic layer of the preferential arrangement of backup server), promptly guaranteeing under the situation that base layer data is not lost, remove the reinforced layer data that self back up, thereby saved the disk array resource of self.
Concrete, before this step was carried out, this client need be obtained the corresponding relation of basic layer, reinforced layer and logic reel number from video server.When needs base layer data, reinforced layer data, this client need get access to base layer data, the reinforced layer data of correspondence according to this corresponding relation from the logical volume of storage server correspondence.
Among the present invention, client can be selected the playback mode according to the needs of self, and for example, when client only needed other quality of data of lowermost level, this client can be selected quick playback mode, and promptly client only need be obtained base layer data from storage server; And when client needed the quality of data of highest level, this client can be selected full playback mode, and promptly this client need be obtained base layer data and all corresponding reinforced layer data from storage server.
Certainly, in actual applications, because the quality of data is to be divided into different quality scales according to the hierarchical coding mode, for example, for the hierarchical coding mode of a basic layer and three reinforced layers, it is other to be divided into four kinds of quality-classes; Be respectively minimum data quality scale, first quality of data rank, second quality of data rank and the maximum data quality scale, wherein, quality scale is high more, and the quality of data is high more.
Because the playback mode of client is corresponding with the quality of data, as can be seen, playback mode among the present invention is not limited to quick playback mode and full playback mode, for corresponding with the quality of data, among the present invention the playback mode is defined as minimum quality playback mode, the first quality playback mode ... first water playback mode, by that analogy, several qualities of data are arranged, then corresponding several playback modes.For example, in the hierarchical coding mode of an above-mentioned basic layer and three reinforced layers, corresponding playback mode is respectively minimum quality playback mode (corresponding minimum data quality scale, be playback mode fast), the first quality playback mode (corresponding first quality of data rank), the second quality playback mode (corresponding second quality of data rank) and first water playback mode (corresponding the maximum data quality scale, be playback mode) entirely.
Further, when client needed the data of minimum data quality scale, then Dui Ying playback mode was a minimum quality playback mode, and client need be obtained base layer data from storage server.When client needed first other data of quality of data level, then Dui Ying playback mode was the first quality playback mode, and client need be obtained base layer data and reinforced layer data from storage server.When client needed second other data of quality of data level, then Dui Ying playback mode was the second quality playback mode, and client need be obtained base layer data and two reinforced layer data from storage server.When client needed the data of the maximum data quality scale, then Dui Ying playback mode was a first water playback mode, and client need be obtained base layer data and all reinforced layer data from storage server.
Among the present invention, when if the user selects minimum quality playback mode in client, then client only need be carried out playback to base layer data, be that client only need be carried out index to base layer data, read base layer data to storage server, thereby make the efficient of quick playback to be greatly improved.
When if the user selects first water playback mode in client, then client need read base layer data and all corresponding reinforced layer data from storage server, and base layer data and all corresponding reinforced layer data fed back to the user, so that the user can access the quality of data of an optimum.Wherein, this client can not repeat them here according to the index search of this base layer data secondary index to reinforced layer data correspondence.
In actual applications, if the user (for example selects other playback modes in client, the first quality playback mode etc.) time, then client need read base layer data and corresponding part reinforced layer data from storage server, and base layer data and corresponding part reinforced layer data are fed back to the user.
The processing procedure of this step specifically comprises:
(1) when the needs playback, client is set up the ISCSI interface channel to storage server.
(2) client retrieves secondary index information, and find base layer data position and reinforced layer Data Position.
(3) client sends the ISCSI request of data to storage server, and only asks base layer data according to the base layer data position.
(4) when the user needed minimum quality playback mode, client was shown to the user with this base layer data.
(5) when the user needs first water playback mode, client sends the ISCSI request of data according to the reinforced layer Data Position, asks all reinforced layer data, and base layer data and all reinforced layer data are shown to the user.
(6) when the user needs other playback modes (for example, the first quality playback mode etc.), client sends the ISCSI request of data according to the reinforced layer Data Position, requested part reinforced layer data, and base layer data and part reinforced layer data be shown to the user.For example, when the user needs the first quality playback mode etc., the reinforced layer data of request first, and the reinforced layer data of base layer data and first are shown to the user.
Wherein, do not have ordinal relation successively between above-mentioned step 504, step 505, step 506 and the step 507, can adjust according to the actual needs, promptly the step among the present invention can be adjusted according to actual needs.
A kind of data handling system that the present invention proposes based on hierarchical coding, as shown in Figure 9, comprise video server 91, storage server 92 and encoder 93, described video server 91 is connected respectively with encoder 93 with storage server 92, and described storage server 92 is connected with encoder 93; Wherein,
Described video server 91 is used to formulate storage policy and storage plan, and described storage policy is sent to described storage server 92, and described storage plan is sent to described encoder 93; Carried the data hierarchy information of a basic layer and at least one reinforced layer in described storage policy and the storage plan.
Described storage server 92, being used for according to described storage policy is that described basic layer is set up storage resources, and is that described reinforced layer is set up storage resources according to described storage policy.
Concrete, described storage server 92 also is used for self memory property is detected, and when described memory property is lower than preset first threshold value, preferential base layer data of storing described encoder by the corresponding storage resources of basic layer; And/or, notify described encoder 93 preferentially base layer data to be stored on the corresponding storage resources.When notifying described encoder 93 preferentially base layer data to be stored on the corresponding storage resources, described storage server 92 is to notify described encoder 93 preferentially base layer data to be stored on the corresponding storage resources by the asynchronous message of ISCSI.
In addition, described storage server 92 also is used for self storage resources is detected, and when described storage resources is lower than default second threshold value, preferentially discharge the storage resources of described reinforced layer correspondence, remove the reinforced layer data in the storage resources of described reinforced layer correspondence.
Described encoder 93 is used for the result according to hierarchical coding, base layer data is stored on the basic layer of corresponding storage resources of described storage server 92; And with the reinforced layer storage to the storage resources of the reinforced layer correspondence of described storage server 92.
As shown in figure 10, the invention allows for a kind of encoder, be applied to comprise that described encoder comprises in the system of encoder and storage server:
Concrete, described transceiver module 103 need store base layer data on the basic layer of corresponding storage resources of described storage server; And with the reinforced layer storage to the storage resources of the reinforced layer correspondence of described storage server.
Further, when the memory property of described storage server was lower than predetermined threshold value, described transceiver module 103 also needed to receive the information of the preferential storage base layer data of described storage server notice; And preferentially base layer data is stored on the basic layer of corresponding storage resources of described storage server.
As shown in figure 11, the invention allows for a kind of storage server, be applied to comprise that described storage server comprises in the system of encoder and storage server:
Set up module 112, be connected with described acquisition module, being used for according to described data hierarchy information is that the data of a basic layer and at least one reinforced layer are set up storage resources respectively;
Concrete, this processing module 113 also is used for self memory property is detected, and when described memory property is lower than preset first threshold value, preferential base layer data of storing described encoder by the corresponding storage resources of basic layer; And/or, notify described encoder preferentially base layer data to be stored on the corresponding storage resources of described basic layer.Wherein, this processing module 113 is to notify described encoder preferentially base layer data to be stored on the corresponding storage resources of described basic layer by the asynchronous message of ISCSI.
In addition, this processing module 113 also is used for self storage resources is detected, and when described storage resources is lower than default second threshold value, preferentially discharges the storage resources of described reinforced layer correspondence, removes the reinforced layer data in the storage resources of described reinforced layer correspondence.
As shown in figure 12, the invention allows for a kind of video server, be applied to comprise in the system of video server, encoder and storage server, storage plan and storage policy have been pre-established in the described video server, all carried the data hierarchy information of a basic layer and at least one reinforced layer in described storage plan and the storage policy, described video server comprises:
Transceiver module 121 is used for described storage plan is sent to described encoder, according to described storage plan data hierarchy is encoded into a basic layer and at least one reinforced layer by described encoder;
And described storage policy sent to described storage server, be that the data of a basic layer and at least one reinforced layer are set up storage resources respectively by described storage server according to described storage policy.
Processing module 122 is connected with described transceiver module, and the indexed mode that is used to adopt hierarchical coding is that the storage resources on the described storage server is set up index; Be specially:
For the pairing storage resources of basic layer set up basic layer index, and set up the reinforced layer index for the pairing storage resources of reinforced layer, described basic layer index and described reinforced layer index have corresponding relation.
As shown in figure 13, the invention allows for a kind of backup server, be applied to comprise that described backup server comprises in the system of storage server and backup server:
Set up module 131, be used to the data of a basic layer and at least one reinforced layer to set up storage resources respectively;
The preferential storage resources that discharges described reinforced layer correspondence is removed the reinforced layer data in the storage resources of described reinforced layer correspondence.
As shown in figure 14, the invention allows for a kind of client, be applied to comprise that described client comprises in the system of storage server and client:
First acquisition module 141 is used to obtain the quality of data rank of self needs, and obtains corresponding data readback mode according to described quality of data rank; For example, when the user needed the highest quality of data, then quality of data rank was the maximum data quality scale, and corresponding data readback mode is a first water playback mode.
Second acquisition module 142 is connected with described first acquisition module 141, is used for obtaining base layer data according to the data readback mode that described first acquisition module 141 obtains at least from described storage server.
Concrete, described second acquisition module 142 specifically is used for:
When described data readback mode is minimum quality playback mode, from described storage server, obtain base layer data according to basic layer index; Perhaps,
When described data readback mode is first water playback mode, from described storage server, obtain base layer data according to basic layer index, and from described storage server, obtain all reinforced layer data according to pairing all the reinforced layer index of described basic layer index; Perhaps,
When described data readback mode is other quality playback modes, from described storage server, obtain base layer data according to basic layer index, and from described storage server, obtain part reinforced layer data according to the pairing part reinforced layer of described basic layer index index.
Through the above description of the embodiments, those skilled in the art can be well understood to the present invention and can realize by hardware, also can realize by the mode that software adds necessary general hardware platform.Based on such understanding, technical scheme of the present invention can embody with the form of software product, it (can be CD-ROM that this software product can be stored in a non-volatile memory medium, USB flash disk, portable hard drive etc.) in, comprise some instructions with so that computer equipment (can be personal computer, server, the perhaps network equipment etc.) carry out the described method of each embodiment of the present invention.
It will be appreciated by those skilled in the art that accompanying drawing is the schematic diagram of a preferred embodiment, module in the accompanying drawing or flow process might not be that enforcement the present invention is necessary.
It will be appreciated by those skilled in the art that the module in the device among the embodiment can be distributed in the device of embodiment according to the embodiment description, also can carry out respective change and be arranged in the one or more devices that are different from present embodiment.The module of the foregoing description can be merged into a module, also can further split into a plurality of submodules.
The invention described above sequence number is not represented the quality of embodiment just to description.
More than disclosed only be several specific embodiment of the present invention, still, the present invention is not limited thereto, any those skilled in the art can think variation all should fall into protection scope of the present invention.
Claims (13)
1. the data processing method based on hierarchical coding is characterized in that, is applied to comprise in the system of encoder and storage server, said method comprising the steps of:
Described encoder obtains storage plan, has carried the data hierarchy information of a basic layer and at least one reinforced layer in the described storage plan;
Described encoder is encoded into a basic layer and at least one reinforced layer according to described data hierarchy information with data hierarchy;
Described encoder stores the data of a described basic layer and at least one reinforced layer respectively on the storage resources corresponding in the described storage server.
2. the method for claim 1 is characterized in that, described encoder stores described basic layer of data with at least one reinforced layer respectively on the storage resources corresponding in the described storage server and specifically comprises:
Described encoder stores base layer data on the basic layer of corresponding storage resources of described storage server;
Described encoder with the reinforced layer storage to the storage resources of the reinforced layer correspondence of described storage server.
3. method as claimed in claim 1 or 2 is characterized in that, described method also comprises:
When the memory property of described storage server was lower than predetermined threshold value, described encoder received the information of the preferential storage base layer data of described storage server notice;
Described encoder preferentially stores base layer data on the basic layer of corresponding storage resources of described storage server.
4. the data processing method based on hierarchical coding is characterized in that, is applied to comprise in the system of encoder and storage server, said method comprising the steps of:
Described storage server obtains storage policy, has carried the data hierarchy information of a basic layer and at least one reinforced layer in the described storage policy;
Described storage server is that the data of a basic layer and at least one reinforced layer are set up storage resources respectively according to described data hierarchy information;
Described storage server receives the data from described encoder, and the base layer data in the described data is stored on the corresponding storage resources of basic layer; With the reinforced layer storage to the storage resources of reinforced layer correspondence.
5. method as claimed in claim 4 is characterized in that, also comprises:
Described storage server detects self memory property, and when described memory property is lower than preset first threshold value,
The base layer data that described storage server is preferentially stored described encoder by the corresponding storage resources of basic layer; And/or,
Described storage server notifies described encoder preferentially base layer data to be stored on the corresponding storage resources of described basic layer.
6. method as claimed in claim 5 is characterized in that, specifically comprises on the storage resources that described storage server notifies described encoder preferentially base layer data to be stored into described basic layer correspondence:
Described storage server notifies described encoder preferentially base layer data to be stored on the corresponding storage resources of described basic layer by the asynchronous message of ISCSI.
7. method as claimed in claim 4 is characterized in that, also comprises:
Described storage server detects self storage resources, and when described storage resources is lower than default second threshold value,
The preferential storage resources that discharges described reinforced layer correspondence of described storage server is removed the reinforced layer data in the storage resources of described reinforced layer correspondence.
8. data processing method based on hierarchical coding, it is characterized in that, be applied to comprise in the system of video server, encoder and storage server, storage plan and storage policy have been pre-established in the described video server, all carried the data hierarchy information of a basic layer and at least one reinforced layer in described storage plan and the storage policy, said method comprising the steps of:
Described video server sends to described encoder with described storage plan, according to described storage plan data hierarchy is encoded into a basic layer and at least one reinforced layer by described encoder;
And described storage policy sent to described storage server, be that the data of a basic layer and at least one reinforced layer are set up storage resources respectively by described storage server according to described storage policy.
9. method as claimed in claim 8 is characterized in that, also comprises:
The indexed mode of described video server employing hierarchical coding is that the storage resources on the described storage server is set up index; Be specially:
Described video server is set up basic layer index for the pairing storage resources of basic layer, and sets up the reinforced layer index for the pairing storage resources of reinforced layer, and described basic layer index and described reinforced layer index have corresponding relation.
10. the data processing method based on hierarchical coding is characterized in that, is applied to comprise in the system of storage server and backup server, said method comprising the steps of:
Described backup server is that the data of a basic layer and at least one reinforced layer are set up storage resources respectively;
Described backup server backs up base layer data in the described storage server by using the corresponding storage resources of basic layer; Perhaps, described backup server independently backs up reinforced layer data in the described storage server by the storage resources that uses the corresponding storage resources of basic layer and independently back up the base layer data in the described storage server and pass through use reinforced layer correspondence.
11. method as claimed in claim 10 is characterized in that, also comprises:
Described backup server detects self storage resources, and when described storage resources is lower than predetermined threshold value,
The preferential storage resources that discharges described reinforced layer correspondence of described backup server is removed the reinforced layer data in the storage resources of described reinforced layer correspondence.
12. the data processing method based on hierarchical coding is characterized in that, is applied to comprise in the system of storage server and client, said method comprising the steps of:
Described client is obtained the quality of data rank that self needs, and obtains corresponding data readback mode according to described quality of data rank;
Described client is obtained base layer data at least according to described data readback mode from described storage server.
13. method as claimed in claim 12 is characterized in that, described client is obtained base layer data at least according to described data readback mode and is specifically comprised from described storage server:
When described data readback mode was minimum quality playback mode, described client was obtained base layer data according to basic layer index from described storage server; Perhaps,
When described data readback mode is first water playback mode, described client is obtained base layer data according to basic layer index from described storage server, and obtains all reinforced layer data according to pairing all the reinforced layer index of described basic layer index from described storage server; Perhaps,
When described data readback mode is other quality playback modes, described client is obtained base layer data according to basic layer index from described storage server, and obtains part reinforced layer data according to the pairing part reinforced layer of described basic layer index index from described storage server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910223268 CN101720041A (en) | 2009-11-23 | 2009-11-23 | Hierarchical coding-based data processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910223268 CN101720041A (en) | 2009-11-23 | 2009-11-23 | Hierarchical coding-based data processing method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101720041A true CN101720041A (en) | 2010-06-02 |
Family
ID=42434543
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200910223268 Pending CN101720041A (en) | 2009-11-23 | 2009-11-23 | Hierarchical coding-based data processing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101720041A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103634552A (en) * | 2012-08-28 | 2014-03-12 | 华为技术有限公司 | Monitoring video storage method, system and central management server |
| CN104767964A (en) * | 2014-01-08 | 2015-07-08 | 株式会社日立制作所 | Video storage control and optimization method and device, and video monitoring system |
| CN105681818A (en) * | 2016-01-07 | 2016-06-15 | 中国联合网络通信集团有限公司 | Picture cloud storage method and device |
| JP2020178216A (en) * | 2019-04-17 | 2020-10-29 | 日本放送協会 | Terminal device, streaming distribution system, and program |
-
2009
- 2009-11-23 CN CN 200910223268 patent/CN101720041A/en active Pending
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103634552A (en) * | 2012-08-28 | 2014-03-12 | 华为技术有限公司 | Monitoring video storage method, system and central management server |
| CN104767964A (en) * | 2014-01-08 | 2015-07-08 | 株式会社日立制作所 | Video storage control and optimization method and device, and video monitoring system |
| CN104767964B (en) * | 2014-01-08 | 2018-09-18 | 株式会社日立制作所 | The method and its device, video monitoring system of video storage control and optimization |
| CN105681818A (en) * | 2016-01-07 | 2016-06-15 | 中国联合网络通信集团有限公司 | Picture cloud storage method and device |
| CN105681818B (en) * | 2016-01-07 | 2019-01-08 | 中国联合网络通信集团有限公司 | A kind of picture cloud storage method and device |
| JP2020178216A (en) * | 2019-04-17 | 2020-10-29 | 日本放送協会 | Terminal device, streaming distribution system, and program |
| JP7235576B2 (en) | 2019-04-17 | 2023-03-08 | 日本放送協会 | Terminal device, streaming distribution system, and program |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20160232055A1 (en) | Methods for distributing erasure-coded fragments in a geo-distributed storage system and devices thereof | |
| US8559322B2 (en) | Link state detection method and system | |
| US7957358B2 (en) | Apparatus, method, and computer readable medium for transmitting data via a plurality of network interfaces | |
| US7941628B2 (en) | Allocation of heterogeneous storage devices to spares and storage arrays | |
| US20130298135A1 (en) | Dynamically Allocating Multitier Applications Based Upon Application Requirements and Performance Reliability of Resources | |
| US20090210618A1 (en) | Apparatus and method to allocate resources in a data storage library | |
| US20180039543A1 (en) | Systems and methods for storing, maintaining, and accessing objects in storage system clusters | |
| CN105183565A (en) | Computer and service quality control method and device | |
| US10908960B2 (en) | Resource allocation based on comprehensive I/O monitoring in a distributed storage system | |
| CN107623643A (en) | A kind of data packet forwarding method and device | |
| CN102298547A (en) | Data backup method and device | |
| CN101808127A (en) | Data backup method, system and server | |
| CN101720041A (en) | Hierarchical coding-based data processing method | |
| CN102291256A (en) | Monitoring data query method and equipment | |
| KR101621752B1 (en) | Distributed Storage Apparatus using Locally Repairable Fractional Repetition Codes and Method thereof | |
| CN109981768A (en) | I/o multipath planning method and equipment in distributed network storage system | |
| CN111399766B (en) | Data storage method, data reading method, device and system in storage system | |
| WO2019161547A1 (en) | Dynamic configuration method, terminal device, network device and computer storage medium | |
| US20140269569A1 (en) | Apparatuses and methods for data transmission and reception | |
| CN102314322B (en) | Data processing method and device based on RAID (redundant array of independent disks) | |
| CN115834587A (en) | Method and device for selecting target storage server and electronic equipment | |
| US8452902B2 (en) | Methods for transmitting buffer size information | |
| KR102001572B1 (en) | Distributed file system and method for managing data the same | |
| CN102325171B (en) | Data storage method in monitoring system and system | |
| CN111694692A (en) | Data storage erasure method, device and equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Open date: 20100602 |