CN109710782B - Bayonet picture management optimization method - Google Patents
Bayonet picture management optimization method Download PDFInfo
- Publication number
- CN109710782B CN109710782B CN201811452370.3A CN201811452370A CN109710782B CN 109710782 B CN109710782 B CN 109710782B CN 201811452370 A CN201811452370 A CN 201811452370A CN 109710782 B CN109710782 B CN 109710782B
- Authority
- CN
- China
- Prior art keywords
- picture
- file
- file block
- block
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000005457 optimization Methods 0.000 title claims abstract description 17
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 230000007774 longterm Effects 0.000 description 4
- 239000012634 fragment Substances 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a bayonet picture management optimization method, which comprises the following steps: when receiving a picture, writing picture data into a buffer area of a corresponding channel according to the channel from which the picture comes, and recording index information of the picture into a hash table; when the picture data of the buffer memory area of a certain channel reaches the set size of the file block, merging the picture data of the buffer memory area into a file block, writing the file block into a disk as a whole, and recording the index of the file block in a database; after the file block is successfully written, deleting the content corresponding to the file block, which is cached in the cache area, and deleting the index information corresponding to the picture of the file block in the hash table. The method of combining a plurality of picture files into one file block improves the IO utilization rate of a file system, so that the storage of the files can be executed according to the running sequence of the magnetic head, and the picture data from different channels are written into different cache areas, thereby being convenient to manage.
Description
Technical Field
The invention relates to the field of picture storage, in particular to a bayonet picture management optimization method.
Background
With the development of the popularization information age of the bayonet video, the requirement on a high-concurrency storage system is higher and higher, and the file system has the defects of low speed of concurrent writing of small files and a large amount of disk fragments generated after long-term use, so that the writing performance is lower and lower, and the long-term storage of large-scale picture small files is not facilitated. And when more files are available, the files are difficult to quickly and accurately locate, and the efficiency of inquiring and reading the files is low.
Disclosure of Invention
The invention aims to provide a bayonet picture management optimization method which is used for solving the problems that the speed of writing small files concurrently in the existing file system is low and a large amount of disk fragments can be generated after long-term use.
The invention is realized in the following way:
the invention provides a bayonet picture management optimization method, which comprises the following steps:
when receiving a picture, writing picture data into a buffer area of a corresponding channel according to the channel from which the picture comes, and recording index information of the picture into a hash table;
when the picture data of the buffer memory area of a certain channel reaches the set size of the file block, merging the picture data of the buffer memory area into a file block, writing the file block into a disk as a whole, and recording the index of the file block in a database;
after the file block is successfully written, deleting the content corresponding to the file block, which is cached in the cache area, and deleting the index information corresponding to the picture of the file block in the hash table.
Further, the timeout time for writing the file into the disk is set, and once the timeout time is reached, the picture data of the corresponding channel buffer area are combined into a file block to be written into the disk.
Further, when receiving the picture, writing the picture data into the buffer area of the corresponding channel according to the channel from which the picture comes, and recording the index information of the picture into the hash table specifically includes:
when receiving the picture, judging whether the picture is a picture file of a channel with a cache area according to the transmitted channel information, if so, writing the picture data into the cache area of the corresponding channel, and if not, creating a new channel cache area for storage.
Further, the method further comprises:
when inquiring the picture file, judging whether the picture file is stored in the disk according to the inquiry information of the user;
if the inquired picture file is not stored in the disk, finding the picture information of the combined large file from the cached hash table, and then inquiring the picture file information required by the user through the information recorded by the file header; if the inquired picture files are stored in the disk, searching file block information corresponding to the picture files from a database, finding file blocks according to the file block information in the database, and combining file head index information of each picture file in the file blocks to accurately position specific contents of each picture file;
and sending specific information of the picture file to a user.
Further, the method further comprises:
when the picture file is read, the specific position and information of the picture file are found according to the picture file inquiring method, and whether the picture file is stored in a cache or a disk is judged according to the specific information of the obtained picture file;
if the picture file is stored in the disk, reading out the specific content of the picture file from the specific position of the specific file block by combining the specific information of the picture file, and then transmitting the specific content of the picture file through a transmitting program;
if the picture file is stored in the cache, the specific content of the picture file is read from the cache and transmitted through a transmitting program.
Further, the method further comprises:
when deleting and covering the picture file, judging whether the residual space of the disk is smaller than a set value, and if so, starting to circularly cover the file;
sequencing the files from a database according to the sequence of storing the picture files, finding the earliest stored file block as the file block to be deleted, and finding the specific position of the file block to be deleted;
and deleting the designated file block according to the specific position of the obtained file block, and deleting the index record of the file block in the database.
Compared with the prior art, the invention has the following beneficial effects:
according to the bayonet picture management optimization method provided by the invention, as a method of combining a plurality of picture files into one file block is adopted, the IO (input/output) utilization rate of a file system is improved, so that the file storage can be executed according to the running sequence of a magnetic head, picture data from different channels are written into different cache areas, namely, the picture data are distinguished according to different front-end card port devices during caching, management is convenient, and index information of cached pictures is recorded into a hash table, so that a user can conveniently inquire and read the currently written file. Meanwhile, when the overlay file is deleted, as the path for storing the file can be quickly found from the hash of the memory and the file block deleted once is larger, the quick deletion and rewriting of the picture file are facilitated, and the increasing small file picture writing requirement is met. The method has the advantages that less disk fragments are generated in long-term use, the utilization rate of the disk is improved, and the effect of better storing small picture files is achieved.
Drawings
Fig. 1 is a flowchart of a method for writing a picture file in a bayonet picture management optimization method according to an embodiment of the present invention;
fig. 2 is a flowchart of a method for querying a picture file in a bayonet picture management optimization method according to an embodiment of the present invention;
fig. 3 is a flowchart of a method for reading a picture file in a bayonet picture management optimization method according to an embodiment of the present invention;
fig. 4 is a flowchart of a method for deleting and covering a picture file in a bayonet picture management optimization method according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The embodiment of the invention provides a bayonet picture management optimization method, which comprises a picture file writing method, as shown in fig. 1, wherein the picture file writing method specifically comprises the following steps:
s1.1, when receiving a picture, writing picture data into a cache area of a corresponding channel according to the channel from which the picture comes, and recording index information of the picture into a hash table;
s1.2, when the picture data of a buffer memory area of a certain channel reaches the set size of a file block, merging the picture data of the buffer memory area into a file block, writing the whole file block into a disk, and recording the index of the file block in a database;
s1.3, after the file block is successfully written, deleting the content corresponding to the file block, which is cached in the cache area, and simultaneously deleting the index information corresponding to the picture of the file block in the hash table.
The above steps are described in detail below.
In the step S1.1, when receiving the picture, it is determined whether the picture is a picture file of a channel having a buffer area according to the transmitted channel information, if so, the picture data is written into the buffer area of the corresponding channel, and if not, a new channel buffer area is created for storage. Different channels correspond to different front-end clamping devices, picture data from the same channel are written into the same buffer area, and picture data from different channels are written into different buffer areas, namely, when in buffer, the different front-end clamping devices are distinguished, and small files from the same front-end clamping device are buffered into a larger file. And recording the index information of the cached pictures into a hash table, so that a user can conveniently inquire and read the current written file.
In step S1.2, the size of the file block is set in advance, after receiving the picture, whether the buffer area corresponding to the channel can be inserted into the buffer according to the set size of the file block is determined, if the size of the file block is not reached, the received file is buffered continuously, if the size of the file block is reached, the index information of the file block is recorded in the database, and the picture data of the buffer area is combined into a file block and written into the disk. Preferably, the timeout period for writing the file into the disk is also set, once the timeout period is reached, the picture data of the buffer area of the channel is merged into the file block to be written into the disk no matter whether the picture data of the buffer area of the channel reaches the set file block size, so that the lost data is prevented from being too much when abnormal conditions occur, the safety of the written data is ensured as much as possible, and the merging timeout period can be set according to the safety level required by the system.
In the step S1.3, the index information of the file in the hash table is deleted while deleting the content of the cached file block, so that the used cache space can be released.
As shown in fig. 2, preferably, the bayonet picture management optimization method of the present embodiment further includes a picture file query method, which specifically includes the following steps:
s2.1, judging whether the picture file is stored in the disk according to the query information of a user when the picture file is queried, wherein the judging standard is that whether a file block meeting the query condition of the user exists in a database or not is queried, if so, the picture file is stored in the disk, and if not, the picture file is not stored in the disk;
s2.2, if the inquired picture file is not stored in the disk, finding the picture information of the combined large file from the cached hash table, and then inquiring the picture file information required by the user through the information recorded by the file header; if the inquired picture files are stored in the disk, searching file block information corresponding to the picture files from a database, finding file blocks according to the file block information in the database, and combining file head index information of each picture file in the file blocks to accurately position specific contents of each picture file;
s2.3, specific information of the picture file is sent to a user.
As shown in fig. 3, preferably, the bayonet picture management optimization method of the present embodiment further includes a method for reading a picture file, which specifically includes the following steps:
s3.1, when the picture file is read, the specific position and information of the picture file are found according to the picture file inquiring method, and whether the picture file is stored in a cache or a magnetic disk is judged according to the specific information of the obtained picture file;
s3.2, if the picture file is stored in the magnetic disk, reading out the specific content of the picture file from the designated position of the designated file block by combining the specific information of the picture file, and transmitting the specific content of the picture file through a transmitting program;
and S3.3, if the picture file is stored in the cache, reading the specific content of the picture file from the cache and transmitting the specific content through a transmitting program, wherein the cached data is still stored.
As shown in fig. 4, preferably, the bayonet picture management optimization method of the present embodiment further includes a method for deleting and covering a picture file, which specifically includes the following steps:
s4.1, when deleting the coverage of the picture file, judging whether the residual space of the disk is smaller than a set value, for example, 5%, and if the residual space is smaller than the set value, starting to circularly cover the file;
s4.2, sorting the files from the database according to the sequence of storing the picture files, finding the earliest stored file block as the file block to be deleted, and finding the specific position of the file block to be deleted;
and S4.3, deleting the designated file block according to the specific position of the obtained file block, releasing the storage space, and deleting the index record of the file block in the database.
Those of ordinary skill in the art will appreciate that all or a portion of the steps in the various methods of the embodiments may be implemented by a program that instructs associated hardware, the program may be stored on a computer readable storage medium, the storage medium may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, alternatives, and improvements that fall within the spirit and scope of the invention.
Claims (2)
1. The bayonet picture management optimization method is characterized by comprising the following steps of:
s1, writing a picture file;
s1.1, when receiving a picture, writing picture data into a cache area of a corresponding channel according to the channel from which the picture comes, and recording index information of the picture into a hash table; the method specifically comprises the following steps: when receiving a picture, judging whether the picture is a picture file of a channel with a cache area according to the transmitted channel information, if so, writing the picture data into the cache area of the corresponding channel, and if not, creating a new channel cache area for storage; different channels correspond to different front-end card port devices, picture data from the same channel is written into the same buffer area, and picture data from different channels is written into different buffer areas;
s1.2, when the picture data of a buffer memory area of a certain channel reaches the set size of a file block, merging the picture data of the buffer memory area into a file block, writing the whole file block into a disk, and recording the index of the file block in a database; setting the overtime of writing the file into the disk, and merging the picture data in the corresponding channel cache area into a file block to be written into the disk once the overtime is reached; setting merging timeout time according to the security level required by the system;
s1.3, after the file block is successfully written, deleting the content corresponding to the file block, which is cached in the cache area, and simultaneously deleting index information corresponding to the picture of the file block in the hash table;
s2, inquiring the picture file;
s2.1, judging whether the picture file is stored in a disk according to query information of a user when the picture file is queried, wherein the judging standard is that whether a file block meeting the query condition of the user exists in a database, if so, the file block information corresponding to the picture file is searched from the database, then the file block is found according to the file block information in the database, the specific content of each picture file is accurately positioned by combining with the file head index information of each picture file in the file block, and then S2.3 is executed; otherwise, the data is not stored in the disk, and S2.2 is executed;
s2.2, judging whether a file to be queried is in a cache region, if so, finding out picture information of a combined large file from a cached hash table, and then querying picture file information required by a user through information recorded by a file header; otherwise, the picture file inquiry fails;
s2.3, specific information of the picture file is sent to a user;
s3, reading a picture file;
s3.1, when the picture file is read, the specific position and information of the picture file are found according to the picture file inquiring method, and whether the picture file is stored in a cache or a magnetic disk is judged according to the specific information of the obtained picture file;
s3.2, if the picture file is stored in the magnetic disk, reading out the specific content of the picture file from the designated position of the designated file block by combining the specific information of the picture file, and transmitting the specific content of the picture file through a transmitting program; otherwise, executing S3.3;
s3.3, judging whether the picture file is stored in a cache, if so, reading the specific content of the picture file from the cache and transmitting the specific content through a transmitting program, wherein the cached data is still stored; otherwise, the picture file is read failure.
2. The bayonet picture management optimization method according to claim 1, characterized in that the method further comprises:
when deleting and covering the picture file, judging whether the residual space of the disk is smaller than a set value, and if so, starting to circularly cover the file;
sequencing the files from a database according to the sequence of storing the picture files, finding the earliest stored file block as the file block to be deleted, and finding the specific position of the file block to be deleted;
and deleting the designated file block according to the specific position of the obtained file block, and deleting the index record of the file block in the database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811452370.3A CN109710782B (en) | 2018-11-30 | 2018-11-30 | Bayonet picture management optimization method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811452370.3A CN109710782B (en) | 2018-11-30 | 2018-11-30 | Bayonet picture management optimization method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109710782A CN109710782A (en) | 2019-05-03 |
| CN109710782B true CN109710782B (en) | 2023-11-28 |
Family
ID=66254436
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811452370.3A Active CN109710782B (en) | 2018-11-30 | 2018-11-30 | Bayonet picture management optimization method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109710782B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110222209B (en) * | 2019-06-14 | 2020-01-03 | 北京马赫谷科技有限公司 | Picture storage method, query method, device and access system |
| CN117850730B (en) * | 2024-03-08 | 2024-05-28 | 青岛罗博数码科技有限公司 | Method and system for displaying pictures by intelligent pen box |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103092775A (en) * | 2013-01-31 | 2013-05-08 | 武汉大学 | Spatial data double cache method and mechanism based on key value structure |
| CN105511802A (en) * | 2015-11-24 | 2016-04-20 | 北京达沃时代科技有限公司 | Buffer memory writing method and apparatus and synchronizing method and apparatus for disk cache region |
| CN106227794A (en) * | 2016-07-20 | 2016-12-14 | 北京航空航天大学 | The storage method and apparatus of dynamic attribute data in temporal diagram data |
| CN106354724A (en) * | 2015-07-15 | 2017-01-25 | 阿里巴巴集团控股有限公司 | File archiving method and file reading method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9542328B2 (en) * | 2015-01-26 | 2017-01-10 | International Business Machines Corporation | Dynamically controlling a file system write cache |
-
2018
- 2018-11-30 CN CN201811452370.3A patent/CN109710782B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103092775A (en) * | 2013-01-31 | 2013-05-08 | 武汉大学 | Spatial data double cache method and mechanism based on key value structure |
| CN106354724A (en) * | 2015-07-15 | 2017-01-25 | 阿里巴巴集团控股有限公司 | File archiving method and file reading method and device |
| CN105511802A (en) * | 2015-11-24 | 2016-04-20 | 北京达沃时代科技有限公司 | Buffer memory writing method and apparatus and synchronizing method and apparatus for disk cache region |
| CN106227794A (en) * | 2016-07-20 | 2016-12-14 | 北京航空航天大学 | The storage method and apparatus of dynamic attribute data in temporal diagram data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109710782A (en) | 2019-05-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107870973B (en) | File storage system capable of accelerating simultaneous playback of multiple paths of monitoring | |
| CN102779180B (en) | The operation processing method of data-storage system, data-storage system | |
| CN110764706B (en) | Storage system, data management method, and storage medium | |
| KR101422557B1 (en) | Predictive data-loader | |
| CN103226965B (en) | Based on the audio/video data access method of time bitmap | |
| CN102393854B (en) | Method and device obtaining audio/video data | |
| CN106657213A (en) | File transmission method and device | |
| CN106649467B (en) | Blue-ray disc library filing management method and system | |
| CN112558873B (en) | Embedded audio and video data storage, retrieval and deletion method and device and memory | |
| CN102479250A (en) | Disk caching method for embedded browser | |
| CN109710782B (en) | Bayonet picture management optimization method | |
| CN104331428A (en) | Storage and access method of small files and large files | |
| CN109240607B (en) | File reading method and device | |
| CN109144413A (en) | A kind of metadata management method and device | |
| CN103399823A (en) | Method, equipment and system for storing service data | |
| CN101872639A (en) | Method and device for storing and retrieving image data on digital hard disk video recorder | |
| US20190287571A1 (en) | Method and device for playing video | |
| CN102567490A (en) | Method and apparatus for recovering description information and caching data in database | |
| CN112637616A (en) | Object storage method, system and server | |
| US20050257000A1 (en) | NAS with worm function | |
| CN101986302B (en) | Media file association method and device | |
| CN117667316A (en) | Asynchronous multithreading file processing method, device and storage medium | |
| CN107430546A (en) | A kind of file updating method and storage device | |
| CN110750566A (en) | Data processing method and device, cache system and cache management platform | |
| CN111597152A (en) | Solid state disk file system management method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |