CN115409679A - Data time sequence recovery method, system, equipment and medium based on thread processing - Google Patents
Data time sequence recovery method, system, equipment and medium based on thread processing Download PDFInfo
- Publication number
- CN115409679A CN115409679A CN202110580325.1A CN202110580325A CN115409679A CN 115409679 A CN115409679 A CN 115409679A CN 202110580325 A CN202110580325 A CN 202110580325A CN 115409679 A CN115409679 A CN 115409679A
- Authority
- CN
- China
- Prior art keywords
- threads
- processing
- time
- temporary storage
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention provides a data time sequence recovery method, a system, equipment and a medium based on thread processing, wherein the method comprises the following steps: polling and processing the collected image frame data by utilizing threads in the thread pool; and sequencing the processing result data obtained by processing the warp Cheng Lunxun based on the time identification. The method writes the real-time image frame data of the same path into the temporary storage window through multi-thread concurrent processing, and then sends the processing result data with smaller time identification to the processing queue based on the temporary storage window, so that the image frame data are prevented from generating time sequence disorder, and the processing result data from the video image data of the same path are ensured to be sent to the next-stage application according to the sequence of acquisition time.
Description
Technical Field
The present invention relates to the field of computer system technologies, and in particular, to a method, a system, a device, and a medium for data timing recovery based on thread processing.
Background
In the smart city field, video capture devices such as web cameras are distributed in buildings, streets and parks, and real-time video image capture is an important component of digital infrastructure construction. The real-time image acquisition is a means for acquiring real-time image information by utilizing a modern technology, and plays an important role in the modern multimedia technology. The method has wide application in the fields of daily life, biomedicine, aerospace and the like. The speed and quality of image acquisition directly affect the overall effect of the product.
At present, a real-time image acquisition system mainly comprises two parts, namely acquisition and preprocessing of a real-time video image, wherein the acquisition part acquires frame data of a real-time video stream, and the preprocessing part processes the video frame data into pictures or video data in a preset format.
However, in the method of using multiple video frame data processing thread groups, because most of application scenes based on real-time images have the required image data sorted according to time, and after multithreading is introduced to concurrently and jointly process the acquired image frames, the CPU processing time obtained by each thread is different, so that different time consumption is easily caused for processing the same image frame, and data with later acquisition time is transmitted to the next stage earlier than data with earlier acquisition time, so that the application receives image data with disordered acquisition time.
Disclosure of Invention
The invention provides a data time sequence recovery method, a data time sequence recovery system, data time sequence recovery equipment and a data time sequence recovery medium based on thread processing, which are used for overcoming the defect that the time sequence of image frame data is disordered when multi-thread concurrent processing is carried out on the same path of real-time images in the prior art and ensuring the sequence of images on the acquisition time when the multi-thread concurrent processing is carried out on the same path of image acquisition queue data.
In a first aspect, the present invention provides a data timing recovery method based on thread processing, including: polling and processing the collected image frame data by utilizing threads in the thread pool; and sequencing the processing result data obtained by the thread polling processing based on the time identification.
In one embodiment, the sorting the processing result data obtained by the thread polling processing based on the time identifier includes: judging whether the capacity of a temporary storage window is full according to the length of the processing result data written into the temporary storage window in advance and the length of a preset temporary storage window; if the capacity of the temporary storage window is not full, writing the current processing result data into the temporary storage window for queuing; otherwise, firstly sending the prior processing result data with the minimum corresponding time identifier in the temporary storage window to a processing queue, and then writing the current processing result data into the temporary storage window; wherein the prior processing result data in the processing queue is arranged according to the time identification.
In one embodiment, the determining whether the size of the temporary storage window is full according to the length of the processing result data previously written into the temporary storage window and the length of a preset temporary storage window includes: checking the length of the processing result data previously written into the temporary storage window; comparing the length to the length of the temporal window; if the length is smaller than the length of the temporary storage window, the capacity of the temporary storage window is not full; if the length is equal to the length of the temporary storage window, the capacity of the temporary storage window is full.
In one embodiment, before the determining whether the size of the scratch pad window is not full according to the length of the processing result data previously written into the scratch pad window and the length of the scratch pad window, the method includes: and setting the length of the temporary storage window.
In one embodiment, the setting the length of the temporal window includes: calculating the average time consumption of the image frame data acquired by the threads in the thread pool at one time; selecting a maximum value in the average time consumption, and in a time period from the moment when the maximum value corresponds to the acquired image frame data to the moment when the processing result data of the image frame data corresponding to the maximum value is written into the temporary storage window, acquiring and writing the total number of data of the temporary storage window into all threads in the thread pool; the length of the temporary storage window is not less than the total number of the data.
In one embodiment, the length of the scratch pad window is equal to the number of threads in the thread pool; or the length of the temporary storage window is equal to the number of threads in the thread pool plus one.
In one embodiment, before processing the acquired image frame data by polling threads in a thread pool, the method comprises the following steps: estimating the number of threads required in unit time based on the frequency of the collected image frame data and the collected image frame data; and adjusting the capacity of the thread pool according to the number of threads required in the unit time.
In one embodiment, the estimating the number of threads needed per unit time includes: estimating the average processing time of the thread processing once acquired image frame data in the thread pool; calculating the total consumption time of all threads in the thread pool for data processing based on the frequency of the collected image frame data and the average processing time; and rounding up based on the total consumed time to obtain the number of threads required in unit time.
In one embodiment, the adjusting the capacity of the thread pool according to the number of threads needed in the unit time includes: judging whether the number of threads required in the unit time is 0 or not; if yes, adjusting the number of threads of the current thread pool to be 1, and suspending the threads; otherwise, judging the size between the number of threads needed in the unit time and the number of threads in the thread pool so as to adjust the capacity of the thread pool.
In one embodiment, the determining a size between the number of threads needed in the unit time and the number of threads in the thread pool to adjust the thread pool capacity includes: if the number of threads needed in the unit time is equal to the number of threads in the thread pool, not adjusting the number of threads in the current thread pool; if the number of threads needed in the unit time is larger than the number of threads in the thread pool, increasing the number of threads in the current thread pool according to the difference value between the number of threads needed in the unit time and the number of threads in the thread pool; and if the number of threads required in the unit time is less than the number of threads in the thread pool, reducing the number of threads in the current thread pool according to the difference between the number of threads in the thread pool and the number of threads required in the unit time.
In a second aspect, the present invention provides a data timing recovery system based on thread processing, including: the processing module is used for processing the acquired image frame data by utilizing the thread polling in the thread pool; and the queue module is used for sequencing the processing result data obtained by the thread polling processing based on the time identification.
In a third aspect, the present invention provides an electronic device, which includes a memory and a memory storing a computer program, and when the processor executes the program, the processor implements the steps of the data timing recovery method based on thread processing according to the first aspect or the second aspect.
In a fourth aspect, the present invention provides a processor-readable storage medium storing a computer program for causing a processor to execute the steps of the method for restoring data timing based on thread processing according to the first or second aspect.
According to the data timing recovery method, device, equipment and medium based on thread processing, multithreading concurrent processing is performed on the same path of real-time image frame data to be written into the temporary storage window, and processing result data with smaller internal time identification are sent to the processing queue based on the temporary storage window, so that time sequence confusion of image frame data is avoided, and the processing result data from the same path of video image data are sent to the next-stage application according to the sequence of acquisition time; the number of threads in the thread pool is dynamically adjusted to avoid data blockage caused by less threads in the processing process, or avoid waste of system resources caused by a thread Cheng Jiaoduo in the processing process.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a flow chart of a data timing recovery method based on thread processing according to the present invention;
FIG. 2 is a flow diagram of dynamically adjusting thread pools in accordance with the present invention;
FIG. 3 is a flow chart illustrating a process for writing data into a processing queue of a temporary storage window according to the present invention;
FIG. 4 is a schematic structural diagram of a data timing recovery system based on thread processing according to the present invention;
FIG. 5 is a second schematic diagram of the data timing recovery system based on thread processing according to the present invention;
fig. 6 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 respectively shows a flow chart of a data timing recovery method based on thread processing according to the present invention, where the method includes:
s01, polling and processing the collected image frame data by using threads in a thread pool;
and S02, sequencing the processing result data obtained by processing the meridian Cheng Lunxun based on the time identification.
It should be noted that the execution subject of the data timing recovery method based on thread processing provided by the present invention may be a computer device, such as a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a Personal Digital Assistant (PDA).
S0N in this specification does not represent the order of the data timing recovery method based on thread processing, and the data timing recovery method based on thread processing according to the present invention is specifically described below with reference to fig. 2 to 3.
And step S01, polling and processing the collected image frame data by using threads in the thread pool.
In this embodiment, the polling processing of the acquired image frame data includes: converting the image size, e.g., reducing the image resolution, reducing the image size, etc.; and/or, the compressed image, for example, the pixel points in the image frame data, usually represent the color mode RGB by three primary colors, and the space is large, generally 5-6 million, and for transmission, it can be compressed into the format of BMP, PNG or JPG, etc.
In an alternative embodiment, before the polling processing of the collected image frame data by the thread in the thread pool, the method comprises the following steps:
acquiring image frame data in real time according to acquisition task requirements, and writing the image frame data into a cache queue Q c . It should be noted that, when image frame data collected by the central line Cheng Lunxun in the subsequent thread pool, the thread is buffered in the buffer queue Q c When the collected image frame data is extracted; the collected image frame data is derived from videos and can be obtained based on a camera, video playing software or video equipment and the like. The acquisition task queues are multiple and are used for acquiring image frame data in real time according to acquisition task requirements and transmitting the acquired image frame data to the buffer queue Q c Each collection task corresponds to a unique cache queue Q c 。
In addition, the frame data refers to one frame of image forming the video, and one frame of image in the video is captured to obtain a primary picture in the video stream. The queue is a First-In First-Out (FIFO) linear table, is a common data structure, and is usually realized by a chain table or an array In specific application; the queue allows only insertion operations at the back end (called real) and deletion operations at the front end (called front), the definition of the task mainly depending on the object of the processing task.
When the image frame data is collected, the time mark of the collected image frame data is used as the time mark for identifying according to the collection time or the time in the collected video, so that the sequence of the collected image frame data relative to the pictures in the video stream can be distinguished.
In an optional embodiment, because the capacity of the thread pool determines the number of concurrent tasks that can be processed within a specified time, if the capacity of the thread pool is small, the tasks are likely to queue and wait to enter the thread pool, which causes data congestion; if the capacity of the thread pool is large, system resources are wasted, and therefore before the collected image frame data is polled and processed by using threads in the thread pool, the method includes:
estimating the number of threads required in unit time based on the frequency of the collected image frame data and the collected image frame data; the capacity of the thread pool is adjusted according to the number of threads required in a unit time, see fig. 2. Therefore, under the condition of multi-path real-time image acquisition, a plurality of threads in the thread pool continuously poll and process acquired image frame data to estimate the number of required threads, so that the number of the threads is dynamically adjusted to meet the requirement of dynamically increasing and decreasing multi-path real-time image acquisition.
Estimating the number of threads needed per unit time, comprising: estimating the average processing time of the image frame data acquired by the threads in the thread pool at one time; calculating the total consumption time of all threads for data processing in the thread pool based on the frequency of the collected image frame data and the average processing time; and rounding up based on the total consumed time to obtain the number of threads required in unit time. Specifically, the method comprises the following steps:
firstly, the average processing time of the threads in the thread pool for processing the image frame data collected once is estimated, and the average processing time
Expressed as: t is pi
Wherein m represents the current thread pool capacity, namely the number of threads contained in the current thread pool; j is denoted as the jth thread in the thread pool; t is pij Denoted as the jth threadAnd (5) processing the processing time of the primary data in the ith acquisition task pair column.
Secondly, calculating the total consumption time of all threads in the thread pool for data processing based on the frequency and the average processing time of the collected image frame data, and calculating the total consumption time T p Expressed as:
wherein n is the total number of the acquisition task queues, and fci is the frequency of the acquired image frame data, i.e. the newly increased number of the image frame data acquired by the acquisition task queues in unit time.
Finally, rounding up based on the total consumption time to obtain the number of threads needed in unit time and the number of threads C needed in unit time p Expressed as:
C p ＝ceil(T p /1s)
wherein ceil is an ceiling function.
Adjusting the capacity of the thread pool according to the number of threads needed in unit time, comprising: judging whether the number of threads required in unit time is 0 or not; if so, i.e. C p If not, adjusting the number of threads in the current thread pool to be 1, and suspending the threads; otherwise, judging the size between the number of threads needed in unit time and the number of threads in the thread pool to adjust the capacity of the thread pool.
More specifically, determining the size between the number of threads needed in a unit time and the number of threads in the thread pool to adjust the thread pool capacity includes:
if the number of threads needed in a unit of time is equal to the number of threads in the thread pool, i.e. C p = m, the number of threads in the current thread pool is not adjusted; if the number of threads needed in the unit time is larger than the number of threads in the thread pool, namely C p If m is greater than m, the difference value C between the number of threads needed in unit time and the number of threads in the thread pool is calculated p -m increases the number of threads in the current thread pool and wakes up the threads; if required in unit timeThe number of threads is less than the number of threads in the thread pool, i.e. C p If m is less than m, according to the difference value m-C between the number of threads in the thread pool and the number of threads needed in unit time p The number of threads in the current thread pool is reduced and the threads are woken up.
And S02, sequencing the processing result data obtained by processing the meridian Cheng Lunxun based on the time identification.
In this embodiment, sorting the processing result data obtained by processing the meridian Cheng Lunxun based on the time identifier includes: judging whether the capacity of the temporary storage window is full according to the length of the processing result data written into the temporary storage window in advance and the length of a preset temporary storage window; if the capacity of the temporary storage window is not full, writing the current processing result data into the temporary storage window for queuing; otherwise, the data of the previous processing result with the minimum corresponding time mark in the temporary storage window is sent to a processing queue Q p Writing the current processing result data into a temporary storage window; wherein the processing queue Q p The preceding processing result data in the array are arranged according to time marks. Specifically, the method for judging whether the capacity of the temporary storage window is full according to the length of the processing result data written into the temporary storage window in advance and the length of a preset temporary storage window comprises the following steps: checking the length of the processing result data previously written into the temporary storage window; comparing the length with the length of the temporal window; if the length is smaller than the length of the temporary storage window, the capacity of the temporary storage window is not full; if the length is equal to the length of the register window, the register window is full.
It should be noted that, the buffer window and the processing queue Q in the initial state p No data exists in the temporary storage window, so that in an initial state, data needs to be written into the temporary storage window once, namely the processing result data obtained in advance is written into the temporary storage window; and comparing the length of the processing result data currently written into the temporary storage window with the length of the temporary storage window to judge whether the current processing result data is directly written into the temporary storage window.
In an alternative embodiment, before determining whether the size of the scratch pad window is not full according to the length of the processing result data previously written into the scratch pad window and the length of the scratch pad window, the method includes: the length of the temporary storage window is set. Specifically, the length of the temporal window is set, including: calculating the average time consumption of the image frame data acquired by the threads in the thread pool at one time; selecting a maximum value in average time consumption, and in a time period from the moment when the maximum value corresponds to the acquired image frame data to the moment when the processing result data of the image frame data corresponding to the maximum value is written into the temporary storage window, acquiring and writing the total number of data of the temporary storage window into all threads in the thread pool; the length of the temporary storage window is not less than the total number of data.
It should be noted that there is uncertainty about the time consumed by processing data once per thread if the length L of the scratch pad window is large w If the processing result data is too short, the sequence of the processing result data written into the temporary storage window in the acquisition task queue is smaller than the sequence of the data in all the temporary storage windows in the acquisition task queue, and the processing queue Q cannot be guaranteed by the step of sorting the processing result data obtained by the thread processing based on the time identifier p The time sequence of the internal online processing result data; if the length L of the temporary storage window w If the size is too large, the memory resource of the server will be wasted, and therefore, the length L of the temporary storage window w The length L of the temporary storage window is set according to the number of threads in the thread pool and the time difference spent on processing data once among the threads w The calculation process of (c) is as follows:
firstly, the average time consumption of the threads in the thread pool for processing the image frame data collected once is calculated. Specifically, assume that the number of the acquisition task queues is n, and the ith acquisition task queue is marked as Q i . Let a total of m threads, the jth thread be denoted as P j The average processing time of the jth thread processing the image frame data in the ith acquisition task queue is recorded as
The processing time length of the jth thread for processing the image frame data in the ith acquisition task queue is marked as T pij (k) Where k represents the processing duration of a certain data, the average time consumption is expressed as:
where K represents the number of statistics.
Secondly, get
Maximum value of (1), is noted
Wherein j =1, …, m,
the corresponding thread is P, and the maximum value is
The time corresponding to the collected image frame data is recorded as t p Will be the maximum value
The time when the corresponding image frame data is processed by the thread P and written into the temporary storage window is recorded as t q Then at t p -t q In the time period, the maximum number of the data which are processed by other threads and written into the temporary storage window is as follows:
where floor is expressed as a floor function and other threads are at t p -t q The maximum number of data collected and written into the temporary storage window in a time period plus the total number of data written into the temporary storage window by the thread P itself is represented as:
finally, the length L of the temporary storage window is set w Guarantee L w Not less than C. It should be noted that, if the time stamp corresponding to the processing result data written into the temporary storage window by the thread P is the minimum, the processing of writing the thread P into the temporary storage window may be performedComparing the result data with the processing result data written into the temporary storage window by other threads to select the processing result data with the minimum time mark from the temporary storage window and write the processing result data into the processing queue Q p To ensure that even if the thread P gets the collected image frame data from the collection task queue earlier and the data finally enters the temporary storage window because of the longer thread processing time, the data can be put into the processing queue Q in advance p Thereby guaranteeing a processing queue Q p The timing of the data within the queue.
In an alternative embodiment, the length L of the scratch window is calculated w = m, since the time consumed for processing the same data among threads is similar, the length L of the register window is set to avoid the influence of the fluctuation of the processing time during the processing of a single data w The method comprises the following steps: l is w ＝m+1。
When the length L of the temporary storage window w When the temporary storage window is judged to be full or not, judging the length of the processing result data written into the temporary storage window in advance based on the number m of threads in the thread pool directly; when the length L of the temporary storage window w When = m +1, when determining whether the temporary storage window is full, the determination may be directly based on the length of the processing result data previously written into the temporary storage window and the number of threads in the thread pool plus one. For example, when the length L of the register window w If = m, the step of determining whether the scratch pad window is full is as follows, referring to fig. 3.
First, set the initial state, the temporary window and the processing queue Q p Has no data in it, and the length of the temporary storage window is L w And the number of the threads in the thread pool is m.
Secondly, write once data into the temporary storage window, that is, write the processing result data obtained firstly into the temporary storage window, and check the corresponding length of the data, and record it as L c 。
Subsequently, L is judged c And m, if L c <m, if the capacity of the temporary storage window is not full, directly writing the data into the temporary storage window for queuing; if L is c If = m, the capacity of the temporary storage window is full, and the place with the minimum time mark is found from the written data in the temporary storage windowThe data of the processing result is sent to a processing queue Q p (ii) a Then put L into c And writing the corresponding processing result data into the temporary storage window. It should be noted that, the transmission to the processing queue Q is performed p The previous processing result data in the queue is transmitted in time-stamp order, and thus, the processing queue Q p The preceding processing result data within is sorted by time stamp.
It should be noted that the length L of the temporary storage window w When = m +1, it is determined whether the temporary storage window is full with the length L of the reference temporary storage window w If it is not less than m, determining whether the temporary storage window is full, which is not described herein.
In an optional embodiment, after sorting the processing result data obtained by the thread processing based on the time identifier, the method includes:
and transmitting the processing result data sequenced according to the time identification to the next stage for processing. In this embodiment, multiple transfer threads will process queue Q p The processing result data in the step (2) is sent to the next stage so as to ensure that the processing result data from the same path of video image data is sent to the next stage for application according to the sequence of the acquisition time.
In summary, the same path of real-time image frame data subjected to multi-thread concurrent processing is written into the temporary storage window, and the processing result data with smaller time identification in the same path of real-time image frame data is sent to the processing queue based on the temporary storage window, so that the image frame data is prevented from generating time sequence disorder, and the processing result data from the same path of video image data is ensured to be sent to the next-stage application according to the sequence of acquisition time; the number of threads in the thread pool is dynamically adjusted to avoid data blockage caused by less threads in the processing process, or avoid waste of system resources caused by a thread Cheng Jiaoduo in the processing process.
The data timing recovery system based on thread processing provided by the present invention is described below, and the data timing recovery system based on thread processing described below and the data timing recovery method based on thread processing described above may be referred to correspondingly.
Referring to fig. 4-5, fig. 4-5 illustrate a data timing recovery system based on thread processing, comprising:
the processing module 1 processes the collected image frame data by utilizing the polling of threads in the thread pool;
and the queue module 2 is used for sequencing the processing result data obtained by the thread polling processing based on the time identification.
In this embodiment, the processing module 1 includes a thread pool unit, which is used to poll and process the collected image frame data. Specifically, the thread pool unit includes: a conversion subunit for converting the image size, for example, reducing the image resolution, reducing the image size, and the like; and/or, a compression subunit, configured to compress an image, for example, a pixel point in image frame data, where the three primary colors are usually used to represent a color mode RGB, the space of the color mode RGB is large, and is typically 5 to 6 megabytes, and for convenience of transmission, the color mode RGB can be compressed into a format such as BMP, PNG, or JPG.
The queue module 2 comprises a judging unit, a temporary storage window unit and a buffer queue unit, wherein the judging unit judges whether the capacity of the temporary storage window unit is full according to the length of the processing result data which is written into the temporary storage window unit in advance and the length of a preset temporary storage window; if the capacity of the temporary storage window unit is not full, the threads in the thread pool unit write the current processing result data into the temporary storage window unit for queuing; otherwise, the temporary storage window unit sends the previous processing result data with the minimum corresponding time mark in the temporary storage window unit to the processing queue unit, and the thread unit writes the current processing result data into the temporary storage window unit; wherein, the prior processing result data in the processing queue unit are arranged according to the time mark.
In order to facilitate the judgment of whether the current temporary storage window capacity is full, the judgment unit comprises an identification subunit and a first judgment subunit, wherein the identification subunit checks the length of the processing result data written into the temporary storage window in advance; the judging subunit compares the length with the length of the temporary storage window; if the length is smaller than the length of the temporary storage window, the capacity of the temporary storage window is not full; if the length is equal to the length of the register window, the register window is full.
In an alternative embodiment, there is uncertainty in the time it takes to process data once per thread if the scratch window is over-sizedIf the processing result data is short, the sequence of the processing result data written into the temporary storage window in the acquisition task queue is easily smaller than the sequence of the data in all the temporary storage windows in the acquisition task queue, and the processing queue Q cannot be guaranteed according to the step of sequencing the processing result data obtained by the thread processing based on the time identifier p The time sequence of the internal online processing result data; if the length L of the temporary storage window w If the size of the buffer window is too large, the memory resource of the server is wasted, and therefore, the length of the buffer window needs to be set before determining whether the capacity of the buffer window is not full.
The queue module 2 further comprises: the first computing unit is used for computing the average time consumption of the image frame data acquired by the thread processing in the thread pool at one time; the second calculation unit is used for selecting the maximum value in the average time consumption, and calculating the total number of data collected and written into the temporary storage window by all threads in the thread pool in a time period from the moment when the maximum value corresponds to the collected image frame data to the moment when the processing result data of the image frame data corresponding to the maximum value is written into the temporary storage window; and the setting unit is used for setting the length of the temporary storage window to be not less than the total number of the data. It should be noted that, the specific calculation method may be described with reference to the above method embodiment, and is not described herein again.
In an alternative embodiment, the length L of the scratch pad window is calculated w = m, because the time consumed for processing the same data among threads is similar, in order to avoid the influence of the fluctuation of the processing time when the length of the temporary storage window is processed by single data, the length L of the temporary storage window is set w The method comprises the following steps: l is w = m +1. In an optional embodiment, because the capacity of the thread pool determines the number of concurrent tasks that can be processed within a specified time, if the capacity of the thread pool is small, tasks are likely to queue to wait for entering the thread pool, and data congestion is caused; if the capacity of the thread pool is large, system resources will be wasted, and therefore, the processing module 1 further includes:
the estimating unit is used for estimating the number of threads required in unit time based on the frequency of the collected image frame data and the collected image frame data; and the adjusting unit is used for adjusting the capacity of the thread pool according to the number of threads required in unit time. Therefore, under the condition of multi-path real-time image acquisition, a plurality of threads in the thread pool continuously poll and process acquired image frame data to estimate the number of required threads, so that the number of the threads is dynamically adjusted to meet the requirement of dynamically increasing and decreasing multi-path real-time image acquisition.
In order to facilitate accurate estimation of the number of threads required per unit time, the estimation unit includes: the first estimation subunit estimates the average processing time of the thread processing once acquired image frame data in the thread pool; the second estimation subunit calculates the total consumption time of all threads for data processing in the thread pool based on the frequency of the collected image frame data and the average processing time; and the rounding unit rounds upwards based on the total consumed time to obtain the number of threads required in unit time.
In addition, the adjusting unit includes: a second judgment subunit which judges whether the number of threads required in unit time is 0; the adjusting subunit adjusts the number of threads in the thread pool unit according to the judgment result of the second judging subunit, and when the second judging subunit judges that the number of threads required in unit time is 0, the adjusting subunit adjusts the number of threads in the current thread pool to be 1 and suspends the threads; and when the second judging subunit judges that the number of threads required in the unit time is not 0, the subunit is adjusted to judge the size between the number of threads required in the unit time and the number of threads in the thread pool, so as to adjust the capacity of the thread pool according to the difference value of the number of threads required in the unit time and the number of threads in the thread pool.
Specifically, adjusting the thread pool capacity according to the difference between the two comprises: if the number of threads needed in unit time is equal to the number of threads in the thread pool, the number of threads in the current thread pool is not adjusted; if the number of threads required in unit time is larger than the number of threads in the thread pool, increasing the number of threads in the current thread pool according to the difference value between the number of threads required in unit time and the number of threads in the thread pool; and if the number of threads required in unit time is less than the number of threads in the thread pool, reducing the number of threads in the current thread pool according to the difference between the number of threads in the thread pool and the number of threads required in unit time. It should be noted that, the specific adjustment method can be described with reference to the above method embodiment, and is not described herein again.
In an optional embodiment, the system further comprises: and the acquisition module is used for acquiring image video frame data in real time according to the acquisition task requirement. Specifically, the acquisition module comprises an acquisition unit and a buffer queue unit, wherein the acquisition unit is used for acquiring image frame data in real time according to the acquisition task requirement and writing the image frame data into the buffer queue unit; the buffer queue unit is used for storing the image frame data collected by the collecting unit.
In an optional embodiment, the system further includes a transmission module, which is composed of a plurality of transmission threads, and is configured to send the video image data in the queue module 2 to a next stage, where the same queue module 2 has only one sending thread, so as to ensure that processing result data derived from the same path of video image data is sent to a next stage application according to the collection time sequence.
Fig. 6 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 6: a processor (processor) 610, a communication interface (communication interface) 620, a memory (memory) 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 complete communication with each other through the communication bus 640. The processor 610 may invoke computer programs in the memory 630 to perform the steps of the data timing recovery method based on thread processing, including, for example: polling and processing the collected image frame data by using threads in the thread pool; and sequencing the processing result data obtained by processing the warp Cheng Lunxun based on the time identification.
In addition, the logic instructions in the memory 630 may be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
In another aspect, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer readable storage medium, the computer program including program instructions, when the program instructions are executed by a computer, the computer being capable of executing the steps of the method for executing data timing recovery based on thread processing, which are provided by the above methods, for example, the method includes: polling and processing the collected image frame data by using threads in the thread pool; and sequencing the processing result data obtained by processing the warp Cheng Lunxun based on the time identification.
On the other hand, an embodiment of the present application further provides a processor-readable storage medium, where the processor-readable storage medium stores a computer program, where the computer program is configured to cause the processor to execute the method provided in each of the above embodiments, for example, including: polling and processing the collected image frame data by utilizing threads in the thread pool; and sequencing the processing result data obtained by processing the warp Cheng Lunxun based on the time identification.
The processor-readable storage medium can be any available medium or data storage device that can be accessed by a processor, including, but not limited to, magnetic memory (e.g., floppy disks, hard disks, magnetic tape, magneto-optical disks (MOs), etc.), optical memory (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor memory (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND FLASH), solid State Disks (SSDs)), etc.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (13)
1. A data time sequence recovery method based on thread processing is characterized by comprising the following steps:
polling and processing the collected image frame data by utilizing threads in the thread pool;
and sequencing the processing result data obtained by the thread polling processing based on the time identification.
2. The method for restoring data timing sequence based on thread processing according to claim 1, wherein the sorting the processing result data obtained by the thread polling processing based on the time identifier comprises:
judging whether the capacity of the temporary storage window is full according to the length of the processing result data written into the temporary storage window in advance and the length of a preset temporary storage window;
if the capacity of the temporary storage window is not full, writing the current processing result data into the temporary storage window for queuing;
otherwise, firstly sending the prior processing result data with the minimum corresponding time identifier in the temporary storage window to a processing queue, and then writing the current processing result data into the temporary storage window; wherein the prior processing result data in the processing queue is arranged according to the time identification.
3. The method of claim 2, wherein the determining whether the buffer window capacity is full according to the length of the processing result data previously written into the buffer window and a preset buffer window length comprises:
checking the length of the processing result data previously written into the temporary storage window;
comparing the length to the length of the temporal window;
if the length is smaller than the length of the temporary storage window, the capacity of the temporary storage window is not full;
if the length is equal to the length of the temporary storage window, the capacity of the temporary storage window is full.
4. The method of claim 2, wherein before determining whether the size of the scratch pad window is less than the full size according to the length of the processing result data previously written into the scratch pad window and the length of the scratch pad window, the method comprises:
and setting the length of the temporary storage window.
5. The method of claim 4, wherein the setting the length of the scratch pad window comprises:
calculating the average time consumption of the image frame data acquired by the threads in the thread pool at one time;
selecting the maximum value in the average time consumption, and calculating the total number of data collected and written into the temporary storage window by all threads in the thread pool in a time period from the time when the maximum value corresponds to the collected image frame data to the time when the processing result data of the image frame data corresponding to the maximum value is written into the temporary storage window;
the length of the temporary storage window is not less than the total number of the data.
6. The method according to claim 4, wherein the length of the temporary window is equal to the number of threads in the thread pool; alternatively, the first and second electrodes may be,
the length of the temporary storage window is equal to the number of threads in the thread pool plus one.
7. The method for restoring data timing based on thread processing according to claim 1, prior to polling and processing the collected image frame data by using threads in the thread pool, comprising:
estimating the number of threads required in unit time based on the frequency of the acquired image frame data and the acquired image frame data;
and adjusting the capacity of the thread pool according to the number of threads required in the unit time.
8. The method of claim 7, wherein the estimating the number of threads needed per unit time comprises:
estimating the average processing time of the thread processing once acquired image frame data in the thread pool;
calculating the total consumption time of all threads in the thread pool for data processing based on the frequency of the acquired image frame data and the average processing time;
and rounding up based on the total consumed time to obtain the number of threads required in unit time.
9. The method according to claim 7, wherein the adjusting the capacity of the thread pool according to the number of threads needed in the unit time comprises:
judging whether the number of threads required in the unit time is 0 or not;
if so, adjusting the number of the threads of the current thread pool to be 1, and suspending the threads;
otherwise, judging the size between the number of threads needed in the unit time and the number of threads in the thread pool to adjust the capacity of the thread pool.
10. The method according to claim 9, wherein the determining a size between the number of threads needed in the unit time and the number of threads in the thread pool to adjust the thread pool capacity comprises:
if the number of threads needed in the unit time is equal to the number of threads in the thread pool, not adjusting the number of threads in the current thread pool;
if the number of threads needed in the unit time is larger than the number of threads in the thread pool, increasing the number of threads in the current thread pool according to the difference value between the number of threads needed in the unit time and the number of threads in the thread pool;
and if the number of threads required in the unit time is less than the number of threads in the thread pool, reducing the number of threads in the current thread pool according to the difference between the number of threads in the thread pool and the number of threads required in the unit time.
11. A data timing recovery system based on thread processing, comprising:
the processing module is used for processing the acquired image frame data by utilizing the thread polling in the thread pool;
and the queue module is used for sequencing the processing result data obtained by the thread polling processing based on the time identification.
12. An electronic device comprising a processor and a memory storing a computer program, wherein the processor implements the steps of the method for data timing recovery based on thread processing according to any one of claims 1 to 10 when executing the computer program.
13. A processor-readable storage medium, having stored thereon a computer program for causing a processor to execute the steps of the method for thread-processing-based data timing recovery according to any one of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110580325.1A CN115409679A (en) | 2021-05-26 | 2021-05-26 | Data time sequence recovery method, system, equipment and medium based on thread processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110580325.1A CN115409679A (en) | 2021-05-26 | 2021-05-26 | Data time sequence recovery method, system, equipment and medium based on thread processing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115409679A true CN115409679A (en) | 2022-11-29 |
Family
ID=84155059
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110580325.1A Pending CN115409679A (en) | 2021-05-26 | 2021-05-26 | Data time sequence recovery method, system, equipment and medium based on thread processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115409679A (en) |
-
2021
- 2021-05-26 CN CN202110580325.1A patent/CN115409679A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20160240224A1 (en) | Reference and non-reference video quality evaluation | |
| KR101423916B1 (en) | Method and apparatus for recognizing the plural number of faces | |
| CN115052141B (en) | System and method for event camera data processing | |
| US20230144483A1 (en) | Method for encoding video data, device, and storage medium | |
| US9407823B2 (en) | Handling video frames compromised by camera motion | |
| CN111698555B (en) | Video frame extraction processing method and device | |
| CN114567796A (en) | Frame loss method, device, server and medium | |
| CN113691733A (en) | Video jitter detection method and device, electronic equipment and storage medium | |
| CN110956648A (en) | Video image processing method, device, equipment and storage medium | |
| CN110852196B (en) | Face recognition information display method and device | |
| CN115409679A (en) | Data time sequence recovery method, system, equipment and medium based on thread processing | |
| CN113329175A (en) | Snapshot method, device, electronic device and storage medium | |
| CN112887717A (en) | Intelligent terminal video coding optimization method and device, equipment and storage medium | |
| CN113744139A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN113038261A (en) | Video generation method, device, equipment, system and storage medium | |
| CN115509739A (en) | High-concurrency scheduling and analyzing system for real-time intelligent perception of videos | |
| CN111083413B (en) | Image display method and device, electronic equipment and storage medium | |
| CN114302100A (en) | Video analysis method and device, electronic equipment and storage medium | |
| Kim et al. | Self-adaptive machine learning operating systems for security applications | |
| WO2023123714A1 (en) | Image recognition method and apparatus, and device | |
| CN117793454A (en) | Video stream processing method and device, electronic equipment and storage medium | |
| Sieczkowski et al. | Use of a Raspberry PI single-board computer for image acquisition and transmission | |
| CN112311734B (en) | Image feature extraction method of multi-channel video, electronic equipment and storage medium | |
| CN118138801B (en) | Video data processing method and device, electronic equipment and storage medium | |
| CN115599574B (en) | Graphic processing system, electronic component, electronic device, and information processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |