CN110493598A - Method for processing video frequency and relevant apparatus - Google Patents
Method for processing video frequency and relevant apparatus Download PDFInfo
- Publication number
- CN110493598A CN110493598A CN201910738333.7A CN201910738333A CN110493598A CN 110493598 A CN110493598 A CN 110493598A CN 201910738333 A CN201910738333 A CN 201910738333A CN 110493598 A CN110493598 A CN 110493598A
- Authority
- CN
- China
- Prior art keywords
- data
- artificial intelligence
- layer
- input
- process device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
This application involves a kind of method for processing video frequency and relevant apparatus, wherein electronic equipment includes memory, processor and stores the computer program that can be run on a memory and on a processor, and the processor realizes the method for processing video frequency when executing described program.
Description
Technical field
This application involves technical field of video processing, and in particular to the video processing side based on artificial intelligence process device chip
Method and relevant apparatus.
Background technique
Currently, when using depth learning technology processing video data, it is necessary first to be solved on decoder to video
Code, then the picture of decoded output is input on central processor CPU and carries out the processing such as color space conversion, will treated knot
Fruit is transmitted on artificial intelligence process device chip again and executes the reasoning results, as shown in Figure 1.
There are following technological deficiencies for the prior art: (1) processing can bring the consumption of cpu resource, (2) number before doing on CPU
According to artificial intelligence process device chip is transferred to again from decoder transfers to CPU, there are biggish time overheads.(3) usually in CPU
On finish the problems such as also needing to consider the alignment of data and put after pre-treatment, further consume the resource of CPU.
Summary of the invention
One embodiment of the application provides a kind of method for processing video frequency, which comprises
Artificial intelligence process device chip receives picture frame；Wherein, described image frame is obtained based on video decoding to be processed
；
The artificial intelligence process device chip executes the corresponding binary instruction of deep learning model in off-line operation file,
Described image frame is handled；Wherein, the off-line operation file is the framework based on the artificial intelligence process device chip
What information and the corresponding binary instruction of the deep learning model obtained.
Another embodiment of the application provides a kind of method for processing video frequency, which comprises
Obtain the Schema information and deep learning model of artificial intelligence process device chip；
The deep learning model is compiled according to the Schema information of artificial intelligence process device chip, is generated corresponding
Binary instruction；
According to the Schema information of the artificial intelligence process device chip, the corresponding binary instruction of the deep learning model
Obtain off-line operation file；Wherein, the off-line operation file is handled for the input data of the deep learning model；
The input data is video to be processed.
Another method for processing video frequency that one embodiment of the application provides, which comprises
Obtain the Schema information and deep learning model of artificial intelligence process device chip；
The deep learning model is compiled according to the Schema information of artificial intelligence process device chip, is generated corresponding
Binary instruction；
The artificial intelligence process device chip receives picture frame and the binary instruction；Wherein, described image frame is base
It is obtained in video decoding to be processed；
The artificial intelligence process device chip executes the corresponding binary instruction of the deep learning model, to described image
Frame is handled.
The a kind of electronic equipment that one embodiment of the application provides, comprising: memory, processor and be stored in memory
Computer program that is upper and can running on a processor, which is characterized in that the processor is realized above-mentioned when executing described program
The method for processing video frequency.
It is executable to be stored thereon with processor for a kind of computer readable storage medium that one embodiment of the application provides
Program, run the executable program of the processor to execute method for processing video frequency described above.
Using the above method, the technical program is transformed the structure of deep learning model, in input layer and hidden layer
Between data prediction layer is set, pre- place can be completed to the picture frame of video in artificial intelligence process device (AI) chip in this way
(Data Format Transform) function is managed, to alleviate the burden of CPU.An image frame data is not needed simultaneously to transmit from decoding chip
To CPU, the copying into for data of progress between CPU and AI chip is not needed more and is copied out, thus reduce the number of data transmission, solution
Certainly IO memory access bottleneck problem.
Detailed description of the invention
Fig. 1 is flow chart of the prior art using depth learning technology processing video data.
Fig. 2 is the software stack structural schematic diagram of artificial intelligent processor chip；
Fig. 3 is one of the flow diagram of method for processing video frequency shown in one embodiment of the application.
Fig. 4 is the two of the flow diagram of the method for processing video frequency shown in one embodiment of the application.
Fig. 5 is the three of the flow diagram of the method for processing video frequency shown in one embodiment of the application.
Fig. 6 is the schematic diagram of the video process apparatus shown in the application one embodiment.
Fig. 7 is the schematic diagram of the electronic equipment shown in the application one embodiment.
Specific embodiment
Below in conjunction with the attached drawing in the embodiment of the present application, technical solutions in the embodiments of the present application carries out clear, complete
Site preparation description, it is clear that described embodiment is some embodiments of the present application, instead of all the embodiments.Based on this Shen
Please in embodiment, those skilled in the art's every other embodiment obtained without making creative work,
It shall fall in the protection scope of this application.
It should be appreciated that claims hereof, specification and attached drawing in term " first ", " second ", " third " and
" 4th " etc. is not use to describe a particular order for distinguishing different objects.The description and claims of this application
Used in term " includes " and "comprising" indicate described feature, entirety, step, operation, the presence of element and/or component,
But one or more of the other feature, entirety, step, operation, the presence or addition of element, component and/or its set is not precluded.
It is also understood that mesh of the term used in this present specification merely for the sake of description specific embodiment
, and be not intended to limit the application.As used in present specification and claims, unless context
Other situations are clearly indicated, otherwise " one " of singular, "one" and "the" are intended to include plural form.It should also be into one
Step understands that the term "and/or" used in present specification and claims refers to one in the associated item listed
A or multiple any combination and all possible combinations, and including these combinations.
For the ease of better understanding the technical program, technical term involved in the embodiment of the present application is first explained below.
The software stack of artificial intelligence process device: referring to fig. 2, which includes artificial intelligence application 200, artificial
Intelligent framework 202, artificial intelligence learning database 204, artificial intelligence run-time library 206 and driving 208.Next have to it
Body illustrates:
The corresponding different application scenarios of artificial intelligence application 200, provide corresponding intelligent algorithm model.The algorithm mould
Type can be parsed directly by the programming interface of artificial intelligence frame 202, in a wherein possible implementation, by artificial
Intelligence learning library 204 by intelligent algorithm model conversion be binary instruction, call artificial intelligence run-time library 206 to two into
System instruction is converted to artificial intelligence learning tasks, which is placed in task queue, by 208 scheduling of driving
Artificial intelligence learning tasks in task queue allow the artificial intelligence process device of bottom to execute.Another possible realization wherein
In mode, artificial intelligence run-time library 206 can also be called directly, the off-line operation file of previously cured generation is run, subtracts
The intermediate expense of few software architecture, improves operational efficiency.
Artificial intelligence process device chip: also referred to as application specific processor, for specific application or the processor in field.Example
Such as: graphics processor (Graphics Processing Unit, abbreviation: GPU) also known as shows core, vision processor, display
Chip is one kind specially in PC, work station, game machine and some mobile devices (such as tablet computer, smart phone)
The application specific processor of upper image operation work.Another example is: neural network processor (Neural Processing Unit, abbreviation:
It NPU), is a kind of application specific processor that matrix multiplication operation is directed in the application of artificial intelligence field, using " data-driven is simultaneously
The framework of row calculating " is especially good at the mass multimedia data of processing video, image class.
Reasoning (Inference): it is made prediction using trained model file to new data, it is only necessary to pass through network
Carry out propagated forward.Wherein, the step of solving supervision machine problem concerning study with neural network are as follows: using GPU to magnanimity number of tags
According to deep neural network training is carried out, iteration is needed when training carries out propagated forward and backpropagation by network, eventually
Generate trained model file.
Traditionally, decoding chip is decoded video to be processed, obtains corresponding picture frame.Picture frame is transmitted to
It is pre-processed on CPU, then the picture frame pre-processed is transmitted on AI chip and makes inferences processing.However, to figure on CPU
The consumption of cpu resource can be brought as frame does pretreatment；Image frame data to be processed is transferred to people again from decoder transfers to CPU
It is made inferences on work intelligent processor chip, there are biggish time overheads.
Based on this, the technical program is transformed the structure of deep learning model, so that between input layer and hidden layer
Data prediction layer is set, improved deep learning model is parsed through artificial intelligence process device chip software stack, acquisition pair
The binary instruction answered.Schema information and the deep learning model corresponding two based on the artificial intelligence process device chip
The off-line operation file that system instruction obtains.The picture frame of decoding chip output in this way is conveyed directly to artificial intelligence process device core
On piece, artificial intelligence process device chip execute the corresponding binary instruction of improved refreshing team learning model, carry out to picture frame
Processing.In this way, CPU is without pre-processing picture frame, picture frame is transferred to manually again there are no that need to be transferred to CPU from decoding chip
It is made inferences on intelligent processor chip.Solves IO memory access bottleneck problem while not consuming the resource of CPU.
Based on foregoing description, the application proposes a kind of technical solution, as shown in figure 3, one embodiment for the application is shown
A kind of one of the flow diagram of method for processing video frequency 1000 out.
As shown in 3 figures, in step S110, artificial intelligence process device chip receives picture frame.The picture frame is based on wait locate
What the video decoding of reason obtained.
In step S120, artificial intelligence process device chip execute in off-line operation file deep learning model corresponding two into
System instruction, handles picture frame.The off-line operation file is the Schema information based on the artificial intelligence process device chip
What binary instruction corresponding with deep learning model obtained.
In the present embodiment, deep learning model includes input layer, data prediction layer, hidden layer and output layer.Input
Input data of the output data of layer as data prediction layer.Input of the output data of data prediction layer as hidden layer
Data.Input data of the output data of hidden layer as output layer.Data prediction layer is used for the output data to input layer
Carry out the adjustment of color parameter, image size parameter.
For example, input layer is for receiving picture frame.The input of data prediction layer is the picture frame of the first format.Through data
Picture frame is converted to the second format by the first format by the processing of pretreatment layer.Data prediction layer exports the figure of the second format
As frame.Hidden layer is for making inferences calculating to the picture frame of the second format.The input of hidden layer can be data prediction layer
Output.Output layer, for the reasoning and calculation result of hidden layer to be exported to outside chip.
Optionally, data prediction layer can be used for data to be converted into rgb format by yuv format.Data prediction
It can be used for zooming in and out data, translate, the transformation such as rotation and alignment of data.
Optionally, the target that hidden layer can be used in input picture frame is classified, is detected, identified and is tracked.Into
One step, hidden layer can provide the calculating such as recognition of face, attitude detection and target tracking.
Optionally, the data format that received picture frame is in step S110 is the first format.The picture frame be based on pair
Video to be processed is decoded operation acquisition.
Handling picture frame in step S120 may include: that picture frame is converted to the second lattice by the first format
Formula；And calculating is made inferences to the picture frame of the second format, obtain reasoning and calculation result.
Further, the first format can be yuv format.Second format can be rgb format.For example, the first format
It can be YUV444, YUV422 or YUV420 format.Second format can be the formats such as RGB, BGR, RGBA, RGB0, herein
It is not listed one by one.
Optionally, the second format can also be the lattice after the calculating such as diminution, rotation, the zero padding carried out to the first format
Formula.
In practical applications, artificial intelligence process device chip obtains the corresponding binary instruction of deep learning model；Wherein,
The binary instruction is in off-line operation file；The off-line operation file includes the neural network model corresponding two
System instruction, constant table, input/output data scale, data layout description information and parameter information；Wherein, the data cloth
It is pre- that office's description information refers to that the hardware feature based on artificial intelligence process device chip carries out input/output data layout and type
Processing；Constant table, input/output data scale and parameter information are determined based on the neural network model；The parameter information
For the weight data in the neural network model；The constant table needs to make for being stored with during execution binary instruction
Data.
The artificial intelligence process device chip executes the binary instruction and handles described image frame.
In practical applications, not no oneself the operating system of artificial intelligence process device chip, through central processor CPU to mind
It is parsed through network model, translates into the binary instruction that artificial intelligence process device chip can be allowed to identify, binary system is referred to
It enables and the Schema information premature cure of artificial intelligence process device chip generates off-line operation file.Artificial intelligence process device in this way
Chip directly runs the off-line operation file of previous cured generation.
Using the above method, all pretreatments of picture frame can be completed in artificial intelligence process device (AI) chip, no
It needs image frame data to be transferred to central processing unit from decoding chip, pretreated image frame data is not needed to transmit yet
Give artificial intelligence process device chip.To reduce the number of data transmission, the working efficiency of video processing is improved.
As shown in figure 4, for a kind of flow diagram of method for processing video frequency 2000 shown in one embodiment of the application
Two.
As shown in figure 4, obtaining the Schema information and deep learning model of artificial intelligence process device chip in step S210.
In step S220, the deep learning model is compiled according to the Schema information of artificial intelligence process device chip, generation pair
The binary instruction answered.In step S230, according to the Schema information of the artificial intelligence process device chip, the deep learning mould
The corresponding binary instruction of type obtains off-line operation file；Wherein, the off-line operation file is used for the deep learning model
Input data handled；The input data is video to be processed.
Optionally, off-line operation file may include: the corresponding binary instruction of deep learning model, constant table, input/
Output data scale, data layout description information and parameter information.Data layout description information can refer to based on artificial intelligence
The hardware feature of processor chips is laid out to input/output data and type pre-processes.Constant table, input/output data
Scale and parameter information can be determined based on deep learning model.Parameter information is the weight data in deep learning model.Often
Number table needs data to be used for being stored with to execute in binary instruction calculating process.
Optionally, off-line operation file can be with further include: off-line operation file version information, artificial intelligence process device chip
Schema information.Wherein, the Schema information of artificial intelligence process device chip is by chip architecture version number or function statement come table
Sign.
It optionally, may include: input layer, preprocessing layer, hidden layer and output layer in the off-line operation file.Input layer
For receiving picture frame.Picture frame is converted to the second format by the first format by data prediction layer.Wherein the second format is used for
Reasoning and calculation.The input of data prediction layer can be the received picture frame of input layer.Hidden layer, for being carried out to picture frame
Reasoning and calculation.The input of hidden layer can be the output of data prediction layer.Output layer, for the reasoning and calculation knot hidden layer
Fruit exports to outside chip.
Optionally, data prediction layer can be used for data to be converted into rgb format by yuv format.Data prediction
It can be used for zooming in and out data, translate, the transformation such as rotation and alignment of data.
Further, data prediction layer can be also used for a picture frame and be converted to the second format by the first format.More into
One step, the first format can be yuv format.Second format can be rgb format.For example, the first format can be YUV444,
YUV422 YUV420 format.Second format can be the formats such as RGB, BGR, RGBA, RGB0, numerous to list herein.
In addition, the second format can also be the format after the calculating such as diminution, rotation, the zero padding carried out to the first format.
Optionally, the target that hidden layer can be used in input picture frame is classified, is detected, identified and is tracked.Into
One step, hidden layer can provide the calculating such as recognition of face, attitude detection and target tracking.
Using the above method generate off-line operation file, under be loaded in the memory of artificial intelligence process device chip.Offline
The structure of the corresponding deep learning model of binary instruction in operating file changes, and increases data prediction layer.When
The off-line operation file is performed, which can call directly the binary system in off-line operation file
Instruction, first pre-processes picture frame, is then based on pretreatment and makes inferences calculating.Since picture frame is directly entered at artificial intelligence
It manages on device chip, needs not move through CPU, alleviate the burden of central processing unit.Reduce the number of data transmission, Jin Erti simultaneously
Data-handling efficiency is risen.
As shown in figure 5, three of the flow diagram for the method for processing video frequency 3000 shown in one embodiment of the application.
As shown in figure 5, step S310, obtains the Schema information and deep learning model of artificial intelligence process device chip.Step
Rapid S320 is compiled deep learning model according to the Schema information of artificial intelligence process device chip, generate corresponding two into
System instruction.Step S330, artificial intelligence process device chip receive picture frame and binary instruction.Step S340, at artificial intelligence
It manages device chip and executes the corresponding binary instruction of deep learning model, picture frame is handled.
Optionally, the deep learning model in step S310 may include preprocessing layer, be used for picture frame by the first lattice
Formula is converted into the second format.
Optionally, handling picture frame in step S340 may include: that picture frame is converted by the first format
Second format.Further, the first format can be yuv format.Second format can be rgb format.For example, the first format
It can be YUV444, YUV422 or YUV420 format.Second format can be the formats such as RGB, BGR, RGBA, RGB0, herein
It is not listed one by one.
Optionally, the second format can also be the lattice after the calculating such as diminution, rotation, the zero padding carried out to the first format
Formula.Correspondingly, preprocessing layer can be used for zooming in and out the image in picture frame, translate, rotate, and can be used for data pair
Together.
Optionally, step S330 received picture frame be decoding chip generate decoding picture frame.
Using the above method, can in artificial intelligence process device (AI) chip to the picture frame of decoder chip output into
The function of row reasoning operation, to alleviate the burden of CPU.The technical program does not need an image frame data from decoding core simultaneously
Piece is transferred to central processing unit, does not need more for image frame data to be transferred to artificial intelligence process device chip.To reduce number
According to the number of transmission, the working efficiency of video processing is improved.
As shown in fig. 6, this application provides the video process apparatus 5000 shown in one embodiment, including image receives mould
Block 501 and processing module 502.
Image receiver module 501 receives picture frame, which obtained based on video decoding to be processed.Handle mould
Block 502 executes the corresponding binary instruction of deep learning model in off-line operation file, to the received figure of image receiver module 501
As frame is handled.The off-line operation file is Schema information and deep learning model pair based on artificial intelligence process device chip
What the binary instruction answered obtained.
As shown in fig. 7, the embodiment of the present application also provides a kind of electronic equipment.The electronic equipment can be a kind of chip.It should
Chip may include output unit 601, input unit 602, processor 603, memory 604, communication interface 605 and memory
Unit 606.
Memory 604 is used as a kind of non-transient computer readable memory, and can be used for storing software program, computer can hold
Line program and module, it is corresponding for a kind of method for processing video frequency based on artificial intelligence process device chip as described above
Program instruction/module.
Processor 603 is stored in software program, instruction and module in storage cut-off by operation, thereby executing electronics
The various function application and data processing of equipment, the i.e. method of realization above-described embodiment description.
Memory 604 may include storing program area and storage data area, wherein storing program area can store operation system
Application program required for system, at least one function；Storage data area, which can be stored, uses created number according to electronic device
According to etc..In addition, memory 604 may include high-speed random access memory, it can also include non-transitory memory, such as extremely
A few disk memory, flush memory device or other non-transitory solid-state memories.In some embodiments, memory
604 it is optional include the memory remotely located relative to processor 603, these remote memories can pass through network connection to electricity
Sub- equipment.
Present invention also provides one embodiment, which includes: decoder and artificial intelligence process device core
Piece.Wherein, decoder is decoded to obtain picture frame to the video frame of input.Artificial intelligence process device chip carries out picture frame
Pretreatment and reasoning and calculation, obtain reasoning and calculation result according to pretreated picture frame.
Optionally, image processing apparatus further include:
First memory is connect with decoder, and first memory can also be connect with artificial treatment chip.Decoder and people
Work intelligent processor chip slaughters dragon and crosses first memory communication connection.Further, decoder is decoding calculated result deposit the
First presumptive area of one memory, artificial intelligence process device obtain picture frame from the first presumptive area of first memory.
Closer, second fate of the artificial intelligence process device chip reasoning and calculation result deposit first memory
Domain, central processing unit are connect with connection types such as PCIe buses with first memory.Central processing unit is by dma mode, from the
Second presumptive area of one memory obtains reasoning and calculation result.
Optionally, decoder and artificial intelligence process device chip pass through assembly line cooperating.
Utilize above-mentioned image processing apparatus, it may not be necessary to which central processing unit carries out pre-treatment operation, does not also need solution
Image data after code is transferred to central processing unit, does not need the data that pre-treatment operation obtains more and transmits from central processing unit
To artificial intelligence process device chip.To alleviate the work load of central processing unit, reduce the number of data transmission, in turn
Improve the working efficiency of video processing.
The embodiment of the present application also provides a kind of computer readable storage medium, is stored thereon with the executable journey of processor
Sequence, processor execute the program for executing process as described above.
It should be understood that above-mentioned Installation practice is only illustrative, the device of the application can also be by another way
It realizes.For example, the division of units/modules described in above-described embodiment, only a kind of logical function partition, in actual implementation may be used
To there is other division mode.For example, multiple units, module or component can combine, or be desirably integrated into another system,
Or some features can be ignored or does not execute.
The unit as illustrated by the separation member or module can be and be physically separated, and may not be and physically divides
It opens.It can be physical unit as unit or the component of module declaration, may not be physical unit, it can be located at one
In device, or it may be distributed on multiple devices.The scheme of embodiment can select according to the actual needs in the application
Some or all of unit therein is realized.
In addition, unless otherwise noted, each functional unit/module in each embodiment of the application can integrate at one
In units/modules, it is also possible to each unit/module and physically exists alone, it can also be with two or more units/modules collection
At together.Above-mentioned integrated units/modules both can take the form of hardware realization, can also be using software program module
Form is realized.
If the integrated units/modules are realized in the form of hardware, which can be digital circuit, simulation electricity
Road etc..The physics realization of hardware configuration includes but is not limited to transistor, memristor etc..Unless otherwise noted, the place
Reason device can be any hardware processor, such as CPU, GPU, FPGA, DSP and ASIC appropriate etc..Unless otherwise noted, institute
Stating storage unit can be any magnetic storage medium appropriate or magnetic-optical storage medium, for example, resistive formula memory RRAM
(Resistive Random Access Memory), dynamic random access memory DRAM (Dynamic Random Access
Memory), static random access memory SRAM (Static Random-Access Memory), enhancing dynamic randon access
Memory EDRAM (Enhanced Dynamic Random Access Memory), high bandwidth memory HBM (High-
Bandwidth Memory), mixing storage cube HMC (Hybrid Memory Cube) etc..
If the integrated units/modules realized in the form of software program module and as independent product sale or
In use, can store in a computer-readable access to memory.Based on this understanding, the technical solution essence of the application
On all or part of the part that contributes to existing technology or the technical solution can be with the shape of software product in other words
Formula embodies, which is stored in a memory, including some instructions are used so that a computer
Equipment (can for personal computer, server or network equipment etc.) execute each embodiment the method for the application whole or
Part steps.And memory above-mentioned includes: USB flash disk, read-only memory (ROM, Read-Only Memory), random access memory
Various Jie that can store program code such as device (RAM, Random Access Memory), mobile hard disk, magnetic or disk
Matter.
In the above-described embodiments, it all emphasizes particularly on different fields to the description of each embodiment, there is no the portion being described in detail in some embodiment
Point, reference can be made to the related descriptions of other embodiments.Each technical characteristic of above-described embodiment can be combined arbitrarily, to make
Description is succinct, and combination not all possible to each technical characteristic in above-described embodiment is all described, as long as however, these
Contradiction is not present in the combination of technical characteristic, all should be considered as described in this specification.
The embodiment of the present application is described in detail above, specific case used herein to the principle of the application and
Embodiment is expounded, and the explanation of above embodiments is only used for helping to understand the present processes and its core concept.Together
When, those skilled in the art according to the thought of the application, make in specific embodiment and application range based on the application
Change or deform place, shall fall in the protection scope of this application.In conclusion the content of the present specification should not be construed as to the application
Limitation.
Claims (13)
1. a kind of method for processing video frequency, which is characterized in that the described method includes:
Artificial intelligence process device chip receives picture frame；Wherein, described image frame is obtained based on video decoding to be processed；
The artificial intelligence process device chip executes the corresponding binary instruction of deep learning model in off-line operation file, to institute
Picture frame is stated to be handled；Wherein, the off-line operation file is the Schema information based on the artificial intelligence process device chip
What binary instruction corresponding with the deep learning model obtained.
2. the method as described in claim 1, which is characterized in that the deep learning model includes input layer, data prediction
Layer, hidden layer and output layer；Input data of the output data of the input layer as the data prediction layer；The data
Input data of the output data of pretreatment layer as the hidden layer, the output data of the hidden layer is as the defeated of output layer
Enter data.
3. method according to claim 2, which is characterized in that the data prediction layer is for the output to the input layer
Data carry out the adjustment of color parameter, image size parameter, and described image frame is converted to the second format by the first format.
4. according to the method described in claim 2, wherein, the off-line operation file includes: that the deep learning model is corresponding
Binary instruction, constant table, input/output data scale, data layout description information and parameter information；Wherein, the number
Refer to the hardware feature based on the artificial intelligence process device chip to input/output data layout and class according to layout description information
Type is pre-processed；The constant table, input/output data scale and parameter information are determined based on the neural network model；
The parameter information is the weight data in the neural network model；The constant table is for being stored with execution binary instruction
Data to be used are needed in the process.
5. a kind of method for processing video frequency, which is characterized in that the described method includes:
Obtain the Schema information and deep learning model of artificial intelligence process device chip；
The deep learning model is compiled according to the Schema information of artificial intelligence process device chip, generate corresponding two into
System instruction；
It is obtained according to the Schema information of the artificial intelligence process device chip, the corresponding binary instruction of the deep learning model
Off-line operation file；Wherein, the off-line operation file is handled for the input data of the deep learning model；It is described
Input data is video to be processed.
6. method as claimed in claim 5, which is characterized in that the off-line operation file includes the deep learning model pair
Binary instruction, constant table, input/output data scale, data layout description information and the parameter information answered；Wherein, described
Data layout description information refers to the hardware feature based on artificial intelligence process device chip to input/output data layout and type
It is pre-processed；Constant table, input/output data scale and parameter information are determined based on the deep learning model；The ginseng
Number information is the weight data in the deep learning model；The constant table executes binary instruction operation for being stored with
Data to be used are needed in journey.
7. method as claimed in claim 5, which is characterized in that the off-line operation file further include: off-line operation file version
The Schema information of this information, artificial intelligence process device chip；Wherein, the Schema information of the artificial intelligence process device chip passes through
Chip architecture version number or function statement are to characterize.
8. method as claimed in claim 5, which is characterized in that the deep learning model includes input layer, data prediction
Layer, hidden layer and output layer；The input data of the input layer is described image frame；The output data of the input layer is as institute
State the input data of data prediction layer；Input data of the output data of the data prediction layer as the hidden layer,
Input data of the output data of the hidden layer as output layer.
9. method according to claim 8, which is characterized in that the data prediction layer to picture frame carry out color parameter,
Described image frame is converted to the second format by the first format by the adjustment of image size parameter.
10. a kind of method for processing video frequency, which is characterized in that the described method includes:
Obtain the Schema information and deep learning model of artificial intelligence process device chip；
The deep learning model is compiled according to the Schema information of artificial intelligence process device chip, generate corresponding two into
System instruction；
The artificial intelligence process device chip receives picture frame and the binary instruction；Wherein, described image frame be based on to
What the video decoding of processing obtained；
The artificial intelligence process device chip executes the corresponding binary instruction of the deep learning model, to described image frame into
Row processing.
11. according to the method described in claim 10, it is characterized in that, the depth model includes preprocessing layer, the figure
As frame is converted to the second format from the first format.
12. a kind of electronic equipment, comprising: memory, processor and storage are on a memory and the meter that can run on a processor
Calculation machine program, which is characterized in that the processor realizes the described in any item methods of Claims 1 to 4 when executing described program；
Or the processor realizes claim 5~9 described in any item methods when executing described program；Or the processor is held
Claim 10~11 described in any item methods are realized when row described program.
13. a kind of computer readable storage medium is stored thereon with the executable program of processor, which is characterized in that operation institute
It states the executable program of processor and 1~4 described in any item methods is required with perform claim；Or the operation processor can be held
Capable program requires 5~9 described in any item methods with perform claim；Or the program that the operation processor can be performed is to hold
The described in any item methods of row claim 10~11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910738333.7A CN110493598A (en) | 2019-08-12 | 2019-08-12 | Method for processing video frequency and relevant apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910738333.7A CN110493598A (en) | 2019-08-12 | 2019-08-12 | Method for processing video frequency and relevant apparatus |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110493598A true CN110493598A (en) | 2019-11-22 |
Family
ID=68550351
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910738333.7A Pending CN110493598A (en) | 2019-08-12 | 2019-08-12 | Method for processing video frequency and relevant apparatus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110493598A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180121732A1 (en) * | 2016-11-03 | 2018-05-03 | Samsung Electronics Co., Ltd. | Data recognition model construction apparatus and method for constructing data recognition model thereof, and data recognition apparatus and method for recognizing data thereof |
| CN108108662A (en) * | 2017-11-24 | 2018-06-01 | 深圳市华尊科技股份有限公司 | Deep neural network identification model and recognition methods |
| CN108712630A (en) * | 2018-04-19 | 2018-10-26 | 安凯(广州)微电子技术有限公司 | A kind of internet camera system and its implementation based on deep learning |
| CN109034159A (en) * | 2018-05-28 | 2018-12-18 | 北京捷通华声科技股份有限公司 | image information extracting method and device |
| CN109544440A (en) * | 2018-12-29 | 2019-03-29 | 北京知存科技有限公司 | A kind of picture processing chip, System and method for |
| CN109711367A (en) * | 2018-12-29 | 2019-05-03 | 北京中科寒武纪科技有限公司 | Operation method, device and Related product |
| CN109862391A (en) * | 2019-03-18 | 2019-06-07 | 网易（杭州）网络有限公司 | Video classification methods, medium, device and calculating equipment |
-
2019
- 2019-08-12 CN CN201910738333.7A patent/CN110493598A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180121732A1 (en) * | 2016-11-03 | 2018-05-03 | Samsung Electronics Co., Ltd. | Data recognition model construction apparatus and method for constructing data recognition model thereof, and data recognition apparatus and method for recognizing data thereof |
| CN108108662A (en) * | 2017-11-24 | 2018-06-01 | 深圳市华尊科技股份有限公司 | Deep neural network identification model and recognition methods |
| CN108712630A (en) * | 2018-04-19 | 2018-10-26 | 安凯(广州)微电子技术有限公司 | A kind of internet camera system and its implementation based on deep learning |
| CN109034159A (en) * | 2018-05-28 | 2018-12-18 | 北京捷通华声科技股份有限公司 | image information extracting method and device |
| CN109544440A (en) * | 2018-12-29 | 2019-03-29 | 北京知存科技有限公司 | A kind of picture processing chip, System and method for |
| CN109711367A (en) * | 2018-12-29 | 2019-05-03 | 北京中科寒武纪科技有限公司 | Operation method, device and Related product |
| CN109862391A (en) * | 2019-03-18 | 2019-06-07 | 网易（杭州）网络有限公司 | Video classification methods, medium, device and calculating equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11087504B2 (en) | Transforming grayscale images into color images using deep neural networks | |
| CN111386536A (en) | Semantically consistent image style conversion | |
| US20210019555A1 (en) | Generating video frames using neural networks | |
| CN111797992A (en) | Machine learning optimization method and device | |
| Hidaka et al. | Webdnn: Fastest dnn execution framework on web browser | |
| CN111178507A (en) | Atlas convolution neural network data processing method and device | |
| TW202022719A (en) | Model-based prediction method and device | |
| CN113159273B (en) | Neural network training method and related equipment | |
| CN111915555B (en) | 3D network model pre-training method, system, terminal and storage medium | |
| CN110717584A (en) | Neural network compiling method, compiler, computer device, and readable storage medium | |
| CN113095475A (en) | Neural network training method, image processing method and related equipment | |
| CN113065997B (en) | Image processing method, neural network training method and related equipment | |
| WO2021120177A1 (en) | Method and apparatus for compiling neural network model | |
| CN109711540B (en) | Computing device and board card | |
| CN113850012B (en) | Data processing model generation method, device, medium and electronic equipment | |
| CN110490317A (en) | Neural network computing device and operation method | |
| Shi et al. | Object detection based on Swin Deformable Transformer‐BiPAFPN‐YOLOX | |
| CN113850298A (en) | Image identification method and device and related equipment | |
| CN109325512A (en) | Image classification method and device, electronic equipment, computer program and storage medium | |
| CN113570678A (en) | Image coloring method and device based on artificial intelligence and electronic equipment | |
| WO2023185541A1 (en) | Model training method and related device | |
| CN110503596A (en) | Method for processing video frequency, device, electronic equipment and computer readable storage medium | |
| CN110493598A (en) | Method for processing video frequency and relevant apparatus | |
| Tiwari et al. | NCS based ultra low power optimized machine learning techniques for image classification | |
| CN116168267A (en) | Training method, device, equipment and medium for multi-task neural network model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Room 644, scientific research complex building, No. 6, South Road, Academy of Sciences, Haidian District, Beijing 100086 Applicant after: Zhongke Cambrian Technology Co.,Ltd. Address before: Room 644, scientific research complex building, No. 6, South Road, Academy of Sciences, Haidian District, Beijing 100086 Applicant before: Beijing Zhongke Cambrian Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191122 |
|
| RJ01 | Rejection of invention patent application after publication |