CN115147840B - Artificial intelligence system and method for character recognition - Google Patents
Artificial intelligence system and method for character recognition Download PDFInfo
- Publication number
- CN115147840B CN115147840B CN202110355563.2A CN202110355563A CN115147840B CN 115147840 B CN115147840 B CN 115147840B CN 202110355563 A CN202110355563 A CN 202110355563A CN 115147840 B CN115147840 B CN 115147840B
- Authority
- CN
- China
- Prior art keywords
- image
- image data
- shared memory
- character recognition
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013473 artificial intelligence Methods 0.000 title claims abstract description 159
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000012545 processing Methods 0.000 claims abstract description 47
- 239000000872 buffer Substances 0.000 claims description 38
- 230000005540 biological transmission Effects 0.000 claims description 32
- 230000008569 process Effects 0.000 claims description 23
- 238000004422 calculation algorithm Methods 0.000 claims description 17
- 238000012805 post-processing Methods 0.000 claims description 12
- 238000003709 image segmentation Methods 0.000 claims description 8
- 230000003139 buffering effect Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000011176 pooling Methods 0.000 claims description 4
- 238000012549 training Methods 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 2
- 238000007781 pre-processing Methods 0.000 description 13
- 230000006870 function Effects 0.000 description 8
- 238000012544 monitoring process Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Landscapes
- Character Discrimination (AREA)
Abstract
An artificial intelligence system and method for character recognition, the artificial intelligence system comprising: a shared memory; the processing chip is used for analyzing the artificial intelligent AI model for character recognition, determining an operator to be operated and the operation sequence thereof and storing the operator and the operation sequence thereof into the shared memory; the programmable logic device is used for acquiring the acquired original image containing the character to be recognized, processing the image into image data, storing the image data into the shared memory, reading the operation sequence from the shared memory, scheduling the corresponding operation operator to operate the image data, obtaining an image character recognition result and storing the image character recognition result into the shared memory; the processing chip is also used for reading image data from the shared memory and inputting the image data into the programmable logic device. The embodiment of the application has the advantages of low power consumption, low delay, low cost, easy expansion and the like, and is suitable for being used in an edge embedded system.
Description
Technical Field
Embodiments of the present application relate to the field of artificial intelligence, and more particularly, but not exclusively, to an artificial intelligence system and method for character recognition.
Background
With the development and wide application of AI (ARTIFICIAL INTELLIGENCE ) technology, an increasing challenge is presented to AI computation in different scenarios. The application of AI calculation gradually expands from the initial cloud to the edge embedded system.
In some technologies, when an AI method is applied to an edge embedded system to identify characters, the power consumption and the cost of the whole system are high, and a high-complexity system is unfavorable for arrangement, so that the cost performance of the edge embedded system for identifying characters by applying the AI method is not high.
Disclosure of Invention
The following is a summary of the subject matter described in detail herein and is not intended to limit the scope of the appended claims.
The embodiment of the application provides an artificial intelligence system for character recognition, which comprises the following components:
a shared memory;
The processing chip is used for analyzing the artificial intelligent AI model for character recognition, determining an operator to be operated and the operation sequence thereof, and storing the operator and the operation sequence into the shared memory;
the programmable logic device is used for acquiring the acquired original image containing the character to be recognized, processing the image into image data, storing the image data into the shared memory, reading the operation sequence from the shared memory, scheduling a corresponding operator to operate the image data, obtaining an image character recognition result and storing the image character recognition result into the shared memory;
The processing chip is also used for reading the image data from the shared memory and inputting the image data into the programmable logic device.
The embodiment of the application also provides a method for character recognition, which is applied to the artificial intelligence system for character recognition, and comprises the following steps:
The processing chip analyzes an artificial intelligence AI model for character recognition, determines an operator to be operated and an operation sequence thereof, and stores the operator and the operation sequence in a shared memory;
the programmable logic device acquires the acquired original image containing the character to be recognized, processes the original image into image data and stores the image data into the shared memory;
the processing chip reads the image data and inputs the image data to the programmable logic device;
And the programmable logic device reads the operation sequence scheduling corresponding operation operator from the shared memory, performs operation on the image data to obtain an image character recognition result, and stores the image character recognition result in the shared memory.
The AI system of the embodiment of the application can realize character recognition by adopting a lightweight and low-power-consumption device, reduces the complexity of AI model inference and character recognition, solves the problems of high complexity, high power consumption and high cost when an AI method is applied to recognize characters in an edge embedded system, can load the AI model for character recognition by using a low-power-consumption lightweight system at the middle and low ends, can reduce the power consumption of the whole system, reduce the cost and improve the cost performance and the efficiency of recognizing the characters by applying the AI method in the edge embedded system.
Other aspects will become apparent upon reading and understanding the accompanying drawings and detailed description.
Drawings
The accompanying drawings are included to provide an understanding of the principles of the application, and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain, without limitation, the principles of the application.
FIG. 1 is a schematic diagram of an artificial intelligence system for character recognition according to an embodiment of the present application;
FIG. 2 is a schematic diagram of an AI system implemented using a system-on-chip in one embodiment;
FIG. 3 is a flow chart of a method for performing character recognition according to an embodiment of the present application;
FIG. 4 is a schematic flow chart of digital identification in an example;
fig. 5 is a schematic structural diagram of an AI system for digital recognition in an example;
FIG. 6 is a schematic diagram of an image preprocessing module in an example;
fig. 7 is a schematic diagram of an example digitally identified AI model parallel inference module.
Detailed Description
The present application has been described in terms of several embodiments, but the description is illustrative and not restrictive, and it will be apparent to those of ordinary skill in the art that many more embodiments and implementations are possible within the scope of the described embodiments. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are possible. Any feature or element of any embodiment may be used in combination with or in place of any other feature or element of any other embodiment unless specifically limited.
The present application includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The disclosed embodiments, features and elements of the present application may also be combined with any conventional features or elements to form a unique inventive arrangement as defined in the appended claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive arrangements to form another unique inventive arrangement as defined in the appended claims. It is therefore to be understood that any of the features shown and/or discussed in the present application may be implemented alone or in any suitable combination. Accordingly, the embodiments are not to be restricted except in light of the attached claims and their equivalents. Further, various modifications and changes may be made within the scope of the appended claims.
Furthermore, in describing representative embodiments, the specification may have presented the method and/or process as a particular sequence of steps. However, to the extent that the method or process does not rely on the particular order of steps set forth herein, the method or process should not be limited to the particular sequence of steps described. Other sequences of steps are possible as will be appreciated by those of ordinary skill in the art. Accordingly, the particular order of the steps set forth in the specification should not be construed as limitations on the claims below. Furthermore, the claims directed to the method and/or process should not be limited to the performance of their steps in the order written, and one skilled in the art can readily appreciate that the sequences may be varied and still remain within the spirit and scope of the embodiments of the present application.
An embodiment of the present application provides an artificial intelligence system for character recognition, as shown in fig. 1, including:
A shared memory 10;
the processing chip 11 is used for analyzing the artificial intelligent AI model for character recognition, determining an operator to be operated and the operation sequence thereof, and storing the operator and the operation sequence thereof into the shared memory 10;
the programmable logic device 12 is configured to acquire an acquired original image containing a character to be identified, process the acquired original image into image data, store the image data in the shared memory 10, read an operation sequence from the shared memory 10, schedule a corresponding operator to operate on the image data, obtain an image character identification result, and store the image character identification result in the shared memory;
the processing chip 11 is also used to read image data from the shared memory 10 and input it into the programmable logic device 12.
The AI system of the embodiment can adopt lightweight and low-power-consumption devices to realize character recognition, reduce complexity of AI model inference and character recognition, solve the problems of high complexity, high power consumption and high cost when the AI method is applied to identify characters in the edge embedded system, load the AI model for character recognition by using a low-power-consumption lightweight system at the middle and low ends, reduce the power consumption of the whole system, reduce the cost and improve the cost performance and the efficiency of the edge embedded system for identifying characters by applying the AI method.
In this embodiment, the AI model for character recognition may be any AI model that can recognize and detect characters from an original image, and may be obtained by a machine learning method. The character recognition result can be regarded as an output result of the AI model (or an inferred conclusion called the AI model) when the image data is input to the AI model for character recognition.
In this embodiment, the original image may be output together when the character recognition result is output.
In this embodiment, each operator may be considered as a layer in the AI model, or as a step in the operation performed by the AI model. The operators may be saved, but not limited to, in the form of functions or tasks, etc., and the corresponding operators may be called, but not limited to, by function names or task names, etc., when scheduling the operators according to the execution order.
In this embodiment, a plurality of operators may be loaded in advance in the programmable logic device, where loading operators may, but is not limited to, configuring the programmable logic device according to a hardware description language for implementing the operators, or implementing the operators in the programmable logic device through other hardware implementation manners. The operators loaded in the programmable logic device contain all operators in the AI model to be run.
The AI model for character recognition in the present embodiment may be an AI model employed in any one of the character recognition fields, and the present application is not limited thereto. The AI model may be a pre-trained model; the AI model may be updated during the recognition process.
In this embodiment, various forms of shared memory may be employed, or the shared memory may be included in a programmable logic device.
In some exemplary embodiments, the programmable logic device processes the raw image into the image data, comprising: dividing the original image into a plurality of groups of image data, wherein each group of image data corresponds to an image of a character;
the step of scheduling corresponding operation operators according to the operation sequence to operate the image data to obtain an image character recognition result comprises the following steps: a plurality of scheduled model operation accelerator cores perform operation in parallel; each scheduled model operation accelerator core respectively carries out operation on one group of image data to obtain the image character recognition result corresponding to the group of image data;
The processing chip is further used for scheduling the corresponding number of model operation accelerator cores in the programmable logic device according to the number of the groups of the image data, reading the plurality of groups of the image data from the shared memory, inputting the image data into the model operation accelerator cores in a one-to-one correspondence mode, and combining the image character recognition results corresponding to the plurality of groups of the image data into a final recognition result.
In the present embodiment, by dividing the original image into image data of the corresponding group number in accordance with the number of characters contained, image character recognition can be facilitated. For some AI models, for example, the digital identification AI model, the numbers 0 to 9 are identified based on unit numbers, i.e., the AI model infers 1-bit numbers at a time, so that an image segmentation process is required, for the digital identification AI model, the acquired original image may be segmented, and the N-bit numbers are segmented into N groups of image data, each group of image data running an AI model inference to identify the numbers of the group of image data.
In other embodiments, any number of characters may be divided into a group, the original image may be divided, or the character recognition may be directly performed without dividing the original image, and the character recognition may be set according to the AI model used, which is not limited in this aspect of the present application.
In this embodiment, all existing operators of the programmable logic device where each model operation accelerator core is located can be used, so that no matter which operators are determined to be operated by the processing chip, each model operation accelerator core can be guaranteed to have the capability of independently completing operation, and all hardware implementations of required operation operations are included in the accelerator core. Each operator may be distributed at a different address and timing, and the model operation accelerator core may schedule according to the address and timing of the operator when using a particular operator.
In this embodiment, after the original image is divided into multiple sets of image data, each set of image data is respectively input to a model operation accelerator core corresponding to one to perform parallel operation, and the processing mode of the parallel operation enables an AI model inference conclusion for all characters to be generated within the time of single character recognition, thereby playing the role of accelerating operation.
In this embodiment, determining the model operation accelerator core to be operated, and combining the image character recognition results corresponding to the multiple sets of image data into the final recognition result is completed by the processing chip, when the processing chip inputs N sets of image data into the N model operation accelerator cores, the processing chip may sequentially input, for example, the image data of the first character is input into the first model operation accelerator core, the image data of the second character is input into the second model operation accelerator core, and so on, and then the image character recognition results of each image data are respectively obtained according to the input sequence, and the final recognition result obtained after sequentially combining in sequence is the character to be recognized in the original image.
In other embodiments, a preset algorithm may be used to allocate image data to the model operation accelerator cores, and corresponding addresses storing the recognition results of the image characters may be allocated in advance to each image data or each model operation accelerator core to be operated, so that the sequence of the characters to be recognized in the original image can be restored during combination, and the purpose may be achieved in other manners.
In other embodiments, multiple characters in the sequence of characters to be recognized may be recognized in a serial fashion, or multiple characters may be recognized at once, such that only one model operation accelerator core may be invoked.
In some exemplary embodiments, the character to be recognized is a reading on a dashboard;
the artificial intelligence system further comprises:
The wireless transmission module is used for sending the final identification result;
the processing chip is also used for driving the wireless transmission module and sending the final identification result to the wireless transmission module.
In the market application of instruments and meters, intelligent instruments such as intelligent electric meters, intelligent instruments, intelligent gas meters and the like are mostly used, and functions of remote meter reading, remote control, monitoring and the like can be realized by a built-in processor unit (CPU) and a wired and wireless transmission module. However, the old non-intelligent instruments and meters in the early stage have no remote meter reading function and need manual meter reading. If the instrument is in the severe environments such as underground mine, the difficulty and the danger coefficient of manual meter reading are large, and the condition monitoring is not facilitated. If the intelligent instrument is replaced, not only is the resource waste caused, but also great expenditure is brought. The artificial intelligent system of the embodiment completes the function of using the AI model to monitor the instrument together through the mutual matching of the processing chip and the programmable logic device, so that the AI-friendly non-intelligent instrument monitoring system based on the edge embedded system is formed, the respective advantages of the processing chip and the programmable logic device can be fully utilized, the reading identification and monitoring of the old non-intelligent instrument can be realized only by less logic resources and limited data computing capacity, the identified reading can be transmitted to a remote terminal in a wireless way, the manpower resources are saved, and the AI can be applied to realize the digital identification and remote monitoring of the non-intelligent instrument only by being additionally arranged at the front end of the non-intelligent instrument, the application field of the AI is expanded, and the intelligent system has the advantages of low power consumption, low delay, low cost, easiness in expansion and the like.
In some exemplary embodiments, the processing chip may be, but is not limited to, an MCU (Micro Control Unit ) and the programmable logic device may be, but is not limited to, an FPGA (Field Programmable GATE ARRAY ); the MCU is connected with the shared memory through a system bus, and the FPGA is connected with the shared memory through a parallel bus; the operator is realized by an FPGA kernel.
In this embodiment, the AI system for character recognition may be a system on a chip.
In this embodiment, the operations performed by the processing chip and the programmable logic device described above may be, but are not limited to, performed by an MCU core and an FPGA core, respectively.
The embodiment can realize an AI system through an MCU and an FPGA system on chip; the MCU and the FPGA system-on-chip refers to a system-on-chip which is formed by connecting an FPGA, an MCU, a memory, an external device and the like with an FPGA core. Based on the programmable characteristic of the FPGA, the architecture has good expansibility.
In one implementation manner of the embodiment, the AI System uses a lightweight MCU and a SoC (System On Chip) of a low-power FPGA as shown in fig. 2 as a carrier, and performs data interaction between the MCU and the FPGA through a shared memory, so as to realize character recognition. The system on chip can be externally connected with a camera to obtain the acquired original image, can directly output an image character recognition result or a further processing result by the MCU, and can also be internally or externally connected with a wireless transmission module to send the image character recognition result or the further processing result.
In one implementation of this embodiment, the operators include one or more of the following types of operators: full join operation (full Connected) operator, matrix transform operation (Reshape) operator, flexible maximum transmission operation (SoftMax) operator, normal convolution operation (Conv 2D) operator, deep convolution operation (DepthwiseConv D) operator, max pooling operation (MaxPool D) operator, average pooling operation (AveragePool D) operator.
AI models containing the above-mentioned non-enumerated operators may be selected as desired, as the present application is not limited in this regard.
In this embodiment, the AI model may be considered to include, in addition to the operators, an input layer and an output layer, which are respectively configured to input image data (directly or through an input/output buffer) to a corresponding operator in the AI model, and output the image character recognition result directly to the shared memory or through the input/output buffer to the shared memory; the input layer may be implemented by an MCU core and the output layer may be implemented by an FPGA.
In some exemplary embodiments, the programmable logic device includes:
The camera controller is used for controlling the camera to collect images and controlling the reading and writing time sequence and the cache of the collected original images;
The first input/output buffer is used for buffering the original image acquired by the camera and sending the original image to the shared memory through the parallel bus.
In this embodiment, a camera may be disposed in front of the artificial intelligence system, and the programmable logic device controls the camera to collect images through the camera controller, buffers the collected original images in the first input/output buffer, and then sends the collected original images to the shared memory through the parallel bus.
In other embodiments, the HDMI device may be used to transmit the original image to the artificial intelligence system after the image is acquired, and the present application does not limit the form of acquiring the original image.
In one implementation of this embodiment, the programmable logic device may further include:
a second input-output buffer for holding the original image read from the shared memory;
The image gray level algorithm module is used for carrying out gray level binarization processing on the original image and converting the original image into a gray level image;
The image segmentation algorithm module is used for segmenting the gray image into images with preset sizes and buffering the images into the second input/output buffer;
The image enhancement algorithm module is used for carrying out Fourier transform on the image with the preset size to obtain the image data, and caching the image data into the second input/output buffer;
The second input-output buffer is further configured to store the image data to the shared memory via a parallel bus.
After the image segmentation algorithm module segments the images, the images with preset sizes obtained after segmentation can be sequentially stored in an N-segment address space of the second input/output buffer according to the arrangement sequence before segmentation; the image enhancement algorithm module may sequentially perform fourier transform on the segmented image and sequentially store the segmented image back into the N-segment address space of the second input-output buffer. The input/output buffer may be separately provided to sequentially store the divided or enhanced images, which is not limited in the embodiment of the present application.
In this embodiment, the camera controller and the modules may be configured in the FPGA core through, but not limited to, a hardware description language.
In one implementation manner of this embodiment, the processing chip may include:
The AI model interpreter is used for analyzing the AI model for character recognition, calculating the data type and all operators of the AI model, determining the operators to be operated and the operation sequence thereof according to the calculation result, and storing the operators and the operation sequence thereof into the shared memory;
and the AI model inference conclusion post-processing module is used for combining a plurality of groups of image character recognition results into the final recognition result and outputting the final recognition result.
In this embodiment, the AI model interpreter may be implemented, but is not limited to, by software code stored in the MCU core.
In this embodiment, the MCU may read the image data from the shared memory through the third input/output buffer, input the image data to the corresponding model operation accelerator core, and after the model operation accelerator core completes the operation and saves the image character recognition result to the shared memory, the AI model inference conclusion post-processing module may combine the plurality of groups of image character recognition results into a final recognition result, and transmit the final recognition result to the wireless transmission module.
In this embodiment, the final recognition result may be directly output to an external device, or output to other components in the artificial intelligence system, such as, but not limited to, a wireless transmission module; when the artificial intelligence system comprises a wireless transmission module, a wireless transmission module driver can be further included in the processing chip to drive the wireless transmission module.
In other embodiments, if image segmentation is not performed and multiple image character recognition results are not obtained, the processing chip may not include the AI model inference conclusion post-processing module described above.
In some exemplary embodiments, the AI model for character recognition is a character recognition AI model obtained by training sample data in the cloud; wherein the sample data is a character data source.
In other embodiments, the AI model is not limited to be from the cloud, and may be, for example, trained or downloaded by other devices and then input to the AI system, or stored in a designated location for the AI system to read by itself.
The artificial intelligence system of the embodiment realizes the function of character recognition through the mutual matching of the processing chip and the programmable logic device, and can fully utilize the performances of the processing chip and the programmable logic device in the character recognition process by reasonably distributing the working contents of the processing chip and the programmable logic device, thereby improving the processing speed, being beneficial to reducing the complexity of the system and the performance requirements on related hardware devices, and providing more possibility for realizing the edge-end embedded AI system with low cost, low power consumption and low delay.
The embodiment of the application also provides a method for character recognition, which is applied to the artificial intelligence system for character recognition described in any of the above embodiments, and the method is shown in fig. 3, and includes steps S310-S340:
s310, analyzing an artificial intelligent AI model for character recognition by a processing chip, determining an operator to be operated and an operation sequence thereof, and storing the operator to a shared memory;
S320, the programmable logic device acquires the acquired original image containing the character to be recognized, processes the image into image data and stores the image data into the shared memory;
s330, the processing chip reads the image data and inputs the image data to the programmable logic device;
S340, the programmable logic device reads the operation operators corresponding to the operation sequence scheduling from the shared memory, performs operation on the image data to obtain an image character recognition result, and stores the image character recognition result in the shared memory.
In the above steps, S310 may be performed once initially, and then S320-S340 may be performed multiple times, that is, after performing analysis and determination for one AI model, character recognition may be performed on multiple original images, or multiple character recognition operations may be performed. Step S320 may be executed in parallel with other steps, for example, when the MCU parses the AI model, the FPGA may process one or more original images in parallel, store the obtained image data in the shared memory, after the model is parsed, determine the operator and the operation sequence to be operated, read the processed image data to perform character recognition, and may continue to process the original images into image data in parallel in the processes of recognizing and storing the image character recognition result. In addition, the character recognition results obtained can be outputted in parallel when the recognition is performed. Of course, a processing mode of all serial or partial serial may be adopted, for example, after the model analysis is completed, the operator to be operated and the operation sequence are determined, the original image is obtained and processed into image data, and the next original image is obtained after the image character recognition result is obtained according to the image data. In this embodiment, whether to use the parallel mode and which steps are specifically parallel may be determined by themselves according to requirements, device performance, and the like, which is not limited in this embodiment.
In some exemplary embodiments, the character to be recognized is a reading on a dashboard;
The method further comprises the steps of:
And the processing chip reads the image character recognition result from the shared memory, processes the image character recognition result into a final recognition result, and drives the wireless transmission module to send the final recognition result.
According to the method, the digital image of the instrument panel can be collected as the original image, the digital identification AI model is used for deducing and identifying the reading of the instrument, the reading is wirelessly transmitted to a remote place, and the remote monitoring of the non-intelligent instrument can be realized.
In this embodiment, a wireless transmission module may be built into the artificial intelligence system to wirelessly transmit the identified meter reading to the remote terminal; the wireless transmission module is not used as a part of the artificial intelligence system, but is connected with the artificial intelligence system in an external connection mode; the wireless transmission module driver can be built in the MCU core and used for controlling the sending and receiving of the wireless transmission module. The implementation form of wireless transmission may be set as needed, which is not limited in this embodiment.
Further implementation details can be found in the above-described embodiments.
The above embodiment is described below with an example of an application scenario in which dial numbers of a non-intelligent meter are monitored remotely.
The digital identification AI system in the edge embedded system is based on SoC realization of a lightweight MCU and a middle-low-end low-power consumption FPGA, can collect and identify a digital image of an instrument panel, and can be transmitted to a remote place in a wireless manner. The AI system comprises an image acquisition module, an image preprocessing module, a digital identification AI model parallel inference module, an AI model inference conclusion post-processing module and a wireless transmission module.
In this example, as shown in fig. 4, the flow of digital recognition by the AI system performs training of digital recognition AI models according to a digital data source at the cloud end to obtain AI models for digital recognition (hereinafter referred to as digital recognition AI models), and numbers 0 to 9 in the image can be inferred based on the trained AI models. An image acquisition module in the AI system acquires dial images of the instrument to be detected from an external camera as acquired original images, and an image preprocessing module processes the original images into image data. The digital identification AI model parallel deducing module obtains digital identification AI model deducing conclusion (namely image character identification result) according to the image data and the digital identification AI model. The AI model deducing conclusion post-processing module processes the obtained digital identification AI model deducing conclusion to obtain a complete N-bit number, namely, the number display (namely, the reading) of the non-intelligent instrument, and the complete N-bit number is used as a final identification result to be transmitted to the wireless transmission module, and the wireless transmission module transmits the complete N-bit number to the remote terminal. The working process of each module in the AI system is shown by solid arrows in FIG. 4, the flow paths of data input and output by each module are shown by dotted arrows in FIG. 4, and the original image acquired by the image acquisition module is stored in the shared memory for reading by the image preprocessing module; the image data processed by the image preprocessing module is stored in a shared memory for reading by a digital identification AI model parallel deducing module; the image character recognition result (called AI model inference conclusion in this example) obtained by the digital recognition AI model parallel inference module is saved to the shared memory, and is read by the AI model inference conclusion post-processing module to obtain a complete N-bit number and transmitted to the wireless transmission module.
In this example, the structure of the AI system is shown in fig. 5, and the SoC includes an MCU core, an FPGA core, and a shared memory, and is externally connected to a camera through an image acquisition module. The MCU core is connected with the shared memory through a system bus, and the FPGA core is connected with the shared memory through a parallel bus. The MCU kernel comprises an AI model interpreter, an AI model conclusion post-processing module and a wireless transmission module driver. The FPGA kernel comprises the image acquisition module, the image preprocessing module and N model operation accelerator kernels, and each model operation accelerator kernel comprises all operation operators in the artificial intelligent system. As in the dashed circle of fig. 5, the AI model interpreter and the N model operation accelerator cores are part of the AI model parallel inference module. Where N is the number of characters contained in the original image.
Five modules in this example are described below:
(1) Image acquisition module
The image acquisition module is externally connected with a camera and is used for acquiring the instrument panel number of the non-intelligent instrument.
The image acquisition module is positioned in the FPGA core and is realized by using FPGA logic resources, the camera acquires the input image data, and the input image data is input into the image acquisition module through the SoC port. The image acquisition module comprises:
the camera controller is used for controlling the image acquisition of the camera equipment and controlling the reading and writing time sequence and the cache of the acquired original image;
the first input/output buffer is used for buffering image data collected by the camera or the HDMI interface device.
The image data acquired by the image acquisition module is stored into a shared memory in the chip through a parallel bus. The shared memory can be directly accessed by the FPGA core and the MCU core at the same time, and data can be read and written in real time.
(2) Image preprocessing module
The original image acquired by the image acquisition module is based on the pixel resolution of the camera equipment, the general resolution is higher, and in the edge embedded system, the hardware resource limitation of the embedded system is limited, and the used AI model can identify and detect the image with generally lower resolution, so that the image preprocessing module is required to preprocess the acquired original image data to generate the image data suitable for the AI model.
In addition, the general meter is a multi-bit digital representation, such as 5-bit numbers (i.e. a sequence containing 5 numbers), while the number identification AI model identifies numbers 0 to 9 based on unit numbers, i.e. the AI model infers 1-bit numbers at a time, so that an image segmentation process is required to be performed, an image acquired by an image acquisition module is segmented, N-bit numbers are segmented into N groups of image data, each group of image data is subjected to AI model inference once, and one-bit number corresponding to the group of image data is identified.
The image preprocessing module is located in the FPGA kernel and is realized by using FPGA logic resources, as shown in fig. 6, and an image gray level algorithm module, an image segmentation algorithm module, an image enhancement algorithm module and a second input/output buffer are built in and are respectively used for image graying, clipping, transformation enhancement and input/output buffering.
And the image preprocessing module accesses the on-chip shared memory through the parallel bus, reads the original image saved to the shared memory by the image acquisition module, and caches the original image to the second input/output buffer.
And the image gray level algorithm module is internally provided with Scalar algorithm, executes gray level binarization on the original image and converts the original image into a gray level image.
The image segmentation algorithm module is used for segmenting N-bit numbers in the gray level images into N groups of images with preset sizes, each group of images comprises 1-bit numbers, and the segmented N groups of images are sequentially stored in N segments of address spaces of the fourth input/output buffer respectively.
And the image enhancement algorithm module is internally provided with an FFT (Fourier transform) algorithm, performs Fourier transform on the image with the preset size to obtain image data, and caches the image data in the fourth input/output buffer.
Image data generated by image preprocessing is cached in a fourth input/output buffer and stored in a shared memory through a parallel bus. The second input/output buffer and the fourth input/output buffer in this example may be the same input/output buffer.
(3) Digital identification AI model parallel inference module
The image preprocessing module preprocesses the generated image data as a data input for the AI model used by the digital recognition AI model parallel inference module.
As shown in fig. 7, the digital recognition AI model parallel inference module includes an AI model interpreter, N model operation accelerator cores, and an input-output buffer, wherein the N model operation accelerator cores implement AI model parallel inference of N sets of image data.
In the digital identification AI model parallel deducing module, an AI model interpreter of an MCU inner core part is connected with and accesses a shared memory through a system bus, the digital identification AI model is analyzed through an MCU software method, the data type of the AI model and all model operators are calculated, the model operators of the corresponding types needing to be operated and the operation sequence are counted, and the statistical analysis conclusion of the model operators is stored into the shared memory through the system bus. N model operation accelerator cores of the FPGA core part are connected through a parallel bus and access to a shared memory, each model operation accelerator core can use all operation operators in the artificial intelligent system, and each model operation accelerator core can operate a group of image data according to the model operation operators counted by the AI model interpreter and the operation sequence to identify a number corresponding to the group of image data.
The MCU reads N groups of image data generated by the image preprocessing module from the shared memory, sequentially stores the N groups of image data into N sections of addresses of the third input/output buffer according to the arrangement sequence of numbers corresponding to the image data in the meter panel reading, reads a model arithmetic operator and an operation sequence which are generated by the AI model interpreter and need to be operated, dynamically allocates the number of model arithmetic accelerator cores which need to be operated in parallel according to the number of the groups of the read image data (in the example, N model arithmetic accelerator cores are dynamically allocated), transmits the model arithmetic operator and the operation sequence which need to be operated to the N model arithmetic accelerator cores, sequentially reads the N groups of image data of the third input/output buffer according to the sequence from 1 to N, and inputs the N model arithmetic accelerator cores which need to be operated in parallel according to the sequence from 1 to N.
Each model operation accelerator core operates on the image data according to the model operation operator and the operation sequence which are generated by the AI model interpreter and need to be operated, and deduces and identifies the numbers of the group of image data. After each model operation accelerator core completes the calculation of an AI model operation operator, a digital identification AI model inference conclusion is generated, the N-segment addresses are sequentially stored in the fifth input/output buffer according to the sequence from 1 to N, and then the N-segment addresses are sequentially stored in the shared memory through a parallel bus. If N is 3, AI model inference conclusion (three numbers) corresponding to each of the three sets of image data 1, 2 and 3 are stored in the 1 st, 2 nd and 3 rd segment addresses of the fifth input/output buffer respectively.
The third input/output buffer and the fifth input/output buffer in this example may be the same input/output buffer.
(4) AI model conclusion post-processing module
And the AI model conclusion post-processing module is positioned in the MCU core and connected with the shared memory through a system bus, and is used for combining single digits inferred and identified by N model operation accelerator cores in the digital identification AI model parallel inference module into a complete N-bit digit according to the arrangement sequence in the fifth input/output buffer, namely the digit display of the monitored non-intelligent instrument.
The AI model deducing conclusion post-processing module sequentially reads N digital identification AI model deducing conclusion (for example, N is 3, the first digit, the second digit and the third digit are read from the 1 st, the 2 nd and the 3 rd segment addresses of the fifth input-output buffer) from the shared memory according to the sequence from 1 to N through a system bus, combines the N deduced digits into a complete N digit (for example, if N is 3, the first digit is shifted left by 2 digits, the second digit is shifted left by 1 digit, so as to obtain 3 digits), and transmits the complete N digit to the wireless transmission module.
The positions and sizes of the N-segment addresses in the fourth and fifth input-output buffers may be set in advance.
(5) Wireless transmission module
The digital identification AI system of the example is internally provided with a wireless transmission module which is used for transmitting the instrument number generated by the AI model conclusion post-processing module to a remote terminal through wireless, so as to realize remote monitoring of the non-intelligent instrument. And the wireless transmission module driver is positioned in the MCU core and used for controlling the sending and receiving of the wireless transmission module.
In this example, for low power consumption and miniature, the digital identification AI models used are AI models in TinyML fields, and may include 29 layers of operators such as Conv2D, depthwiseConv2D, fullyConnected, averagePooling D, reshape, softMax, an image input data layer and an inference conclusion output data layer implemented by the MCU. At the cloud, through machine learning, the model learns a large number of digital data sources, trains an AI model that can be accurately used for the above-described digital recognition, and the AI system of this example infers the numbers 0 to 9 in the original image based on the trained AI model.
In other embodiments, other AI models for digital identification may be employed, as the application is not limited in this regard.
Those of ordinary skill in the art will appreciate that all or some of the steps, systems, functional modules/units in the apparatus, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between the functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed cooperatively by several physical components. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
Claims (10)
1. An artificial intelligence system for character recognition, comprising: a shared memory;
The processing chip is used for analyzing the artificial intelligent AI model for character recognition, determining an operator to be operated and the operation sequence thereof, and storing the operator and the operation sequence into the shared memory;
the programmable logic device is used for acquiring the acquired original image containing the character to be recognized, processing the image into image data, storing the image data into the shared memory, reading the operation sequence from the shared memory, scheduling a corresponding operator to operate the image data, obtaining an image character recognition result and storing the image character recognition result into the shared memory;
The processing chip is also used for reading the image data from the shared memory and inputting the image data into the programmable logic device; wherein,
The programmable logic device processes the original image into the image data, comprising: dividing the original image into a plurality of groups of image data, wherein each group of image data corresponds to an image of a character;
The step of reading the operation sequence from the shared memory, and scheduling a corresponding operator to operate on the image data to obtain an image character recognition result, comprises the following steps: a plurality of scheduled model operation accelerator cores perform operation in parallel; each scheduled model operation accelerator core respectively carries out operation on one group of image data to obtain the image character recognition result corresponding to the group of image data;
The processing chip is further used for scheduling the corresponding number of model operation accelerator cores in the programmable logic device according to the number of the groups of the image data, reading a plurality of groups of the image data from the shared memory, inputting the image data into the model operation accelerator cores in a one-to-one correspondence mode, and combining the image character recognition results corresponding to the plurality of groups of the image data into a final recognition result.
2. The artificial intelligence system of claim 1, wherein:
the character to be identified is a reading on an instrument panel;
the artificial intelligence system further comprises:
The wireless transmission module is used for sending the final identification result;
the processing chip is also used for driving the wireless transmission module and sending the final identification result to the wireless transmission module.
3. The artificial intelligence system of claim 1 or 2, wherein:
The processing chip is a micro control unit MCU, and the programmable logic device is a field programmable gate array FPGA; the artificial intelligence system is a system on a chip; the MCU is connected with the shared memory through a system bus, and the FPGA is connected with the shared memory through a parallel bus;
The operator is realized by an FPGA kernel.
4. The artificial intelligence system of claim 3, wherein:
The operators comprise one or more of the following types of operators: full join operator, matrix transform operator, flexible maximum transmission operator, normal convolution operator, deep convolution operator, max pooling operator, average pooling operator.
5. The artificial intelligence system of claim 1, wherein the programmable logic device comprises:
The camera controller is used for controlling the camera to collect images and controlling the reading and writing time sequence and the cache of the collected original images;
The first input/output buffer is used for buffering the original image acquired by the camera and sending the original image to the shared memory through the parallel bus.
6. The artificial intelligence system of claim 5, wherein the programmable logic device further comprises:
a second input-output buffer for holding the original image read from the shared memory;
The image gray level algorithm module is used for carrying out gray level binarization processing on the original image and converting the original image into a gray level image;
The image segmentation algorithm module is used for segmenting the gray image into images with preset sizes and buffering the images into the second input/output buffer;
The image enhancement algorithm module is used for carrying out Fourier transform on the image with the preset size to obtain the image data, and caching the image data into the second input/output buffer;
The second input-output buffer is further configured to store the image data to the shared memory via a parallel bus.
7. The artificial intelligence system according to claim 6, wherein:
The processing chip comprises:
The AI model interpreter is used for analyzing the AI model for character recognition, calculating the data type and all operators of the AI model, determining the operators to be operated and the operation sequence thereof according to the calculation result, and storing the operators and the operation sequence thereof into the shared memory;
and the AI model inference conclusion post-processing module is used for combining a plurality of groups of image character recognition results into the final recognition result and outputting the final recognition result.
8. The artificial intelligence system of any one of claims 1-2, 4-7, wherein:
The AI model for character recognition is a character recognition AI model obtained through sample data training at the cloud; wherein the sample data is a character data source.
9. A method for character recognition, applied to the artificial intelligence system for character recognition according to any one of claims 1 to 8, the method comprising:
The processing chip analyzes an artificial intelligence AI model for character recognition, determines an operator to be operated and an operation sequence thereof, and stores the operator and the operation sequence in a shared memory;
the programmable logic device acquires the acquired original image containing the character to be recognized, processes the original image into image data and stores the image data into the shared memory;
the processing chip reads the image data and inputs the image data to the programmable logic device;
the programmable logic device reads the operation sequence scheduling corresponding operation operator from the shared memory, performs operation on the image data to obtain an image character recognition result, and stores the image character recognition result in the shared memory; wherein,
The programmable logic device processes the original image into the image data, comprising: dividing the original image into a plurality of groups of image data, wherein each group of image data corresponds to an image of a character;
The step of reading the operation sequence from the shared memory, and scheduling a corresponding operator to operate on the image data to obtain an image character recognition result, comprises the following steps: a plurality of scheduled model operation accelerator cores perform operation in parallel; each scheduled model operation accelerator core respectively carries out operation on one group of image data to obtain the image character recognition result corresponding to the group of image data;
The processing chip also schedules the corresponding number of model operation accelerator cores in the programmable logic device according to the number of the groups of the image data, reads a plurality of groups of the image data from the shared memory, inputs the image data into the model operation accelerator cores in a one-to-one correspondence manner, and combines the image character recognition results corresponding to the image data into a final recognition result.
10. The method of claim 9, wherein the character to be recognized is a reading on a dashboard; the method further comprises the steps of:
And the processing chip reads the image character recognition result from the shared memory, processes the image character recognition result into a final recognition result, and drives the wireless transmission module to send the final recognition result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110355563.2A CN115147840B (en) | 2021-03-31 | 2021-03-31 | Artificial intelligence system and method for character recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110355563.2A CN115147840B (en) | 2021-03-31 | 2021-03-31 | Artificial intelligence system and method for character recognition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115147840A CN115147840A (en) | 2022-10-04 |
| CN115147840B true CN115147840B (en) | 2024-05-17 |
Family
ID=83405535
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110355563.2A Active CN115147840B (en) | 2021-03-31 | 2021-03-31 | Artificial intelligence system and method for character recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115147840B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102044063A (en) * | 2010-12-23 | 2011-05-04 | 中国科学院自动化研究所 | FPGA (Field Programmable Gate Array) and DSP (Digital Signal Processor) based machine vision system |
| CN109766293A (en) * | 2019-02-01 | 2019-05-17 | 京微齐力(北京)科技有限公司 | Connect the circuit and System on Chip/SoC of FPGA and artificial intelligence module on chip |
| CN109932953A (en) * | 2017-12-19 | 2019-06-25 | 陈新 | Intelligent supercomputer programmable controller |
| CN110727633A (en) * | 2019-09-17 | 2020-01-24 | 广东高云半导体科技股份有限公司 | Edge artificial intelligence computing system framework based on SoC FPGA |
| CN110991522A (en) * | 2019-11-29 | 2020-04-10 | 山东万腾电子科技有限公司 | Character recognition method and system and industrial intelligent gateway |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10802992B2 (en) * | 2016-08-12 | 2020-10-13 | Xilinx Technology Beijing Limited | Combining CPU and special accelerator for implementing an artificial neural network |
-
2021
- 2021-03-31 CN CN202110355563.2A patent/CN115147840B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102044063A (en) * | 2010-12-23 | 2011-05-04 | 中国科学院自动化研究所 | FPGA (Field Programmable Gate Array) and DSP (Digital Signal Processor) based machine vision system |
| CN109932953A (en) * | 2017-12-19 | 2019-06-25 | 陈新 | Intelligent supercomputer programmable controller |
| CN109766293A (en) * | 2019-02-01 | 2019-05-17 | 京微齐力(北京)科技有限公司 | Connect the circuit and System on Chip/SoC of FPGA and artificial intelligence module on chip |
| CN110727633A (en) * | 2019-09-17 | 2020-01-24 | 广东高云半导体科技股份有限公司 | Edge artificial intelligence computing system framework based on SoC FPGA |
| CN110991522A (en) * | 2019-11-29 | 2020-04-10 | 山东万腾电子科技有限公司 | Character recognition method and system and industrial intelligent gateway |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115147840A (en) | 2022-10-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111797976B (en) | Training method, image recognition method, device, equipment and medium for neural network | |
| CN106960219B (en) | Picture identification method and device, computer equipment and computer readable medium | |
| CN110647920A (en) | Transfer learning method and device in machine learning, equipment and readable medium | |
| CN110956202B (en) | Image training method, system, medium and intelligent device based on distributed learning | |
| US20200250545A1 (en) | Split network acceleration architecture | |
| EP3886000A1 (en) | Methods, systems, articles of manufacture, and apparatus to decode zero-value-compression data vectors | |
| EP3564863B1 (en) | Apparatus for executing lstm neural network operation, and operational method | |
| CN114723033B (en) | Data processing method, data processing device, AI chip, electronic device and storage medium | |
| CN110738235B (en) | Pulmonary tuberculosis judging method, device, computer equipment and storage medium | |
| CN111832666B (en) | Medical image data amplification method, device, medium, and electronic apparatus | |
| CN111708511A (en) | Data compression for neural networks | |
| US11544542B2 (en) | Computing device and method | |
| CN105700956A (en) | Distributed job processing method and system | |
| CN114781635B (en) | Model deployment method, device, equipment and medium | |
| US20210064955A1 (en) | Methods, apparatuses, and computer program products using a repeated convolution-based attention module for improved neural network implementations | |
| WO2021198809A1 (en) | Feature reordering based on sparsity for improved memory compression transfers during machine learning jobs | |
| CN112396072B (en) | Image classification acceleration method and device based on ASIC (application specific integrated circuit) and VGG16 | |
| CN115147840B (en) | Artificial intelligence system and method for character recognition | |
| CN108520532B (en) | Method and device for identifying motion direction of object in video | |
| EP3716150A1 (en) | Improved image segmentation using a neural network translation model | |
| CN109829004B (en) | Data processing method, device and equipment based on unmanned vehicle and storage medium | |
| KR20210038027A (en) | Method for Training to Compress Neural Network and Method for Using Compressed Neural Network | |
| CN113222121B (en) | Data processing method, device and equipment | |
| CN114897133A (en) | Universal configurable Transformer hardware accelerator and implementation method thereof | |
| CN113139617B (en) | Power transmission line autonomous positioning method and device and terminal equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |