CN115147840A - Artificial intelligence system and method for character recognition - Google Patents
Artificial intelligence system and method for character recognition Download PDFInfo
- Publication number
- CN115147840A CN115147840A CN202110355563.2A CN202110355563A CN115147840A CN 115147840 A CN115147840 A CN 115147840A CN 202110355563 A CN202110355563 A CN 202110355563A CN 115147840 A CN115147840 A CN 115147840A
- Authority
- CN
- China
- Prior art keywords
- image
- image data
- shared memory
- character recognition
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000013473 artificial intelligence Methods 0.000 title claims abstract description 170
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000012545 processing Methods 0.000 claims abstract description 50
- 239000000872 buffer Substances 0.000 claims description 37
- 230000005540 biological transmission Effects 0.000 claims description 32
- 230000008569 process Effects 0.000 claims description 18
- 238000004422 calculation algorithm Methods 0.000 claims description 17
- 238000012805 post-processing Methods 0.000 claims description 12
- 238000003709 image segmentation Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000011176 pooling Methods 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 2
- 238000012549 training Methods 0.000 claims description 2
- 230000009466 transformation Effects 0.000 claims 1
- 238000007781 pre-processing Methods 0.000 description 13
- 230000006870 function Effects 0.000 description 8
- 238000012544 monitoring process Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000003139 buffering effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012821 model calculation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
Images
Landscapes
- Character Discrimination (AREA)
Abstract
An artificial intelligence system and method for character recognition, the artificial intelligence system comprising: a shared memory; the processing chip is used for analyzing an artificial intelligence AI model for character recognition, determining an operator to be operated and an operation sequence thereof and storing the operator to the shared memory; the programmable logic device is used for acquiring an acquired original image containing characters to be recognized, processing the acquired original image into image data and storing the image data into the shared memory, reading an operation sequence from the shared memory, scheduling corresponding arithmetic operators to carry out operation on the image data to obtain an image character recognition result and storing the image character recognition result into the shared memory; the processing chip is also used for reading image data from the shared memory and inputting the image data into the programmable logic device. The embodiment of the application has the advantages of low power consumption, low time delay, low cost, easy expansion and the like, and is suitable for being used in an edge end embedded system.
Description
Technical Field
The embodiment of the application relates to but not limited to the field of artificial intelligence, in particular to an artificial intelligence system and method for character recognition.
Background
With the development and wide application of AI (Artificial Intelligence) technology, AI computation under different scenes poses more and more challenges. The application of AI computing is gradually expanded from the beginning cloud to the edge-end embedded system.
In some technologies, when the AI method is applied to identify characters in the edge-end embedded system, the power consumption and cost of the whole system are high, and the highly complex system is not favorable for arrangement, so that the cost performance of the AI method applied to identify characters in the edge-end embedded system is not high.
Disclosure of Invention
The following is a summary of the subject matter described in detail herein and is not intended to limit the scope of the appended claims.
The embodiment of the application provides an artificial intelligence system for character recognition, which comprises:
a shared memory;
the processing chip is used for analyzing an artificial intelligence AI model for character recognition, determining an operator to be operated and an operation sequence thereof and storing the operator to the shared memory;
the programmable logic device is used for acquiring an acquired original image containing characters to be recognized, processing the acquired original image into image data and storing the image data into the shared memory, reading the running sequence from the shared memory, scheduling corresponding arithmetic operators to carry out operation on the image data to obtain an image character recognition result and storing the image character recognition result into the shared memory;
the processing chip is also used for reading the image data from the shared memory and inputting the image data into the programmable logic device.
The embodiment of the present application further provides a method for performing character recognition, which is applied to the above artificial intelligence system for performing character recognition, and the method includes:
the processing chip analyzes an artificial intelligence AI model for character recognition, determines an operator to be operated and an operation sequence thereof and stores the operator to a shared memory;
the programmable logic device acquires an acquired original image containing characters to be recognized, processes the acquired original image into image data and stores the image data into the shared memory;
the processing chip reads the image data and inputs the image data to the programmable logic device;
and the programmable logic device reads the corresponding arithmetic operator of the operation sequence scheduling from the shared memory, performs operation on the image data to obtain an image character recognition result, and stores the image character recognition result in the shared memory.
The AI system of the embodiment of the application can realize character recognition by adopting light-weight and low-power-consumption devices, reduces the complexity of AI model inference and character recognition, solves the problems of high complexity, high power consumption and high cost when the AI method is applied to recognizing characters in an edge-end embedded system, can load the AI model for character recognition by using a low-power-consumption light-weight system at a middle-low end, can reduce the power consumption of the whole system, reduce the cost and improve the cost performance and efficiency of the edge-end embedded system for recognizing characters by applying the AI method.
Other aspects will be apparent upon reading and understanding the attached drawings and detailed description.
Drawings
The accompanying drawings are included to provide an understanding of the present disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the examples serve to explain the principles of the disclosure and not to limit the disclosure.
FIG. 1 is a schematic diagram of an artificial intelligence system for performing character recognition in accordance with an embodiment of the present application;
FIG. 2 is a diagram illustrating an AI system implemented using a system-on-chip in one embodiment;
FIG. 3 is a flowchart illustrating a method for character recognition according to an embodiment of the present application;
FIG. 4 is a flow chart illustrating number identification in an example;
FIG. 5 is a schematic structural diagram of an AI system for numeric identification in an example;
FIG. 6 is a schematic diagram of an exemplary image pre-processing module;
fig. 7 is a schematic diagram of an example digital recognition AI model parallel inference module.
Detailed Description
The description herein describes embodiments, but is intended to be exemplary, rather than limiting and it will be apparent to those of ordinary skill in the art that many more embodiments and implementations are possible that are within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are possible. Any feature or element of any embodiment may be used in combination with or instead of any other feature or element in any other embodiment, unless expressly limited otherwise.
The present application includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The embodiments, features and elements disclosed herein may also be combined with any conventional features or elements to form a unique inventive aspect as defined by the appended claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive aspects to form yet another unique inventive aspect, as defined by the appended claims. Thus, it should be understood that any of the features shown and/or discussed in this application may be implemented individually or in any suitable combination. Accordingly, the embodiments are not limited except as by the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.
Further, in describing representative embodiments, the specification may have presented the method and/or process as a particular sequence of steps. However, to the extent that the method or process does not rely on the particular order of steps set forth herein, the method or process should not be limited to the particular sequence of steps described. Other orders of steps are possible as will be understood by those of ordinary skill in the art. Accordingly, the particular order of the steps set forth in the specification should not be construed as limitations on the claims appended hereto. Furthermore, the claims directed to the method and/or process should not be limited to the performance of their steps in the order written, and one skilled in the art can readily appreciate that the sequences may be varied and still remain within the spirit and scope of the embodiments of the present application.
An embodiment of the present application provides an artificial intelligence system for performing character recognition, as shown in fig. 1, including:
a shared memory 10;
the processing chip 11 is used for analyzing an artificial intelligence AI model for character recognition, determining an operator to be operated and an operation sequence thereof and storing the operator to the shared memory 10;
the programmable logic device 12 is used for acquiring an acquired original image containing characters to be recognized, processing the acquired original image into image data, storing the image data into the shared memory 10, reading an operation sequence from the shared memory 10, scheduling corresponding arithmetic operators to perform operation on the image data, obtaining an image character recognition result and storing the image character recognition result into the shared memory;
the processing chip 11 is also used to read image data from the shared memory 10 into the programmable logic device 12.
The AI system of the embodiment can realize character recognition by adopting light-weight and low-power-consumption devices, reduces the complexity of AI model inference and character recognition, solves the problems of high complexity, high power consumption and high cost when an AI method is applied to recognizing characters in an edge-end embedded system, can load the AI model for character recognition by using a low-power-consumption light-weight system at a middle-low end, reduces the power consumption of the whole system, reduces the cost, and improves the cost performance and the efficiency of the edge-end embedded system for recognizing characters by applying the AI method.
In this embodiment, the AI model for character recognition may be any AI model that can recognize and detect a character from an original image, and may be obtained by a machine learning method. The character recognition result can be regarded as an output result of an AI model (or referred to as an inference result of the AI model) for character recognition when image data is input to the AI model.
In this embodiment, the original image may also be output together when the character recognition result is output.
In this embodiment, each operator may be considered as a layer in the AI model or as a step in an operation performed by the AI model. The operators can be stored in a form of function or task, and the like, but not limited to calling the corresponding operators by function name or task name, and the like when the operators are scheduled according to the running sequence.
In this embodiment, a plurality of operators may be loaded in the programmable logic device in advance, where loading the operators may refer to, but is not limited to, configuring the programmable logic device according to a hardware description language for implementing the operators, or implementing the plurality of operators in the programmable logic device through other hardware implementation manners. The arithmetic operators loaded in the programmable logic device comprise all the arithmetic operators in the AI model to be operated.
The AI model for performing character recognition in this embodiment may be an AI model used in any character recognition field, which is not limited in this application. The AI model may be a pre-trained model; the AI model may be updated during the identification process.
In this embodiment, various forms of shared memory may be employed, or the shared memory may be included in a programmable logic device.
In some exemplary embodiments, the programmable logic device processing the raw image into the image data comprises: dividing the original image into a plurality of groups of image data, wherein each group of image data corresponds to an image of a character;
the scheduling of the corresponding arithmetic operator according to the running sequence to perform operation on the image data to obtain an image character recognition result includes: a plurality of scheduled model operation accelerator cores perform operation in parallel; each scheduled model operation accelerator core respectively operates a group of image data to obtain an image character recognition result corresponding to the group of image data;
the processing chip is further configured to schedule a corresponding number of model operation accelerator cores in the programmable logic device according to the number of groups of the image data, read the plurality of groups of image data from the shared memory, input the image data to the model operation accelerator cores in a one-to-one correspondence manner, and combine the image character recognition results corresponding to the plurality of groups of image data into a final recognition result.
In this embodiment, the original image is divided into image data of the corresponding number of groups according to the number of characters included, thereby facilitating image character recognition. For some AI models, such as a digital identification AI model, the numbers 0 to 9 are identified based on unit numbers, that is, the AI model infers 1 digit at a time, so that an image segmentation process is required.
In other embodiments, any number of characters may be divided into one group to segment the original image, or the character recognition may be directly performed without segmenting the original image, which may be set according to the AI model used, which is not limited in this application.
In this embodiment, all the existing operation operators of the programmable logic device in which each model operation accelerator core is located may be used, so that no matter which operation operators to be operated are determined by the processing chip, each model operation accelerator core may be ensured to have the capability of independently completing operations, and all the required hardware implementations of the operation operations are included in the accelerator core. Each operator can be distributed in different addresses and time sequences, and when a certain operator is used by the model operation accelerator core, scheduling can be carried out according to the addresses and the time sequences of the operators.
In this embodiment, after the original image is divided into a plurality of groups of image data, each group of image data is respectively input to the one-to-one corresponding model operation accelerator cores for parallel operation, and this parallel operation processing manner enables AI model inference conclusions for all characters to be generated within a single character recognition time, thereby playing a role in accelerating operation.
In this embodiment, determining the model operation accelerator core to be operated, and combining the image character recognition results corresponding to the multiple sets of image data into the final recognition result are completed by the processing chip, and when the processing chip inputs N sets of image data into N model operation accelerator cores, the N sets of image data may be input in sequence, for example, the image data of a first character is input into a first model operation accelerator core, the image data of a second character is input into a second model operation accelerator core, and so on, then the image character recognition results of each image data are respectively obtained according to the input sequence, and the final recognition result obtained after sequentially combining the image character recognition results in the original image is the character to be recognized.
In other embodiments, a preset algorithm may also be used to allocate image data to the model operation accelerator core, a corresponding address for storing an image character recognition result may be pre-allocated to each image data or each model operation accelerator core that needs to be operated, a sequence of characters may also be ensured during combination to restore a sequence of characters to be recognized in an original image, and other manners may also be used to achieve the purpose, which is not limited in this application.
In other embodiments, multiple characters in the sequence of characters to be recognized may be recognized in a serial manner, or multiple characters may be recognized at one time, so that only one model operation accelerator core may be called.
In some exemplary embodiments, the character to be recognized is a reading on a dashboard;
the artificial intelligence system further comprises:
the wireless transmission module is used for sending the final identification result;
the processing chip is also used for driving the wireless transmission module and sending the final identification result to the wireless transmission module.
In the instrument market application, most of the intelligent instruments, such as an intelligent electric meter, an intelligent instrument, an intelligent gas meter and the like, are used, a built-in processor unit (CPU) and wired and wireless transmission modules are used, and functions of remote meter reading, remote control, monitoring and the like can be realized. However, the early old-fashioned non-intelligent instrument and meter does not have the function of remote meter reading and needs manual meter reading. If the instrument is in severe environments such as underground mines and the like, the difficulty and danger coefficient of manual meter reading are large, and the condition monitoring is not facilitated. If all the intelligent meters are replaced, not only can resources be wasted, but also great expenses can be brought. The artificial intelligence system of the embodiment can jointly complete the function of monitoring the instrument by using the AI model through the mutual matching of the processing chip and the programmable logic device, thereby forming an AI-friendly non-intelligent instrument monitoring system based on an edge-end embedded system, so that the respective advantages of the processing chip and the programmable logic device can be fully utilized, the reading identification and monitoring of the old non-intelligent instrument can be realized only by less logic resources and limited data computing capacity, the identified reading can be wirelessly transmitted to a remote terminal, the manpower resources are saved, the artificial intelligence system can be applied to the front end of the non-intelligent instrument to realize the digital identification and the remote monitoring of the non-intelligent instrument by using the AI only by being additionally arranged on the front end of the non-intelligent instrument, the AI application field is expanded, and the artificial intelligence system has the advantages of low power consumption, low time delay, low cost, easy expansion and the like.
In some exemplary embodiments, the processing chip may be, but is not limited to, an MCU (Micro Control Unit), and the Programmable logic device may be, but is not limited to, an FPGA (Field Programmable Gate Array); the MCU is connected with the shared memory through a system bus, and the FPGA is connected with the shared memory through a parallel bus; the operation operator is realized by an FPGA kernel.
In this embodiment, the AI system for performing character recognition may be a system on a chip.
In this embodiment, the operations performed by the processing chip and the programmable logic device described above may be, but are not limited to be, completed in the MCU core and the FPGA core, respectively.
The embodiment can realize the AI system through one MCU and the FPGA on-chip system; the MCU and FPGA on-chip system is formed by connecting an FPGA, an MCU, a memory, external equipment and the like with an FPGA core. Based on the programmable characteristic of the FPGA, the framework has good expansibility.
In an implementation manner of this embodiment, the AI System uses a lightweight MCU and a SoC (System On Chip) of a low power consumption FPGA (System On Chip) as shown in fig. 2 as carriers, and the MCU and the FPGA perform data interaction through a shared memory to realize character recognition. The system on chip can be externally connected with a camera to obtain an acquired original image, can directly output an image character recognition result or a further processing result by the MCU, and can also be internally or externally connected with a wireless transmission module to send the image character recognition result or the further processing result.
In an implementation manner of this embodiment, the operators include one or more types of operators: a Fully Connected operation (full Connected) operator, a matrix transform operation (Reshape) operator, a flexible maximum transport operation (SoftMax) operator, a common convolution operation (Conv 2D) operator, a deep convolution operation (DepthwiseConv 2D) operator, a max pooling operation (MaxPool 2D) operator, an average pooling operation (AveragePool 2D) operator.
The AI model including the above-mentioned non-listed operators may be selected as needed, which is not limited in the present application.
In this embodiment, the AI model may be regarded as including, in addition to the operator, an input layer and an output layer, which are respectively used to input image data (directly or through an input/output buffer) to the corresponding operator in the AI model, and output an image character recognition result to the shared memory directly or through the input/output buffer; the input layer may be implemented by an MCU core and the output layer may be implemented by an FPGA.
In some exemplary embodiments, the programmable logic device comprises:
the camera controller is used for controlling the camera to collect images and controlling the reading and writing time sequence and the cache of the collected original images;
and the first input and output buffer is used for caching the original image collected by the camera and sending the original image to the shared memory through the parallel bus.
In this embodiment, a camera may be arranged in front of the artificial intelligence system, the programmable logic device controls the camera to perform image acquisition through the camera controller, and the acquired original image is cached in the first input/output buffer and then sent to the shared memory through the parallel bus.
In other embodiments, the original image may also be transmitted to the artificial intelligence system by using an HDMI device after the image is acquired, and the form of acquiring the original image is not limited in the present application.
In an implementation manner of this embodiment, the programmable logic device may further include:
a second input/output buffer for holding the original image read from the shared memory;
the image gray level algorithm module is used for carrying out gray level binarization processing on the original image and converting the original image into a gray level image;
the image segmentation algorithm module is used for segmenting the gray level image into images with preset sizes and caching the images into the second input and output buffer;
the image enhancement algorithm module is used for carrying out Fourier transform on the image with the preset size to obtain the image data and caching the image data to the second input and output buffer;
the second input output buffer is also used to store the image data to the shared memory through a parallel bus.
After the image is segmented by the image segmentation algorithm module, the images with the preset size obtained after segmentation can be sequentially stored in the N sections of address spaces of the second input and output buffer according to the arrangement sequence before segmentation; the image enhancement algorithm module can sequentially perform Fourier transform on the segmented images and sequentially store the images back to the N sections of address spaces of the second input and output buffer. The input/output buffer may also be separately configured to sequentially store the segmented or image-enhanced images, which is not limited in this embodiment of the present application.
In this embodiment, the camera controller and each module may be configured in an FPGA core through a hardware description language, but not limited to this.
In an implementation manner of this embodiment, the processing chip may include:
the AI model interpreter is used for analyzing the AI model for character recognition, calculating the data type and all arithmetic operators of the AI model, determining the arithmetic operators to be operated and the operation sequence thereof according to the calculation result, and storing the arithmetic operators and the operation sequence in the shared memory;
and the AI model inference conclusion post-processing module is used for combining the multiple groups of image character recognition results into the final recognition result and outputting the final recognition result.
In this embodiment, the AI model interpreter may be implemented, but is not limited to being, by software code stored in the MCU core.
In this embodiment, the MCU may read image data from the shared memory through the third input/output buffer, and input the image data to the corresponding model operation accelerator cores, and after the model operation accelerator cores complete operations and store the image character recognition results in the shared memory, the AI model conclusion inference post-processing module may combine the multiple sets of image character recognition results into a final recognition result, and transmit the final recognition result to the wireless transmission module.
In this embodiment, the final recognition result may be directly output to an external device, or output to other components in the artificial intelligence system, such as but not limited to a wireless transmission module; when the artificial intelligence system comprises a wireless transmission module, the processing chip may further comprise a wireless transmission module driver to drive the wireless transmission module.
In other embodiments, if the image segmentation is not performed and the multiple image character recognition results are not obtained, the processing chip may not include the AI model inference conclusion post-processing module.
In some exemplary embodiments, the AI model for character recognition is a character recognition AI model obtained by training sample data in the cloud; wherein, the sample data is a character data source.
In other embodiments, the AI model is not limited to be from the cloud, and may be input to the AI system after being trained or downloaded by other devices, or stored in a designated location for the AI system to read by itself.
The artificial intelligence system of the embodiment realizes the function of character recognition by the mutual matching of the processing chip and the programmable logic device, and makes full use of the performance of the processing chip and the programmable logic device by reasonably distributing the working contents of the processing chip and the programmable logic device in the character recognition process, thereby improving the processing speed, being beneficial to reducing the complexity of the system, reducing the performance requirements on related hardware devices, and providing more possibilities for realizing the edge-end embedded AI system with low cost, low power consumption and low time delay.
An embodiment of the present application further provides a method for performing character recognition, which is applied to the artificial intelligence system for performing character recognition according to any of the above embodiments, and the method is shown in fig. 3, and includes steps S310 to S340:
s310, the processing chip analyzes an artificial intelligence AI model for character recognition, determines an operator to be operated and an operation sequence thereof and stores the operator to a shared memory;
s320, the programmable logic device acquires the acquired original image containing the character to be recognized, processes the original image into image data and stores the image data into the shared memory;
s330, the processing chip reads the image data and inputs the image data to the programmable logic device;
and S340, the programmable logic device reads the corresponding arithmetic operator of the operation sequence scheduling from the shared memory, and performs operation on the image data to obtain an image character recognition result, and stores the image character recognition result in the shared memory.
In the above steps, S310 may be executed once initially, and then S320-S340 may be executed multiple times, that is, after one parsing and determining is performed on one AI model, character recognition may be performed on multiple original images, or multiple character recognition operations may be performed. Step S320 may be executed in parallel with other steps, for example, when the MCU parses the AI model, the FPGA may process one or more original images in parallel, store the obtained image data in the shared memory, after the model is parsed, determine the operation operator and the operation sequence to be operated, read the processed image data for character recognition, and continue to process the original images into image data in parallel in the processes of recognizing and storing the image character recognition results. In addition, the obtained character recognition results can be output in parallel when performing recognition. Of course, a full serial or partial serial processing mode may be adopted, for example, after the model analysis is completed, the operation operator to be operated and the operation sequence are determined, the original image is obtained and processed into image data, and the next original image is obtained after the image character recognition result is obtained according to the image data. In this embodiment, whether a parallel manner is adopted and which steps are specifically parallel may be determined by self according to requirements, device performance, and the like, which is not limited in this embodiment.
In some exemplary embodiments, the character to be recognized is a reading on a dashboard;
the method further comprises the following steps:
and the processing chip reads the image character recognition result from the shared memory, processes the image character recognition result into a final recognition result, and drives the wireless transmission module to send the final recognition result.
According to the method, the digital image of the instrument panel can be collected as the original image, the digital identification AI model is used for deducing and identifying the reading of the instrument, the reading is transmitted to a remote place in a wireless mode, and remote monitoring of the non-intelligent instrument can be achieved.
In this embodiment, a wireless transmission module may be built in the artificial intelligence system so as to wirelessly transmit the recognized meter reading to the remote terminal; the wireless transmission module is not used as a part of the artificial intelligence system, but is connected with the artificial intelligence system in an external connection mode; the wireless transmission module driver can be internally arranged in the MCU kernel and used for controlling the sending and receiving of the wireless transmission module. The implementation form of wireless transmission may be set as needed, and this embodiment does not limit this.
Further implementation details can be found in the above-described embodiments.
The above embodiment is described below by using an example, and the application scenario of the example is to remotely monitor the dial numbers of a non-smart meter.
The AI system is realized on the basis of a light MCU and a SoC of a middle-low-power-consumption FPGA, can acquire and identify digital images of an instrument panel and can transmit the digital images to a remote place in a wireless manner. The AI system comprises an image acquisition module, an image preprocessing module, a digital identification AI model parallel inference module, an AI model inference conclusion post-processing module and a wireless transmission module.
In this example, as shown in fig. 4, the AI system performs digital recognition, and trains a digital recognition AI model according to a digital data source at the cloud end to obtain an AI model for digital recognition (hereinafter, referred to as digital recognition AI model), and numbers 0 to 9 in an image can be inferred based on the trained AI model. An image acquisition module in the AI system acquires a dial plate image of the instrument to be detected from an external camera as an acquired original image, and an image preprocessing module processes the original image into image data. And the digital recognition AI model parallel inference module obtains the inference conclusion (namely the image character recognition result) of the digital recognition AI model according to the image data and the digital recognition AI model. The AI model inference conclusion post-processing module processes the obtained digital identification AI model inference conclusion to obtain a complete N-digit number, namely the digital display (reading number) of the non-intelligent instrument, and the complete N-digit number is used as a final identification result and is transmitted to the wireless transmission module and is transmitted to the remote terminal by the wireless transmission module. The working process of each module in the AI system is shown by a solid arrow in fig. 4, the flow path of data input and output by each module is shown by a dotted arrow in fig. 4, and the original image acquired by the image acquisition module is stored in the shared memory for reading and use by the image preprocessing module; the image data obtained by the image preprocessing module is stored in a shared memory for reading and using by a digital identification AI model parallel inference module; the image character recognition result (referred to as AI model inference result in this example) obtained by the digital recognition AI model parallel inference module is stored in the shared memory, and is read by the AI model inference result post-processing module to obtain a complete N-digit number, and the complete N-digit number is transmitted to the wireless transmission module.
In this example, the AI system has a structure as shown in fig. 5, where the SoC includes an MCU core, an FPGA core, and a shared memory, and is externally connected to the camera through an image acquisition module. The MCU kernel is connected with the shared memory through a system bus, and the FPGA kernel is connected with the shared memory through a parallel bus. The MCU kernel comprises an AI model interpreter, an AI model conclusion inference post-processing module and a wireless transmission module driver. The FPGA kernel comprises the image acquisition module, the image preprocessing module and N model operation accelerator cores, and each model operation accelerator core comprises all operation operators in the artificial intelligence system. As in the dotted circle of fig. 5, the AI model interpreter and the N model computation accelerator cores are components of the AI model parallel inference module. Where N is the number of characters contained in the original image.
The following describes the five modules in this example, respectively:
(1) Image acquisition module
The image acquisition module is externally connected with a camera and is used for acquiring the instrument panel number of the non-intelligent instrument.
The image acquisition module is positioned in the FPGA kernel and is realized by using FPGA logic resources, and the camera acquires input image data and inputs the image data into the image acquisition module through the SoC port. The image acquisition module includes:
the camera controller is used for controlling image acquisition of the camera equipment, and controlling the reading and writing time sequence and the cache of the acquired original image;
and the first input/output buffer is used for buffering the image data collected by the camera or the HDMI interface equipment.
The image data collected by the image collecting module is stored in the shared memory in the chip through the parallel bus. The shared memory can be directly accessed by the FPGA kernel and the MCU kernel at the same time, and data can be read and written in real time.
(2) Image preprocessing module
The original image acquired by the image acquisition module is based on the pixel resolution of the camera equipment, and the general resolution is higher, but in the edge-end embedded system, the hardware resource limitation of the embedded system is limited, and the used image resolution which can be identified and detected by the AI model is generally lower, so that the acquired original image data needs to be preprocessed by the image preprocessing module to generate the image data suitable for the AI model.
In addition, the general instruments are represented by multi-digit numbers, for example, 5 digits (i.e., a sequence including 5 digits), and the digital identification AI model identifies the digits 0 to 9 based on a unit digit, i.e., the AI model infers 1 digit at a time, so that image segmentation processing needs to be performed to segment the image acquired by the image acquisition module, N digits are segmented into N sets of image data, and each set of image data is subjected to AI model inference once to identify a digit corresponding to the set of image data.
The image preprocessing module is located in an FPGA core, and is implemented by using FPGA logic resources, as shown in fig. 6, the image gray scale algorithm module, the image segmentation algorithm module, the image enhancement algorithm module, and the second input/output buffer are built in, and are respectively used for image graying, clipping, transform enhancement, and input/output buffering.
And the image preprocessing module accesses the on-chip shared memory through the parallel bus, reads the original image stored in the shared memory by the image acquisition module, and caches the original image in the second input/output buffer.
And the image gray level algorithm module is internally provided with a Scalar algorithm and is used for carrying out gray level binarization on the original image and converting the gray level binarization into a gray level image.
And the image segmentation algorithm module is used for segmenting N digits in the gray level image into N groups of images with preset sizes, each group of images comprises 1 digit, and the N groups of segmented images are respectively and sequentially stored in N sections of address spaces of the fourth input and output buffer.
And the image enhancement algorithm module is internally provided with an FFT (Fourier transform) algorithm, performs Fourier transform on the image with the preset size to obtain image data, and caches the image data in a fourth input/output buffer.
And image data generated by image preprocessing is cached in a fourth input/output buffer and is stored in the shared memory through a parallel bus. The second input/output buffer and the fourth input/output buffer in this example may be the same input/output buffer.
(3) Digital recognition AI model parallel inference module
The image preprocessing module preprocesses the generated image data as data input for the AI model used by the digital identification AI model parallel inference module.
As shown in fig. 7, the digital recognition AI model parallel inference module includes an AI model interpreter, N model operation accelerator cores and an input-output buffer, where the N model operation accelerator cores implement AI model parallel inference of N sets of image data.
In the digital recognition AI model parallel inference module, an AI model interpreter of an MCU kernel part is connected and accessed to a shared memory through a system bus, the digital recognition AI model is analyzed through an MCU software method, the data type and all model arithmetic operators of the AI model are calculated, the model arithmetic operators of the corresponding types and the operation sequence which need to be operated are counted, and the statistical analysis conclusion of the model arithmetic operators is stored in the shared memory through the system bus. N model operation accelerator cores of the FPGA kernel part are connected and access to a shared memory through a parallel bus, each model operation accelerator core can use all operation operators in the artificial intelligence system, and each model operation accelerator core can operate a group of picture data according to the model operation operators needing to operate and the operation sequence counted by the AI model interpreter, so that a number corresponding to the group of picture data is identified.
The MCU reads N groups of image data generated by the image preprocessing module from the shared memory, sequentially stores the N groups of image data in N sections of addresses of the third input and output buffer according to the arrangement sequence of the numbers corresponding to the image data in the reading of the instrument panel, reads the model operation operators and the operation sequence which are generated by the AI model interpreter and need to be operated, dynamically allocates the number of the model operation accelerator cores which need to be operated in parallel according to the group number of the read image data (in the example, N groups of image data are provided, N model operation accelerator cores are dynamically allocated), transmits the model operation operators and the operation sequence which need to be operated to the N model operation accelerator cores, sequentially reads N groups of image data of the third input and output buffer according to the sequence, and correspondingly inputs the N groups of image data to the dynamically allocated model operation accelerator cores one by one according to the sequence from 1 to N.
And each model operation accelerator core operates the image data according to the model operation operator which needs to operate and the operation sequence generated by the AI model interpreter, and deduces and identifies the number of the group of image data. And after finishing the calculation of the AI model calculation operator, each model operation accelerator core generates a digital identification AI model inference conclusion, sequentially stores the numerical identification AI model inference conclusion to N sections of addresses of a fifth input/output buffer according to the sequence from 1 to N, and sequentially stores the numerical identification AI model inference conclusion to N sections of addresses of a shared memory through a parallel bus. For example, if N is 3, AI model inference results (three numbers) corresponding to three sets of image data, 1, 2, and 3, are stored in the 1 st, 2 nd, and 3 rd segment addresses of the fifth input/output buffer, respectively.
The third input/output buffer and the fifth input/output buffer in this example may be the same input/output buffer.
(4) AI model inference conclusion post-processing module
And the AI model conclusion deduction post-processing module is connected with the shared memory through a system bus and is used for combining the single digits deduced and recognized by the N model operation accelerator cores in the digital recognition AI model parallel deduction module into a complete N-bit digit according to the arrangement sequence in the fifth input and output buffer, namely the digit display of the monitored non-intelligent instrument.
The AI model inference conclusion post-processing module sequentially reads N number identification AI model inference conclusions from the shared memory through the system bus according to the sequence from 1 to N (for example, N is 3, and first, second, and third digits are read from the 1 st, 2 nd, and 3 rd segment addresses of the fifth input/output buffer), combines the N number of inference digits into a complete N number through shift calculation (for example, if N is 3, the first digit is shifted to the left by 2 bits, and the second digit is shifted to the left by 1 bit, so as to obtain a 3 digit), and transmits the complete N number digit to the wireless transmission module.
The positions and sizes of the N-segment addresses in the fourth and fifth input-output buffers can be preset.
(5) Wireless transmission module
The digital identification AI system of the example is internally provided with a wireless transmission module which is used for transmitting the instrument number generated by the AI model conclusion deduction post-processing module to a remote terminal through wireless transmission so as to realize the remote monitoring of the non-intelligent instrument. And the wireless transmission module driver is positioned in the MCU kernel and used for controlling the sending and receiving of the wireless transmission module.
In the present example, for the consideration of low power consumption and miniaturization, the digital recognition AI models used are all AI models in the TinyML field, and may include 29 layers of operators such as Conv2D, depthwiseConv2D, fullyConnected, averagepowing 2D, reshape, softMax, and the like, as well as an image input data layer and an inference result output data layer implemented by the MCU. At the cloud, through machine learning, the model learns a large number of digital data sources, trains an AI model that can be accurately used for the above-mentioned digital recognition, and the AI system of this example infers numbers 0 to 9 in the original image based on the trained AI model.
In other embodiments, other AI models for performing numerical identification may be used, and the present application is not limited thereto.
It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.
Claims (11)
1. An artificial intelligence system for performing character recognition, comprising: a shared memory;
the processing chip is used for analyzing an artificial intelligence AI model for character recognition, determining an operator to be operated and an operation sequence thereof and storing the operator to the shared memory;
the programmable logic device is used for acquiring an acquired original image containing characters to be recognized, processing the acquired original image into image data and storing the image data into the shared memory, reading the running sequence from the shared memory, scheduling corresponding arithmetic operators to carry out operation on the image data to obtain an image character recognition result and storing the image character recognition result into the shared memory;
the processing chip is also used for reading the image data from the shared memory and inputting the image data into the programmable logic device.
2. The artificial intelligence system of claim 1,
the programmable logic device processing the raw image into the image data, comprising: dividing the original image into a plurality of groups of image data, wherein each group of image data corresponds to an image of a character;
the scheduling of the corresponding arithmetic operator according to the running sequence to perform operation on the image data to obtain an image character recognition result includes: a plurality of scheduled model operation accelerator cores perform operation in parallel; each scheduled model operation accelerator core respectively operates a group of image data to obtain an image character recognition result corresponding to the group of image data;
the processing chip is further configured to schedule a corresponding number of model operation accelerator cores in the programmable logic device according to the number of groups of the image data, read the plurality of groups of image data from the shared memory, input the image data to the model operation accelerator cores in a one-to-one correspondence manner, and combine the image character recognition results corresponding to the plurality of groups of image data into a final recognition result.
3. The artificial intelligence system of claim 2, wherein:
the character to be recognized is a reading on an instrument panel;
the artificial intelligence system further comprises:
the wireless transmission module is used for sending the final identification result;
the processing chip is further used for driving the wireless transmission module and sending the final identification result to the wireless transmission module.
4. The artificial intelligence system of any of claims 1-3, wherein:
the processing chip is a Micro Control Unit (MCU), and the programmable logic device is a Field Programmable Gate Array (FPGA); the artificial intelligence system is a system on chip; the MCU is connected with the shared memory through a system bus, and the FPGA is connected with the shared memory through a parallel bus;
the operation operator is realized by an FPGA kernel.
5. The artificial intelligence system of claim 4, wherein:
the operators include one or more of the following types of operators: the system comprises a full-connection operator, a matrix transformation operator, a flexible maximum transmission operator, a common convolution operator, a depth convolution operator, a maximum pooling operator and an average pooling operator.
6. The artificial intelligence system of claim 1 wherein the programmable logic device comprises:
the camera controller is used for controlling the camera to collect images and controlling the reading and writing time sequence and the cache of the collected original images;
and the first input and output buffer is used for caching the original image collected by the camera and sending the original image to the shared memory through the parallel bus.
7. The artificial intelligence system of claim 6, wherein the programmable logic device further comprises:
a second input-output buffer for holding the original image read from the shared memory;
the image gray level algorithm module is used for carrying out gray level binarization processing on the original image and converting the gray level binarization processing into a gray level image;
the image segmentation algorithm module is used for segmenting the gray level image into images with preset sizes and caching the images into the second input and output buffer;
the image enhancement algorithm module is used for carrying out Fourier transform on the image with the preset size to obtain the image data and caching the image data to the second input and output buffer;
the second input output buffer is also used to store the image data to the shared memory through a parallel bus.
8. The artificial intelligence system of claim 7, wherein:
the processing chip comprises:
the AI model interpreter is used for analyzing the AI model for character recognition, calculating the data type and all arithmetic operators of the AI model, determining the arithmetic operators to be operated and the operation sequence thereof according to the calculation result, and storing the arithmetic operators and the operation sequence in the shared memory;
and the AI model inference conclusion post-processing module is used for combining the multiple groups of image character recognition results into the final recognition result and outputting the final recognition result.
9. The artificial intelligence system of any one of claims 1-3, 5-8, wherein:
the AI model for character recognition is a character recognition AI model obtained by training sample data at the cloud end; wherein, the sample data is a character data source.
10. A method for character recognition, applied in the artificial intelligence system for character recognition of any one of claims 1-9, the method comprising:
the processing chip analyzes an artificial intelligence AI model for character recognition, determines an operator to be operated and an operation sequence thereof and stores the operator to a shared memory;
the programmable logic device acquires an acquired original image containing characters to be recognized, processes the acquired original image into image data and stores the image data into the shared memory;
the processing chip reads the image data and inputs the image data to the programmable logic device;
and the programmable logic device reads the corresponding arithmetic operator of the operation sequence scheduling from the shared memory, performs operation on the image data to obtain an image character recognition result, and stores the image character recognition result in the shared memory.
11. The method of claim 10, wherein the character to be recognized is a reading on a dashboard; the method further comprises the following steps:
and the processing chip reads the image character recognition result from the shared memory, processes the image character recognition result into a final recognition result, and drives a wireless transmission module to send the final recognition result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110355563.2A CN115147840B (en) | 2021-03-31 | 2021-03-31 | Artificial intelligence system and method for character recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110355563.2A CN115147840B (en) | 2021-03-31 | 2021-03-31 | Artificial intelligence system and method for character recognition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115147840A true CN115147840A (en) | 2022-10-04 |
| CN115147840B CN115147840B (en) | 2024-05-17 |
Family
ID=83405535
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110355563.2A Active CN115147840B (en) | 2021-03-31 | 2021-03-31 | Artificial intelligence system and method for character recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115147840B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102044063A (en) * | 2010-12-23 | 2011-05-04 | 中国科学院自动化研究所 | FPGA (Field Programmable Gate Array) and DSP (Digital Signal Processor) based machine vision system |
| US20180046913A1 (en) * | 2016-08-12 | 2018-02-15 | DeePhi Technology Co., Ltd. | Combining cpu and special accelerator for implementing an artificial neural network |
| CN109766293A (en) * | 2019-02-01 | 2019-05-17 | 京微齐力(北京)科技有限公司 | Connect the circuit and System on Chip/SoC of FPGA and artificial intelligence module on chip |
| CN109932953A (en) * | 2017-12-19 | 2019-06-25 | 陈新 | Intelligent supercomputer programmable controller |
| CN110727633A (en) * | 2019-09-17 | 2020-01-24 | 广东高云半导体科技股份有限公司 | Edge artificial intelligence computing system framework based on SoC FPGA |
| CN110991522A (en) * | 2019-11-29 | 2020-04-10 | 山东万腾电子科技有限公司 | Character recognition method and system and industrial intelligent gateway |
-
2021
- 2021-03-31 CN CN202110355563.2A patent/CN115147840B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102044063A (en) * | 2010-12-23 | 2011-05-04 | 中国科学院自动化研究所 | FPGA (Field Programmable Gate Array) and DSP (Digital Signal Processor) based machine vision system |
| US20180046913A1 (en) * | 2016-08-12 | 2018-02-15 | DeePhi Technology Co., Ltd. | Combining cpu and special accelerator for implementing an artificial neural network |
| CN109932953A (en) * | 2017-12-19 | 2019-06-25 | 陈新 | Intelligent supercomputer programmable controller |
| CN109766293A (en) * | 2019-02-01 | 2019-05-17 | 京微齐力(北京)科技有限公司 | Connect the circuit and System on Chip/SoC of FPGA and artificial intelligence module on chip |
| CN110727633A (en) * | 2019-09-17 | 2020-01-24 | 广东高云半导体科技股份有限公司 | Edge artificial intelligence computing system framework based on SoC FPGA |
| US20210081770A1 (en) * | 2019-09-17 | 2021-03-18 | GOWN Semiconductor Corporation | System architecture based on soc fpga for edge artificial intelligence computing |
| CN110991522A (en) * | 2019-11-29 | 2020-04-10 | 山东万腾电子科技有限公司 | Character recognition method and system and industrial intelligent gateway |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115147840B (en) | 2024-05-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111797976B (en) | Training method, image recognition method, device, equipment and medium for neural network | |
| US11907760B2 (en) | Systems and methods of memory allocation for neural networks | |
| CN106960219B (en) | Picture identification method and device, computer equipment and computer readable medium | |
| CN108416327B (en) | Target detection method and device, computer equipment and readable storage medium | |
| CN110956202B (en) | Image training method, system, medium and intelligent device based on distributed learning | |
| CN110647920A (en) | Transfer learning method and device in machine learning, equipment and readable medium | |
| CN111209910A (en) | Systems, methods, and non-transitory computer-readable media for semantic segmentation | |
| CN109949219B (en) | Reconstruction method, device and equipment of super-resolution image | |
| CN111832666B (en) | Medical image data amplification method, device, medium, and electronic apparatus | |
| US20220083857A1 (en) | Convolutional neural network operation method and device | |
| CN110738235B (en) | Pulmonary tuberculosis judging method, device, computer equipment and storage medium | |
| US11544542B2 (en) | Computing device and method | |
| US11651191B2 (en) | Methods, apparatuses, and computer program products using a repeated convolution-based attention module for improved neural network implementations | |
| CN112163601A (en) | Image classification method, system, computer device and storage medium | |
| CN111338695A (en) | Data processing method based on pipeline technology and related product | |
| US20200175324A1 (en) | Segmentation of target areas in images | |
| CN108520532B (en) | Method and device for identifying motion direction of object in video | |
| CN115147840B (en) | Artificial intelligence system and method for character recognition | |
| EP3716150A1 (en) | Improved image segmentation using a neural network translation model | |
| CN115984822A (en) | SNN traffic sign identification method based on space attention and related device | |
| KR20210038027A (en) | Method for Training to Compress Neural Network and Method for Using Compressed Neural Network | |
| CN111104965A (en) | Vehicle target identification method and device | |
| CN115994892A (en) | Lightweight medical image segmentation method and system based on ghostnet | |
| CN114897133A (en) | Universal configurable Transformer hardware accelerator and implementation method thereof | |
| CN109102074B (en) | Training device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |