CN113939792A - User interface for machine language model creation - Google Patents

Abstract

This disclosure presents devices, methods, and computer-readable media for creating a user interface for a machine learning model. An application developer may select a machine learning template from a plurality of templates that are appropriate for the type of data used in their application (304). The templates may include a plurality of templates for classification of image, text, sound, motion, and form data. The graphical user interface (300) allows for the intuitive selection of training data (316), validation data (318), and integration of the trained model (314) into an application. The user interface also displays numerical scores for both training accuracy (508) and validation accuracy (510) using the test data. The application provides a live mode that allows machine learning models to be executed on the mobile device to allow the model to be tested according to data from one or more of the sensors (i.e., cameras or microphones) on the mobile device.

Description

User interface for machine language model creation

Cross Reference to Related Applications

This application claims the benefits and priority of U.S. provisional application No. 62/855,958 filed on day 1,6, 2019, and U.S. patent application No. 16/670,914 filed on day 31, 10, 2019; each of the above-mentioned patent applications is incorporated by reference herein in its entirety.

Technical Field

The present disclosure relates generally to techniques and user interfaces for generating and training machine learning models for application development.

Background

Machine learning techniques can be implemented by application developers to make predictions and decisions without the use of explicit instructions, but instead relying on patterns and effects. Machine learning techniques create mathematical models based on sample data (referred to as "training data"). Machine learning techniques may be used in various applications, such as email filtering, computer vision, and predictive analysis. Models developed for machine learning are often developed with applications separate from those used to create the applications. These separate machine learning tools may also involve different programming languages than the application. Providing techniques to implement machine learning tools into the same integrated development environment in which applications are developed may allow models to be trained, tested, and seamlessly integrated into applications employing these machine learning techniques more easily.

Disclosure of Invention

Certain embodiments relate to techniques (e.g., methods, memory or non-transitory computer-readable media storing code or instructions executable by one or more processors) for creating a user interface for a machine learning model in an integrated development environment. The disclosed machine learning tools may allow the entire flow of training and testing models to be integrated into an application using an interactive graphical user interface. Machine Learning (ML) templates may allow developers to easily create custom models without having to program separate machine learning models. For example, the training data may be retrieved from a storage device or may be integrated from a live capture/recording of images or sound. Both stored data and live data may be used to train and test the model, both within the application.

In various implementations, the techniques may be implemented in an application development program. The techniques may allow a developer to select from a plurality of available templates. For example, templates can be categorized into image, text, sound, motion, and table data. The machine learning model may allow classification of data and/or regression of real-valued targets within a set of data.

These and other embodiments of the invention are described in detail below. For example, other embodiments relate to systems, devices, and computer-readable media associated with the methods described herein.

Further areas of applicability of the present disclosure will become apparent from the detailed description provided hereinafter. It should be understood that the detailed description and specific examples, while indicating various embodiments, are intended for purposes of illustration only and are not intended to necessarily limit the scope of the disclosure.

Drawings

FIG. 1 illustrates an exemplary simplified diagram of a system for generating a trained machine learning model for application development.

FIG. 2 illustrates an exemplary graphical user interface for selecting templates for generating a trained machine learning model.

FIG. 3 illustrates an exemplary graphical user interface for selecting training data for a sound classifier.

FIG. 4 illustrates an exemplary graphical user interface for dragging training data for a sound classifier.

FIG. 5 illustrates an exemplary graphical user interface for training a machine learning model.

Fig. 6 illustrates an exemplary graphical user interface for the output of a machine learning model, in particular, a sound classifier.

FIG. 7 illustrates an exemplary graphical user interface of a progress bar for a technique for generating a trained machine learning model.

Fig. 8 illustrates an exemplary graphical user interface for the output of a machine learning model, in particular, an image classifier.

FIG. 9 is an exemplary flow diagram of a technique for generating a trained machine learning model for application development.

Fig. 10 illustrates a block diagram for an exemplary device to perform one or more techniques described herein.

In the drawings, similar components and/or features may have the same reference label. Further, various components of the same type may be distinguished by following the reference label by a connection number and a second label that distinguishes among the similar components. If only the first reference label is used in the specification, the description is applicable to any one of the similar components having the same first reference label irrespective of the second reference label.

Detailed Description

Certain embodiments of the present disclosure relate to an apparatus, computer-readable medium, and method for creating a machine learning model in an integrated development environment. The machine learning tool provides the developer with options regarding the type of data to choose from. A developer may select a template appropriate for the type of data used in their application.

The machine learning tools may allow developers to easily create trained machine learning models for incorporation into applications. The machine learning tools may be application-based and may run as standalone applications or in an Integrated Development Environment (IDE) of the operating system. In some embodiments, the trained machine learning model created by the tool (e.g., provided as a standalone application) can then be incorporated into a program being developed in the IDE. This enables developers to take advantage of the inherent development tools provided by operating systems, rather than attempting to incorporate models developed by third party programs. This also allows for easier testing of applications in an integrated development environment.

Applications incorporating machine learning models may be executed on a portable computing device, such as a laptop or tablet. By allowing selection of model types and/or model sources from a graphical user interface, the machine learning tool may be user-friendly. For example, the various model types may include an image analysis model, a sound analysis model, a text analysis model, a tabular data analysis model, and so on. The model source may include a computer or network location of training data. The graphical user interface may allow a developer to drag and drop training data into a designated area. Similarly, a developer may select evaluation or test data through an intuitive user interface. The machine learning tool may provide a graphical workflow to assist developers in developing and training various phases of the machine learning model. Further, the graphical user interface may provide the developer with a graphical depiction of the various stages and sub-stages used to generate the trained model. In some examples, the graphical user interface may display a numerical score that utilizes both training accuracy and validation accuracy of the test data.

I. Generating a trained machine learning model

The disclosure herein describes various user-friendly techniques for generating trained machine learning models. In supervised learning, the technique builds a mathematical model from a set of data that contains both inputs and desired outputs. For example, if the task is to determine whether an image contains an object, the training data for the supervised learning technique may include images (inputs) with and without the object, and each image may have a label (output) that specifies whether it contains an object. The techniques may be used for a variety of different types of data. The technique may include selecting a template for a model type (e.g., image analysis, text analysis, sound analysis, motion analysis, and tabular data analysis). Next, the technique can include inputting training data for training the model. Part of the training process may include identifying and selecting test data to evaluate the model. After the model has been trained, the model may be evaluated using the test data. The techniques may generate one or more scores as a result of the evaluation. The trained models may be stored and derived for use in other applications.

Fig. 1 shows a simplified diagram of a technique 100 for machine learning model generation. The machine learning tool 102 may be a software application executing on a computing device. The machine learning tool 102 may include a presentation of a plurality of graphical user interfaces to allow developers to interact with routines.

A. Selecting a template

A developer may first consider whether their application should incorporate machine learning. Machine learning is suitable for performing specific tasks without the use of explicit instructions. Machine learning techniques accomplish these tasks by learning and using patterns found in the data set. Some typical uses of machine learning techniques include assisting manual data entry, detecting unwanted email messages (otherwise known as spam), recommending products, diagnostic techniques, customer segmentation and life value prediction, predictive maintenance, financial analysis, voice analysis, image recognition, text recognition, and form data analysis.

If machine learning techniques are suitable for the application, the developer may use the machine learning tool 102 to generate a trained machine learning model 104. The machine learning tool 102 can present a template selector that presents options for different categories of data from which a developer can select. Categories may include general categories of image, text, sound, motion, and tabular data. Each category may also have a sub-category. For example, image analysis may have sub-categories of image classification and object detection.

The machine learning tool 102 may provide a template module 110. The template module 110 may provide a graphical user interface to allow a developer to select a template from a plurality of templates. Templates may be organized by category and subcategory, corresponding to different model types. Such graphical user interfaces allow developers to select from different graphical representations of the aforementioned categories. The graphical user interface also provides a description of the particular template. For example, a developer may select a template by clicking on a description with a pointing device or selecting on a touch screen. After selecting the template, the machine learning tool 102 may open a project view that may guide the developer through the steps of building the trained model. The stages for generating the trained machine learning model may include problem development, data collection, training the model, and deploying the trained model.

Fig. 2 illustrates an exemplary graphical user interface 200 for selecting templates for generating a trained machine learning model. The template GUI 200 depicts a template list 202 that may include image, text, sound, motion, and form templates, although in other embodiments other templates may be provided. The developer may also display "all," as shown in FIG. 2, which displays all templates in a scrollable presentation. The categories 204 may be displayed over icons 206 of the templates. As shown in FIG. 2, GUI 200 shows two templates for image categories 204. Fig. 2 depicts an icon 206 and a label 208 for each template in the category 204. For example, fig. 2 depicts icons 206 and labels 208 for an "image classifier" template and an "object detector" template. Fig. 2 shows that an image classifier template is selected and a text description box 210 provides additional information for the template.

The following is a brief description of the different classification types of the model.

1. Image analysis template

In some embodiments, the image analysis classification may have two different types of models. One model type is an image classification model that may be trained to classify objects depicted in a visual image. The second type of model is an object detection model. The object detection model allows for the detection of a specific object, animal or human, in an image. In one non-limiting example, a developer may want to create a flower image classification model. The flower classification model may analyze images of the flowers to improve a core image classification model to detect particular flowers. For example, a flower classification model may correctly classify images of roses, narcissus, sunflowers, etc. from images captured by a user's smartphone.

Image classification is the question of identifying what label to apply to an image from a set of categories. The first step is to collect training data 106. The training data 106 may include a plurality of image files. The image file may be in a variety of image file formats. The image file formats may include Joint Photographic Experts Group (JPEG), exchangeable image file format (Exif), Tagged Image File Format (TIFF), Graphics Interchange Format (GIF), Windows Bitmap (BMP), Portable Network Graphics (PNG), and Better Portable Graphics (BPG). Other formats exist and may be used. Developers can capture many different types of images that reflect the real data that the classifier can ultimately see, and then mark them. Ideally, the data should reflect the actual use of the application. Thus, for example, if a developer is creating a custom image classifier to be used by a user on their smartphone, the developer should collect pictures and possibly fewer screenshots from the smartphone.

The image classification model may provide an initial category label that provides insight about the image (e.g., whether the image was taken indoors or outdoors). The image classification model may also detect a room (e.g., a kitchen) in which the image was captured. Object detection may add a bounding box around the object in the image and suggest a label for the object in the bounding box.

The machine learning tool 102 also incorporates live mode. Using live mode, a developer can take images from a smartphone or other electronic device and test the accuracy of the image classifier during development.

2. Text analysis template

The machine learning tool 102 may employ at least two types of models for text analysis. One type of model is natural language processing, which helps to process and understand the meaning of human language text. All machine learning in natural language driven natural language processing is done on the device to protect the privacy of the user. The text analysis model may include text classification and word tagging. The text classifier takes a block of text, such as a sentence or paragraph or an entire document, and applies tags thereto. Text classification can be used to determine sentiment analysis, spam analysis, and topic analysis.

In a test classification workflow, the training data 106 may include original text files organized in folders whose authentic labels are the names of the folders. Alternatively, the training data may use comma separated values, where the text labels and the real labels are separated by commas. JavaScript formats are also supported.

In the text classifier, a language identifier may identify the native language of the text. The language identifier may identify a dominant language of the text file. The language assumption API may specify a maximum number of languages contained in the text file and calculate an associated probability that the text is in one of these languages. The text may then be converted into a token, which may be further changed to a feature value. The training data may then be used to train a core model of the text classifier by comparing the output to the real data.

For example, a text classifier may be used to automatically evaluate multiple comments. Thus, for text classification, a developer may create an application to understand user comments and expectations to automatically know whether a given comment is a positive comment or a negative comment, or in between. Such a process may be referred to as sentiment classification. Another exemplary use of a text classifier is to classify articles by using article summaries or perhaps even just article titles. Another possible implementation is an automated travel agent that can determine the topic category that the customer is asking. The text classifier may determine whether the customer is interested in a hotel or restaurant or flight.

The word tagger may consider a sentence as a sequence of words and then apply a tag in context to each word in the sentence. The word tagger may generate a custom name tagger for automatic name recognition of custom names, such as product names. The word tagger may divide the sentence portion of the text, either as sequential words or as text blocks in a text classifier. For example, the word tagger may tag words as people, organizational entities, slang words, and the like. The word tagger parses the plain text and interprets the structure of the words.

All machine learning for natural language processing may be performed on the device to protect the privacy of the user.

Other features of text analysis include automatic voice tagging and named entity recognition. The named entity recognition API can return search results related to the named entity in the text. Based on what the user has browsed in the web search tool, the named entity identification can suggest names and locations that are relevant to the browsing history, thereby enhancing the search experience in the application.

3. Sound analysis template

The machine learning tool 102 may employ one or more sound analysis templates. The sound classifier can be used to identify a particular instrument, such as a barbell, cymbal, drum, guitar, etc. Laughter and silence can also be classified. A sound classifier may be trained using a plurality of sound segments. Once trained, the sound classifier may analyze the sample sound segments and determine the classification of the sound at different time windows.

The sound classifier can be combined with other features (e.g., live recording features), which would enable live listening and near real-time analysis of sound. In this live analysis mode, the machine learning tool 102 may listen to a microphone on the computing device locally or remotely by using sound input from a microphone of a companion device such as a smartphone.

4. Motion analysis template

The machine learning tool 102 may employ one or more motion analysis templates. The motion classifier may identify particular motions captured from one or more sensors (e.g., accelerometers) on, for example, a smartphone or wearable device. For example, motion analysis templates may be trained in an appropriate fashion for the forehand and backhand swings in tennis. The machine learning tool 102 enables developers to integrate motion analysis templates into applications for improving tennis swings. Similar routines may be written to identify a particular workout. The motion analysis model may be trained to identify specific exercises and capture the number of repetitions of each exercise.

5. Form data analysis template

The machine learning tool 102 may employ one or more templates for analyzing the form data. The tabular data may be structured data organized in a tabular format. One example may be house sales information for a given area. The house sales information may include characteristics of the house, such as the number of bedrooms, the number of bathrooms, the total square feet of the house, proximity to school, and the sales price of the house. The developer may want to use this information to predict sales prices based on one or more of these characteristics. The machine learning tool 102 may import the data as a Comma Separated Values (CSV) file or JavaScript format. The template may allow for easy calculations, such as price per square foot. The template may also allow for easy filtering of data, such as filtering out less than a certain number of houses in a bedroom.

B. Input of training data for generating machine learning modelsInto

Returning to fig. 1, the machine learning tool 102 may allow a developer to input training data 106. The training data 106 may include a plurality of files stored in a memory of the computing device or on a network. The training module 116 may use the training data 106 to modify the selected template parameters to cause the trained model to be able to identify or classify image, sound, motion, or form data based on the selected template and the training data. The training data 106 may include metadata that identifies one or more characteristics of the file (e.g., image, sound, or sensor data characteristics). The machine learning tool 102 supports a number of algorithms, such as boosted tree regression, random forests, and the like. All these algorithms are represented by their classification. To train the selected model, the developer would only have to write one line of code that identifies the target and the data source. Thus, in this case, the developer may select linear regression or regularized linear regression, and the developer will identify the house data and the column is the price.

The training data 106 may be structured data files of different formats. The data files may be associated with metadata files. In the case of structured data files, folders may provide information about files contained within the file. For example, for a flower classifier, a file containing a rose image may be labeled "rose". The machine learning tool 102 may identify the name of the folder and associate features of the images contained in the folder with the rose. In the case of sound analysis, the folder may contain the sound clip of the guitar. The folder may be labeled "guitar" and the machine learning tool 102 may associate the characteristics of the sound clip contained within the folder with the guitar.

The training data 106 may be contained in a memory of a computing device on which the machine learning tool 102 is executing. The training data 106 may also be located in a storage device connected to the computing device. Alternatively, the training data 106 may be stored in a database provided from a cloud-based service.

In various embodiments, the data may be collected and tagged as a dictionary, with string tags corresponding to an array of images. The data set may also be organized into a hierarchical directory structure such that the label is the name of the folder in which all the images are contained. Other data sources may also exist, such as a single folder containing the tagged filename.

The machine learning tool 102 may include a training data input module 112. The training data input module may include an Application Programming Interface (API) for extracting training data 106 and associated data structures from a memory of the computing device. The training data 106 and associated data structures may be imported from a database server. The database server may be a cloud-based server accessible over a network (e.g., the internet).

The machine learning tool 102 may provide errors and warnings to developers at various trigger points in the process. For example, when a developer drags and drops the training data 106, the technique may perform data analysis on the training data 106 and may display errors and warnings on a user interface. At each stage of the process, the machine learning tool 102 may identify and display errors and warnings to developers. Some errors and warnings may send developers back to the previous stage to correct. In addition, the machine learning tool 102 also has a completely separate view to provide insight into the data independent of the particular task. For example, the separate view may inform the developer by plotting a distribution of labels for possible classifications. In a separate view, the histogram may depict multiple annotations for each of the different classifications of the classifier for the identified data source. The detailed view may inform the developer whether several images are duplicate or very similar. This allows developers to understand the richness of the data.

Turning to fig. 3, an input graphical user interface 300 for selecting training data for a sound classifier is shown. The input graphical user interface 300 lists the name 302 of the project file and the model template 304. The machine learning tool 102 shown in FIG. 1 may allow a developer to select model templates from various model sources that may be further trained using training data. As shown in FIG. 3, the name 302 of the project file is "MySoundClassifier 2" and the model template 304 is "first test". In input block 306, the number of detected classifications for the "first test" model template may be displayed. The training block 308 may indicate the accuracy of the trained model on the selected training data. Verification block 310 may show the accuracy of the trained model on the verification data. Test block 312 may show the accuracy of the trained model on the test data. Output box 314 may display the name of the output file of the trained model. In FIG. 3, output block 314 indicates that the model has not been trained.

The input graphical user interface 300 allows a developer to select model templates 304, training data 316, validation data 318, and test data 320. The graphical user interface 300 provides a first drop-down menu 322 to allow the developer to select the location of the training data 316. Training data 316 may be uploaded from a file in memory of the computing device. Alternatively, the training data 316 may also be dragged and dropped onto the training data box 317 from another file on the computing device. The developer may select one or more files to be used as verification data 318 and test data 320. In some embodiments, a developer may select a particular classification of data to use as training data 316.

The input graphical user interface 300 provides a second drop down menu 324 for selecting the validation data 318. The validation data 318 may be uploaded from a file in memory of the computing device. Alternatively, the verification data 318 may be dragged and dropped onto the verification data box 319. The developer may select one or more files to use as the validation data 318. The developer may select "auto" in which the machine learning tool 102 withholds a preselected amount of training data 316 (e.g., 10% of the training data 316) as validation data 318.

The input graphical user interface 300 provides a third drop-down menu 326 for the test data 320. After the model has been trained and validated, the developer may upload additional test data 320 to test the classification of the trained model. Test data 320 may be uploaded from a file in memory of the computing device. Alternatively, test data 320 may be dragged and dropped onto test data box 321.

The input graphical user interface 300 provides a status box 328 indicating "training data is needed". This prompts the developer to identify training data 316 using the interface.

Fig. 4 illustrates an exemplary graphical user interface 400 for selecting training data for a sound classifier. Fig. 4 is a continuation of the graphical user interface 300 shown in fig. 3. The graphical user interface 400 lists the name 402 of the project file and the model template 404. As shown in FIG. 4, the name 402 of the project file is "MySoundClassifier 2" and the model template 404 is "first test". In input block 406, the number of classifications detected in training data 416 is displayed. Training box 408 shows the accuracy of the model on the selected training data. The verification block 410 shows the accuracy of the model on the verification data. Test block 412 shows the accuracy of the trained model on the test data. The output box 414 shows the name of the output file. In fig. 4, output block 414 indicates that the model has not been trained.

In fig. 4, the graphical user interface 400 shows that training data 416 has been added. The machine language tool 102 has identified 1,608 entries in the training data 416. Input box 404 indicates that machine language tool 102 has identified 50 data classes in training data 416.

Graphical user interface 400 allows a developer to select training data 416, validation data 418, and test data 420. The graphical user interface 400 provides a first drop-down menu 422 for selecting a location of the training data 416. Training data 416 may be uploaded from a file in memory of the computing device. Alternatively, training data 416 may be dragged and dropped onto training data box 417. The developer may select one or more files to use as training data 416. In some embodiments, a developer may select a particular classification of data to use as training data 416. Upon selection of the training data 416, the graphical user interface may display the number of items 421 contained in the training data 416.

The graphical user interface 400 provides a second drop-down menu 424 for selecting the verification data 418. The validation data 418 may be uploaded from a file in memory of the computing device. Alternatively, the validation data 418 may be dragged and dropped onto the validation data box 419. The developer may select one or more files to use as verification data 418. The developer may also select "auto" from a second drop-down menu 424, where the machine learning tool 102 obscures a preselected amount of training data 416 as validation data 418.

The graphical user interface 400 provides a third drop-down menu 420 for the test data 420. After the model has been trained and validated, the developer may upload additional test data 420 to test the classification of the trained model. Test data 420 may be uploaded from a file in memory of the computing device. Alternatively, test data 420 may be dragged and dropped onto test data box 423.

The graphical user interface 400 provides a status box 428 indicating that the model is "ready for training". The "training ready" notification prompts the developer to perform the training process by selecting the play button 430.

Returning to FIG. 1, the training data 106 may be stored in a file on the desktop of the integrated development environment. The developer may simply drag the training data folder to the designated area. The machine learning tool 102 may determine how many categories to classify by the number of folders of training data and provide some high-level summaries. For example, in the flower classifier example, the machine learning tool 102 may understand that a developer is generating a 17-class classifier with 2000 training images.

C. Training machine learning model

For example, the following machine learning model may be employed: linear regression, logistic regression, decision trees, Support Vector Machines (SVMs), naive bayes, K nearest neighbors (knns), K-means clustering, random forests, dimensionality reduction algorithms, gradient boosting algorithms (GBMs) (e.g., XGBoost, LightGBM, and castboost). Various techniques may employ one or more algorithms to train the model to achieve the desired goal of target accuracy, such as using gradient methods with back propagation, ensemble learning, variational bayes methods, and adaboost.

After the training data 106 is imported, the model may be trained. The developer may begin training by selecting play button 430, as shown in FIG. 4. The training module 116 modifies the model using the training data 106 to predict the desired outcome. The training data 106 may extract one or more features from each training data asset. Training module 116 may associate one or more features of a training data asset with a tag provided by an associated data structure. The tags may also take the form of metadata associated with the training data 106. The training module 116 analyzes hundreds, thousands, or even millions of data elements to refine a Machine Learning (ML) model into a trained model. Ideally, after training, the model may predict the expected output when new, unseen data (i.e., test data 108) is presented.

The training module 116 may incorporate autocomplete features for incorporating the training data 106. The training module 116 allows identification of training data in the form of a label dictionary and data array, data source, or specified model training parameters. The graphical user interface of the training module may include a console that informs the developer that training module 116 is extracting features and the number of iterations the training module is running. The graphical user interface may provide a quick view to inform the developer how many examples have been evaluated by training module 116 and the number of classifications that have been evaluated.

Fig. 5 illustrates a training graphical user interface 500 for generating a machine learning model. Fig. 5 is a continuation of the graphical user interface 400 depicted in fig. 4. The graphical user interface 500 lists the name 502 of the project file and the model template 504. As shown in FIG. 5, the name 502 of the project file is "MySoundClassifier" and the model template 504 is identified as "first test". The graphical user interface 500 depicts a progress indicator 516 showing the progress of the machine learning technique. Input block 506 shows that 50 classes are detected in the training data. The training box 508 shows that the current training accuracy of the model is 93%. Verification block 510 shows that the accuracy of the trained model on the verification data is 82%. Test box 512 shows that the accuracy of the trained model on the test data is 73%. Output box 514 shows icon 516 and the size (in bytes) of the output file. The output file shown in fig. 5 is 9.3 Megabytes (MB).

The training graphical user interface 500 depicts a line graph of the training accuracy line 518 and the validation accuracy line 520 over the number of iterations of the model. The accuracy block 522 may provide the developer with further insight into the accuracy of the trained model for different classifications. For example, the training graphical user interface 500 may depict the precision rate 526 and recall rate 528 for each category 524 of data. Precision rate refers to the percentage of the correlation result. The precision rate can be calculated as the true positive result divided by the actual result (true positive result plus false positive result). Recall refers to the percentage of the total correlation results that the trained model correctly calculated. The recall rate can be calculated as true positive divided by the predicted result or (true positive plus false negative). The accuracy of the training model can be calculated as true positives plus true negatives divided by the total result. The trained model shown in fig. 5 is a sound classifier, and thus the model indicates various classifications of sound (i.e., glass break, sneeze, rooster, etc.).

The graphical user interface 500 provides a status box 528 indicating "training is completed after 1 hour and 30 minutes in 2019, 5 months and 26 days at night 11: 31". The developer may make a copy of the results by selecting the "make copy" button 534. The developer may also drag and drop the executable output file icon 515 into a file directory on the computing device.

Fig. 6 shows an output graphical user interface 600 for the output of a machine learning model, in particular a sound classifier. Fig. 6 is a continuation of the training graphical user interface 500 depicted in fig. 5. Output graphical user interface 600 may be selected by selecting output file box 614. The output graphical user interface 600 lists the name 602 of the project file and the model template 604. As shown in FIG. 6, the name 602 of the project file is "MySoundClassifier" and the model template 604 is "first test". Input box 606 shows that 50 classes are detected in the training data. Training box 608 shows that the current training accuracy of the trained model is 93%. The verification block 610 shows that the accuracy of the trained model on the verification data is 82%. Test box 612 shows that the accuracy of the model on the test data is 73%. Output box 614 shows icon 615 and the size of the output file (in bytes).

The output graphical user interface 600 allows selection of one of several test files 616. The audio file 618 with the label "2-98866-a-47. wav" has been selected in fig. 6. Audio file 618 can be played by selecting play button 620. The output graphical user interface 600 provides a graphical depiction 622 of the sound waves. The graphical depiction 622 displays a bar 624 that shows the current location of execution of the audio file 618. The graphical depiction 622 displays the current time 626 and a category label 628 above and below the graphical depiction 622. The portion of the audio file 618 in fig. 6 has received the category label 628 for "airplane". The output graphical user interface 600 provides a detail section 630. For example, the detail section 630 may include: the full file name of the output file, the size of the file (in megabytes), the model name, the file author, the license identifier, and the file description.

The output graphical user interface 600 provides a status box 632 indicating "training is complete after 1 hour 30 minutes in 2019, 5 month 26 night 11: 31". The developer may make a copy of the results by selecting the "make copy" button 634. The developer may also drag and drop executable output file 614 into a file directory on the computing device.

Fig. 7 illustrates a progress bar 700 for a technique of generating a trained machine learning model. The progress bar 700 is divided into three phases: settings 702, training 704, and output 706. Progress bar 700 shows input box 708 showing that 50 classifications are detected in the training data. Training box 710 shows that the current training accuracy of the trained model is 93%. Verification block 712 shows that the accuracy of the trained model on the verification data is 82%. Test block 714 shows that the accuracy of the trained model on the test data is 73%. Output box 716 shows the name and size (in bytes) of the output file.

Fig. 8 shows an output graphical user interface 800 for the output of a machine learning model, in particular, an image classifier. The output graphical user interface 800 may be selected by selecting the output file 818. The output graphical user interface 800 lists the name 802 of the project file and the model template 804. As shown in FIG. 8, the name 802 of the project file is "FlowerClassifier", and the model template 804 is "My model". Output graphical user interface 800 displays two data sources 806, specifically, flower training data 807 and flower test data 809. The setup block 810 shows that 17 classes are detected in the training data. Training box 812 shows that the current training accuracy of the trained model is 100%. Verification block 814 shows that the accuracy of the trained model on the verification data is 73%. Test block 816 shows that the accuracy of the trained model on the test data is 82%. Output box 818 shows an icon 819 and the size (in bytes) of the output file.

Fig. 8 shows a displayed test image 820. An information box 822 for the test image 820 may also be displayed. Information box 822 may present the file name of the test image and the percentage of the trained model that depicts the test image 820 as a particular classification. For example, for the bellflower image depicted in fig. 8, information box 822 indicates that the trained model classified the image as 100% bellflower and 0% as snowdrop, fritillary bulb, crocus, and tulip. The output graphical user interface 800 provides a drag-and-drop interface 824 for evaluating other test images. The output graphical user interface 800 also provides summary details 826 for the image classifier being executed.

In some embodiments, a migration learning technique may be employed. Migration learning is a machine learning method in which a trained model developed for a task is reused as a starting point for a model on a second task. Migration learning differs from traditional machine learning in that a pre-trained model that has been used for another task is used to jump on to start the development process on a new task or problem. The benefit of the migration learning is that it can speed up the time it takes to develop and train the model by reusing these patches or modules of the developed model. This helps to speed up the model training process and speed up the results.

Often, a model that has been trained on a large dataset may provide portions of its code to other models. The module may be used to build a new model on a new data set. This is another method for speeding up the model building process and obtaining efficiency in your machine learning workflow.

The migration learning technique may be applied to the top of the model already present in the machine learning tool. Using these techniques, the last few layers of the model can be retrained to specific data, so the system no longer needs millions of data sets to be trained. In addition to shortening the time required for training, this can also result in much smaller models falling from hundreds of megabytes to only a few megabytes or even kilobytes for thousands of images.

Returning to fig. 1, the user interface of the machine learning tool 102 may allow the developer to drag-and-drop the added training data into the model and observe the builder in a live view. The training module 116 may employ a graphics processing unit of the computing system to perform the analysis. If multiple sub-phases exist, training module 116 may provide an indication of progress during training, even within a training phase. The progress indication may be a graphical representation or textual information to inform the developer of the training progress. As training progresses, the machine learning tool 102 may calculate and display the accuracy of the training model. The machine learning tool may dynamically update the accuracy as the training progresses. The machine learning tool may even distinguish the accuracy of different training classes. For example, in a flower classifier, the machine learning tool 102 may distinguish between the accuracy of a rose and the accuracy of carnation. This may provide the developer with insight into the model.

Previous machine learning tools did not save information in the various steps of generating the trained model. Thus, the developer cannot return to see what was done to create the machine learning model. The disclosed technology allows developers to see how the trained models were developed to make modifications easier. By saving the information to include accuracy and verification settings, developers can make changes to the training data to test the impact on model generation. Furthermore, previous models do not provide live interaction with mobile devices. By allowing developers to use data from mobile devices (e.g., a user's smart phone), this allows machine learning tools to be tested by providing end-user interaction with the machine language model before the application into which the model is incorporated even completes.

The training module 116 may detect errors during data training. In some cases, the data may be in an incorrect format. In some cases, one or more classifications for data may have insufficient data. Training module 116 may display warnings and errors. An exemplary warning may be due to data imbalance. For example, there may be 100 more rose images than a chrysanthemum. Errors and warnings may even be represented graphically. Training module 116 may provide recommendations for developers to fix errors and warnings. In some implementations, training module 116 may automatically correct the error (e.g., rebalance the classification).

One way in which developers can influence model development is by balancing data. One way to balance the data may be by removing duplicates. Another way in which a developer can modify a model is by adding noise to the image in the case where the image is very clear. The developer may also modify the orientation of the image.

The training module 116 inserts checkpoints at various points during the training of the model. The use of checkpoints allows developers to return to previous versions of the model. Thus, if the developer finds that including a new batch of training data would adversely affect the trained model, the developer can easily return to the version of the trained model at the previous checkpoint that occurred before the new batch of training data was included. This saves the developer time during the training process because the developer can not only easily see the impact of new data, but can also return to an earlier version of the model to modify the training data to improve the trained model.

Training module 116 may also schedule when resources of the computing device are available (e.g., after hours or at night). This allows developers to easily use the resources of the computing device to present computing tasks and run intensive processing when demand for other tasks is low.

D. Selecting test data to evaluate a model

The machine learning tool 102 may allow a developer to input test data 108. The test data 108 is data that the machine learning tool 102 has not been used for training and that has not been seen previously by the machine learning tool 102.

Test data 108 may include one or more metadata files associated with test data 108. The metadata file may identify the actual identity of the file. For a classification model, test data 108 should include files for several different classifications. For example, for a flower image classifier, the test data 108 should include multiple images of different flowers (e.g., roses, carnations, narcissus, etc.). Each image will have metadata that provides the true identity of each image of test data 108.

The machine learning tool 102 may include a test data input module 114. Test data input module 114 may include an Application Programming Interface (API) for retrieving test data 108 and associated data structures from a memory of a computing device. Test data 108 and associated data structures may be imported from a database server. The database server may be a cloud-based server accessible over a network (e.g., the internet). In some embodiments, test data input module 114 may be the same module as training data input module 112.

E. Evaluating trained models using test data

The machine learning tool 102 may include an evaluation module 118 for evaluating the model using the test data 108. The evaluation module 118 can evaluate the trained model by analyzing features of the test data 108 that were not previously seen. Test data 108 may include metadata that may provide the true identity of the test data. In this way, the machine learning tool 102 may compare the output of the trained model to the real data. In this way, the machine learning tool 102 may calculate the accuracy of the trained model. The graphical user interface may display the percentage accuracy of the model. For example, for a flower classifier, test data 108 may include multiple images of different types of flowers. The evaluation module 118 may test the accuracy of the flower classifier model by comparing the output of the flower classifier to ground truth. The machine learning tool 102 allows the categories to be sorted by accuracy.

Another feature of the machine learning tool 102 is that it allows a developer to drag images into the trained model and analyze the output. FIG. 8 illustrates a drag-and-drop interface 824 that allows a user to drag an image to test a trained model. Other models may provide similar interfaces to test other types of files (e.g., a sound classifier may allow dragging sound clips into a drag-and-drop interface to classify sound files). This provides the developer with details about the individual predictions for the model. A developer may want to use images from a camera on an electronic device (e.g., a smartphone). The machine learning tool 102 provides a live mode in which the developer can switch to a mode that allows images to be captured from the smartphone and tested. Previously, a developer would have to write an application to play on a smartphone with a model, then use the application on the smartphone and write code to display some graphical details there. The machine learning model 102 is application-based and allows the model to be easily tested on a smartphone during development without writing a single line of code for the application.

F. Storing and exporting trained models

The machine learning tool 102 may include a storage module 120 for storing the trained and evaluated model in a memory of the computing system. The model may also be stored in cloud storage accessible via a network (i.e., the internet).

The machine learning tool 102 may also include an export/import module 122. Export/import module 122 may convert the trained model into lines of code for incorporation into an application. The trained model derivation can be done in an integrated development environment. Code export may be as simple as dragging an icon of a trained model into a folder of an application.

Export/import module 122 may also allow developers to import models that are stored in memory of the computing device. Once imported, the model may be retrained with additional training data and reevaluated with additional test data.

Techniques for generating machine learning models using graphical user interfaces

The technology discloses a graphical user interface for easily building a machine learning model. These techniques may be used by novice developers who may have little machine learning language programming experience. The user interface guides the developer to select templates based on the type of model, to select training data, to identify test data, and to derive trained models. The techniques may also utilize real-time sensor information from one or more devices to evaluate the trained model. Fig. 9 shows a flow diagram of a process 900 for a technique for generating a machine learning model using a graphical user interface. The method may be performed on an electronic device (e.g., a laptop computer, a desktop computer, or a tablet computer).

At 902, the technique can include receiving a selection of a template of a plurality of templates as a machine learning model, each template corresponding to a classification of a data type. The selection may be made via a user interface. The template may be presented on a graphical user interface. The template may be presented by a classification of a plurality of classifications. The classifications may include images, text, sounds, motion, and tables. The template may be selected by interacting with a pointing device (e.g., mouse, trackball, touch screen display). Selection of a template may be indicated on the graphical user interface by highlighting or changing the color and/or shading of icons and labels on the display.

At 904, the technique can include identifying a location of a plurality of training data, the plurality of training data including a first plurality of structured data records and a first associated metadata record, each metadata record of the first associated metadata record identifying at least a category label of the first plurality of structured data records. The training data may be located on the graphical user interface using a drop down menu. The training data may be located in a folder stored in a memory of the computing device. The training data may be located in a folder in a database in the cloud-based network. The developer identifies the location of the training data by dragging and dropping the selected training data into a designed training data box on the graphical user interface.

In some embodiments, the technique may include analyzing the plurality of training data to determine a number of classifications. In some implementations, the techniques may determine whether each of the plurality of classifications conforms to the plurality of training data. In this way, the techniques may generate an error or an alert if, for example, audio training data is identified for the image classifier, and vice versa. The technique may also include selecting one or more of the plurality of determined classifications for training a Machine Learning (ML) model. The developer may want to exclude one or more of the determined classifications when training the model. For example, the developer may exclude "roses" from the classification of the flower classifier. The determined number of classifications may be displayed on a graphical user interface.

At 906, the technique can include training a Machine Learning (ML) model by analyzing each of the first plurality of structured data records and the first associated metadata record. The training may analyze each of the plurality of structured data records and first associated metadata records for one or more elements in order to train a model. The elements may depend on the type of classifier. For example, image elements may be identified for an image classifier, text elements may be used for a text analyzer, and so on.

At 908, the technique can include identifying a location of a plurality of validation data, the plurality of validation data including a second plurality of structured data records and a second associated metadata record, each metadata record of the second associated metadata record identifying at least a category label of the second plurality of structured data records. The verification data may be located on the graphical user interface using a drop down menu. The verification data may be located in a folder stored in a memory of the computing device. The validation data may be located in a folder in a database in the cloud-based network. The developer identifies the location of the validation data by dragging and dropping the selected training data into a designed training data box on the graphical user interface. The technique may also receive a selection that may automatically select the plurality of validation data from a portion of the training data (i.e., 10% of the training data may be suppressed and not used for training but for validation).

At 910, the technique can include validating the trained model by analyzing each of the second plurality of structured data records to generate an identification of each of the second plurality of structured data records. The accuracy of the trained data may be determined by comparing the identified classification to the classification tags in the metadata of the second plurality of structured data records. Precision and recall may be calculated for each data category.

At 912, the technique may include displaying an accuracy score by comparing the identification of each of the second plurality of structured data records with the second associated metadata record to generate an accuracy score. Validation accuracy may be calculated during training and displayed as a numerical percentage on a graphical user interface, or plotted against iteration number in an online graph. The precision and recall may be displayed for each data category.

At 914, the technique can include generating code for a trained model that includes the trained model and is executable on the mobile device to classify data obtained from sensors of the mobile device. In various embodiments, the code may be executable code. The code may be saved as a file. Since the file may include executable code, the model may run on other mobile devices (e.g., smart phones, wearable devices, or tablet computing devices). In this way, a developer can test the machine learning model on-device for one or more beta testers without having to complete the coding of the application. Testing with images, sounds, or motion received from other mobile devices provides developers insight into how models work from actual sensors on the mobile devices. The sensor may include a camera, microphone, motion sensor (e.g., accelerometer), keyboard, keypad, or touch screen display. In some embodiments, the technique may receive feedback from the one or more beta users regarding the accuracy of the classification provided by the model.

In some embodiments, the technique may include analyzing the plurality of training data to determine a plurality of classifications, each classification corresponding to a different classification label. The technique may include comparing the classification labels of the plurality of training data to determine consistency with the categories of data types.

In some embodiments, the technique may include analyzing the plurality of training data to determine a plurality of classifications, each classification corresponding to a different classification label. The technique may include displaying a list of the classifications of the plurality of training data on a graphical user interface. The technique may include receiving a selection of one or more of a plurality of classifications for training a machine learning model.

In some embodiments, the technique may include displaying a list of the classifications of the plurality of training data on a graphical user interface. The technique may include receiving a selection of one or more of a plurality of classifications for training a machine learning model.

In some embodiments, the technique may include receiving an identification of a location of the plurality of training data by selecting an icon associated with the training data and dragging the icon onto a designated area on a graphical user interface.

In some embodiments, identifying the location of the plurality of validation data comprises automatically selecting a random portion of the training data. Random portions of the training data may be prevented from training the machine learning model.

In some embodiments, the technique may include receiving a selection that will automatically select the plurality of validation data from the training data. The technique may include preventing a preselected percentage of training data training. The blocked training data may include a second plurality of structured data records. The technique may include validating the trained model by analyzing each of the second plurality of structured data records to generate an identification of each of the second plurality of structured data records and the second associated metadata record. The technique may include generating an accuracy score by comparing the identification of each of the second plurality of structured data records to the second associated metadata record.

The technique may include detecting that a threshold of the plurality of training data is reached. The threshold may be the number of training data records required to generate the machine language model. The technique may include automatically training the machine learning model when a threshold amount of training data is identified. The technique can include generating code for a trained model.

In various aspects, a computing device includes one or more memories and one or more processors in communication with the one or more memories and configured to execute instructions stored in the one or more memories to perform any of the operations disclosed herein.

In various aspects, a non-transitory computer-readable medium stores a plurality of instructions that, when executed by one or more processors of a computing device, cause the one or more processors of the computing device to perform operations comprising any of the operations disclosed herein.

It should be appreciated that the specific steps illustrated in fig. 9 provide a particular technique for capturing and aggregating operational metrics for third party applications according to various embodiments of the present disclosure. Other sequences of steps may also be performed according to alternative embodiments. For example, alternative embodiments of the present invention may perform the steps outlined above in a different order. Moreover, the various steps shown in FIG. 9 may include multiple sub-steps that may be performed in various sequences as appropriate to the individual step. Also, additional steps may be added or removed depending on the particular application. One of ordinary skill in the art would recognize many variations, modifications, and alternatives.

Exemplary device

Each of the methods described herein may be implemented by a computer system. Each of the steps of these methods may be performed automatically by a computer system and/or may have input/output related to a user. For example, a user may provide input for each step in a method, and each of these inputs may be in response to a particular output requesting such input, where the output is generated by a computer system. Each input may be received in response to a corresponding request output. Further, the input may be received as a data stream from a user, from another computer system, retrieved from a memory location, retrieved over a network, requested from a web service, and so forth. Likewise, the output may be provided as a data stream to a user, to another computer system, saved in a memory location, sent over a network, provided to a web service, and so forth. In short, each step of the methods described herein may be performed by a computer system and may involve any number of inputs, outputs, and/or requests to and from the computer system, which may or may not involve a user. Those steps that do not involve the user can be said to be performed automatically by the computer system without human intervention. Thus, it will be understood in light of this disclosure that each step of each method described herein can be altered to include input and output to and from the user, or can be accomplished automatically by a computer system without manual intervention, with any determination made by a processor. Furthermore, some embodiments of each of the methods described herein may be implemented as a set of instructions stored on a tangible, non-transitory storage medium to form a tangible software product.

The computing device may be a laptop computer with data stored locally in memory. Alternatively, multiple computing devices may be used for distributed processing with data located in a cloud service. The machine learning tool may also be executed on a tablet computer.

Fig. 10 is a block diagram of an exemplary device 1000, which may be a mobile device on which the techniques may be performed. The device 1000 generally includes a computer-readable medium (memory) 1002, a processing system 1004, input/output (I/O) subsystems 1006, radio circuitry 1008, and audio circuitry 1010, including a speaker 1050 and a microphone 1052. These components may be coupled by one or more communication buses or signal lines 1003. Device 1000 may be any portable electronic device, including a handheld computer, a tablet, a mobile phone, a laptop computer, a tablet, a media player, a wearable device, a Personal Digital Assistant (PDA), a key fob, a car key, a door access card, a multifunction device, a mobile phone, a portable gaming device, an automotive display unit, and so forth, including a combination of two or more of these items.

Device 1000 can be a multifunction device with display 1054. According to some embodiments, the display 1054 may be a touch screen. The touch screen optionally displays one or more graphics within a User Interface (UI). In some embodiments, the user can select one or more of these graphics by making a gesture on the graphics, for example, with one or more fingers or with one or more styluses. In some embodiments, selection of one or more graphics will occur when the user breaks contact with the one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (left to right, right to left, up, and/or down), and/or a rolling of a finger (right to left, left to right, up, and/or down) that has made contact with device 1000. In some implementations, or in some cases, inadvertent contact with a graphic does not select the graphic. For example, when the gesture corresponding to the selection is a tap, a swipe gesture that swipes over the application icon optionally does not select the corresponding application. Device 1000 may optionally further include one or more physical buttons, such as a "home" button, or a menu button. Optionally, the menu button is used to navigate to any application in a set of applications that are optionally executed on computing device 1000. Alternatively, in some embodiments, the menu button is implemented as a soft key in a graphical user interface displayed on the touch screen.

Device 1000 may incorporate display 1054. The display 1054 may be a Liquid Crystal Display (LCD), Organic Light Emitting Diode (OLED), Active Matrix Organic Light Emitting Diode (AMOLED), super active matrix light emitting diode (AMOLED), Thin Film Transistor (TFT), in-plane switch (IPS), or thin film transistor liquid crystal display (TFT-LCD), which are commonly found in computing devices. Display 1054 may be a touch screen display of a computing device.

In one embodiment, device 1000 includes a touch screen, menu buttons, push buttons for turning the device on and off and locking the device, one or more volume adjustment buttons, a Subscriber Identity Module (SIM) card slot, a headset jack, and a docking/charging external port. Depressing the button optionally serves to turn the device on/off by depressing the button and holding the button in a depressed state for a predefined time interval; locking the device by depressing the button and releasing the button before the predefined time interval has elapsed; and/or unlocking the device or initiating an unlocking process. In an alternative embodiment, device 1000 also accepts voice input through a microphone for activating or deactivating certain functions. Device 1000 also optionally includes one or more contact intensity sensors for detecting the intensity of contacts on the touch screen, and/or one or more tactile output generators for generating tactile outputs for a user of device 1000.

In one illustrative configuration, the device 1000 may include at least one computer-readable medium (memory) 1002 and one or more processing units (or processors) 1018. The processor 1018 may be suitably implemented in hardware, software, or a combination thereof. Computer-executable instructions or firmware implementations of processor 1018 may include computer-executable instructions written in any suitable programming language to perform the various functions described.

Computer-readable media (memory) 1002 may store program instructions that are loadable and executable on processor 1018, and data generated during execution of the programs. Depending on the configuration and type of device 1000, memory 1002 may be volatile memory (such as Random Access Memory (RAM)) and/or non-volatile memory (such as Read Only Memory (ROM), flash memory, etc.). The device 1000 may have one or more memories. Apparatus 1000 may also include additional removable and/or non-removable storage devices, including but not limited to magnetic, optical, and/or tape storage devices. The disk drives and their associated non-transitory computer-readable media may provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for the devices. In some implementations, the memory 1002 may include a variety of different types of memory, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), or ROM. Although the volatile memory described herein can be referred to as RAM, any volatile memory that will not retain the data stored therein after being unplugged from a host and/or power source is suitable.

Memory 1002 and additional storage, whether removable or non-removable, are examples of non-transitory computer-readable storage media. For example, non-transitory computer readable storage media may include volatile or nonvolatile, removable or non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Memory 1002 and additional storage are examples of non-transitory computer storage media. Additional types of computer storage media that may be present in device 1000 include, but are not limited to: phase change RAM (pram), SRAM, DRAM, RAM, ROM, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), Digital Video Disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by device 1000. Combinations of any of the above should also be included within the scope of non-transitory computer readable storage media. Based on the disclosure and teachings provided herein, a person of ordinary skill in the art will be able to contemplate other ways and/or methods of implementing the various embodiments. However, as described above, the computer-readable storage medium does not include a transitory medium such as a carrier wave or the like.

Alternatively, computer readable communication media may include computer readable instructions, program modules or other data transmitted within a data signal such as a carrier wave or other transmission means. However, as used herein, computer-readable storage media does not include computer-readable communication media.

Device 1000 may also contain a communication connection 1008 that allows device 1000 to communicate with a data store, another device or server, a user terminal, and/or other devices via one or more networks. Such networks may include any one or combination of many different types of networks, such as a wired network, the internet, a wireless network, a cellular network, a satellite network, other private and/or public networks, or any combination thereof. Device 1000 may also include I/O devices 1006 such as touch input devices, keyboards, mice, pens, voice input devices, displays, speakers, printers, etc.

It should be apparent that the architecture shown in fig. 10 is merely one example of an architecture for device 1000, and device 1000 may have more or fewer components or a different configuration of components than shown. The various components shown in fig. 10 may be implemented in hardware, software, or a combination of both hardware and software, including one or more signal processing circuits and/or application specific integrated circuits.

Wireless circuitry 1008 is conventional circuitry (such as antenna) used for communicating over a wireless link or network to one or more other devicesA wire system, a Radio Frequency (RF) transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a memory, etc.). The wireless circuitry 1008 may use various protocols, such as those described herein. The wireless circuitry 1008 may, for example, have a wireless interface for one wireless protocol (e.g.,

) And a separate component for another wireless protocol, such as Ultra Wideband (UWB). Different antennas may be used for different protocols.

The radio circuit 1008 is coupled to the processing system 1004 via a peripheral interface 1016. The interface 1016 may include conventional components for establishing and maintaining communication between peripheral devices and the processing system 1004. Voice information and data information received through the wireless circuitry 1008 (e.g., in a voice recognition or voice command application) are sent to the one or more processors 1018 via the peripheral interface 1016. The one or more processors 1018 can be configured to process various data formats for one or more application programs 1034 stored on the computer-readable medium (memory) 1002.

Peripheral interface 1016 couples the device's input and output peripherals to processor 1018 and computer-readable medium 1002. The one or more processors 1018 are in communication with the computer-readable medium 1002 via a controller 1020. The computer-readable medium 1002 may be any device or medium that can store code and/or data for use by the one or more processors 1018. Media 1002 may include a hierarchy of memories including a cache, a primary memory, and a secondary memory.

Device 1000 can also include a power system 1042 for powering the various hardware components. Power system 1042 can include a power management system, one or more power sources (e.g., battery, Alternating Current (AC)), a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., a Light Emitting Diode (LED)), and any other components typically associated with the generation, management, and distribution of power in a mobile device.

In some embodiments, the apparatus 1000 includes a camera 1044. In some embodiments, device 1000 includes a sensor 1046. The sensors 1046 may include accelerometers, compasses, gyroscopes, pressure sensors, audio sensors, light sensors, barometers, and the like. The sensor 1046 may be used to sense positional aspects, such as auditory or optical marking of the position.

In some embodiments, device 1000 may include a GPS receiver, sometimes referred to as GPS unit 1048. A mobile device may use a satellite navigation system, such as the Global Positioning System (GPS), to obtain positioning information, timing information, altitude, or other navigation information. During operation, the GPS unit may receive signals from GPS satellites orbiting the Earth. The GPS unit analyzes the signal to estimate the time of transmission and the distance. The GPS unit may determine the current location (current position) of the mobile device. Based on these estimates, the mobile device may determine a location fix, altitude, and/or current speed. The location fix may be geographic coordinates such as latitude information and longitude information.

The one or more processors 1018 run various software components stored in the media 1002 to perform various functions of the device 1000. In some embodiments, the software components include an operating system 1022, a communication module (or set of instructions) 1024, a location module (or set of instructions) 1026, a machine learning module 1028, an interactive development environment 1030, and other applications (or sets of instructions) 1034.

Operating system 1022 may be any suitable operating system, including iOS, Macintosh operating system (Mac OS), Darwin, Quatros real-time operating system (RTXC), LINUX, UNIX, OS X, Microsoft Windows, or an embedded operating system such as VxWorks. The operating system may include various programs, instruction sets, software components, and/or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.), and facilitates communication between various hardware and software components. Operating system 1022 is system software that manages computer hardware and software resources and provides general-purpose services to computer programs. For example, operating system 1022 can manage interaction between a user interface module and one or more user applications. The various embodiments may also be implemented in a wide variety of operating environments, which in some cases may include one or more user computers, devices, or processing devices that may be used to operate any of a number of applications. The user devices or client devices may include any of a number of general purpose personal computers, such as desktop or laptop computers running a standard operating system, as well as cellular devices, wireless devices, and handheld devices running mobile software and capable of supporting multiple networking protocols and instant messaging protocols. This system may also include multiple workstations running any of a variety of commercially available operating systems and other known applications for purposes such as development and database management. These devices may also include other electronic devices such as virtual terminals, thin clients, gaming systems, and other devices capable of communicating via a network.

The communication module 1024 facilitates communication with other devices through the one or more external ports 1036 or via the wireless circuitry 1008, and includes various software components for processing data received from the wireless circuitry 1008 and/or the external ports 1036. An external port 1036 (e.g., a Universal Serial Bus (USB), firewire, lightning connector, 60-pin connector, etc.) is adapted to couple to other devices directly or indirectly through a network (e.g., the internet, a wireless Local Area Network (LAN), etc.).

Location/motion module 1026 can assist in determining the current location (e.g., coordinates or other geographic location identifier) and motion of device 1000. Modern positioning systems include satellite-based positioning systems such as Global Positioning System (GPS), cellular network positioning based on "cell ID", and Wi-Fi positioning technology based on Wi-Fi networks. GPS also relies on the visibility of multiple satellites to determine a position estimate, which may not be visible (or have weak signals) indoors or in "urban canyons". In some embodiments, location/motion module 1026 receives data from GPS unit 1048 and analyzes the signals to determine the current location of the mobile device. In some embodiments, location/motion module 1026 may use Wi-Fi or cellular location technology to determine the current location. For example, knowledge of nearby cell sites and/or Wi-Fi access points and knowledge of their locations can be used to estimate the location of the mobile device. Information identifying the Wi-Fi or cellular transmitter is received at the wireless circuit 1008 and communicated to the location/motion module 1026. In some embodiments, the location module receives one or more transmitter IDs. In some embodiments, the sequence of transmitter IDs may be compared to a reference database (e.g., a cell ID database, a Wi-Fi reference database) that maps or correlates transmitter IDs to location coordinates of corresponding transmitters and calculates estimated location coordinates of device 1000 based on the location coordinates of the corresponding transmitters. Regardless of the particular positioning technology used, the position/motion module 1026 receives information from which a position fix can be derived, interprets the information, and returns position information, such as geographic coordinates, latitude/longitude, or other position fix data.

The electronic device may include a machine learning module 1028. The machine learning module 1028 receives a selection of one or more model templates, inputs a plurality of training data, trains a model on the training data, validates the model, and generates an executable file for the trained model.

The electronic device may also include an interactive development environment 1030. The interactive development environment in operating system 1022 may include a suite of software development tools for developers to generate software applications. The integrated development environment 1030 may support source code for the following programming languages: C. c + +, Objective-C, Objective-C + +, Java, AppleScript, Python, Ruby, ResEdit (Rez), and Swift, with various programming models, including but not limited to Cocoa, Carbon, and Java. Some integrated development environments also support GNU Pascal, Free Pascal, Ada, C #, Perl, and D.

The integrated development environment 1030 may include developer documentation, built-in interface builders, applications for constructing graphical user interfaces. The version of the integrated development environment may include a modified version of the GNU compiler set. The integrated development environment 1030 may support the development of microcomputers, portable computing system operating systems, smart phones, wearable devices, media access devices, and software development suites. The integrated development environment 1030 may also include a downloadable simulator.

The one or more application programs 1034 on the mobile device can include any application installed on the device 1000, including but not limited to a browser, an address book, a contact list, email, instant messaging, word processing, keyboard emulation, desktop applets, JAVA-enabled applications, encryption, digital rights management, voice recognition, voice replication, a music player (playing back recorded music stored in one or more files, such as MP3 files or Advanced Audio Coding (AAC) files), and so forth.

There may be other modules or sets of instructions (not shown) such as a graphics module, a time module, etc. For example, the graphics module may include various conventional software components for rendering, animating and displaying graphical objects (including, but not limited to, text, web pages, icons, digital images, animations and the like) on the surface of the display. In another example, the timer module may be a software timer. The timer module may also be implemented in hardware. The time module may maintain various timers for any number of events.

The I/O subsystem 1006 may be coupled to a display system (not shown), which may be a touch-sensitive display. The display system displays visual output to a user in a GUI. The visual output may include text, graphics, video, and any combination thereof. Some or all of the visual outputs may correspond to user interface objects. Although the display may use LED (light emitting diode) technology, LCD (liquid crystal display) technology, or LPD (light emitting polymer display) technology, other display technologies may be used in other embodiments.

In some embodiments, the I/O subsystem 1006 may include a display and user input devices such as a keyboard, mouse, and/or touch pad. In some implementations, the I/O subsystem 1006 can include a touch-sensitive display. The touch sensitive display may also accept input from a user based on haptic and/or tactile contact. In some embodiments, the touch sensitive display forms a touch sensitive surface for accepting user input. The touch-sensitive display/surface (along with any associated modules and/or sets of instructions in medium 1002) detects contact (and any movement or release of the contact) on the touch-sensitive display and converts the detected contact into interaction with user interface objects, such as one or more soft keys displayed on the touch screen when the contact occurs. In some embodiments, the point of contact between the touch-sensitive display and the user corresponds to one or more fingers of the user. The user may contact the touch-sensitive display using any suitable object or appendage, such as a stylus, pen, finger, or the like. The touch-sensitive display surface may use any suitable touch-sensitive technology to detect contact and any movement or release thereof, including capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch-sensitive display.

In addition, the I/O subsystem may also be coupled to one or more other physical control devices (not shown), such as buttons, keys, switches, rocker buttons, dials, slide switches, levers, LEDs, etc., for controlling or performing various functions, such as power control, speaker volume control, phone ring tone loudness, keyboard entry, scrolling, hold, menu, lock screen, clear and end communication, etc. In some embodiments, in addition to a touch screen, device 1000 may include a touch pad (not shown) for activating or deactivating particular functions. In some embodiments, the trackpad is a touch-sensitive area of the device that, unlike a touchscreen, does not display visual output. The trackpad may be a touch-sensitive surface separate from the touch-sensitive display, or an extension of the touch-sensitive surface formed by the touch-sensitive display.

In some embodiments, some or all of the operations described herein may be performed using an application executing on a user's device. The circuits, logic modules, processors, and/or other components may be configured to perform various operations described herein. Those skilled in the art will appreciate that such configuration can be accomplished through design, setup, interconnection, and/or programming of specific components, depending on the implementation, and that the configured components may or may not be reconfigurable for different operations, again depending on the implementation. For example, a programmable processor may be configured by providing appropriate executable code; a dedicated logic circuit may be configured by appropriately connecting logic gates and other circuit elements; and so on.

Most embodiments utilize at least one network familiar to those skilled in the art, using any of a variety of commercially available protocols to support communications, such as the transmission control protocol/internet protocol (TCP/IP), the open systems interconnection model (OSI), the File Transfer Protocol (FTP), universal plug and play (UpnP), the Network File System (NFS), the Common Internet File System (CIFS), and AppleTalk. The network may be, for example, a local area network, a wide area network, a virtual private network, the internet, an intranet, an extranet, a public switched telephone network, an infrared network, a wireless network, and any combination thereof.

In implementations utilizing a web server, the web server may run any of a variety of server or mid-tier applications, including hypertext transfer protocol (HTTP) servers, File Transfer Protocol (FTP) servers, Common Gateway Interface (CGI) servers, data servers, Java servers, and business application servers. One or more servers may also be capable of executing programs or scripts in response to requests from user devices, such as by executing one or more applications that may be implemented in any programming language, such as

C. C # or C + + or any scripting language such as Perl, Python, or TCL, and combinations thereof. The one or more servers may also include database servers, including but not limited to those available from

And

those commercially available.

Such programs may also be encoded and transmitted using carrier wave signals adapted for transmission over wired, optical, and/or wireless networks conforming to various protocols, including the internet. Likewise, a computer-readable medium according to an embodiment of the present invention may be created using a data signal encoded by such a program. The computer readable medium encoded with the program code may be packaged with a compatible device or provided separately from other devices (e.g., via internet download). Any such computer-readable medium may exist or be located within a single computer product (e.g., a hard drive, a CD, or an entire computer system), and may exist or be located within different computer products within a system or network. The computer system may include a monitor, printer, or other suitable display for providing any of the results mentioned herein to a user.

The environment may include various data stores and other memory and storage media, as described above. These may reside in various locations, such as on storage media local to one or more computers or remote from any or all of the computers on the network (and/or in one or more computers). In a particular set of embodiments, the information may reside in a Storage Area Network (SAN) familiar to those skilled in the art. Similarly, any necessary files for performing the functions attributed to a computer, server, or other network device may be stored locally and/or remotely as desired. When the system includes computerized devices, each such device may include hardware elements that may be electrically coupled via a bus, including, for example, at least one Central Processing Unit (CPU), at least one input device (e.g., a mouse, keyboard, controller, touch screen, or keypad), and at least one output device (e.g., a display device, printer, or speaker). Such systems may also include one or more storage devices, such as magnetic disk drives, optical storage devices, and solid state storage devices such as RAM or ROM, as well as removable media devices, memory cards, flash memory cards, and the like.

Such devices may also include a computer-readable storage media reader, a communication device (e.g., modem, network card (wireless or wired), infrared communication device, etc.), and working memory as described above. The computer-readable storage media reader can be connected to or configured to receive non-transitory computer-readable storage media representing remote, local, fixed, and/or removable storage devices, as well as storage media for temporarily and/or more permanently containing, storing, transmitting, and retrieving computer-readable information. The system and various devices may also typically include a number of software applications, modules, services or other elements located within the at least one working memory device, including an operating system and application programs, such as a client application or browser. It will be appreciated that alternative embodiments may have many variations in accordance with the above. For example, customized hardware might also be used and/or particular elements might be implemented in hardware, software (including portable software, such as applets), or both. In addition, connections to other devices such as network input/output devices may be used.

Any of the software components or functions described in this patent application may be implemented as software code executed by a processor using any suitable computer language, such as, for example, Java, C + +, C #, Objective-C, Swift, or a scripting language using, for example, conventional or object-oriented techniques, such as Perl or Python. The software code may be stored on a computer-readable medium as a series of instructions or commands to enable storage and/or transmission. Suitable non-transitory computer readable media may include Random Access Memory (RAM), Read Only Memory (ROM), magnetic media such as a hard drive or floppy disk, or optical media such as a Compact Disc (CD) or DVD (digital versatile disc), flash memory, or the like. The computer readable medium may be any combination of such storage devices or transmission devices.

Computer programs incorporating the various features of the present disclosure may be encoded on a variety of computer-readable storage media; suitable media include magnetic disks or tapes, optical storage media such as Compact Disks (CDs) or DVDs (digital versatile disks), flash memory, and the like. The computer readable storage medium encoded with the program code may be packaged with a compatible device or provided separately from other devices. Further, the program code may be encoded and transmitted over wired, optical, and/or wireless networks conforming to a variety of protocols, including the internet, to allow distribution, such as via internet download. Any such computer-readable media may reside or be located within a single computer product (e.g., a solid state drive, a hard drive, a CD, or an entire computer system), and may exist or be located within different computer products within a system or network. The computer system may include a monitor, printer, or other suitable display for providing any of the results mentioned herein to a user.

The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the disclosure as set forth in the claims.

As described above, one aspect of the present technology is to collect and use data from various sources to display rectangular content in non-rectangular areas. The present disclosure contemplates that, in some instances, such collected data may include personal information data that uniquely identifies or may be used to contact or locate a particular person. Such personal information data may include demographic data, location-based data, phone numbers, email addresses, twitter IDs, home addresses, data or records related to the user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other identifying or personal information.

The present disclosure recognizes that the use of such personal information data in the present technology may be useful to benefit the user. For example, the personal information data may be used to display information about the operational metrics of the third party application. Thus, the use of such personal information data may be presented to the user on the display. In addition, the present disclosure also contemplates other uses for which personal information data is beneficial to a user. For example, health and fitness data may be used to provide insight into the overall health condition of a user, or may be used as positive feedback for individuals using technology to pursue health goals.

The present disclosure contemplates that entities responsible for collecting, analyzing, disclosing, transmitting, storing, or otherwise using such personal information data can comply with established privacy policies and/or privacy practices. In particular, such entities should enforce and adhere to the use of privacy policies and practices that are recognized as meeting or exceeding industry or government requirements for maintaining privacy and security of personal information data. Such policies should be easily accessible to users and should be updated as data is collected and/or used. Personal information from the user should be collected for legitimate and legitimate uses by the entity and not shared or sold outside of these legitimate uses. Furthermore, such acquisition/sharing should be performed after receiving user informed consent. Furthermore, such entities should consider taking any necessary steps to defend and secure access to such personal information data, and to ensure that others who have access to the personal information data comply with their privacy policies and procedures. In addition, such entities may subject themselves to third party evaluations to prove compliance with widely accepted privacy policies and practices. In addition, policies and practices should be adjusted to the particular type of personal information data collected and/or accessed, and to applicable laws and standards including specific considerations of jurisdiction. For example, in the united states, the collection or acquisition of certain health data may be governed by federal and/or state laws, such as the health insurance association and accountability act (HIPAA); while other countries may have health data subject to other regulations and policies and should be treated accordingly. Therefore, different privacy practices should be maintained for different personal data types in each country.

Regardless of the foregoing, the present disclosure also contemplates embodiments in which a user selectively prevents use or access to personal information data. That is, the present disclosure contemplates that hardware elements and/or software elements may be provided to prevent or block access to such personal information data. For example, with respect to third party application evaluation techniques, the inventive technique may be configured to allow a user to opt-in or opt-out of participating in the collection of personal information data during or at any time after registration service. As another example, the user may choose not to provide personal information to be displayed. As another example, the user may choose to limit the amount of personal data to maintain or completely inhibit the display of personal data. In addition to providing "opt-in" and "opt-out" options, the present disclosure contemplates providing notifications related to accessing or using personal information. For example, the user may be notified that their personal information data can be accessed when the application is downloaded, and then be reminded again just before the personal information data is accessed by the application.

Further, it is an object of the present disclosure that personal information data should be managed and processed to minimize the risk of inadvertent or unauthorized access or use. Once the data is no longer needed, the risk can be minimized by limiting data collection and deleting data. In addition, and when applicable, including in certain health-related applications, data de-identification may be used to protect the privacy of the user. De-identification may be facilitated by removing particular identifiers (e.g., date of birth, etc.), controlling the amount or specificity of stored data (e.g., collecting location data at a city level rather than at an address level), controlling how data is stored (e.g., aggregating data among users), and/or other methods, as appropriate.

Thus, while the present disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, the present disclosure also contemplates that various embodiments may be implemented without the need to access such personal information data. That is, various embodiments of the present technology do not fail to function properly due to the lack of all or a portion of such personal information data. For example, the content may be selected and delivered to the user by inferring preferences based on non-personal information data or an absolute minimum amount of personal information, such as content requested by a device associated with the user, other non-personal information available to third party application evaluation techniques, or publicly available information.

Other variations are within the spirit of the disclosure. Accordingly, while the disclosed technology is susceptible to various modifications and alternative constructions, certain illustrated embodiments thereof are shown in the drawings and have been described above in detail. It should be understood, however, that there is no intention to limit the disclosure to the specific forms disclosed, but on the contrary, the intention is to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of the disclosure as defined by the appended claims.

The use of the terms "a" and "an" and "the" and similar referents in the context of describing the disclosed embodiments (especially in the context of the following claims) is to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The terms "comprising," "having," "including," and "containing" are to be construed as open-ended terms (i.e., meaning "including, but not limited to,") unless otherwise noted. The term "connected" is to be construed as being partially or wholly contained, attached or joined together, even if there is something intervening. The phrase "based on" should be understood as open-ended, and not limiting in any way, and is intended to be interpreted or otherwise understood as "based at least in part on" where appropriate. Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., "such as") provided herein, is intended merely to better illuminate embodiments of the disclosure and does not pose a limitation on the scope of the disclosure unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the disclosure.

Unless specifically stated otherwise, an extraction language such as the phrase "X, Y or at least one of Z" is understood in context to be commonly used to present items, terms, etc., which may be X, Y or Z, or any combination thereof (e.g., X, Y and/or Z). Thus, such disjunctive language is not generally intended to, and should not, imply that certain embodiments require that at least one of X, at least one of Y, or at least one of Z each be present. Additionally, joint language such as the phrase "at least one of X, Y, and Z" should also be understood to mean X, Y, Z or any combination thereof, including "X, Y and/or Z," unless specifically stated otherwise.

Preferred embodiments of this disclosure are described herein, including the best mode known to the inventors for carrying out the disclosure. Variations of those preferred embodiments may become apparent to those of ordinary skill in the art upon reading the foregoing description. The inventors expect skilled artisans to employ such variations as appropriate, and the inventors intend for the disclosure to be practiced otherwise than as specifically described herein. Accordingly, this disclosure includes all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Moreover, this disclosure includes any combination of the above-described elements in all possible variations thereof unless otherwise indicated herein or otherwise clearly contradicted by context.

All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and were set forth in its entirety herein.

Claims (20)

1. A method for generating a machine learning model using a graphical user interface, the method comprising execution by an electronic device, comprising:

receiving, via a user interface, a selection of a template of a plurality of templates that define a machine learning model, each template corresponding to a category of data types;

identifying a location of a plurality of training data, the plurality of training data including a first plurality of structured data records and first associated metadata records, each of the first associated metadata records identifying at least a classification label of the first plurality of structured data records;

training the machine learning model by analyzing each of the first plurality of structured data records and the first associated metadata record to generate a trained model;

identifying a location of a plurality of validation data, the plurality of validation data including a second plurality of structured data records and a second associated metadata record, each of the second associated metadata records identifying at least the category label of the second plurality of structured data records;

validating the trained model by analyzing each of the second plurality of structured data records to generate an identification of each of the second plurality of structured data records;

displaying the accuracy score by comparing the identification of each of the second plurality of structured data records with the second associated metadata record to generate an accuracy score; and

generating code for the trained model, the code comprising the trained model and executable on a mobile device to classify data obtained from a sensor of the mobile device.

2. The method of claim 1, further comprising:

analyzing the plurality of training data to determine a plurality of classifications, each classification corresponding to a different classification label; and

comparing the classification labels of the plurality of training data to determine consistency with the category of the data type.

3. The method of claim 1, further comprising:

analyzing the plurality of training data to determine a plurality of classifications, each classification corresponding to a different classification label;

displaying a list of the classifications of the plurality of training data on the graphical user interface; and

receiving a selection of one or more of the plurality of classifications for training the machine learning model.

4. The method of any of claims 2 to 3, further comprising:

displaying a list of the classifications of the plurality of training data on the graphical user interface; and

receiving a selection of one or more of the plurality of classifications for training the machine learning model.

5. The method of any of claims 1-3, further comprising receiving the identification of the location of the plurality of training data by selecting an icon associated with the training data and dragging the icon onto a designated area on the graphical user interface.

6. The method of any of claims 1-3, wherein identifying the location of the plurality of validation data comprises automatically selecting a random portion of the training data, wherein the random portion of the training data is prevented from training the machine learning model.

7. The method of any of claims 1 to 3, further comprising:

receiving a selection that will automatically select the plurality of validation data from the training data;

blocking a preselected percentage of the training data from the training, the blocked training data comprising the second plurality of structured data records;

validating the trained model by analyzing each of the second plurality of structured data records to generate the identification of each of the second plurality of structured data records and the second associated metadata record; and

generating the accuracy score by comparing the identification of each of the second plurality of structured data records to the second associated metadata record.

8. The method of any of claims 1 to 3, further comprising:

detecting that a threshold of the plurality of training data is reached; and

automatically training the machine learning model; and

generating the code for the trained model.

9. A computing device, comprising:

one or more memories; and

one or more processors in communication with the one or more memories and configured to execute instructions stored in the one or more memories to perform operations comprising:

receiving, via a user interface, a selection of a template of a plurality of templates that define a machine learning model, each template corresponding to a category of data types;

identifying a location of a plurality of training data, the plurality of training data including a first plurality of structured data records and first associated metadata records, each of the first associated metadata records identifying at least a classification label of the first plurality of structured data records;

training the machine learning model by analyzing each of the first plurality of structured data records and the first associated metadata record to generate a trained model;

identifying a location of a plurality of validation data, the plurality of validation data including a second plurality of structured data records and a second associated metadata record, each of the second associated metadata records identifying at least the category label of the second plurality of structured data records;

validating the trained model by analyzing each of the second plurality of structured data records to generate an identification of each of the second plurality of structured data records;

displaying the accuracy score by comparing the identification of each of the second plurality of structured data records with the second associated metadata record to generate an accuracy score; and

generating code for the trained model, the code comprising the trained model and executable on a mobile device to classify data obtained from a sensor of the mobile device.

10. The computing device of claim 9, further comprising instructions stored in the one or more memories for performing operations comprising:

analyzing the plurality of training data to determine a plurality of classifications, each classification corresponding to a different classification label; and

comparing the classification labels of the plurality of training data to determine consistency with the category of the data type.

11. The computing device of claim 9, further comprising instructions stored in the one or more memories for performing operations comprising:

analyzing the plurality of training data to determine a plurality of classifications, each classification corresponding to a different classification label;

displaying a list of the classifications of the plurality of training data on a graphical user interface; and

receiving a selection of one or more of the plurality of classifications for training the machine learning model.

12. The computing device of any of claims 10 to 11, further comprising instructions stored in the one or more memories for performing operations comprising:

displaying a list of the classifications of the plurality of training data on the user interface; and

receiving a selection of one or more of the plurality of classifications for training the machine learning model.

13. The computing device of claim 9, further comprising instructions stored in the one or more memories for performing operations comprising: receiving the identification of the locations of the plurality of training data by selecting an icon associated with the training data and dragging the icon onto a designated area on a graphical user interface.

14. The computing device of claim 9, wherein identifying the location of the plurality of validation data comprises automatically selecting a random portion of the training data, wherein the random portion of the training data is prevented from training the machine learning model.

15. The computing device of claim 9, further comprising instructions stored in the one or more memories for performing operations comprising:

receiving a selection that will automatically select the plurality of validation data from the training data;

blocking a preselected percentage of the training data from the training, the blocked training data comprising the second plurality of structured data records;

validating the trained model by analyzing each of the second plurality of structured data records to generate the identification of each of the second plurality of structured data records and the second associated metadata record; and

generating the accuracy score by comparing the identification of each of the second plurality of structured data records to the second associated metadata record.

16. The computing device of claim 9, further comprising instructions stored in the one or more memories for performing operations comprising:

detecting that a threshold of the plurality of training data is reached;

automatically training the machine learning model; and

generating the code for the trained model.

17. A non-transitory computer-readable medium storing a plurality of instructions that, when executed by one or more processors of a computing device, cause the one or more processors of the computing device to perform operations comprising:

receiving, via a user interface, a selection of a template of a plurality of templates that define a machine learning model, each template corresponding to a category of data types;

identifying a location of a plurality of training data, the plurality of training data including a first plurality of structured data records and first associated metadata records, each of the first associated metadata records identifying at least a classification label of the first plurality of structured data records;

training the machine learning model by analyzing each of the first plurality of structured data records and the first associated metadata record to generate a trained model;

identifying a location of a plurality of validation data, the plurality of validation data including a second plurality of structured data records and a second associated metadata record, each of the second associated metadata records identifying at least the category label of the second plurality of structured data records;

validating the trained model by analyzing each of the second plurality of structured data records to generate an identification of each of the second plurality of structured data records;

displaying the accuracy score by comparing the identification of each of the second plurality of structured data records with the second associated metadata record to generate an accuracy score; and

generating code for the trained model, the code comprising the trained model and executable on a mobile device to classify data obtained from a sensor of the mobile device.

18. The non-transitory computer-readable medium of claim 17, the further instructions, when executed by one or more processors of the computing device, cause the one or more processors of the computing device to perform operations comprising:

analyzing the plurality of training data to determine a plurality of classifications, each classification corresponding to a different classification label; and

comparing the classification labels of the plurality of training data to determine consistency with the category of the data type.

19. The non-transitory computer-readable medium of claim 17, the further instructions, when executed by one or more processors of the computing device, cause the one or more processors of the computing device to perform operations comprising:

analyzing the plurality of training data to determine a plurality of classifications, each classification corresponding to a different classification label;

displaying a list of the classifications of the plurality of training data on a graphical user interface; and

receiving a selection of one or more of the plurality of classifications for training the machine learning model.

20. The non-transitory computer-readable medium of any of claims 17 to 19, the further instructions, when executed by one or more processors of the computing device, cause the one or more processors of the computing device to perform operations comprising:

receiving a selection that will automatically select the plurality of validation data from the training data;

blocking a preselected percentage of the training data from the training, the blocked training data comprising the second plurality of structured data records;

validating the trained model by analyzing each of the second plurality of structured data records to generate the identification of each of the second plurality of structured data records and the second associated metadata record; and

generating the accuracy score by comparing the identification of each of the second plurality of structured data records to the second associated metadata record.

Images