How to implement artificial intelligence solutions

How to
implement
Artificial
Intelligence
solutions
ByCarlosToxtli

Index
● Overview
● Discriminative techniques
● Machine Learning
● Deep Learning
● Reinforcement Learning
● Generative techniques
● Conclusions

What is Artificial Intelligence?
Artificial intelligence (Artificial Intelligence, or AI) is the simulation of human
intelligence processes by machines, especially computer systems. These
processes include learning (the acquisition of information and rules for the
use of information), reasoning (using the rules to reach approximate or
definitive conclusions) and self-correction.

Predictions of the Mexican elections

World Cup Predictions
They are not usually very accurate since there
are many factors at play.

It is able to understand us
Although we give
very basic concepts,
AI can create more
complex concepts.

Create videos controlling a person
It has been possible to
create videos
supplanting someone's
identity.

Songs composed by AI
Song that is
composed from
the sound (hum) of
the human voice.

1958
The computer will be the first device to
think as the human brain. It can
understand its surroundings,
recognize you speak your name and
translate in real time.
It is possible to build brains that
could reproduce themselves on an
assembly and which would be
conscious of their existence.
Rosenblatt murio en un accidente

Geoffrey Hinton is the father of neural networks

Fun fact about George Boole
Geoffrey Hinton is a great-great-great-great
grandson of George Boole.
He changed the world from the mechanical
industrial era to the digital era.

AI is also in the interest of governments
Geoffrey Hinton
currently resides in
Canada and the
government is
investing in being a
reference in the AI
scene.

Getting started in Artificial
Intelligence

How do I start?
The first thing you should know is that you start either as an implementer of
existing techniques or as a creator of new techniques.

Creator of new techniques (Scientist)
In order to propose new algorithms, deep knowledge is required in several
areas, including:
● Linear algebra
● Statistics
● Equation systems
● Calculus
● Parallel computing
● Data structures
● Programming languages (usually C or Python)
… Among other areas

As an implementer you just need to know how each technique differs from
another and know which one to use depending on the problem. The process
of implementing only implies adapting the inputs to which each algorithm
requires, without the need to program a new algorithm. This requires:
● Data processing
● Programming languages (usually C or Python)
● Parallel computing
Among others
Implementer of existing techniques (Engineering)

This talk will focus on how to implement
While it is true that the great discoveries come from the generation of new
algorithms, reaching a sufficient level of understanding of the foundations of
artificial intelligence takes time and usually requires postgraduate studies.
As an implementer you will eventually be able to improve existing
techniques.

The evolution of the methods
A simple way to understand how to implement AI is compared to a
conventional programming method. A method is a piece of code that runs
and returns a result, example:
house_price = get_price( house_location )
The result can be searched in a database or calculated based on an existing
formula (existing rules).

The evolution of the methods
When we implement an AI technique, it is implemented in a manner similar
to a method, but there is no defined formula to obtain the result, and it
returns 2 values, a predicted value and the percentage of certainty.
precio_casa, certainty_percentage = predict_price( house_location )
The result is inferred based on previous data. There is no exact formula, it is
always estimated based on similar data.

To start a classic example is the Hot Dog and No Hot Dog. For that hundreds
of Hot Dogs photos and hundreds of photos of anything else are collected.
With that many of the techniques detect precisely if there is a Hot Dog. This is
what the function would look like
detected_class, certainty_percentage = is_a_hot_dog( image )
Where the detected_class is “Hot Dog”
And certainty_percentage ranges from 0 to 1
If the value is higher to a threshold (i.e. 0.5)
then It is a Hot Dog
The most basic example: Hot dog / Not Hot Dog

It is complicated
even for humans

Hot dog / Not Hot Dog use case
As in the SIlicon
Valley series

Detecting cats and dogs
In this case we need 3 data sets, hundreds of photos of dogs,
hundreds of cats and hundreds of anything else.
detected_classes, certainty_percentages = cat_or_dog( images )
Because they are more than one category, the function returns
a data array, where:
detected_classes[0] is “dog” y certainty_percentages[0] is 0.2
detected_classes[1] is “cat” y certainty_percentages[1] is 0.7
detected_classes[2] is “other” y certainty_percentages[2] is 0.1
Then it is more likely to be a cat.

Self-driving cars use the same principles
A car that drives itself, even if it sounds very complex, is just the
implementation of several AI functions, this is a very simple example:
detected_classes, certainty_percentages = vision_sensor_1( camera_image )
IF detected_classes[0] == ‘Another car’ and certainty_percentages[0] < .1
AND detected_classes[1] == ‘Pedestrian’ and certainty_percentages[1] < .1
Then SWITCH_LINE()

Autonomous car engineers use models
Dedicated to making autonomous cars is something very attainable for
anyone, since the vast majority of tests are done on models and not on real
cars. Once it is 100% efficient in models, tests are carried out on cars. An
autonomous car laboratory can be set up at a very low cost.

Classification and Regression
We have already learned these concepts
with examples of Supervised Learning
Regression predicts a value, as in the
case of the house, a house closer to a
shopping center and main roads, returns
a value in larger price, example 1,000,000
Classification predicts a category, in
the examples the "classes" or categories
to search are returned and based on that
it returns us if it is "dog" or "cat". We
know in advance the categories.

But there is an extra case (Clustering)
When we don't know the categories of the data (Unsupervised
Learning) we need the computer to group those that resemble
each other, then give them a name:
class_found, certainty_percentage = find_category( data[0] )
This case is called Clustering. An example may be to give pictures
of drawn letters, the result should be to group them into
separate letters.

Now you know the basics
With what has been learned, it is enough to raise a problem and solve it using
different types of learning, now we will see what techniques exist and when
each one is used.

Now it’s time to learn how to implement it

Machine Learning requires proper data
Machine Learning is the general concept of how a computer can learn.
Learning comes from receiving new information and adapting its parameters
to correctly identify new cases. This requires a lot of information, but you
also need to know what parameters ("features") to analyze.

Features?
Yes, we need to find based on
what parameters to
differentiate from one
another, for example there are
very similar species of fish.
In what characteristics would
you identify to identify them.
Color, size, number of fins?

The next step is to “annotate” (labeling)
Since we have each record of each specimen now manually we must specify
what "class" is, for example:
And then when unseen data are presented, we can predict.
Color Size Fins CLASS
Golden 20 5 Tilapia
Silver 10 6 Tuna
Silver 30 6 Salmon
Silver 20 6 ???

With the Classes and Features we can start
Ready now if we can feed any algorithm to get a result.
The most used are:
● kNN
● Bayes
● Support Vector Machines
● Neural Networks
● Decision Trees
● Random Forest
● ...
Among many others

Now we have to train the model
Once we have the Classes, the
Features and choose an algorithm. We
proceed to train it, the outcome is a
file called Model with the rules to
classify data not seen before.

But without good features, it won't be accurate
Example:
To predict
The features are not enough, we need to gather more data or features that
are more discriminative.
Golden 20 5 Tilapia
Silver 10 6 Tuna
Silver 30 6 Salmon
Silver 20 6 ???

Preparing the proper data is vital

Frameworks
● Sklearn
● Weka
● Matlab

Code
from sklearn.linear_model import LinearRegression
from sklearn.datasets import load_iris
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.2)
clf = LinearRegression()
classifier.fit(X_train, y_train)
y_pred = classifier.predict(X_test)
print('accuracy is',accuracy_score(y_pred, y_test))

When to use it?
● Simple regression problems
○ How much the rent should cost in certain area
○ How much should I charge for specific amount of work
● Problems where we want to define a rule that separates two
categories that are similar, i.e. Premium or Basic price for customers
under certain parameters (number of rooms vs number of cars)

Code
from sklearn.tree import DecisionTreeClassifier, export_graphviz
import pydotplus
iris = load_iris()
clf = DecisionTreeClassifier().fit(iris.data, iris.target)
dot_data = export_graphviz(clf, out_file=None, filled=True, rounded=True,
feature_names=iris.feature_names,
class_names=['Versicolor','Setosa','Virginica'])
graph = pydotplus.graph_from_dot_data(dot_data)

When to use it?
● When we need to know what decisions the machine is taking
● When we need to explain to others how the features are evaluated
● When there are no much features

Code
from sklearn.ensemble import RandomForestClassifier
iris = load_iris()
clf = RandomForestClassifier(n_estimators=100)

When to use it?
● When we want to know alternatives of how to evaluate a problem.
● When we want to manually discard flows that are biased
● When we want to manage ensembles from one single method.

Code
from sklearn.naive_bayes import GaussianNB
iris = load_iris()
clf = GaussianNB()

When to use it?
● When we want to know the probabilities of the different cases.
● When we need a probabilistic model.
● When we need an easy way to prove in paper.

Code
from sklearn.neighbors import NearestNeighbors
iris = load_iris()
clf = NearestNeighbors(n_neighbors=5)

When to use it?
● When intuition says that the problem can be solved from getting thee
most similar option.
● When the information is no exhaustive.
● When we want to justify the decision of the algorithm in a common
human reasoning.

Code
from sklearn.cluster import KMeans
from sklearn.datasets import load_digits
from sklearn.decomposition import PCA
from sklearn.preprocessing import scale
digits = load_digits()
data = scale(digits.data)
reduced_data = PCA(n_components=2).fit_transform(data)
print(reduced_data)
kmeans = KMeans(init='k-means++', n_clusters=10, n_init=10)
kmeans.fit(reduced_data)
kmeans.predict(reduced_data[:10])

When to use it?
● When we don’t know how to understand the data
● When we want to optimize resources by grouping related elements.
● When we want that the computer creates the labels for us.

Code
from sklearn.svm import SVC
iris = load_iris()
classifier = SVC()

When to use it?
● It was the most effective technique before Neural Networks, it can
achieve excellent results with less processing.
● Mathematically speaking, it is based in very strong math principles, it
creates complex multidimensional hyperplanes that separates the classes
precisely.
● It is not a white box technique, but may be the best option for problems
where we want to get the best of Machine Learning approach without
dealing with Neural Networks.

Logistic Regression
No regularization
L2 regularization

Code
from sklearn.linear_model import LogisticRegression
iris = load_iris()
clf = LogisticRegression(multi_class='multinomial')

When to use it?
● When we want to optimize a regression
● When we want to binarize the output
● As a preliminary analysis before implementing neural networks

Code
from sklearn.linear_model import Perceptron
iris = load_iris()
clf = Perceptron()

When to use it?
● When we have very few features and there is no extra details that can be
extracted from hidden layers.
● There are in fact neural networks, and we do not need alway to use
them for deep learning these can be used for machine learning when we
benchmark with other machine learning techniques.
● When we want to get the power of neural networks and we don’t have
much computational power.

Artificial Neural Networks = Multi-Layer Perceptron

Frameworks
● Tensorflow
● Keras
● Pythorch
● Sklearn

When to use it?
● Classifiers when common machine Learning Algorithms performs
poorly.
● Models with much features.
● Multiple classes projects.

Convolutional Neural Networks (CNN)

All that knowledge to train a CNN to do this ...

Frameworks
● Tensorflow
● Keras
● Pytorch
● Caffe

When to use it?
● When we want to process images
● When we want to process videos
● When we have highly dimensional data.

Recurrent Neural Networks (RNN)

RNN
These have memory that keeps
the context.

Frameworks
● Tensorflow
● Keras
● Pytorch

When to use it?
● When sequences are provided
○ Text sequences
○ Image sequences (videos)
○ Time series
● When we need to provide an ordered output

That’s all for Deep Learning techniques

When to use it?
● When we want to benchmark models
● When different models are stronger when these are evaluated together
● When the individual processing is not exhaustive

Frameworks
● TPOT
● MLBox
● H2O
● Google AutoML

When to use it?
● On every new model
● When we have enough time to train multiple models
● When we don’t know wich hyperparameters are better.

Last but not least - Reinforcement Learning

Frameworks
● OpenAI Gym
● Google Dopamine
● RLLib
● Keras-RL
● Tensorforce
● Facebook Horizon

When to use it?
● When a robot explores a place and needs to learn from the environment.
● When we can try as much as we can in a simulator.
● When we want to find the most optimal path

Techniques to improve the learning process

Principal Component Analysis (PCA)
Feature selection

When to use it?
● When we have too much features and we do not know which of them
are useful.
● When we want to reduce the dimensionality of our model.
● When we want to plot our decision boundaries.

When to use it?
● When we have limited data
● When we want to help our model to generalize more
● When our unseen data comes in very different formats.

That was too much information!

Discriminative: Predicts from Data

Generative: Generates from data distribution

Examples of Generative Artificial
The following images intermix the
styles of two domains

Generative models
● Autoencoders
● Adversarial Networks
● Sequence Models
● Transformers

When to use it?
● When we want to compress data.
● When we need to change one type of input to other type of output.
● When we don’t need much variability in the generated data.

Generative Adversarial Networks

When to use it?
● When we need to transfer a style
● When we need more variability in the generated output
● When we need to keep context in the generation.

When to use it?
● When we generate text
● When we generate the next sequence from a serie
● When the order in the generated output matters.

When to use it?
● When context is an essential part of the generated output
● When we need to keep consistency in the frequency space.
● When we have enough computational resources.

Summary
● Predict values from numeric features (Regression)
○ Linear regression
○ Polynomial regressions
○ Any machine learning technique without an activation function
● Predict categories from labeled numeric data (Classification)
○ Logistic Regression
○ k-Nearest Neighbors
○ Decision Tree
○ Random Forest
○ Support Vector Machine
○ Perceptron
○ Multi Layer Perceptron
○ Naive Bayes

Summary
● Find labels from unlabeled data (Clustering)
○ K-means
○ LDA
○ NMF
● Detect objects in an image (Classification)
○ Convolutional Neural Networks
● Detect the category of a text or video (Classification)
○ Recurrent Neural Networks
● Learn to navigate and find optimal paths (Try and error)
○ Reinforcement Learning
● Generate images
○ Autoencoders
○ Generative Adversarial Networks

Summary
● Generate audio and text
○ Sequence models
○ Transformers
● Explainable approaches
○ k-Nearest Neighbors
○ Naive Bayes
○ Decision Trees
○ Linear regression
○ K-means

Resources
● Working examples ready to run online (Jupyter)
○ http://bit.ly/awesome-machine-learning
● Communities around the world
○ http://theschool.ai

Conclusions
● Depending on the application that we want to develop we need different
techniques and frameworks
● The model training is the 20% of the work, processing the data to have
something useful is the remaining 80%.
● We need a lot of data to train a model, if you don’t have enough data,
there are always augmentation from transformations and generations
techniques to create more synthetic data form the existing.
● We don’t need to be a data scientist to implement Artificial Intelligence
solutions, we only need to know how the techniques work and how to
tune up.
● Now that you know everything you need, it's time to put it into practice.

Thanks
@ctoxtli
http://www.carlostoxtli.com
http://facebook.com/carlos.toxtli

How to implement artificial intelligence solutions

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à How to implement artificial intelligence solutions

Similaire à How to implement artificial intelligence solutions (20)

Plus de Carlos Toxtli

Plus de Carlos Toxtli (20)

Dernier

Dernier (20)

How to implement artificial intelligence solutions

Notes de l'éditeur