Machine Learning Algorithms

Unsupervised learning, class type clusterning

Methods to assign a set of objects into groups.

Objects in a cluster are more similar to each other than to those in other clusters. Enables understanding of the differences as well as the similarities within the data.

Prefers data that is in groupings given some form of distance (Euclidean, Manhattan, or others)

No restriction

Ward hierarchical clustering, k-means, Gaussian Mixture Models, spectral, Birch, Affinity propogation, fuzzy clustering

For a given K, finds K clusters by iteratively moving cluster centers to the cluster centers of gravity and adjusting the cluster set assignments.

Supervised learning, instance based

K-NN is an algorithm that can be used when you have a objects that have been classified or labeled and other similar objects that haven't been classified or labeled yet, and you want a way to automatically label them.

1: Simple; 2: Powerful; 3: Lazy, no training involved; 4: Naturally handles multiclass classification and regression

1: Performs poorly on high-dimensionality datasets; 2: Expensive and slow to predict new instances; 3: Must define a meaningful distance function;

Good for measuring distance based approximations, good for outlier detection

Low-dimensional datasets

1: Computer security: intrusion detection; 2: Fault detection in semiconducter manufacturing; 3: Video content retrieval; 4: Gene expression

Each node in the tree tests a single attribute, decisions are represented by the edges leading away from that node with leaf nodes representing the final decision.

1: Fast; 2: Robust to noise and missing values; 3: Accurate

1: Complex trees are hard to interpret; 2: Duplication within the same sub-tree is possible

1: Star classification; 2: Medical diagnosis; 3: Credit risk analysis

CART, ID3

Supervised or unsupervised with class: Markovian

Markov models are a kind of probabilistic model often used in language modeling. The observations are assumed to follow a Markov chain, where each observation is independent of all past observations given the previous one.

Markov chains are useful models of many natural processes and the basis of powerful techniques in probabilistic inference and randomized algorithms.

Generally works well for system information where the Markov assumption holds

Prefers time series data and memoryless information

Temporal pattern recognition such as speech, handwriting, gesture recognition, part-of-speech tagging, musical score following, and bioinformatics.

Markov chains, Hidden Markov Models

Supervised learning, regression class

Trying to fit a linear continuous function to the data to predict results.

Can be univariate or multivariate.

1: Very fast - runs in constant time, 2: Easy to understand the model, 3: Less prone to overfitting

1: Unable to model complex relationships, 2: Unable to capture nonlinear relationships without first transforming the inputs

1: Prefers continuous variables; 2: A first look at a dataset; 3: Numerical data with lots of features

Low restriction on problems it can solve

1: Fitting a line

Supervised learning; used for classification; probabalistic approach

Given its simplicity and the assumption that the independent variables are statistically independent, Naive Bayes models are effective classification tools that are easy to use and interpret. Naive Bayes is particularly appropriate when the dimensionality of the independent space is high. For the reasons given above, Naive Bayes can often outperform other more sophisticated classification methods. A variety of methods exist for modeling the conditional distributions of the inputs including normal, lognormal, gamma, and Poisson.

1: Easy to use and interpret; 2: Works well with high dimensional problems

Works on problems where the inputs are independent from each other

Prefers problems where the probability will always be greater than zero for each class

A variety of methods exist for modeling the conditional distributions of the inputs including normal, lognormal, gamma, and Poisson.

Supervised learning; nonlinear functional approximation

With experience, networks can learn, as feedback strengthens or inhibits connections that produce certain results. Each layer depends on the calculations done on the layer before it.

1: Extremely powerful, can model even very complex relationships; 2: No need to understand the underlying data; 3: Almost works by "magic"

1: Prone to overfitting; 2: Long training time; 3: Requires significant computing power for large datasets; 4: Model is essentially unreadable; 5: Work best with "homogenous" data where features all have similar meanings

Prefers binary inputs

Little restriction bias

1: Images; 2: Video; 3: "Human-intelligence" type tasks like driving or flying; 4: Robotics

Deep learning

Supervised learning for defining a decision boundary

Divides an instance space by finding the line that is as far as possible from both classes. This line is called the "maximum-margin hyperplane".

Only the points near the hyperplane are important. These points near the boundary are called the support vectors.

1: Can model complex, nonlinear relationships; 2: Robust to noise (because they maximize margins)

1: Need to select a good kernel function; 2: Model parameters are difficult to interpret; 3: Sometimes numerical stability problems; 4: Requires significant memory and processing power

Works where there is a definite distinction between two classifications

Prefers binary classification problems

1: Text classification; 2: Image classification; 3: Handwriting recognition