We live in an internet of information and an internet of things.
The trifecta of childhood dreams, rockets, robots and radio has never seen a
brighter time. They are the three R's for the digital age.
Oz is giving something to the Tin Man, that he didn't already have - an artificial brain.
Adding to the seismic shift we find ourselves in is the explosion of machine learning (ML) techniques. There are two kinds of detonation taking place.
The first is the increase in the number of fundamental algorithms.
The second is in permutations or variants of those algorithms.
Consider the garden of fundamental algorithms:
1. Linear Regression (Supervised ML)
2. Logistic Regression (Supervised)
3. Decision Tree (Supervised)
4. Random Forest – Ensemble of Decision Trees (Supervised)
5. Support Vector Machines - SVM (Supervised)
6. Naive Bayes (Supervised)
7. Gradient Boosting Algorithms - XG Boost (Supervised)
8. Convolutional Neural Network – CNN (Supervised)
9. Recurrent Neural Network – RNN (Supervised)
10. K-Nearest Neighbors – kNN (Supervised)
11. K-Means (Unsupervised)
12. Dimensionality Reduction: Principal Components Analysis – PCA (Unsupervised)
13. Generative Adversarial Networks – GANs (Both)
14. Reinforcement Learning – RL (Neither)
15. Attention Mechanisms that Prioritize Machine Learning Operations (Self Supervised)
Consider the garden of a specific ML device, the autoencoder, we have:
1. Denoising Auto-encoders
2. Sparse Autoencoders
3. Deep Autoencoders
4. Contractive Autoencoders
5. Undercomplete Autoencoders
6. Convolutional Autoencoders
7. Variational Autoencoders
8. Recurrent Autoencoders
9. SeqToSeq Autoencoders
For each kind of ML algorithm and variant, we require the following representations to understand them:
1. The underlying equations
2. The block diagram showing how the parts fit together
3. An outline of the code that computes them
4. An animation of the progression of the algorithm
For algorithm selection and cost estimation would also like to know:
1. The time complexity
2. The space complexity
3. The loss function
4. The hyperparameters, learning rate, activation function, etc.
During the execution of an algorithm, it is useful to watch the loss function decrease, as this portrays the learning that is taking place.
Without these algorithm representations, selection and cost estimation we are just shooting in the dark. This especially so when we are comparing our work with that of others.
So that is my note on clarity.
References