Machine Learning
Articles, notes, and tutorials filed under Machine Learning.
Compile Caffe without Root Privileges
On shared servers or HPC clusters, you often don’t have sudo access, which makes installing system-level dependencies impossible. In this tutorial, we are going to introduce how to install Caffe entirely in user space, without root privileges. We assume that you have installed Anaconda and CUDA on your PC. Note: This tutorial is based on CUDA 9.0, Caffe 1.0, protobuf 3.2.0, and OpenCV 3.4.3. You may need to adjust the version numbers for your own setup.
Gradient Boosting Decision Tree
In the previous article, we’ve talked about AdaBoost which combines output of weak learners into a weighted sum that represents the final output of the boosted classifier. If you know little about AdaBoost or additive model, we highly recommend you read the article first. Gradient boosting is a machine learning technique for regression and classification problems, which produces a prediction model in the form of an ensemble of weak prediction models, typically decision trees. It builds the model in a stage-wise fashion like other boosting methods do, and it generalizes them by allowing optimization of an arbitrary differentiable loss function.
AdaBoost
AdaBoost, short for “Adaptive Boosting”, is a machine learning meta-algorithm formulated by Yoav Freund and Robert Schapire who won the Gödel Prize in 2003 for their work. The output of the other learning algorithms (weak learners) is combined into a weighted sum that represents the final output of the boosted classifier. AdaBoost is adaptive in the sense that subsequent weak learners are tweaked in favor of those instances misclassified by previous classifiers. AdaBoost is sensitive to noisy data and outliers and is quite robust to overfitting.
Random Forest
Random forests are an ensemble learning method for classification, regression, and other tasks. They operate by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes (classification) or the mean prediction (regression) of the individual trees. Each tree is grown on a random subset of the training data and considers only a random subset of the features at each split, so the forest as a whole corrects for a decision tree’s habit of overfitting its training set: when it is time to make a prediction, the trees vote and the majority wins.
Random Projection
Introduction In mathematics and statistics, random projection is a technique used to reduce the dimensionality of a set of points which lie in Euclidean space. Random projection methods are known for their simplicity and low error rates compared with other dimensionality-reduction techniques, and experiments show that they preserve pairwise distances well. Consider a problem as follows: We have a set of $n$ points in a high-dimensional Euclidean space $\mathbf{R}^d$. We want to project the points onto a space of low dimension $\mathbf{R}^k$ in such a way that pairwise distances of the points are approximately the same as before.
Decision Tree
What’s a decision tree? A decision tree is a flowchart-like structure in which each internal node represents a “test” on an attribute (e.g. whether Income is below 50K), each branch represents an outcome of that test, and each leaf node holds a prediction: a class label for classification, or a value for regression. A path from the root to a leaf is therefore a chain of conditions that ends in a decision.