owai.com

Discriminative models.

Discriminative models directly map inputs to outputs using conditional distribution or prediction function.

• Logistic regression/maximum entropy classifiers
• Linear discriminant analysis
• Support vector machines
• Boosting
• Conditional random fields
• Linear regression
• Neural networks

Generative models.

Generative model assume a common distribution that govern by parameters and find parameters values that best fit the training data.

Notes on statistical distance measures:

1. Euclidean distance:The straight line distance between two points. Summing the square root of the squared differences between each coordinate.
2. Cosine distance: Dividing the dot product of two vectors by the product of their lengths.
3. Manhattan distance: The distance between two points measured along axes at right angles. Summing the absolute values of the difference between each coordinate.
4. Hamming distance: The number of bits which differ between two binary strings.

I like this paper on machine learning , it is by L.G. Valiant

Abstract:
Humans appear to be able to learn new concepts without needing to be programmed explicitly in any conventional sense. In this paper we regard learning as the phenomenon of knowledge acquisition in the absence of explicit programming.We give a precise methodology for studying this phenomenon from a computational viewpoint. It consists of choosing an appropriate information gathering mechanism, the learning protocol, and exploring the class of concepts that can be learned using it in a reasonable (polynomial) number of steps. Although inherent algorithmic complexity appears to set serious limits to the range of concepts that can be learned, we show that there are some important nontrivial classes of propositional concepts that can be learned in a realistic sense.

A Theory of the Learnable (full paper in pdf)