Introduction to Machine Learning

UNIT III

Chapter: 8: Supervised Learning

Syllabus

Introduction to machine learning - Linear Regression Models: Least squares, single & multiple variables, Bayesian linear regression, gradient descent, Linear Classification Models: Discriminant function - Probabilistic discriminative model - Logistic regression, Probabilistic generative model - Naive Bayes, Maximum margin classifier - Support vector machine, Decision Tree, Random forests.

Introduction to Machine Learning

• Machine Learning (ML) is a sub-field of Artificial Intelligence (AI) which concerns with developing computational theories of learning and building learning machines.

• Learning is a phenomenon and process which has manifestations of various aspects. Learning process includes gaining of new symbolic knowledge and development of cognitive skills through instruction and practice. It is also discovery of new facts and theories through observation and experiment.

• Machine Learning Definition: A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.

• Machine learning is programming computers to optimize a performance criterion using example data or past experience. Application of machine learning methods to large databases is called data mining.

• It is very hard to write programs that solve problems like recognizing a human face. We do not know what program to write because we don't know how our brain does it. Instead of writing a program by hand, it is possible to collect lots of examples that specify the correct output for a given input.

• A machine learning algorithm then takes these examples and produces a program that does the job. The program produced by the learning algorithm may look very different from a typical hand-written program. It may contain millions of numbers. If we do it right, the program works for new cases as well as the ones we trained it on.

• Main goal of machine learning is to devise learning algorithms that do the learning automatically without human intervention or assistance. The machine learning paradigm can be viewed as "programming by example." Another goal is to develop computational models of human learning process and perform computer simulations.

• The goal of machine learning is to build computer systems that can adapt and learn from their experience.

• Algorithm is used to solve a problem on computer. An algorithm is a sequence of instruction. It should carry out to transform the input to output. For example, for addition of four numbers is carried out by giving four number as input to the algorithm and output is sum of all four numbers. For the same task, there may be various algorithms. It is interested to find the most efficient one, requiring the least number of instructions or memory or both.

• For some tasks, however, we do not have an algorithm.

How Machines Learn?

Machine learning typically follows three phases:

1. Training: A training set of examples of correct behavior is analyzed and some representation of the newly learnt knowledge is stored. This is some form of rules.

2. Validation: The rules are checked and, if necessary, additional training is given. Sometimes additional test data are used, but instead, a human expert may validate the rules, or some other automatic knowledge - based component may be used. The role of the tester is often called the opponent.

3. Application: The rules are used in responding to some new situation.

• Fig. 8.1.1 shows phases of ML.

Why Machine Learning is Important?

• Machine learning algorithms can figure out how to perform important tasks by generalizing from examples.

• Machine learning provides business insight and intelligence. Decision makers are provided with greater insights into their organizations. This adaptive technology is being used by global enterprises to gain a competitive edge.

• Machine learning algorithms discover the relationships between the variables of a system (input, output and hidden) from direct samples of the system.

• Following are some of the reasons:

1) Some tasks cannot be defined well, except by examples. For example: Recognizing people.

2) Relationships and correlations can be hidden within large amounts of data. To solve these problems, machine learning and data these relationships.

3) Human designers often produce machines that do not work as well as desired in the environments in which they are used.

4) The amount of knowledge available about certain tasks might be too large for explicit encoding by humans.

5) Environments change time to time.

6) New knowledge about tasks is constantly being discovered by humans.

• Machine learning also helps us find solutions of many problems in computer vision, speech recognition and robotics. Machine learning uses the theory of statistics in building mathematical models, because the core task is making inference from a sample.

• Learning is used when:

1. Human expertise does not exist (navigating on Mars),

2. Humans are unable to explain their expertise (speech recognition)

3. Solution changes in time (routing on a computer network)

4. Solution needs to be adapted to particular cases (user biometrics)

Ingredients of Machine Learning

The ingredients of machine learning are as follows:

1. Tasks: The problems that can be solved with machine learning. A task is an abstract representation of a problem. The standard methodology in machine learning is to learn one task at a time. Large problems are broken into small, reasonably independent sub-problems that are learned separately and then recombined.

• Predictive tasks perform inference on the current data in order to make predictions. Descriptive tasks characterize the general properties of the data in the database.

2. Models: The output of machine learning. Different models are geometric models, probabilistic models, logical models, grouping and grading.

• The model-based approach seeks to create a modified solution tailored to each new application. Instead of having to transform your problem to fit some standard algorithm, in model-based machine learning you design the algorithm precisely to fit your problem. 

• Model is just made up of set of assumptions, expressed in a precise mathematical form. These assumptions include the number and types of variables in the problem domain, which variables affect each other, and what the effect of changing one variable is on another variable.

• Machine learning models are classified as: Geometric model, Probabilistic model and Logical model.

3. Features: The workhorses of machine learning. A good feature representation is central to achieving high performance in any machine learning task.

• Feature extraction starts from an initial set of measured data and builds derived values intended to be informative, non redundant, facilitating the subsequent learning and generalization steps.

• Feature selection is a process that chooses a subset of features from the original features so that the feature space is optimally reduced according to a certain criterion.