Model Validation IN R

 MODEL VALIDATION

Description of Model Validation

Model validation is the process of evaluating a trained model on test data set. This provides the generalization ability of a trained model. 

There are many ways to get the training and test data sets for model validation like:

• 3-way holdout method of getting training, validation and test data sets.

• k-fold cross-validation with independent test data set.

• Leave-one-out cross-validation with independent test data set.

Role / Importance

In machine learning, model validation is referred to as the process where a trained model is evaluated with a testing data set. The testing data set is a separate portion of the same data set from which the training set is derived. The main purpose of using the testing data set is to test the generalization ability of a trained model.

Model validation is carried out after model training. Together with model training, model validation aims to find an optimal model with the best performance.

PROBLEM 1 : Iris Data Set

Model Validation

Source Code

library(caret)

# load the iris dataset

data(iris)

#Bootstrap

# define training control

train_control <- trainControl(method="boot", number=100)

# train the model

model <- train(Species~., data=iris, trControl=train_control, method="nb")

# summarize results

print(model)

Output

PROBLEM 2 : Diabetes Data Set

Source Code

library(caret)

diabet<-read.csv('C:/Semester 6/Data Science/diabetes.csv')

diabet$Outcome<-as.factor(diabet$Outcome)

# Bootstrap

# define training control

train_control <- trainControl(method="boot", number=100)

# train the model

model <- train(diabet$Outcome~diabet$BMI, data=diabet, trControl=train_control, method="nb")

# summarize results

print(model)

Output