RECURSIVE PARTITIONING AND REGRESSION TREES (RPART) - Detailed Example on Classification in R

R Code - Bank Subscription Marketing - Classification {RECURSIVE PARTITIONING AND REGRESSION TREES}

 R Code  for Recursive Partitioning and Regression Trees (RPART)
Data Set:- Bank Marketing https://www.blogger.com/blogger.g?blogID=4359514443959080595#editor/target=post;postID=5426750358330739357
Source of Data Set:-  UCI Repository (http://archive.ics.uci.edu/ml/datasets/Bank+Marketing)
The Code includes the following:-
1. Data Exploration - Missing Values, Outliers
2. Data Visualisation
3.  Correlation Matrix
4. Data Partitioning
5. Package used "rpart"
6. Model Tuning and Plots
7. Confusion Matrix


## The data has been imported using Import Dataset option in R Environment
## The data set can be obtained from http://archive.ics.uci.edu/ml/datasets/Bank+Marketing
## DATASET UNDERSTANDING
head(bank_full) ## Displays first 6 rows for each variable
str(bank_full) ## Describes each variables
summary(bank_full) ## Provides basic statistical information of each variable
## DATA EXPLORATION - Check for Missing Data
## Option 1
is.na(bank_full) ## Displays True for a missing value
## Since it is a large dataset, graphical display of missing values will prove to be easier
##Option 2
require(Amelia)
missmap(bank_full,main="Missing Data - Bank Subscription", col=c("red","grey"),legend=FALSE)
## No red colour stripes are visible. hence no missing values.
##Option 3
summary(bank_full) ## displays missing values if any under every variable

## DATA VISUALISATION
## Use Box plots (Only for continuous variables)- To Check Ouliers
boxplot(bank_full$age~bank_full$subscribed, main=" AGE",ylab="age of customers",xlab="Subscribed")
boxplot(bank_full$balance~bank_full$subscribed, main=" BALANCE",ylab="Balance of customers",xlab="Subscribed")
boxplot(bank_full$lastday~bank_full$subscribed, main=" LAST DAY",ylab="Last day of contact",xlab="Subscribed")
boxplot(bank_full$lastduration~bank_full$subscribed, main="LAST DURATION",ylab="Last duration of contact",xlab="Subscribed")
boxplot(bank_full$numcontacts~bank_full$subscribed, main="NUM CONTACTS",ylab="number of contacts",xlab="Subscribed")
boxplot(bank_full$pdays~bank_full$subscribed, main=" Previous DAYS",ylab="Previous days of contact",xlab="Subscribed")
boxplot(bank_full$pcontacts~bank_full$subscribed, main=" Previous Contacts",ylab="Previous Contacts with customers",xlab="Subscribed")
## Though some outliers are observed in Previous contacts, NumContacts and LastDuration, they have not bee removed keeping their significance into consideration
## Use Histograms (For both continuous and categorical variables)
## These histograms provide details abpout Skewness, Normal Distribution etc
## Function to create histograms for continuous variables with normal curve
bank_Conthist<-function(VarName,NumBreaks,xlab,main,lengthxfit) ## xlab and main should be mentioned under quotes as they are characters
{
  hist(VarName,breaks=NumBreaks,col="yellow",xlab=xlab,main=main)
  xfit<-seq(min(VarName),max(VarName),length=lengthxfit)
  yfit<-dnorm(xfit,mean=mean(VarName),sd=sd(VarName))
  yfit<-yfit*diff(h$mids[1:2])*length(VarName)
  lines(xfit,yfit,col="red",lwd=3)
}
bank_Conthist(bank_full$age,10,"age of customers","AGE",30)
bank_Conthist(bank_full$balance,50,"Balance of customers","Balance",100)
## Balance is more skewed towards to Negative or Zero
bank_Conthist(bank_full$lastday,5,"Last Day of contact","LAst Day",10)
bank_Conthist(bank_full$lastduration,100,"LastDuration of COntact","Last Duration",10)
## Last Duration is more skewed towards 0 to 100 secs.
bank_Conthist(bank_full$numcontacts,30,"Number of Contacts","NUmContacts",20)
## NUmContacts are more skewed towards 1
bank_Conthist(bank_full$pdays,30,"Previous Days of contacts","PDays",20)
## Many were not contacted previously
bank_Conthist(bank_full$pcontacts,20,"Previous Contacts","PContacts",10)
## Since many were not contacted previously, therefore Pcontacts is 0

## Barplots for Categorical Variables
barplot(table(bank_full$job),col="red",main="JOB")
barplot(table(bank_full$marital),col="green",main="Marital")
barplot(table(bank_full$education),col="red",main="Education")
barplot(table(bank_full$creditdefault),col="red",main="Credit Default")
## Since Credit Default is highly skewed towards NO, this shall be removed from further analysis
bank_full[5]<-NULL
str(bank_full)
barplot(table(bank_full$housingloan),col="red",main="Housing Loan")
barplot(table(bank_full$personalloan),col="blue",main="Personal Loan")
barplot(table(bank_full$lastcommtype),col="red",main="Last communication type")
barplot(table(bank_full$lastmonth),col="violet",main="Last Month")
barplot(table(bank_full$poutcome),col="magenta",main="Previous Outcome")
## Correlation Matrix among input (or independent) continuous variables
bank_full.cont<-data.frame(bank_full$age,bank_full$balance,bank_full$lastday,bank_full$lastduration,bank_full$numcontacts,bank_full$pdays,bank_full$pcontacts)
str(bank_full.cont)
cor(bank_full.cont)
## It can be observed that No two variables are highly correlated

 ## Partitioning Data into Train and Test datasets in 70:30
library(caret)
set.seed(1234567)
train2<-createDataPartition(bank_full$subscribed,p=0.7,list=FALSE)
train<-bank_full[train2,]
test<-bank_full[-train2,]
write.table(bank_full,"bank1.csv",sep=",")
str(train)
str(test)

## CLASSIFICATION USING Recursive partitioning and regressive trees(RPART) Model

 
install.packages("rpart")
library(rpart)
## Various parameters that control resursive partioning and regressive trees
control<-rpart.control(minsplit=30,minbucket=10,cp=0.01,maxcomplete=6,maxsurrogate=8,usersurrogate=2,xval=15,surrogatestyle=0,maxdepth=15)
## generally minimum no. of observations in any terminal leaf node(minbucket)=minimum no. of observations that must exist in a node in order to split (minsplit)/3
##Training the model
train.rpart<-rpart(subscribed~.,data=train,control=control)
##Prints the rpart object
print(train.rpart)
##Summarizes the model
summary(train.rpart,digits=getOption("digits"),cp=0,save="train.rpart")
## MEan Variance Plot- Graphical representation of deviance of a node divided by the number of observations at the node
meanvar(train.rpart)  ## Drawn for Regression plots..Not significant for Classification plots
##Shows paths to the selected nodes- It returns the names list where each element contains the splits on the path from the root to the selected node
path.rpart(train.rpart,nodes=c(10,11,12,13),pretty=0,print.it=TRUE)
##Plot the rpart object
plot(train.rpart,uniform=FALSE,branch=1,compress=TRUE,margin=0)
##Label the plot drawn above (TREE Dendogram)
text(train.rpart,splits=TRUE,use.n=TRUE)
##PostScript Presentation Plot
post(train.rpart,file="",title="Bank Data-RPART",use.n=TRUE)
##Snip-Contains the nodes that remain after selected subtrees are snipped off
train.rpart.snip<-snip.rpart(train.rpart,toss=5)  ## trim subtree at node=5
plot(train.rpart.snip)
text(train.rpart.snip,splits=TRUE,use.n=TRUE)
post(train.rpart.snip,file="",title="Bank Data-RPART",use.n=TRUE)
## R squared plots for different splits
rsq.rpart(train.rpart)
## Prediction on TEST data set using Trained model
##Prediction with Class Probabilities
test.rpart.prob<-predict(train.rpart,test,type="prob")
test.rpart.prob
## Level Numbers- Predicted
test.rpart.vector<-predict(train.rpart,test,type="vector")
test.rpart.vector
## Factors - Predicted
test.rpart.factor<-predict(train.rpart,test,type="class")
test.rpart.factor
## Matrix of Level numbers, class frequencies, and probabilities
test.rpart.matrix<-predict(train.rpart,test,type="matrix")
test.rpart.matrix
## Misclassification table or Confusion Matrix
table(test.rpart.vector,test$subscribed)
## Residuals from a fitted rpart
residuals(train.rpart,type=c("usual","pearson","deviance"))  ## This has more significance in Regression type dataset
##Predicted values after cross vaildation for a set of complexity parameter values
##cp is desired list of complexity values. By default takes values from cptable component fit
##Displays cptable
printcp(train.rpart)
train.rpart.cv<-xpred.rpart(train.rpart,xval=15,cp=4)
## Find MSE (Mean squared Error) with cross-validated model- Cross Validated Error estimate
## More valid for regression analysis
train.error<-(train.rpart.cv-train$subscribed)^2
apply(train.error,2,sum)
## Visual representation of cross validation results-Plots a complexity parameter table
plotcp(train.rpart,minline=TRUE,lty=3,col="red",upper=c("size","splits","none"))
##Cost complexity pruning-Determines nested sequence of subtrees by recursively snipping off least important splits
prune(train.rpart,cp=4)

RPART TREE
 

VISUAL REPRESENTATION OF CROSS VALIDATION RESULTS

SUPPORT VECTOR MACHINE (SVM) - Detailed Example on Classification in R

R Code - Bank Subscription Marketing - Classification {SUPPORT VECTOR MACHINE}

 R Code  for Support Vector Machine (SVM)
Data Set:- Bank Marketing https://www.blogger.com/blogger.g?blogID=4359514443959080595#editor/target=post;postID=5426750358330739357
Source of Data Set:-  UCI Repository (http://archive.ics.uci.edu/ml/datasets/Bank+Marketing)
The Code includes the following:-
1. Data Exploration - Missing Values, Outliers
2. Data Visualisation
3.  Correlation Matrix
4. Data Partitioning
5. Package used "e1071"
6. Model Tuning and Plots
7. Confusion Matrix


## The data has been imported using Import Dataset option in R Environment
## The data set can be obtained from http://archive.ics.uci.edu/ml/datasets/Bank+Marketing
## DATASET UNDERSTANDING
head(bank_full) ## Displays first 6 rows for each variable
str(bank_full) ## Describes each variables
summary(bank_full) ## Provides basic statistical information of each variable
## DATA EXPLORATION - Check for Missing Data
## Option 1
is.na(bank_full) ## Displays True for a missing value
## Since it is a large dataset, graphical display of missing values will prove to be easier
##Option 2
require(Amelia)
missmap(bank_full,main="Missing Data - Bank Subscription", col=c("red","grey"),legend=FALSE)
## No red colour stripes are visible. hence no missing values.
##Option 3
summary(bank_full) ## displays missing values if any under every variable

## DATA VISUALISATION
## Use Box plots (Only for continuous variables)- To Check Ouliers
boxplot(bank_full$age~bank_full$subscribed, main=" AGE",ylab="age of customers",xlab="Subscribed")
boxplot(bank_full$balance~bank_full$subscribed, main=" BALANCE",ylab="Balance of customers",xlab="Subscribed")
boxplot(bank_full$lastday~bank_full$subscribed, main=" LAST DAY",ylab="Last day of contact",xlab="Subscribed")
boxplot(bank_full$lastduration~bank_full$subscribed, main="LAST DURATION",ylab="Last duration of contact",xlab="Subscribed")
boxplot(bank_full$numcontacts~bank_full$subscribed, main="NUM CONTACTS",ylab="number of contacts",xlab="Subscribed")
boxplot(bank_full$pdays~bank_full$subscribed, main=" Previous DAYS",ylab="Previous days of contact",xlab="Subscribed")
boxplot(bank_full$pcontacts~bank_full$subscribed, main=" Previous Contacts",ylab="Previous Contacts with customers",xlab="Subscribed")
## Though some outliers are observed in Previous contacts, NumContacts and LastDuration, they have not bee removed keeping their significance into consideration
## Use Histograms (For both continuous and categorical variables)
## These histograms provide details abpout Skewness, Normal Distribution etc
## Function to create histograms for continuous variables with normal curve
bank_Conthist<-function(VarName,NumBreaks,xlab,main,lengthxfit) ## xlab and main should be mentioned under quotes as they are characters
{
  hist(VarName,breaks=NumBreaks,col="yellow",xlab=xlab,main=main)
  xfit<-seq(min(VarName),max(VarName),length=lengthxfit)
  yfit<-dnorm(xfit,mean=mean(VarName),sd=sd(VarName))
  yfit<-yfit*diff(h$mids[1:2])*length(VarName)
  lines(xfit,yfit,col="red",lwd=3)
}
bank_Conthist(bank_full$age,10,"age of customers","AGE",30)
bank_Conthist(bank_full$balance,50,"Balance of customers","Balance",100)
## Balance is more skewed towards to Negative or Zero
bank_Conthist(bank_full$lastday,5,"Last Day of contact","LAst Day",10)
bank_Conthist(bank_full$lastduration,100,"LastDuration of COntact","Last Duration",10)
## Last Duration is more skewed towards 0 to 100 secs.
bank_Conthist(bank_full$numcontacts,30,"Number of Contacts","NUmContacts",20)
## NUmContacts are more skewed towards 1
bank_Conthist(bank_full$pdays,30,"Previous Days of contacts","PDays",20)
## Many were not contacted previously
bank_Conthist(bank_full$pcontacts,20,"Previous Contacts","PContacts",10)
## Since many were not contacted previously, therefore Pcontacts is 0

## Barplots for Categorical Variables
barplot(table(bank_full$job),col="red",main="JOB")
barplot(table(bank_full$marital),col="green",main="Marital")
barplot(table(bank_full$education),col="red",main="Education")
barplot(table(bank_full$creditdefault),col="red",main="Credit Default")
## Since Credit Default is highly skewed towards NO, this shall be removed from further analysis
bank_full[5]<-NULL
str(bank_full)
barplot(table(bank_full$housingloan),col="red",main="Housing Loan")
barplot(table(bank_full$personalloan),col="blue",main="Personal Loan")
barplot(table(bank_full$lastcommtype),col="red",main="Last communication type")
barplot(table(bank_full$lastmonth),col="violet",main="Last Month")
barplot(table(bank_full$poutcome),col="magenta",main="Previous Outcome")
## Correlation Matrix among input (or independent) continuous variables
bank_full.cont<-data.frame(bank_full$age,bank_full$balance,bank_full$lastday,bank_full$lastduration,bank_full$numcontacts,bank_full$pdays,bank_full$pcontacts)
str(bank_full.cont)
cor(bank_full.cont)
## It can be observed that No two variables are highly correlated

 ## Partitioning Data into Train and Test datasets in 70:30
library(caret)
set.seed(1234567)
train2<-createDataPartition(bank_full$subscribed,p=0.7,list=FALSE)
train<-bank_full[train2,]
test<-bank_full[-train2,]
write.table(bank_full,"bank1.csv",sep=",")
str(train)
str(test)


## CLASSIFICATION USING SUPPORT VECTOR MACHINE(SVM) Model
 

install.packages("e1071")
library(e1071)
## TUNING to perform cross-validation to know about optimum cost parameter for SVM modelling. Tune() performs only ten-fold cross validation here.
bank_full.tune<-tune(svm,subscribed~.,data=bank_full,kernel="linear",ranges=list(cost=c(0.01,0.1,1,5,10,100)))
## Diffferent kernels such as Polynomial, Radial Basis and Sigmoid can also be used for tuning.
summary(bank_full.tune)
bestmodal.tune<-bank_full.tune$best.model
summary(bestmodal.tune)
##We get cost=0.01 as optimum one with kernel = linear.
##In the following example, Polynomial function is being used
train.svm<-svm(subscribed~.,train,kernel="polynomial",cost=0.01,scale=TRUE,degree=3,gamma=1)
summary(train.svm)
plot(train.svm,train)  ## Not possible for large data-sets
## Testing data using the trained model
test.svm<-predict(train.svm,test)
## Misclassification Table
table(predict=test.svm,truth=test$subscribed)

RANDOM FORESTS-Detailed Solved Classification example in R (Includes Dependence and error Plots, MDS, Nodes and confusion matrices etc)

 R Code  for RANDOM FOREST
Data Set:- Bank Marketing 
Source of Data Set:-  UCI Repository (http://archive.ics.uci.edu/ml/datasets/Bank+Marketing)
The Code includes the following:-
1. Data Exploration - Missing Values, Outliers
2. Data Visualisation
3.  Correlation Matrix
4. Data Partitioning
5. RANDOM FOREST - Detailed
6. R Package- randomForest



## The data has been imported using Import Dataset option in R Environment
## The data set can be obtained from http://archive.ics.uci.edu/ml/datasets/Bank+Marketing
## DATASET UNDERSTANDING
head(bank_full) ## Displays first 6 rows for each variable
str(bank_full) ## Describes each variables
summary(bank_full) ## Provides basic statistical information of each variable
## DATA EXPLORATION - Check for Missing Data
## Option 1
is.na(bank_full) ## Displays True for a missing value
## Since it is a large dataset, graphical display of missing values will prove to be easier
##Option 2
require(Amelia)
missmap(bank_full,main="Missing Data - Bank Subscription", col=c("red","grey"),legend=FALSE)
## No red colour stripes are visible. hence no missing values.
##Option 3
summary(bank_full) ## displays missing values if any under every variable

## DATA VISUALISATION
## Use Box plots (Only for continuous variables)- To Check Ouliers
boxplot(bank_full$age~bank_full$subscribed, main=" AGE",ylab="age of customers",xlab="Subscribed")
boxplot(bank_full$balance~bank_full$subscribed, main=" BALANCE",ylab="Balance of customers",xlab="Subscribed")
boxplot(bank_full$lastday~bank_full$subscribed, main=" LAST DAY",ylab="Last day of contact",xlab="Subscribed")
boxplot(bank_full$lastduration~bank_full$subscribed, main="LAST DURATION",ylab="Last duration of contact",xlab="Subscribed")
boxplot(bank_full$numcontacts~bank_full$subscribed, main="NUM CONTACTS",ylab="number of contacts",xlab="Subscribed")
boxplot(bank_full$pdays~bank_full$subscribed, main=" Previous DAYS",ylab="Previous days of contact",xlab="Subscribed")
boxplot(bank_full$pcontacts~bank_full$subscribed, main=" Previous Contacts",ylab="Previous Contacts with customers",xlab="Subscribed")
## Though some outliers are observed in Previous contacts, NumContacts and LastDuration, they have not bee removed keeping their significance into consideration
## Use Histograms (For both continuous and categorical variables)
## These histograms provide details abpout Skewness, Normal Distribution etc
## Function to create histograms for continuous variables with normal curve
bank_Conthist<-function(VarName,NumBreaks,xlab,main,lengthxfit) ## xlab and main should be mentioned under quotes as they are characters
{
  hist(VarName,breaks=NumBreaks,col="yellow",xlab=xlab,main=main)
  xfit<-seq(min(VarName),max(VarName),length=lengthxfit)
  yfit<-dnorm(xfit,mean=mean(VarName),sd=sd(VarName))
  yfit<-yfit*diff(h$mids[1:2])*length(VarName)
  lines(xfit,yfit,col="red",lwd=3)
}
bank_Conthist(bank_full$age,10,"age of customers","AGE",30)
bank_Conthist(bank_full$balance,50,"Balance of customers","Balance",100)
## Balance is more skewed towards to Negative or Zero
bank_Conthist(bank_full$lastday,5,"Last Day of contact","LAst Day",10)
bank_Conthist(bank_full$lastduration,100,"LastDuration of COntact","Last Duration",10)
## Last Duration is more skewed towards 0 to 100 secs.
bank_Conthist(bank_full$numcontacts,30,"Number of Contacts","NUmContacts",20)
## NUmContacts are more skewed towards 1
bank_Conthist(bank_full$pdays,30,"Previous Days of contacts","PDays",20)
## Many were not contacted previously
bank_Conthist(bank_full$pcontacts,20,"Previous Contacts","PContacts",10)
## Since many were not contacted previously, therefore Pcontacts is 0

## Barplots for Categorical Variables
barplot(table(bank_full$job),col="red",main="JOB")
barplot(table(bank_full$marital),col="green",main="Marital")
barplot(table(bank_full$education),col="red",main="Education")
barplot(table(bank_full$creditdefault),col="red",main="Credit Default")
## Since Credit Default is highly skewed towards NO, this shall be removed from further analysis
bank_full[5]<-NULL
str(bank_full)
barplot(table(bank_full$housingloan),col="red",main="Housing Loan")
barplot(table(bank_full$personalloan),col="blue",main="Personal Loan")
barplot(table(bank_full$lastcommtype),col="red",main="Last communication type")
barplot(table(bank_full$lastmonth),col="violet",main="Last Month")
barplot(table(bank_full$poutcome),col="magenta",main="Previous Outcome")
## Correlation Matrix among input (or independent) continuous variables
bank_full.cont<-data.frame(bank_full$age,bank_full$balance,bank_full$lastday,bank_full$lastduration,bank_full$numcontacts,bank_full$pdays,bank_full$pcontacts)
str(bank_full.cont)
cor(bank_full.cont)
## It can be observed that No two variables are highly correlated

## Partitioning Data into Train and Test datasets in 70:30
library(caret)
set.seed(1234567)
train1<-createDataPartition(bank_full$subscribed,p=0.7,list=FALSE)
train<-bank_full[train1,]
test<-bank_full[-train1,]
## Without Any Transformations on the Variables, Random Forest Model has been Applied.
 ## RANDOM FOREST FOR CLASSIFICATION
library(randomForest)
## Tune Random Forest to obtain optimal mtry (No. of independent or predictor variables to be sampled at each split) parameter
tuneRF(train[,1:15],train$subscribed,stepFactor=2,improve=0.01,plot=TRUE,doBest=FALSE)
## optimal mtry can be used while training the model, but this has been excluded for this dataset
## Train data set for training the random forest model-- Ensure that your systme is equiped with higher memory space to run random forests
train.RandomForest=randomForest(subscribed~.,data=train,ntree=500,proximity=TRUE,importance=TRUE)
## Independent Variable Importance
importance(train.RandomForest,type=1)
## Independent Variable Importance Plot
varImpPlot(train.RandomForest,sort=TRUE,type=1)
## Independent or Predictor Variables actually used in the Random Forest model
varUsed(train.RandomForest,by.tree=FALSE,count=TRUE)
##size of tree in Random Forest
a<-treesize(train.RandomForest,terminal=TRUE)
hist(a)  ## Histogram to see the importance of each variable
## To compute oulying measures or outliers based on proximity matrix
b=outlier(train.RandomForest)
## graphical plot of outlier
plot(b,type='h',col=c("red","blue")[as.numeric(train$subscribed)])
## MultiDimensional scaling plot for trained model
MDSplot(train.RandomForest,train$subscribed,k=2,palette=rep(1,2),pch=as.numeric(train$subscribed))
## Partial Dependence Plot to observe the marginal effect of a variable on the classification class probabilities
## Lets observe the partial dependence of age on YES category class
partialPlot(train.RandomForest,train,age,"yes")
## Predict on the TEST data using the trained Random Forest Model
test.RandomForest=predict(train.RandomForest,test,predict.all=TRUE,proximity=TRUE,nodes=TRUE)
## Table showing the actual subscribed as per data and predicted subscribed by the trained model
table(observed=test$subscribed,predicted=test.RandomForest)
##Nodes Matrix
str(attr(test.RandomForest,"nodes"))
## confusion Matrix
table(test.RandomForest,test$subscribed)
## Plot margin of predictions on test dataset--- Here, positive margin means correct classification
margin(test.RandomForest)
plot(margin(test.RandomForest,sort=TRUE))
## Plot error rates or MSE(mean square error) for test dataset
plot(test.RandomForest,type=1)

LOGISTIC REGRESSION and C5.0 DECISION TREE Detailed solved example in Classification -R Code - Bank Subscription Marketing

 R Code  for LOGISTIC REGRESSION and C5.0 DECISION TREE
Data Set:- Bank Marketing 
Source of Data Set:-  UCI Repository (http://archive.ics.uci.edu/ml/datasets/Bank+Marketing)
The Code includes the following:-
1. Data Exploration - Missing Values, Outliers
2. Data Visualisation
3.  Correlation Matrix
4. Data Partitioning
5. Logistic Regression
6. C5.0 Decision Tree
6. Confusion Matrix with ROC



## The data has been imported using Import Dataset option in R Environment
## The data set can be obtained from http://archive.ics.uci.edu/ml/datasets/Bank+Marketing
## DATASET UNDERSTANDING
head(bank_full) ## Displays first 6 rows for each variable
str(bank_full) ## Describes each variables
summary(bank_full) ## Provides basic statistical information of each variable
## DATA EXPLORATION - Check for Missing Data
## Option 1
is.na(bank_full) ## Displays True for a missing value
## Since it is a large dataset, graphical display of missing values will prove to be easier
##Option 2
require(Amelia)
missmap(bank_full,main="Missing Data - Bank Subscription", col=c("red","grey"),legend=FALSE)
## No red colour stripes are visible. hence no missing values.
##Option 3
summary(bank_full) ## displays missing values if any under every variable

## DATA VISUALISATION
## Use Box plots (Only for continuous variables)- To Check Ouliers
boxplot(bank_full$age~bank_full$subscribed, main=" AGE",ylab="age of customers",xlab="Subscribed")
boxplot(bank_full$balance~bank_full$subscribed, main=" BALANCE",ylab="Balance of customers",xlab="Subscribed")
boxplot(bank_full$lastday~bank_full$subscribed, main=" LAST DAY",ylab="Last day of contact",xlab="Subscribed")
boxplot(bank_full$lastduration~bank_full$subscribed, main="LAST DURATION",ylab="Last duration of contact",xlab="Subscribed")
boxplot(bank_full$numcontacts~bank_full$subscribed, main="NUM CONTACTS",ylab="number of contacts",xlab="Subscribed")
boxplot(bank_full$pdays~bank_full$subscribed, main=" Previous DAYS",ylab="Previous days of contact",xlab="Subscribed")
boxplot(bank_full$pcontacts~bank_full$subscribed, main=" Previous Contacts",ylab="Previous Contacts with customers",xlab="Subscribed")
## Though some outliers are observed in Previous contacts, NumContacts and LastDuration, they have not bee removed keeping their significance into consideration
## Use Histograms (For both continuous and categorical variables)
## These histograms provide details abpout Skewness, Normal Distribution etc
## Function to create histograms for continuous variables with normal curve
bank_Conthist<-function(VarName,NumBreaks,xlab,main,lengthxfit) ## xlab and main should be mentioned under quotes as they are characters
{
  hist(VarName,breaks=NumBreaks,col="yellow",xlab=xlab,main=main)
  xfit<-seq(min(VarName),max(VarName),length=lengthxfit)
  yfit<-dnorm(xfit,mean=mean(VarName),sd=sd(VarName))
  yfit<-yfit*diff(h$mids[1:2])*length(VarName)
  lines(xfit,yfit,col="red",lwd=3)
}
bank_Conthist(bank_full$age,10,"age of customers","AGE",30)
bank_Conthist(bank_full$balance,50,"Balance of customers","Balance",100)
## Balance is more skewed towards to Negative or Zero
bank_Conthist(bank_full$lastday,5,"Last Day of contact","LAst Day",10)
bank_Conthist(bank_full$lastduration,100,"LastDuration of COntact","Last Duration",10)
## Last Duration is more skewed towards 0 to 100 secs.
bank_Conthist(bank_full$numcontacts,30,"Number of Contacts","NUmContacts",20)
## NUmContacts are more skewed towards 1
bank_Conthist(bank_full$pdays,30,"Previous Days of contacts","PDays",20)
## Many were not contacted previously
bank_Conthist(bank_full$pcontacts,20,"Previous Contacts","PContacts",10)
## Since many were not contacted previously, therefore Pcontacts is 0

## Barplots for Categorical Variables
barplot(table(bank_full$job),col="red",main="JOB")
barplot(table(bank_full$marital),col="green",main="Marital")
barplot(table(bank_full$education),col="red",main="Education")
barplot(table(bank_full$creditdefault),col="red",main="Credit Default")
## Since Credit Default is highly skewed towards NO, this shall be removed from further analysis
bank_full[5]<-NULL
str(bank_full)
barplot(table(bank_full$housingloan),col="red",main="Housing Loan")
barplot(table(bank_full$personalloan),col="blue",main="Personal Loan")
barplot(table(bank_full$lastcommtype),col="red",main="Last communication type")
barplot(table(bank_full$lastmonth),col="violet",main="Last Month")
barplot(table(bank_full$poutcome),col="magenta",main="Previous Outcome")
## Correlation Matrix among input (or independent) continuous variables
bank_full.cont<-data.frame(bank_full$age,bank_full$balance,bank_full$lastday,bank_full$lastduration,bank_full$numcontacts,bank_full$pdays,bank_full$pcontacts)
str(bank_full.cont)
cor(bank_full.cont)
## It can be observed that No two variables are highly correlated

## Partitioning Data into Train and Test datasets in 70:30
library(caret)
set.seed(1234567)
train1<-createDataPartition(bank_full$subscribed,p=0.7,list=FALSE)
train<-bank_full[train1,]
test<-bank_full[-train1,]
## Without Any Transformations on the Variables, following are the Classification Models Applied.
## LOGISTIC REGRESSION
train.logistic=glm(subscribed~age+job+marital+education+housingloan+balance+personalloan+lastcommtype+lastday+lastmonth+lastduration+numcontacts+pdays+pcontacts+poutcome,binomial(link="logit"),train)
summary(train.logistic)    ## Display the Results. One can remove the variables which are not significant (if P value>0.05) and re-run the model to make it more effective in terms of accurate prediction.
##Deviance was reduced to 22845-18828= 4017 in 40 degrees of freedom (31648-316408)
## Significance Test
1-pchisq(4017,df=40)
## Since its valus is 0, the NUll hypothesis is rejected and the model developed is significant
##Model Diagnostic Plots with train data set
plot(train.logistic)
##Using model for testing the TEST data set
##Initial Probabilities are generated
test.prob<-predict.glm(train.logistic,test[,1:15],type="response")
## PLOT ROC curve to finalise your cut off value
library(pROC)
ROC=roc(subscribed~test.prob,data=test)
plot(ROC)  ## Here cut off value = 0.5 has been selected.
## Based on the problem condition, Higher Specificity OR Higher Sensitivity can be selected.In this Scenario, higher sensitivity is preferable
##Now convert the probabilitis to classes/levels with cut off as 0.5
test.prediction<-cut(test.prob,c(-Inf,0.5,Inf),labels=c("No","Yes"))
test.prediction
summary(test.prediction)
##Confusion Matrix
table(test.prediction,test$subscribed)
## 2. C5.0 Decision Tree
library(C50)## Control for C5.0 Model
C5.0Control(subset=TRUE,bands=2,winnow=TRUE,noGlobalPruning=FALSE,CF=0.25,minCases=2,label="C5.0 Outcome")## Train Model using C5.0
train.c50=C5.0(train[,1:15],train$subscribed,trials=10,rules=FALSE,control=C5.0Control(),costs=NULL)
## Variable Importance Measure
C5imp(train.c50,metric="usage",pct=TRUE)## Lower % Usage variables can be dicarded and the Train function is re-run to get better accuracy
summary(train.c50) ## Check the percentage of accuracy of the model## Predict on the TEST data using trained Model
test.c50=predict(object=train.c50,newdata=test[,1:15],trials=1,type="class")
table(test.c50,test$subscribed)  ## CONFUSION MATRIX from C5.0

Following is the ROC curve of Logistic Regression for the above solution