How To Detect Credit Card Fraud Through Transaction Time Using Machine Learning

In this section, we will be creating and training our model for predicting whether a transaction is fraudulent. Since there are multiple algorithms we can use to build our model, we will compare the accuracy scores after testing and pick the most accurate algorithm.

For my use, I exported the normalized, equal parts fraud and not-fraud data to a .csv file called “cleaned credit card.csv”. You can use the code below to do this or the equalized df generated earlier to provide the same functionality.

equalized_df.to_csv('datasets/cleaned credit card.csv')

Let’s begin by importing the different models we will be used for classification.

In [9]:

from xgboost import XGBClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
from sklearn.naive_bayes import GaussianNB
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.neighbors import KNeighborsClassifier

From this list, we are using XGBoost, DecisionTree, RandomForest, Naive Bayes, LogisticRegressoin, SVC, and KNeighborsClassifier to perform our predictions. We will then see which algorithm produces the highest accuracy and select it as our algorithm of choice for future use. We also want to partition our dataset into training, testing, and validation, so let’s add a method for that ability.

In [10]:

from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

Now, we can begin building and training our model.

Let’s split our data into test, train, and validation using train_test_split.

Our testing will take three phases: testing, training, and validation. Training is first, and it’s where our model generates “intuition” about how to approach fraudulent and not fraudulent transactions. It is similar to a student studying and developing knowledge about a topic before an exam.

The testing phase is where we see how the model performs against data where we know the outcome. This would be the exam if we continue the analogy from before. The algorithms will perform differently, similar to how students score differently on exams. We generate an accuracy score from this phase to compare the different algorithms.

Validation testing ensures that the model isn’t overfitting to our specific dataset. Overfitting is when the model develops an intuition too specific to the training set. Overfitting is a problem because our model is no longer flexible. It may work on the initial set, but subsequent uses will cause our model to fail. Continuing the exam analogy, the validation testing phase is like another exam version with different questions. If a student happened to cheat on the first exam by knowing the questions, the second exam will give a better representation of performance.

Note that verification doesn’t completely disprove or prove to overfit, but the testing does give insight into it.

training,test = train_test_split(equalized_df, train_size = 0.7, test_size = 0.3, shuffle=True)
training, valid = train_test_split(training, train_size = 0.7, test_size =0.3, shuffle=True)

training_label = training.pop(‘Class’)
test_label = test.pop(‘Class’)
valid_label = valid.pop(‘Class’)

We assign the ‘Class’ column to be our label as that is what we are trying to classify by. Our training and testing will use these labels to compare the predicted output versus the actual output.

In [12]:

import pprint

pp = pprint.PrettyPrinter(indent=4)

# instantiate the algorithms
xgb = XGBClassifier()
dtc = DecisionTreeClassifier()
rfc = RandomForestClassifier()
nbc = GaussianNB()
LR = LogisticRegression(random_state=0, solver=’lbfgs’,multi_class=’multinomial’)
SVM = SVC(kernel=’rbf’, C=1,gamma=’auto’)
knn = KNeighborsClassifier(n_neighbors=3)

# train the models
xgb.fit(training, training_label)
dtc.fit(training, training_label)
rfc.fit(training, training_label)
nbc.fit(training, training_label)
LR.fit(training, training_label)
SVM.fit(training, training_label)
knn.fit(training, training_label)

# try and predict an outcome from the test set
xgb_predict = xgb.predict(test)
dtc_predict = dtc.predict(test)
rfc_predict = rfc.predict(test)
nbc_predict = nbc.predict(test)
LR_predict = LR.predict(test)
SVM_predict = SVM.predict(test)
knn_predict = knn.predict(test)

# judge accuracy using built-in function
accuracy = dict()
accuracy[‘XGBoost’] = accuracy_score(test_label, xgb_predict)
accuracy[‘Naive_bayes’] = accuracy_score(test_label, nbc_predict)
accuracy[‘DecisionTree’] = accuracy_score(test_label, dtc_predict)
accuracy[‘RandomForest’] = accuracy_score(test_label,rfc_predict)
accuracy[‘support_vector_Machines’] = accuracy_score(test_label,SVM_predict)
accuracy[‘Linear Regression’] = accuracy_score(test_label,LR_predict)
accuracy[‘KNN’] = accuracy_score(test_label,knn_predict)c:\cocolevio\5411_alpha\ffea_notebooks\venv\lib\site-packages\sklearn\ensemble\forest.py:245: FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.
“10 in version 0.20 to 100 in 0.22.”, FutureWarning)

The accuracies for the different algorithms are shown below, sorted by algorithm name and then its decimal accuracy:

In [13]:

pp.pprint(accuracy){   'DecisionTree': 0.9932432432432432,
    'KNN': 0.9493243243243243,
    'Linear Regression': 0.9797297297297297,
    'Naive_bayes': 0.9797297297297297,
    'RandomForest': 0.9864864864864865,
    'XGBoost': 0.9932432432432432,
    'support_vector_Machines': 0.9290540540540541
}

From the preliminary testing, we can see that all of the algorithms are above 90% accuracy. While this is good, we also must test for overfitting our model with the data we have. So, we avoid this overfitting by testing with a validation set. If the validation set accuracy is also high, we can be slightly more confident that our model isn’t overfitting.

However, it appears as if XGBoost and DecisionTree are performing the best with a perfect score on the testing set.