Page 231 - AI Computer 10

3. Consider an employee hiring system based on AI employed by an organisation to help them shortlist
candidates. The AI model is a classifier model. Create a Confusion matrix for the model and populate
it with sample data for 500 candidates. Using the confusion matrix, calculate the various evaluation
metrics for the model. Prepare a report in Word, illustrating your model and showing all the calculations.

Competency Based Questions

1. You are developing a machine learning model for predicting whether a student will pass an exam based
on their study hours and past performance. The dataset is balanced. Which evaluation metric should you
primarily focus on?

a. F1 Score b. Precision
c. Error d. Accuracy
2. A bank is using an AI model to detect fraudulent transactions. The dataset is highly imbalanced, with 98%
non-fraudulent and 2% fraudulent cases. Which metric should be prioritized?

a. F1 Score b. Precision
c. Recall d. Accuracy

3. An AI model predicts whether an email is spam or not. It correctly identifies 90% of spam emails but also
incorrectly classifies 15% of legitimate emails as spam. Which metric should be improved, and why?
a. F1 Score b. Precision
c. Recall d. Accuracy

4. A company uses AI to screen job applicants. If the model has high False Negatives, what issue might
arise?

a. Unqualifi ed candidates might be rejected
b. Unqualifi ed candidates might be accepted

c. Qualifi ed candidates might be rejected
d. Qualifi ed candidates might be accepted

Case Study Based Questions

1. The IT cell of a bank developed a model to predict whether a customer will default on a loan. The
confusion matrix based on the model’s performance is:

Actual Default Actual No Default

Predicted Default 80 20
Predicted No Default 40 860
Answer the following question.
a. What is the recall for detecti ng loan defaulters?

b. What is the precision for the given data?
c. Why is recall more important than precision in this case?

97
97