HR Analytics

Background?You are the head of analytics for an online retailer focused on athleisure and sports attire for mature women and men.  The executive team of your company mirrors the customer demographic they target, and they believe that having a salesforce that reflects their targeted market segment, both in terms of demographics and athletic interest, is a critical driver of their success.  Your company has seen a spike in searches for novel sports equipment inspired by the summer Olympics in Tokyo, e.g., high-end longboard skateboards, helmets and athletic attire.  Your company wants to expand to capitalize on this trend, by developing new lines of clothing and gear.  The executive team is excited by the potential opportunity and wants to move the company rapidly in a new direction with a new line of business.

Given the tight labor market, your company is concerned about missing the current trend by taking too much time to fill positions.  The Head of HR gives you data for the company’s sales employees who have been hired within the past 3 years, including those who left the company within the past year.   The data includes pre-hire resume details such as address, date of birth, hobbies and personal interest, education, technical certificates and training courses, prior sales experience in the same line of business, as well as current details, such as annual performance review results, promotion history, manager assessment of their long-term potential, sales target achievement and customer satisfaction ratings. The Head of HR asks you to develop a model to screen and predict candidates who will become successful new sales employees for the new line of business.

Assume that you developed your model based on the 3 years of data.  You validate your model using a test sample of your dataset.  You have calculated the following confusion matrix and associated parameters:

Predicted QualifiedPredicted NonqualifiedActual Qualified10515Actual Nonqualified15100

Sensitivity = 88%

Sensitivity is the true positive rate.  It is the number of positive predictions as percent of the total number of actual qualified candidates.

Specificity = 87%

Specificity is the true negative rate.  It is the number of negative predictions as percent of the total number of actual unqualified candidates.

Precision = 88%

Precision is the positive predictive value.  It is the number of correct/actual positive predictions as a percent of the total number of positive predictions.

Accuracy = 87%

Accuracy is the number of correctly predicted candidates (positive & negative) as a percent of the total number of candidates.

Questions:

Which of the variables listed above (second paragraph) would you use to define success of sales employees in order to develop your model? (10 points)

The choice of independent variables for your model has to be based on their power to predict the dependent variable AND their availability in job candidates’ resumes that you intend to screen.  Which of the variables listed above (second paragraph) would you test as predictors of success in your model? (10 points)

What are the false positive rate and the false negative rate? Describe what false positive and false negative mean.  (10 points)

If your goal is to develop a model to screen resumes and identify candidates to be invited for an interview, which type of error is worse – false positive or false negative? Explain the rationale for your answer. (10 points)

If you want to improve the performance of your model to identify candidates to be invited for an interview, and minimize the error that you defined in question 4, which parameter (sensitivity, specificity, precision, accuracy) would you use to guide your work? (10 points)

If your goal is to develop a model to identify candidates who will receive a job offer, which type of error is worse – false positive or false negative? Explain the rationale for your answer. (10 points)

If you want to improve the performance of your model to identify candidates to receive a job offer, and minimize the error that you defined in question 6, which parameter (sensitivity, specificity, precision, accuracy) would you use to guide your work? (10 points)

The Head of HR is impressed by your work, and she wants to use your model to identify candidates who will receive a job offer.  Would you recommend using your model to make the decision to offer jobs to candidates?  Discuss the rationale for your answer. (10 points)

Fast forward one year.  The company deployed the model that you developed with the 3 years of data on current employees, and people were hired based on your predictions.  The Head of HR has now come back to you with a concern that not all of the new hires were “good”.  Eleven of the 100 people hired were not qualified, and did not work out.  What parameter in the confusion matrix would you use to understand if your model worked better than you expected, as well as you expected or worse than you expected?  How well did the model work?   Explain the rationale for your answers.  (10 points)

Describe any limitations and/or concerns associated with your approach. (10 points)

HR Analytics
Background:
You are the head of analytics for an online retailer focused on athleisure and
sports attire for mature women and men. The executive team of your
company mirrors the customer demographic they target, and they believe that
having a salesforce that reflects their targeted market segment, both in terms
of demographics and athletic interest, is a critical driver of their success. Your
company has seen a spike in searches for novel sports equipment inspired by
the summer Olympics in Tokyo, e.g., high-end longboard skateboards, helmets
and athletic attire. Your company wants to expand to capitalize on this trend,
by developing new lines of clothing and gear. The executive team is excited
by the potential opportunity and wants to move the company rapidly in a new
direction with a new line of business.
Given the tight labor market, your company is concerned about missing the
current trend by taking too much time to fill positions. The Head of HR gives
you data for the company’s sales employees who have been hired within the
past 3 years, including those who left the company within the past year. The
data includes pre-hire resume details such as address, date of birth, hobbies
and personal interest, education, technical certificates and training courses,
prior sales experience in the same line of business, as well as current
details, such as annual performance review results, promotion history,
manager assessment of their long-term potential, sales target achievement and
customer satisfaction ratings. The Head of HR asks you to develop a model to
screen and predict candidates who will become successful new sales
employees for the new line of business.
Assume that you developed your model based on the 3 years of data. You
validate your model using a test sample of your dataset. You have calculated
the following confusion matrix and associated parameters:
Predicted Qualified
Predic
Actual Qualified
105
15
Actual Nonqualified
15
100
•
o Sensitivity = 88%
Sensitivity is the true positive rate. It is the number of positive
predictions as percent of the total number of actual qualified
candidates.
o Specificity = 87%
Specificity is the true negative rate. It is the number of negative
predictions as percent of the total number of actual unqualified
candidates.
o Precision = 88%
Precision is the positive predictive value. It is the number of
correct/actual positive predictions as a percent of the total number of
positive predictions.
o Accuracy = 87%
Accuracy is the number of correctly predicted candidates (positive &
negative) as a percent of the total number of candidates.
Questions:
1. Which of the variables listed above (second paragraph) would you use
to define success of sales employees in order to develop your model?
(10 points)
2. The choice of independent variables for your model has to be based on
their power to predict the dependent variable AND their availability in
job candidates’ resumes that you intend to screen. Which of the
variables listed above (second paragraph) would you test as predictors
of success in your model? (10 points)
3. What are the false positive rate and the false negative rate? Describe
what false positive and false negative mean. (10 points)
4. If your goal is to develop a model to screen resumes and identify
candidates to be invited for an interview, which type of error is worse false positive or false negative? Explain the rationale for your answer.
(10 points)
5. If you want to improve the performance of your model to identify
candidates to be invited for an interview, and minimize the error that
you defined in question 4, which parameter (sensitivity, specificity,
precision, accuracy) would you use to guide your work? (10 points)
6. If your goal is to develop a model to identify candidates who will
receive a job offer, which type of error is worse – false positive or false
negative? Explain the rationale for your answer. (10 points)
7. If you want to improve the performance of your model to identify
candidates to receive a job offer, and minimize the error that you
defined in question 6, which parameter (sensitivity, specificity,
precision, accuracy) would you use to guide your work? (10 points)
8. The Head of HR is impressed by your work, and she wants to use your
model to identify candidates who will receive a job offer. Would you
recommend using your model to make the decision to offer jobs to
candidates? Discuss the rationale for your answer. (10 points)
9. Fast forward one year. The company deployed the model that you
developed with the 3 years of data on current employees, and people
were hired based on your predictions. The Head of HR has now come
back to you with a concern that not all of the new hires were
“good”. Eleven of the 100 people hired were not qualified, and did not
work out. What parameter in the confusion matrix would you use to
understand if your model worked better than you expected, as well as
you expected or worse than you expected? How well did the model
work? Explain the rationale for your answers. (10 points)
10. Describe any limitations and/or concerns associated with your
approach. (10 points)
Assessment
Point values for each question are stated above. Each question will be
evaluated based on the following criteria:
1. Your ability to derive insights from the data provided.
2. The clarity of your logic, and the accuracy of your answers.