Inferring Vocational Interests through an AI-based Chatbot: Examining Psychometric Properties of Machine-inferred Vocational Interests Scores
Start Date
17-10-2025 11:00 AM
End Date
17-10-2025 11:30 AM
Submission Type
Competitive Paper
Track
Entrepreneurship
Abstract
Artificial intelligence (AI) and Machine Learning (ML) have been hot topics recently and have been applied in various areas of psychological research. Increasing attention has been focused on using AI/ML to predict the human mind, especially personality (e.g., Hickman et al., 2021; Youyou et al., 2015). This new approach has generally been referred to as AI-based psychological assessment (Fan et al., 2023). The research results showed some promises of this new approach in measuring individual differences (Azucar et al., 2018; Settanni et al., 2018). However, other than personality, this new approach has seldom been applied to the measurement of other types of individual differences such as career interests, emotional intelligence, cognitive ability, etc. The present research focuses on vocational interests, which reflect relatively stable individual differences in motives, goals, and personal strivings both within and beyond the vocational domain (Stoll et al., 2017; Van Iddekinge et al., 2011). Vocational interests hold a crucial position in applied psychology in that they are important noncognitive factors that influence major work and life outcomes, and their assessment deserves innovation (Chernyshenko et al., 2011). In the current research, I explore the feasibility of using an AI-based chatbot to measure vocational interests. Specifically, I build predictive models using text scripts to predict vocational interests and then comprehensively examine the psychometric properties of machine-inferred interests scores: reliability, internal structure, convergent and discriminant validity, and criterion-related validity.
Inferring Vocational Interests through an AI-based Chatbot: Examining Psychometric Properties of Machine-inferred Vocational Interests Scores
Artificial intelligence (AI) and Machine Learning (ML) have been hot topics recently and have been applied in various areas of psychological research. Increasing attention has been focused on using AI/ML to predict the human mind, especially personality (e.g., Hickman et al., 2021; Youyou et al., 2015). This new approach has generally been referred to as AI-based psychological assessment (Fan et al., 2023). The research results showed some promises of this new approach in measuring individual differences (Azucar et al., 2018; Settanni et al., 2018). However, other than personality, this new approach has seldom been applied to the measurement of other types of individual differences such as career interests, emotional intelligence, cognitive ability, etc. The present research focuses on vocational interests, which reflect relatively stable individual differences in motives, goals, and personal strivings both within and beyond the vocational domain (Stoll et al., 2017; Van Iddekinge et al., 2011). Vocational interests hold a crucial position in applied psychology in that they are important noncognitive factors that influence major work and life outcomes, and their assessment deserves innovation (Chernyshenko et al., 2011). In the current research, I explore the feasibility of using an AI-based chatbot to measure vocational interests. Specifically, I build predictive models using text scripts to predict vocational interests and then comprehensively examine the psychometric properties of machine-inferred interests scores: reliability, internal structure, convergent and discriminant validity, and criterion-related validity.
