A Multimodal Approach to Confidence, Emotion, and Personality Assessment in Virtual Interviews
DOI:
https://doi.org/10.5269/bspm.82391Abstract
Therecruitmentlandscapeemphasizesprioritizingcandidatestechnicalskillsoverinterpersonal
skills suchasemotional intelligence, suitablebehaviour, communicationstyleandadaptability. Traditional
systemsanalysetextorskillbasedresponses,therebymissingouttoanalysebehaviourandrealtimeemotions
ininterview. Thiscreatesgapinanalysingthecandidatestruepersonalityprofileandstyleof interaction.
Therefore, there isnecessityofmultimodal systemtoevaluate ,whatcandidatespeak,howtheyspeakand
howtheypresentthemselvesduringinterview.Toaddressthis limitations, thisstudyproposesamultimodal
emotionrecognitionsystemwhichstrengthensforpersonalityassessmentindigitalinterviews.Thisframework
processesbyintegratinglinguisticcues fromtextual transcripts,paralinguisticcues fromspeechsignalsand
visualexpressionsfromvideoframes.Textbasedemotionfeaturesareextractedusingafinetunedtransformer
model,achieved43%accuracy. Audiobasedemotional cuesarerecognizedusingaspeechemotionpipeline,
achieved63%accuracy.Facialexpressionsareanalysedusingvisionbasedmodule.Earlyfusionstrategyhas
beenappliedto integratethesemultiplemodalitieswhichenables thesystemtogetconsolidatedemotional
statefromtheinterview.Thismultimodalarchitecturegivesagreatsupporttoevaluatecandidatecomplete
profile,whichmakes therecruitmentdecisionholistic. Thismakes theorganizations tohirethecandidates
whoarenotonlytechnicallystrongbutalsowithgoodinterpersonalskills.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Boletim da Sociedade Paranaense de Matemática
This work is licensed under a Creative Commons Attribution 4.0 International License.
When the manuscript is accepted for publication, the authors agree automatically to transfer the copyright to the (SPM).
The journal utilize the Creative Common Attribution (CC-BY 4.0).
How to Cite
