A Study of “Multimodal Depression Detection Using Machine Learning

Abstract

Depression is a major mental-health concern characterized by persistent disturbances in mood, interest, cognition, behavior, and daily functioning. The increasing availability of digital behavioural data has encouraged researchers to investigate artificial intelligence (AI) and machine-learning (ML) approaches for supporting depression screening and assessment. While conventional approaches frequently depend on questionnaires and clinical interviews, machine-learning research has explored potentially informative signals from text, speech, facial behaviour, video, social-media activity, physiological measurements, and other behavioural sources. Multimodal machine learning is particularly important because depression is expressed through interconnected linguistic, vocal, visual, and behavioural characteristics that may not be adequately represented by a single data source. This article descriptively reviews published research on multimodal machine-learning approaches for depression detection using secondary data. The analysis covers conventional ML, deep learning, recurrent neural networks, attention mechanisms, Transformers, graph-based approaches, contrastive learning, and emerging large-language-model-based systems. The reviewed evidence indicates a gradual movement from single-modality approaches toward multimodal representation and fusion, particularly involving text, audio, and video. However, differences in datasets, clinical labels, populations, preprocessing procedures, evaluation measures, and fusion strategies restrict direct comparison of reported performance. Privacy, bias, interpretability, dataset imbalance, and external validation remain important challenges. The study concludes that multimodal ML provides a promising research direction for depression screening support, but automated detection should complement rather than replace professional clinical assessment