DefinePK

Abstract

Detecting emotions on social media is crucial for applications such as mental health monitoring and brand analytics. However, existing models often overlook inter-modal interactions, disregard cultural variations, and rely on computationally expensive architectures. We propose LightSAED, a lightweight cross-modal transformer that fuses textual, visual, and emoji data to detect emotions, sarcasm, and emotional intensity in tweets. LightSAED introduces three key innovations: (1) a dynamic cross-modal attention mechanism for effective multimodal fusion, (2) a dedicated sarcasm detection sub-layer trained with explicit supervision, and (3) a hierarchical cultural adaptation layer leveraging region-specific embeddings based on sociolinguistic features. We also present TwemoInt++, a curated dataset of 50,000+ tweets, annotated for emotion, sarcasm, and intensity, stratified into ten culturally defined regions. Extensive experiments show that LightSAED outperforms state-of-the-art baselines, improving emotion accuracy by 6.2% and sarcasm detection F1-score by 9.8%. Robustness tests against noisy data and adversarial examples further validate its reliability. To enhance efficiency, pruning and 8-bit quantization reduce inference time by 42% and model size by 63%, enabling real-time edge deployment on resource-constrained devices. Despite its advancements, challenges remain in handling ambiguous cultural cues and low-resource languages, paving the way for future enhancements