⚡ Try ConG-MoE v2 Live
Type any Tamil-English comment or click a sample chip below to inspect real-time classification and specialist head routing.
Why Tanglish Breaks Standard NLP
70% of movie comments are positive. Naive classifiers optimize for majority accuracy, burying genuine criticism. Four linguistic bottlenecks worsen this:
1Spelling Variation
No standard Tanglish orthography: padam appears as padham, paadam, or batham. Tokenizers treat each as a new word, creating extreme sparsity.
2Agglutinative Suffixes
Suffixes (-la, -oda, -ku) stick to English roots (trailer-la, hero-oda). 85.8% of words in social feeds are Out-of-Vocabulary in transliteration dictionaries.
3Emoji Sarcasm
Cleaning scripts discard non-ASCII emojis, yet they carry the emotion: “padam mass 🤮” is sarcastic, but stripped of emojis it looks like glowing praise.
4Mid-Sentence Reversals
“Sema hype aana climax mokka”: Bag-of-words ignores clause turns, so the positive words overpower the negative clause following aana (but).
How ConG-MoE v2 Works
ConG-MoE v2 replaces resource-heavy transformers with a fast, discourse-guided 3-head mixture of experts:
Head 1 · Statistical Classifier
Class-weighted logistic regression over 35,000 sub-word n-grams (10k words + 25k word-bounded chars). Penalizes minority errors by up to 12.5×.
Head 2 · Conjunction Pivot Monitor
Watches polarity changes across pivot conjunctions (aana, aanal, irundhalum, but). Opposing sentiments on either side vote Mixed.
Head 3 · Affective Emoji Resolver
Catches emotional dissonance between text praise and disgusted emojis (e.g., praise alongside 🤮 or 👎).
Benchmark Results
Tested on 5,000 held-out Tanglish comments. ConG-MoE v2 preserves minority recall at sub-millisecond CPU speeds.
Each dot is one of 584 true Mixed comments in the test set. The baseline dumps 438 of them into Positive; ConG-MoE v2 moves them into the correct Mixed bin.
Compare Architectures on Held-Out Test Set (N = 5,000):
Table II. Held-out test evaluation. ConG-MoE v2 cuts collapse from 75.0% to 14.04% at 0.33 ms CPU latency.
Error Diagnostics & Confusion Matrix:
Qualitative Case Studies
See how ConG-MoE v2 avoids the majority-class trap on real comments:
The Research Team
Dr. Kannimuthu S
Professor & Research Guide, Department of Computer Science and Engineering.
Department of Information Technology, Karpagam College of Engineering, Coimbatore, Tamil Nadu, India.
📚 View Academic Citations & Direct Paper Links (14 References)
- P. Bojanowski, E. Grave, A. Joulin, T. Mikolov, "Enriching Word Vectors with Subword Information," TACL, 2017. ACL Anthology ↗ · arXiv ↗
- B. R. Chakravarthi, V. Muralidaran, R. Priyadharshini, J. P. McCrae, "Corpus Creation for Sentiment Analysis in Code-Mixed Tamil-English Text," SLTU-CCURL, 2020. ACL Anthology ↗ · arXiv ↗
- B. R. Chakravarthi et al., "Overview of the Track on Sentiment Analysis for Dravidian Languages in Code-Mixed Text," FIRE / DravidianLangTech, 2021. arXiv ↗ · ACL Anthology ↗
- A. Conneau et al., "Unsupervised Cross-lingual Representation Learning at Scale (XLM-R)," ACL, 2020. ACL Anthology ↗ · arXiv ↗
- J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," NAACL-HLT, 2019. ACL Anthology ↗ · arXiv ↗
- S. Dowlagar, R. Mamidi, "Transformers with Class Weight and Focal Loss for Offensive Language Identification in Dravidian Code-Mixed Text," DravidianLangTech / SemEval, 2021. ACL Anthology ↗ · SemEval ↗
- D. Kakwani et al., "IndicNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian Languages," Findings of EMNLP, 2020. ACL Anthology ↗ · arXiv ↗
- S. Khanuja et al., "MuRIL: Multilingual Representations for Indian Languages," arXiv, 2021. arXiv ↗ · Google Research ↗
- S. V. Kogilavani et al., "Sentiment Analysis of Dravidian Code-Mixed Text Using Machine Learning Approaches," FIRE, 2020. CEUR-WS PDF ↗
- B. Premjith, K. P. Soman, P. Poornachandran, "A Subword-Level Deep Learning Architecture for Sentiment Analysis in Dravidian Code-Mixed Text," FIRE, 2020. CEUR-WS PDF ↗
- R. Sennrich, B. Haddow, A. Birch, "Neural Machine Translation of Rare Words with Subword Units (BPE)," ACL, 2016. ACL Anthology ↗ · arXiv ↗
- V. Srivastava, M. Singh, "PHAST: Phonetic-Aware Subword Tokenization for Code-Mixed Languages," DravidianLangTech, 2021. ACL Anthology ↗ · arXiv ↗
- V. Bhatia, "Tamil-English (Tanglish) Comments for Sentiment Analysis Dataset," Kaggle Datasets, 2021. Kaggle Dataset ↗
- K. Shanmugavadivel et al., "Sentiment Analysis in Code Mixture Language / Hybrid CNN-BiLSTM," DravidianLangTech / ICCIPR, 2023. ACL Anthology ↗