DefinePK

Research Objective

To automatically distinguish and classify seven basic hand gestures representing symbolic emotions or expressions (happy, sad, neutral, disgust, scared, anger, and surprise) in Pakistan Sign Language using deep learning techniques, specifically transfer learning with the VGG16 architecture.

Methodology

The study proposes a transfer learning approach using the VGG16 architecture, pre-trained on ImageNet, to classify seven basic hand gestures representing emotions in Pakistan Sign Language. A dataset of 455 images collected from 65 individuals was used. The performance of VGG16 was compared with two optimizers (SGD and Adam) and other architectures like AlexNet, LeNet05, and ResNet50.

Methodology Flowchart

                        graph TD
    A[Data Collection: 455 images, 65 individuals, 7 emotion gestures] --> B[Data Preprocessing: Resize images, 75:25 train/test split]
    B --> C[Model Selection: VGG16 architecture]
    C --> D[Transfer Learning: Use pre-trained weights from ImageNet]
    D --> E[Model Adaptation: Replace classifier layers for 7 classes]
    E --> F[Optimizer Selection: Adam, SGD]
    F --> G[Training: Train adapted VGG16 model]
    G --> H[Evaluation: Accuracy, Loss, Precision, Recall, F1-Score, AUC, Confusion Matrix]
    H --> I[Comparison: VGG16Adam vs VGG16SGD vs AlexNet, LeNet05, ResNet50]
    I --> J[Conclusion: VGG16 with Adam achieves highest accuracy]                    

Discussion

The study highlights the effectiveness of transfer learning, particularly with the VGG16 architecture and Adam optimizer, for accurate hand sign interpretation, even with limited and low-quality data. The research addresses the challenge of interpreting sign language automatically, aiming to bridge the communication gap for deaf and hard-of-hearing individuals.

Key Findings

The proposed VGG16 architecture with the Adam optimizer achieved a high accuracy of 99.98% on a minimal and low-quality dataset of 455 images for seven hand gesture classes. Other tested architectures (AlexNet, LeNet05, ResNet50) and VGG16 with SGD optimizer did not perform as well.

Conclusion

The research successfully demonstrates the efficacy of a transfer learning-based VGG16 model with the Adam optimizer for interpreting Pakistan Sign Language expressions through hand gestures, achieving near-perfect accuracy. This approach offers a promising avenue for developing real-time sign language recognition systems.

Fact Check

1. Accuracy Achieved: The study reports a high accuracy of 99.98% for the proposed VGG16 architecture with the Adam optimizer.
2. Dataset Size: The dataset used comprised 455 images collected from 65 individuals.
3. Number of Classes: The model was trained to classify seven basic hand gestures representing emotions.

Mind Map