Introduction
Autism spectrum disorder (ASD) is a neuro-developmental condition that affects how people communicate, interact, and behave. ASD can be diagnosed through various methods, such as observing the behavior of the individual, interviewing the parents or caregivers, or using brain imaging techniques. However, these methods have some limitations, such as being time-consuming, subjective, or expensive.
In recent years, a new approach has emerged that uses deep learning (DL) to analyze images or videos of the brain or the face of the individual. DL is a branch of artificial intelligence that can learn complex patterns from large amounts of data. DL can be applied to different types of images or videos, such as functional magnetic resonance imaging (fMRI), electroencephalography (EEG), eye tracking, facial expression, or gesture recognition.
A recent paper provides a systematic review of the DL-based approach involving the analysis of images or videos in autism research. The paper covers studies that were published from 2017 to June 2023 and were indexed in PubMed, IEEE Xplore, ACM Digital Library, and Google Scholar. The paper categorizes the studies based on the different features extracted as input for the DL-based approach, such as brain connectivity, brain activation, eye movement, facial expression, or gesture. The paper also reviews the existing public and private datasets that include images or videos for autism research, and discusses the different rehabilitation strategies that have been shown to be beneficial for ASD individuals. Finally, the paper identifies the various challenges and future directions for the automated detection, classification, and rehabilitation of ASD using DL.
Why Use DL for ASD Diagnosis?
The paper argues that DL has several advantages over traditional methods for ASD diagnosis, such as:
- DL can learn from large and complex datasets without requiring much human intervention or prior knowledge.
- DL can handle noisy, incomplete, or heterogeneous data, which are common in ASD research.
- DL can extract high-level and abstract features from images or videos that can capture the subtle differences between ASD and typical individuals.
- DL can achieve high accuracy and generalization performance, which can improve the reliability and validity of ASD diagnosis.
- DL can provide interpretable and explainable results, which can help understand the underlying mechanisms and biomarkers of ASD.
How to Use DL for ASD Diagnosis?
The paper reviews the different types of images or videos that can be used as input for the DL-based approach, and the different DL models that can be applied to them. The paper categorizes the studies into four groups, based on the type of image or video:
- Brain imaging: This group includes studies that use fMRI or EEG images to measure the brain activity or connectivity of ASD individuals. The paper reviews the different DL models that can be used to analyze these images, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), autoencoders, or graph neural networks. The paper also discusses the different types of features that can be extracted from these images, such as functional connectivity, regional homogeneity, amplitude of low-frequency fluctuations, or graph theoretical measures.
- Eye tracking: This group includes studies that use eye tracking images or videos to measure the eye movement or gaze patterns of ASD individuals. The paper reviews the different DL models that can be used to analyze these images or videos, such as CNNs, RNNs, or attention models. The paper also discusses the different types of features that can be extracted from these images or videos, such as fixation duration, saccade amplitude, scan path, or saliency map.
- Facial expression: This group includes studies that use facial expression images or videos to measure the emotional or social responses of ASD individuals. The paper reviews the different DL models that can be used to analyze these images or videos, such as CNNs, RNNs, or generative adversarial networks (GANs). The paper also discusses the different types of features that can be extracted from these images or videos, such as facial landmarks, facial action units, facial emotion recognition, or facial expression synthesis.
- Gesture recognition: This group includes studies that use gesture recognition images or videos to measure the motor or social skills of ASD individuals. The paper reviews the different DL models that can be used to analyze these images or videos, such as CNNs, RNNs, or pose estimation models. The paper also discusses the different types of features that can be extracted from these images or videos, such as body joints, body parts, body pose, or gesture classification.
What are the Challenges and Future Directions for DL-based ASD Diagnosis?
The paper identifies several challenges and future directions for the DL-based approach for ASD diagnosis, such as:
- Data availability and quality: The paper notes that there is a lack of large and diverse datasets that include images or videos for autism research. The paper suggests that more efforts should be made to collect, annotate, and share such datasets, and to ensure their quality, privacy, and ethics.
- Data integration and fusion: The paper notes that ASD is a complex and heterogeneous condition that involves multiple modalities and domains. The paper suggests that more research should be done to integrate and fuse different types of images or videos, as well as other types of data, such as genetic, behavioral, or clinical data, to provide a more comprehensive and accurate diagnosis of ASD.
- Model interpretability and explainability: The paper notes that DL models are often seen as black boxes that provide little insight into their decision-making process. The paper suggests that more research should be done to make the DL models more interpretable and explainable, and to provide feedback and guidance to the users, such as clinicians, therapists, or parents.
- Model evaluation and validation: The paper notes that DL models are often evaluated and validated using metrics that may not reflect the real-world scenarios or the needs of the users. The paper suggests that more research should be done to evaluate and validate the DL models using more realistic and meaningful metrics, such as clinical relevance, user satisfaction, or social impact.
Conclusion
The paper concludes that the use of DL for the precise and affordable diagnosis of autism is increasing substantially. The paper provides a comprehensive and systematic review of the DL-based approach involving the analysis of images or videos in autism research. The paper also discusses the existing datasets, the rehabilitation strategies, and the challenges and future directions for the DL-based approach. The paper hopes that its findings will benefit researchers, therapists, psychologists, and relevant stakeholders to advance ASD screening, monitoring, and diagnosis with the aid of a DL-based approach that entails image or video analysis.
FAQ
How many studies were included in the systematic review and what were their main characteristics?
The authors included 51 studies in the systematic review. The studies were published from 2017 to June 2023. The studies involved participants from different age groups, ranging from infants to adults. The studies used different types of images or videos, such as fMRI, EEG, eye tracking, facial expression, or gesture recognition. The studies used different types of DL models, such as CNNs, RNNs, autoencoders, graph neural networks, GANs, attention models, or pose estimation models. The studies reported different types of outcomes, such as accuracy, sensitivity, specificity, or F1-score.
What are the existing public and private datasets that include images or videos for ASD research?
The paper reviews the existing public and private datasets that include images or videos for ASD research, such as ABIDE, ABIDE II, ADHD-200, Autism Brain Imaging Data Exchange (ABIDE), Autism Brain Imaging Data Exchange II (ABIDE II), Autism Diagnostic Interview-Revised (ADI-R), Autism Diagnostic Observation Schedule (ADOS), Autism Phenome Project (APP), Autism Speaks, Autism Treatment Network (ATN), Autism Video Dataset (AVD), Child Mind Institute (CMI), Emotion Recognition in the Wild (EmotiW), Face and Gesture Recognition Research Network (FG-NET), Facial Expression Recognition and Analysis Challenge (FERA), Facial Expression Recognition Challenge (FERC), First Impressions, Gaze Tracking in the Wild (GTW), Human Connectome Project (HCP), International Affective Picture System (IAPS), International Affective Digitized Sounds (IADS), Kinetic Family Drawing (KFD), Kinetic House-Tree-Person Drawing (KHTP), Kinetic School Drawing (KSD), Mindboggle, Multimodal Autism Phenotype Dataset (MAPD), Multimodal Social Interaction Dataset (MSID), NIMH Data Archive (NDA), OpenfMRI, OpenNeuro, Pediatric Imaging, Neurocognition, and Genetics (PING), Philadelphia Neurodevelopmental Cohort (PNC), Social Interaction and Communication Database (SICD), Social Signal Interpretation (SSI), and YouTube Faces.
What are the rehabilitation strategies that have been shown to be beneficial for ASD individuals?
The paper discusses the rehabilitation strategies that have been shown to be beneficial for ASD individuals, such as social skills training, cognitive behavioral therapy, applied behavior analysis, virtual reality, serious games, and robot-assisted therapy.
Source:
https://www.sciencedirect.com/science/article/abs/pii/S0952197623013696