EFFICIENT PREPROCESSING STEPS FOR STRUCTURAL MAGNETIC RESONANCE IMAGES USED IN AUTISM DIAGNOSIS MODE

Abstract

ABSTRACT Convolutional neural networks (CNNs) have driven significant progress in many fields including image classification. In recent years, CNNs have gained substantial attention in the field of medical image classification due to its exceptional ability to automatically extract features from images. As a result, they are widely used in many medical image-based diagnosis systems. The proposed invention developed a simple CNN architecture to diagnose Autism Spectrum Disorder (ASD) from structural magnetic resonance imaging(sMRl) omages. In this study, we designed a 3D CNN called 3D MANe! (Modified AlexNet) to process three- dimensional structural MRI images. Data collected from a multisite ABIDE II repository were pre-processed, normalized and resampled. The dataset was extended through data augmentation to facilitate effective model training. The model was trained on a cloudbased GPU with the RAM capacity of 45GiB and evaluated on the test data. The test results showed that the model98.71% correctly predicted the autistic MRis. The proposed model's performance was validated by comparing it with pre-defined models. The 3D MANet achieved the highest classification accuracy among all models tested, and its simple architecture ensures it is computationally less intensive.

Specification

DESCRIPTION
FJELD OF THE INVENTION
[0001) The present invention relates to the diagnosis of the neuro developmental disorder
autism using brain images. The structural magnetic resonance imaging images were used to
predict this disorder. Multisite brain images were collected and processed to remove noise
and nonnalize the intensity variations. Then the processed 3D-brain images were then
-divided into training and testing sets and the 30 MANet model was developed. The-deep
learning model was trained on the training set and predictions were made on the test data.
This model achieved an accuracy rate of 98.71% and an AUC value of 0.9941. This deep
learning model can assist clinicians in the diagnosis of autism spectrum disorder.
BACKGROUND OF THE INVENTION
[0002) Autism is a neurodevelopmental disorder that impacts the patients' social
relationships, interactions, and communication. It also leads to restrictive and repetitive
behaviours. The root cause of this disorder is unclear, but the early diagnosis can significantly
improve the quality of life of the children affected. Currently, diagnosis involves screening,
interviews with parents and care takers and asking them to complete questionnaires.
Clinicians assign the scores based on the responses, and a diagnosis of autism is made
accordingly. However, this process is too long and time taking due to the high demand for
psychiatrists, clinicians and physicians. The availability of medical professionals and their
appointment schedules further delay the diagnosis. Thus, there is an urgent need for an
automated diagnosis system to predict autism more quickly and with less reliance on medical
professionals.
)0003] Neuroimaging techniques are more sophisticated and are important tool for
neurology and mental health diagnosis. The brain imaging techniques such as sMRL
fMRI and DTI" images play a vital role in diagnosing neurodevelopmental disorders.
The identification of biomarkers with clear neural underpinnings in ASD would be
helpful in ensuring an early and accurate diagnosis as well as an optimally effective
treatment. Structural and functional magnetic resonance imaging has the potential to
reveal brain abnormalities of ASD that could be used as biomarkers of the disease.
[0004) Examination of brain anatomy using Magnetic resonance imaging allows
researchers and clinicians to investigate the brain noninvasively. Due to its high
spatial resolution and contrast sensitivity, structural MRI is widely used to investigate
brain morphology. As such, medical imaging plays an important role in early
diagnosis of autism spectrum disorder. Traditional diagnostic techniques are not
efficient, as diagnoses are often made after the manifestation of substantial
9PATFNT
a)'"~--.
f·HFt,it-iAT ,-. ' ' .-- •• ' • ' '""T
N
-Q)
C)
Ill
D..
"Q) -1-- N
E....
0
-LL. CIO .....
Lt)
N
CIO
0 .....
-::1'
-::1'
N
0
~
Lt) .......... .....
M -..... -::1'
N
0
N
-I CJ
Sheet 2
Anncxure-2
behavioural disorders. It is crucial to diagnose these conditions early, before visible
behavioural symptoms arise.
100051 In recent years, deep learning (DL) techniques have become increasingly
popular in diagnosing neurological disorders like autism, as they reduce physician's
workloads. These deep learning algorithms are artificial intelligence applications that
can automatically learn and detect patterns within large datasets. Deep learningbased
applications uses neural networks that can make consistent, highperformance
predictions using complex and nonlinear relationships between
features. They can identify intricate patterns that are not visible to the human eye.
ASD is an extremely complicated neurodevelopmental disorder that can be
diagnosed with the help of medical imaging and optimal deep learning algorithms.
[0006] Patent id CN211862821U developed an auxiliary ASD diagnosis method, based on
eye movement techniques towards face recognition and emotion perception tasks. In the data
acquisition process, the subjects were guided to complete a video-watching task, and their
eye movements were recorded using an eye tracker. These data were pre-processed and an
. automatic feature extraction was perfmmed by combining a convolutional neural ·network
(CNN) using deep learning. The neural network classifier was trained using this model, and
finally, the auxiliary diagnosis of ASD was realized(Shenzhen Institute of Advanced
Technology of CAS, 20 19).
[0007] A European patent with an international publication id W02013062937 invented a
mobile tool that provides vide-based screening of children for risk of having autism spectrum
disorder. This invention was basically designed to speed up the diagnosis process and expand
population coverage. It also aimed to reduce the cost and time involved in obtaining a
diagnosis. Additionally, it addressed the issue of variability in a subject's behaviour outside
of routine conditions. The invention utilized a machine learning protocol for analyzing the
behavioural data. However, it still relies on the responses of parents or caregivers to predict
the disorder which may lead to misdiagnosis(W ALL, 20 14).
10008] A patent with the publication number 20190298245 invented a method for early
diagnosis of autism in children. This invention used four movement sensitivity parameters
during the interaction of the examined child with a touch screen. The movement
abnormalities are the core features of autism, so this invention used the movement sensitivity
parameters to diagnose autism in children. The results extracted from the touch screen fed
into a machine learning model using the K-fold cross validation method. From the prediction
results, the receiver operating characteristics curve was generated, and specificity and
sensitivity were calculated.(Pawel JARMOLKOWICZ (Dobre Miasto), 2019)
BRIEF SUMMARY OF THE INVENTION
10009] Medical imaging utilizes various techniques to produce images of the human body's
intemal structures. These images playing a vital role in assisting the medical professionals
with diagnosing, treating and monitoring diseases or disorders. Medical imaging is becoming
essential across various biomedical research and clinical practice fields. For example,
radiologists can identify and quantify tumors from MRI and CT scans while neuroscientists
detect regional metabolic brain activity from positron emission tomography (PET).
1001 OJ Medical professionals face several challenges when using medical imaging in their
clinical practice. These challenges include low image resolution, high noise levels, low
9Pt.Ti=r,iT
en''''~"··
(HENNP.,I
N
r------------------------------------------------------------------------------------
-Q)
C)
Ill
D..
Q) -1-- N
E...
0
-LL. CIO .....
1.1')
N
CIO
0 .....
-::1'
-::1'
N
0
~
1.1') .......... .....
-.M... . -::1'
N
0
N
-I CJ
9enP' A.T., ,>,_=_f -.. ,i T..
N
Sheet 3
Anncxure-2
contrast, geometric defmmations, and the presence of ar1i facts. Preprocessing. is a key
solution to address these issues. Additionally, obtaining sufficient medical image data from a
single site for a particular disorder is difficult. Deep leaming models require large data sets to
train the model. It is inevitable to collect the data from different sites. However, multisite data
captured from different scanners are often affected by the varying intensity levels, which can
negatively impact the perfom1ance of the deep learning models during prediction. Therefore,
hannonizing multisite data through preprocessing is crucial for improving model
performance.
(0011) Preprocessing tasks such as registration, skull stripping, smoothing, bias correction are
the common steps perfonned on medical images. This invention outlines a set of
preprocessing tasks, including Fuzzy C-Means (FO.CM) nonnalization and resampling.
Nmmalization effectively harmonizes multisite data by .establishing a uniform intensity range
for MRI sacns. Resampling reduces the file size by aligning the preprocessed MRI image in
to 'TI space'. These tasks are applied to structural brain MR images to improve their quality.
Additionally, the proposed invention augments the small rlataset. at the intensity level to
expand the data necessary for-training the deep learning· model. The proposed model, riluiled
3D MANe! is designed with appropriate filters and fine-tuned using hyper parameters to
prevent overfilling and ensure smooth convergence. Model training begins by splitting the
data set into training and testing sets. The invention utilizes a 'CUDA' environment in a
cloud-based system. The model training performed on a 16GiB GPU, 45Gib RAM. After
training, the test results are evaluated on the test data. The model performance is measured in
terms of Accuracy, Loss, Sensitivity, Specificity, FI-Score, and Area Under Curve (AUC).
BRIEF DESCRIPTION OF THE ORA WINGS
[0013] The following description describes the present invention in terms of processing steps
required to implement an embodiment of the invention. These steps can be performed by an
appropriately programmed computer, with python installed and the sMRJ image processing
software like FSL (fMRJ software Library). In addition, neuroimaging libraries like nileam,
nibabel are used in this autism diagnosis model. The data set has been taken from the public
repository I 00 known as ABIDE (Autism Brain Imaging Data Exchange-H)). The
participants in this proposed system are aged between 5 to 13 years. sMR I images from five
different sites have been collected and used comprising a total.of 144 autistic and 218 nonautistic
images .. Due to the limited amount of test and training data, the dataset has been
augmented I 04 to triple its original size.
(0014( The proposed model was implemented m a cloud-based environment utilizing a
'CUDA' processor with the A400 GPU and 45GiB RAM used. The GPU processor provided
efficient speed for both neural network training and inferencing. The images were augmented
offline and split into training and testing set. The deep leaming model was developed in a
TensorFiow 2.9.1 environment.
(0015] Figure 2 shows the flow diagram of the basic pre-processing steps performed on the
dataset. The first step 20 I registers the input image to the MNJ 152 I mm standard brain
template. The next process skull stripping 202 removes the non-brain regions like skull, neck,
and eyes from the sMRJ image. This process reduces the tile size and minimizes the
computational resources needed for model training. Additionally, it significantly simplifies
the image, narrowing the area of analysis .
(0016] The bias conection process 203 removes the bias signal caused by the variations in
the capturing device, and the surrounding environment. Removing these signals can
significantly improve model performance. Smoothing 204 is the last step in the basic
preprocessing method, which uses a median filter to smooth the mages. This process effectively removes the noise form the sMRI images and improves the signal-to-noise ratio
(SNR).
100171 Figure 3 shows the normalization process. The data collected from multiple sites may
exhibit varying tissue intensity level due to differences in capturing devices and
environmental factors. This nom1alization process establishes uniform intensity levels across
all images from different sites. It takes the input from the pre-processed data and a library
called "intensity normalization" must be downloaded before we proceeding. Fuzzy C-Means
normalization is then applied to the input image, keeping the white matter intensity level 0-
1.5 as the target intensity. Finally, the histogram of the normalized images is visualized to
assess the effect of the normalization process.
10018) Figure 4 illustrates the resampling process. Resampling altars the dimensions of the
voxels in the image to reduce computational complexity while preserving the essential
inf01mation needed for diagnosis. This process is accomplished using the library called
'Torchio' 401 which includes a method called 'resample' with an argument 'Tl '. This
operatiotlr_esamples alljmages to the_Tl image space 402. In the_proposed method, this
sJgm!Jcantly reduces the fde size of each sMRI image by nearly half.
10019) Figure 5 depicts the data augmentation process. The data augmentation accomplished
using the Torchio library 50 I. Three augmentation methods are chosen to augment the entire
dataset. Intensity based augmentation methods such as Random Gamma 502, Random Bias
503 and Random Noise 504 are selected. These intensity-based augmentation methods
harmonize the data from multiple sites, leading to improved model generalizability. By
employing these three-augmentation techniques, the dataset is expanded to three times its
original size.
[0020) Figure 6 shows the results of the normalization process. Data from three different sites
have been nonnalizcd and their histograms arc presented in the figure. The intensity
variations of different images range from 0 to 1.5, indicating that the multisite data has been
successfitlly nom1alized using the Fuzzy C-Means method.
[00211 Figure 7 shows the network architecture of 3D MANet. The pre-processed,
normalized, resampled and augmented 3-D sMRI images were used as input for training the
deep learning model. The training images were resized to dimensions of (128, 128,60). The
training process achieved the accuracy of I 00%, and the model was trained for I 00 epochs
with the early stopping option enabled.
[0022) Figure 8 presents the accuracy chart of the 30 MANet model, displaying both the
training (blue line) and validation accuracy (red line) of the model. The model achieved a
validation accuracy of98.71%. Figure 9 shows the model loss chart of the 30 MANet model
indicating the training (blue line) and validation loss (red line). The recorded validation loss
corresponding to the above accuracy is 0.2950. The model's sensitivity is 99.58%, specificity
is 97.35%, Fl_score is 0.99, and Area Under Curve (AUC) value is 0.9941. The invention
can be described and explained with the accompanying drawings in which:
Figure I: The steps involved in the overall system of the proposed invention.
Figure 2: The now diagram of the basic pre-processing steps used.
Figure 3: Flow diagram shows the image normalization process
Figure 4: Flow diagram shows the rcsampling process
Figure 5: Flow diagram shows the data augmentation process
Figure 6: Histogram of normalized images
Figure 7: The architecture of 3D MANet
Figure 8: The 3D MANe! model accuracy chart
Figure 9: The 3D MANe! model loss chart
DETAILED DESCRIPTION
Sheet 6
(0023) Pre-processing: The proposed invention starts with the basic pre-processing tasks to
prepare the sMRJ images for the deep learning model. Four main pre-processing steps arc
performed initially: image registration, skull stripping, bias cotTection and smoothing. The
registration process aligns the input sMRI image with the standard image template
MN1152_Tl_lmm (Montreal Neurological Institute), which is an average of 152 normal
MRI scans. During this linear registration, the 12-parametric affine body transformations are
applied. This registration process is accomplished using FSL's Flirt program, with the cost
function set to conelation ratio and the interpolation method as spline interpolation.
(0024] Skull stripping is the next pre-process performed to remove the non-brain region from
the brain image. This operation is essential to reduce the computational complexity and to
minimize the storage requirements. Non-brain regions such as neck, skull, eyes are not
significant for brain analysis and can be removed during this skull stripping process. FSL's
Bet 2 program is used to remove these non-brain regions by taking the fractional intensity
threshold value as an argument.
(0025] Bias concction removes bias field from the MR image. The bias field is a thin signal
across the MR image which may not be visible to the human eye, but it can degrade the
volumetric analysis of cerebral tissues. The proposed invention utilizes the N4 bias correction
technique, supported by the simple ITK tool kit. Comparing to other bias cotTection methods,
N4 bias conection technique yields better results. The next pre-processing step, smoothing
removes the noise from the sMRJ data and helps to recover the underlying signal. The present
invention used a median filter with the kernel size of 3*3*3 .
(0026( Normalization: Image normalization is an important step in preparing the data for the
deep learning model. The goal is to move the intensity value close to 0 while maintaining the
relationship between the voxels. The proposed invention used Fuzzy C-Mean normalization
from the intensity normalization library. Using this normalization method we kept the white
matter range (0-1.5) as the threshold value and nonnalized the sMRI images. As a result, all
the images in the dataset are aligned between the minimum intensity value of 0 and a
maximum intensity value of 1.5. This normalization process is very important to harmonize
the data from multiple sites, before passing it to the neural network .
(0027] Resampling: Resampling changes the dimensions of the voxel and the overall image
size, reducing the computational complexity when handling high-resolution medical images
while still retaining sufficient information for diagnosis and analysis. Resampling also used
to standardize the resolution across different datasets, making them more comparable and
facilitating consistent analysis. In this proposed system, multi-site data is resampled by
aligning it to the 'T I space'. This step reduces the file size of the image data by nearly half,
significantly decreasing the computational requirements needed to manage large volumes of
medical data.
(0028( Data Augmentation: Data augmentation is the process of artificially generating new
data from the existing data to train any deep learning model. By making small changes to the
original data, data augmentation can increase the dataset's size and improve the robustness
and generalization of the deep teaming models. It is also useful for con·ecting imbalanced
datasets and helps to mitigate the problem of model overfilling. In the proposed invention,
intensity-level data augmentation techniques are applied to the sMRI dataset. Three methodsrandom
gamma, random bias, and random noise are introduced, effectively increasing the
dataset size to a more reasonable level. To introduce the model with images of different
intensities and to train effectively, we choose to use intensity level data augmentation
methods. As a result, the augmented dataset is expanded three t.imes the size of the original.
(0029( Model building and training: The proposed model is a simple convolutional neural
network (CNN) model with the sequential structure. It consists of five convolutional layers,
two fully connected layers, three pooling layers and one output layer with a sigmoid
activation function. The internal layers are activated through ReLU activation function. The
dropout layer has been employed to tackle the problem of model overfitting. Hyper
parameters such as kernel initializer and regularizers are used to facilitate the model
convergence and regulate loss values, fm1her preventing model overfitting. Before training,
the dataset is split into training and testing sets with the split ratio of 0.27. The input image is
resized to dimensions of ( 128, 128,60). During training the learning rate is set to 0.000 I, and
the optimizer is configured as Adam. The model is trained for I 00 epochs with the early
stopping option enabled and a patience value 25 is set. The model achieves the training
accuracy of I 00% with the cotTesponding loss value recorded at 0.2086.
(0030] Model testing and prediction: The trained model has been evaluated on the test data,
yielding an accuracy of98.71% and a loss value of0.2950. In addition to accuracy and loss,
other evaluation metrics such as sensitivity, specificity, F !-score, and Area Under Curve
(AUC) were also assessed. The efficiency of the model was evaluated by comparing the
proposed model (3D MANe!) with predefined architecmres including 3D LeNet, 3DAiexNet,
3D ResNet-18. The proposed model, 30 MANet, outperforms all other models, achieving a
top accuracy rate of 98.71%. The below table shows the classification performance of
MANet compared to different models.


0012] We claim,
A Modified AlexNet (MANet) model to diagnose Autism Spectrum Disorder (ASD) from
structural Magnetic Resonance Imaging Images, comprising:
I. We claim the n01malization process for multisite data
• Fuzzy C-Means normalization with respect to the white matter with the
intensity range between (0-1.5)
2. We claim the resampling process of aligning the image to the 'TI space' which
reduces-the-file-size by nearly half
• Reduces the computational requirements needed for the model to train.
3. We claim the parameter values used in intensity level data augmentation methods,
• The parameter log gamma set with the value of 0.8 for Random Gamma
augmentation.
• The parameter values of 0.4 and 3 are set as coefficients and order respectively
for Random Bias augmentation.
• The parameter value between 0 to 0.1 used for Random Noise augmentation.
4. We claim the number of filters in the MANet convolution architecture
• conv3d-7 first convolutional layer with 48 filters results the feature map of
size (64*64*30)
• conv3d I -7 second convolutional layer with 128 filters results the feature
map of size (31 *31*14)
• conv3d_2-7 third convolutional layer with 192 filters results the feature map
of size (15*15*6)
• conv3d_3 -7 fout1h convolutional layer with 192 filters results the feature map
of size (15* 15*6)
• conv3d_ 4 -7 fifth convolutional layer with 128 filters results the feature map
of size (15*15*6)
5. We claim the hyper parameter values used in the 3D MANer architecture
• Kernel regularizer= 12 with the value ofO.OOI
• Kernel initialet='he-nonnal'
6. We claim the training and testing data split ratio of 0.27
7. We claim the model learning rate value ofO.OOOI
8. We claim the depth value for the 3D sMRJ data which gave the highest accuracy in
prediction, depth;6Q slices per image
9. We claim the accuracy value of98. 71% of this classification model in predicting the
autism
I 0. We claim the area under curve value of A UC;0.9941 for this autism diagnosis model

Documents