Segmentace lézí roztroušené sklerózy pomocí hlubokých neuronových sítí
Loading...
Date
Authors
ORCID
Advisor
Referee
Mark
A
Journal Title
Journal ISSN
Volume Title
Publisher
Vysoké učení technické v Brně. Fakulta elektrotechniky a komunikačních technologií
Abstract
Hlavným zámerom tejto diplomovej práce bola automatická segmentácia lézií sklerózy multiplex na snímkoch MRI. V rámci práce boli otestované najnovšie metódy segmentácie s využitím hlbokých neurónových sietí a porovnané prístupy inicializácie váh sietí pomocou preneseného učenia (transfer learning) a samoriadeného učenia (self-supervised learning). Samotný problém automatickej segmentácie lézií sklerózy multiplex je veľmi náročný, a to primárne kvôli vysokej nevyváženosti datasetu (skeny mozgov zvyčajne obsahujú len malé množstvo poškodeného tkaniva). Ďalšou výzvou je manuálna anotácia týchto lézií, nakoľko dvaja rozdielni doktori môžu označiť iné časti mozgu ako poškodené a hodnota Dice Coefficient týchto anotácií je približne 0,86. Možnosť zjednodušenia procesu anotovania lézií automatizáciou by mohlo zlepšiť výpočet množstva lézií, čo by mohlo viesť k zlepšeniu diagnostiky individuálnych pacientov. Našim cieľom bolo navrhnutie dvoch techník využívajúcich transfer learning na predtrénovanie váh, ktoré by neskôr mohli zlepšiť výsledky terajších segmentačných modelov. Teoretická časť opisuje rozdelenie umelej inteligencie, strojového učenia a hlbokých neurónových sietí a ich využitie pri segmentácii obrazu. Následne je popísaná skleróza multiplex, jej typy, symptómy, diagnostika a liečba. Praktická časť začína predspracovaním dát. Najprv boli skeny mozgu upravené na rovnaké rozlíšenie s rovnakou veľkosťou voxelu. Dôvodom tejto úpravy bolo využitie troch odlišných datasetov, v ktorých boli skeny vytvárané rozličnými prístrojmi od rôznych výrobcov. Jeden dataset taktiež obsahoval lebku, a tak bolo nutné jej odstránenie pomocou nástroju FSL pre ponechanie samotného mozgu pacienta. Využívali sme 3D skeny (FLAIR, T1 a T2 modality), ktoré boli postupne rozdelené na individuálne 2D rezy a použité na vstup neurónovej siete s enkodér-dekodér architektúrou. Dataset na trénovanie obsahoval 6720 rezov s rozlíšením 192 x 192 pixelov (po odstránení rezov, ktorých maska neobsahovala žiadnu hodnotu). Využitá loss funkcia bola Combo loss (kombinácia Dice Loss s upravenou Cross-Entropy). Prvá metóda sa zameriavala na využitie predtrénovaných váh z ImageNet datasetu na enkodér U-Net architektúry so zamknutými váhami enkodéra, resp. bez zamknutia a následného porovnania s náhodnou inicializáciou váh. V tomto prípade sme použili len FLAIR modalitu. Transfer learning dokázalo zvýšiť sledovanú metriku z hodnoty približne 0,4 na 0,6. Rozdiel medzi zamknutými a nezamknutými váhami enkodéru sa pohyboval okolo 0,02. Druhá navrhnutá technika používala self-supervised kontext enkodér s Generative Adversarial Networks (GAN) na predtrénovanie váh. Táto sieť využívala všetky tri spomenuté modality aj s prázdnymi rezmi masiek (spolu 23040 obrázkov). Úlohou GAN siete bolo dotvoriť sken mozgu, ktorý bol prekrytý čiernou maskou v tvare šachovnice. Takto naučené váhy boli následne načítané do enkodéru na aplikáciu na náš segmentačný problém. Tento experiment nevykazoval lepšie výsledky, s hodnotou DSC 0,29 a 0,09 (nezamknuté a zamknuté váhy enkodéru). Prudké zníženie metriky mohlo byť spôsobené použitím predtrénovaných váh na vzdialených problémoch (segmentácia a self-supervised kontext enkodér), ako aj zložitosť úlohy kvôli nevyváženému datasetu.
This master thesis focused on automatic segmentation of Multiple Sclerosis (MS) lesions on MRI images. We tested the latest methods of segmentation using Deep Neural Networks and compared the approaches of weight initialization by transfer learning and self-supervised learning. The automatic segmentation of MS lesions is a very challenging task, primarily due to the high imbalance of the dataset (brain scans usually contain only a small amount of damaged tissue). Another challenge is a manual annotation of these lesions, as two different doctors can mark other parts of the brain as damaged and the Dice Coefficient of these annotations is approximately 0.86, which further underlines the complexity of this task. The possibility of simplifying the annotation process by automatization could improve the lesion load determination and might lead to better diagnostic of each individual patient. Our goal was to propose two techniques that use transfer learning to pre-train weights to later improve the performance of existing segmentation models. The theoretical part describes the division of artificial intelligence, machine learning and deep neural networks and their use in image segmentation. Afterwards, the work provides a description of Multiple Sclerosis, its types, symptoms, diagnosis and treatment. The practical part begins with data preprocessing. Firstly, brain scans were adjusted to the same resolution with the same voxel size. This was needed due to the usage of three different datasets, in which the scans had been created by devices from different manufacturers. One dataset also included the skull, therefore it was necessary to remove it by an FSL tool, leaving only the patient's brain in the scan. The preprocessed data were 3D scans (FLAIR, T1 and T2 modalities), which were cut into individual 2D slices and used as an input for the neural network with encoder-decoder architecture. The whole dataset consisted a total of 6,720 slices with a resolution of 192 x 192 pixels for training (after removing slices where the mask was empty). Loss function was Combo loss (combination of Dice Loss with modified Cross-Entropy). The first technique was to use the pre-trained weights from the ImageNet dataset on encoder in U-Net network, with and without locked encoder weights, respectively, and compare the results with random weight initialization. In this case, we used only the FLAIR modality. Transfer learning has proven to increase the metrics from approximately 0.4 to 0.6. The difference between encoder with and without locked weights was about 0.02. The second proposed technique was to use a self-supervised context encoder with Generative Adversarial Networks (GAN) to pre-train the weights. This network used all three modalities also with the empty slices (23,040 slices in total). The purpose of GAN was to recreate the brain image, which was covered by a checkerboard. Weights learned during this training were later loaded for the encoder to apply to our segmentation problem. The following experiment did not show any improvement, with a DSC value of 0.29 and 0.09, with and without a locked encoder, respectively. Such a decrease in performance might have been caused by the use of weights pre-trained on two distant problems (segmentation and self-supervised context encoder) or by difficulty of the task considering the hugely unbalanced dataset.
This master thesis focused on automatic segmentation of Multiple Sclerosis (MS) lesions on MRI images. We tested the latest methods of segmentation using Deep Neural Networks and compared the approaches of weight initialization by transfer learning and self-supervised learning. The automatic segmentation of MS lesions is a very challenging task, primarily due to the high imbalance of the dataset (brain scans usually contain only a small amount of damaged tissue). Another challenge is a manual annotation of these lesions, as two different doctors can mark other parts of the brain as damaged and the Dice Coefficient of these annotations is approximately 0.86, which further underlines the complexity of this task. The possibility of simplifying the annotation process by automatization could improve the lesion load determination and might lead to better diagnostic of each individual patient. Our goal was to propose two techniques that use transfer learning to pre-train weights to later improve the performance of existing segmentation models. The theoretical part describes the division of artificial intelligence, machine learning and deep neural networks and their use in image segmentation. Afterwards, the work provides a description of Multiple Sclerosis, its types, symptoms, diagnosis and treatment. The practical part begins with data preprocessing. Firstly, brain scans were adjusted to the same resolution with the same voxel size. This was needed due to the usage of three different datasets, in which the scans had been created by devices from different manufacturers. One dataset also included the skull, therefore it was necessary to remove it by an FSL tool, leaving only the patient's brain in the scan. The preprocessed data were 3D scans (FLAIR, T1 and T2 modalities), which were cut into individual 2D slices and used as an input for the neural network with encoder-decoder architecture. The whole dataset consisted a total of 6,720 slices with a resolution of 192 x 192 pixels for training (after removing slices where the mask was empty). Loss function was Combo loss (combination of Dice Loss with modified Cross-Entropy). The first technique was to use the pre-trained weights from the ImageNet dataset on encoder in U-Net network, with and without locked encoder weights, respectively, and compare the results with random weight initialization. In this case, we used only the FLAIR modality. Transfer learning has proven to increase the metrics from approximately 0.4 to 0.6. The difference between encoder with and without locked weights was about 0.02. The second proposed technique was to use a self-supervised context encoder with Generative Adversarial Networks (GAN) to pre-train the weights. This network used all three modalities also with the empty slices (23,040 slices in total). The purpose of GAN was to recreate the brain image, which was covered by a checkerboard. Weights learned during this training were later loaded for the encoder to apply to our segmentation problem. The following experiment did not show any improvement, with a DSC value of 0.29 and 0.09, with and without a locked encoder, respectively. Such a decrease in performance might have been caused by the use of weights pre-trained on two distant problems (segmentation and self-supervised context encoder) or by difficulty of the task considering the hugely unbalanced dataset.
Description
Keywords
augmentácia dát, hlboké učenie, konfrontačná generatívna sieť, kontext enkodér, prenesené učenie, samoriadené učenie, segmentácia obrazu, skleróza multiplex, spracovanie medicínskych obrázkov, Tensorflow, Context Encoder, Data Augmentation, Deep Learning, Generative Adversarial Networks, Image Segmentation, Medical Images Processing, Multiple Sclerosis, Self-Supervised Learning, Tensorflow, Transfer Learning
Citation
SASKO, D. Segmentace lézí roztroušené sklerózy pomocí hlubokých neuronových sítí [online]. Brno: Vysoké učení technické v Brně. Fakulta elektrotechniky a komunikačních technologií. 2021.
Document type
Document version
Date of access to the full text
Language of document
en
Study field
bez specializace
Comittee
prof. Ing. Zdeněk Smékal, CSc. (předseda)
prof. Ing. Karel Bartušek, DrSc. (místopředseda)
Ing. Hicham Atassi, Ph.D. (člen)
Ing. David Kubánek, Ph.D. (člen)
doc. Ing. Jaroslav Sklenář, CSc. (člen)
Ing. Petr Ilgner (člen)
Date of acceptance
2021-06-08
Defence
Student prezentoval výsledky své práce a komise byla seznámena s posudky.
Student obhájil bakalářskou práci a odpověděl na otázky členů komise a oponenta.
Otázky oponenta:
Jaký další postup byste navrhoval za účelem zvýšení výkonnosti GAN modelu? Student odpověděl na otázku.
Result of defence
práce byla úspěšně obhájena
Document licence
Standardní licenční smlouva - přístup k plnému textu bez omezení