| | SLO | ENG | Cookies and privacy

Bigger font | Smaller font

Show document

Title:Metoda prilagodljivega uglaševanja slojev konvolucijskih nevronskih mrež pri strojnem učenju s prenosom znanja : doktorska disertacija
Authors:Vrbančič, Grega (Author)
Podgorelec, Vili (Mentor) More about this mentor... New window
Files:.pdf DOK_Vrbancic_Grega_2021.pdf (5,35 MB)
MD5: F6160109D814FC296BAB8822A55BC6BD
 
Language:Slovenian
Work type:Doctoral dissertation (mb31)
Typology:2.08 - Doctoral Dissertation
Organization:FERI - Faculty of Electrical Engineering and Computer Science
Abstract:V doktorski disertaciji predstavimo problematiko izbire uglaševanih slojev konvolucijskih nevronskih mrež pri strojnem učenju s prenosom znanja. Z izvedeno analizo vpliva izbire uglaševanih slojev konvolucijske nevronske mreže na uspešnost učenja potrdimo domnevo, da je primerna izbira uglaševanih slojev s ciljem doseganja visoke klasifikacijske uspešnosti odvisna od izbrane arhitekture konvolucijske nevronske mreže ter ciljnega problema oz. izbrane podatkovne zbirke. Z namenom naslovitve problema izbire uglaševanih slojev razvijemo in predlagamo prilagodljivo metodo DEFT, ki temelji na algoritmu diferencialne evolucije in deluje popolnoma samodejno, ne glede na uporabljeno arhitekturo konvolucijske nevronske mreže ali ciljni problem. Zaradi velike časovne kompleksnosti predlagane metode v nadaljevanju razvijemo in predlagamo na funkciji izgube temelječo metriko LDM, ki v zgodnji fazi učenja uspešno zaznava manj primerne izbire uglaševanih slojev, kar nam omogoča, da za zaznane manj primerne izbire uglaševanih slojev predčasno zaključimo učenje in na tak način zmanjšamo časovno zahtevnost predlagane metode. Uspešnost predlagane metode ovrednotimo z uporabo treh različnih arhitektur globokih konvolucijskih mrež nad tremi raznolikimi slikovnimi podatkovnimi zbirkami. Klasifikacijsko uspešnost predlagane metode z in brez uporabe metrike LDM smo primerjali s klasičnimi pristopi učenja globokih konvolucijskih nevronskih mrež. Primerjavo izvedemo z uporabo najpogostejših klasifikacijskih metrik, časom, potrebnim za učenje, ter porabljenim številom epoh. Rezultate smo preverili z uporabo klasičnih metod statistične analize kot tudi z naprednim pristopom Bayesove analize. Izsledki slednje so potrdili tezo, da je mogoče z uporabo metode prilagodljivega uglaševanja slojev konvolucijske nevronske mreže uspešno nasloviti problem izbire slojev ter da lahko z uporabo metrike LDM za zaznavo manj primernih izbir uglaševanih slojev učinkovito zmanjšamo število epoh, potrebnih za učenje, ob doseganju primerljivih rezultatov.
Keywords:strojno učenje, globoko učenje, učenje s prenosom znanja, klasifikacija, uglaševanje, optimizacija
Year of publishing:2021
Place of performance:Maribor
Publisher:[G. Vrbančič]
Number of pages:XXV, 166 str.
Source:Maribor
UDC:004.032.26:004.85(043.3)
COBISS_ID:82430723 New window
Views:157
Downloads:20
Metadata:XML RDF-CHPDL DC-XML DC-RDF
Categories:KTFMB - FERI
:
  
Average score:(0 votes)
Your score:Voting is allowed only for logged in users.
Share:AddThis
AddThis uses cookies that require your consent. Edit consent...

Hover the mouse pointer over a document title to show the abstract or click on the title to get all document metadata.

Licences

License:CC BY-NC-SA 4.0, Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International
Link:http://creativecommons.org/licenses/by-nc-sa/4.0/
Description:A Creative Commons license that bans commercial use and requires the user to release any modified works under this license.
Licensing start date:10.06.2021

Secondary language

Language:English
Title:Method for adaptive fine-tuning of convolutional neural network layers using transfer learning
Abstract:In this Doctoral Dissertation, we present the problem of selecting fine-tunable layers when utilizing transfer learning with the fine-tuning approach for training deep convolutional neural networks. With the conducted empirical analysis of layer selection impact on the training performance, we confirmed the assumption that the most suitable selection of fine-tuned layers depends on the chosen convolutional neural network architecture, as well as on the target problem. In order to address the problem of selecting the most suitable combination of fine-tunable layers, we developed and proposed an adaptive method, DEFT, based on a differential evolution algorithm, which works in a straightforward automatic manner using different convolutional neural network architectures. Due to the high time complexity of the proposed method, we developed and proposed a metric derived from the loss value, which is capable of detecting less suitable selections of fine-tunable layers at an early stage of training, which allows us to terminate training early, and, thus, reduce the time complexity of the proposed method. The performance of the proposed method was evaluated by utilizing three different convolutional neural network architectures against three different image datasets. Classification performance of the proposed DEFT method, with or without the proposed metric LDM, was compared against conventional approaches for training convolutional neural networks. The performance comparison was conducted using the most common classification metrics, consumed time for training, and consumed number of epochs. The statistical analysis of the obtained results was conducted using conventional statistical methods, as well as modern Bayesian analysis based approaches. The results confirmed the initial thesis that the problem of layer selection when utilizing transfer learning with fine-tuning, can be addressed successfully using the proposed adaptive DEFT method, and that utilization of the proposed LDM metric reduces the number of epochs needed for training effectively, while achieving comparable results.
Keywords:machine learning, deep learning, transfer learning, classification, fine-tuning, optimization


Comments

Leave comment

You have to log in to leave a comment.

Comments (0)
0 - 0 / 0
 
There are no comments!

Back
Logos of partners University of Maribor University of Ljubljana University of Primorska University of Nova Gorica