Estudo de algoritmos de otimização inspirados na natureza aplicados ao treinamento de redes neurais artificiais
Carregando...
Arquivos
Citações na Scopus
Tipo de produção
Dissertação
Data
2021
Autores
Bouzon, M. F.
Orientador
Rodrigues, Paulo Sérgio Silva
Periódico
Título da Revista
ISSN da Revista
Título de Volume
Citação
BOUZON, M. F. Estudo de algoritmos de otimização inspirados na natureza aplicados ao treinamento de redes neurais artificiais. 2021. 117 f. Dissertação (Mestrado em Engenharia Elétrica) - Centro Universitário FEI, São Bernardo do Campo, 2021. Disponível em: https://doi.org/10.31414/EE.2021.D.131321.
Texto completo (DOI)
Palavras-chave
Redes neurais (Computação)
Resumo
Redes Neurais Artificiais (RNAs) são técnicas de aprendizado de máquina e inteligência artificial muito populares, propostas desde os anos 50. Entre seus maiores desafios estão o treinamento de parâmetros tais como pesos, parâmetros das funções de ativação e constantes, assim como dos seus hiper-parâmetros, como a arquitetura das redes e densidade de neurônios por camada. Entre os algoritmos mais conhecidos para a otimização paramétrica das redes estão o Adam e o Retropropagação ou Backpropagation (BP), aplicados sobretudo em arquiteturas populares como o Perceptron multicamadas ou Multilayer Perceptron (MLP), Rede Neural Recorrente ou Recurrent Neural Network (RNN), Long-Short Term Memory (LSTM), Rede neural
de Base Radial ou Radial Basis Function Neural Network (RBFNN), entre muitas outras. Recentemente, o grande sucesso das redes neurais profundas, as chamadas Deep Learnings, bem como das redes totalmente conectadas, tem enfrentado problemas de tempo de treinamento e utilização de hardware especializado. Esses desafios deram novo fôlego à utilização de algoritmos de otimização para o treinamento dessas redes, e mais recentemente aos algoritmos inspirados na natureza, os chamados Inspirados na natureza ou Nature-Inspired (NI). Essa estratégia, embora não seja uma técnica tão recente, ainda não obteve grande atenção de pesquisadores, necessitando hoje de maior número de testes experimentais e avaliação, sobretudo devido ao recente aparecimento de uma gama muito maior de algoritmos NI. Alguns dos elementos que carecem
de atenção, sobretudo para os NI mais recentes, estão relacionados principalmente ao tempo de convergência e estudos sobre o uso de diferentes funções custo. Assim, a presente dissertação de mestrado tem por objetivo realizar testes, comparação e estudos sobre algoritmos NI aplicados ao treinamento de redes neurais. Foram testados algoritmos NI tanto tradicionais quanto recentes, sob vários pontos de vista, incluindo o tempo de convergência e funções objetivos, elementos que receberam até o momento pouca atenção dos pesquisadores em testes prévios. Os resultados mostraram que a utilização de algoritmos NI para treinamento de RNAs tradicionais obtiveram resultados com boa classificação, similares a algoritmos populares como o Adam e o Algoritmo Backpropagation com Momento (BPMA), mas superando esses algoritmos em termos de tempo de convergência na ordem de 20 a mais de 70%, dependendo da rede e dos parâmetros envolvidos. Isso indica que a estratégia de usar algoritmos NI, sobretudo os mais recentes, para treinamento de redes neurais é um método promissor que pode impactar cada vez
mais no tempo e qualidade dos resultados das aplicações recentes e futuras de aprendizado de máquina e inteligência artificial.
Artificial Neural Networks are a popular machine learning and artificial intelligence technique, proposed since the 1950s. Among their greatest challenges is the training of parameters such as weights, parameters of the activation functions and constants, as well as their yperparameters, such as network architecture and density of neurons per layer. Among the best known algorithms for parametric optimization of networks are Adam and BP, applied mainly in popular architectures such as MLP, RNN, LSTM, Feed-forward Neural Network (FNN), RBFNN, among many others. Recently, the great success of deep neural networks, known as Deep Learnings, as well as fully connected networks, has faced problems with training time and the use of specialized hardware. These challenges gave new impetus to the use of optimization algorithms for the training of these networks, and more recently to the algorithms inspired by nature, also called as NI. This strategy, although not a recent technique, has not yet received much attention from researchers, requiring today a greater number of experimental tests and evaluation, mainly due to the recent appearance of a much larger range of algorithms NI. Some of the elements that need attention, especially for the most recent NI, are mainly related to the time of convergence and studies on the use of different cost functions. Thus, the present master’s dissertation aims to perform tests, comparisons, and studies on algorithms NI applied to the training of neural networks. Both traditional and recent NI algorithms were tested, from many perspectives, including convergence time and cost functions, elements that until now have received little attention from researchers in previous tests. The results showed that the use of NI algorithms for the training of traditional RNAs obtained results with good classification, similar to popular algorithms such as Adam and BPMA, but surpassing these algorithms in terms of convergence time in 20 up to 70%, depending on the network and the parameters involved. This indicates that the strategy of using NI algorithms, especially the most recent ones, for training neural networks is a promising method that can impact the time and quality of the results of recent and future machine learning applications and artificial intelligence.
Artificial Neural Networks are a popular machine learning and artificial intelligence technique, proposed since the 1950s. Among their greatest challenges is the training of parameters such as weights, parameters of the activation functions and constants, as well as their yperparameters, such as network architecture and density of neurons per layer. Among the best known algorithms for parametric optimization of networks are Adam and BP, applied mainly in popular architectures such as MLP, RNN, LSTM, Feed-forward Neural Network (FNN), RBFNN, among many others. Recently, the great success of deep neural networks, known as Deep Learnings, as well as fully connected networks, has faced problems with training time and the use of specialized hardware. These challenges gave new impetus to the use of optimization algorithms for the training of these networks, and more recently to the algorithms inspired by nature, also called as NI. This strategy, although not a recent technique, has not yet received much attention from researchers, requiring today a greater number of experimental tests and evaluation, mainly due to the recent appearance of a much larger range of algorithms NI. Some of the elements that need attention, especially for the most recent NI, are mainly related to the time of convergence and studies on the use of different cost functions. Thus, the present master’s dissertation aims to perform tests, comparisons, and studies on algorithms NI applied to the training of neural networks. Both traditional and recent NI algorithms were tested, from many perspectives, including convergence time and cost functions, elements that until now have received little attention from researchers in previous tests. The results showed that the use of NI algorithms for the training of traditional RNAs obtained results with good classification, similar to popular algorithms such as Adam and BPMA, but surpassing these algorithms in terms of convergence time in 20 up to 70%, depending on the network and the parameters involved. This indicates that the strategy of using NI algorithms, especially the most recent ones, for training neural networks is a promising method that can impact the time and quality of the results of recent and future machine learning applications and artificial intelligence.