Frameworks for trading stocks in the financial market using data mining techniques and multi-swarm optimization algorithms for dynamic and multimodal enviroments

Detalhes bibliográficos
Autor(a) principal: BRASILEIRO, Rodrigo de Carvalho
Data de Publicação: 2019
Tipo de documento: Tese
Idioma: eng
Título da fonte: Repositório Institucional da UFPE
Texto Completo: https://repositorio.ufpe.br/handle/123456789/35865
Resumo: Financial time series behave similarly to a data stream, that is, a set of input elements that arrive continuously and sequentially over time. So, a time series may present concept drifts, which is the change in the data generating process. This phenomenon negatively affects forecasting methods that rely on observing the past behavior of the series to predict future values. Many papers report the use of data mining techniques and computational intelligence to predict the future direction of stock prices, uncovering patterns in time series data to support decision making for financial market operations. The traditional optimization algorithms proposed in the literature generally assume that the environment is static, assuming that the time series data generation distribution is the same over the period of interest. Another problem is that, sometimes these methods do not take into account the possibility that the function to be optimized has multiple peaks and, in this case, is represented by multimodal functions. However, multimodality is one of the known features of real-time financial time series optimization problems. Furthermore, several methods involving optimization algorithms have been proposed in the literature, however most of them do not consider real world problems. The main contribution of this work is a decision support system capable of dealing with concept changes and multimodality in the financial time series environment. To achieve this goal, we propose two modelos that aim to find patterns in financial time series, using multi-swarms to improve particle initialization, thus avoiding local optimum in the final optimization phase. In addition, the models use a validation step with the early stopping criteria to avoid overfitting. In contrast to the first proposed model, the second one considers two consecutive generations of populations to detect changes in time series, then a statistical test is used to check for changes in the environment to avoid false positives. Once a change is detected, the second model performs a series of actions to find new patterns, replacing obsolete ones. The patterns discovered by the models are used in conjunction with proposed investment rules to support decisions and help investors maximize profit on their stock market operations. Experiments using 82 stocks from the S&P100 index were tested with a confidence level of 95%, showing that the proposed method is able to improve results when concept drifts are considered.
id UFPE_fe784def577a4c3b224f92785edb3e43
oai_identifier_str oai:repositorio.ufpe.br:123456789/35865
network_acronym_str UFPE
network_name_str Repositório Institucional da UFPE
repository_id_str 2221
spelling BRASILEIRO, Rodrigo de Carvalhohttp://lattes.cnpq.br/2004648403365793http://lattes.cnpq.br/5194381227316437OLIVEIRA, Adriano Lorena Inácio de2019-12-18T19:51:32Z2019-12-18T19:51:32Z2019-05-23BRASILEIRO, Rodrigo de Carvalho. Frameworks for trading stocks in the financial market using data mining techniques and multi-swarm optimization algorithms for dynamic and multimodal enviroments. 2019. Tese (Doutorado em Ciência da Computação) – Universidade Federal de Pernambuco, Recife, 2019.https://repositorio.ufpe.br/handle/123456789/35865Financial time series behave similarly to a data stream, that is, a set of input elements that arrive continuously and sequentially over time. So, a time series may present concept drifts, which is the change in the data generating process. This phenomenon negatively affects forecasting methods that rely on observing the past behavior of the series to predict future values. Many papers report the use of data mining techniques and computational intelligence to predict the future direction of stock prices, uncovering patterns in time series data to support decision making for financial market operations. The traditional optimization algorithms proposed in the literature generally assume that the environment is static, assuming that the time series data generation distribution is the same over the period of interest. Another problem is that, sometimes these methods do not take into account the possibility that the function to be optimized has multiple peaks and, in this case, is represented by multimodal functions. However, multimodality is one of the known features of real-time financial time series optimization problems. Furthermore, several methods involving optimization algorithms have been proposed in the literature, however most of them do not consider real world problems. The main contribution of this work is a decision support system capable of dealing with concept changes and multimodality in the financial time series environment. To achieve this goal, we propose two modelos that aim to find patterns in financial time series, using multi-swarms to improve particle initialization, thus avoiding local optimum in the final optimization phase. In addition, the models use a validation step with the early stopping criteria to avoid overfitting. In contrast to the first proposed model, the second one considers two consecutive generations of populations to detect changes in time series, then a statistical test is used to check for changes in the environment to avoid false positives. Once a change is detected, the second model performs a series of actions to find new patterns, replacing obsolete ones. The patterns discovered by the models are used in conjunction with proposed investment rules to support decisions and help investors maximize profit on their stock market operations. Experiments using 82 stocks from the S&P100 index were tested with a confidence level of 95%, showing that the proposed method is able to improve results when concept drifts are considered.As séries temporais financeiras comportam-se de forma semelhante a um fluxo de dados, ou seja, um conjunto de elementos de entrada que chegam de forma contínua e sequencial ao longo do tempo. Então, uma série temporal pode apresentar uma mudança de conceito, que é a mudança no processo gerador dos dados. Este fenômeno afeta negativamente os métodos de previsão que se baseiam na observação do comportamento passado da série para prever valores futuros. Muitos trabalhos relatam o uso de técnicas de mineração de dados e inteligência computacional para prever a direção futura dos preços das ações, descobrindo padrões nos dados das séries temporais para fornecer suporte a decisões para as operações realizadas no mercado financeiro. Os algoritmos de otimização tradicionais propostos na literatura geralmente consideram que o ambiente é estático, supondo que a distribuição geradora dos dados das séries temporais seja a mesma ao longo do período de interesse. Outro problema é que algumas vezes estes métodos não levam em consideração a possibilidade da função a ser otimizada ter múltiplos picos e, neste caso, ser representada através de funções multimodais. No entanto, a multimodalidade é uma das características conhecidas dos problemas de otimização em séries temporais financeiras no mundo real. Além disso, diversos métodos envolvendo algoritmos de otimização foram propostos na literatura, no entanto a maior parte deles não levam em consideração os problemas do mundo real. A principal contribuição deste trabalho é um sistema de suporte a decisão capaz de tratar as mudanças de conceito no ambiente e multimodalidade das séries temporais financeiras. Para atingir foi proposto dois modelos que visam encontrar padrões em séries temporais financeiras, usando multi-enxames para melhorar a inicialização das partículas, portanto evitando ótimos locais na fase final da otimização. Além disso, os modelos usam um conjunto de validação com o critério de parada antecipada para evitar overfitting. Em contraste com o primeiro modelo proposto, o segundo considera duas gerações consecutivas das populações para detectar as mudanças nas séries temporais, posteriormente um teste estatístico é usado para verificar se houve mudanças no ambiente, procurando evitar falsos positivos. Após detectar uma mudança, o segundo modelo executa uma série de medidas com o objetivo de encontrar novos padrões, substituindo os obsoletos. Os padrões descobertos pelos modelos são usados em conjunto com as regras de investimento propostas para apoiar as decisões e ajudar os investidores a maximizar o lucro em suas operações no mercado de ações. Experimentos usando 82 ações do índice S&P100 foram testados com nível de confiança de 95%, mostrando que o método proposto é capaz de melhorar os resultados quando as mudanças de conceito são consideradas.engUniversidade Federal de PernambucoPrograma de Pos Graduacao em Ciencia da ComputacaoUFPEBrasilAttribution-NonCommercial-NoDerivs 3.0 Brazilhttp://creativecommons.org/licenses/by-nc-nd/3.0/br/info:eu-repo/semantics/openAccessInteligência computacionalMineração de dadosFluxo contínuo de dadosFrameworks for trading stocks in the financial market using data mining techniques and multi-swarm optimization algorithms for dynamic and multimodal enviromentsinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/doctoralThesisdoutoradoreponame:Repositório Institucional da UFPEinstname:Universidade Federal de Pernambuco (UFPE)instacron:UFPETEXTTESE Rodrigo de Carvalho Brasileiro.pdf.txtTESE Rodrigo de Carvalho Brasileiro.pdf.txtExtracted texttext/plain294339https://repositorio.ufpe.br/bitstream/123456789/35865/4/TESE%20Rodrigo%20de%20Carvalho%20Brasileiro.pdf.txt4cbb13ef311434293449a5978751b73eMD54THUMBNAILTESE Rodrigo de Carvalho Brasileiro.pdf.jpgTESE Rodrigo de Carvalho Brasileiro.pdf.jpgGenerated Thumbnailimage/jpeg1298https://repositorio.ufpe.br/bitstream/123456789/35865/5/TESE%20Rodrigo%20de%20Carvalho%20Brasileiro.pdf.jpg27d40a73ae35edadd2c395e33a678d8aMD55ORIGINALTESE Rodrigo de Carvalho Brasileiro.pdfTESE Rodrigo de Carvalho Brasileiro.pdfapplication/pdf3379235https://repositorio.ufpe.br/bitstream/123456789/35865/1/TESE%20Rodrigo%20de%20Carvalho%20Brasileiro.pdf794fcf0edef2fef66a428d35f1dacb19MD51CC-LICENSElicense_rdflicense_rdfapplication/rdf+xml; charset=utf-8811https://repositorio.ufpe.br/bitstream/123456789/35865/2/license_rdfe39d27027a6cc9cb039ad269a5db8e34MD52LICENSElicense.txtlicense.txttext/plain; charset=utf-82310https://repositorio.ufpe.br/bitstream/123456789/35865/3/license.txtbd573a5ca8288eb7272482765f819534MD53123456789/358652019-12-19 02:15:43.79oai:repositorio.ufpe.br:123456789/35865TGljZW7Dp2EgZGUgRGlzdHJpYnVpw6fDo28gTsOjbyBFeGNsdXNpdmEKClRvZG8gZGVwb3NpdGFudGUgZGUgbWF0ZXJpYWwgbm8gUmVwb3NpdMOzcmlvIEluc3RpdHVjaW9uYWwgKFJJKSBkZXZlIGNvbmNlZGVyLCDDoCBVbml2ZXJzaWRhZGUgRmVkZXJhbCBkZSBQZXJuYW1idWNvIChVRlBFKSwgdW1hIExpY2Vuw6dhIGRlIERpc3RyaWJ1acOnw6NvIE7Do28gRXhjbHVzaXZhIHBhcmEgbWFudGVyIGUgdG9ybmFyIGFjZXNzw612ZWlzIG9zIHNldXMgZG9jdW1lbnRvcywgZW0gZm9ybWF0byBkaWdpdGFsLCBuZXN0ZSByZXBvc2l0w7NyaW8uCgpDb20gYSBjb25jZXNzw6NvIGRlc3RhIGxpY2Vuw6dhIG7Do28gZXhjbHVzaXZhLCBvIGRlcG9zaXRhbnRlIG1hbnTDqW0gdG9kb3Mgb3MgZGlyZWl0b3MgZGUgYXV0b3IuCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwoKTGljZW7Dp2EgZGUgRGlzdHJpYnVpw6fDo28gTsOjbyBFeGNsdXNpdmEKCkFvIGNvbmNvcmRhciBjb20gZXN0YSBsaWNlbsOnYSBlIGFjZWl0w6EtbGEsIHZvY8OqIChhdXRvciBvdSBkZXRlbnRvciBkb3MgZGlyZWl0b3MgYXV0b3JhaXMpOgoKYSkgRGVjbGFyYSBxdWUgY29uaGVjZSBhIHBvbMOtdGljYSBkZSBjb3B5cmlnaHQgZGEgZWRpdG9yYSBkbyBzZXUgZG9jdW1lbnRvOwpiKSBEZWNsYXJhIHF1ZSBjb25oZWNlIGUgYWNlaXRhIGFzIERpcmV0cml6ZXMgcGFyYSBvIFJlcG9zaXTDs3JpbyBJbnN0aXR1Y2lvbmFsIGRhIFVGUEU7CmMpIENvbmNlZGUgw6AgVUZQRSBvIGRpcmVpdG8gbsOjbyBleGNsdXNpdm8gZGUgYXJxdWl2YXIsIHJlcHJvZHV6aXIsIGNvbnZlcnRlciAoY29tbyBkZWZpbmlkbyBhIHNlZ3VpciksIGNvbXVuaWNhciBlL291IGRpc3RyaWJ1aXIsIG5vIFJJLCBvIGRvY3VtZW50byBlbnRyZWd1ZSAoaW5jbHVpbmRvIG8gcmVzdW1vL2Fic3RyYWN0KSBlbSBmb3JtYXRvIGRpZ2l0YWwgb3UgcG9yIG91dHJvIG1laW87CmQpIERlY2xhcmEgcXVlIGF1dG9yaXphIGEgVUZQRSBhIGFycXVpdmFyIG1haXMgZGUgdW1hIGPDs3BpYSBkZXN0ZSBkb2N1bWVudG8gZSBjb252ZXJ0w6otbG8sIHNlbSBhbHRlcmFyIG8gc2V1IGNvbnRlw7pkbywgcGFyYSBxdWFscXVlciBmb3JtYXRvIGRlIGZpY2hlaXJvLCBtZWlvIG91IHN1cG9ydGUsIHBhcmEgZWZlaXRvcyBkZSBzZWd1cmFuw6dhLCBwcmVzZXJ2YcOnw6NvIChiYWNrdXApIGUgYWNlc3NvOwplKSBEZWNsYXJhIHF1ZSBvIGRvY3VtZW50byBzdWJtZXRpZG8gw6kgbyBzZXUgdHJhYmFsaG8gb3JpZ2luYWwgZSBxdWUgZGV0w6ltIG8gZGlyZWl0byBkZSBjb25jZWRlciBhIHRlcmNlaXJvcyBvcyBkaXJlaXRvcyBjb250aWRvcyBuZXN0YSBsaWNlbsOnYS4gRGVjbGFyYSB0YW1iw6ltIHF1ZSBhIGVudHJlZ2EgZG8gZG9jdW1lbnRvIG7Do28gaW5mcmluZ2Ugb3MgZGlyZWl0b3MgZGUgb3V0cmEgcGVzc29hIG91IGVudGlkYWRlOwpmKSBEZWNsYXJhIHF1ZSwgbm8gY2FzbyBkbyBkb2N1bWVudG8gc3VibWV0aWRvIGNvbnRlciBtYXRlcmlhbCBkbyBxdWFsIG7Do28gZGV0w6ltIG9zIGRpcmVpdG9zIGRlCmF1dG9yLCBvYnRldmUgYSBhdXRvcml6YcOnw6NvIGlycmVzdHJpdGEgZG8gcmVzcGVjdGl2byBkZXRlbnRvciBkZXNzZXMgZGlyZWl0b3MgcGFyYSBjZWRlciDDoApVRlBFIG9zIGRpcmVpdG9zIHJlcXVlcmlkb3MgcG9yIGVzdGEgTGljZW7Dp2EgZSBhdXRvcml6YXIgYSB1bml2ZXJzaWRhZGUgYSB1dGlsaXrDoS1sb3MgbGVnYWxtZW50ZS4gRGVjbGFyYSB0YW1iw6ltIHF1ZSBlc3NlIG1hdGVyaWFsIGN1am9zIGRpcmVpdG9zIHPDo28gZGUgdGVyY2Vpcm9zIGVzdMOhIGNsYXJhbWVudGUgaWRlbnRpZmljYWRvIGUgcmVjb25oZWNpZG8gbm8gdGV4dG8gb3UgY29udGXDumRvIGRvIGRvY3VtZW50byBlbnRyZWd1ZTsKZykgU2UgbyBkb2N1bWVudG8gZW50cmVndWUgw6kgYmFzZWFkbyBlbSB0cmFiYWxobyBmaW5hbmNpYWRvIG91IGFwb2lhZG8gcG9yIG91dHJhIGluc3RpdHVpw6fDo28gcXVlIG7Do28gYSBVRlBFLCBkZWNsYXJhIHF1ZSBjdW1wcml1IHF1YWlzcXVlciBvYnJpZ2HDp8O1ZXMgZXhpZ2lkYXMgcGVsbyByZXNwZWN0aXZvIGNvbnRyYXRvIG91IGFjb3Jkby4KCkEgVUZQRSBpZGVudGlmaWNhcsOhIGNsYXJhbWVudGUgbyhzKSBub21lKHMpIGRvKHMpIGF1dG9yIChlcykgZG9zIGRpcmVpdG9zIGRvIGRvY3VtZW50byBlbnRyZWd1ZSBlIG7Do28gZmFyw6EgcXVhbHF1ZXIgYWx0ZXJhw6fDo28sIHBhcmEgYWzDqW0gZG8gcHJldmlzdG8gbmEgYWzDrW5lYSBjKS4KRepositório InstitucionalPUBhttps://repositorio.ufpe.br/oai/requestattena@ufpe.bropendoar:22212019-12-19T05:15:43Repositório Institucional da UFPE - Universidade Federal de Pernambuco (UFPE)false
dc.title.pt_BR.fl_str_mv Frameworks for trading stocks in the financial market using data mining techniques and multi-swarm optimization algorithms for dynamic and multimodal enviroments
title Frameworks for trading stocks in the financial market using data mining techniques and multi-swarm optimization algorithms for dynamic and multimodal enviroments
spellingShingle Frameworks for trading stocks in the financial market using data mining techniques and multi-swarm optimization algorithms for dynamic and multimodal enviroments
BRASILEIRO, Rodrigo de Carvalho
Inteligência computacional
Mineração de dados
Fluxo contínuo de dados
title_short Frameworks for trading stocks in the financial market using data mining techniques and multi-swarm optimization algorithms for dynamic and multimodal enviroments
title_full Frameworks for trading stocks in the financial market using data mining techniques and multi-swarm optimization algorithms for dynamic and multimodal enviroments
title_fullStr Frameworks for trading stocks in the financial market using data mining techniques and multi-swarm optimization algorithms for dynamic and multimodal enviroments
title_full_unstemmed Frameworks for trading stocks in the financial market using data mining techniques and multi-swarm optimization algorithms for dynamic and multimodal enviroments
title_sort Frameworks for trading stocks in the financial market using data mining techniques and multi-swarm optimization algorithms for dynamic and multimodal enviroments
author BRASILEIRO, Rodrigo de Carvalho
author_facet BRASILEIRO, Rodrigo de Carvalho
author_role author
dc.contributor.authorLattes.pt_BR.fl_str_mv http://lattes.cnpq.br/2004648403365793
dc.contributor.advisorLattes.pt_BR.fl_str_mv http://lattes.cnpq.br/5194381227316437
dc.contributor.author.fl_str_mv BRASILEIRO, Rodrigo de Carvalho
dc.contributor.advisor1.fl_str_mv OLIVEIRA, Adriano Lorena Inácio de
contributor_str_mv OLIVEIRA, Adriano Lorena Inácio de
dc.subject.por.fl_str_mv Inteligência computacional
Mineração de dados
Fluxo contínuo de dados
topic Inteligência computacional
Mineração de dados
Fluxo contínuo de dados
description Financial time series behave similarly to a data stream, that is, a set of input elements that arrive continuously and sequentially over time. So, a time series may present concept drifts, which is the change in the data generating process. This phenomenon negatively affects forecasting methods that rely on observing the past behavior of the series to predict future values. Many papers report the use of data mining techniques and computational intelligence to predict the future direction of stock prices, uncovering patterns in time series data to support decision making for financial market operations. The traditional optimization algorithms proposed in the literature generally assume that the environment is static, assuming that the time series data generation distribution is the same over the period of interest. Another problem is that, sometimes these methods do not take into account the possibility that the function to be optimized has multiple peaks and, in this case, is represented by multimodal functions. However, multimodality is one of the known features of real-time financial time series optimization problems. Furthermore, several methods involving optimization algorithms have been proposed in the literature, however most of them do not consider real world problems. The main contribution of this work is a decision support system capable of dealing with concept changes and multimodality in the financial time series environment. To achieve this goal, we propose two modelos that aim to find patterns in financial time series, using multi-swarms to improve particle initialization, thus avoiding local optimum in the final optimization phase. In addition, the models use a validation step with the early stopping criteria to avoid overfitting. In contrast to the first proposed model, the second one considers two consecutive generations of populations to detect changes in time series, then a statistical test is used to check for changes in the environment to avoid false positives. Once a change is detected, the second model performs a series of actions to find new patterns, replacing obsolete ones. The patterns discovered by the models are used in conjunction with proposed investment rules to support decisions and help investors maximize profit on their stock market operations. Experiments using 82 stocks from the S&P100 index were tested with a confidence level of 95%, showing that the proposed method is able to improve results when concept drifts are considered.
publishDate 2019
dc.date.accessioned.fl_str_mv 2019-12-18T19:51:32Z
dc.date.available.fl_str_mv 2019-12-18T19:51:32Z
dc.date.issued.fl_str_mv 2019-05-23
dc.type.status.fl_str_mv info:eu-repo/semantics/publishedVersion
dc.type.driver.fl_str_mv info:eu-repo/semantics/doctoralThesis
format doctoralThesis
status_str publishedVersion
dc.identifier.citation.fl_str_mv BRASILEIRO, Rodrigo de Carvalho. Frameworks for trading stocks in the financial market using data mining techniques and multi-swarm optimization algorithms for dynamic and multimodal enviroments. 2019. Tese (Doutorado em Ciência da Computação) – Universidade Federal de Pernambuco, Recife, 2019.
dc.identifier.uri.fl_str_mv https://repositorio.ufpe.br/handle/123456789/35865
identifier_str_mv BRASILEIRO, Rodrigo de Carvalho. Frameworks for trading stocks in the financial market using data mining techniques and multi-swarm optimization algorithms for dynamic and multimodal enviroments. 2019. Tese (Doutorado em Ciência da Computação) – Universidade Federal de Pernambuco, Recife, 2019.
url https://repositorio.ufpe.br/handle/123456789/35865
dc.language.iso.fl_str_mv eng
language eng
dc.rights.driver.fl_str_mv Attribution-NonCommercial-NoDerivs 3.0 Brazil
http://creativecommons.org/licenses/by-nc-nd/3.0/br/
info:eu-repo/semantics/openAccess
rights_invalid_str_mv Attribution-NonCommercial-NoDerivs 3.0 Brazil
http://creativecommons.org/licenses/by-nc-nd/3.0/br/
eu_rights_str_mv openAccess
dc.publisher.none.fl_str_mv Universidade Federal de Pernambuco
dc.publisher.program.fl_str_mv Programa de Pos Graduacao em Ciencia da Computacao
dc.publisher.initials.fl_str_mv UFPE
dc.publisher.country.fl_str_mv Brasil
publisher.none.fl_str_mv Universidade Federal de Pernambuco
dc.source.none.fl_str_mv reponame:Repositório Institucional da UFPE
instname:Universidade Federal de Pernambuco (UFPE)
instacron:UFPE
instname_str Universidade Federal de Pernambuco (UFPE)
instacron_str UFPE
institution UFPE
reponame_str Repositório Institucional da UFPE
collection Repositório Institucional da UFPE
bitstream.url.fl_str_mv https://repositorio.ufpe.br/bitstream/123456789/35865/4/TESE%20Rodrigo%20de%20Carvalho%20Brasileiro.pdf.txt
https://repositorio.ufpe.br/bitstream/123456789/35865/5/TESE%20Rodrigo%20de%20Carvalho%20Brasileiro.pdf.jpg
https://repositorio.ufpe.br/bitstream/123456789/35865/1/TESE%20Rodrigo%20de%20Carvalho%20Brasileiro.pdf
https://repositorio.ufpe.br/bitstream/123456789/35865/2/license_rdf
https://repositorio.ufpe.br/bitstream/123456789/35865/3/license.txt
bitstream.checksum.fl_str_mv 4cbb13ef311434293449a5978751b73e
27d40a73ae35edadd2c395e33a678d8a
794fcf0edef2fef66a428d35f1dacb19
e39d27027a6cc9cb039ad269a5db8e34
bd573a5ca8288eb7272482765f819534
bitstream.checksumAlgorithm.fl_str_mv MD5
MD5
MD5
MD5
MD5
repository.name.fl_str_mv Repositório Institucional da UFPE - Universidade Federal de Pernambuco (UFPE)
repository.mail.fl_str_mv attena@ufpe.br
_version_ 1802310679850385408