Real Time Delta Extraction Based on Triggers to Support Data Warehousing
Autor(a) principal: | |
---|---|
Data de Publicação: | 2013 |
Outros Autores: | , , , , |
Tipo de documento: | Artigo de conferência |
Idioma: | eng |
Título da fonte: | Repositório Institucional da UNESP |
Texto Completo: | http://dx.doi.org/10.1109/PDCAT.2013.52 http://hdl.handle.net/11449/197452 |
Resumo: | Nowadays large corporations require integrated data from diverse sources, leading to the use of data warehouse architectures for this purpose. To bypass problems related to the use of computational resources to process large volumes of data, an ETL (Extract, Transform and Load) technique with zero latency can be used, that works by constantly processing small data loads. Among the extraction techniques of the zero latency ETL are the use of logs, triggers, materialized views and timestamps. This paper proposes a structure capable of performing this task by means of triggers and a tool developed for the automatic generation of the SQL (Structured Query Language) code to create these trigger, besides showing its performance and comparing it to other techniques. Said method is relevant for the extraction of portions of selected information as it permits to combine conventional and real time ETL techniques. |
id |
UNSP_b3055715db39a1796c4cf203cfab79e3 |
---|---|
oai_identifier_str |
oai:repositorio.unesp.br:11449/197452 |
network_acronym_str |
UNSP |
network_name_str |
Repositório Institucional da UNESP |
repository_id_str |
2946 |
spelling |
Real Time Delta Extraction Based on Triggers to Support Data Warehousingdata warehouseETL (Extract Transform and Load)CDC (Change Data Capture)triggerautomatic code generationNowadays large corporations require integrated data from diverse sources, leading to the use of data warehouse architectures for this purpose. To bypass problems related to the use of computational resources to process large volumes of data, an ETL (Extract, Transform and Load) technique with zero latency can be used, that works by constantly processing small data loads. Among the extraction techniques of the zero latency ETL are the use of logs, triggers, materialized views and timestamps. This paper proposes a structure capable of performing this task by means of triggers and a tool developed for the automatic generation of the SQL (Structured Query Language) code to create these trigger, besides showing its performance and comparing it to other techniques. Said method is relevant for the extraction of portions of selected information as it permits to combine conventional and real time ETL techniques.Sao Paulo State Univ, Dept Ciencias Computacao & Estat, Sao Paulo, BrazilSao Paulo State Univ, Dept Ciencias Computacao & Estat, Sao Paulo, BrazilIeeeUniversidade Estadual Paulista (Unesp)Valencio, Carlos Roberto [UNESP]Marioto, Matheus Henrique [UNESP]Donega Zafalon, Geraldo Francisco [UNESP]Machado, Jose Marcio [UNESP]Momente, Julio Cesar [UNESP]Horng, S. J.2020-12-10T22:32:01Z2020-12-10T22:32:01Z2013-01-01info:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/conferenceObject293-297http://dx.doi.org/10.1109/PDCAT.2013.522013 International Conference On Parallel And Distributed Computing, Applications And Technologies (pdcat). New York: Ieee, p. 293-297, 2013.http://hdl.handle.net/11449/19745210.1109/PDCAT.2013.52WOS:000361018500046Web of Sciencereponame:Repositório Institucional da UNESPinstname:Universidade Estadual Paulista (UNESP)instacron:UNESPeng2013 International Conference On Parallel And Distributed Computing, Applications And Technologies (pdcat)info:eu-repo/semantics/openAccess2021-10-23T14:47:59Zoai:repositorio.unesp.br:11449/197452Repositório InstitucionalPUBhttp://repositorio.unesp.br/oai/requestopendoar:29462021-10-23T14:47:59Repositório Institucional da UNESP - Universidade Estadual Paulista (UNESP)false |
dc.title.none.fl_str_mv |
Real Time Delta Extraction Based on Triggers to Support Data Warehousing |
title |
Real Time Delta Extraction Based on Triggers to Support Data Warehousing |
spellingShingle |
Real Time Delta Extraction Based on Triggers to Support Data Warehousing Valencio, Carlos Roberto [UNESP] data warehouse ETL (Extract Transform and Load) CDC (Change Data Capture) trigger automatic code generation |
title_short |
Real Time Delta Extraction Based on Triggers to Support Data Warehousing |
title_full |
Real Time Delta Extraction Based on Triggers to Support Data Warehousing |
title_fullStr |
Real Time Delta Extraction Based on Triggers to Support Data Warehousing |
title_full_unstemmed |
Real Time Delta Extraction Based on Triggers to Support Data Warehousing |
title_sort |
Real Time Delta Extraction Based on Triggers to Support Data Warehousing |
author |
Valencio, Carlos Roberto [UNESP] |
author_facet |
Valencio, Carlos Roberto [UNESP] Marioto, Matheus Henrique [UNESP] Donega Zafalon, Geraldo Francisco [UNESP] Machado, Jose Marcio [UNESP] Momente, Julio Cesar [UNESP] Horng, S. J. |
author_role |
author |
author2 |
Marioto, Matheus Henrique [UNESP] Donega Zafalon, Geraldo Francisco [UNESP] Machado, Jose Marcio [UNESP] Momente, Julio Cesar [UNESP] Horng, S. J. |
author2_role |
author author author author author |
dc.contributor.none.fl_str_mv |
Universidade Estadual Paulista (Unesp) |
dc.contributor.author.fl_str_mv |
Valencio, Carlos Roberto [UNESP] Marioto, Matheus Henrique [UNESP] Donega Zafalon, Geraldo Francisco [UNESP] Machado, Jose Marcio [UNESP] Momente, Julio Cesar [UNESP] Horng, S. J. |
dc.subject.por.fl_str_mv |
data warehouse ETL (Extract Transform and Load) CDC (Change Data Capture) trigger automatic code generation |
topic |
data warehouse ETL (Extract Transform and Load) CDC (Change Data Capture) trigger automatic code generation |
description |
Nowadays large corporations require integrated data from diverse sources, leading to the use of data warehouse architectures for this purpose. To bypass problems related to the use of computational resources to process large volumes of data, an ETL (Extract, Transform and Load) technique with zero latency can be used, that works by constantly processing small data loads. Among the extraction techniques of the zero latency ETL are the use of logs, triggers, materialized views and timestamps. This paper proposes a structure capable of performing this task by means of triggers and a tool developed for the automatic generation of the SQL (Structured Query Language) code to create these trigger, besides showing its performance and comparing it to other techniques. Said method is relevant for the extraction of portions of selected information as it permits to combine conventional and real time ETL techniques. |
publishDate |
2013 |
dc.date.none.fl_str_mv |
2013-01-01 2020-12-10T22:32:01Z 2020-12-10T22:32:01Z |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/conferenceObject |
format |
conferenceObject |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
http://dx.doi.org/10.1109/PDCAT.2013.52 2013 International Conference On Parallel And Distributed Computing, Applications And Technologies (pdcat). New York: Ieee, p. 293-297, 2013. http://hdl.handle.net/11449/197452 10.1109/PDCAT.2013.52 WOS:000361018500046 |
url |
http://dx.doi.org/10.1109/PDCAT.2013.52 http://hdl.handle.net/11449/197452 |
identifier_str_mv |
2013 International Conference On Parallel And Distributed Computing, Applications And Technologies (pdcat). New York: Ieee, p. 293-297, 2013. 10.1109/PDCAT.2013.52 WOS:000361018500046 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
2013 International Conference On Parallel And Distributed Computing, Applications And Technologies (pdcat) |
dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/openAccess |
eu_rights_str_mv |
openAccess |
dc.format.none.fl_str_mv |
293-297 |
dc.publisher.none.fl_str_mv |
Ieee |
publisher.none.fl_str_mv |
Ieee |
dc.source.none.fl_str_mv |
Web of Science reponame:Repositório Institucional da UNESP instname:Universidade Estadual Paulista (UNESP) instacron:UNESP |
instname_str |
Universidade Estadual Paulista (UNESP) |
instacron_str |
UNESP |
institution |
UNESP |
reponame_str |
Repositório Institucional da UNESP |
collection |
Repositório Institucional da UNESP |
repository.name.fl_str_mv |
Repositório Institucional da UNESP - Universidade Estadual Paulista (UNESP) |
repository.mail.fl_str_mv |
|
_version_ |
1803650376085274624 |