Methodology to filter out outliers in high spatial density data to improve maps reliability

Detalhes bibliográficos
Autor(a) principal: Maldaner,Leonardo Felipe
Data de Publicação: 2022
Outros Autores: Molin,José Paulo, Spekken,Mark
Tipo de documento: Artigo
Idioma: eng
Título da fonte: Scientia Agrícola (Online)
Texto Completo: http://old.scielo.br/scielo.php?script=sci_arttext&pid=S0103-90162022000100102
Resumo: ABSTRACT The considerable volume of data generated by sensors in the field presents systematic errors; thus, it is extremely important to exclude these errors to ensure mapping quality. The objective of this research was to develop and test a methodology to identify and exclude outliers in high-density spatial data sets, determine whether the developed filter process could help decrease the nugget effect and improve the spatial variability characterization of high sampling data. We created a filter composed of a global, anisotropic, and an anisotropic local analysis of data, which considered the respective neighborhood values. For that purpose, we used the median to classify a given spatial point into the data set as the main statistical parameter and took into account its neighbors within a radius. The filter was tested using raw data sets of corn yield, soil electrical conductivity (ECa), and the sensor vegetation index (SVI) in sugarcane. The results showed an improvement in accuracy of spatial variability within the data sets. The methodology reduced RMSE by 85 %, 97 %, and 79 % in corn yield, soil ECa, and SVI respectively, compared to interpolation errors of raw data sets. The filter excluded the local outliers, which considerably reduced the nugget effects, reducing estimation error of the interpolated data. The methodology proposed in this work had a better performance in removing outlier data when compared to two other methodologies from the literature.
id USP-18_35511c05468b1db0310ee13c7b937fd6
oai_identifier_str oai:scielo:S0103-90162022000100102
network_acronym_str USP-18
network_name_str Scientia Agrícola (Online)
repository_id_str
spelling Methodology to filter out outliers in high spatial density data to improve maps reliabilityprecision agriculturelocal analysismap accuracyABSTRACT The considerable volume of data generated by sensors in the field presents systematic errors; thus, it is extremely important to exclude these errors to ensure mapping quality. The objective of this research was to develop and test a methodology to identify and exclude outliers in high-density spatial data sets, determine whether the developed filter process could help decrease the nugget effect and improve the spatial variability characterization of high sampling data. We created a filter composed of a global, anisotropic, and an anisotropic local analysis of data, which considered the respective neighborhood values. For that purpose, we used the median to classify a given spatial point into the data set as the main statistical parameter and took into account its neighbors within a radius. The filter was tested using raw data sets of corn yield, soil electrical conductivity (ECa), and the sensor vegetation index (SVI) in sugarcane. The results showed an improvement in accuracy of spatial variability within the data sets. The methodology reduced RMSE by 85 %, 97 %, and 79 % in corn yield, soil ECa, and SVI respectively, compared to interpolation errors of raw data sets. The filter excluded the local outliers, which considerably reduced the nugget effects, reducing estimation error of the interpolated data. The methodology proposed in this work had a better performance in removing outlier data when compared to two other methodologies from the literature.Escola Superior de Agricultura "Luiz de Queiroz"2022-01-01info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersiontext/htmlhttp://old.scielo.br/scielo.php?script=sci_arttext&pid=S0103-90162022000100102Scientia Agricola v.79 n.1 2022reponame:Scientia Agrícola (Online)instname:Universidade de São Paulo (USP)instacron:USP10.1590/1678-992x-2020-0178info:eu-repo/semantics/openAccessMaldaner,Leonardo FelipeMolin,José PauloSpekken,Markeng2021-01-13T00:00:00Zoai:scielo:S0103-90162022000100102Revistahttp://revistas.usp.br/sa/indexPUBhttps://old.scielo.br/oai/scielo-oai.phpscientia@usp.br||alleoni@usp.br1678-992X0103-9016opendoar:2021-01-13T00:00Scientia Agrícola (Online) - Universidade de São Paulo (USP)false
dc.title.none.fl_str_mv Methodology to filter out outliers in high spatial density data to improve maps reliability
title Methodology to filter out outliers in high spatial density data to improve maps reliability
spellingShingle Methodology to filter out outliers in high spatial density data to improve maps reliability
Maldaner,Leonardo Felipe
precision agriculture
local analysis
map accuracy
title_short Methodology to filter out outliers in high spatial density data to improve maps reliability
title_full Methodology to filter out outliers in high spatial density data to improve maps reliability
title_fullStr Methodology to filter out outliers in high spatial density data to improve maps reliability
title_full_unstemmed Methodology to filter out outliers in high spatial density data to improve maps reliability
title_sort Methodology to filter out outliers in high spatial density data to improve maps reliability
author Maldaner,Leonardo Felipe
author_facet Maldaner,Leonardo Felipe
Molin,José Paulo
Spekken,Mark
author_role author
author2 Molin,José Paulo
Spekken,Mark
author2_role author
author
dc.contributor.author.fl_str_mv Maldaner,Leonardo Felipe
Molin,José Paulo
Spekken,Mark
dc.subject.por.fl_str_mv precision agriculture
local analysis
map accuracy
topic precision agriculture
local analysis
map accuracy
description ABSTRACT The considerable volume of data generated by sensors in the field presents systematic errors; thus, it is extremely important to exclude these errors to ensure mapping quality. The objective of this research was to develop and test a methodology to identify and exclude outliers in high-density spatial data sets, determine whether the developed filter process could help decrease the nugget effect and improve the spatial variability characterization of high sampling data. We created a filter composed of a global, anisotropic, and an anisotropic local analysis of data, which considered the respective neighborhood values. For that purpose, we used the median to classify a given spatial point into the data set as the main statistical parameter and took into account its neighbors within a radius. The filter was tested using raw data sets of corn yield, soil electrical conductivity (ECa), and the sensor vegetation index (SVI) in sugarcane. The results showed an improvement in accuracy of spatial variability within the data sets. The methodology reduced RMSE by 85 %, 97 %, and 79 % in corn yield, soil ECa, and SVI respectively, compared to interpolation errors of raw data sets. The filter excluded the local outliers, which considerably reduced the nugget effects, reducing estimation error of the interpolated data. The methodology proposed in this work had a better performance in removing outlier data when compared to two other methodologies from the literature.
publishDate 2022
dc.date.none.fl_str_mv 2022-01-01
dc.type.driver.fl_str_mv info:eu-repo/semantics/article
dc.type.status.fl_str_mv info:eu-repo/semantics/publishedVersion
format article
status_str publishedVersion
dc.identifier.uri.fl_str_mv http://old.scielo.br/scielo.php?script=sci_arttext&pid=S0103-90162022000100102
url http://old.scielo.br/scielo.php?script=sci_arttext&pid=S0103-90162022000100102
dc.language.iso.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv 10.1590/1678-992x-2020-0178
dc.rights.driver.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv text/html
dc.publisher.none.fl_str_mv Escola Superior de Agricultura "Luiz de Queiroz"
publisher.none.fl_str_mv Escola Superior de Agricultura "Luiz de Queiroz"
dc.source.none.fl_str_mv Scientia Agricola v.79 n.1 2022
reponame:Scientia Agrícola (Online)
instname:Universidade de São Paulo (USP)
instacron:USP
instname_str Universidade de São Paulo (USP)
instacron_str USP
institution USP
reponame_str Scientia Agrícola (Online)
collection Scientia Agrícola (Online)
repository.name.fl_str_mv Scientia Agrícola (Online) - Universidade de São Paulo (USP)
repository.mail.fl_str_mv scientia@usp.br||alleoni@usp.br
_version_ 1748936465741512704