Entropy Measures vs. Kolmogorov Complexity

• Autor(a) principal: André Souto
• Data de Publicação: 2011
• Outros Autores: Luís Filipe Antunes, Andreia Teixeira, Armando Matos
• Tipo de documento: Artigo
• Idioma: eng
• Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
• Texto Completo: http://repositorio.inesctec.pt/handle/123456789/2961; http://dx.doi.org/10.3390/e13030595

Kolmogorov complexity and Shannon entropy are conceptually different measures. However, for any recursive probability distribution, the expected value of Kolmogorov complexity equals its Shannon entropy, up to a constant. We study if a similar relationship holds for R´enyi and Tsallis entropies of order α, showing that it only holds for α = 1. Regarding a time-bounded analogue relationship, we show that, for some distributions we have a similar result. We prove that, for universal time-bounded distribution mt(x), Tsallis and Rényi entropies converge if and only if α is greater than 1. We also establish the uniform continuity of these entropies.