Constrained Information Maximization to Control Internal Representatio
Autor(a) principal: | |
---|---|
Data de Publicação: | 1997 |
Tipo de documento: | Artigo |
Idioma: | eng |
Título da fonte: | Journal of the Brazilian Computer Society |
Texto Completo: | http://old.scielo.br/scielo.php?script=sci_arttext&pid=S0104-65001997000200005 |
Resumo: | In the present paper, we propose a constrained information maximization method to control internal representations obtained in a course of learning. We focus upon hidden units and define information in hidden units acquired by learning. Internal representations are transformed by controlling this information. To control internal representations, a constraint is introduced in information maximization that total output from all the hidden units is a constant. By changing values of the constant, it is possible to generate many kinds of different internal representations, corresponding to the information content in hidden units. For example, we can obtain compact output patterns and specialized patterns of hidden units by changing the constant. We applied the constrained information maximization method to alphabet character recognition problems and a rule acquisition problem of an artificial language close to English. In the experiments, we were especially concerned with the generation of specialized hidden units, one of the typical example of the control of internal representations. Experimental results confirmed that we can control internal representations to produce specialized hidden units and to detect and extract main features of input patterns |
id |
UFRGS-28_71ca443a16f3d798786a10906efb57ae |
---|---|
oai_identifier_str |
oai:scielo:S0104-65001997000200005 |
network_acronym_str |
UFRGS-28 |
network_name_str |
Journal of the Brazilian Computer Society |
repository_id_str |
|
spelling |
Constrained Information Maximization to Control Internal RepresentatioInformation maximizationconstraintInternal representationspecialized hidden unitsrule acquisitionIn the present paper, we propose a constrained information maximization method to control internal representations obtained in a course of learning. We focus upon hidden units and define information in hidden units acquired by learning. Internal representations are transformed by controlling this information. To control internal representations, a constraint is introduced in information maximization that total output from all the hidden units is a constant. By changing values of the constant, it is possible to generate many kinds of different internal representations, corresponding to the information content in hidden units. For example, we can obtain compact output patterns and specialized patterns of hidden units by changing the constant. We applied the constrained information maximization method to alphabet character recognition problems and a rule acquisition problem of an artificial language close to English. In the experiments, we were especially concerned with the generation of specialized hidden units, one of the typical example of the control of internal representations. Experimental results confirmed that we can control internal representations to produce specialized hidden units and to detect and extract main features of input patternsSociedade Brasileira de Computação1997-07-01info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersiontext/htmlhttp://old.scielo.br/scielo.php?script=sci_arttext&pid=S0104-65001997000200005Journal of the Brazilian Computer Society v.4 n.1 1997reponame:Journal of the Brazilian Computer Societyinstname:Sociedade Brasileira de Computação (SBC)instacron:UFRGS10.1590/S0104-65001997000200005info:eu-repo/semantics/openAccessKamimura,Ryotaroeng1998-10-07T00:00:00Zoai:scielo:S0104-65001997000200005Revistahttps://journal-bcs.springeropen.com/PUBhttps://old.scielo.br/oai/scielo-oai.phpjbcs@icmc.sc.usp.br1678-48040104-6500opendoar:1998-10-07T00:00Journal of the Brazilian Computer Society - Sociedade Brasileira de Computação (SBC)false |
dc.title.none.fl_str_mv |
Constrained Information Maximization to Control Internal Representatio |
title |
Constrained Information Maximization to Control Internal Representatio |
spellingShingle |
Constrained Information Maximization to Control Internal Representatio Kamimura,Ryotaro Information maximization constraint Internal representation specialized hidden units rule acquisition |
title_short |
Constrained Information Maximization to Control Internal Representatio |
title_full |
Constrained Information Maximization to Control Internal Representatio |
title_fullStr |
Constrained Information Maximization to Control Internal Representatio |
title_full_unstemmed |
Constrained Information Maximization to Control Internal Representatio |
title_sort |
Constrained Information Maximization to Control Internal Representatio |
author |
Kamimura,Ryotaro |
author_facet |
Kamimura,Ryotaro |
author_role |
author |
dc.contributor.author.fl_str_mv |
Kamimura,Ryotaro |
dc.subject.por.fl_str_mv |
Information maximization constraint Internal representation specialized hidden units rule acquisition |
topic |
Information maximization constraint Internal representation specialized hidden units rule acquisition |
description |
In the present paper, we propose a constrained information maximization method to control internal representations obtained in a course of learning. We focus upon hidden units and define information in hidden units acquired by learning. Internal representations are transformed by controlling this information. To control internal representations, a constraint is introduced in information maximization that total output from all the hidden units is a constant. By changing values of the constant, it is possible to generate many kinds of different internal representations, corresponding to the information content in hidden units. For example, we can obtain compact output patterns and specialized patterns of hidden units by changing the constant. We applied the constrained information maximization method to alphabet character recognition problems and a rule acquisition problem of an artificial language close to English. In the experiments, we were especially concerned with the generation of specialized hidden units, one of the typical example of the control of internal representations. Experimental results confirmed that we can control internal representations to produce specialized hidden units and to detect and extract main features of input patterns |
publishDate |
1997 |
dc.date.none.fl_str_mv |
1997-07-01 |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/article |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
format |
article |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
http://old.scielo.br/scielo.php?script=sci_arttext&pid=S0104-65001997000200005 |
url |
http://old.scielo.br/scielo.php?script=sci_arttext&pid=S0104-65001997000200005 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
10.1590/S0104-65001997000200005 |
dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/openAccess |
eu_rights_str_mv |
openAccess |
dc.format.none.fl_str_mv |
text/html |
dc.publisher.none.fl_str_mv |
Sociedade Brasileira de Computação |
publisher.none.fl_str_mv |
Sociedade Brasileira de Computação |
dc.source.none.fl_str_mv |
Journal of the Brazilian Computer Society v.4 n.1 1997 reponame:Journal of the Brazilian Computer Society instname:Sociedade Brasileira de Computação (SBC) instacron:UFRGS |
instname_str |
Sociedade Brasileira de Computação (SBC) |
instacron_str |
UFRGS |
institution |
UFRGS |
reponame_str |
Journal of the Brazilian Computer Society |
collection |
Journal of the Brazilian Computer Society |
repository.name.fl_str_mv |
Journal of the Brazilian Computer Society - Sociedade Brasileira de Computação (SBC) |
repository.mail.fl_str_mv |
jbcs@icmc.sc.usp.br |
_version_ |
1754734669489766400 |