Statistical Language Models applied to News Generation

• Autor(a) principal: João Ricardo Pintas Soares
• Data de Publicação: 2017
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
• Texto Completo: https://repositorio-aberto.up.pt/handle/10216/106475
• topic: Engenharia electrotécnica, electrónica e informática; Electrical engineering, Electronic engineering, Information engineering

Natural Language Generation (NLG) is a subfield of Artificial Intelligence. Its main goal is to produce understandable text in natural language, from a non-linguistic data input.Automated News Generation is a promising subject in the area of Computational Journalism, which uses NLG to create tools that helps journalists in the news production, automating some steps. These tools need a large amount of structured data as input and, for this reason, sports is a very natural subject to use, because the data is very well organized. The automatization of steps, in the news production, brings benefits to journalists, namely the tools can summarize data and make it readable instantly. Then they just have to adjust it, making the process of production a lot faster. The need for this agile process was the main motivation of this dissertation. The goal of this dissertation is to implement an Automated News Generation algorithm with the collaboration of ZOS, Lda. who owns the zerozero.pt project, an online social media publisher with one of the largest football databases in the world. They will provide a dataset for exploration and research in this field. This dissertation continues the work done by João Aires, in 2016, when he wrote a dissertation about this same topic. In this dissertation will be used a different approach to address the problem.The primary objective is to use Statistical Language Models to generate news from scratch, applying them to a system where the user can generate sentences about a specific match.Zerozero.pt saves data of more than 6000 matches per week and produces news for an average of 100 games per week. After a manual analysis of a part of that data, was decided that a news piece would be divided in 4 parts: Introduction, Goals, Sent offs and Conclusion. With the creation of Statistical Language Models for each part it is possible to summarize each match, making it easier to use this large amount of structured data and consequently increase the journalist's productivity.The evaluation of the system will be done using manual evaluation such as inquiries. This way, it will be possible to analyze and discuss the obtained results.