Implementation of the Low-Cost Work Stealing Algorithm for parallel computations

• Main Author: Custódio, Rafael Guerreiro
• Publication Date: 2022
• Format: Master thesis
• Language: eng
• Source: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
• Download full: http://hdl.handle.net/10362/151099
• topic: Scheduling algorithms; Work stealing; Parallel computing; Load balancing; Low-Cost Work Stealing; Domínio/Área Científica::Engenharia e Tecnologia::Engenharia Eletrotécnica, Eletrónica e Informática

For quite a while, CPU’s clock speed has stagnated while the number of cores keeps increasing. Because of this, parallel computing rose as a paradigm for programming on multi-core architectures, making it critical to control the costs of communication. Achieving this is hard, creating the need for tools that facilitate this task. Work Stealing (WSteal) became a popular option for scheduling multithreaded com- putations. It ensures scalability and can achieve high performance by spreading work across processors. Each processor owns a double-ended queue where it stores its work. When such deque is empty, the processor becomes a thief, attempting to steal work, at random, from other processors’ deques. This strategy was proved to be efficient and is still currently used in state-of-the-art WSteal algorithms. However, due to the concur- rent nature of the deque, local operations require expensive memory fences to ensure correctness. This means that even when a processor is not stealing work from others, it still incurs excessive overhead due to the local accesses to the deque. Moreover, the pure receiver-initiated approach to load balancing, as well as, the randomness of the targeting of a victim makes it not suitable for scheduling computations with few or unbalanced parallelism. In this thesis, we explore the various limitations of WSteal in addition to solutions proposed by related work. This is necessary to help decide on possible optimizations for the Low-Cost Work Stealing (LCWS) algorithm, proposed by Paulino and Rito, that we implemented in C++. This algorithm is proven to have exponentially less overhead than the state-of-the-art WSteal algorithms. Such implementation will be tested against the canonical WSteal and other variants that we implemented so that we can quantify the gains of the algorithm.