High level description

In distributed systems, effective synchronization across multiple nodes is essential for resource access control. However, traditional locking mechanisms often face challenges related to scalability and latency. This thesis will focus on developing a sharding-based approach to low-latency distributed locking, optimizing concurrency management, and ensuring task integrity in multi-node systems.

Who are we looking for?

Master of Science in Computer Science/Engineering

Project description

At Knightec, we rely on task queues in our customer deliveries to ensure efficient, reliable processing. In this work, we aim to improve the performance of these systems by employing a sharded locking approach to reduce contention and improve scalability across distributed environments.

Task Queue with Exactly-Once Delivery Guarantee

Task queues are fundamental in distributed systems, and ensuring exactly-once task delivery is critical to avoid duplicate work or lost tasks. This section will explore how distributed locking can ensure that tasks are delivered and processed exactly once, even across multiple servers. We will focus on how sharded locks can improve the efficiency of task dispatching and handling within these queues, which is directly applicable to Knightec's task queue-based solutions for our customers.

Sharding for In-Memory Concurrency Management

The core of the proposed solution is to shard the locking mechanism itself. Instead of using a single, global lock, the queue or resource will be divided into shards, with each shard maintaining its own lock. This will allow each server instance to manage a portion of the work in parallel, significantly reducing contention and improving performance. This section will detail how this sharded locking approach can be applied to in-memory concurrency management on each server, optimizing both task processing and resource access in distributed systems.