Concurrency

Concurrency control regulator that optimizes thread count by detecting system scheduling delays.

The Concurrency regulator maintains optimal performance by measuring the system's ability to promptly execute threads. It uses a dedicated OS thread to perform brief sleep operations, analyzing delays between requested and actual wake times to detect various forms of contention and interference. This approach is similar to the jHiccup tool, but uses standard deviation analysis via the Regulator framework to make automatic adjustments.

The probe mechanism relies on the operating system's thread scheduling behavior. In a healthy system, the probe thread wakes up from sleep very close to the requested time. Various conditions can delay thread wake-ups, including OS scheduler overload, CPU throttling, excessive context switching, hypervisor interference, and hardware power management. By measuring these delays, the regulator detects when the system is struggling to handle the current thread count.

When wake-up delays show high jitter, indicating degraded thread scheduling, the regulator reduces the number of workers. When delays are consistent and the system maintains its target load, the regulator gradually increases workers. This approach automatically finds the optimal thread count for the current system conditions and available CPU resources.