kyo.scheduler.regulator

Admission control regulator that prevents scheduler overload by measuring queuing delays.

The Admission regulator protects the system from overload by monitoring scheduler queuing delays and selectively rejecting tasks when delays increase. It maintains an admission percentage that adjusts dynamically based on measured delays.

==Queuing Delay Measurement==

The regulator probes for queuing delays by periodically submitting special timing tasks into the scheduler and measuring how long they wait before execution. A probe task simply measures the time between its creation and execution. High variance or increasing delays in these measurements indicate scheduler congestion, triggering reductions in the admission rate to alleviate pressure.

==Rejection Mechanism==

Tasks are rejected using a deterministic hashing mechanism that provides stable and consistent admission decisions within time windows. Each task key (string or integer) is hashed using a large prime number multiplication and the current time window to generate a value between 0-99. Tasks with hash values above the current admission percentage are rejected.

The rotation mechanism is particularly important for fairness when task keys represent user identifiers:

Without rotation, users whose IDs hash to high values would face persistent rejection during system pressure
This could lead to poor user experience where some users are consistently locked out while others maintain access
Rotation ensures rejection patterns shift periodically, giving previously rejected users opportunities for admission
The rotation window duration can be tuned to balance stability and fairness

For example, with a 60-minute rotation window:

User A might be rejected in the first window due to their ID's hash value
In the next window, the time-based component changes the hash calculation
User A now has a different effective priority and may be admitted while others are rejected
This prevents any user from experiencing extended denial of service

This approach ensures:

Consistent decisions within each rotation window
Even distribution of rejections across the key space
Periodic rotation to prevent unfair persistent rejection
Load balancing through prime number distribution
Fair access patterns over time for all users

==Load Shedding Pattern==

The system responds to increasing pressure through a gradual and predictable load shedding pattern:

As pressure increases and admission percentage drops, tasks with highest hash values are rejected
Additional tasks are rejected if pressure continues building
System stabilizes at lower load with a stable subset of traffic
During recovery, admission percentage gradually increases
Previously rejected tasks may be admitted in new time windows

==Backpressure Characteristics==

This design creates an effective backpressure mechanism with several key characteristics. Load reduces predictably as the admission percentage drops, avoiding the oscillation patterns common with random rejection strategies. The system maintains a stable subset of flowing traffic within each time window, while providing natural queue-like behavior for rejected requests.

==Distributed Systems Context==

The admission control mechanism is particularly effective in microservices architectures. Consistent rejection patterns help downstream services manage their own load effectively, while enabling client libraries to implement intelligent backoff strategies. The predictable degradation and recovery patterns make it easier to maintain system stability across service boundaries.

Value parameters

config: Configuration parameters controlling admission behavior
loadAvg: A supplier that provides the current system load average
nowMillis: Current time supplier for delay measurements
rotationWindow: Duration after which the rejection pattern rotates to allow previously rejected keys another chance
schedule: Function to schedule probe tasks in the scheduler
timer: Timer for scheduling periodic regulation

Attributes

See also: Regulator for details on the underlying regulation mechanism
Companion: object
Supertypes: class Regulator

class Object

trait Matchable

class Any

Attributes

Companion: class
Supertypes: class Object

trait Matchable

class Any
Self type: Admission.type

Concurrency control regulator that optimizes thread count by detecting system scheduling delays.

The Concurrency regulator maintains optimal performance by measuring the system's ability to promptly execute threads. It uses a dedicated OS thread to perform brief sleep operations, analyzing delays between requested and actual wake times to detect various forms of contention and interference. This approach is similar to the jHiccup tool, but uses standard deviation analysis via the Regulator framework to make automatic adjustments.

The probe mechanism relies on the operating system's thread scheduling behavior. In a healthy system, the probe thread wakes up from sleep very close to the requested time. Various conditions can delay thread wake-ups, including OS scheduler overload, CPU throttling, excessive context switching, hypervisor interference, and hardware power management. By measuring these delays, the regulator detects when the system is struggling to handle the current thread count.

When wake-up delays show high jitter, indicating degraded thread scheduling, the regulator reduces the number of workers. When delays are consistent and the system maintains its target load, the regulator gradually increases workers. This approach automatically finds the optimal thread count for the current system conditions and available CPU resources.

Value parameters

config: Configuration parameters controlling concurrency adjustment
loadAvg: A supplier that provides the current system load average
nowNanos: Current time supplier for wake-up delay measurements
sleep: Function to perform sleep probes (must use OS thread sleep)
timer: Timer for scheduling periodic regulation
updateConcurrency: Callback to update the number of worker threads

Attributes

See also: Regulator for details on the underlying regulation mechanism
Companion: object
Supertypes: class Regulator

class Object

trait Matchable

class Any

Attributes

Companion: class
Supertypes: class Object

trait Matchable

class Any
Self type: Concurrency.type

Configuration parameters controlling regulator behavior.

The configuration determines how regulators collect and analyze timing measurements, and how aggressively they respond to detected issues. These parameters balance responsiveness against stability.

Value parameters

collectInterval: Interval between probe measurements
collectWindow: Size of the moving window used for standard deviation calculation
jitterLowerThreshold: Low standard deviation threshold that allows load increase
jitterUpperThreshold: High standard deviation threshold that triggers load reduction
loadAvgTarget: Target load level - load must meet this for increases
regulateInterval: Interval between regulation adjustments
stepExp: Controls how quickly consecutive adjustments escalate

Attributes

Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

A self-tuning regulator that dynamically adjusts scheduler behavior based on system performance metrics. This base class provides automatic adjustment of scheduler parameters based on real-time performance measurements and statistical analysis of timing variations.

==Measurement Collection==

The regulator collects timing measurements through periodic probes at configured intervals. These measurements are stored in a moving window, which provides an efficient way to maintain recent performance history while automatically discarding old data. This approach enables quick detection of emerging performance trends while smoothing out momentary irregularities.

==Jitter Analysis==

Collected measurements are analyzed using a moving standard deviation calculation to determine system stability. This "jitter" metric reveals performance characteristics such as sudden instability, ongoing systemic issues, and recovery patterns. The analysis focuses on detecting significant deviations that indicate potential performance problems.

==Adjustment Mechanism==

Based on the jitter analysis, the regulator makes incremental adjustments to maintain system stability. When jitter exceeds the upper threshold, the regulator adjusts using negative steps. Conversely, when jitter falls below the lower threshold and load meets the target, it adjusts with positive steps. Step sizes increase exponentially with consecutive adjustments in the same direction but reset when the direction changes, providing both responsiveness and stability.

==Configuration==

The regulator's behavior is controlled through configuration parameters that define the measurement window size, collection and regulation intervals, jitter thresholds, target load, and step escalation rate. These parameters can be tuned to match specific system characteristics and performance requirements.

Value parameters

config: Configuration parameters for the regulator
loadAvg: A supplier that provides the current system load average (0.0 to 1.0)
timer: Timer used for scheduling periodic measurements and adjustments

Attributes

See also: Config for configuration parameters

MovingStdDev for jitter calculation details

Admission for admission control implementation

Concurrency for concurrency control implementation
Note: Implementations must provide probe() and update() methods to define measurement collection and adjustment application respectively.
Supertypes: class Object

trait Matchable

class Any
Known subtypes: class Admission

class Concurrency

kyo.scheduler.regulator

Members list

Type members

Classlikes

Value parameters

Attributes

Attributes

Value parameters

Attributes

Attributes

Value parameters

Attributes

Value parameters

Attributes