The actual clock frequency at which Intel's CPUs are clocked is complex and is governed by multiple mechanisms that perform dynamic frequency scaling based on the available headroom on power consumption.
As the technology improves, more transistors are getting packed into same per unit area of silicon. Moreover, overall silicon area of the mciroprocessor chip keeps growing as more functionality is integrated into the chip. The net result is that the power consumption of microprocessors is growing and power dissipation per unit area is also growing. It is the power dissipation per unit area that constraints the CPU clock frequencies. Intel has implemented a number of mechanisms into their architectures to extract additional performance through higher frequency whenever the power and thermal budgets allow.
- Enhanced Intel SpeedStep Technology (EIST) - Introduced with Pentium M, 2005
- Dynamic Acceleration Technology (DAT) - Introduced with Modified Pentium M/Core 2006
- Turbo Boost Technology (TBT) - Introduced with Nehalem in 2008
- Turbo Boost Technology 2.0 (TBT 2.0) - Introduced with Sandy Bridge in 2010
- Turbo Boost Max Technology 3.0 (TBMT) - Introduced with Broadwell E in 2016
- Speed Shift Technology (SST) - Introduced with Skylake in 2015
- Thermal Velocity Boost (TVB) - Introduced with Coffee Lake H in 2018
- Speed Select Technology (SST) - Introduced with Cascade Lake in 2019
Non-AVX, AVX-2.0 and AVX-512 Base FrequenciesIn past, microprocessors used to be marketed with one base frequency specified in the datasheets. For modern microprocessors available on Kogence, Intel specifies three base clock frequency for each of core in the socket. Base clock frequencies is the minimum guaranteed performance of each core in the socket. There is a base frequency for non-AVX instructions, base frequencies for AVX-2.0 instructions and base frequency for AVX-512 instructions. Base clock frequency for non-AVX instructions is the highest and this is what is advertised in the summary data sheets. As AVX-2.0 and AVX-512 instructions are more complex and consume more power, they are clocked at lower frequencies. Base frequencies used to calculate the actual clock frequency at which each core is clocked is determined for each core in the socket independently of other cores. So if we some cores are executing non-AVX instructions then may be clocked at higher frequency then the cores that are executing the AVX-512 instructions. It should be noted that AVX-2.0 and AVX-512 instructions perform more FLOPS per cycle. So even with slower clock rate, portion of your code that is utilizing AVX-2.0 or AVX-512 instructions would achieve higher FLOPS per second than the code that is utilizing non-AVX instructions.
Low Power Modes (P-States)
With Enhanced Intel SpeedStep Technology (EIST), each unused core can be "switched OFF" to save overall power consumption. Each core comes with a series of frequencies and associated voltages. This tuple is known as P-state and this P-state table is stored within the read-only processor model specific register (MSR) and is used to ensure that frequencies do not exceed the lower or upper bound.. An example frequency table is shown below. The lower bound is called the Low Frequency Mode (LFM) and is the lowest frequency-voltage operating point for a given processor. This is the "switched OFF" mode and is triggered when CPU is not doing anything. The upper bound is called the High Frequency mode (HFM) and is the highest frequency-voltage operating point. This is the "switched ON" mode and is triggered when CPU is loaded. Note that the HFM frequency is usually referred to by its advertised name: Base Frequency.
|Example P-State Table|
|1.21 V||2.8 GHz (HFM)|
|1.18 V||2.4 GHz|
|1.05 V||2.0 GHz|
|0.96 V||1.6 GHz|
|0.93 V||1.3 GHz|
|0.86 V||900 MHz|
|0.80 V||600 MHz (LFM)|
The CPUs will switch around between the various P-States as needed and as dictated by the operating system. Intel Xeon scalable processors available on Kogence come with Per-Core-P-State (PCPS), meaning each core operates at independent voltage and frequency to manage power consumption.
Turbo Boost 1.0 and 2.0
Turbo Boost Technology (TBT) is a microprocessor technology developed by Intel that attempts to enable temporary higher performance by opportunistically and automatically increasing the each core's clock frequency beyond the guaranteed minimum performance (i.e. the advertised base frequency).
Under various workloads, especially ones that are relatively low in power demands and/or are lightly threaded or not threaded at all, the processor can take advantage of the headroom by increasing the clock frequency while staying within the TDP (thermal design power) limits. The decision to kick into turbo boost is automatic and algorithmic in nature based on a number of factors such as: estimated current consumption, estimated power consumption, core temperature, and the number of active cores. The number of active cores, which Intel defines as cores in "C0" or "C1" states ("C3" and "C6" states are 'inactive'), dictates the upper limit. Generally, the more active cores, the lower the turbo boost available on top of the base clock frequency. For example, a dual-core 2 GHz CPU may allow a boost of 266.66 MHz (to 2266.66 MHz) when a single core is active but may only allow 133.33 MHz (to 2133.33 MHz) when two cores are active. Note that only the base clock frequency is the guaranteed performance of each core. So if each core is not only active but is doing heavy workload then the entire socket may be clocked at the base frequency and no boost may be applied. Just like base frequencies are controlled independently for each core in the socket, the turbo boost applied to a given core is also controlled independently of other cores in the socket. But calculation of boost to apply to each core does depend on the workload on all other cores.
BIOS allows for Turbo Boost to be disabled or enabled for entire socket and needs to be supported by the OS. In Kogence orchestrations, Turbo Boost enabled and supported by all hardware and OS we offer.
TBT 1.0 was first introduced in the Nehalem microarchitecture. Intel Turbo Boost Technology 1.0 was focused on improving performance of single-threaded applications on multicore Nehalem processors. TBT 2.0 was introduced later in 2011 in the Sandy Bridge microarchitecture and increased the scope of the metrics that are monitored to calculate turbo boost beyond just the threaded nature of the workloads.
Non-AVX Turbo, AVX-2.0 Turbo, AVX-512 Turbo
Just like the base frequencies are different for different instruction sets, the turbo boost is also dependent on the instruction set. If a given core is executing AVX-512 instructions then then base clock frequency applied that particular core will be lower than the base frequency applied to other cores doing non-AVX instructions. Similarly the turbo boost that the core executing the AX-512 instructions can get will also be different from the boost that cores executing non-AVX instructions can get.
Turbo Boost Max Technology 3.0
No two chips are the same; this is true even if both dies came from the very same wafer. Likewise, when stretching a processor to its limits, even the individual cores on a single monolithic integrated circuit produce slightly different performance characteristics. Testing individual cores by disabling all the other cores will reveal which ones happen to have better thermal and voltage properties and are stable at higher frequencies. They will each slightly differ.
Intel attempts to exploit those "superior cores" with Turbo Boost Max Technology. During the manufacturing process, Intel is able to test each die and determine which cores possess the best overclocking capabilities. That information is then stored in the CPU in order from best to worst.
With the help of the BIOS and the Intel driver for Windows or Linux, demanding workloads will be migrated to the best available core. The processor will then attempt to temporarily increase the frequency to TBMT range (somewhere around 200 MHz higher than Turbo Boost maximum frequency). Demanding workloads is workload that exceeds the configured utilization threshold (the default value given by Intel is 90%). Certain software can be manually tagged which will automatically affinitize them to the superior cores.
TBMT 3.0 was first introduced in processors marketed under Broadwell brand name. For Broadwell, TBMT applies only to 1 core. For Skylake, TBMT applies to 2 cores. With the introduction of Cascade Lake X, Intel improved the way TBMT works by operating on four cores instead of two.
Speed Select Technology was introduced with Cascade Lake. Speed Select Technology (SST) is a microprocessor power management technology introduced by Intel that allows for throughput and per-core performance configurations, allowing for the prioritization of performance for certain workloads running on specific cores by sacrificing the performance of other cores.
Processors support up to three SST Power Profiles (SST-PP) which can be used to configure the server for different kind of workloads. Profiles allow for fine tuning depending on the number of cores that require higher performance. In other words, the fewer the cores, the higher the performance for prioritized cores. And vice versa, the more cores, the lower the performance for prioritized cores. Profiles are manufactured to ensure base and turbo frequencies operate as desired within the TDP limits of the SKU.
Speed Select Technology is a set of power management controls that allows a system administrator to customize per-core performance. By configuring the performance of specific cores and affinitizing workloads to those cores, higher software performance can be achieved. SST supports multiple types of customizations:
- Frequency Prioritization (SST-CP) - allows specific cores to clock higher by reducing the frequency of cores running lower-priority software.
- Speed Select Base Freq (SST-BF) - allows specific cores to run higher base frequency (P1) by reducing the base frequencies (P1) of other cores.