Lesson 37 · Computer architecture

Multicore processors

Around 2004 clock speeds stopped climbing: a faster clock meant more power and more heat than a chip could shed. So chip makers spent their growing transistor budget on more cores, complete processors side by side on one chip. Each core has its own small L1 and L2 caches and shares one bigger L3. Watch how the caches keep their copies of a value in agreement (the MESI states and write-invalidate), why Amdahl's law caps the speed-up of a 90% parallel program at 10 times, and what hyper-threading adds.

14 min11 steps5 questionsAdvanced

Fig 37 · Interactive

Loading figure…

or ← →

Check yourself

5 quick questions

Pick an answer to see why. Your score saves with your progress.

01Why did chip makers stop raising clock speeds around 2004 and add cores instead?

Dynamic power is proportional to capacitive load × voltage² × frequency. Pushing the clock past about 3.6 GHz made chips too hot, while transistor counts kept growing, so the extra transistors went into more cores.

02Core 0 and core 2 both hold a copy of x in state S. Core 0 wants to write x. What happens first?

With write-invalidate, the writer first tells every other cache to drop its copy (I, invalid). Only then does it write, and its copy becomes M, modified: the only up-to-date copy.

03Which MESI state means: only this cache holds the line, and it has not been changed?

E, exclusive: the only copy, still the same as in memory. M is also the only copy but changed; S may be one of several unchanged copies; I must not be used.

04A program is 90% parallel. How much faster can it run on 4 cores, and on endless cores?

Amdahl's law: time = serial + parallel / n. Out of 10 units: 1 + 9/4 = 3.25 units, 10 / 3.25 ≈ 3.1 times faster. With endless cores the serial 1 unit remains, so the limit is 10 / 1 = 10 times.

05What does hyper-threading (simultaneous multithreading) give one core?

One SMT core holds the state of two threads and runs instructions from both on one set of execution units. When one waits for memory, the other keeps the units busy: Intel measured up to about 30% more, not double.

Key terms

Glossary

Core: A complete processor inside a chip: it fetches and runs its own stream of instructions. A multicore chip has several.
Clock speed: How many steps a processor takes per second, in gigahertz (GHz): billions of ticks per second.
Power wall: The limit, reached around 2004, where a faster clock would make a chip use more power and heat than it can get rid of.
Cache coherence: Keeping every cached copy of the same memory address in agreement, so no core reads a stale value.
MESI: Four states a cached line can be in: Modified (only copy, changed), Exclusive (only copy, unchanged), Shared (several unchanged copies) and Invalid (do not use).
Bus: Shared wires that connect the caches; what one cache sends, all the others can see.
Amdahl's law: Speed-up on n cores = 1 / ((1 − p) + p / n), where p is the part of the work that can run in parallel; the serial part sets the limit.
Simultaneous multithreading (SMT): One core running two threads at once on the same execution units; Intel calls it Hyper-Threading.

Fine print

Notes and sources

Dynamic power is proportional to capacitive load × voltage² × switching frequency. Intel's x86 clock rates rose for about 25 years to 3.6 GHz (Pentium 4 Prescott, 2004, 103 W) and then flattened while power fell back below 100 W (Patterson and Hennessy, Computer Organization and Design, section 1.7, The Power Wall; Sutter, The Free Lunch Is Over, 2005).
Per-core L1 and L2 caches with a shared last-level cache are the common layout; for example the AMD Ryzen 7 7700X has 1 MB of L2 per core and 32 MB of L3 shared by its 8 cores. Latencies are typical: L1 about 1 ns, L2 3-10 ns, L3 10-20 ns (Hennessy and Patterson, Computer Architecture, figure 2.1).
MESI (Modified, Exclusive, Shared, Invalid) is a write-invalidate snooping protocol, first published as the Illinois protocol by Papamarcos and Patel (ISCA 1984). Coherence is kept per cache line (64 bytes on Intel and AMD chips), not per variable. Real chips use variants such as MOESI or MESIF and often a ring or mesh with directories instead of one bus (Hennessy and Patterson, chapter 5).
Amdahl's law: speed-up = 1 / ((1 − p) + p / n) for a parallel fraction p on n cores (Amdahl, AFIPS 1967). With p = 0.9: 4 cores give 3.08×, 16 cores 6.4×, and no number of cores can exceed 1 / 0.1 = 10×.
Simultaneous multithreading issues instructions from several threads in the same core each cycle (Tullsen, Eggers and Levy, ISCA 1995). Intel's Hyper-Threading runs two threads per core; Intel reported up to about 30% more performance on server benchmarks (Marr et al., Intel Technology Journal, 2002).

Quiz Unlocks after the last step.