GPUs: thousands of small cores
A CPU has a few big, clever cores; a GPU has thousands of small, simple ones. Brighten a picture both ways: the CPU's cores take a few pixels at a time, while the GPU sends one instruction and every lane changes its own pixel at once. Then look inside: threads march in warps of 32, fast memory feeds them, branches split them, and the same trick powers the matrix maths behind AI.
5 quick questions
Pick an answer to see why. Your score saves with your progress.
01How many CUDA cores does an NVIDIA GeForce RTX 4090 have, compared with the 24 cores of an Intel Core i9-13900K?
NVIDIA's Ada whitepaper lists 128 streaming multiprocessors with 128 CUDA cores each: 16,384. They are small lanes that do arithmetic, not full CPU cores.
02What does SIMD stand for?
One instruction (such as px = px + 40) is applied to many pieces of data at once (one pixel per lane). That is how the GPU brightens all 32 pixels in one step.
03On an NVIDIA GPU, how many threads are grouped into one warp?
The CUDA Programming Guide groups threads into warps of 32. All the lanes of a warp run the same instruction together. AMD calls its group a wavefront, of 32 or 64 threads.
04Half the threads in a warp take the if branch and half take the else. What happens?
A warp issues one instruction at a time. Lanes that do not follow the branch are masked off and wait, so divergent code takes both paths one after the other. That is why GPUs are poor at branchy code.
05Which job suits a GPU best?
Every number in a matrix product is its own independent sum of products, so thousands of lanes can each work out one. Step-by-step, decision-heavy work stays on the CPU.
Glossary
- Core
- The part of a processor that runs instructions. A CPU has a few big cores; a GPU has thousands of small ones.
- GPU
- Graphics processing unit: a processor with thousands of simple cores built to do the same sum on lots of data at once.
- CUDA core
- NVIDIA's name for one small arithmetic lane in its GPUs. It is much simpler than a CPU core.
- SIMD
- Single instruction, multiple data: one instruction applied to many pieces of data at the same time.
- SIMT
- Single instruction, multiple threads: NVIDIA's version of SIMD, where each lane runs its own thread but a whole warp shares one instruction at a time.
- Warp
- A group of 32 threads on an NVIDIA GPU that run the same instruction together. AMD calls the group a wavefront (32 or 64 threads).
- Warp divergence
- When threads in one warp take different branches, so the paths run one after the other and some lanes sit idle.
- Memory bandwidth
- How many bytes a chip can move to or from its memory each second, for example 1,008 GB/s on an RTX 4090.
Notes and sources
- Teaching model: the CPU is drawn with 4 cores and the GPU with 32 lanes (one warp); the picture is 32 grey pixels. Real chips have many more of each.
- Core counts: Intel Core i9-13900K, 24 cores (8 performance + 16 efficient), max memory bandwidth 89.6 GB/s (Intel ARK); NVIDIA GeForce RTX 4090, 128 SMs x 128 = 16,384 CUDA cores, 1,008 GB/s (NVIDIA Ada whitepaper).
- A "CUDA core" is not a CPU core: it is one lane that does arithmetic. Instructions are fetched and scheduled per warp, not per lane, which is why a CUDA core is so much smaller.
- CPU cores do some SIMD too: Intel's AVX2 instructions work on 256-bit registers, eight 32-bit numbers at a time. A GPU takes the same idea to thousands of lanes.
- Warps: NVIDIA groups threads in 32s (CUDA Programming Guide, "Warps and SIMT"); lanes that skip a branch are masked off while the others run it. AMD uses wavefronts of 32 or 64 (GPUOpen RDNA Performance Guide).
Quiz Unlocks after the last step.