Amdahl's Law

It’s fascinating to see that a law presented in 1967 can be, nearly 60 years later, so accurate and relevant. Amdahl’s law, named after computer scientist Gene Myron Amdahl (1922 - 2015), can be stated as:

The overall performance improvement gained by optimizing a single part of a system is limited by the fraction of time that the improved part is actually used.

Gene Myron Amdahl was an American computer architect and high-tech entrepreneur, known for his work on mainframe computers at IBM and later his own companies. Amdahl’s law states a fundamental limitation of parallel computing.

For the mathematicians among us:

$$\text{Speedup } (S) = \frac{1}{(1 - P) + \frac{P}{N}}$$

Where:

• $S$: the overall theoretical speedup
• $P$: the proportion of the task/code that can be parallelized or improved ($0 \le P \le 1$)
• $1 - P$: the sequential (serial) portion that cannot be improved.
• $N$: the number of processors or the speedup factor applied to the parallel part.

Back in the day, this formula was used to explain that, no matter how many tasks a computer (later on, a PC) could use in parallel, the serial part of the work creates a ceiling, expressed as $S_{\max} = \frac{1}{1 - P}$. A basic example: if 10% of a program must run sequentially, the maximum possible speedup can never exceed 10x, no matter how many processors you add.

The scenario

Fast forward to the LLM and AI era: the sequential part can also be human intervention or control, necessary in many AI automated tasks. Let’s have a look at a scenario in a hospital:

Imagine a hospital network using an AI system to screen 1000 X-ray scans per day for anomalies.

• The parallel portion ($P$ = 95%): the AI can process thousands of scans simultaneously across cloud servers. It instantly filters out clear scans, flags potential issues, and pre-fills reports.
• The sequential portion ($1 - P$ = 5%): a human radiologist must physically review every scan flagged as “high risk”, double-check the AI’s findings, and sign off on the final diagnosis. No matter how much computing power the hospital adds, this step doesn’t get any faster: it runs at human speed. Because a human can only review one scan at a time, this step is entirely sequential.

In Amdahl’s terms, this is the serial part: not because it can never be split up, but because the upgrade we are paying for doesn’t touch it.

Calculating the “human bottleneck”

Let’s see what happens to the hospital’s overall throughput if the engineering team upgrades the AI computing cluster to make the parallel processing phase virtually instantaneous:

$$S = \frac{1}{0.05 + \frac{0.95}{N}}$$

Compute Factor (N)	AI Processing Time	Human Review Time (5%)	Overall Speedup Factor
Baseline Automation (N = 1)	95% of the day	5% of the day	1.0×
Upgraded Hardware (N = 5)	19% of the day	5% of the day	4.17× faster
Massive Supercomputer (N = 100)	0.95% of the day	5% of the day	16.8× faster
"Infinite" Compute (N → ∞)	0% of the day	5% of the day	20.0× (Absolute Max)

Key insights on Human-in-the-loop bottlenecks

Amdahl’s law explains exactly why automating a workflow often yields disappointing results if the human element isn’t re-engineered:

• The hard multiplier limit: even if the AI software runs instantly ($N \to \infty$), the hospital can never achieve more than a 20x overall speedup because the radiologist’s manual verification still takes up 5% of the original total baseline time.
• The cognitive context-switch: in real life, human intervention often introduces an even worse sequential penalty. If the human has to wait for a prompt, read a summary, orient themselves, and make a choice, they introduce a “warm-up” delay. This effectively increases the sequential fraction (1 - P), dragging the maximum speedup ceiling down even lower.
• The solution: to break past this Amdahl limit, you cannot just buy faster computers. You must change the human workflow — either by increasing the AI’s autonomous decision threshold (reducing the 5% human portion to 1% brings the ceiling to 100x) or by introducing horizontal human scaling (hiring more radiologists to parallelize the human element).

In the next blog, we will look deeper into this, with examples in the ERP and finance world!