Some time ago I posted an apology peice
for Python’s tail caling results. I apologized for communicating performance
results without noticing a compiler bug had occured.
I can proudly say today that I am partially retracting that apology, but
only for two platforms—MacOs AArch64 (XCode Clang) and Windows x86-64 (MSVC).
In our own experiments, the tail calling interpreter for CPython
was found to beat the computed
goto interpreter by 5% on pyperformance on AArch64 macOS using XCode Clang,
and roughly 15% on pyperformance on Windows on an experimental internal
version of MSVC. The Windows build is against a switch-case interpreter, but
this in theory shouldn’t matter too much, more on that in the next section.
This is of course, a hopefully accurate result. I tried to be more diligent
here, but I am of course not infallible. However, I have found that sharing early and making a fool of myself often works well, as it has led to people catching bugs in my code, so I shall continue doing so :).
Also this assumes the change doesn’t get reverted later in Python 3.15’s
development cycle.
Brief background on interpreters
Just a recap. There are two popular current ways of writing C-based
interpreters.
Switch-cases:
switch (opcode) {
case INST_1: ...
case INST_2: ...
}
Where we just switch-case to the correct instruction handler.
And the other popular way is a
GCC/Clang extension called labels-as-values/computed gotos.
Which is basically the same idea, but to instead jump to the address of the
next label. Traditionally, the key optimization here is that it needs
only one jump to go to the next instruction, while in the switch-case
interpreter, a naiive compiler would need two jumps.
With modern compilers however, the benefits of the computed gotos is a lot less,
mainly because modern compilers have gotten better and modern hardware
has also gotten better. In Nelson Elhage’s excellent investigation
on the next kind of interpreter,
the speedup of computed gotos over switch case on modern Clang was
only in the low single digits on pyperformance.
A 3rd way that was suggested decades ago, but not really entirely feasible
is call/tail-call threaded interpreters. In this scheme, each bytecode
handler is its own function, and we tail-call from one handler to the next
in the instruction stream:
This wasn’t too feasible in C for one main reason—tail call optimization
was merely an optimization. It’s something the C compiler might do, or
might not do. This means if you’re unlucky and the C compiler chooses not
to perform the tail call, your interpreter might stack overflow!
Some time ago, Clang introduced __attribute__((musttail)), which allowed
for mandating that a call must be tail-called. Otherwise, the compilation
will fail. To my knowledge, the first time this was popularized for use
in a mainstream interpreter was in Josh Haberman’s Protobuf blog post.
Later on, Haoran Xu noticed that the GHC calling convention combined with
tail calls produced efficient code. They used this for their baseline
JIT in a paper and termed the technique Copy-and-Patch.
So where are we now?
After using a fixed XCode Clang, our performance numbers on CPython
3.14/3.15 suggest that the tail calling interpreter does provide a
modest speedup over computed gotos. Around the 5% geomean range on
pyperformance.
Advertisement
To my understanding, uv already ships Python 3.14 on macOS with tail calling,
which might be responsible for some of the speedups you see on there.
We’re planning to ship the official 3.15 macOS binaries on python.org with
tail calling as well.
However, you’re not here for that. The title of this blog post
is clearly about MSVC Windows x86-64. So what about that?
Tail-calling for Windows
[!CAUTION]
The features for MSVC discussed below are to my knowledge, undocumented.
They are not guaranteed to always be around unless the MSVC team decide to keep them. Use at your own risk!
Advertisement
These are the preliminary pyperformance results
for CPython on MSVC with tail-calling vs
switch-case. Any number above 1.00x is a speedup
(e.g. 1.01x == 1% speedup), anything below 1.00x is a slowdown.
The speedup is a geomtric mean of around 15-16%, with a
range of ~60% slowdown (one or two outliers) to 78% speedup.
However, the key thing is that the vast majority of benchmaarks sped up!
Chart credits to Michael Droettboom
[!WARNING]
These results are on an experimental internal MSVC compiler, public results below.
To verify this and make sure I wasn’t wrong yet again, I checked the results
on my machine with Visual Studio 2026. These are the results from this issue.
Mean +- std dev: [spectralnorm_tc_no] 146 ms +- 1 ms -> [spectralnorm_tc] 98.3 ms +- 1.1 ms: 1.48x faster
Mean +- std dev: [nbody_tc_no] 145 ms +- 2 ms -> [nbody_tc] 107 ms +- 2 ms: 1.35x faster
Mean +- std dev: [bm_django_template_tc_no] 26.9 ms +- 0.5 ms -> [bm_django_template_tc] 22.8 ms +- 0.4 ms: 1.18x faster
Mean +- std dev: [xdsl_tc_no] 64.2 ms +- 1.6 ms -> [xdsl_tc] 56.1 ms +- 1.5 ms: 1.14x faster
So yeah, the speedups are real! For a large-ish library like xDSL, we see
a 14% speedup, while for smaller microbenchmarks like nbody and spectralnorm,
the speedups are greater.
Thanks to Chris Eibl and Brandt Bucher, we managed to get the PR for this
on MSVC over the finish line. I also want to sincerely thank the MSVC team. I can’t say this enough: they have been a joy to work with and
I’m very impressed by what they’ve done, and I want to congratulate them
on releasing Visual Studio 2026.
This is now listed in the What’s New for 3.15 notes:
Builds using Visual Studio 2026 (MSVC 18) may now use the new tail-calling interpreter. Results on an early experimental MSVC compiler reported roughly 15% speedup on the geometric mean of pyperformance on Windows x86-64 over the switch-case interpreter. We have observed speedups ranging from 15% for large pure-Python libraries to 40% for long-running small pure-Python scripts on Windows. (Contributed by Chris Eibl, Ken Jin, and Brandt Bucher in gh-143068. Special thanks to the MSVC team including Hulon Jenkins.)
I used to believe the the tail calling interpreters get their speedup
from better register use. While I still believe that now, I suspect that is
not the main reason for speedups in CPython.
My main guess now is that tail calling resets compiler heuristics to sane levels, so that compilers can do their jobs.
Let me show an example, at the time of writing, CPython 3.15’s interpreter loop
is around 12k lines of C code. That’s 12k lines in a single function
for the switch-case and computed goto interpreter.
Advertisement
This has caused many issues for compilers in the past, too many to list in fact.
I have a EuroPython 2025 talk about this. In short, this overly large function
breaks a lot of compiler heuristics.
One of the most beneficial optimisations is inlining. In the past, we’ve found
that compilers sometimes straight up refuse to inline even the
simplest of functions in that 12k loc eval loop. I want to stress that this
is not the fault of the compiler. It’s actually doing the correct
thing—you usually don’t want to increase the code size of something already
super large. Unfortunately, this does’t bode well for our interpreter.
You might say just write the interpreter in assembly!
However, the whole point of this exercise is to not do that.
Ok enough talk, let’s take a look at the code now. Taking a real
example, we examine BINARY_OP_ADD_INT which adds two Python integers.
Cleaning up the code so it’s readable, things look like this:
TARGET(BINARY_OP_ADD_INT) {
// Increment the instruction pointer.
_Py_CODEUNIT* const this_instr = next_instr;
frame->instr_ptr = next_instr;
next_instr += 6;
_PyStackRef right = stack_pointer[-1];
// Check that LHS is an int.
PyObject *value_o = PyStackRef_AsPyObjectBorrow(left);
if (!_PyLong_CheckExactAndCompact(value_o)) {
JUMP_TO_PREDICTED(BINARY_OP);
}
// Check that RHS is an int.
// ... (same code as above for LHS)
// Add them together.
PyObject *left_o = PyStackRef_AsPyObjectBorrow(left);
PyObject *right_o = PyStackRef_AsPyObjectBorrow(right);
res = _PyCompactLong_Add((PyLongObject *)left_o, (PyLongObject *)right_o);
// If the addition fails, fall back to the generic instruction.
if (PyStackRef_IsNull(res)) {
JUMP_TO_PREDICTED(BINARY_OP);
}
// Close the references.
PyStackRef_CLOSE_SPECIALIZED(left, _PyLong_ExactDealloc);
PyStackRef_CLOSE_SPECIALIZED(right, _PyLong_ExactDealloc);
// Write to the stack, and dispatch.
stack_pointer[-2] = res;
stack_pointer += -1;
DISPATCH();
}
Seems simple enough, let’s take a look at the assembly for switch-case on
VS 2026. Note again, this is a non-PGO build for easy source information,
PGO generally makes some of these problems go away, but not all of them:
The Xiaoxin AI Tablet Pro GT has a comparatively thin design offset by two rear-facing cameras. (Image source: Lenovo) The Xiaoxin AI Tablet Pro GT […]
