The Past and Future of GIL

PyCon China · 2019

Jiayuan Zhang

Who Am I?

  • Full-Stack Developer at IQIYI.Inc
  • Python, JavaScript, Lisp and (Rust)

  • Open source contributor, werkzeug, requests, doom-emacs, etc.

  • Coding with Emacs, organizing my life with org-mode

Outline

  • What is GIL
  • How GIL works
  • Remove GIL
  • The future

Part I

What is GIL?

# single_threaded.py
import time
from threading import Thread

COUNT = 50000000

def countdown(n):
    while n > 0:
        n -= 1

start = time.time()
countdown(COUNT)
end = time.time()

print('Time taken in seconds -', end - start)
$ python single_threaded.py
Time taken in seconds - 6.20024037361145
# multi_threaded.py
import time
from threading import Thread

COUNT = 50000000

def countdown(n):
    while n > 0:
        n -= 1

t1 = Thread(target=countdown, args=(COUNT//2,))
t2 = Thread(target=countdown, args=(COUNT//2,))

start = time.time()
t1.start()
t2.start()
t1.join()
t2.join()
end = time.time()

print('Time taken in seconds -', end - start)
$ python multi_threaded.py
Time taken in seconds - 6.924342632293701
$ python multi_threaded.py
Time taken in seconds - 6.924342632293701
$ python single_threaded.py
Time taken in seconds - 6.20024037361145

What is GIL ?

  • Global Interpreter Lock
  • Mutex, pthread (Linux) or win thread (Windows), controlled by OS
  • Allows only one thread to execute Python code at any point in time
  • Bottleneck in CPU-bound and multi-threaded code

What is GIL in Depth

  • Is Python thread safe?
  • "Lock" on what?

Is Python Thread Safe?

L.append(x)
L1.extend(L2)
x = L[i]
x = L.pop()
L1[i:j] = L2
L.sort()
x = y
x.field = y
D[x] = y
D1.update(D2)
D.keys()
i = i+1
L.append(L[-1])
L[i] = L[j]
D[x] = D[x] + 1

Safe

Not Safe

What kinds of global value mutation are thread-safe?

Why?

How Python Runs?

Python Source Code (.py files)

Python Interpreter

Result

What is Python Interpreter?

Python Source Code (.py files)

Result

Compiler

Bytecode

Virtual Machine

Python Interpreter

Library Modules

"Lock" on What?

Python Source Code (.py files)

Result

Compiler

Bytecode

Virtual Machine

Python Interpreter

Library Modules

GIL

Try Some Bytecode

Is `number += 1` thread safe?
from dis import dis

dis(lambda x: x+1)

Try Some Bytecode

  1           0 LOAD_FAST                0 (x)
              2 LOAD_CONST               1 (1)
              4 BINARY_ADD
              6 RETURN_VALUE

GIL

GIL

GIL

GIL

Not Thread Safe!!!

Part II

How GIL works?

I/O Bound Model

Thread 1

Thread 2

I/O

Release GIL

Acquire GIL

I/O

Release GIL

Acquire GIL

I/O

Release GIL

Acquire GIL

  • When a thread is running, it holds the GIL
  • GiL released on I/O (read, write, send, recv, etc.)

David Beazley Understand GIL

OS

CPU Bound Model

Thread 1

Thread 2

Check

Release GIL

Acquire GIL

Check

Release GIL

Acquire GIL

Release GIL

Acquire GIL

  • CPU bound threads that never perform I/O are handled as a special case
  • A "check" occurs every 100 "ticks"

Check

run 100 ticks

run 100 ticks

run 100 ticks

run 100 ticks

David Beazley Understand GIL

OS

Some OS Stuff

  • The operating system has a priority queue of threads/processes ready to run
  • Signaled threads simply enter that queue
  • The operating system then runs the process or thread with the highest priority
  • It may or may not be the signaled thread

David Beazley Understand GIL

Real CPU Bound Thread Situation

Thread 1

Thread 2

  • Hundreds to thousands of checks might occur before a thread context switch

David Beazley Understand GIL

run 100 ticks

OS

Release GIL

Acquire GIL

run 100 ticks

Release GIL

Acquire GIL

Release GIL

Acquire GIL

check

check

check

run 100 ticks

Idle

run 100 ticks

CPU 1

CPU 1

Thread Thrashing

Thread 1

Thread 2

  • When thread 2 wakes up, the GIL is already gone

David Beazley Understand GIL

run 100 ticks

OS

Release GIL

Acquire GIL

run 100 ticks

Release GIL

Acquire GIL

Release GIL

Acquire GIL

check

check

check

run 100 ticks

run 100 ticks

CPU 1

CPU 2

Wake

Acquire GIL

(fails)

Wake

Acquire GIL

(fails)

Wake

Acquire GIL

(success)

A Better GIL after Python 3.2

  • It aims to fix thread thrashing
  • Current thread will voluntarily release the GIL if it runs out of TIMEOUTs
  • If other thread acquire the GIL, the current thread will release the GIL after 5ms
  • A thread runs until `gil_drop_request` gets set to 1

But, the GIL is still there...

Part III

Remove GIL?

Why GIL?

  • There was no multi-core computer when Python was created
  • Python is designed to be easy-to-use, so you don't need to care about the memory stuff
  • GIL prevents deadlocks (as there is only one lock)
  • GIL provides a performance increase to single-threaded programs as only one lock needs to be managed
  • CPython uses Reference Counting

Before Removing the GIL...

  1. Reference Counting
  2. Globals and statics in the interpreter
  3. The C extension parallelism and reentrancy issues need to be handled as do places in the code where atomicity is required
  4. You can't breaking all of the C extensions

I'd welcome a set of patches into Py3k only if the performance for a single-threaded program (and for a multi-threaded but I/O-bound program) does not decrease.

-- Guido van Rossum

Related Works

  • 1995, Greg Stein, a fork of Python 1.5
  • Larry Hastings' Gilectomy (on hold)
  • Many other implementations...

It's Hard!!!

But, Who Cares?

  • Users with threaded, CPU bound Python code
  • Basically no one else

Use C extensions!

Other Solutions

  • Multi-processing (not recommend)
  • C extension modules
    • Rewrite CPU bound code in C (you need to control memory by yourself)
    • Release the GIL around that code
  • Corountine

Part IV

The Future

PEP 554

  • Multiple Interpreters in the Stdlib
  • By Eric Snow (to GIL or not to GIL: the Future of Multi-Core CPython)
  • Will release in CPython 3.9 (maybe)
  • CPython has supported multiple interpreters in the same process since version 1.5 (1997) via the C-API
  • Introduce the stdlib interpreters modules, high-level interface to subinterpreters
  • Functionality for sharing data between interpreters (channel)

Subinterpreters Model

Process

Interpreter

Interpreter

GIL

GIL

Thread

Thread

Thread

  • Sub-interpreter can't access other sub-interpreters' variables

How to Share Data?

  • A mechanism centers around "channels"
  • Similiar to queues and pipes
  • Objects are not shared between iterpreters since they are tied to the interpreter in which they were created
  • Only the following types will be supported fo sharing:
    • None
    • bytes
    • str
    • int
  • PEP 3118 buffer objects
  • PEP 554 channels

Support for other basic types (e.g. bool, float) will be added later

How to Share Data

Process

Interpreter

Interpreter

GIL

GIL

Thread

Thread

Thread

Channel

Advantages

  • Isolation
    • Each interpreter has its own copy of all modules, classes, functions, and variables
    • But process-global state remains shared (file descriptors, builtin types...)
  • Potentially performance
    • multiprocessing < subinterpreter < threads ?
  • Provide a direct route to an alternate concurrency mode

In Progress

That's All. Thanks!!!

Contacts

Wechat

（请备注公司/学校 + 姓名）