Skip to content

gh-142518: Document thread-safety guarantees of list operations #142519

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lysnikolaou wants to merge 9 commits into
base: main
Choose a base branch
from
Open

gh-142518: Document thread-safety guarantees of list operations #142519

Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
fd400f0
gh-142518: Document thread-safety guarantees of list operations
lysnikolaou Dec 4, 2025
38483c9
Address feedback; add more details
lysnikolaou Dec 11, 2025
cc5c2f0
Merge branch 'main' into list-thread-safety-docs
lysnikolaou Dec 11, 2025
2292896
Add everything to code blocks
lysnikolaou Dec 11, 2025
2b9e711
Improve wording around atomicity; specify exact types
lysnikolaou Dec 11, 2025
688b25a
Better explain lock-free and atomicity
lysnikolaou Dec 17, 2025
11e8072
Use correct ref for atomic
lysnikolaou Dec 18, 2025
8728c54
Merge branch 'main' into list-thread-safety-docs
lysnikolaou Dec 18, 2025
f09430b
Fix lint error
lysnikolaou Dec 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Address feedback; add more details
  • Loading branch information
@lysnikolaou
lysnikolaou committed Dec 11, 2025
commit 38483c98f094f4e547dcee3ba74b541c3e726b17
113 changes: 70 additions & 43 deletions Doc/library/stdtypes.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1369,49 +1369,6 @@ Lists are mutable sequences, typically used to store collections of
homogeneous items (where the precise degree of similarity will vary by
application).

.. note::

Individual operations on :class:`list` instances such as
:meth:`~list.append`, :meth:`~list.pop`, ``lst[i] = x``, and ``x = lst[i]``
are atomic and will not corrupt the list or crash when called concurrently
from multiple threads:

.. code-block::
:class: good

# These operations are safe to call concurrently from multiple threads
lst.append(item) # atomically appends item
lst.pop() # atomically removes and returns last item
lst[i] = value # atomically replaces item at index i
item = lst[i] # atomically retrieves item at index i

One exception to this rule is :meth:`~list.extend`. Its atomicity
guarantees depend on the iterable being passed. When the iterable is
a :class:`list`, a :class:`tuple`, a :class:`set`, a :class:`frozenset`,
a :class:`dict` or a :ref:`dictionary view object <dict-views>`, the
operation is atomic. Otherwise, an iterator is created which can be
concurrently modified by another thread.

Operations that involve multiple accesses, as well as iteration, are not
atomic. For example, the following patterns are not thread-safe:

.. code-block::
:class: bad

# NOT atomic: read-modify-write
lst[i] = lst[i] + 1

# NOT atomic: check-then-act
if lst:
item = lst.pop()

# NOT thread-safe: iteration while modifying
for item in lst:
process(item) # another thread may modify lst

Consider external synchronization when sharing :class:`list` instances
across threads. See :ref:`freethreading-python-howto` for more information.

.. class:: list(iterable=(), /)

Lists may be constructed in several ways:
Expand Down Expand Up @@ -1479,6 +1436,76 @@ application).
list appear empty for the duration, and raises :exc:`ValueError` if it can
detect that the list has been mutated during a sort.

.. admonition:: Thread safety

Most individual operations on :class:`list` instances are atomic:

.. code-block::
:class: good

# The following operations are atomic
lst1 + lst2 # atomic concatenation of two lists
x * lst # atomic repeat of lst x times
item = lst[i] # atomically retrieves item at index i
lst[i] = value # atomically replaces item at index i
lst *= x # atomically extend the list x times

# Calls to the following list methods are atomic
lst.clear()
lst.copy()
lst.append(item)
lst.insert(idx, item)
lst.pop(idx)
lst.remove(item)
lst.reverse()
lst.sort()

The following operations/methods are not atomic:

.. code-block::
:class: maybe
Copy link
Member

@encukou encukou Dec 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this could say something like “Operations/methods that involve iteration are generally not atomic, except when used with specific built-in types”, and iteration itself can be moved here?
Mentioning iteration might help people make sense of this, i.e. it's no longer two arbitrary lists of operations/methods.

Then the bad section below would be left only with examples of “manually” combining multiple operations.

Copy link
Member Author

@lysnikolaou lysnikolaou Dec 11, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know how I feel about this. "Operations/methods that involve iteration are generally not atomic" is probably not a mnemonic we want people to use, because there are methods that are atomic but traverse the list. Granted most of those are ones that also mutate it, but e.g. list.copy doesn't.

But the idea of separating iteration from manually combining multiple operations is good. Maybe we should do just that?

Copy link
Member

@encukou encukou Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good.
(I guess it's “operations that involve arbitrary iterators and/or comparison functions”, but that's too long; readers whom that would help can figure it out from the list.)

As for iteration, it sounds like the guarantees are the same for single-threaded code: iteration of a list that is being modified may skip elements or yield repeated elements, but will not crash or produce elements that were never part of the list. Is that right?

Is this the place to document list iterators -- i.e. what happens if you use a shared iterator in several threads?

Copy link
Member Author

@lysnikolaou lysnikolaou Dec 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've completely reworded the docs to account for lock-free operations. Is this better?

I think documenting iterators should be done in the Iterator docs, not really individually for each iterator type.


lst.index(item)
lst.count(item)
item in lst

The :meth:`~list.index` and :meth:`~list.count` methods, and the ``in``
operator, iterate the list without holding a lock. They are safe to call
concurrently but may return results affected by concurrent modifications.

:meth:`~list.extend` is always atomic with respect to the target list.
However, the operation is fully atomic only when the iterable that's passed
to ``extend`` is a :class:`list`, a :class:`tuple`, a :class:`set`, a
:class:`frozenset`, a :class:`dict` or a
:ref:`dictionary view object <dict-views>`. Otherwise, an iterator is
created which can be concurrently modified by another thread. The same
applies to inplace concatenation of list with other iterables when using
``lst += iterable``.

Similarly, assigning to a list slice with ``lst[i:j] = obj`` is always
atomic with respect to the target list, but ``obj`` is only locked when it
is also a :class:`list`.

Operations that involve multiple accesses, as well as iteration, are not
atomic. For example:

.. code-block::
:class: bad

# NOT atomic: read-modify-write
lst[i] = lst[i] + 1

# NOT atomic: check-then-act
if lst:
item = lst.pop()

# NOT thread-safe: iteration while modifying
for item in lst:
process(item) # another thread may modify lst

Consider external synchronization when sharing :class:`list` instances
across threads. See :ref:`freethreading-python-howto` for more information.


.. _typesseq-tuple:

Expand Down
Loading