gh-143120: pixi builds for free-threading and TSAN #142872

crusaderky · 2025-12-17T12:08:15Z

Follow-up to #142469

Add pixi infrastructure for

free-threading: compiles with --disable-gil
tsan-free-threading: compiles with --disable-gil --with-thread-sanitizer

Currently, free-threading seems to work well on Linux.
tsan crashes on build: [EDIT read comment below for solution]

[...]/build_tsan ./python -E -S -m sysconfig --generate [...]
 │ │ -posix-vars ;\
[...]
 │ │ Bus error (core dumped)
 │ │ make: *** [Makefile:1162: pybuilddir.txt] Error 135

@kumaraditya303 @lucascolley any idea what's wrong here?

Issue: Add pixi builds for TSan + free-threading to aid downstream development. #143120

crusaderky · 2025-12-17T12:08:35Z

Tools/pixi-packages/free-threading/pixi.toml

copy-paste of asan/pixi.toml

crusaderky · 2025-12-17T12:08:51Z

Tools/pixi-packages/free-threading/recipe.yaml

+  script:
+    file: ../build.sh
+    env:
+      PYTHON_VARIANT: "free-threading"


copy-paste of asan/recipe.yaml, except this one line

crusaderky · 2025-12-17T12:09:04Z

Tools/pixi-packages/tsan-free-threading/pixi.toml

copy-paste of asan/pixi.toml

crusaderky · 2025-12-17T12:09:21Z

Tools/pixi-packages/tsan/recipe.yaml

+  script:
+    file: ../build.sh
+    env:
+      PYTHON_VARIANT: "tsan"


copy-paste of asan/recipe.yaml, except this one line

lucascolley · 2025-12-17T12:10:08Z

@kumaraditya303 @lucascolley any idea what's wrong here?

no clue sorry I haven't seen this before

crusaderky · 2025-12-18T17:23:27Z

The tsan crash is due to the default mmap_rnd_bits on Ubuntu 24.04; it goes away after lowering it:

$ sudo sysctl vm.mmap_rnd_bits
vm.mmap_rnd_bits = 32  # too high
$ sudo sysctl vm.mmap_rnd_bits=28  # reduce it
vm.mmap_rnd_bits = 28

ubuntu-latest github actions workers seems to be unaffected.

hugovk · 2025-12-23T21:56:05Z

There's a lot of copy/paste here, which can make maintenance harder. Does pixi have some concept of code reuse or parametrisation? Maybe not the best example, but something like GitHub Actions' reusable workflows?

ngoldbaum · 2025-12-23T21:58:36Z

@lucascolley is there anything we can do right now to reduce the duplication here? Or does that require fixes in pixi?

lucascolley · 2025-12-23T21:59:35Z

As mentioned in the README already, this is blocked on prefix-dev/pixi#4599.

ngoldbaum

I opened an issue which is needed to make this mergeable. I'm also playing with a NumPy recipe based on this.

ngoldbaum · 2025-12-23T20:34:55Z

Tools/pixi-packages/README.md

- `asan`: ASan-instrumented build with `PYTHON_ASAN=1`
+- `free-threading`
+- `asan`: ASan-instrumented build
+- `tsan`: free-threading, TSan-instrumented build


Should probably be called tsan-free-threading. There should probably also be an asan-free-threading. @lucascolley is there a way to avoid the combinatorial explosion of variants here? Ideally you'd be able to somehow combine these but I think that's probably not tenable at the moment.

I think the answer to this is no because of prefix-dev/pixi#4599
Barring the above, we could write a template + generation script and then commit the output to git? Very ugly if you ask me.

Yes, I would vote for keeping things simple but verbose at the minute until there is a proper solution upstream

Renamed to tsan-free-threading

ngoldbaum · 2025-12-23T20:36:04Z

Tools/pixi-packages/build.sh

    echo "BUILD TYPE: ASAN"
    BUILD_DIR="../build_asan"
    CONFIGURE_EXTRA="--with-address-sanitizer"
-    export PYTHON_ASAN="1"


why delete this?

I did not find it to be used anywhere. @lucascolley any insight?

I think it can be removed, yes. I forgot halfway through whether it did anything on the CPython side

crusaderky · 2026-01-02T15:56:06Z

Tools/pixi-packages/build.sh

+    cp "${BUILD_PREFIX}/lib/clang/21/lib/darwin/libclang_rt.asan_osx_dynamic.dylib" "${PREFIX}/lib/"
+    cp "${BUILD_PREFIX}/lib/clang/21/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib" "${PREFIX}/lib/"


I don't have straightforward means to test this. @lucascolley / @ngoldbaum could you give it a go?

The build dependencies list {compiler('c')}, which I believe translates to clang-21, plus and explicit reference to clang-19. Here we're referencing clang-21.
@lucascolley is this deliberate?

maybe we can drop the clang-19

happy to give this a go but would be easier once there is a NumPy build using this.

I'm able to get past the rpath errors I was seeing before on my Mac using this hack. Thanks!

crusaderky · 2026-01-02T16:16:43Z

Ready for final review

ngoldbaum · 2026-01-05T22:58:03Z

@lucascolley and I chatted a little about this on the pixi discord. It seems that the way these packages are defined, the numpy build in the CI for numpy/numpy#30510 ends up having Python packages installed in a GIL-enabled 3.15 envrionment, while the build happens in a 3.15t environment:

numpy on 🎋 tsan-ft-pixi is 📦 v2.5.0.dev0 via 🐍
❯ ls -s pixi-packages/tsan-free-threading/.pixi/build/work/numpy-K8YFxN4GfUg-ME03ejjoGjs/host_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.15t/site-packages/
╭───┬────────────────────┬──────┬───────┬────────────────╮
│ # │        name        │ type │ size  │    modified    │
├───┼────────────────────┼──────┼───────┼────────────────┤
│ 0 │ README.txt         │ file │ 119 B │ 18 minutes ago │
│ 1 │ pip                │ dir  │ 288 B │ 12 minutes ago │
│ 2 │ pip-25.3.dist-info │ dir  │ 288 B │ 12 minutes ago │
╰───┴────────────────────┴──────┴───────┴────────────────╯

numpy on 🎋 tsan-ft-pixi is 📦 v2.5.0.dev0 via 🐍
❯ ls -s pixi-packages/tsan-free-threading/.pixi/build/work/numpy-K8YFxN4GfUg-ME03ejjoGjs/host_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.15/site-packages/
╭────┬─────────────────────────────────────┬──────┬────────┬────────────────╮
│  # │                name                 │ type │  size  │    modified    │
├────┼─────────────────────────────────────┼──────┼────────┼────────────────┤
│  0 │ Cython                              │ dir  │  736 B │ 12 minutes ago │
│  1 │ cython-3.2.4.dist-info              │ dir  │  320 B │ 12 minutes ago │
│  2 │ cython.py                           │ file │  632 B │ 21 hours ago   │
│  3 │ meson-1.10.0.dist-info              │ dir  │  352 B │ 12 minutes ago │
│  4 │ meson_python-0.18.0.dist-info       │ dir  │  288 B │ 12 minutes ago │
│  5 │ mesonbuild                          │ dir  │ 1.5 kB │ 12 minutes ago │
│  6 │ mesonpy                             │ dir  │  288 B │ 12 minutes ago │
│  7 │ packaging                           │ dir  │  576 B │ 12 minutes ago │
│  8 │ packaging-25.0.dist-info            │ dir  │  352 B │ 12 minutes ago │
│  9 │ pyproject_metadata                  │ dir  │  256 B │ 12 minutes ago │
│ 10 │ pyproject_metadata-0.10.0.dist-info │ dir  │  288 B │ 12 minutes ago │
│ 11 │ pyximport                           │ dir  │  160 B │ 12 minutes ago │
│ 12 │ tomli                               │ dir  │  224 B │ 12 minutes ago │
│ 13 │ tomli-2.3.0.dist-info               │ dir  │  288 B │ 12 minutes ago │
╰────┴─────────────────────────────────────┴──────┴────────┴────────────────╯

This is either a bug in rattler-build or it's a bug in the package definitions here.

If it's the latter - maybe the package definitions here need to define the ABI pin that the conda-forge python-freethreading package defines? Or some other way to indicate the Python ABI in the metadata for the package. Maybe @h-vetinari or @isuruf have ideas.

isuruf · 2026-01-05T23:05:23Z

The python builds here (technically added in the previous PRs) do not have the correct metadata to be used as a conda package. Either you should fix the metadata here or not use them downstream.

lucascolley · 2026-01-05T23:06:32Z

how can we fix the metadata?

isuruf · 2026-01-05T23:08:54Z

You need the python_abi run requirement which in turn require a proper build string.

isuruf · 2026-01-05T23:13:07Z

In order to not let this repo be held up against conda-forge/python_abi, you'll need to add python_abi to the recipe as well.

crusaderky · 2026-01-06T10:18:57Z

@lucascolley and I chatted a little about this on the pixi discord. It seems that the way these packages are defined, the numpy build in the CI for numpy/numpy#30510 ends up having Python packages installed in a GIL-enabled 3.15 envrionment, while the build happens in a 3.15t environment:

Does this imply that if I use the ASAN numpy recipe from numpy main I end up running with a regular (non-ASAN) cpython?

However what you're saying doesn't seem to add up. If I create a project

[workspace]
channels = ["https://prefix.dev/conda-forge"]
platforms = ["linux-64", "linux-aarch64", "osx-64", "osx-arm64"]
preview = ["pixi-build"]

[dependencies]
python.git = "https://github.com/crusaderky/cpython"
python.subdirectory = "Tools/pixi-packages/tsan-free-threading"
python.rev = "tsan"
numpy.git = "https://github.com/ngoldbaum/numpy"
numpy.subdirectory = "pixi-packages/tsan-free-threading"
numpy.rev = "tsan-ft-pixi"

then I get

$ pixi ls
Name             Version     Build                      Size  Kind   Source
_libgcc_mutex    0.1         conda_forge            2.50 KiB  conda  https://prefix.dev/conda-forge
_openmp_mutex    4.5         2_gnu                 23.07 KiB  conda  https://prefix.dev/conda-forge
bzip2            1.0.8       hda65f42_8           254.24 KiB  conda  https://prefix.dev/conda-forge
ca-certificates  2026.1.4    hbd8a1cb_0           143.08 KiB  conda  https://prefix.dev/conda-forge
icu              78.1        h33c6efd_0            12.13 MiB  conda  https://prefix.dev/conda-forge
libexpat         2.7.3       hecca717_0            74.85 KiB  conda  https://prefix.dev/conda-forge
libffi           3.5.2       h9ec8514_0            56.47 KiB  conda  https://prefix.dev/conda-forge
libgcc           15.2.0      he0feb66_16         1018.36 KiB  conda  https://prefix.dev/conda-forge
libgomp          15.2.0      he0feb66_16          589.14 KiB  conda  https://prefix.dev/conda-forge
liblzma          5.8.1       hb9d3cd8_2           110.25 KiB  conda  https://prefix.dev/conda-forge
libmpdec         4.0.0       hb9d3cd8_0            89.05 KiB  conda  https://prefix.dev/conda-forge
libsqlite        3.51.1      hf4e2dac_1           921.34 KiB  conda  https://prefix.dev/conda-forge
libstdcxx        15.2.0      h934c35e_16            5.59 MiB  conda  https://prefix.dev/conda-forge
libuuid          2.41.3      h5347b49_0            39.37 KiB  conda  https://prefix.dev/conda-forge
libxcb           1.17.0      h8a09558_0           386.61 KiB  conda  https://prefix.dev/conda-forge
libzlib          1.3.1       hb9d3cd8_2            59.53 KiB  conda  https://prefix.dev/conda-forge
ncurses          6.5         h2d0b736_3           870.74 KiB  conda  https://prefix.dev/conda-forge
numpy            2.5.0.dev0  hb0f4dca_0                       conda  git+https://github.com/ngoldbaum/numpy?subdirectory=pixi-packages%2Ftsan-free-threading&rev=tsan-ft-pixi#069e0d56b11769b8f2df85ca7a08e5c23928a75b
openssl          3.6.0       h26f9b46_0             3.02 MiB  conda  https://prefix.dev/conda-forge
pthread-stubs    0.4         hb9d3cd8_1002          8.06 KiB  conda  https://prefix.dev/conda-forge
python           3.15        hb0f4dca_0                       conda  git+https://github.com/crusaderky/cpython?subdirectory=Tools%2Fpixi-packages%2Ftsan-free-threading&rev=tsan#9a2a8963286077536f67b89a2123355aeb049a3d
readline         8.3         h853b02a_0           336.99 KiB  conda  https://prefix.dev/conda-forge
tk               8.6.13      noxft_ha0e22de_103     3.13 MiB  conda  https://prefix.dev/conda-forge
xorg-libx11      1.8.12      h4f16b4b_0           816.30 KiB  conda  https://prefix.dev/conda-forge
xorg-libxau      1.0.12      hb03c661_1            14.96 KiB  conda  https://prefix.dev/conda-forge
xorg-libxdmcp    1.1.5       hb03c661_1            20.11 KiB  conda  https://prefix.dev/conda-forge
zstd             1.5.7       hb78ec9c_6           587.28 KiB  conda  https://prefix.dev/conda-forge

which looks healthy? I can't see any GIL-enabled python from conda-forge?

You need the python_abi run requirement which in turn require a proper build string.

I'm trying to understand what this means by reading https://github.com/conda-forge/python-feedstock/blob/main/recipe/meta.yaml.

Are you saying that what we're misssing is something like this?

outputs:
      run_exports:
        noarch:
          - python
        weak:
          - python_abi {{ ver2 }}.* *_{{ abi_tag }}
{% if py_interp_debug == "yes" %}
          - python {{ ver2 }}.* *_debug_{{ abi_tag }}
{% endif %}

lucascolley · 2026-01-06T10:29:30Z

https://github.com/conda-forge/python-feedstock/blob/main/recipe/meta.yaml#L36-L40 looks most relevant to the failure Nathan is seeing. I think it's just the wrong directory rather than the wrong package being installed.

lucascolley · 2026-01-06T10:43:06Z

Actually the exact problem seems described at https://github.com/conda-forge/python-feedstock/blob/main/recipe/build_base.sh#L526-L541

lucascolley · 2026-01-06T10:45:33Z

So maybe we need an equivalent of conda/conda#14053 fixed in rattler

lucascolley · 2026-01-06T11:09:36Z

however rattler seems to get it right when installing python-freethreading from conda-forge at least:

~/sandbox/rattler-pyt via 🧚 v0.62.2 
❯ pixi add python-freethreading cowpy
✔ Added python-freethreading >=3.14.2,<4
✔ Added cowpy >=1.1.5,<2

~/sandbox/rattler-pyt via 🧚 v0.62.2 took 2s 
❯ ls -s .pixi/envs/default/lib/python3.14t/site-packages
╭───┬───────────────────────┬──────┬───────┬────────────────╮
│ # │         name          │ type │ size  │    modified    │
├───┼───────────────────────┼──────┼───────┼────────────────┤
│ 0 │ README.txt            │ file │ 119 B │ a month ago    │
│ 1 │ conda-site.pth        │ file │ 256 B │ 22 seconds ago │
│ 2 │ cowpy                 │ dir  │ 128 B │ 23 seconds ago │
│ 3 │ cowpy-1.1.5.dist-info │ dir  │ 320 B │ 23 seconds ago │
╰───┴───────────────────────┴──────┴───────┴────────────────╯

lucascolley · 2026-01-06T11:12:49Z

can you try this diff Guido?

diff --git a/Tools/pixi-packages/tsan-free-threading/recipe.yaml b/Tools/pixi-packages/tsan-free-threading/recipe.yaml
index dfae06ad7a5..3417e8b92b1 100644
--- a/Tools/pixi-packages/tsan-free-threading/recipe.yaml
+++ b/Tools/pixi-packages/tsan-free-threading/recipe.yaml
@@ -18,4 +18,6 @@ build:
     env:
       PYTHON_VARIANT: "tsan-free-threading"
+  python:
+    site_packages_path: "lib/python3.13t/site-packages"

 # derived from https://github.com/conda-forge/python-feedstock/blob/main/recipe/meta.yaml

ngoldbaum · 2026-01-06T16:09:42Z

can you try this diff Guido?

Maybe I did something wrong, but I attempted to apply this patch over at ngoldbaum@39df6fe and then updated numpy at ngoldbaum/numpy@d28bebc and it doesn't seem to work, unfortunately, at least on my local setup. CI will test it when a runner picks it up:

https://github.com/numpy/numpy/actions/runs/20754129965/job/59591290742?pr=30510

ngoldbaum · 2026-01-06T16:15:21Z

It looks like the option in the recipe isn't having any effect:

bash-5.3$ ls lib/python3.15/site-packages/
Cython					meson_python-0.18.0.dist-info		mesonpy					pyproject_metadata			tomli
cython-3.2.4.dist-info			meson-1.10.0.dist-info			packaging				pyproject_metadata-0.10.0.dist-info	tomli-2.3.0.dist-info
cython.py				mesonbuild				packaging-25.0.dist-info		pyximport
bash-5.3$ ls lib/python3.15t/site-packages/
pip			pip-25.3.dist-info	README.txt

lucascolley · 2026-01-06T16:43:40Z

I tried it locally and it at least works with pixi global install:

Tools/pixi-packages/tsan-free-threading on 🎋 tsan [🔨?] via 🧚 v0.62.2
❯ pixi global install -e pyt --path ./python-3.15-h60d57d3_0.conda
└── pyt (installed)
    ├─ dependencies: python 3.15
    └─ exposes: idle3, idle3.15, pip3, pip3.15, pydoc3, pydoc3.15, python, python3, python3-config, python3.15, python3.15-config, python3.15t, python3.15t-config

Tools/pixi-packages/tsan-free-threading on 🎋 tsan [🔨?] via 🧚 v0.62.2 took 2s
❯ pixi global install -e pyt cowpy
└── pyt (installed)
    ├─ dependencies: python 3.15, cowpy 1.1.5
    └─ exposes: idle3, idle3.15, pip3, pip3.15, pydoc3, pydoc3.15, python, python3, python3-config, python3.15, python3.15-config, python3.15t, python3.15t-config, cowpy

Tools/pixi-packages/tsan-free-threading on 🎋 tsan [🔨?] via 🧚 v0.62.2
❯ ls -s ~/.pixi/envs/pyt/lib/python3.13t/site-packages/
╭───┬───────────────────────┬──────┬───────┬───────────────╮
│ # │         name          │ type │ size  │   modified    │
├───┼───────────────────────┼──────┼───────┼───────────────┤
│ 0 │ cowpy                 │ dir  │ 128 B │ 2 minutes ago │
│ 1 │ cowpy-1.1.5.dist-info │ dir  │ 320 B │ 2 minutes ago │
╰───┴───────────────────────┴──────┴───────┴───────────────╯

lucascolley · 2026-01-06T16:59:06Z

ah probably that only works for noarch packages and we need something like https://github.com/conda-forge/python-feedstock/blob/c90e3cfe0a44db1b0741359d68c330cd7d861b16/recipe/sitecustomize.py

isuruf · 2026-01-06T19:05:29Z

ah probably that only works for noarch packages and we need something like https://github.com/conda-forge/python-feedstock/blob/c90e3cfe0a44db1b0741359d68c330cd7d861b16/recipe/sitecustomize.py

That workaround is for older conda versions not handling noarch packages properly.

lucascolley · 2026-01-06T19:18:08Z

Hmm yeah on second thought meson-python should be in the right place, let me try to reproduce Nathan's failure

lucascolley · 2026-01-06T20:08:23Z

can you try this diff Guido?

Maybe I did something wrong, but I attempted to apply this patch over at ngoldbaum@39df6fe and then updated numpy at ngoldbaum/numpy@d28bebc and it doesn't seem to work, unfortunately, at least on my local setup. CI will test it when a runner picks it up:

numpy/numpy/actions/runs/20754129965/job/59591290742?pr=30510

doh should have checked the CI log earlier, it is working! Just worked locally for me too

ngoldbaum · 2026-01-06T20:41:02Z

doh should have checked the CI log earlier, it is working! Just worked locally for me too

Oh hey, it worked! That's so strange - I wonder what's different on my setup. It looks like pixi is up-to-date...

lucascolley · 2026-01-06T20:42:16Z

if in doubt try pixi clean, maybe something is being cached that shouldn't be

ngoldbaum · 2026-01-06T20:52:33Z

if in doubt try pixi clean, maybe something is being cached that shouldn't be

Nope, doesn't work. Here's the build log after pixi clean and pixi clean cache: https://gist.github.com/ngoldbaum/8ade90dd534410c6c13e349f23e2d077

The only difference I see compared with the working build on CI is that my build used uv 0.9.22 but the CI build used 0.9.21.

bedevere-app bot added the awaiting review label Dec 17, 2025

crusaderky commented Dec 17, 2025

View reviewed changes

Tools/pixi-packages/free-threading/pixi.toml

Copy link

Contributor Author

crusaderky Dec 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copy-paste of asan/pixi.toml

crusaderky commented Dec 17, 2025

View reviewed changes

Tools/pixi-packages/tsan-free-threading/pixi.toml

Copy link

Contributor Author

crusaderky Dec 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copy-paste of asan/pixi.toml

crusaderky commented Dec 17, 2025

View reviewed changes

crusaderky marked this pull request as draft December 17, 2025 12:10

bedevere-app bot removed the awaiting review label Dec 17, 2025

crusaderky changed the title ~~WIP pixi builds for free-threading and tsan~~ WIP pixi builds for free-threading and TSAN Dec 17, 2025

This was referenced Dec 23, 2025

Add pixi builds for TSan + free-threading to aid downstream development. #143120

Open

Add note to reduce ASLR randomness for TSan Quansight-Labs/free-threaded-compatibility#304

Merged

hugovk changed the title ~~WIP pixi builds for free-threading and TSAN~~ gh-143120: WIP pixi builds for free-threading and TSAN Dec 23, 2025

hugovk added the skip news label Dec 23, 2025

hugovk added the infra CI, GitHub Actions, buildbots, Dependabot, etc. label Dec 23, 2025

ngoldbaum reviewed Dec 23, 2025

View reviewed changes

ngoldbaum mentioned this pull request Dec 23, 2025

ENH: TSan pixi package definitions numpy/numpy#30510

Draft

crusaderky force-pushed the tsan branch from eb0149f to 242081e Compare January 2, 2026 15:53

crusaderky commented Jan 2, 2026

View reviewed changes

crusaderky added 4 commits January 2, 2026 16:03

pixi builds for free-threading and tsan

d930e12

Rename to tsan-free-threading

57d45b9

Add libclang_rt hack

5ff29b4

Add troubleshooting for tsan

9a2a896

crusaderky force-pushed the tsan branch from b528931 to 9a2a896 Compare January 2, 2026 16:03

crusaderky marked this pull request as ready for review January 2, 2026 16:05

bedevere-app bot added the awaiting review label Jan 2, 2026

crusaderky changed the title ~~gh-143120: WIP pixi builds for free-threading and TSAN~~ gh-143120: pixi builds for free-threading and TSAN Jan 2, 2026

		cp "${BUILD_PREFIX}/lib/clang/21/lib/darwin/libclang_rt.asan_osx_dynamic.dylib" "${PREFIX}/lib/"
		cp "${BUILD_PREFIX}/lib/clang/21/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib" "${PREFIX}/lib/"

Uh oh!

gh-143120: pixi builds for free-threading and TSAN #142872

Are you sure you want to change the base?

gh-143120: pixi builds for free-threading and TSAN #142872

Conversation

crusaderky commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lucascolley commented Dec 17, 2025

Uh oh!

crusaderky commented Dec 18, 2025

Uh oh!

hugovk commented Dec 23, 2025

Uh oh!

ngoldbaum commented Dec 23, 2025

Uh oh!

lucascolley commented Dec 23, 2025

Uh oh!

ngoldbaum left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lucascolley Jan 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

crusaderky commented Jan 2, 2026

Uh oh!

ngoldbaum commented Jan 5, 2026

Uh oh!

isuruf commented Jan 5, 2026

Uh oh!

lucascolley commented Jan 5, 2026

Uh oh!

isuruf commented Jan 5, 2026

Uh oh!

isuruf commented Jan 5, 2026

Uh oh!

crusaderky commented Jan 6, 2026

Uh oh!

lucascolley commented Jan 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lucascolley commented Jan 6, 2026

Uh oh!

lucascolley commented Jan 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lucascolley commented Jan 6, 2026

Uh oh!

lucascolley commented Jan 6, 2026

Uh oh!

ngoldbaum commented Jan 6, 2026

crusaderky commented Dec 17, 2025 •

edited

Loading

lucascolley Jan 2, 2026 •

edited

Loading

lucascolley commented Jan 6, 2026 •

edited

Loading

lucascolley commented Jan 6, 2026 •

edited

Loading

lucascolley commented Jan 6, 2026 •

edited

Loading