Python: Sandbox the requirements update script a bit
We currently get errors trying to run the requirements update process.
The issue is that pip tries to set up packages that require various
host tools just to run the setup.py script.
The patch here runs the update process in a container.
$ bazel run //tools/python:requirements.update
WARNING: Overflow when watching local filesystem for changes... temporarily falling back to manually checking files for changes
INFO: Build option --compilation_mode has changed, discarding analysis cache.
INFO: Analyzed target //tools/python:requirements.update (84 packages loaded, 27041 targets configured).
INFO: Found 1 target...
Target //tools/python:requirements.update up-to-date:
bazel-bin/tools/python/requirements.update
INFO: Elapsed time: 3.172s, Critical Path: 0.09s
INFO: 4 processes: 4 internal.
INFO: Build completed successfully, 4 total actions
INFO: Running command line: bazel-bin/tools/python/requirements.update tools/python/requirements.txt tools/python/requirements.lock.txt None None None //tools/python:requirements.update
Updating tools/python/requirements.lock.txt
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> [45 lines of output]
Collecting setuptools
Downloading setuptools-69.0.3-py3-none-any.whl (819 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 819.5/819.5 kB 11.4 MB/s eta 0:00:00
Collecting wheel
Downloading wheel-0.42.0-py3-none-any.whl (65 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.4/65.4 kB 6.2 MB/s eta 0:00:00
Collecting pycairo
Downloading pycairo-1.25.1.tar.gz (347 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 347.1/347.1 kB 16.1 MB/s eta 0:00:00
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Installing backend dependencies: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Building wheels for collected packages: pycairo
Building wheel for pycairo (pyproject.toml): started
Building wheel for pycairo (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error
× Building wheel for pycairo (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [14 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-39
creating build/lib.linux-x86_64-cpython-39/cairo
copying cairo/__init__.py -> build/lib.linux-x86_64-cpython-39/cairo
copying cairo/__init__.pyi -> build/lib.linux-x86_64-cpython-39/cairo
copying cairo/py.typed -> build/lib.linux-x86_64-cpython-39/cairo
warning: build_py: byte-compiling is disabled, skipping.
running build_ext
'pkg-config' not found.
Command ['pkg-config', '--print-errors', '--exists', 'cairo >= 1.15.10']
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pycairo
Failed to build pycairo
ERROR: Could not build wheels for pycairo, which is required to install pyproject.toml-based projects
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
Traceback (most recent call last):
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/rules_python/python/pip_install/pip_compile.py", line 127, in <module>
cli()
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__click/click/core.py", line 1137, in __call__
return self.main(*args, **kwargs)
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__click/click/core.py", line 1062, in main
rv = self.invoke(ctx)
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__click/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__click/click/core.py", line 763, in invoke
return __callback(*args, **kwargs)
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__click/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__pip_tools/piptools/scripts/compile.py", line 487, in cli
results = resolver.resolve(max_rounds=max_rounds)
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__pip_tools/piptools/resolver.py", line 266, in resolve
has_changed, best_matches = self._resolve_one_round()
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__pip_tools/piptools/resolver.py", line 356, in _resolve_one_round
their_constraints.extend(self._iter_dependencies(best_match))
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__pip_tools/piptools/resolver.py", line 469, in _iter_dependencies
dependencies = self.repository.get_dependencies(ireq)
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__pip_tools/piptools/repositories/local.py", line 85, in get_dependencies
return self.repository.get_dependencies(ireq)
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__pip_tools/piptools/repositories/pypi.py", line 250, in get_dependencies
self._dependencies_cache[ireq] = self.resolve_reqs(
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__pip_tools/piptools/repositories/pypi.py", line 213, in resolve_reqs
results = resolver._resolve_one(reqset, ireq)
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__pip/pip/_internal/resolution/legacy/resolver.py", line 509, in _resolve_one
dist = self._get_dist_for(req_to_install)
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__pip/pip/_internal/resolution/legacy/resolver.py", line 462, in _get_dist_for
dist = self.preparer.prepare_linked_requirement(req)
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__pip/pip/_internal/operations/prepare.py", line 438, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__pip/pip/_internal/operations/prepare.py", line 524, in _prepare_linked_requirement
dist = _get_prepared_distribution(
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__pip/pip/_internal/operations/prepare.py", line 68, in _get_prepared_distribution
abstract_dist.prepare_distribution_metadata(
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__pip/pip/_internal/distributions/sdist.py", line 38, in prepare_distribution_metadata
self._prepare_build_backend(finder)
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__pip/pip/_internal/distributions/sdist.py", line 70, in _prepare_build_backend
self.req.build_env.install_requirements(
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__pip/pip/_internal/build_env.py", line 196, in install_requirements
self._install_requirements(
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__pip/pip/_internal/build_env.py", line 254, in _install_requirements
call_subprocess(
File "/home/james/.cache/bazel/_bazel_james/a53c1076cfa8612afda853f7c8f2a68f/execroot/org_frc971/bazel-out/k8-fastbuild/bin/tools/python/requirements.update.runfiles/pypi__pip/pip/_internal/utils/subprocess.py", line 224, in call_subprocess
raise error
pip._internal.exceptions.InstallationSubprocessError: pip subprocess to install build dependencies exited with 1
Change-Id: Idc0e66994c7e0282f7ef8825f737fd407c0cc0a4
Signed-off-by: Philipp Schrader <philipp.schrader@gmail.com>
diff --git a/tools/python/update_helper_files/Dockerfile b/tools/python/update_helper_files/Dockerfile
new file mode 100644
index 0000000..3a4d42c
--- /dev/null
+++ b/tools/python/update_helper_files/Dockerfile
@@ -0,0 +1,10 @@
+# This Dockerfile sets up a container with the minimum number of things to make
+# //tools/python:requirements.update target happy.
+
+FROM debian:12
+
+RUN apt update
+RUN DEBIAN_FRONTEND=noninteractive apt install --no-install-recommends -y clang
+RUN DEBIAN_FRONTEND=noninteractive apt install --no-install-recommends -y python3
+RUN DEBIAN_FRONTEND=noninteractive apt install --no-install-recommends -y pkgconf
+RUN DEBIAN_FRONTEND=noninteractive apt install --no-install-recommends -y libcairo2-dev