Add a script to allow more users to easily upload artifacts
This makes it so that anyone with access to buildkite can upload an
artifact to the build server. Existing protections entail:
1. Doesn't allow users to run this against anything other than the
master branch.
2. Does not allow overwriting existing files.
3. Does not touch files outside of the
/data/files/frc971/Build-Dependencies/ directory.
Certainly no riskier than our regular CI at this stage.
This does cause a surprising amount of disk usage on the software971
server, due to bazel it extracting the amd64 toolchains.
Steps required to get this set up were:
1. Install the buildkite-agent apt package.
2. Configure the token in /buildkite/buildkite-agent.cfg, and set
queue=deploy.
3. Start the buildkite-agent service.
4. Add the buildkite-agent user to the www-data group.
5. Add a new gerrit public key for the buildkite user corresponding to
the new agent.
6. Set up the https://buildkite.com/spartan-robotics/deploy-artifact
pipeline.
Signed-off-by: James Kuszmaul <jabukuszmaul+collab@gmail.com>
Change-Id: I89416d23e8d75102314c7aa1dbe8b9fd64f4f762
diff --git a/tools/rehosting/rehost_test.py b/tools/rehosting/rehost_test.py
new file mode 100644
index 0000000..da316e2
--- /dev/null
+++ b/tools/rehosting/rehost_test.py
@@ -0,0 +1,37 @@
+import unittest
+import unittest.mock
+from tools.rehosting import rehost
+from pathlib import Path
+import os
+
+
+class TestRehost(unittest.TestCase):
+
+ def test_url_validation(self):
+ self.assertEqual("http://google.com",
+ rehost.validate_url("http://google.com"))
+ self.assertRaisesRegex(Exception, "Invalid URL", rehost.validate_url,
+ "file:///some/secret")
+ self.assertRaisesRegex(Exception, "Invalid URL", rehost.validate_url,
+ "http://10.0.0.0/secret")
+
+ def test_url_to_path(self):
+ test_dir = os.getenv("TEST_TMPDIR", "/tmp/")
+ with unittest.mock.patch.object(rehost, "BUILD_DEPENDENCIES_PATH",
+ test_dir):
+ existing_file = test_dir + "/exists.com"
+ with open(existing_file, 'w') as f:
+ f.write('string')
+ self.assertEqual(
+ Path(test_dir) / "example.com/foo/bar",
+ rehost.url_to_path("https://example.com/foo/bar"))
+ self.assertRaisesRegex(ValueError,
+ f"not in the subpath of '{test_dir}'",
+ rehost.url_to_path,
+ "https://example.com/../../bar")
+ self.assertRaisesRegex(FileExistsError, "There is already a file",
+ rehost.url_to_path, "https://exists.com")
+
+
+if __name__ == "__main__":
+ unittest.main()