CVE pipeline by djgalloway · Pull Request #2614 · ceph/ceph-build

djgalloway · 2026-06-16T23:39:35Z

Packages get pushed to pulp.front (internal only). Containers get pushed to an internal-only quay registry.

This enables us to build packages and containers from ceph-private.git privately. This includes Jenkins build logs, artifacts, packages, and containers. Packages will be pushed to the internal-only pulp instance and containers to the internal-only quay instance. To avoid job duplication, each job pulls in the respective Jenkinsfiles e.g., - cve-pipelinen -> ceph-dev-pipeline - cve-source-dist -> ceph-source-dist The only difference is the parameters we pass to the job. The key differences there are CEPH_REPO points to ceph-private.git whereas ceph-dev-pipeline is not capable of doing so. We also tell the pipeline not to use a public chacra instance. Signed-off-by: David Galloway <david.galloway@ibm.com>

This logic was chosen to preserve logic the THROWAWAY parameter provides regardless of package repository repository choice. Signed-off-by: David Galloway <david.galloway@ibm.com>

Signed-off-by: David Galloway <david.galloway@ibm.com>

Unsure if anything even uses chacra_url. Containerfile uses `jq -r .[0].url`. Signed-off-by: David Galloway <david.galloway@ibm.com>

Signed-off-by: David Galloway <david.galloway@ibm.com>

djgalloway · 2026-06-17T18:11:39Z

curl -s "https://shaman.ceph.com/api/search/?project=ceph&distros=centos/9/x86_64&&ref=wip-pulp-container&flavor=default&sha1=21297a321fbc3edb12858b25bdb878cc78aed4bb" | jq .
[
  {
    "url": "https://pulp.front.sepia.ceph.com/pulp/content/repos/ceph/wip-pulp-container/21297a321fbc3edb12858b25bdb878cc78aed4bb/centos/9/flavors/default/",
    "chacra_url": "https://pulp.front.sepia.ceph.com/pulp/content/repos/ceph/wip-pulp-container/21297a321fbc3edb12858b25bdb878cc78aed4bb/centos/9/flavors/default/",
    "ref": "wip-pulp-container",
    "sha1": "21297a321fbc3edb12858b25bdb878cc78aed4bb",
    "distro": "centos",
    "distro_version": "9",
    "distro_codename": null,
    "modified": "2026-06-17 18:04:30.190595",
    "status": "ready",
    "flavor": "default",
    "project": "ceph",
    "archs": [
      "x86_64",
      "source"
    ],
    "extra": {
      "version": "21.3.0-187-g21297a32",
      "package_manager_version": "21.3.0-187.g21297a32",
      "build_url": "https://jenkins.ceph.com/job/cve-pipeline/18/",
      "root_build_cause": "",
      "node_name": "10.20.192.23+soko13",
      "job_name": "cve-pipeline"
    }
  }
]

curl https://pulp.front.sepia.ceph.com/pulp/content/repos/ceph/wip-pulp-container/21297a321fbc3edb12858b25bdb878cc78aed4bb/centos/9/flavors/default/

<html>
<head><title>Index of repos/ceph/wip-pulp-container/21297a321fbc3edb12858b25bdb878cc78aed4bb/centos/9/flavors/default/</title></head>
<body bgcolor="white">
<h1>Index of repos/ceph/wip-pulp-container/21297a321fbc3edb12858b25bdb878cc78aed4bb/centos/9/flavors/default/</h1>
<hr><pre><a href="../">../</a>
<a href="SRPMS/">SRPMS/</a>                                                                                                
<a href="noarch/">noarch/</a>                                                                                               
<a href="x86_64/">x86_64/</a>                                                                                               
</pre><hr></body>
</html>

djgalloway · 2026-06-17T18:24:40Z

Requires a minimal change to the Containerfile as well ceph/ceph#69555

dmick

mostly good, a few questions

dmick · 2026-06-20T00:02:03Z

+    PACKAGE_MANAGER_VERSION=""
+    if [ -r "$WORKSPACE/pulp_repo_info" ]; then
+        source "$WORKSPACE/pulp_repo_info"
+    fi


is it an error not to have a PACKAGE_MANAGER_VERSION? How does this synchronize?

It gets written here: https://github.com/ceph/ceph-build/pull/2614/changes/BASE..045d140e7cc55c72739aef8c6fbf3af18d7ac860#diff-e43616de828404d15446d65b52a9351b6042eefe915702c04074e6b674e46d71R424-R425.

pulp_upload.sh runs immediately before notify_shaman_pulp_repo.sh in the same step:

ceph-build/ceph-dev-pipeline/build/Jenkinsfile

Lines 621 to 622 in 49b2430

./scripts/pulp_upload.sh

./scripts/notify_shaman_pulp_repo.sh ready ceph ${os.name} ${os.version_name} $ARCH https://pulp.front.sepia.ceph.com/pulp/content/repos/ceph/${env.BRANCH}/${env.SHA1}/${os.name}/${os.version_name}/flavors/${env.FLAVOR}/

OK. So maybe mention in a comment "PACKAGE_MANAGER_VERSION is set in this file I'm about to source" and have an error condition in case it doesn't arrive for whatever reason, just to harden it a bit. Two scripts changing independently are easier to break in future

dmick · 2026-06-22T20:52:46Z

+    PACKAGE_MANAGER_VERSION=""
+    if [ -r "$WORKSPACE/pulp_repo_info" ]; then
+        source "$WORKSPACE/pulp_repo_info"
+    fi


OK. So maybe mention in a comment "PACKAGE_MANAGER_VERSION is set in this file I'm about to source" and have an error condition in case it doesn't arrive for whatever reason, just to harden it a bit. Two scripts changing independently are easier to break in future

zmc

I very much appreciate the approach here. Just a couple very minor questions / suggestions

zmc · 2026-06-23T23:09:47Z

+                    repo_base_url = pulp_repo_base
+                  } else {
+                    spec_project_url = "${chacra_url}r/ceph/${env.BRANCH}/${env.SHA1}/${os.name}/${os.version_name}/flavors/${env.FLAVOR}/"
+                    repo_base_url = "${chacra_url}/r/ceph/${env.BRANCH}/${env.SHA1}/${os.name}/${os.version_name}/flavors/${env.FLAVOR}"


there might be a typo here. I see the trailing / and assume it's desired, but the ${chacra_url}r/ceph vs ${chacra_url}/r/ceph feels less intentional. I'd rather see something like:

def string_a = "foo/bar" def string_b = "${string_a}/"

zmc · 2026-06-23T23:11:24Z

                    export FLAVOR="${env.FLAVOR}"

                    ./scripts/pulp_upload.sh
+                    ./scripts/notify_shaman_pulp_repo.sh ready ceph ${os.name} ${os.version_name} $ARCH https://pulp.front.sepia.ceph.com/pulp/content/repos/ceph/${env.BRANCH}/${env.SHA1}/${os.name}/${os.version_name}/flavors/${env.FLAVOR}/


can we find a way to just use spec_project_url ?

djgalloway added 6 commits June 12, 2026 13:46

ceph-dev-pipeline: Add CHACRA_URL parameter

329da2b

This logic was chosen to preserve logic the THROWAWAY parameter provides regardless of package repository repository choice. Signed-off-by: David Galloway <david.galloway@ibm.com>

pulp_upload.sh: Do not overwrite BRANCH name

66ce43e

Signed-off-by: David Galloway <david.galloway@ibm.com>

pulp_upload.sh: Create repository before upload

f9bf330

Signed-off-by: David Galloway <david.galloway@ibm.com>

ceph-dev-pipeline: Notify shaman of pulp repo URLs

045d140

Unsure if anything even uses chacra_url. Containerfile uses `jq -r .[0].url`. Signed-off-by: David Galloway <david.galloway@ibm.com>

ceph-dev-pipeline: Pulp repo support in ceph-release RPM's ceph.repo

49b2430

Signed-off-by: David Galloway <david.galloway@ibm.com>

djgalloway requested review from dmick and zmc June 18, 2026 20:48

dmick reviewed Jun 20, 2026

View reviewed changes

dmick approved these changes Jun 22, 2026

View reviewed changes

zmc reviewed Jun 23, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

CVE pipeline#2614

CVE pipeline#2614
djgalloway wants to merge 6 commits into
mainfrom
cve-pipeline

djgalloway commented Jun 16, 2026 •

edited

Loading

Uh oh!

djgalloway commented Jun 17, 2026

Uh oh!

djgalloway commented Jun 17, 2026 •

edited

Loading

Uh oh!

dmick left a comment

Uh oh!

Uh oh!

Uh oh!

dmick Jun 20, 2026

Uh oh!

djgalloway Jun 22, 2026

Uh oh!

dmick Jun 22, 2026

Uh oh!

dmick Jun 22, 2026

Uh oh!

zmc left a comment

Uh oh!

zmc Jun 23, 2026

Uh oh!

zmc Jun 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	./scripts/pulp_upload.sh
	./scripts/notify_shaman_pulp_repo.sh ready ceph ${os.name} ${os.version_name} $ARCH https://pulp.front.sepia.ceph.com/pulp/content/repos/ceph/${env.BRANCH}/${env.SHA1}/${os.name}/${os.version_name}/flavors/${env.FLAVOR}/

Uh oh!

Conversation

djgalloway commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

djgalloway commented Jun 17, 2026

Uh oh!

djgalloway commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dmick left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

dmick Jun 20, 2026

Choose a reason for hiding this comment

Uh oh!

djgalloway Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

dmick Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

dmick Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

zmc left a comment

Choose a reason for hiding this comment

Uh oh!

zmc Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

zmc Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

djgalloway commented Jun 16, 2026 •

edited

Loading

djgalloway commented Jun 17, 2026 •

edited

Loading