Skip to content

MSTransferor: use different rucio account for RelVal input data transfers#10880

Open
amaltaro wants to merge 5 commits into
dmwm:masterfrom
amaltaro:fix-10253-take1
Open

MSTransferor: use different rucio account for RelVal input data transfers#10880
amaltaro wants to merge 5 commits into
dmwm:masterfrom
amaltaro:fix-10253-take1

Conversation

@amaltaro

@amaltaro amaltaro commented Oct 28, 2021

Copy link
Copy Markdown
Contributor

Fixes #10253

Status

In development

Description

This PR will provide 2 commits:

  1. MSTransferor: supporting a different rucio account for RelVal input data placement
  2. MSRuleCleaner: consider rules under this new relval rucio account during the workflow cleanup.
    TODO: changes at Global and Local WorkQueue level are also required. Likely better in a different issue/PR.

Note that a new key/value pair (relval_nodes_out_of_space) will be reported via the REST API.

Is it backward compatible (if not, which system it affects?)

MAYBE (yes, in terms of code/interface, no in terms of options provided for data location)

Related PRs

None

External dependencies / deployment changes

Deployment changes: dmwm/deployment#1101

@cmsdmwmbot

Copy link
Copy Markdown

Jenkins results:

  • Python2 Unit tests: failed
    • 4 new failures
  • Python3 Unit tests: failed
    • 4 new failures
    • 1 tests no longer failing
    • 1 changes in unstable tests
  • Python2 Pylint check: failed
    • 1 warnings and errors that must be fixed
    • 5 warnings
    • 31 comments to review
  • Python3 Pylint check: failed
    • 1 warnings and errors that must be fixed
    • 5 warnings
    • 35 comments to review
  • Pylint py3k check: succeeded
  • Pycodestyle check: succeeded
    • 18 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12533/artifact/artifacts/PullRequestReport.html

@cmsdmwmbot

Copy link
Copy Markdown

Jenkins results:

  • Python2 Unit tests: failed
    • 5 new failures
    • 1 tests deleted
    • 1 tests added
  • Python3 Unit tests: failed
    • 5 new failures
    • 1 tests deleted
    • 1 tests added
    • 1 changes in unstable tests
  • Python2 Pylint check: failed
    • 11 warnings and errors that must be fixed
    • 12 warnings
    • 79 comments to review
  • Python3 Pylint check: failed
    • 14 warnings and errors that must be fixed
    • 12 warnings
    • 86 comments to review
  • Pylint py3k check: failed
    • 2 warnings
  • Pycodestyle check: succeeded
    • 49 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12534/artifact/artifacts/PullRequestReport.html

@cmsdmwmbot

Copy link
Copy Markdown

Jenkins results:

  • Python2 Unit tests: failed
    • 7 new failures
    • 1 tests deleted
    • 1 tests added
  • Python3 Unit tests: failed
    • 7 new failures
    • 1 tests deleted
    • 1 tests added
    • 3 changes in unstable tests
  • Python2 Pylint check: failed
    • 11 warnings and errors that must be fixed
    • 9 warnings
    • 78 comments to review
  • Python3 Pylint check: failed
    • 16 warnings and errors that must be fixed
    • 9 warnings
    • 85 comments to review
  • Pylint py3k check: failed
    • 2 warnings
  • Pycodestyle check: succeeded
    • 45 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12535/artifact/artifacts/PullRequestReport.html

@cmsdmwmbot

Copy link
Copy Markdown

Jenkins results:

  • Python2 Unit tests: failed
    • 3 new failures
    • 1 tests deleted
    • 1 tests added
  • Python3 Unit tests: failed
    • 3 new failures
    • 1 tests deleted
    • 1 tests added
    • 1 changes in unstable tests
  • Python2 Pylint check: failed
    • 11 warnings and errors that must be fixed
    • 9 warnings
    • 78 comments to review
  • Python3 Pylint check: failed
    • 16 warnings and errors that must be fixed
    • 9 warnings
    • 85 comments to review
  • Pylint py3k check: failed
    • 2 warnings
  • Pycodestyle check: succeeded
    • 45 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12536/artifact/artifacts/PullRequestReport.html

more changes to the way we remove workflows from the queue

fix PycurlRucio
fixed unit tests

another unit test fix
@cmsdmwmbot

Copy link
Copy Markdown

Jenkins results:

  • Python2 Unit tests: succeeded
    • 1 tests deleted
    • 1 tests added
  • Python3 Unit tests: succeeded
    • 1 tests deleted
    • 1 tests added
    • 3 changes in unstable tests
  • Python2 Pylint check: failed
    • 12 warnings and errors that must be fixed
    • 9 warnings
    • 80 comments to review
  • Python3 Pylint check: failed
    • 13 warnings and errors that must be fixed
    • 9 warnings
    • 87 comments to review
  • Pylint py3k check: failed
    • 2 warnings
  • Pycodestyle check: succeeded
    • 45 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12537/artifact/artifacts/PullRequestReport.html

@cmsdmwmbot

Copy link
Copy Markdown

Jenkins results:

  • Python2 Unit tests: succeeded
    • 1 tests deleted
    • 1 tests added
    • 1 changes in unstable tests
  • Python3 Unit tests: succeeded
    • 1 tests deleted
    • 1 tests no longer failing
    • 1 tests added
    • 2 changes in unstable tests
  • Python2 Pylint check: failed
    • 12 warnings and errors that must be fixed
    • 9 warnings
    • 80 comments to review
  • Python3 Pylint check: failed
    • 13 warnings and errors that must be fixed
    • 9 warnings
    • 87 comments to review
  • Pylint py3k check: failed
    • 2 warnings
  • Pycodestyle check: succeeded
    • 45 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12539/artifact/artifacts/PullRequestReport.html

@cmsdmwmbot

Copy link
Copy Markdown

Can one of the admins verify this patch?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Adopt a different account for RelVal input data placement in MSTransferor

2 participants