Skip to content

[9.1] feat (PS): add custom param to production step - #8874

Open
arrabito wants to merge 5 commits into
DIRACGrid:integrationfrom
arrabito:feat/add-custom-param-to-ps
Open

arrabito wants to merge 5 commits into
DIRACGrid:integrationfrom
arrabito:feat/add-custom-param-to-ps

Conversation

@arrabito

@arrabito arrabito commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

BEGINRELEASENOTES

  • PS
    NEW: add the possibility to define custom parameters for the production steps

ENDRELEASENOTES

@arrabito
arrabito force-pushed the feat/add-custom-param-to-ps branch from fba73a1 to 040b153 Compare November 6, 2026 09:07

@fstagni fstagni left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are also no tests added.

for paramName, paramValue in (customParameters or {}).items():
res = self.transClient.setTransformationParameter(transID, paramName, paramValue)
if not res["OK"]:
deleteRes = self.transClient.deleteTransformation(transID)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain your reasoning here? As I read it, if you fail to set a parameter you remove the whole transformation.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is also worth logging which parameter failed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, there is no reason to delete the transformation here. I'll remove it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, there is no reason to delete the transformation here. I'll remove it.

Thinking a little bit further, the original reason for which I added the deletion here it's to not leave partially configured transformations.

Does the logic make sense for you?

return S_OK()

def addTransformationStep(self, stepID, prodID):
def addTransformationStep(self, stepID, prodID, customParameters=None):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If one of these parameter is "Status", or "Body", will it overwrite the status or body of the transformation? Is that the scope?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In principle Body should not be overwritten as it's in the _STEP_ATTRIBUTES , see:

https://github.com/arrabito/DIRAC/blob/feat/add-custom-param-to-ps/src/DIRAC/ProductionSystem/Client/ProductionStep.py#L51

but it's true the Status would be.

I guess that I should also add Status to the list of _STEP_ATTRIBUTES and also all other Fields in the Transformations table, right?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In principle Body should not be overwritten as it's in the _STEP_ATTRIBUTES , see:

https://github.com/arrabito/DIRAC/blob/feat/add-custom-param-to-ps/src/DIRAC/ProductionSystem/Client/ProductionStep.py#L51

but it's true the Status would be.

I guess that I should also add Status to the list of _STEP_ATTRIBUTES and also all other Fields in the Transformations table, right?

I believe so.

@arrabito

arrabito commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

There are also no tests added.

You mean unit tests or updating the existing integration tests for the Production System?

@arrabito
arrabito force-pushed the feat/add-custom-param-to-ps branch from b487083 to a9f8e6d Compare November 9, 2026 08:14
@arrabito

arrabito commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

I've added some unit tests. I see however that some CI tests are failing but it does not seem related to my changes.
Let me know if you need further input from me.

@fstagni

fstagni commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

I've added some unit tests. I see however that some CI tests are failing but it does not seem related to my changes. Let me know if you need further input from me.

Prob fixed by #8876

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants