Error saving project configuration to file 'file:/files/project-config.json' Could not delete "file:///files/project-config.json". #4951

nasrips · 2025-02-25T16:37:49Z

nasrips
Feb 25, 2025

Hello All,
Hope you're doing great.
I am using docker to run apache hop and I am getting two different behaviors.
The workflow was designed using Hop GUI 2.11.0.

1. Using hop:2.12.0

I am getting the exceptions below before the workflow starts.
PS: the workflow completed successfully!

2. Using hop:2.11.0

with the same docker config, I am facing a failure.

Logs using (hop:2.12.0)

2025/02/25 16:23:15 - Running the entrypoint script with PID 7
2025/02/25 16:23:15 - The project folder for poc-hop is set to: /files/
2025/02/25 16:23:15 - The specified project folder exists
2025/02/25 16:23:15 - Registering project poc-hop in the Hop container configuration
2025/02/25 16:23:15 - /opt/hop/hop-conf.sh --project=poc-hop --project-create --project-home='/files/' --project-config-file='project-config.json'
Creating project 'poc-hop'
Project 'poc-hop' was created for home folder : /files/
Error found during execution!
picocli.CommandLine$ExecutionException: There was an error handling options
at org.apache.hop.config.HopConfig.run(HopConfig.java:101)
at org.apache.hop.config.HopConfig.main(HopConfig.java:181)
Caused by: org.apache.hop.core.exception.HopException:
Error handling environment configuration options

Error saving project configuration to file 'file:/files/project-config.json'
Could not delete "file:///files/project-config.json".

    at org.apache.hop.projects.project.ManageProjectsOptionPlugin.handleOption(ManageProjectsOptionPlugin.java:196)
    at org.apache.hop.config.HopConfig.run(HopConfig.java:92)
    ... 1 more

Caused by: org.apache.hop.core.exception.HopException:
Error saving project configuration to file 'file:/files/project-config.json'
Could not delete "file:///files/project-config.json".

    at org.apache.hop.projects.project.Project.saveToFile(Project.java:133)
    at org.apache.hop.projects.project.ManageProjectsOptionPlugin.createProject(ManageProjectsOptionPlugin.java:389)
    at org.apache.hop.projects.project.ManageProjectsOptionPlugin.handleOption(ManageProjectsOptionPlugin.java:170)
    ... 2 more

Caused by: org.apache.commons.vfs2.FileSystemException: Could not delete "file:///files/project-config.json".
at org.apache.commons.vfs2.provider.AbstractFileObject.deleteSelf(AbstractFileObject.java:467)
at org.apache.commons.vfs2.provider.AbstractFileObject.delete(AbstractFileObject.java:423)
at org.apache.commons.vfs2.provider.AbstractFileObject.delete(AbstractFileObject.java:386)
at org.apache.hop.core.vfs.HopVfs.getOutputStream(HopVfs.java:420)
at org.apache.hop.projects.project.Project.saveToFile(Project.java:130)
... 4 more
Caused by: org.apache.commons.vfs2.FileSystemException: Could not delete "/files/project-config.json".
at org.apache.commons.vfs2.provider.local.LocalFile.doDelete(LocalFile.java:88)
at org.apache.commons.vfs2.provider.AbstractFileObject.deleteSelf(AbstractFileObject.java:460)
... 8 more

2025/02/25 16:23:20 - Registering environment dev-env in the Hop container configuration
2025/02/25 16:23:20 - /opt/hop/hop-conf.sh --environment-create --environment=dev-env --environment-project=poc-hop --environment-config-files='/files/environments/dev-config.json,/files/environments/dev-sensitive-config.json'
Creating environment 'dev-env'
Environment 'dev-env' was created in Hop configuration file /opt/hop/config/hop-config.json
Found existing environment configuration file: /files/environments/dev-config.json
Found existing environment configuration file: /files/environments/dev-sensitive-config.json
dev-env
Purpose: Apache Hop docker container
Project name: poc-hop
Config file: /files/environments/dev-config.json
Config file: /files/environments/dev-sensitive-config.json
2025/02/25 16:23:26 - Running a single hop workflow / pipeline (/files/workflows/pen2ldw_DataLoader.hwf)
2025/02/25 16:23:28 - HopRun - Referencing environment 'dev-env' for project poc-hop' in Apache Hop docker container
2025/02/25 16:23:28 - HopRun - Enabling project 'poc-hop'
2025/02/25 16:23:29 - HopRun - Starting workflow: /files/workflows/pen2ldw_DataLoader.hwf
2025/02/25 16:23:29 - pen2ldw_DataLoader - Start of workflow execution
2025/02/25 16:23:29 - pen2ldw_DataLoader - Starting action [File exists]
2025/02/25 16:23:29 - pen2ldw_DataLoader - Starting action [Check DB connections]
2025/02/25 16:23:29 - pen2ldw_DataLoader - Starting action [ParentChildWmlPenelopData]
2025/02/25 16:23:29 - pen2ldw_DataLoader - Launched action [ParentChildWmlPenelopData] in parallel.
2025/02/25 16:23:29 - pen2ldw_DataLoader - Starting action [OnlyXmlAttributePenelopData]
2025/02/25 16:23:29 - pen2ldw_DataLoader - Launched action [OnlyXmlAttributePenelopData] in parallel.
2025/02/25 16:23:29 - ParentChildWmlPenelopData - Using run configuration [local]
2025/02/25 16:23:29 - OnlyXmlAttributePenelopData - Using run configuration [local]
2025/02/25 16:23:29 - ParentChildWmlPenelopData - Executing this pipeline using the Local Pipeline Engine with run configuration 'local'
2025/02/25 16:23:29 - OnlyXmlAttributePenelopData - Executing this pipeline using the Local Pipeline Engine with run configuration 'local'
2025/02/25 16:23:29 - ParentChildWmlPenelopData - Execution started for pipeline [ParentChildWmlPenelopData]
...........................

Logs using (hop:2.11.0)

Creating project 'poc-hop'
Project 'poc-hop' was created for home folder : /files/
Error found during execution!
picocli.CommandLine$ExecutionException: There was an error handling options
at org.apache.hop.config.HopConfig.run(HopConfig.java:101)
at org.apache.hop.config.HopConfig.main(HopConfig.java:181)
Caused by: org.apache.hop.core.exception.HopException:
Error handling environment configuration options

Error saving project configuration to file 'file:/files/project-config.json'
Could not delete "file:///files/project-config.json".

    at org.apache.hop.projects.project.ManageProjectsOptionPlugin.handleOption(ManageProjectsOptionPlugin.java:196)
    at org.apache.hop.config.HopConfig.run(HopConfig.java:92)
    ... 1 more

Caused by: org.apache.hop.core.exception.HopException:
Error saving project configuration to file 'file:/files/project-config.json'
Could not delete "file:///files/project-config.json".

    at org.apache.hop.projects.project.Project.saveToFile(Project.java:133)
    at org.apache.hop.projects.project.ManageProjectsOptionPlugin.createProject(ManageProjectsOptionPlugin.java:389)
    at org.apache.hop.projects.project.ManageProjectsOptionPlugin.handleOption(ManageProjectsOptionPlugin.java:170)
    ... 2 more

Caused by: org.apache.commons.vfs2.FileSystemException: Could not delete "file:///files/project-config.json".
at org.apache.commons.vfs2.provider.AbstractFileObject.deleteSelf(AbstractFileObject.java:467)
at org.apache.commons.vfs2.provider.AbstractFileObject.delete(AbstractFileObject.java:423)
at org.apache.commons.vfs2.provider.AbstractFileObject.delete(AbstractFileObject.java:386)
at org.apache.hop.core.vfs.HopVfs.getOutputStream(HopVfs.java:420)
at org.apache.hop.projects.project.Project.saveToFile(Project.java:130)
... 4 more
Caused by: org.apache.commons.vfs2.FileSystemException: Could not delete "/files/project-config.json".
at org.apache.commons.vfs2.provider.local.LocalFile.doDelete(LocalFile.java:88)
at org.apache.commons.vfs2.provider.AbstractFileObject.deleteSelf(AbstractFileObject.java:460)
... 8 more

2025/02/25 16:32:26 - Registering environment dev-env in the Hop container configuration
2025/02/25 16:32:26 - /opt/hop/hop-conf.sh --environment-create --environment=dev-env --environment-project=poc-hop --environment-config-files='/files/environments/dev-config.json,/files/environments/dev-sensitive-config.json'
Creating environment 'dev-env'
Environment 'dev-env' was created in Hop configuration file /opt/hop/config/hop-config.json
Found existing environment configuration file: /files/environments/dev-config.json
Found existing environment configuration file: /files/environments/dev-sensitive-config.json
dev-env
Purpose: Apache Hop docker container
Project name: poc-hop
Config file: /files/environments/dev-config.json
Config file: /files/environments/dev-sensitive-config.json
2025/02/25 16:32:31 - Running a single hop workflow / pipeline (/files/workflows/pen2ldw_DataLoader.hwf)
2025/02/25 16:32:34 - HopRun - Referencing environment 'dev-env' for project poc-hop' in Apache Hop docker container
2025/02/25 16:32:34 - HopRun - Enabling project 'poc-hop'
2025/02/25 16:32:34 - HopRun - Starting workflow: /files/workflows/pen2ldw_DataLoader.hwf
2025/02/25 16:32:34 - pen2ldw_DataLoader - Start of workflow execution
2025/02/25 16:32:34 - pen2ldw_DataLoader - ERROR: A serious error occurred during workflow execution:
2025/02/25 16:32:34 - pen2ldw_DataLoader - Error validating or creating folder '/files//metadata/workflow-log'to store JSON serialized objects in from class org.apache.hop.reflection.workflow.meta.WorkflowLog
2025/02/25 16:32:34 - pen2ldw_DataLoader - ERROR: org.apache.hop.core.exception.HopException:
2025/02/25 16:32:34 - pen2ldw_DataLoader - Error validating or creating folder '/files//metadata/workflow-log'to store JSON serialized objects in from class org.apache.hop.reflection.workflow.meta.WorkflowLog
2025/02/25 16:32:34 - pen2ldw_DataLoader -
2025/02/25 16:32:34 - pen2ldw_DataLoader - at org.apache.hop.metadata.serializer.json.JsonMetadataProvider.getSerializer(JsonMetadataProvider.java:101)
2025/02/25 16:32:34 - pen2ldw_DataLoader - at org.apache.hop.metadata.serializer.multi.MultiMetadataSerializer.loadAll(MultiMetadataSerializer.java:165)
2025/02/25 16:32:34 - pen2ldw_DataLoader - at org.apache.hop.reflection.workflow.xp.WorkflowStartLoggingXp.callExtensionPoint(WorkflowStartLoggingXp.java:56)
2025/02/25 16:32:34 - pen2ldw_DataLoader - at org.apache.hop.reflection.workflow.xp.WorkflowStartLoggingXp.callExtensionPoint(WorkflowStartLoggingXp.java:43)
2025/02/25 16:32:34 - pen2ldw_DataLoader - at org.apache.hop.core.extension.ExtensionPointMap.callExtensionPoint(ExtensionPointMap.java:149)
2025/02/25 16:32:34 - pen2ldw_DataLoader - at org.apache.hop.core.extension.ExtensionPointHandler.callExtensionPoint(ExtensionPointHandler.java:41)
2025/02/25 16:32:34 - pen2ldw_DataLoader - at org.apache.hop.workflow.Workflow.executeFromStart(Workflow.java:387)
2025/02/25 16:32:34 - pen2ldw_DataLoader - at org.apache.hop.workflow.Workflow.startExecution(Workflow.java:313)
2025/02/25 16:32:34 - pen2ldw_DataLoader - at org.apache.hop.workflow.engines.local.LocalWorkflowEngine.startExecution(LocalWorkflowEngine.java:248)
2025/02/25 16:32:34 - pen2ldw_DataLoader - at org.apache.hop.run.HopRun.runWorkflow(HopRun.java:437)
2025/02/25 16:32:34 - pen2ldw_DataLoader - at org.apache.hop.run.HopRun.runWorkflow(HopRun.java:388)
2025/02/25 16:32:34 - pen2ldw_DataLoader - at org.apache.hop.run.HopRun.run(HopRun.java:205)
2025/02/25 16:32:34 - pen2ldw_DataLoader - at org.apache.hop.run.HopRun.main(HopRun.java:929)
2025/02/25 16:32:34 - pen2ldw_DataLoader - Workflow execution finished

Am I missing something here?
Thank you for your help.
BR.

bamaer · 2025-02-25T18:44:36Z

bamaer
Feb 25, 2025
Collaborator

Looks like a file/folder permission issue.
Just to be clear: can you run the exact same docker run command without issues in 2.11 and with this issue in 2.12?

3 replies

nasrips Feb 25, 2025
Author

The error below is always displayed regardless the version:
Error saving project configuration to file 'file:/files/project-config.json'
Could not delete "file:///files/project-config.json".

Using hop 2.12 the workflow continue running and insert the data, but with hop 2.11 (also 2.10 and 2.9) I got the error below in addition.
2025/02/25 16:32:34 - pen2ldw_DataLoader - ERROR: A serious error occurred during workflow execution:
2025/02/25 16:32:34 - pen2ldw_DataLoader - Error validating or creating folder '/files//metadata/workflow-log'to store JSON serialized objects in from class org.apache.hop.reflection.workflow.meta.WorkflowLog
2025/02/25 16:32:34 - pen2ldw_DataLoader - ERROR: org.apache.hop.core.exception.HopException:
2025/02/25 16:32:34 - pen2ldw_DataLoader - Error validating or creating folder '/files//metadata/workflow-log'to store JSON serialized objects in from class org.apache.hop.reflection.workflow.meta.WorkflowLog
2025/02/25 16:32:34 - pen2ldw_DataLoader -
2025/02/25 16:32:34 - pen2ldw_DataLoader - at org.apache.hop.metadata.serializer.json.JsonMetadataProvider.getSerializer(JsonMetadataProvider.java:101)

So I tried to create a workflow logger and then I got the error below (a new one) :

2025/02/25 17:20:45 - pen2ldw_DataLoader - Start of workflow execution
2025/02/25 17:20:45 - pen2ldw_DataLoader - Stop workflow execution
2025/02/25 17:20:45 - pen2ldw_DataLoader - ERROR: A serious error occurred during workflow execution:
2025/02/25 17:20:45 - pen2ldw_DataLoader - Error handling Workflow Log metadata object 'workflow-log' at the start of pipeline: pen2ldw_DataLoader
2025/02/25 17:20:45 - pen2ldw_DataLoader -
2025/02/25 17:20:45 - pen2ldw_DataLoader - Error validating or creating folder '/files//metadata/pipeline-probe'to store JSON serialized objects in from class org.apache.hop.reflection.probe.meta.PipelineProbe
2025/02/25 17:20:45 - pen2ldw_DataLoader - ERROR: org.apache.hop.core.exception.HopException:
2025/02/25 17:20:45 - pen2ldw_DataLoader - Error handling Workflow Log metadata object 'workflow-log' at the start of pipeline: pen2ldw_DataLoader
2025/02/25 17:20:45 - pen2ldw_DataLoader -
2025/02/25 17:20:45 - pen2ldw_DataLoader - Error validating or creating folder '/files//metadata/pipeline-probe'to store JSON serialized objects in from class org.apache.hop.reflection.probe.meta.PipelineProbe

The issue with the rights is at the host level? should I run docker using the hop user who need to have rw on the volume where the code and config files live?
Thank you for your help.

hansva Feb 25, 2025
Collaborator

It will have to be a RW volume indeed at the moment. Maybe we should figure out a way to make hop work with RO rights inside the container. The process currently creates a new project and when it does that it will update some files.

in-da Feb 25, 2025

I see, I will give it a try and keep you updated.
Thank you for your help!

Navigation Menu

Repository navigation

Error saving project configuration to file 'file:/files/project-config.json' Could not delete "file:///files/project-config.json". #4951

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 1 comment 3 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Select a reply

Uh oh!

Error saving project configuration to file 'file:/files/project-config.json' Could not delete "file:///files/project-config.json". #4951

Uh oh!

Uh oh!

nasrips Feb 25, 2025

1. Using hop:2.12.0

2. Using hop:2.11.0

Logs using (hop:2.12.0)

Logs using (hop:2.11.0)

Replies: 1 comment · 3 replies

Uh oh!

bamaer Feb 25, 2025 Collaborator

Uh oh!

Uh oh!

nasrips Feb 25, 2025 Author

Uh oh!

hansva Feb 25, 2025 Collaborator

Uh oh!

Uh oh!

in-da Feb 25, 2025

nasrips
Feb 25, 2025

Replies: 1 comment 3 replies

bamaer
Feb 25, 2025
Collaborator

nasrips Feb 25, 2025
Author

hansva Feb 25, 2025
Collaborator