copy from object-store to local filesystems

[ap--]

Hi @barneygale

Happy new year 🎆 ☺️

A universal-pathlib user just reported an issue that made me think about a likely very common scenario, when more ReadablePath subclasses become available. When using the .copy() / .copy_into() from a ReadablePath-like object for an objectstore-like filesystem to a local filesystem there's a potential issue:

Let's say I have an S3Path(ReadablePath) class that provides access to some data on s3 under the following two keys:

• s3://bucket/object
• s3://bucket/object/extra

Then it's impossible to represent the data with the same structure (object as a file and object/ as a directory) locally. Since most users will use pathlib.Path as their WritablePath subclass for copy targets, I was wondering if there should be some error handling added to WritablePath._copy_from. Maybe as a callback so that users can resolve if there is a name conflict. That error handling callback would have to be exposed in the 4 (copy|move)(_into)? methods.

What are your thoughts on this?
I'd be happy to work on a PR in cpython or here in case you think it makes sense to.

Cheers,
Andreas

xref: fsspec/filesystem_spec#1968
xref: fsspec/universal_pathlib#528

[ap--]

I implemented a version of _copy_from with an on_name_collision handler here that I am planning to roll out in universal-pathlib. If you think that this could be a viable solution I can work on a PR ☺️