Skip to content

Issue #8671 : Read and write Windows shares through an SMB VFS connection - #8717

Merged
mattcasters merged 6 commits into
apache:mainfrom
mattcasters:issue-8671
Oct 6, 2026
Merged

mattcasters merged 6 commits into
apache:mainfrom
mattcasters:issue-8671

Conversation

@mattcasters

@mattcasters mattcasters commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Named SMB connection so Hop can read and write a Windows or Samba share without mounting it. The scheme is the connection name (finance:///reports/daily.csv). Host, share, and password stay on the metadata.

The client is smbj and speaks SMB 2.0.2 through 3.1.1. SMB1 is not offered. Sign-in in this version is NTLMv2 (username, password, optional domain) and guest. Kerberos is the next authenticator and plugs into the same smb-connection metadata type.

Integration tests run as integration-tests/smb in the Jenkins suite. Samba is a compose service with SMB1 off, NTLMv2 only, and mandatory signing, plus a second service that requires SMB3 encryption. The file operations are shared pipelines under integration-tests/vfs. Pointing the other VFS drivers at those pipelines is #8719.

fixes #8671


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Run mvn clean install apache-rat:check to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
  • If you have a group of commits related to the same change, please squash your commits into one and force push your branch using git rebase -i.
  • Mention the appropriate issue in your description (for example: addresses #123), if applicable.

To make clear that you license your contribution under the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

…connection

Named SMB2/SMB3 connection over smbj, with NTLMv2 and guest authentication.
Kerberos stays a later authenticator on the same metadata type. Integration
tests run against Samba in Docker with SMB1 disabled.
Run create, write, read, append, list, copy, move, and delete against
Samba from integration-tests/smb, including a rooted folder, SMB3
encryption, a bad password, and a missing share. Drop the
Testcontainers image. Relative paths resolve against the parent, a
move creates missing folders, and the share root cannot be deleted.
@mattcasters
mattcasters marked this pull request as ready for review October 1, 2026 18:54
@mattcasters

Copy link
Copy Markdown
Contributor Author

A few issues:

Missing bouncycastle lib during assembly.
Update dependency to org.bouncycastle:bcprov-jdk18on:1.86 (security)

Plus working with the VFS explorer:

2026-10-02T13:28:49,958 WARN  [com.hierynomus.smbj.share.Directory] - Directory close failed for \\some-windows-server\share\folder1,DiskShare[\\some-windws-server\share],SMB2FileId{persistentHandle=c5 34 30 07 71 95 bf 6a}
com.hierynomus.smbj.common.SMBRuntimeException: DiskShare has already been closed
        at com.hierynomus.smbj.share.Share.send(Share.java:373)
        at com.hierynomus.smbj.share.Share.sendReceive(Share.java:367)
        at com.hierynomus.smbj.share.Share.closeFileId(Share.java:180)
        at com.hierynomus.smbj.share.Open.close(Open.java:92)
        at com.hierynomus.smbj.share.Open.closeSilently(Open.java:97)
        at com.hierynomus.smbj.connection.LeaseManager.close(LeaseManager.java:103)
        at com.hierynomus.smbj.connection.Connection.close(Connection.java:192)
        at com.hierynomus.smbj.connection.Connection.close(Connection.java:162)
        at org.apache.hop.vfs.smb.SmbjSmbShare.closeQuietly(SmbjSmbShare.java:375)
        at org.apache.hop.vfs.smb.SmbjSmbShare.close(SmbjSmbShare.java:272)
        at org.apache.hop.vfs.smb.SmbConnectionTester.test(SmbConnectionTester.java:46)
        at org.apache.hop.vfs.smb.metadata.SmbConnection.testConnection(SmbConnection.java:158)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at org.apache.hop.core.gui.plugin.GuiWidgetMethodInvoker.invoke(GuiWidgetMethodInvoker.java:44)
        at org.apache.hop.ui.core.gui.GuiCompositeWidgets.lambda$5(GuiCompositeWidgets.java:948)
        at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:91)
        at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4363)
        at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1217)
        at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4161)
        at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3749)
        at org.apache.hop.ui.hopgui.HopGui.open(HopGui.java:879)
        at org.apache.hop.ui.hopgui.HopGui.main(HopGui.java:697)

@mattcasters
mattcasters marked this pull request as draft October 2, 2026 11:35
@mattcasters

Copy link
Copy Markdown
Contributor Author

Successfully tested against corporate shares.

@mattcasters
mattcasters marked this pull request as ready for review October 5, 2026 09:47
@mattcasters

mattcasters commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor Author

Writing to multiple xlsx files at once succeeded. However, this is shown on the console 60 seconds after completion of my pipeline, and every 60 seconds afterwards:

com.hierynomus.protocol.transport.TransportException: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: java.util.concurrent.TimeoutException: Timeout expired
        at com.hierynomus.protocol.transport.TransportException$1.wrap(TransportException.java:30)
        at com.hierynomus.protocol.transport.TransportException$1.wrap(TransportException.java:24)
        at com.hierynomus.protocol.commons.concurrent.Futures.get(Futures.java:47)
        at com.hierynomus.smbj.share.TreeConnect.close(TreeConnect.java:70)
        at com.hierynomus.smbj.share.Share.close(Share.java:116)
        at com.hierynomus.smbj.session.Session.logoff(Session.java:236)
        at com.hierynomus.smbj.session.Session.close(Session.java:279)
        at com.hierynomus.smbj.connection.Connection.close(Connection.java:185)
        at com.hierynomus.smbj.connection.Connection.close(Connection.java:162)
        at com.hierynomus.smbj.connection.Connection.handleError(Connection.java:303)
        at com.hierynomus.smbj.transport.PacketReader.run(PacketReader.java:59)
        at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: java.util.concurrent.TimeoutException: Timeout expired
        at com.hierynomus.protocol.commons.concurrent.PromiseBackedFuture.get(PromiseBackedFuture.java:59)
        at com.hierynomus.protocol.commons.concurrent.CancellableFuture.get(CancellableFuture.java:84)
        at com.hierynomus.protocol.commons.concurrent.Futures.get(Futures.java:42)
        ... 9 more
Caused by: com.hierynomus.smbj.common.SMBRuntimeException: java.util.concurrent.TimeoutException: Timeout expired
        at com.hierynomus.smbj.common.SMBRuntimeException$1.wrap(SMBRuntimeException.java:28)
        at com.hierynomus.smbj.common.SMBRuntimeException$1.wrap(SMBRuntimeException.java:22)
        at com.hierynomus.protocol.commons.concurrent.Promise.retrieve(Promise.java:136)
        at com.hierynomus.protocol.commons.concurrent.PromiseBackedFuture.get(PromiseBackedFuture.java:57)
        ... 11 more
Caused by: java.util.concurrent.TimeoutException: Timeout expired
        ... 13 more

I suspect there's a timer going off to clean up and it goes wrong.

@mattcasters
mattcasters marked this pull request as draft October 5, 2026 14:32
@mattcasters
mattcasters marked this pull request as ready for review October 5, 2026 14:56
@mattcasters
mattcasters marked this pull request as draft October 5, 2026 14:57
@mattcasters
mattcasters marked this pull request as ready for review October 6, 2026 07:08
@mattcasters
mattcasters merged commit ef564f0 into apache:main Oct 6, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request]: VFS driver for SMB

1 participant