Skip to content

Bytes paths (path_encoding=None) only work in some methods #84

Open

Description

@shcheklein

With sftp_client_kwargs={"path_encoding": None} (needed for servers with non-UTF-8 file names, see #39), asyncssh only accepts paths as bytes. SSHFileSystem passes bytes paths to the server unchanged, but only some methods handle them.

Work with bytes paths: ls (but see the bug below), cat_file, open, mkdir, makedirs, rm_file, rmdir.

Fail with TypeError:

  • info, and everything built on it (exists, isdir, isfile, size, modified): _info calls path.rstrip("/") with a str argument.
  • rm: checks _isdir first, so it hits the same error.
  • put_file, pipe_file: self._parent(path) uses fsspec's _parent, which checks "/" in path. posixpath.dirname(path) works for both str and bytes.
  • cp_file, checksum: build shell commands with shlex.quote, which only accepts str.
  • fsspec's generic methods (walk, find, glob, get, put, copy) assume str paths throughout.

Bug in ls: _ls skips entries with file_attr.filename in ["", ".", ".."]. asyncssh only decodes file names when the path is str, so with a bytes path the names are bytes and b"." / b".." aren't skipped. On servers that list those entries (OpenSSH does), fs.ls(b"/dir") returns b"/dir/." and b"/dir/..". The mock server used in tests doesn't return them, so the tests don't catch it.

Possible scope: make sshfs's own methods handle bytes (info, rm, put_file, pipe_file, cp_file, checksum, and the ls filter). Full support in fsspec's generic methods isn't realistic, so the docs should list which methods work with bytes paths.

Raised in the review of #77: #77 (comment)

Activity

  1. kushaldave002 commented on Nov 19, 2026

    @kushaldave002

    I reproduced the bytes-path failures in #84 and am preparing a focused fix. It keeps byte separators/parents intact for metadata and uploads, filters byte-valued dot entries in listings, and uses SFTP copy/streamed MD5 for bytes paths rather than decoding them or passing them to shell quoting. Seven regression cases fail on current main; the fix and full checks are underway.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions