You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
path.iterdir() yields path itself as the first item (with S3) #226
I wouldn't call it a problem. There's something stored under that key in the bucket. We should just define / document behavior of upath in cases like these.
I wasn't aware of marker directories at the time of submitting this issue. This seems to be exactly what was going on, but I don't have access to an AWS environment at the moment to verify this.
Nevertheless, I think your fix makes total sense, I'd say let's go for it.
Edit: found the other article you've linked in your tests, nice
Activity
ap-- commentedon Jun 21, 2024
Hi @danielgafni
Thank you for reporting! Could you check 3 things:
Cheers,
Andreas 😊
danielgafni commentedon Jun 21, 2024
Hey!
I can answer to (1) and (3) right away:
fsspecversion was2024.3.1Regarding (2), I will be able to check a bit later
danielgafni commentedon Jun 23, 2024
Checked it, seems to be an issue with
fsspec. It has this problem too.ap-- commentedon Jun 23, 2024
I wouldn't call it a problem. There's something stored under that key in the bucket. We should just define / document behavior of upath in cases like these.
danielgafni commentedon Jun 23, 2024
No object with this key exists in the bucket. Also, it happens with any "directory" (common path prefix) in the bucket, not just a specific one.
Also,
aws s3 lsdoesn't contain the problematic keyap-- commentedon Jun 23, 2024
Could you create a PR with a test case in the upath S3 tests that reproduces the issue? That would be super helpful for finding a solution
ap-- commentedon Oct 5, 2025
might be relevant: https://medium.com/cyberark-engineering/the-strange-case-of-amazon-s3-bucket-folders-c8d113a8dd01
ap-- commentedon Jan 13, 2026
@danielgafni could you please verify if #536 resolves your issue?
danielgafni commentedon Jan 13, 2026
I wasn't aware of marker directories at the time of submitting this issue. This seems to be exactly what was going on, but I don't have access to an AWS environment at the moment to verify this.
Nevertheless, I think your fix makes total sense, I'd say let's go for it.
Edit: found the other article you've linked in your tests, nice
ap-- commentedon Jan 13, 2026
Okay. It makes sense for them to be omitted from
.iterdir(). I'll merge the PR.Thanks again for reporting 🙏
danielgafni commentedon Nov 11, 2026
Yay! 🥳