You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the feature you'd like
The on-the-fly conversion during download works, but could be improved for greater convenience:
Decompress files on download (from detected compression to file w/o compression)
Compress files without compression on download (from file w/o compression to specified compression)
Refactor file conversion logic to databusclient/extensions/file_converter.py
Currently the conversion feels slow. I didn't test whether this is reasonable or if this could be improved. An idea would be to add the conversion (and also checksum computation/validation) directly to the download step. Now the files is first downloaded completely and then converted. Maybe this can be squashed into one step.
Why is this feature important?
Explain the benefit or use case. Is your feature request related to a problem?
Follow up on #18. Not urgent, but useful for greater functionality
Describe alternatives you've considered
For now: manually
This makes sense. Integrating conversion and checksum validation into the download step could improve performance.
I’m currently focusing on small cleanups, but happy to help later once the approach is clarified.
One idea I had is to treat the download, (de)compression, and checksum as a small streaming pipeline rather than separate steps — so the download stream passes through optional processors (decompressor, compressor, checksum) before being written to disk.
I could start with a minimal version (gzip + checksum only) to address this issue, and later this could be extended to support more formats or features if that fits the project’s direction. Would this approach make sense here?
Activity
Dev10-sys commentedon Jan 22, 2026
This makes sense. Integrating conversion and checksum validation into the download step could improve performance.
I’m currently focusing on small cleanups, but happy to help later once the approach is clarified.
Tahoora-Tabassum commentedon Jan 29, 2026
One idea I had is to treat the download, (de)compression, and checksum as a small streaming pipeline rather than separate steps — so the download stream passes through optional processors (decompressor, compressor, checksum) before being written to disk.
I could start with a minimal version (gzip + checksum only) to address this issue, and later this could be extended to support more formats or features if that fits the project’s direction. Would this approach make sense here?
feat: Add file_converter extension module (Issue dbpedia#54)
Integer-Ctrl commentedon Sep 17, 2026
This issue is deprecated due to the refactoring of the compression options as part of the GSoC 2026 project