Skip to content

feat(download): opensource internal download library - #88

Merged
jellonek merged 3 commits into
mainfrom
downloader
Nov 12, 2026
Merged

jellonek merged 3 commits into
mainfrom
downloader

Conversation

@jellonek

Copy link
Copy Markdown
Contributor

Closes #71

@jellonek
jellonek requested review from a team as code owners November 10, 2026 07:42
Copilot AI balanced review requested due to automatic review settings November 10, 2026 07:42

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical download defects remain in archive checksum verification, temporary-file isolation, decompression limits, and cache-directory creation, causing valid downloads to fail or exceed safety limits.

9 open issues
What changed in this PR

Adds a reusable download package to cloudctl, laying the groundwork for automatic tool installation and caching described in #71.

Changes:

  • Adds binary, tar.gz, and ZIP downloads with SHA-256 verification, progress output, and size-limit checks.
  • Adds GitHub release downloads that prefer API digests, fall back to .sha256 files, and reuse cached binaries.
  • Introduces shared XDG cache configuration and supporting dependencies.
File Description
pkg/​download/​github.go Implements GitHub digest lookup, checksum fallback, and cached release downloads.
pkg/​download/​download.go Implements download, extraction, checksum verification, and progress helpers.
pkg/​config/​constants.go Defines the user agent and application-specific XDG locations.
go.sum Records checksums for added dependencies.
go.mod Adds dependencies for XDG paths, progress formatting, and logging.

Review effort: Balanced

Comment thread pkg/download/download.go Resolved
Comment thread pkg/download/download.go Resolved
Comment thread pkg/download/download.go Resolved
Comment thread pkg/download/github.go Resolved
Comment thread pkg/download/download.go Resolved
Comment thread pkg/download/download.go Resolved
Comment thread pkg/download/download.go Resolved
Comment thread pkg/download/download.go Resolved
Comment thread pkg/download/download.go Resolved
Signed-off-by: Piotr Skamruk <piotr.skamruk@gmail.com>
@jellonek
jellonek force-pushed the downloader branch 4 times, most recently from 29fd35e to 1c68757 Compare November 10, 2026 10:30
@jellonek
jellonek requested a balanced review from Copilot November 10, 2026 10:33

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread pkg/download/download.go Resolved
Comment thread pkg/download/download.go Resolved
Comment thread pkg/download/download.go Resolved
Comment thread pkg/download/download.go Resolved
@jellonek
jellonek force-pushed the downloader branch 2 times, most recently from 8afaa79 to 9566dac Compare November 10, 2026 10:53
@jellonek
jellonek requested a balanced review from Copilot November 10, 2026 10:54

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A shadowed output handle causes valid archive downloads to panic, while cross-filesystem staging, incorrect checksum selection, and premature cache publication can also leave downloads unusable.

3 open issues
4 resolved since last review

Review effort: Balanced

Comment thread pkg/download/download.go Resolved
Comment thread pkg/download/download.go Resolved
Comment thread pkg/download/github.go Resolved

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved checksum-selection bugs and incomplete size checks can reject valid downloads or accept oversized responses and truncated ZIP binaries.

3 open issues
3 resolved since last review

Review effort: Balanced

Comment thread pkg/download/download.go Resolved
Comment thread pkg/download/download.go Resolved
Comment thread pkg/download/download.go Resolved
Signed-off-by: Piotr Skamruk <piotr.skamruk@gmail.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Shared checksum manifests select the first digest instead of the downloaded asset’s entry, causing valid downloads to fail verification across all three download functions.

1 open issue
3 resolved since last review

Review effort: Balanced

Comment thread pkg/download/download.go Resolved
@onuryilmaz

Copy link
Copy Markdown
Contributor

Dependency Modernization

Three new dependencies added in this PR are worth revisiting before merge.


github.com/OpenPeeDeeP/xdg → replace with github.com/adrg/xdg

OpenPeeDeeP/xdg has not been updated since 2019 (v1.0.0) and is effectively abandoned. The actively maintained alternative github.com/adrg/xdg (last commit: November 2026, v0.5.3) covers the full XDG spec (Config, Data, Cache, State, Runtime, UserDirs) and offers a simpler package-level API — no initialization required:

// current
XDGLocations = xdg.New("cloudoperators", "cloudctl")
// ...
XDGLocations.QueryConfig("cloudctl.yaml")

// with adrg/xdg
xdg.ConfigHome // directly usable, no init

github.com/sirupsen/logrus — maintenance mode

From the logrus README: "Logrus is in maintenance-mode. We will not be adding new features." It only receives security and critical bug fixes. For a new package in this repo, using fmt/os.Stderr directly (consistent with the rest of the codebase) or log/slog (stdlib, Go 1.21+) would be a better fit.


github.com/logrusorgau/aurora — incompatible module

aurora is tagged as v2.0.3+incompatible, meaning its module path was never migrated to v2. It's used for a single colorized debug string visible only at debug log level. If logrus is replaced, this dependency can be dropped entirely; if color output is still needed, the project already pulls in github.com/charmbracelet/lipgloss which handles terminal styling.

@onuryilmaz

Copy link
Copy Markdown
Contributor

Code Review

Critical: DownloadGitHubReleaseBinary deletes the binary it just installed

pkg/download/github.go lines ~145–155:

staged, err := os.CreateTemp(filepath.Dir(destPath), filepath.Base(destPath)+"-*")
// ...
stagedPath := staged.Name()
defer func() {
    _ = os.Remove(stagedPath) // fires AFTER Rename — deletes destPath
}()
// ...
if err := os.Rename(stagedPath, destPath); err != nil {
    return "", err
}
return destPath, nil  // defer removes destPath here

os.Rename moves stagedPath → destPath, then the deferred os.Remove(stagedPath) fires. At that point stagedPath no longer exists, but on some filesystems/OSes the remove targets the already-renamed destination path. Net result: every successful download silently deletes the binary it just installed. The next exec call gets no such file or directory.

Fix: nil out stagedPath on success so the defer becomes a no-op:

defer func() {
    if stagedPath != "" {
        _ = os.Remove(stagedPath)
    }
}()
// ...
if err := os.Rename(stagedPath, destPath); err != nil {
    return "", err
}
stagedPath = "" // disarm the defer
return destPath, nil

openChecksumSource uses filepath.Base on a URL — breaks on Windows

pkg/download/download.go line ~425:

base := filepath.Base(url)  // filepath uses '\' separator on Windows
if w, ok := strings.CutSuffix(base, ".sha256"); ok {
    wantAsset = w
}

On Windows, filepath.Base uses \ as the separator. A URL like https://github.com/.../tool.tar.gz.sha256 contains no backslash, so filepath.Base returns the entire URL string. strings.CutSuffix does not match, wantAsset stays "", and selectChecksumToken falls back to the first token in a multi-entry checksum file — verifying the download against the wrong hash.

Fix: use path.Base (package path, not path/filepath) which always uses /:

import "path"
// ...
base := path.Base(url)

@onuryilmaz

Copy link
Copy Markdown
Contributor

DownloadTarGzFile: double-close and fragile cleanup from defer inside loop

pkg/download/download.go ~line 160:

for {
    // ...
    out, err = os.CreateTemp(...)
    defer func() {
        _ = out.Close()      // fires at function return — double-close
        _ = os.Remove(out.Name()) // fires after os.Rename moved the file
    }()

    _, err = io.CopyN(out, tarReader, header.Size)
    // ...
    err = out.Close()  // explicit close before break
    fileFound = true
    break
}
// ...
os.Rename(out.Name(), filepathDest) // moves the file
// defer fires here: out.Close() again, os.Remove(now-moved path)

Two issues:

  1. out.Close() is called explicitly before break, then the deferred out.Close() fires again at function return → double-close on *os.File.
  2. os.Remove(out.Name()) in the defer fires after os.Rename has already moved the file. On Linux this is a no-op (the path no longer exists), but it's the same structural fragility as the stagedPath defer bug — one refactor away from deleting the installed file.

Fix: move temp file management out of the loop or use an explicit cleanup variable:

var out *os.File
var outPath string
defer func() {
    if out != nil {
        _ = out.Close()
    }
    if outPath != "" {
        _ = os.Remove(outPath)
    }
}()

// inside loop:
out, err = os.CreateTemp(...)
outPath = out.Name()
// ... write, close explicitly ...
out = nil   // disarm the Close defer
// on success path before Rename:
outPath = "" // disarm the Remove defer

@jellonek

Copy link
Copy Markdown
Contributor Author

Double close and rename is not a problem. Both will return an error which is explicitly ignored. They were added as there was no code which would handle cleanup if there was a premature return from the loop.

@jellonek

jellonek commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor Author

on some filesystems/OSes the remove targets the already-renamed destination path.

Which filesystem/OS?

@jellonek

Copy link
Copy Markdown
Contributor Author

Good point about path.Base on url! I'll update the code.

Signed-off-by: Piotr Skamruk <piotr.skamruk@gmail.com>
@jellonek

Copy link
Copy Markdown
Contributor Author

Base on url handling updated.
Dependencies fixed.

@onuryilmaz

Copy link
Copy Markdown
Contributor

@jellonek if you could resolve the open conversation #88 (review) we can merge it

@jellonek
jellonek merged commit f950744 into main Nov 12, 2026
7 checks passed
@jellonek
jellonek deleted the downloader branch November 12, 2026 23:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEAT cloudctl] - Add tool download infrastructure

3 participants