download a file from a GitHub release page

[till]

Hey,

What I have so far is the following:

sources:
  github:
    kind: githubRelease
    spec:
      token: "{{ requiredEnv .github.token }}"
      owner: org
      repository: repo
      versionFilter:
        kind: latest

This seems to work, but then I am stuck on the following:

1. I want to download a file from the release, ideally also filter on it (e.g. filename) to make sure I get the correct one.
2. I want to run an additional command (which pushes the artefact to a remote registry)

My thought is to use a target for that (for the download), but I gave up googling — I can't find any examples.

I think I would also like to use the version to check in the remote registry before I continue. But maybe I am abusing updatecli for something it was not meant to be used for.

Any thoughts?

[dduportal]

Hi @till, you can use a second source of type "file" on short term. Something like:

name: download file from GH release

sources:
  latestHadolintVersion:
    kind: githubrelease
    spec:
      token: "{{ requiredEnv .github.token }}"
      owner: hadolint
      repository: hadolint
  getHadolintLinuxArm:
    kind: file
    dependson:
      - latestHadolintVersion
    spec:
      file: https://github.com/hadolint/hadolint/releases/download/{{ source "latestHadolintVersion" }}/hadolint-Linux-arm64

targets:
  ensureHadolintLinuxArm:
    kind: file
    sourceid: getHadolintLinuxArm
    spec:
      file: ./hadolint

[till]

@olblak I got a bit further, but it seems I am still stuck:

sources:
  getRelease:
    kind: githubrelease
    transformers:
      - trimPrefix: 'v'
    spec:
      token: "{{ requiredEnv .github.token }}"
      owner: paketo-community
      repository: go-generate
      assert: "*.cnb"
      versionFilter:
        kind: latest
  getAsset:
    kind: file
    dependson:
      - getRelease
    spec:
      file: https://github.com/paketo-community/go-generate/releases/download/v{{ source "getRelease" }}/go-generate-{{ source "getRelease" }}.cnb

targets:
  saveAsset:
    kind: file
    sourceid: getAsset
    spec:
      file: ./go-generate.cnb
  publish:
    dependson:
      - saveAsset
    sourceid: getRelease
    kind: shell
    spec:
      command: sudo skopeo copy oci-archive:go-generate.cnb docker://r.planetary-quantum.com/buildpacks/go-generate:{{ source "getRelease" }}

All I see is pipeline failed, and it fails somehow in the target saveAsset where I am trying to save it to a file. But I can't see why it fails? The publish target is skipped. Maybe something gets clipped somewhere because it breaks my shell (re: the output of the blob).

Does the file source follow redirects, etc..? As github download basically gives me an S3 link? From the output of the blob, I believe it does, but I can't tell what else is wrong.

Any idea what it could be?

[dduportal]

I'm (personnally) having mixed feelings. It looks liek you're trying to use updatecli as an automation engine instead of using a command line or a script. What is the rationale behind this?

I would tend to split these processes as 2 distinct things:

• Updatecli as a tool to update textual files in which you pin your dependencies
• Another system (script, command, pipeline, whatever) to read the textual value of pinned dependencies and perform an action.

[till]

You're not wrong. Was doing something similar and wanted to re-use it instead of writing more dreadful bash wrapped in yaml.

So I understand where you're coming from. You already helped me out a lot!

[dduportal]

@till I still think we can improve updatecli as @olblak said. The issue you opened is already a good one! thanks!

[till]

Merci, as always — appreciate all the help. Thanks to both of you! :)

I managed to do finally my workflow with github actions and a little bit of bash.

[olblak]

Another option would be to leverage the shell plugin with something like

name: download file from GH release

sources:
  latestHadolintVersion:
    kind: githubrelease
    spec:
      token: '{{ requiredEnv "GITHUB_TOKEN" }}'
      owner: hadolint
      repository: hadolint

targets:
  cmd:
    kind: shell
    disablesourceinput: true
    spec:
      command: echo https://github.com/hadolint/hadolint/releases/download/{{ source "latestHadolintVersion" }}/hadolint-Linux-arm64

  script:
    kind: shell
    disablesourceinput: true
    spec:
      command: |
        # Run a full sh script
        echo https://github.com/hadolint/hadolint/releases/download/{{ source "latestHadolintVersion" }}/hadolint-Linux-arm64
        echo run another command

The downside of the shell script, is that you may need more tools than updatecli, depending on what you execute.
Also, you may need to design your script to be able to run in dry run otherwise running updatecli diff would be similar to updatecli apply

If that approach interest you more, I could prioritize some documentation work

[till]

I can't really decide, I think diff may not work anyway with what I am trying to do.