Comparing speed of Concourse and GitHub Actions

I’ve been experimenting with GitHub Actions as an alternative to running jobs using Concourse, and needed to compare how long specific tasks took in each setting.

Luckily both systems expose APIs which, with a small amount of persuasion, can pull out this exact information on the command-line using an appropriate client and jq.

Finally, I pumped the data into R for quick analysis.

Concourse

CONCOURSE=https://example-concourse.example
TEAM=$1
PIPELINE=$2
JOB=$3

curl $CONCOURSE/api/v1/teams/$TEAM/pipelines/$PIPELINE/jobs/$JOB/builds --silent \
  | jq '.[] | select(.start_time != null) | (.end_time - .start_time)'
  | R --quiet -e 'x <- scan(file="stdin",quiet=TRUE); summary(x);'

GitHub Actions

This requires the gh tool to interface with GitHub’s API.

ORG=$1
REPO=$2
PIPELINE=$3
JOB=$4

# Get all runs for the pipeline
RUN_IDS=$(gh api -XGET /repos/$ORG/$REPO/actions/runs \
    | jq ".workflow_runs[]
      | select(.status == \"completed\")
      | select(.name == \"$PIPELINE\")
      | [.id]
      | unique
      | .[]")

for RUN in $RUN_IDS; do

  # Get the duration for the specific job
  gh api -XGET /repos/$ORG/$REPO/actions/runs/$RUN/jobs \
    | jq ".jobs[]
        | select(.name == \"$JOB\")
        | (.completed_at | strptime(\"%Y-%m-%dT%H:%M:%SZ\") | mktime) -
          (.started_at | strptime(\"%Y-%m-%dT%H:%M:%SZ\") | mktime)"
done \
  | R --quiet -e 'x <- scan(file="stdin",quiet=TRUE); summary(x);'

Outputs

This pops out values for the minimum, median, mean, maximum run durations in addition to first and third quartiles which is enough information to point us towards where we can optimise our automated pipelines.