19 votes

Install asdf: One Runtime Manager to Rule All Dev Environments

Posted by jdsalaro
Tags: programming, sysadmin, software development, user created
https://jdsalaro.com/tutorial/asdf-single-package-manager-multiple-dev-environments/

7 comments

  1. [3]
    jdsalaro
    Link
    Greetings folks, I wrote a tutorial on how to manage the dumpster fire that arises whenever one has to contribute to projects with very diverse stacks using asdf.vm. It's been a highly debated...

    Greetings folks, I wrote a tutorial on how to manage the dumpster fire that arises whenever one has to contribute to projects with very diverse stacks using asdf.vm. It's been a highly debated topic, so I figured y'all might be interested :D
    As usual, feel free to ask away!

    6 votes
    1. [2]
      TheD00d
      Link Parent
      Wow! I'm not really a dev (sysadmin/labber and infosec) so I don't normally need to contend with this, but man. I can't tell you how many times I've tried install some tool from GitHub and tried...

      Wow! I'm not really a dev (sysadmin/labber and infosec) so I don't normally need to contend with this, but man. I can't tell you how many times I've tried install some tool from GitHub and tried installing modules and other components only for it to turn into a huge mess.

      'asdf' looks like it will be super helpful absolutely adding it to my Desktop anisble script. Bonus for such a well written and visually supported tutorial!

      3 votes
      1. jdsalaro
        Link Parent
        Hey there ! I'm a fellow infosec/automation/backend and product guy! Let me know in case you have free cycles and want to collaborate on research, I've got some ideas :) It's been crazy lately...

        Wow! I'm not really a dev (sysadmin/labber and infosec)

        Hey there ! I'm a fellow infosec/automation/backend and product guy! Let me know in case you have free cycles and want to collaborate on research, I've got some ideas :)

        so I don't normally need to contend with this, but man.

        It's been crazy lately because I'm testing other people's environment and code and without the setup described in the OP I'd had gone crazy by now.

        I can't tell you how many times I've tried install some tool from GitHub and tried installing modules and other components only for it to turn into a huge mess.

        An absolute shit show, I hear you.

        On Reddit someone was complaining that asdf.vm environments cannot be deployed and therefore are not useful, not as Docker containers, well if every project just came with a Dockerfile or docker-compose.yml

        'asdf' looks like it will be super helpful absolutely adding it to my Desktop anisble script.

        Is your desktop ansible script on a public repo somewhere? Curious to cross-checks notes!

        Bonus for such a well written and visually supported tutorial

        Thanks mate! I appreciate it, feel free to stay in the loop via the mailing list or socials, my resolution for this year is to be more active :D I love explaining stuff and teaching, so it really motivates me how good the reception has been; although the debate has been intense as hell.

  2. [4]
    first-must-burn
    Link
    This is interesting. I have to confess the idea of having so many versions of things installed gives me the heebeejeebies. But if one must live in that environment, this seems like a useful tool....

    This is interesting. I have to confess the idea of having so many versions of things installed gives me the heebeejeebies. But if one must live in that environment, this seems like a useful tool.

    I'm curious how often the tool dependencies (like library paths) get snarled up when switching environments. Does the tool only shim the binaries, or does it have a provision to set up environment variables as well?

    One problem that I see is that the global config has the potential to hide dependencies. Have you considered the ability to add a "no-globals" option to the local config files, so that users must explicitly choose a version for that context? That way when the config gets committed, it carries all the version info explicitly. I can see this being the default behavior for configs inside version control, but maybe that is a bridge too far.

    FWIW, I am all-in on containerized development, especially if the app itself will eventually be deployed in containers. For sure, there is a learning curve, but the ability to precisely control the environment seems worth the cost to me. For my dev setups, I like having task as the bootstrap dependency. From there, a task setup-env and a task check-env can check for or install run time dependencies (mainly things like docker, k3d, helm, etc). Then I usually have a task build-dev-env for the dev docker file and task run-dev-env to launch a shell in it. The task file is a great resource for people on the team who don't understand the containerization to use it in a repeatable way. Debugging remote apps is one of the hardest part of working this way, but debug servers running in the container are pretty well supported by things like vscode.

    2 votes
    1. TangibleLight
      Link Parent
      I use this. I haven't encountered issues with libraries or runtime paths. It doesn't have a built in provision for environment variables, everything goes through the shims. And I've never...

      I use this. I haven't encountered issues with libraries or runtime paths. It doesn't have a built in provision for environment variables, everything goes through the shims. And I've never encountered issues with cleanup, since all the environment management goes through the shims and don't leak out into your shell.

      There is the asdf direnv plugin, which allows you to set up environment vars, not using the shims, through an envrc. I haven't encountered issues with cleanup here either, but this one seems more likely to cause issues. You can put arbitrary environment setup in the envrc.

      2 votes
    2. [2]
      jdsalaro
      Link Parent
      I appreciate the feedback! You and me both, but it's the deck of cards I've been dealt :D If that gives you the heebie-jeebies, check this comment on HackerNews out: To be honest, this has only...

      This is interesting.

      I appreciate the feedback!

      I have to confess the idea of having so many versions of things installed gives me the heebeejeebies.

      You and me both, but it's the deck of cards I've been dealt :D

      If that gives you the heebie-jeebies, check this comment on HackerNews out:

      Whish there were some CLI to speed up this process actually. Just cd:ing into a folder should pull everything down for you to run iex/irb/node/etc as if it was native but running through the container.

      But if one must live in that environment, this seems like a useful tool
      I can't emphasize enough how many headaches I've solved or altogether prevented from arising by relying on this setup.

      I'm curious how often the tool dependencies (like library paths) get snarled up when switching environments.

      To be honest, this has only ever happened to me a couple of times (two?) in five years and it was due to brew and how opinionated and careless with its changes it is.

      Does the tool only shim the binaries, or does it have a provision to set up environment variables as well?

      Each plugin is different, since each language runtime operates differently. For example, for Java environment variables such as JAVA_HOME are set. I'll delves bit deeper into astd.vm internals in a future post.

      One problem that I see is that the global config has the potential to hide dependencies. Have you considered the ability to add a "no-globals" option to the local config files, so that users must explicitly choose a version for that context?

      Wouldn't explicitly stating the desired version for the given local repo and pushing it suffice in this case? A no globals with known or customary expected values seems equivalent to a set and pushed local .tool-versions, or am I missing something?

      I am all-in on containerized development, especially if the app itself will eventually be deployed in containers. For sure, there is a learning curve, but the ability to precisely control the environment seems worth the cost to me.

      I certainly agree, with the big plus than by the time you have your environment setup properly sorted out you also have a Dockerfile ready for deployment. However, and this is a big issue, exploring the solution space of challenges and experimenting within Docker containers is a pain in the butt! For that alone I always want to setup a local, native development environment. Also because I feel that if I've set it up locally, in a reproducible manner, I truly understand it and can port it anywhere. Nevertheless, I do agree with you in principle.

      For my dev setups, I like having task as the bootstrap dependency.

      Interesting, this is the second simpler version of make I learn about today ( the second being just )

      From there, a task setup-env and a task check-env can check for or install run time dependencies (mainly things like docker, k3d, helm, etc). Then I usually have a task build-dev-env for the dev docker file and task run-dev-env to launch a shell in it. The task file is a great resource for people on the team who don't understand the containerization to use it in a repeatable way.

      This sounds really cool, do you have one such speced-out task file public somewhere? I'd like to dive into it.

      Debugging remote apps is one of the hardest part of working this way, but debug servers running in the container are pretty well supported by things like vscode.

      It wasn't always the case, but I do agree that they've come a long way. Nevertheless, nothing approaches the ease with which you can debug and continue down the dependency tree when your languages runtime environment is just yet another directory within ~/.asdf/language/version/package

      1 vote
      1. first-must-burn
        Link Parent
        The scenario I'm thinking of is: a directory has its own asdf config set up for java, so the local config captures the java dependency. The config file gets pushed. (This is the correct normal...

        Wouldn't explicitly stating the desired version for the given local repo and pushing it suffice in this case? A no globals with known or customary expected values seems equivalent to a set and pushed local .tool-versions, or am I missing something?

        The scenario I'm thinking of is:

        1. a directory has its own asdf config set up for java, so the local config captures the java dependency. The config file gets pushed. (This is the correct normal flow, as you say
        2. Someone adds a python script that does some task, and it works correctly for them because they have python 3.13 in their global config. They push the script but no local asdf config change.
        3. You pull the change but the script fails because your global config is on python 3.8.

        But if someone had set something no-globals in the local config when settling things up, in step 2, the person adding the script would have gotten an error message trying to use python without setting the version explicitly and corrected it at the moment of the mistake.

        Of course, failure to add the dependency is correctable, and doing it right the first time something you can ask people to "always do". Maybe something you can catch in CI. And if it's a dependency that asdf is not yet shimming, this won't catch it. So maybe you about lump it into all the weird corner cases, like calling python ../otherwise/whatever.py. (Can you tell I'm a stress tester by background?)

        I certainly agree, with the big plus than by the time you have your environment setup properly sorted out you also have a Dockerfile ready for deployment. However, and this is a big issue, exploring the solution space of challenges and experimenting within Docker containers is a pain in the butt!

        The way I usually do this is open up a new dockerfile and a terminal, then do a docker run --rm -it debian:bookworm-slim /bin/bash. Then I just mess around installing things interactively, adding lines to my dockerfile as I go to keep track of what I did. When I'm happy with the environment, I consolidate all the apt installs into a single dockerfile command to reduce the image size and add any manually installed tools as steps.

        This sounds really cool, do you have one such speced-out task file public somewhere?

        I don't have a public example, but I can put a couple inline here.

        First, a really simple one that just does the docker stuff. run-env gives you an interactive environment, or automatic-task can be used to invoke some executable like task automatic-task -- -some-args foo. Note how in each command, some part of the local file system is mounted into the docker container so that things you do inside (like run a compiler) persist outside the container.

        Taskfile (short) 
        # https://taskfile.dev

version: '3'

vars:
  DOCKER_IMAGE_NAME: whatever:whatever

tasks:
  build-image:
    cmds:
      - docker build -t {{.DOCKER_IMAGE_NAME}} .

  build-image-nocache:
    cmds:
      - docker build --no-cache -t {{.DOCKER_IMAGE_NAME}} .

  run-env:
    interactive: true
    cmds:
      - |
        docker run --rm -it \
        -v $(pwd)/files:/files \
        -w /files \
        {{.DOCKER_IMAGE_NAME}} /bin/bash

  automatic-task:
    cmds:
      - |
        docker run --rm -it \
        -v $(pwd)/files:/files \
        -w /files \
        {{.DOCKER_IMAGE_NAME}} \
        some-executable-in-the-container {{.CLI_ARGS}}

        Here's the taskfile from my quote database project. It is a little more complicated because it's configuring a local k3d environment, but it has some of the bootstrapping stuff in it. Basically, if there's a commandline thing that you would do repeatedly, like to set the environment up, I like to put it in a task so that I don't have to remember how to do it.

        Because there are so many commands, I have the names set up in least-specific-word to most-specific-word order so that you can (for example) start typing task docker then tab complete to get all the docker tasks. Task has the ability to import other task files, so some people might break these down into several task files and import them, which would basically put them in a namespace and achieve a similar effect, but with : instead of -.

        Taskfile (long) 
        # https://taskfile.dev

version: '3'

vars: #alphabetical list
  DOCKER_PREFIX: quotable
  HELM_DEPLOYMENT_NAME: quotable
  LOCAL_CLUSTER_NAME: quotable
  LOCAL_REGISTRY_NAME_SUFFIX: quotable.registry.localhost
  LOCAL_REGISTRY_NAME: k3d-{{.LOCAL_REGISTRY_NAME_SUFFIX}}
  LOCAL_REGISTRY_PORT: 34523
  MIGRATE_VERSION: v4.15.1
  MANUAL_BACKUP_JOB_NAME: manual-backup

  REMOTE_REGISTRY_NAME: raybetter/quotable
  GIT_HASH: 
    sh: git log -n 1 --format=%H

  # POSTGRES_LOCAL_DOCKER_NAME: quotable-postgres

tasks:
####################################################################################################
# cluster tasks

  registry-local-create:
    description: create the local registry -- do this before creating the local cluster
    cmds:
      - k3d registry create {{.LOCAL_REGISTRY_NAME_SUFFIX}} --port {{.LOCAL_REGISTRY_PORT}}

  registry-local-delete:
    description: delete the local registry
    cmds:
      - k3d registry delete {{.LOCAL_REGISTRY_NAME}}

  cluster-local-create:
    description: create the local cluster
    preconditions:
      # require the registry be created first
      - sh: '[ "$(k3d registry list --no-headers | grep -c "k3d-quotable.registry.localhost")" -eq 1 ]'
    cmds:
      - k3d cluster create {{.LOCAL_CLUSTER_NAME}} -p "8080:80@loadbalancer" --registry-use k3d-{{.LOCAL_REGISTRY_NAME_SUFFIX}}:{{.LOCAL_REGISTRY_PORT}}
      # let the cluster get going so kubectl will succeed
      - sleep 15
      #this fixes the traefik configuration for local deployment (ignore cert correctness) and turns on ingress logging
      - kubectl apply -f deployment/k3d/traefik-local-config.yaml

  cluster-local-delete:
    description: delete the local cluster
    cmds:
      - k3d cluster delete {{.LOCAL_CLUSTER_NAME}}


  cluster-remote-setup:
    description: set up the terraform for the local cluster
    dir: deployment/remote/terraform
    cmds:
      - terraform init --upgrade
      - terraform validate

  cluster-remote-create:
    description: create a cloud instance, install kubernetes
    dir: deployment/remote/terraform
    interactive: true
    cmds:
      - ssh-agent ./run_terraform.sh
      - chmod 600 quotable_kubeconfig.yaml
      - KUBECONFIG=quotable_kubeconfig.yaml kubectl apply -f ../k8s/letsencrypt.yaml
      - KUBECONFIG=quotable_kubeconfig.yaml kubectl apply -f ../k8s/traefik-cert-fix.yaml

  cluster-remote-list-servers:
    description: list the servers running under the API token
    preconditions:
      - sh: '[ -e "deployment/remote/secrets/hetzner_api_key" ]'
    cmds:
      - HCLOUD_TOKEN=$(cat deployment/remote/secrets/hetzner_api_key) hcloud server list


  cluster-expose-traefik-dashboard:
    description:  open the traefik dashboard on port 9000; run this command then navigate to `http://localhost:9000/dashboard/`
    interactive: true
    vars:
      PODNAME:
        sh: kubectl get pods -n kube-system -l app.kubernetes.io/name=traefik  -o=jsonpath='{.items[0].metadata.name}'
    cmds:
      - kubectl port-forward -n kube-system {{.PODNAME}} 9000
      
####################################################################################################
# deployment tasks

  deploy-secrets-create:
    description: create secrets in the cluster
    cmds:
      - bash deployment/scripts/create_secrets.sh "deployment/secrets/"

  deploy-secrets-delete:
    description: delete secrets in the cluster
    cmds:
      - bash deployment/scripts/delete_secrets.sh


  deploy-local-install:
    description: deploy the helm chart locally
    cmds:
      - helm upgrade --install -f deployment/charts/quotable/values.yaml {{.HELM_DEPLOYMENT_NAME}} deployment/charts/quotable/

  deploy-local-uninstall:
    description: uninstall the local helm chart
    cmds:
      - helm uninstall {{.HELM_DEPLOYMENT_NAME}}

  deploy-remote-regsecret:
    description: create the docker hub regsecret
    cmds:
      - bash deployment/remote/secrets/create_dockerhub_secret.sh

  deploy-remote-install:
    description: deploy the helm chart locally
    cmds:
      - helm upgrade --install -f deployment/remote/k8s/remote-values.hetzner.yaml {{.HELM_DEPLOYMENT_NAME}} deployment/charts/quotable/

  deploy-remote-uninstall:
    description: uninstall the local helm chart
    cmds:
      - helm uninstall {{.HELM_DEPLOYMENT_NAME}}



  deploy-get-db-password:
    description: get the database password from the k8s secret
    cmds:
      - kubectl get secret postgres-secrets -o=jsonpath='{.data.password}' | base64 -d

  deploy-expose-backend:
    description: expose the backend service on localhost at 3030
    interactive: true
    cmds:
      - kubectl port-forward service/backend 3030:3000

  deploy-expose-frontend:
    description: expose the frontend service on localhost at 8082
    interactive: true
    cmds:
      - kubectl port-forward service/frontend 8082:8080

  deploy-expose-db:
    description: expose the db service on localhost at 5342
    interactive: true
    cmds:
      - kubectl port-forward service/quotable-db 5432:5432


  deploy-manual-dbbackup-create:
    description: manually trigger a backup job
    preconditions:
      - sh: '[ "$(kubectl get jobs | grep -c "{{ .MANUAL_BACKUP_JOB_NAME }}")" -eq 0 ]'
    cmds:
      - kubectl create job --from=cronjob/quotable-dbbackup {{ .MANUAL_BACKUP_JOB_NAME }}

  deploy-manual-dbbackup-cleanup:
    description: clean up the manually triggered job
    preconditions:
      - sh: '[ "$(kubectl get jobs | grep -c "{{ .MANUAL_BACKUP_JOB_NAME }}")" -eq 1 ]'
    cmds:
      - kubectl delete job {{ .MANUAL_BACKUP_JOB_NAME }}

####################################################################################################
# docker tasks



  # local

  docker-local-build-and-push-all:
    description: build and push all the local docker images
    cmds:
      - task: docker-local-build-all
      - task: docker-local-push-all

  docker-local-build-all:
    description: build all the docker images with the local tags
    cmds:
      - task: docker-build-frontend
        vars:
          DOCKER_IMAGE_FULL_NAME: "{{.LOCAL_REGISTRY_NAME}}:{{.LOCAL_REGISTRY_PORT}}/{{.DOCKER_PREFIX}}/frontend"
      - task: docker-build-backend
        vars:
          DOCKER_IMAGE_FULL_NAME: "{{.LOCAL_REGISTRY_NAME}}:{{.LOCAL_REGISTRY_PORT}}/{{.DOCKER_PREFIX}}/backend"
      - task: docker-build-dbbackup
        vars:
          DOCKER_IMAGE_FULL_NAME: "{{.LOCAL_REGISTRY_NAME}}:{{.LOCAL_REGISTRY_PORT}}/{{.DOCKER_PREFIX}}/dbbackup"

  docker-local-push-all:
    description: push all the docker images to the local cluster registry
    cmds:
      - task: docker-push
        vars:
          DOCKER_IMAGE_FULL_NAME: "{{.LOCAL_REGISTRY_NAME}}:{{.LOCAL_REGISTRY_PORT}}/{{.DOCKER_PREFIX}}/frontend"
      - task: docker-push
        vars:
          DOCKER_IMAGE_FULL_NAME: "{{.LOCAL_REGISTRY_NAME}}:{{.LOCAL_REGISTRY_PORT}}/{{.DOCKER_PREFIX}}/backend"
      - task: docker-push
        vars:
          DOCKER_IMAGE_FULL_NAME: "{{.LOCAL_REGISTRY_NAME}}:{{.LOCAL_REGISTRY_PORT}}/{{.DOCKER_PREFIX}}/dbbackup"

  # remote  
  docker-remote-build-and-push-all:
    description: build and push all the remote docker images
    cmds:
      - task: docker-remote-build-all
      - task: docker-remote-push-all

  docker-remote-build-all:
    description: build all the docker images with the remote tags
    cmds:
      - task: docker-build-frontend
        vars:
          DOCKER_IMAGE_FULL_NAME: "{{.REMOTE_REGISTRY_NAME}}:frontend-{{.GIT_HASH}}"
      - task: docker-build-backend
        vars:
          DOCKER_IMAGE_FULL_NAME: "{{.REMOTE_REGISTRY_NAME}}:backend-{{.GIT_HASH}}"
      - task: docker-build-dbbackup
        vars:
          DOCKER_IMAGE_FULL_NAME: "{{.REMOTE_REGISTRY_NAME}}:dbbackup-{{.GIT_HASH}}"

  docker-remote-push-all:
    description: push all the docker images to the remote cluster registry
    cmds:
      - task: docker-push
        vars:
          DOCKER_IMAGE_FULL_NAME: "{{.REMOTE_REGISTRY_NAME}}:frontend-{{.GIT_HASH}}"
      - task: docker-push
        vars:
          DOCKER_IMAGE_FULL_NAME: "{{.REMOTE_REGISTRY_NAME}}:backend-{{.GIT_HASH}}"
      - task: docker-push
        vars:
          DOCKER_IMAGE_FULL_NAME: "{{.REMOTE_REGISTRY_NAME}}:dbbackup-{{.GIT_HASH}}"

  # helpers

  docker-build-frontend:
    description: build the frontend docker image
    preconditions:
      - sh: '[ -n "{{.DOCKER_IMAGE_FULL_NAME}}" ]'
    dir: frontend
    cmds:
      - docker build -t "{{.DOCKER_IMAGE_FULL_NAME}}" .

  docker-build-backend:
    description: build the backend docker image
    preconditions:
      - sh: '[ -n "{{.DOCKER_IMAGE_FULL_NAME}}" ]'
    dir: backend
    cmds:
      - docker build -t "{{.DOCKER_IMAGE_FULL_NAME}}" .

  docker-build-dbbackup:
    description: build the dbbackup docker image
    preconditions:
      - sh: '[ -n "{{.DOCKER_IMAGE_FULL_NAME}}" ]'
    dir: dbbackup
    cmds:
      - docker build -t "{{.DOCKER_IMAGE_FULL_NAME}}" .

  docker-push:
    description: push the frontend docker image
    preconditions:
      - sh: '[ -n "{{.DOCKER_IMAGE_FULL_NAME}}" ]'
    dir: frontend
    cmds:
      - docker push "{{.DOCKER_IMAGE_FULL_NAME}}"

####################################################################################################
# development server tasks

  skaffold-start:
    description: build and deploy with skaffold and watch the build for changes
    cmds:
      - skaffold dev --trigger polling

  skaffold-delete:
    description: clean up a skaffold deployment
    cmds:
      - skaffold delete


  # deprecated for now because we have removed the .env in favor of k8s secrets
  # dev-server:
  #   description: run the Go dev server
  #   dir: backend
  #   interactive: true
  #   cmds:
  #     - "(set -a; source .env; set +a; go run . )"

  # frontend-watch:
  #   description: run the npm config to watch and rebuild frontend changes
  #   dir: frontend
  #   interactive: true
  #   cmds:
  #     - npm run watch

####################################################################################################
# Database tasks

  # db-server-start:
  #   description: run the local dev postgres server
  #   preconditions:
  #     - sh: '[ -n "$QUOTABLE_DEV_PG_PASSWORD" ]'
  #   cmds:
  #     # note the use of /tmp/dbdata -- if we try a local directory, it doesn't work in WSL
  #     - |
  #       docker run -d \
  #         -p 5432:5432 \
  #         --name {{.POSTGRES_LOCAL_DOCKER_NAME}} \
  #         -e POSTGRES_PASSWORD="${QUOTABLE_DEV_PG_PASSWORD}" \
  #         -e PGDATA=/var/lib/postgresql/data/pgdata \
  #         -v /tmp/dbdata:/var/lib/postgresql/data \
  #         postgres:14.2

  # db-server-stop:
  #   description: stop the local dev postgres server
  #   cmds:
  #     - docker stop {{.POSTGRES_LOCAL_DOCKER_NAME}}
  #     - docker rm {{.POSTGRES_LOCAL_DOCKER_NAME}}

  db-shell:
    description: run psql shell in the dev postgres server
    interactive: true
    vars:
      PGUSER: quotable
      PODNAME: quotable-db-0
    cmds:
      - |
        kubectl exec -it pod/{{.PODNAME}} -- /bin/bash -c 'PGPASSWORD=$POSTGRES_PASSWORD psql -U {{.PGUSER}} postgres'

  db-new-migration:
    description:  define MIGRATION=name to create a new migration with the given name
    preconditions:
      - sh: '[ -n "{{.MIGRATION}}" ]'
    cmds:
      - migrate create -ext sql -dir backend/migrations/ {{.MIGRATION}}

  db-migrations-up:
    description: run the up migrations
    vars:
      PGUSER: quotable
      PGPASSWORD:
        sh: kubectl get secret postgres-secrets -o=jsonpath='{.data.password}' | base64 -d
    cmds:
      - PGPASSWORD={{.PGPASSWORD}} migrate -database "postgres://{{.PGUSER}}:$PGPASSWORD@localhost:5432/postgres?sslmode=disable" -path backend/migrations up

  db-migrations-down:
    description: run the down migrations
    vars:
      PGUSER: quotable
      PGPASSWORD:
        sh: kubectl get secret postgres-secrets -o=jsonpath='{.data.password}' | base64 -d
      PODNAME: quotable-db-0
    cmds:
      - PGPASSWORD={{.PGPASSWORD}} migrate -database "postgres://{{.PGUSER}}:$PGPASSWORD@localhost:5432/postgres?sslmode=disable" -path backend/migrations down

  db-dump:
    description: dump the database
    preconditions:
      - sh: '[ -n "$TARGETFILE" ] && [ ! -e "$TARGETFILE" ]'
    vars:
      PGUSER: quotable
      PODNAME: quotable-db-0
    cmds:
      - kubectl exec pod/{{.PODNAME}} -- /bin/bash -c 'PGPASSWORD=$POSTGRES_PASSWORD pg_dump -U quotable postgres --column-inserts --data-only' > {{.TARGETFILE}}

  db-restore:
    description: load the database
    preconditions:
      - sh: '[ -n "$TARGETFILE" ] && [ -e "$TARGETFILE" ]'
    vars:
      PGUSER: quotable
      PGPASSWORD:
        sh: kubectl get secret postgres-secrets -o=jsonpath='{.data.password}' | base64 -d
      PODNAME: quotable-db-0
    cmds:
      - kubectl cp {{.TARGETFILE}} quotable-db-0:/tmp/in.sql
      - kubectl exec pod/{{.PODNAME}} -- /bin/bash -c 'cat /tmp/in.sql | PGPASSWORD=$POSTGRES_PASSWORD psql -U {{.PGUSER}} postgres'
      - kubectl exec pod/{{.PODNAME}} -- /bin/bash -c 'rm /tmp/in.sql'

####################################################################################################
# API Tasks


  api-generate-backend:
    description: generate the backend API handling Go code
    vars:
      API_FILE: apis/quote_api.yml
      CONFIG_SERVER: apis/server-gen-config.yml
    preconditions:
      - sh: '[ -e "{{.API_FILE}}" ]'
      - sh: '[ -e "{{.CONFIG_SERVER}}" ]'
    cmds:
      - oapi-codegen -config {{.CONFIG_SERVER}} {{.API_FILE}}

####################################################################################################
# Bootstrap tasks

  bootstrap:
    description: install all development dependencies
    cmds:
      - task: bootstrap-install-homebrew
      - task: bootstrap-install-install-js-tools
      - task: bootstrap-install-install-go-tools
      - task: bootstrap-install-migrate
      - task: bootstrap-install-oapi-codegen
      - task: bootstrap-install-kubectl
      - task: bootstrap-install-k3d
      - task: bootstrap-install-helm
      - task: bootstrap-install-git-secret
      - task: bootstrap-install-aws-cli
      - task: bootstrap-set-environment-variables
      - task: bootstrap-install-remote-tools

  bootstrap-install-go-tools:
    description: install go tools
    cmds:
      - go install honnef.co/go/tools/cmd/staticcheck@latest

  bootstrap-install-js-tools:
    description: install js tools
    cmds:
      - wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash
      - nvm install --lts
      - npm install -g @vue/cli

  bootstrap-install-kubectl:
    description: install kubectl
    cmds:
      - curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
      - sudo mv kubectl /usr/local/bin/kubectl
      - grep -qxF 'source <(kubectl completion bash)' ~/.bashrc || echo 'source <(kubectl completion bash)' >>~/.bashrc

  bootstrap-install-kubectl-aliases:
    description: install kubectl aliases
    cmds:
      
      - grep -qF 'alias kc=' ~/.bashrc || echo 'alias kc=kubectl; complete -F __start_kubectl kc;'  >>~/.bashrc
      - grep -qF 'alias kcw=' ~/.bashrc || echo 'alias kcw="watch kubectl"'  >>~/.bashrc
      - |
        grep -qF 'function kcl() {' ~/.bashrc || echo 'function kcl() {
            kubectl logs -f -l="app.kubernetes.io/service=$1"
        }' >>~/.bashrc

  bootstrap-install-k3d:
    description: install k3d
    cmds:
      - wget -q -O - https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash
      - grep -qxF 'source <(k3d completion bash)' ~/.bashrc  || echo 'source <(k3d completion bash)' >>~/.bashrc

  bootstrap-install-helm:
    description: install helm
    cmds:
      - wget https://get.helm.sh/helm-v3.9.0-linux-amd64.tar.gz
      - tar -zxvf helm-v3.9.0-linux-amd64.tar.gz
      - sudo mv linux-amd64/helm /usr/local/bin/helm
      - rm -r helm-v3.9.0-linux-amd64.tar.gz linux-amd64/
      - helm repo add bitnami https://charts.bitnami.com/bitnami
      - grep -qxF 'source <(helm completion bash)' ~/.bashrc  || echo 'source <(helm completion bash)' >>~/.bashrc

  bootstrap-install-skaffold:
    description: install skaffold
    cmds:
      - curl -Lo skaffold https://storage.googleapis.com/skaffold/releases/latest/skaffold-linux-amd64 && sudo install skaffold /usr/local/bin/ && rm ./skaffold
      - grep -qxF 'source <(skaffold completion bash)' ~/.bashrc  || echo 'source <(skaffold completion bash)' >>~/.bashrc

  bootstrap-install-migrate:
    #helper task to install the migrate CLI
    cmds:
      - go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@{{.MIGRATE_VERSION}}

  bootstrap-install-oapi-codegen:
    #helper task to install the OpenAPI codegen CLI
    cmds:
      - go install github.com/deepmap/oapi-codegen/cmd/oapi-codegen@v1.11.0

  bootstrap-install-remote-tools:
    #helper task to install the git-secret cli
    cmds:
      - brew install terraform
      - brew install hcloud

  bootstrap-install-git-secret:
    #helper task to install the git-secret cli
    cmds:
      - brew install git-secret

  bootstrap-install-homebrew:
    #helper task to install the homebrew
    cmds:
      - /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
      - grep -qF '/home/linuxbrew/.linuxbrew/bin/brew' ~/.bashrc  || echo 'eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"' >> ~/.bashrc

  bootstrap-install-aws-cli:
    cmds:
      - |
        cd /tmp &&
        curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" &&
        unzip awscliv2.zip &&
        sudo ./aws/install &&
        rm -r aws/ awscliv2.zip

  bootstrap-set-environment-variables:
    # helper task does:
    # - add $HOME/go/bin to the path in bashrc if it is not already there
    # - add QUOTABLE_DEV_PG_PASSWORD environment variables to the bashrc if it is not already there
    cmds:
      - grep -qxF 'PATH="${PATH}:$HOME/go/bin"' $HOME/.bashrc || echo 'PATH="${PATH}:$HOME/go/bin"' ```
</details>