Example files for today can be found in my Continuous Learning repository at https://github.com/jantytgat/continuouslearning/tree/main/2023/day002

Introduction

In college, I started tinkering with Debian (Debian GNU/Linux 3.0 - Woody) as my first introduction to other operating systems besides MS-DOS and Windows. The use of deb-packages through apt-get felt so natural that I never got around to using Red Hat or Fedora in a sensible way (yum always felt broken).

I became a heavy user of Ubuntu Server in my homelab, which is still Debian-based, as I wanted to run some more bleeding-edge packages on my servers (at the cost of more maintenance and possible issues). Installing the operating system, however, turned out to be a tedious task if you have to do it over and over again. Creating a template soon became a thing using preseed files, specially with virtualization coming of age in the late 2000s. One of the remaining issues was that I had to keep my template up-to-date or be exposed to (sometimes) lengthy update/upgrade processes.

Enter Packer by HashiCorp.

Packer

Packer makes templating a lot easier because of the involved automation and support for different builders. It also makes it easier to store the configuration of those templates in a Git repository. There is one caveat though: if you are storing the configuration in a version control system, you will likely need the possibility to safely store sensitive data like usernames and passwords.

Enter 1Password CLI.

1Password

Although HashiCorp offers a product to handle sensitive data with Vault, it didn’t make sense for me as I’m already using 1Password. With the recent addition of 1Password CLI, integrating this tool in my workflow is the right thing to do!

Prerequisites

Proxmox

My homelab is based on Proxmox as the hypervisor, which is supported by Packer natively. In my environment, I will use the ISO-builder. For more information on this builder, check https://developer.hashicorp.com/packer/plugins/builders/proxmox/iso.

Everything discussed below should also work for other builders, as the main interface is Packer.

Packer

Make sure Packer is installed according to your platform. Instructions can be found at https://developer.hashicorp.com/packer/downloads.

1Password

Obviously, for this topic you will need an account with 1Password, they have a 14-day trial available.

To get started, you will need to install 1Password CLI and connect to your vault. Use the op --help command to get a list of all available flags and commands.

Configuration

Packer

Build configuration

My build configuration is referring to a builder-source called homelab-ubuntu-22-04 for the builder proxmox-iso It is also referring to a couple of variables:

template_script_repository_url
template_script_repository_name
ssh_provisioner

After the installation has completed, Packer will run a couple of scripts to customize the installation. The scripts are located in a public git repository at https://github.com/jantytgat/iac-scripts, and are initiated through the local scripts template.sh and remove_provisioner_user.sh.

By having the provisioner user removed at the end of the installation sequence, having the credentials of that user in the git repository isn’t a disaster, but it is far from best-practice.

build {
    name = "proxmox-ubuntu-22-04"
    
    sources = ["source.proxmox-iso.homelab-ubuntu-22-04"]

    provisioner "shell" {
        execute_command = "sudo -S -E bash -c '{{ .Vars }} {{ .Path }}'"
        environment_vars = [
            "PROVISIONER_SCRIPT_REPOSITORY_URL=${var.template_script_repository_url}",
            "PROVISIONER_SCRIPT_REPOSITORY_NAME=${var.template_script_repository_name}",
            "PROVISIONER_USERNAME=${var.ssh_provisioner.username}",
        ]
        scripts = [
            "../scripts/template.sh",
            "../scripts/remove_provisioner_user.sh",
        ]
    }
}

Build source

The build source configuration has all the magic information needed to build an image on Proxmox from an ISO-file. I won’t be covering all the details for this builder, but some fields need some explanation.

source "proxmox-iso" "homelab-ubuntu-22-04" {
    # PROXMOX CONNECTION DETAILS
    proxmox_url = "${var.proxmox_url}/api2/json"
    insecure_skip_tls_verify = var.proxmox_skip_certificate_validation
    username = "${var.proxmox_username}@pam"
    password = var.proxmox_password
    node = lower(var.proxmox_node)

    # INSTALLER ISO
    iso_file = "${var.proxmox_iso_storage_name}:iso/${var.proxmox_iso_filename}"
    iso_checksum = var.proxmox_iso_checksum
    unmount_iso = var.proxmox_unmount_iso

    # VM DETAILS
    vm_id = var.proxmox_vm_id
    vm_name = var.proxmox_vm_name

    # CLOUD-INIT
    cloud_init = true
    cloud_init_storage_pool = var.proxmox_vm_storage_name

    #http_directory = var.proxmox_cloudinit_path
    http_content = {
        "/meta-data" = file("cloud-init/meta-data")
        "/user-data" = templatefile("cloud-init/user-data", { "provisioner_username" = var.ssh_provisioner.username, "provisioner_password" = bcrypt(var.ssh_provisioner.password, 10), "provisioner_sshkey" = var.ssh_provisioner.sshkey})
    }

    boot_wait = "5s"
    boot_command = [
        "c",
        "linux /casper/vmlinuz --- autoinstall ds='nocloud-net;s=http://{{.HTTPIP}}:{{.HTTPPort}}/' <enter>",
        "initrd /casper/<wait2s>initrd<enter>",
        "boot<wait><enter>"
    ]

    # VM CONFIGURATION
    os = var.proxmox_vm_os_type

    cpu_type = var.proxmox_vm_cpu_type
    sockets = var.proxmox_vm_cpu_sockets
    cores = var.proxmox_vm_cpu_cores

    memory = var.proxmox_vm_memory

    network_adapters {
        model = var.proxmox_vm_network_adapters.model
        bridge = var.proxmox_vm_network_adapters.bridge
        vlan_tag = var.proxmox_vm_network_adapters.vlan_tag
    }

    disks {
        type = var.proxmox_vm_disks.type
        disk_size = var.proxmox_vm_disks.disk_size
        storage_pool = var.proxmox_vm_disks.storage_pool
        storage_pool_type = var.proxmox_vm_disks.storage_pool_type
        format = var.proxmox_vm_disks.format

    }

    # SSH CONNECTION
    ssh_timeout = var.ssh_provisioner.timeout
    ssh_username = var.ssh_provisioner.username
    ssh_password = var.ssh_provisioner.password
}

http_content

Earlier, I would be providing the cloud-init data to the virtual machine using the configuration directive http_directory. This directive instructs Packer to start a webserver, serving static content from the directory specified. This is static content however, and cannot be made dynamic.

I tried thinking about ways to start an external webserver or write a script to make the identity part in user-data dynamic. Packer to the rescue!!!

The directive http_content allows you to control which data is served by the webserver, while also being able to pass data from variables using the function templatefile():

    http_content = {
        "/meta-data" = file("cloud-init/meta-data")
        "/user-data" = templatefile("cloud-init/user-data", { "provisioner_username" = var.ssh_provisioner.username, "provisioner_password" = bcrypt(var.ssh_provisioner.password, 10), "provisioner_sshkey" = var.ssh_provisioner.sshkey})
    }

Note I am calling the bcrypt() function as well, to make sure the password is hashed and accepted by cloud-init. Without this call, the SSH connection will fail.

The second argument of the function is a map of key-value pairs of strings to be replaced in the template file. In turn, the values are referencing variables from our Packer configuration.

user-data

#cloud-config
autoinstall:
  version: 1
  locale: en_US
  keyboard:
    layout: en
    variant: us
  source:
    id: ubuntu-server-minimized
  storage:
    layout:
      name: direct
  identity:
    hostname: packer-ubuntu-22-04-proxmox
    username: ${provisioner_username}
    password: ${provisioner_password}
  ssh:
    install-server: yes
    allow-pw: yes
    authorized-keys:
      - ${provisioner_sshkey}
  packages:
    - apparmor-profiles
    - apparmor-utils
    - apt-transport-https
    - ca-certificates
    - cloud-init
    - curl
    - fail2ban
    - gnupg2
    - htop
    - iftop
    - net-tools
    - ntp
    - qemu-guest-agent
    - screen
    - traceroute
  late-commands:
      - echo 'Defaults:stager !requiretty' > /target/etc/sudoers.d/stager
      - echo 'stager ALL=(ALL) NOPASSWD:ALL' >> /target/etc/sudoers.d/stager
      - chmod 440 /target/etc/sudoers.d/stager
      - curtin in-target --target=/target -- apt update           
      - curtin in-target --target=/target -- apt upgrade -y

Variables

Definitions

Packer requires all variables used through the configuration to be defined.

variable "proxmox_url" {
    type = string
    description = "URL or IP Address of the Proxmox API, without a HTTP PATH"
    sensitive = false
}

variable "proxmox_port" {
    type = number
    default = 8006
    description = "Port on which the Proxmox API listens"
    sensitive = false
}

variable "proxmox_skip_certificate_validation" {
    type = bool
    default = false
    description = "Skip SSL certificate validation when connecting to the Proxmox API"
    sensitive = false
}

variable "proxmox_username" {
    type = string
    default = "root@pam"
    description = "Proxmox API Username"
    sensitive = true
}

variable "proxmox_password" {
    type = string
    description = "Proxmox API Password"
    sensitive = true
}

variable "proxmox_node" {
    type = string
    description = "Proxmox Node Name that will run the build"
    sensitive = false
}

variable "proxmox_iso_storage_name" {
    type = string
    default = "local"
    description = "Name of the storage repository where the ISO-file can be found"
    sensitive = false
}

variable "proxmox_iso_filename" {
    type = string
    description = "Filename of the ISO-file"
    sensitive = false
}

variable "proxmox_iso_checksum" {
    type = string
    description = "Checksum for the ISO file"
    sensitive = false
}

variable "proxmox_unmount_iso" {
    type = bool
    default = true
    description = "Unmount the ISO-file when building is complete"
    sensitive = false
}

variable "proxmox_template_name" {
    type = string
    default = "template_ubuntu_22_04"
    description = "Template name when Packer has completed the build process"
    sensitive = false
}

variable "proxmox_vm_name" {
    type = string
    default = "packer-ubuntu-22-04"
    description = "Virtual machine name while Packer is running"
    sensitive = false
}

variable "proxmox_vm_id" {
    type = number
    default = 999999001
    description = "Virtual machine ID"
}

variable "proxmox_vm_storage_name" {
    type = string
    description = "Name of the storage pool to store the virtual machine"
    sensitive = false
}

variable "proxmox_vm_os_type" {
    type = string
    default = "l26"
    description = "Virtual machine type"
    sensitive = false
}

variable "proxmox_vm_cpu_type" {
    type = string
    default = "kvm64"
    description = "Virtual machine processor type"
    sensitive = false
}

variable "proxmox_vm_cpu_sockets" {
    type = number
    default = 1
    description = "Virtual machine CPU sockets"
    sensitive = false
}

variable "proxmox_vm_cpu_cores" {
    type = number
    default = 2
    description = "Virtal machine CPU cores"
    sensitive = false
}

variable "proxmox_vm_memory" {
    type = number
    default = 4096
    description = "Virtual machine memory"
    sensitive = false
}

variable "proxmox_vm_network_adapters" {
    type = object({
        model = string
        bridge = string
        vlan_tag = number
    })
    default = {
            model = "virtio"
            bridge = ""
            vlan_tag = 0
    }
}

variable "proxmox_vm_disks" {
    type = object({
        type = string
        disk_size = string
        storage_pool = string
        storage_pool_type = string
        format = string
    })
    default = {
            type = "virtio"
            disk_size = "50G"
            storage_pool = ""
            storage_pool_type = ""
            format = "qcow2"
    }
}

variable "ssh_provisioner" {
    type = object({
        timeout = string
        username = string
        password = string
        sshkey = string
    })
    default = {
        timeout = "5m"
        username = ""
        password = ""
        sshkey = ""
    }
    sensitive = true
}

variable "template_script_repository_url" {
    type = string
    default = "https://github.com/jantytgat/iac-scripts"
    description = "URL of the Git-repository containing all template scripts to configure the image"
    sensitive = false
}

variable "template_script_repository_name" {
    type = string
    default = "iac-scripts"
    description = "Name of the Git-repository containing all template scripts to configure the image"
    sensitive = false
}

Values

Here comes the bulk of the magic involved. Specifying values for the variables typically involves specifying sensitive data, such as usernames and passwords.

We will replace these values with a referency to an item in our 1Password vault.

proxmox_url = "{{ op://clcd-2023/hypervisor/website }}"
proxmox_skip_certificate_validation = true
proxmox_username = "{{ op://clcd-2023/hypervisor/username }}"
proxmox_password = "{{ op://clcd-2023/hypervisor/password }}"
proxmox_node = "{{ op://clcd-2023/hypervisor/Title }}"

proxmox_iso_storage_name = "local"
proxmox_iso_filename = "ubuntu-22.04.1-live-server-amd64.iso"
proxmox_iso_checksum = "sha256:10f19c5b2b8d6db711582e0e27f5116296c34fe4b313ba45f9b201a5007056cb"


proxmox_template_name = "template_ubuntu_22_04"
proxmox_vm_name = "packer-ubuntu-22-04"

proxmox_vm_storage_name = "local"
#proxmox_vm_storage_type = "nfs"


proxmox_vm_network_adapters = {
    model = "virtio"
    bridge = "vmbr1"
    vlan_tag = 3142
}

proxmox_vm_disks = {
    type = "virtio"
    disk_size = "50G"
    storage_pool = "local"
    storage_pool_type = "directory"
    format = "qcow2"
}


ssh_provisioner = {
    timeout = "10m"
    username = "{{ op://clcd-2023/provisioner/username }}"
    password = "{{ op://clcd-2023/provisioner/password }}"
    sshkey = "{{ op://clcd-2023/provisioner/public key }}"
}

1Password

To automate the required calls before Packer can run, I’ve created some scripts to be used in a Makefile. The two scripts are responsible for:

Logging in to 1Password and injecting the sensitive data in a file called build.auto.pkrvars.hcl, which will automatically be picked up by Packer at runtime.
Cleaning up the generated file, so no sensitive data remains on the system after Packer is complete.

If Packer quits unexpectedly or you cancel the operation, you might need to call the cleanup script manually. As always, be vigilant about sensitive data lingering on your workstation!

Makefile

#!/bin/sh
eval $(op signin)
cd packer
op inject -i build.pkrvars.hcl -o build.auto.pkrvars.hcl
packer build -timestamp-ui .

The .gitignore file makes sure no files with a name like *.auto.pkrvars.hcl are committed to git.

Execution

When running make proxmox-ubuntu-22-04 from the command-line:

1Password CLI will ask for your password
1Password CLI will inject all {{ op:// }} references into build.auto.pkrvars.hcl
Launch Packer

Resources

Internal

Title	Link
GitHub	https://github.com/jantytgat/continuouslearning
Scripts for IAC	https://github.com/jantytgat/iac-scripts

External

Title	Link
Packer	https://packer.io
Packer - Documentation	https://developer.hashicorp.com/packer/docs
Packer - templatefile	https://developer.hashicorp.com/packer/docs/templates/hcl_templates/functions/file/templatefile
Packer - Assigning variables	https://developer.hashicorp.com/packer/guides/hcl/variables#assigning-variables
1Password	https://1password.com
1Password CLI - Documentation	https://developer.1password.com/docs/cli/
1Password CLI - Load secrets into config files	https://developer.1password.com/docs/cli/secrets-config-files
gitignore	https://www.toptal.com/developers/gitignore
gitignore - Packer template	https://www.toptal.com/developers/gitignore/api/packer
Ubuntu autoinstall reference	https://ubuntu.com/server/docs/install/autoinstall-reference

Jan Tytgat

Day 002 - Packer integration with 1Password

Introduction

Packer

1Password

Prerequisites

Proxmox

Packer

1Password

Configuration

Packer

Build configuration

Build source

http_content

user-data

Variables

Definitions

Values

1Password

Makefile

Execution

Resources

Internal

External