String Replacement With Shell One-Liners

Using 'sed', 'grep', and 'for'

Table of Contents

I’m about to start working on installing FreeIPA servers, and I need to make a few small changes to variables as stored. The most efficient way was to use a few one-liners.

Rename inventory

I want to rename all ansible hosts as their FQDN. I attack the ini-style inventory file like so:

nc@tiny: ~/0/homelab.ansible/ $ sed -i "s/[a-z1-9]$/&.lan.nathancurry.com/" inventory

This simply matches each line that ends in either a letter or a number, and appends (the & operator) the rest of the FQDN.

Clean up hostvars

First, I needed to rename the files so they match the names in my new inventory file:

nc@tiny: ~/0/homelab.ansible/host_vars $ for i in `ls`; do mv $i $i.lan.nathancurry.com; done

This takes the output of the ls command and pipes that into a script, renaming each file to append the FQDN.

Get rid of FQDN variable

Since I’m using FQDNs for ansible host names, this is no longer necessary. I delete the FQDN line like so:

nc@tiny: ~/0/homelab.ansible/host_vars $ sed -i '/^fqdn/d' *

The operator d deletes the match, which is any line beginning with fqdn

Replace ip_address variable

This variable isn’t ansible best practices. It is recommended that you use FQDN for hostnames, and use the ansible_host fact to store the IP.

nc@tiny: ~/0/homelab.ansible/host_vars $ sed -i 's/ip_address/ansible_host/' *

Clean up the project

In the interest of not making any stupid life-altering mistakes, I’m taking this one rather easy. For example with the ip_address field, I locate where it exists in my project using grep:

nc@tiny: ~/0/homelab.ansible $ grep -r ip_address
roles/proxmox/tasks/deploy_containers.yml:      netif: "{{ '{\"net0\":\"name=eth0,type=veth,bridge=vmbr0,ip6=auto,gw=10.3.3.1,hwaddr=' + hostvars[item].mac_address + ',ip=' + hostvars[item].ip_address + '/25\"}' | default(defaults.netif) }}"
roles/bind/templates/named.conf.local.j2:  allow-update-forwarding {127.0.0.1; {{ hostvars['ns1'].ip_address }}; {{ hostvars['ns2'].ip_address }}; };
roles/bind/templates/named.conf.local.j2:  allow-notify { {{ hostvars['ns1'].ip_address }}; };
roles/bind/templates/named.conf.local.j2:  allow-transfer { {{ hostvars['ns1'].ip_address }}; };
roles/bind/templates/named.conf.local.j2:  masters { {{ hostvars['ns1'].ip_address }}; };
roles/bind/templates/named.conf.local.j2:  allow-update-forwarding {127.0.0.1; {{ hostvars['ns1'].ip_address }}; {{ hostvars['ns2'].ip_address }}; };
roles/bind/templates/named.conf.local.j2:  allow-notify { {{ hostvars['ns1'].ip_address }}; };
roles/bind/templates/named.conf.local.j2:  allow-transfer { {{ hostvars['ns1'].ip_address }}; };
roles/bind/templates/named.conf.local.j2:  masters { {{ hostvars['ns1'].ip_address }}; };
roles/bind/templates/db.reverse.j2:{{ hostvars[host].ip_address|regex_replace('10.3.3.') }}  IN   PTR   {{ hostvars[host].fqdn }}.
roles/bind/templates/db.forward.j2:{{ host }}  IN  A  {{ hostvars[host].ip_address }}
roles/bind/vars/main.yml:#   forward': hostvars[item].ip_address
roles/bind/vars/main.yml:#   last': hostvars[item].ip_address|regex_replace('10.3.3.')

And so on. All the matches in the bind role are what I want, so I move into the bind directory and run:

nc@tiny: ~/0/homelab.ansible/roles/bind $ sed -i 's/ip_address/ansible_host/g' */*

The globs are dangerous in uncontrolled situations but they work well here. The g at the end of the line means sed will match multiple occurrences of the same string in any given line.

Fix hostnames

Now I just need to change the hostnames to the FQDN, which is the same as above. The only thing I ran into was that I used for example a dhcp_ns1 variable:

nc@tiny: ~/0/homelab.ansible/roles/dhcpd $ grep -re 'ns[12]'
templates/dhcpd.conf.j2:  address {{ hostvars['ns1'].ansible_host }};
templates/dhcpd.conf.j2:  peer address {{ hostvars['ns2'].ansible_host }};
templates/dhcpd.conf.j2:  address {{ hostvars['ns2'].ansible_host }};
templates/dhcpd.conf.j2:  peer address {{ hostvars['ns1'].ansible_host }};
templates/dhcpd.conf.j2:  primary {{ hostvars['ns1'].ansible_host }};
templates/dhcpd.conf.j2:  primary {{ hostvars['ns1'].ansible_host }};
templates/dhcpd.conf.j2:  option domain-name-servers {{ dhcp_ns1 }}, {{ dhcp_ns2 }};
vars/main.yml:dhcp_ns1: "{{ hostvars['ns1'].ansible_host }}"
vars/main.yml:dhcp_ns2: "{{ hostvars['ns2'].ansible_host }}"

So I expand the matching to include the opening quote, and before committing, I pipe it through grep again to double check my work:

nc@tiny: ~/0/homelab.ansible/roles/dhcpd $ sed "s/'ns[12]/&.lan.nathancurry.com/g" */* | grep 'ns[12]'
  address {{ hostvars['ns1.lan.nathancurry.com'].ansible_host }};
  peer address {{ hostvars['ns2.lan.nathancurry.com'].ansible_host }};
  address {{ hostvars['ns2.lan.nathancurry.com'].ansible_host }};
  peer address {{ hostvars['ns1.lan.nathancurry.com'].ansible_host }};
  primary {{ hostvars['ns1.lan.nathancurry.com'].ansible_host }};
  primary {{ hostvars['ns1.lan.nathancurry.com'].ansible_host }};
  option domain-name-servers {{ dhcp_ns1 }}, {{ dhcp_ns2 }};
dhcp_ns1: "{{ hostvars['ns1.lan.nathancurry.com'].ansible_host }}"
dhcp_ns2: "{{ hostvars['ns2.lan.nathancurry.com'].ansible_host }}"
nc@tiny: ~/0/homelab.ansible/roles/dhcpd $