Skip to content
Techist Techist

Safely Transform Configuration Files with sed

When you edit a configuration file directly with sed, a simple string replacement can change unintended lines or introduce additional changes when the command is run again.

This article uses GNU sed and combines the following techniques to transform configuration files safely.

  • Limit the modification range with an address
  • Preserve existing indentation with a capture
  • Back up the file before modification
  • Avoid matching a state that has already been replaced
  • Use diff to check the actual changes
  • Verify that running the same transformation again does not change the content

In the example, only mode = development in a specific section of /etc/example-app.conf is changed to mode = production.

Step 1: Prepare a configuration file for testing

Section titled “Step 1: Prepare a configuration file for testing”

First, create a configuration file that contains both lines to change and lines that must remain unchanged.

Terminal window
set -euo pipefail
sudo tee /etc/example-app.conf >/dev/null <<'EOF'
global_option = keep
# BEGIN APP
  mode = development
  workers = 4
# END APP
trailing_option = keep
EOF
sudo rm -f /etc/example-app.conf.bak

The global_option, workers, and trailing_option lines are included to verify that they remain unchanged after the sed transformation.

Step 2: Replace only the target line with an address and capture

Section titled “Step 2: Replace only the target line with an address and capture”

Use an address to select the range from # BEGIN APP through # END APP, then change only the unmodified mode = development line within that range.

Terminal window
sudo sed -E -i.bak '/^# BEGIN APP$/,/^# END APP$/ s/^([[:space:]]*)mode[[:space:]]*=[[:space:]]*development[[:space:]]*$/\1mode = production/' /etc/example-app.conf

This command uses the following mechanisms.

/^# BEGIN APP$/,/^# END APP$/ is a sed address range. The substitution is applied only to the lines between these two markers.

^([[:space:]]*) captures the whitespace at the beginning of the line, and \1 restores it in the replacement. This changes only the value while preserving the existing indentation.

The search condition is also limited to development, so a line that has already been changed to production no longer matches the same substitution. This is the basis for preventing unnecessary changes on repeated runs.

With -i.bak, the pre-change content is saved to /etc/example-app.conf.bak before the original file is updated. If you want to preserve the original backup, do not overwrite it with the same .bak suffix on subsequent runs.

Step 3: Check the backup and the diff

Section titled “Step 3: Check the backup and the diff”

Compare the file before and after the change with diff. Then verify that the backup and the transformed file each exactly match the expected content.

Terminal window
set -euo pipefail


set +e
sudo diff -u /etc/example-app.conf.bak /etc/example-app.conf
diff_rc=$?
set -e
test "$diff_rc" -eq 1


sudo tee /tmp/example-app.before.expected >/dev/null <<'EOF'
global_option = keep
# BEGIN APP
  mode = development
  workers = 4
# END APP
trailing_option = keep
EOF


sudo tee /tmp/example-app.after.expected >/dev/null <<'EOF'
global_option = keep
# BEGIN APP
  mode = production
  workers = 4
# END APP
trailing_option = keep
EOF


sudo cmp -s /tmp/example-app.before.expected /etc/example-app.conf.bak
sudo cmp -s /tmp/example-app.after.expected /etc/example-app.conf
printf '%s\n' 'backup-and-content-ok'

An exit code of 1 from diff means that the compared files differ. In this example, the intended result is that only the mode line has changed.

The cmp checks compare the entire files, confirming both that the backup retains the original content and that lines other than mode were not changed.

Step 4: Run the same transformation again and verify idempotency

Section titled “Step 4: Run the same transformation again and verify idempotency”

When a configuration change is incorporated into a script or automation, rerunning the same command should not corrupt or further alter the configuration.

Calculate the hash of the transformed file, run the same sed expression again, and compare the hash afterward. This second run uses -i instead of -i.bak so that the backup created by the first run is not overwritten.

Terminal window
set -euo pipefail


before=$(sudo sha256sum /etc/example-app.conf | awk '{print $1}')


sudo sed -E -i '/^# BEGIN APP$/,/^# END APP$/ s/^([[:space:]]*)mode[[:space:]]*=[[:space:]]*development[[:space:]]*$/\1mode = production/' /etc/example-app.conf


after=$(sudo sha256sum /etc/example-app.conf | awk '{print $1}')


test "$before" = "$after"
test "$(sudo grep -Fxc '  mode = production' /etc/example-app.conf)" -eq 1
sudo cmp -s /tmp/example-app.before.expected /etc/example-app.conf.bak
printf '%s\n' 'idempotent-rerun-ok'

If the SHA-256 value is identical before and after the repeated run, the second sed execution did not change the file content.

By limiting the modification range with an address and using a specific search condition that includes the current value, you can reduce unintended changes compared with applying an unconditional replacement to the entire configuration file. Combining this with a pre-change backup and a diff check provides both a way to verify the applied changes and a recovery path.

Category: Linux