Secure Initial Configuration of a Linux OpenSSH Server

1. Back to home
2. Linux
3. Secure Initial Configuration of a Linux OpenSSH Server

Secure Initial Configuration of a Linux OpenSSH Server

When administering a Linux server over SSH, you need to properly restrict the authentication methods and the users who are allowed to log in.

Changing multiple SSH settings at once creates a risk that even the administrator will no longer be able to log in. In particular, if password authentication is disabled before public key authentication has been verified, console access may be required for recovery.

This article targets RHEL 8-compatible Linux systems and applies public key authentication, root login restrictions, password authentication restrictions, and access control with AllowUsers step by step while preserving the existing SSH administration path.

Do not close the existing SSH administration session while changing the configuration. Continue only after confirming that you can successfully connect with a new session.

Prerequisites

The following environment is assumed.

• RHEL 8-compatible Linux
• OpenSSH Server is already installed
• sshd is managed with systemd
• The server can be reached from an SSH client
• sudo can be executed on the server
• TCP/22 can be used as the administration path
• A console-based management method is available in case of failure

When verifying public key authentication, the private key actually held by the client and the public key registered on the server must belong to the same key pair.

To make verification easier, the public and private keys are represented as variables below. The key pair shown in the table is a published sample and must not be used in production. In a real environment, replace it with a key pair that you have securely generated and stored yourself.

Variable notation

Variable name	Configuration example	Description
<<SERVER_IP>>	192.0.2.10	Destination IP address of the SSH server
<<ADMIN_USER>>	exampleadmin	Test administrative user that uses public key authentication
<<DENIED_USER>>	exampledenied	Test user used to verify denial by AllowUsers
<<TEST_PUBLIC_KEY>>	ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKdKDcDG6HU9jILOorFOFcE1gqK+3tRifzSFa7aCJcIm	Public key registered for the test users
<<TEST_PRIVATE_KEY_B64>>	LS0tLS1CRUdJTiBPUEVOU1NIIFBSSVZBVEUgS0VZLS0tLS0KYjNCbGJuTnphQzFyWlhrdGRqRUFBQUFBQkc1dmJtVUFBQUFFYm05dVpRQUFBQUFBQUFBQkFBQUFNd0FBQUF0emMyZ3RaV1F5TlRVeApPUUFBQUNDblNnM0F4dWgxUFl5Q3pxS3hUaFhCTllLaXZ0N1VZbjgwaFd1MmdpWENKZ0FBQUlqc0oxY1Y3Q2RYRlFBQUFBdHpjMmd0ClpXUXlOVFV4T1FBQUFDQ25TZzNBeHVoMVBZeUN6cUt4VGhYQk5ZS2l2dDdVWW44MGhXdTJnaVhDSmdBQUFFQ1hVdUVYYjlkMFhFSEQKemRhcXdITjFhdGFVTnBhbGVMdVd4b3MySVBBR1Q2ZEtEY0RHNkhVOWpJTE9vckZPRmNFMWdxSyszdFJpZnpTRmE3YUNKY0ltQUFBQQpBQUVDQXdRRgotLS0tLUVORCBPUEVOU1NIIFBSSVZBVEUgS0VZLS0tLS0K	Base64-encoded value of the OpenSSH private key corresponding to the public key above
<<CONFIG_BACKUP>>	/etc/ssh/sshd_config.example-backup	Backup of sshd_config before the change
<<CONFIG_DROPIN>>	/etc/ssh/sshd_config.d/00-example-hardening.conf	SSH configuration file used for verification

192.0.2.10 is an IP address reserved for documentation. Replace it with the actual destination address.

Because the sample private key is publicly available, it must not be used on Internet-facing servers or in production environments. In production, generate your own private key, store it securely, and register only the corresponding public key on the server.

Step 1: Check the SSH service and current configuration

First, check the status of the SSH service.

sudo systemctl is-active sshd
sudo systemctl is-enabled sshd
sudo systemctl status sshd --no-pager

Confirm that is-active returns active.

Next, before making any changes, validate the configuration syntax and check the main effective values.

sudo sshd -t
sudo sshd -T | grep -E '^(permitrootlogin|passwordauthentication|kbdinteractiveauthentication|pubkeyauthentication|allowusers) '

If sshd -t reports an error, fix the existing configuration before adding hardening settings.

Step 2: Register a public key for a non-root user

Before disabling password authentication, create a non-root administrative user that will use public key authentication.

Also register the same public key for the user that will later be used to verify denial by AllowUsers.

sudo useradd -m <<ADMIN_USER>>
sudo useradd -m <<DENIED_USER>>
for user in <<ADMIN_USER>> <<DENIED_USER>>; do
  home="$(getent passwd "$user" | cut -d: -f6)"
  sudo install -d -m 700 -o "$user" -g "$user" "$home/.ssh"
  printf '%s\n' '<<TEST_PUBLIC_KEY>>' | sudo tee "$home/.ssh/authorized_keys" >/dev/null
  sudo chown "$user:$user" "$home/.ssh/authorized_keys"
  sudo chmod 600 "$home/.ssh/authorized_keys"
  sudo restorecon -RF "$home/.ssh"
done

In a real environment, replace <<TEST_PUBLIC_KEY>> with the public key corresponding to the private key used by the connecting client.

On the client side, you can load the private key into a temporary ssh-agent and verify the connection without saving a new key file to disk.

ssh-agent sh -c 'printf "%s" "<<TEST_PRIVATE_KEY_B64>>" | base64 -d | ssh-add - >/dev/null 2>&1 &&
ssh -o UserKnownHostsFile=/dev/null \
  -o StrictHostKeyChecking=no \
  -o UpdateHostKeys=no \
  -o ControlMaster=no \
  -o ControlPath=none \
  -o BatchMode=yes \
  -o PreferredAuthentications=publickey \
  -o PasswordAuthentication=no \
  -o KbdInteractiveAuthentication=no \
  -o ConnectTimeout=10 \
  <<ADMIN_USER>>@<<SERVER_IP>> '"'"'printf "BASELINE_PUBLIC_KEY_OK\n"'"'"''

If this connection succeeds, you have confirmed that the public key registered on the server and the private key used by the client belong to the same key pair and that public key authentication works for the non-root user.

Next, verify that the user used for the denial test can also connect with the same key before the configuration change.

ssh-agent sh -c 'printf "%s" "<<TEST_PRIVATE_KEY_B64>>" | base64 -d | ssh-add - >/dev/null 2>&1 &&
ssh -o UserKnownHostsFile=/dev/null \
  -o StrictHostKeyChecking=no \
  -o UpdateHostKeys=no \
  -o ControlMaster=no \
  -o ControlPath=none \
  -o BatchMode=yes \
  -o PreferredAuthentications=publickey \
  -o PasswordAuthentication=no \
  -o KbdInteractiveAuthentication=no \
  -o ConnectTimeout=10 \
  <<DENIED_USER>>@<<SERVER_IP>> '"'"'printf "DENIED_USER_KEY_READY\n"'"'"''

This preliminary check allows you to verify later that the connection denial occurred after AllowUsers was applied rather than simply because a valid public key was missing.

Step 3: Back up the SSH configuration file

Before changing the configuration, back up the current sshd_config.

sudo cp -a /etc/ssh/sshd_config <<CONFIG_BACKUP>>
sudo test -s <<CONFIG_BACKUP>>

Keep the backup until all configuration changes are complete so that you can restore the original configuration if necessary.

Step 4: Check the loading order and configure SSH restrictions

In OpenSSH, the first value obtained for the same keyword is used. Therefore, simply creating an additional configuration file is not always sufficient. Values specified earlier in sshd_config or the position of an Include directive can prevent the intended settings from becoming effective.

Before making changes, check the positions of Include, Match, and the relevant keywords in the main configuration file, along with any existing drop-ins. Then, if necessary, add an explicit Include directive at the beginning of the main configuration file so that the new additional configuration file can be loaded first as global configuration.

The following settings will be applied.

Setting	Value	Purpose
PermitRootLogin	no	Prohibit SSH login as root
PubkeyAuthentication	yes	Enable public key authentication
PasswordAuthentication	no	Prohibit password authentication
KbdInteractiveAuthentication	no	Prohibit keyboard-interactive authentication
AllowUsers	Administrative user	Restrict the users allowed to log in via SSH

sudo grep -nE '^[[:space:]]*(Include|Match|PermitRootLogin|PubkeyAuthentication|PasswordAuthentication|KbdInteractiveAuthentication|AllowUsers)([[:space:]]|$)' /etc/ssh/sshd_config || true
for file in /etc/ssh/sshd_config.d/*.conf; do
  [ -f "$file" ] || continue
  printf '### %s\n' "$file"
  sudo grep -nE '^[[:space:]]*(Include|Match|PermitRootLogin|PubkeyAuthentication|PasswordAuthentication|KbdInteractiveAuthentication|AllowUsers)([[:space:]]|$)' "$file" || true
done
sudo install -d -m 755 /etc/ssh/sshd_config.d
printf 'PermitRootLogin no\nPubkeyAuthentication yes\nPasswordAuthentication no\nKbdInteractiveAuthentication no\nAllowUsers <<ADMIN_USER>>\n' | sudo tee <<CONFIG_DROPIN>> >/dev/null
sudo chmod 600 <<CONFIG_DROPIN>>
first_active="$(sudo awk '!/^[[:space:]]*($|#)/ {print; exit}' /etc/ssh/sshd_config)"
if [ "$first_active" != "Include <<CONFIG_DROPIN>>" ]; then
  sudo sed -i '1i Include <<CONFIG_DROPIN>>' /etc/ssh/sshd_config
fi
sudo test -s <<CONFIG_DROPIN>> &&
test "$(sudo awk '!/^[[:space:]]*($|#)/ {print; exit}' /etc/ssh/sshd_config)" = "Include <<CONFIG_DROPIN>>"

This approach allows the new configuration to be evaluated first in the global context not only when no Include directive already exists, but also when authentication settings appear before an existing Include or when an Include would otherwise be processed after a Match block.

When making a similar change in production, clean up duplicate settings and manage the file structure as a permanent configuration.

Step 5: Validate the configuration syntax and effective values

Before reloading the configuration, validate its syntax.

sudo sshd -t

If an error is displayed, do not apply the configuration. Correct the added settings or the position of the Include directive.

Next, check the effective configuration for the administrative user.

sudo sshd -T -C user=<<ADMIN_USER>>,host=localhost,addr=127.0.0.1 | grep -E '^(permitrootlogin|passwordauthentication|kbdinteractiveauthentication|pubkeyauthentication|allowusers) '

Confirm that the following values are effective.

• permitrootlogin no
• passwordauthentication no
• kbdinteractiveauthentication no
• pubkeyauthentication yes
• allowusers exampleadmin

If the values are not as expected, do not proceed with the reload. Recheck the loading order reviewed in Step 4, the existing settings, and the Match conditions.

Step 6: Safely reload the SSH configuration

Reload the sshd configuration only if both the syntax check and the effective configuration check succeed.

Do not close the existing SSH administration session.

sudo sshd -t && sudo systemctl reload sshd
sudo systemctl is-active sshd

After the reload, verify public key authentication for the administrative user from a separate session.

ssh-agent sh -c 'printf "%s" "<<TEST_PRIVATE_KEY_B64>>" | base64 -d | ssh-add - >/dev/null 2>&1 &&
ssh -o UserKnownHostsFile=/dev/null \
  -o StrictHostKeyChecking=no \
  -o UpdateHostKeys=no \
  -o ControlMaster=no \
  -o ControlPath=none \
  -o BatchMode=yes \
  -o PreferredAuthentications=publickey \
  -o PasswordAuthentication=no \
  -o KbdInteractiveAuthentication=no \
  -o ConnectTimeout=10 \
  <<ADMIN_USER>>@<<SERVER_IP>> '"'"'printf "RELOAD_SSH_OK\n"'"'"''

Close the existing administration session only after confirming that a new session can connect successfully.

Step 7: Verify that root login is denied

Try to establish a new SSH connection as root.

ssh-agent sh -c 'printf "%s" "<<TEST_PRIVATE_KEY_B64>>" | base64 -d | ssh-add - >/dev/null 2>&1 &&
if ssh -o UserKnownHostsFile=/dev/null \
  -o StrictHostKeyChecking=no \
  -o UpdateHostKeys=no \
  -o ControlMaster=no \
  -o ControlPath=none \
  -o BatchMode=yes \
  -o PreferredAuthentications=publickey \
  -o ConnectTimeout=10 \
  root@<<SERVER_IP>> true; then
  echo ROOT_LOGIN_UNEXPECTEDLY_ALLOWED
  exit 1
else
  echo ROOT_LOGIN_DENIED
fi'

Along with the connection test, confirm in Step 5 that permitrootlogin no is the effective setting.

Step 8: Verify that password authentication is denied

Disable public key authentication and keyboard-interactive authentication on the client side, then verify that a connection requesting only password authentication cannot be established.

if ssh -o UserKnownHostsFile=/dev/null \
  -o StrictHostKeyChecking=no \
  -o UpdateHostKeys=no \
  -o ControlMaster=no \
  -o ControlPath=none \
  -o BatchMode=yes \
  -o PubkeyAuthentication=no \
  -o KbdInteractiveAuthentication=no \
  -o PreferredAuthentications=password \
  -o ConnectTimeout=10 \
  <<ADMIN_USER>>@<<SERVER_IP>> true; then
  echo PASSWORD_AUTH_UNEXPECTEDLY_ALLOWED
  exit 1
else
  echo PASSWORD_AUTH_DENIED
fi

Also confirm in the effective settings from Step 5 that passwordauthentication no is set.

Step 9: Verify access control with AllowUsers

Try connecting again with the denial-test user that was able to log in with a public key before the configuration change.

ssh-agent sh -c 'printf "%s" "<<TEST_PRIVATE_KEY_B64>>" | base64 -d | ssh-add - >/dev/null 2>&1 &&
if ssh -o UserKnownHostsFile=/dev/null \
  -o StrictHostKeyChecking=no \
  -o UpdateHostKeys=no \
  -o ControlMaster=no \
  -o ControlPath=none \
  -o BatchMode=yes \
  -o PreferredAuthentications=publickey \
  -o PasswordAuthentication=no \
  -o KbdInteractiveAuthentication=no \
  -o ConnectTimeout=10 \
  <<DENIED_USER>>@<<SERVER_IP>> true; then
  echo ALLOWUSERS_UNEXPECTEDLY_ALLOWED
  exit 1
else
  echo ALLOWUSERS_DENIED
fi'

Next, confirm that the administrative user allowed by AllowUsers can still authenticate with a public key.

ssh-agent sh -c 'printf "%s" "<<TEST_PRIVATE_KEY_B64>>" | base64 -d | ssh-add - >/dev/null 2>&1 &&
ssh -o UserKnownHostsFile=/dev/null \
  -o StrictHostKeyChecking=no \
  -o UpdateHostKeys=no \
  -o ControlMaster=no \
  -o ControlPath=none \
  -o BatchMode=yes \
  -o PreferredAuthentications=publickey \
  -o PasswordAuthentication=no \
  -o KbdInteractiveAuthentication=no \
  -o ConnectTimeout=10 \
  <<ADMIN_USER>>@<<SERVER_IP>> '"'"'printf "ALLOWUSERS_ALLOWED\n"'"'"''

By checking both the denied user and the allowed user, you can confirm that access control with AllowUsers is working while the administration path remains available.

Step 10: Restore the original configuration

If a problem occurs, recover using the retained administration session or the console.

Delete the added configuration file and restore the backed-up sshd_config. Because the backup also contains the original Include structure, the explicit Include added in Step 4 will also be restored to its original state.

sudo rm -f <<CONFIG_DROPIN>>
sudo cp -a <<CONFIG_BACKUP>> /etc/ssh/sshd_config

After restoration, validate the syntax and reload sshd only if the check succeeds.

sudo sshd -t && sudo systemctl reload sshd
sudo systemctl is-active sshd

Finally, establish a new SSH connection with the administrative user.

ssh-agent sh -c 'printf "%s" "<<TEST_PRIVATE_KEY_B64>>" | base64 -d | ssh-add - >/dev/null 2>&1 &&
ssh -o UserKnownHostsFile=/dev/null \
  -o StrictHostKeyChecking=no \
  -o UpdateHostKeys=no \
  -o ControlMaster=no \
  -o ControlPath=none \
  -o BatchMode=yes \
  -o PreferredAuthentications=publickey \
  -o PasswordAuthentication=no \
  -o KbdInteractiveAuthentication=no \
  -o ConnectTimeout=10 \
  <<ADMIN_USER>>@<<SERVER_IP>> '"'"'printf "ROLLBACK_SSH_OK\n"'"'"''

Confirm that public key authentication for the administrative user still succeeds after the restoration.

Troubleshooting

Effective settings do not change after writing the drop-in

The existence of an additional configuration file alone does not mean that its settings are actually being used.

Check the following points.

• Is the target file included from sshd_config?
• Is the Include directive placed after an existing definition of the same keyword?
• Is the same keyword set earlier in another drop-in?
• Are the settings inside or outside a Match block as intended?
• Do the effective values from sshd -T -C match the expected values?

Decide whether the configuration can be reloaded based on the result of sshd -T, not merely on the contents of the configuration files.

Permission denied with public key authentication

Confirm that the public key registered on the server and the private key actually presented by the client belong to the same key pair.

Also check the following items.

• Is .ssh owned by the target user and set to mode 700?
• Is authorized_keys owned by the target user and set to mode 600?
• Is the SELinux context correct?
• Is PubkeyAuthentication enabled?
• Is the user being denied by AllowUsers or another access restriction?

SSH connections no longer work

If the existing administration session is still available, use that session to inspect the configuration.

If the existing session has also been lost, restore access through an administration path that does not depend on SSH, such as the virtual machine console.

Summary

To securely perform the initial configuration of OpenSSH Server, it is important to reliably establish an alternative administration path using public key authentication before applying restrictions.

Also, do not assume that a setting is effective simply because it has been written to a drop-in. Check the position of Include, the existing settings, and Match conditions, and proceed with the reload only when both sshd -t and sshd -T return the expected results.

After changing the configuration, keep the existing session open while verifying a new SSH connection, and make sure that you can restore the original configuration from the backup if a problem occurs.

Related articles

• How to Set Up an SSH Server on Windows ServerA comprehensive step-by-step guide to installing and configuring an OpenSSH Server on Windows Server.
• PowerShell Scripting Best Practices GuideComprehensive guide covering PowerShell scripting best practices for readability, performance, error handling, and security.
• SSH Public Key Authentication Client Setup (Windows/Linux)Step-by-step guide on how to configure SSH public key authentication on the client side for Windows and Linux systems.

Category: Linux