I used to rely on DuckDNS to make my workstations accessible over the internet. However, I found that my auth logs were getting spammed with failed login attempts.

Click to see a sample of the auth logs
$ journalctl -f

Feb 27 21:31:54 prometheus sshd[28079]: pam_unix(sshd:auth): check pass; user unknown
Feb 27 21:31:54 prometheus sshd[28079]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=185.196.220.81
Feb 27 21:31:55 prometheus sshd[28079]: Failed password for invalid user user from 185.196.220.81 port 51248 ssh2
Feb 27 21:31:55 prometheus sshd[28077]: Failed password for root from 218.92.0.243 port 54595 ssh2
Feb 27 21:31:56 prometheus sshd[28079]: Received disconnect from 185.196.220.81 port 51248:11: end [preauth]
Feb 27 21:31:56 prometheus sshd[28079]: Disconnected from invalid user user 185.196.220.81 port 51248 [preauth]
Feb 27 21:31:58 prometheus sshd[28083]: Invalid user user from 185.196.220.81 port 37162
Feb 27 21:31:58 prometheus sshd[28083]: pam_unix(sshd:auth): check pass; user unknown
Feb 27 21:31:58 prometheus sshd[28083]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=185.196.220.81
Feb 27 21:31:59 prometheus sshd[28077]: Failed password for root from 218.92.0.243 port 54595 ssh2
Feb 27 21:32:00 prometheus sshd[28083]: Failed password for invalid user user from 185.196.220.81 port 37162 ssh2
Feb 27 21:32:02 prometheus sshd[28077]: Failed password for root from 218.92.0.243 port 54595 ssh2
Feb 27 21:32:03 prometheus sshd[28083]: Received disconnect from 185.196.220.81 port 37162:11: end [preauth]
Feb 27 21:32:03 prometheus sshd[28083]: Disconnected from invalid user user 185.196.220.81 port 37162 [preauth]
Feb 27 21:32:03 prometheus sshd[28077]: Received disconnect from 218.92.0.243 port 54595:11:  [preauth]
Feb 27 21:32:03 prometheus sshd[28077]: Disconnected from authenticating user root 218.92.0.243 port 54595 [preauth]
Feb 27 21:32:03 prometheus sshd[28077]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=218.92.0.243  user=root
Feb 27 21:32:04 prometheus sshd[28105]: Invalid user Admin from 185.196.220.81 port 37176
Feb 27 21:32:04 prometheus sshd[28105]: pam_unix(sshd:auth): check pass; user unknown
Feb 27 21:32:04 prometheus sshd[28105]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=185.196.220.81
Feb 27 21:32:06 prometheus sshd[28105]: Failed password for invalid user Admin from 185.196.220.81 port 37176 ssh2
Feb 27 21:32:06 prometheus sshd[28105]: Received disconnect from 185.196.220.81 port 37176:11: end [preauth]
Feb 27 21:32:06 prometheus sshd[28105]: Disconnected from invalid user Admin 185.196.220.81 port 37176 [preauth]
Feb 27 21:32:07 prometheus sshd[28110]: Invalid user admin from 185.196.220.81 port 52236
Feb 27 21:32:07 prometheus sshd[28110]: pam_unix(sshd:auth): check pass; user unknown
Feb 27 21:32:07 prometheus sshd[28110]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=185.196.220.81
Feb 27 21:32:09 prometheus sshd[28110]: Failed password for invalid user admin from 185.196.220.81 port 52236 ssh2
Feb 27 21:32:10 prometheus sshd[28110]: Received disconnect from 185.196.220.81 port 52236:11: end [preauth]
Feb 27 21:32:10 prometheus sshd[28110]: Disconnected from invalid user admin 185.196.220.81 port 52236 [preauth]
Feb 27 21:32:11 prometheus sshd[28122]: Invalid user admin from 185.196.220.81 port 52246
Feb 27 21:32:11 prometheus sshd[28122]: pam_unix(sshd:auth): check pass; user unknown
Feb 27 21:32:11 prometheus sshd[28122]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=185.196.220.81
Feb 27 21:32:13 prometheus sshd[28122]: Failed password for invalid user admin from 185.196.220.81 port 52246 ssh2
Feb 27 21:32:14 prometheus sshd[28122]: Received disconnect from 185.196.220.81 port 52246:11: end [preauth]
Feb 27 21:32:14 prometheus sshd[28122]: Disconnected from invalid user admin 185.196.220.81 port 52246 [preauth]
Feb 27 21:32:15 prometheus sshd[28146]: Invalid user user from 185.196.220.81 port 52252
Feb 27 21:32:15 prometheus sshd[28146]: pam_unix(sshd:auth): check pass; user unknown
Feb 27 21:32:15 prometheus sshd[28146]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=185.196.220.81
Feb 27 21:32:17 prometheus sshd[28146]: Failed password for invalid user user from 185.196.220.81 port 52252 ssh2
Feb 27 21:32:19 prometheus sshd[28146]: Received disconnect from 185.196.220.81 port 52252:11: end [preauth]
Feb 27 21:32:19 prometheus sshd[28146]: Disconnected from invalid user user 185.196.220.81 port 52252 [preauth]
Feb 27 21:32:35 prometheus sshd[28310]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=218.92.0.230  user=root
Feb 27 21:32:36 prometheus sshd[28310]: Failed password for root from 218.92.0.230 port 30964 ssh2
Feb 27 21:32:38 prometheus sshd[28310]: Failed password for root from 218.92.0.230 port 30964 ssh2
Feb 27 21:32:43 prometheus sshd[28310]: Failed password for root from 218.92.0.230 port 30964 ssh2
Feb 27 21:32:43 prometheus sshd[28310]: Received disconnect from 218.92.0.230 port 30964:11:  [preauth]
Feb 27 21:32:43 prometheus sshd[28310]: Disconnected from authenticating user root 218.92.0.230 port 30964 [preauth]
Feb 27 21:32:43 prometheus sshd[28310]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=218.92.0.230  user=root

Unsurprisingly, this is a common issue. So much so that there are databases of such IP addresses. For example, the last IP address from the logs above has been reported 100k+ times.. When I brought this up to my colleague Ben, he suggested using fail2ban. Looking into more options, I found endlessh to be a good solution. So here is what I did:

  1. Move the actual SSH service to a non-standard port (the default port for SSH is 22). Any port above 1024 that is not in use will do. Let’s say port 22222.
  2. “Trap” the bots on port 22 by running endlessh on port 22. endlessh is simply a service that accepts SSH connections and slowly sends an endless SSH banner to the client, effectively acting as a “tarpit”.

Here is how I set it up on my workstation:

I. Moving the real SSH service to a non-standard port

First, we need to move the actual SSH daemon to a non-22 port. Let us use 22222.

Step 1: Create a custom config file.

Instead of editing the main /etc/ssh/sshd_config file, it is both cleaner and safer to create a custom “module” config file. We will call it custom_ssh_port.conf.

$ sudo nano /etc/ssh/sshd_config.d/custom_ssh_port.conf

In this file, write a single line with the desired port number:

Port 22222

Step 2: Update the firewall (important!)

Before restarting the SSH service, we must allow the new port through the firewall (UFW or universal firewall).

$ sudo ufw allow 22222/tcp

Step 3: Restart the SSH service

Restart the ssh service to apply the changes.

$ sudo systemctl restart ssh
$ sudo systemctl status ssh

This should return something like:

● ssh.service - OpenBSD Secure Shell server
     Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled)
     Active: active (running) since ...

We can also verify that the port is open and listening using ss:

$ sudo ss -tulpn | grep 22222

This should return something like:

tcp   LISTEN 0      128                        0.0.0.0:22222      0.0.0.0:*                                                
tcp   LISTEN 0      128                           [::]:22222         [::]:*

Step 4: Verify that SSH works on the new port

$ ssh -p 22222 localhost

II. Setting up the SSH “tarpit” with endlessh

Now that the real SSH service is running on a non-standard port and the port 22 is free, we can set up endlessh to run on it.

Step 1: Build endlessh from source

Clone the GitHub repository and build it from source.

$ cd ~/Installations
$ git clone https://github.com/skeeto/endlessh.git
$ cd endlessh
$ make

On Ubuntu-based systems (e.g. Pop!_OS), it might show the following message meaning a dependency is missing:

cc -std=c99 -Wall -Wextra -Wno-missing-field-initializers -Os  -ggdb3 -o endlessh endlessh.c

So, let us install the missing dependency:

sudo apt install libc6-dev

And run make again.

Step 2: Install the built binary

Next, move the built binary to a directory so that it is picked up by the PATH variable.

$ sudo mv endlessh /usr/local/bin/

# Verify that it is accessible.
$ endlessh -V

Step 3: Configure systemd

Next, we copy the provided systemd service file to the appropriate directory, and enable it so that it starts on boot.

$ sudo cp util/endlessh.service /etc/systemd/system
$ sudo systemctl enable endlessh

This should return something like:

Created symlink /etc/systemd/system/multi-user.target.wants/endlessh.service → /etc/systemd/system/endlessh.service.

Step 4: Configure the port for endlessh

We need to configure endlessh to listen on port 22. For this, we create the config file for endlessh.

$ sudo mkdir -p /etc/endlessh
$ sudo nano /etc/endlessh/config

In this file, write the following:

Port 22

Step 5: Start the “tarpit”

$ sudo systemctl start endlessh

And verify that it is running:

$ sudo systemctl status endlessh