Endlessh HTML Scoreboard

Botnets

As most security concious people know, there are many people out there running automated attacks on the whole of the internet in general. Among these many attacks are those pointed at the low-hanging-fruit that is OpenSSH on the default port of 22.

Basic Security

On that note, if you run a server exposed to the internet you really need to take a few basic security measures:

Change your SSH port to a non standard port.
Ensure that root logins are disabled in your sshd_config.
- If root must be enabled, only allow connections with a SSH key.
Use ssh keys whenever possible.
- Disable ssh password login whenever possible.
Use a daemon like Fail2ban or DenyHOSTS to auto ban failed attempts. ^{(I prefer fail2ban)}

Auto Ban Daemon

While some may say that using an auto ban daemon is an intermediate/advanced step, if you are running a server exposed to the internet... you need to do it.

DenyHOSTS

denyhosts is an option, and it is fine if it suits your needs. That being said, it is sooo much more limited.

Fail2ban

fail2ban is amazing.
tl;dr you pick a text logfile, write a regex to match log lines that are Bad™, and set a ban action.
The common way this plays out is: read log -> match bad line -> grab ip -> iptables drop ip.

In addition to some of the provided jails you can enable for ssh, I added another jail that matches misc portscan attempts like

Apr 25 23:45:03 localhost sshd[31130]: Bad protocol version identification 'GET / HTTP/1.1' from 104.168.236.191 port 43262 2019-04-25 23:45:03,796 fail2ban.filter [3963]: INFO [sshd-bad-protocol] Found 104.168.236.191 - 2019-04-25 23:45:03

since I run on a non standard port. ^{(line one is auth.log, line two is fail2ban.log)}

endlessh

endlessh is a fancy tool written by Chris Wellons.

Endlessh is an SSH tarpit that very slowly sends an endless, random SSH banner. It keeps SSH clients locked up for hours or even days at a time. The purpose is to put your real SSH server on another port and then let the script kiddies get stuck in this tarpit instead of bothering a real server.

You can read their write up of endlessh in their original blog post.

I first heard of endlessh a few weeks ago on in a discussion on IRC, and was immediately taken by the idea. After getting it set up, I immediately started getting hits in the log.

endlessh setup

Since I'm not a big fan of systemd I just added a line to my crontab to run with logging enabled:

@reboot             root    /usr/local/sbin/endlessh -p 22 -v >/var/log/endlessh.log 2>/var/log/endlessh.err

Scoreboard

Not very long after I had endlessh running I had a novel idea.... Lets make a Scoreboard from the log!

Main goals:

Use the last seven days.
List connection attempts, min time, max time, average time, and total time of Players.
List geo location of Players

Iteration One: Shell Script

For the sheer novelty of it, I decided I would make a shell script just to say I did. ^{(knowing that I would then need to move to python to add more fancy parts)} After finishing the shell script to generate an html scoreboard, I quickly moved on to a more feature full python version.

I may have spent a little too much time adding a table sort script into the html...

Iteration Two: Python

This iteration went a lot faster as I had already mapped out the objectives and had an idea on the processing. It was also an interesting comparison to see how much easier it was to do with a real programming language, as I replaced stop gap measures with more robust processing. While I'm sure there is much that I can improve with my coding, I'm at least pleased that I chunked everything that made sense to be in individual functions. :)

One of the improvements I was able to add after moving to python was the "Top Player" list.

Here you can see my endlessh_scoreboard.py.

Among the future tweaks I think I will end up doing, I think I will have to move from html only to a fancy html5 CSS table. This is due to the lag that grows and grows the larger the table gets, when attempting to sort. A change over to CSS (maybe using bootstrap?) would mean more columns of the table should be sortable as well.

As for the point of this post, the Scoreboard itself, you can find that at pit.demu.red.

Additional Steps

After all of this there was a little bit of additional system tweaking I did. Namely adding a fail2ban jail for Players who connect to endlessh too many times ^{(1000 times in ~five days earns a month ban)}. And adjusting other fail2ban jails to not block the game's port, which would hobble the game.

fail2ban jail.local:

[endlessh]

enabled     = true
maxretry    = 1000
## One month
bantime     = 2419200
## five days
findtime    = 432000
port        = 0:65535
logpath     = /var/log/endlessh.log

endlessh.conf (jail):

# fail2ban endlessh ban

[Definition]

failregex = ^.* ACCEPT host=::ffff:<HOST> port=.*

#ignoreregex =

Future Goals

Besides a revision to move to a CSS table with sorting, I may end up adding some code to generate a ascii table for gopher.

Endlessh HTML Scoreboard

Fri 26 April 2019 SSH SSH / endlessh / score / fail2ban / python / html