Server Hardening: Ubuntu 22.04

Yes, i know. There are tons of script for free or for a fee that can automatize all this.

But it’s my honest opinion that someone should know at least the basics. So, brace yourself cause this is gonna be a long ass post.

Premise

When comes to security there are some key factors that should never be forgotten. Wich are:

• SECURITY IS NOT SOMETHING YOU BUY, IS SOMETHING YOU DO.
• IS NOT POSSIBLE TO AVOID BEING HACKED (but we can make things hard for them, at least)

Physical Security

Where your server is located plays a very important role because that is a Data Center, your home, your mom basements or whatever you need to be sure that all the recommended precautions are taken for avoid unwanted persons entering the perimeter. Now, i’m not gonna cover that, but rather what can you do for make things harder to those who already are in front of your favourite server:

• Update the BIOS in order to avoid booting from external hard drive and cd-rom (yes, they still exists)
• Abilitate password to BIOS and GRUB (and use a strong password for fucksacke)

System Update

I know, when you are in this field one of your main objective is to “crystallize” your system for all the time to come because whatever you have done is making everything works. Agree, but time is changed and we can’t allow anymore a lax environment when comes to security. So, ALWAYS FUCKIN APPLY SYSTEM UPDATES. They come with important security patch ! Here i’m gonna offer 2 different approaches: 1 with the normal command update/upgrade and the other with unattended-updates. The difference you ask ? One gives you more control on what you want to install and other automatically installs everything without asking.

$ sudo apt-get update # Update the package list from repositories
$ sudo apt-get upgrade # Upgrades all packages to their latest version
$ # Alternative to use AFTER using the update command
$ sudo apt list --upgradable # This will show a list of packets ready to be upgraded.
$ sudo apt install --only-upgrade packet_name # With this command you tell to upgrade ONLY the specified packet

$ which unattended-upgrades # This should return something like /usr/bin/unattended-upgrades
$ sudo apt-get install unattended-upgrades # use this only if the command above gave no result
$ sudo dpkg-reconfigure unattended-upgrades # Now should get a dialogue box that prompts to accept automatic download and installation of stable updates

VERY IMPORTANT NOTE – README:When using unattended-updates you may encounter situations that are not properly conscious when comes to production environment. This is because unattended-updates may restart some services that you are using causing an inevitable down and not only.

User Management

There are tons of quotes when comes to power, privilege and so on. One of my favourite is

There is no liberty if the power of judging is not separated from the legislative and the executive.

Montesquieu

Sorry, i wanted to show off my knowledge 😛
Anyway.When comes to servers is always a good idea to have a certain granularity regarding permissions and user in general. But this can be a very wider argument. So, here’s the most basic action that i take/advice:

$ dpkg-statoverride -update -add root sudo 4750 /bin/su # limit the use of su command to the sudoers
$ passwd -l root # Disable the fuckin root account. Is always him the target of brute-force/list attack
$ passwd -u root # In case you run the command above and you realized you can't live without root

It hurt me to specify this…but….DON’T FORGET TO CREATE AN OTHER USER TO REPLACE ROOT

Shared Memory

Shared Memory is a good surface of attack for the hacker who gained access to the server. Let’s make their life harder:

$ nano /etc/fstab # Or use vim instead of nano if you feel confident

And add the following line

tmpfs     /run/shm    tmpfs	defaults,noexec,nosuid	0	0

Temp Folder

The temp folder is one of my favourite cause after a successful attack where i gain access to the system, I use this folder to run/execute/download any kind of code and use it.

So. let’s make my life miserable and give me less “space of attack”

# Create a filesystem of 1GB for the partition /tmp
$ dd if=/dev/zero of=/usr/tmpDSK bs=1024 count=1024000

# Create a backup of the current tmp folder
$ cp -Rpfv /tmp /tmpbck

# Let's mount the new tmp partition and set the right permission
$ mount -t tmpfs -o loop,noexec,nosuid,rw /usr/tmpDSK /tmp
$ chmod 1777 /tmp

# Let's restore the backup and delete it
$ cp -Rpf /tmpbck/* /tmp/
$ rm -rf /tmpbck/*

# Set up tmp into fstab (nano /etc/fstab)
# Add this line -> /usr/tmpDSK /tmp tmpfs loop,nosuid,noexec,rw 0 0

# Let's test it
$ mount -o remount /tmp

Compiler

As you may know or not…compilers are used to compile software. Since we are talking of a production server there should be not any reason why you need an active compiler. So let’s make it harder for an attacker to build something “malevolent” in case they gain access.

$ chmod 000 /usr/bin/byacc
$ chmod 000 /usr/bin/yacc
$ chmod 000 /usr/bin/bcc
$ chmod 000 /usr/bin/kgcc
$ chmod 000 /usr/bin/cc
$ chmod 000 /usr/bin/gcc
$ chmod 000 /usr/bin/*c++
$ chmod 000 /usr/bin/*g++

IP Security

Please, add/update the following lines in your /etc/sysctl.conf

# Ignore ICMP broadcast requests
net.ipv4.icmp_echo_ignore_broadcasts = 1

# Disable source packet routing
net.ipv4.conf.all.accept_source_route = 0
net.ipv6.conf.all.accept_source_route = 0 
net.ipv4.conf.default.accept_source_route = 0
net.ipv6.conf.default.accept_source_route = 0

# Ignore send redirects
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0

# Block SYN attacks
net.ipv4.tcp_max_syn_backlog = 2048
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syn_retries = 5

# Log Martians
net.ipv4.conf.all.log_martians = 1
net.ipv4.icmp_ignore_bogus_error_responses = 1

# Ignore ICMP redirects
net.ipv4.conf.all.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0 
net.ipv6.conf.default.accept_redirects = 0

# Ignore Directed pings
net.ipv4.icmp_echo_ignore_all = 1

# Disable ipv6 (this is optional)
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1

$ sysctl -p # Run this command to reload so the new configuration can take place

Port Scanning

The first step of an attack is to recognize the active services of a server. The most used tool is nmap that have a lot of cool features that im not gonna explain now.

So let’s see how to ban for 1 day whoever try a port scan towards our server and save this rule:

$ iptables -A INPUT   -m recent --name portscan --rcheck --seconds 86400 -j DROP
$ iptables -A FORWARD -m recent --name portscan --rcheck --seconds 86400 -j DROP
$ iptables-save > /home/my_user/my.active.firewall.rules

If you didn’t believe the effect of the above commands and you banned yourself, here how to clean the iptables:

$ iptables -A INPUT   -m recent --name portscan --remove
$ iptables -A FORWARD -m recent --name portscan --remove

SYN Check

SYN packets are normally generated when a client attempts to start a TCP connection to a server, and the client and server exchange a series of messages, which normally is ok. Problem is that this SYN packet are commonly used for DDoS purposes. So, fuck this SYN

$ iptables -A INPUT -p tcp ! --syn -m state --state NEW -j DROP

Null Packets: DROP EM !

Null packets are essentially empty or zero-sized packets that may be used for various purposes, including network scanning and attacks. So, let’s drop em !

$ iptables -A INPUT -p tcp --tcp-flags ALL NONE -j DROP

No PING please

Well, this might be controversial but i love this because:

• Provide security through obscurity: the attacker 9/10 wont bother to go further
• It reduce the attack surface. We are telling the server to not reply at ICMP echo requests
• Prevents Ping Floods. (google around for ICMP-based attacks)

Note: MY_NET_INTERFACE = your server network card.
Run ip a or ifconfig to know whats your network interface

$ iptables -A OUTPUT -p icmp -o MY_NET_INTERFACE -j ACCEPT          
$ iptables -A INPUT -p icmp --icmp-type echo-reply -s 0/0 -i MY_NET_INTERFACE -j ACCEPT     
$ iptables -A INPUT -p icmp --icmp-type destination-unreachable -s 0/0 -i MY_NET_INTERFACE -j ACCEPT  
$ iptables -A INPUT -p icmp --icmp-type time-exceeded -s 0/0 -i MY_NET_INTERFACE -j ACCEPT       
$ iptables -A INPUT -p icmp -i MY_NET_INTERFACE -j DROP 

Do not forget…

If you followed this post to the T and you got at this point launching all the commands blindly (thank you for the trust by the way) you are now at a point where you need to save all the various iptables commands we just gave (otherwise at first re-boot you have to do it all again)

$ iptables-save > /home/my_user/my.active.firewall.rules

SSH – Base Configuration

Here some tips & tricks about securing this fundamental service.
First of all, it’s default configuration file is at /etc/sshd/sshd_config
This file is extremely easy to understand and update. Don’t let me treat you like a child. C’mon.

Now, if you read that file, you may have noticed some “bad” default values that you want to modify/update, like:

• AllowUsers – Normally ssh is allowed to any system user, thats not properly a good idea. So, let’s define the list of users allowed to gain ssh access: AllowUsers user_name1 user_name2 user_name3
• PermitRootLogin – No, you don’t want this, refer to the chapter above on User Management. So this become: PermitRootLogin no
• PermitEmptyPasswords – Do i really have to explain this ? PermitEmptyPasswords no
• PermitUserEnvironment – Prevent that user could bypass some access restriction. PermitUserEnvironment no

SSH – Port Knocking

It involves a series of connection attempts or “knocks” to a predefined sequence of closed ports on a server. Once the correct sequence of knocks is detected, the firewall rules are dynamically modified to open a specific port, allowing access from the source IP that performed the knocking. Pretty cool, right ? Don’t worry it’s very easy (now, at begin with iptables was a pain in the ass)

$ sudo apt install knockd # Yes, that's all...joking, we have to configure the bastard
$ nano /etc/knockd.conf

Opening nano (or vim) with the knockd config file you can realize by yourself that is pretty easy and straight forward. So i won’t bother telling you what do exactly. Bur rather i’m gonna tell you this:

• Change the sequence ! Those are default ports ! Don’t be an idiot by default
• As logging default mechanism it use SysLog, you can change this by removing UseSysLog and put logfile = /var/log/my_knockd.log

Ended with the main config file, let’s go modify the second config file: /etc/default/knockd

Here you just have to update 2 values:

• START_KNOCKD = 1
• KNOCKD_OPTS="-i MY_NET_INTERFACE"

To know your network interface name just run ip a in your terminal

And now, the last final commands and we are all set:

$ sudo systemctl enable knockd && sudo systemctl start knockd

SSH – 2FA

Adding a 2FA to access your server is very uncommon, but hey…i’m a paranoid bastard.

Let’s start unstalling what we need:

$ sudo apt install libpam-google-authenticator

Now let’s tell SSH to use Google Authenticator and run google-authenticator:

$ nano /etc/pam.d/sshd

auth required pam_google_authenticator.so

$ sudo systemctl restart sshd.service
$ nano /etc/ssh/sshd_config
ChallengeResponseAuthentication yes
$ google-authenticator
# Make token time-base: YES
# Update the .google_authenticator file: YES
# Disallow multiple uses: YES
# Increase the original generation time limit: NO
# Enable rate-limiting: YES

Yes, i know…i hate google too. But hey, this is effective and free !

Fail2Ban

This is one of my favourite tools. It scans log files and ban all the IP that show any sign of malicious activity. To install it it’s pretty easy and don’t require any particular configuration (but you are more then welcome to do so ! )

$ sudo apt install fail2ban

If i did correctly my jedi trick you are now watching the following file  /etc/fail2ban/jail.conf

No ? Well…do it !

Information Gathering

Use these words in this context might sound strange. But it’s actually correct.

Most of the time people ask me where to look in case of an attack. Here some tips.

In case of DDoS/DoS you can run the following command in order to have more informations and act:

$ netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

Her some logs that require a particular attention:

• /var/log/messages – The main system logs or current activity logs are available.
• /var/log/auth.log – Authentication logs
• /var/log/kern.log – Kernel logs
• /var/log/cron.log – Crond logs (cron job)
• /var/log/maillog – Mail server logs
• /var/log/boot.log – System boot log
• /var/log/secure – Authentication log
• /var/log/ufw.log – Firewall log
• /var/log/utmp or /var/log/wtmp – Login records file.

I know i know, they are way to many (actually i omitted fews) but thanks the lord almighty we have a lot of tools that can scan all this logs and make a nice report

$ apt-get install logwatch libdate-manip-perl

To have a preview of what you have, run:

$ logwatch | less

Please, take a minute to familiarise with logwatch config file, take a look at: /etc/log.d/conf/logwatch.conf

E-Mail Notification

If you didn’t bother at all applying all that has been described…well…ok…At least implement this script in order to have an e-mail everytime there is a successful login on server shell:

$ nano .bash_profile
# at the end of the file add the following

echo 'ALERT - ACCESS ON:' `date` `who` | mail -s "ALERT - ACCESS GRANTED from `who | awk '{print $6}'`" my@email.com

HIDS

Host Intrusion Detection System are a valid tool that should accompany you along the way of IT Security. My favourite one (and no, they don’t pay me nor they know i exist) is OSSEC. It offers open source solution and on payment too.

Missing Something

If you have any question, idea or noticed that something is wrong or missing PLEASE don’t hesitate to contact me.