How to Secure Your Linux Server: The Complete Guide

Linux server hardening means fortifying and securing a Linux server in order to protect it from vulnerabilities and threats. While total security will always remain a moving target in the security arms race, this article explores some important fundamental steps on how to secure your Linux server.

No security risk is too small to ignore

Unfortunately, even the smallest vulnerabilities can transform into serious security threats. It’s important to remember that any code running on your system could potentially contain flaws that allow intruders to access a command shell or legitimate unprivileged users to elevate their access to superuser rights. No matter how low the level of risk may be, it’s always best to address minor security risks before they become major cybersecurity threats by following an established server hardening process. With Linux server hardening, IT teams can fortify their defenses before any serious threats occur, which will help prevent any major issues from occurring in the first place. 

System updates and security patches

Regularly updating your Linux server is essential for maintaining its security and overall health, especially weak security controls and practices are routinely exploited by threat actors, according to the Cybersecurity & Infrastructure Security Agency (CISA). Software and operating systems are constantly improved and refined to address vulnerabilities and weaknesses. By applying updates promptly, you ensure that known security flaws are patched, reducing the risk of exploitation by cyber threats. Moreover, updates often include bug fixes and performance enhancements, leading to a more stable and efficient server environment. Failing to update regularly can expose the system to potential attacks and compromise its integrity.

The risks of unpatched software vulnerabilities include:

• Malware: Malicious code, such as ransomware, is one of the top cybersecurity threats. According to Norton, global cybercrime is expected to grow by 15% annually, reaching $10.5 trillion by 2025.
• Data Breaches: IBM’s Cost of Data Breach Report 2023 states that the global average cost of a data breach for that year was $4.45 million. Experts have stated that data breaches are one of the leading causes of security vulnerabilities in an IT network.
• Compliance issues: Regular patching keeps you compliant with various regulatory requirements, including PCI-DSS, HIPAA, and GDPR.
• Reduced productivity: Unpatched software could impact operational efficiency, in addition to exposing your organization to cyber threats. Patching not only addresses security vulnerabilities but also fixes bugs that could impact performance.

Choosing the right patching strategies and best practices for your server patch management is crucial for maintaining a secure Linux server. Current enterprise-grade Linux distributions generally all have automated patch management services of some kind, whether by paid subscription service or community efforts. The paid services often go beyond simply patching your system files for security; some of these services also include a solution to live-patch your running kernel, allowing your running systems to stay patched before you even need to reboot them. 

Introduction to package managers

Package managers differ across distributions but tend towards using two main package formats: DEB and RPM. DEB is used on Debian-based systems and is managed by apt; RPM stands for Redhat Package Manager and is managed through dnf or yum for Red Hat/Enterprise Linux-based systems or zypper for SUSE. All package managers facilitate the installation, update, and removal of software packages, in addition to resolving dependencies. Utilize package managers to manage software effectively and streamline the update process. All major package managers are also supported by remote repositories, allowing you to centralize control over updates and software rollouts.

Configuring automatic updates for critical security patches

Configuring your Linux server to automatically install critical security patches ensures that high-priority updates are applied promptly without manual intervention, minimizing the window of vulnerability and reducing the burden on administrators. Enterprise distributions typically allow this to be configured during installation. See the configuration packages below, however, should you wish to set this up on your existing system:

Ubuntu: sudo apt install unattended-upgrades

Red Hat/CentOS: sudo yum install dnf-automatic

SUSE: sudo zypper install yast2-online-update-configuration ->  open YaST, select Software -> Online Update Configuration, and configure to taste.

Exercise caution, however, when enabling automatic updates; these may impact system stability or require compatibility testing. Testing is especially important when using mass remote admin tools such as ansible or puppet to do server updates in a heterogeneous computing environment.

Configuring sudo elevation and shell access

Correctly configuring your user accounts and permissions is essential to having a secure system that still allows the right access. Permissions that were not configured properly can create some issues, so it’s best to ensure that they are set up correctly from the start.

Using the sudo command to its full potential with the right /etc/sudoers configuration lets you limit and permit passwordless root access rights on a per-app basis. Always use the visudo command to edit /etc/sudoers as this maintains the correct permissions on that critical file:

sudo visudo

Make changes similar to these, replacing ninja with your username and adjusting the comma-separated list of allowed commands to suit your use case:

someuser ALL=(ALL:ALL) ALL

to

someuser ALL = NOPASSWD: /bin/systemctl restart nginx, /bin/kill, /usr/sbin/reboot

Then save and exit. Note that sudo effectively reads the /etc/sudoers file from the bottom upwards, so if your changes don’t seem to be effective, try moving your edited line to be near the bottom of the sudoers file.

A word on shells

You can set a user’s default shell by running the chsh command as root, or en masse by editing the /etc/passwd file as root. This is useful and desirable in some cases, such as changing an FTP-only account’s default shell to /sbin/nologin — since such an account should never need to run any terminal commands. Removing shell access in such a case would shrink the system’s potential attack surface.

Securing SSH access with keys and disabling root login

Secure Shell (SSH) is an industry-standard network protocol that allows command-line level remote access to a system. It is a firmly held belief among tech folk that shell access should by design be equivalent to “as if you’re local”, so it’s crucial to limit SSH access to necessary users only, and also to make sure only the right users can do things as root.

Best practices for securing SSH access to Linux servers

It is generally a good idea to entirely disable both root login via SSH and password-based SSH access. While the password text in password-based authentication is transmitted in encrypted form by the SSH host keys that get exchanged during the pre-login SSH client/server handshake, the password is nevertheless transmitted across multiple networks in some fashion. This is another attack surface aspect— If your SSH server’s host keys were generated using an older/insecure algorithm, for instance, then your passwords can be snooped.

SSH key logins are not vulnerable to this sort of attack; your public keys are the only parts that get transmitted in “plaintext”, and they were designed for public distribution. As with your private key itself, the passphrase you use to unlock it during SSH key login is not transmitted over the network.

Use SSH keys for authentication

Note that the code snippets below are annotated with the author’s comments indicated by “##” for instruction’s sake.

1. Generate an SSH key using the ssh-keygen command. By default, the industry-standard RSA encryption key format is used, but it is recommended to use the faster and more secure ed25519 elliptic-curve encryption standard.
   
   

   user@host:~> ssh-keygen -t ed25519   
   Generating public/private ed25519 key pair. Enter file in which to save the key (/home/user/.ssh/id_ed25519): 
   Enter passphrase (empty for no passphrase):
   Enter same passphrase again:
   Your identification has been saved in /home/user/.ssh/id_ed25519
   Your public key has been saved in /home/user/.ssh/id_ed25519.pub
   The key fingerprint is: SHA256:NjPk3GbSIL6wdITiSrEQ3U9uXqqaYJdtS62cwCahKoM user@host
   The key's randomart image is:
   +--[ED25519 256]--+
   |.. . |
   | .. ... |
   |... .+o o |
   |..o. o+=.+ |
   | +. ooooS = |
   |o.o.o++o B |
   |=o *.=.. |
   |E.+.* + |
   |o.o. = |
   +----[SHA256]-----+ 
   user@host:~>

2. Copy your public key to your online hosting platform. If you still have password access to the server in question, you can use the ssh-copy-id command to automagically set up your remote account SSH config files with your newly generated SSH key copied into the right places:
   
   

   ssh-copy-id -i ~/.ssh/id_ed25519.pub remoteuser@remotehost

3. Ways of Using SSH Keys to Login:A) Directly in the command line:
   

   ssh -i ~/.ssh/id_ed25519 remoteuser@remotehost:port

   OR by using the ~/.ssh/config file:

4. Edit your file ~/.ssh/config with any text editor. Do this as the account/user who will be using the SSH config to connect remotely.
5.  Add stanzas in the following format:
   
   

   Host server1   ## Arbitrary label/handle for easy usage
   Hostname server1.example.com   ## Real hostname/IP
   User remote_user_1  ## Remote Linux username
   Port 12345  ## Remote port, default 22
   IdentityFile ~/.ssh/id_ed25519  ## NB! Private key file
   ForwardX11 yes ## Optional, forwards remote GUI apps
   
   Host server2  
   Hostname server2.example.com
   User remote_user_2
   Port 34567 
   IdentityFile ~/.ssh/id_ed25519

   
   There should be vertical whitespace between stanzas.

6. Now you can SSH simply by typing:
   
   

   ssh server1
   

   Since everything else is specified under that stanza label in your .ssh/config file. Protip — if you’ve embraced Linux Tab-completion (aka bash completion), you’ll be happy to know that your SSH config file automatically plugs into most modern Linux distros’ bash-completion systems.

Disable SSH root login and password authentication

1. Make ABSOLUTELY SURE you can login in with your SSH key. You can potentially lock yourself out of your system if this isn’t working properly.
2. Use your favorite editor as root to change the following lines in  /etc/ssh/sshd_config to the following settings:
   
   

   PermitRootLogin no
   
   ChallengeResponseAuthentication no
   
   PasswordAuthentication no
   
   KbdInteractiveAuthentication no

3. Optionally make your SSH server run on a non-standard port:
   
   

   Port 12345

4. Save this file and reload your SSH service:
   
   

   sudo systemctl restart sshd

5. Now try to log in via SSH without the SSH key:
   
   

   ssh remote_user_1@server1 [-p port]
   

   If you cannot log in without specifying an SSH key, you’ve done this step correctly.

Implementing a firewall on Linux servers

If you’re operating a production Linux server and your distribution comes with a pre-configured firewall, then you’re off to a great start. Whenever data from a production server will be exposed to the internet, it’s best to protect it by implementing a firewall. Enterprise Linux/Red Hat variants tend to ship with firewalld, whereas Debian/Ubuntu ships with ufw (“Uncomplicated Firewall”). Check out man firewall-cmd or man ufw for details.

Configuring firewall rules and policies

Firewall rules and policies, once you get past the details, should really boil down to “block everything except what I specifically allow” — again, these are generally preconfigured to sane defaults on modern Linux systems, especially on Enterprise Linux variants. Any use cases not covered by these sane defaults fall outside the scope of this document.

Monitoring and managing the firewall

• Both firewalld and ufw plug into the major logging systems journald and/or [r]syslog.
• Aside from the command-line firewall management tools firewall-cmd and ufw,  other remote management frontends like Cockpit, WebMin, and YaST offer modules which allow for firewall management. Other remote management tools like ansible and puppet also have modules for automatic firewall configuration and management.

Securing network-facing services

Best practices for securing network-facing services:

• Have as little code on your system as possible.
• Run the absolute minimum possible amount of services.
• Don’t run a public-facing service as root. 
• Servers do not need a GUI. Extraneous code and running services = increased attack surface.
• Linux services rely on your securing the system with encryption certificates. Certbot/Let’s Encrypt has the easiest process to generate free encryption certificates.
• Most Linux services listen on separate ports for their secure and insecure connections; see the relevant services’ man pages for details on security and other configuration options.
• You can use the command netstat -tulpen to see the ports your system is listening on (whether or not those ports are blocked by your firewall; this shows listening ports on a per-process basis.) 

Install only necessary third-party packages

Ensure that you limit the number of third-party packages or applications you install on your Linux server. As with any download, practice good cybersecurity hygiene habits to reduce the risk of threats and keep your server as healthy as possible.

A simple yet powerful tip is to install server packages from reputable sources and those that will be used regularly. The last point needs to be emphasized! Even if a package comes from a trusted vendor, it is still prone to risk if it is not regularly used (not to mention that it takes up space in your server).

Use fail2ban

fail2ban is a self-regulating security utility for Linux that automatically checks your server for any automated digital attacks and blocks them. fail2ban can be easily configured by:

1. Type the command: $ sudo apt install fail2ban – y
2. Copy this configuration file: $ sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
3. Restart fail2ban to ensure it works properly: $ sudo service fail2ban restart

You can then configure the utility as you prefer. Regardless, fail2ban alerts you of any multiple login attempts on your remote servers and then block these addresses. Over time, you can examine log files to see a full list of banned IT addresses.

Implement two-factor authentication

Two-factor authentication (2FA) is a must for securing your Linux servers. Essentially, it is a security system that verifies users twice before they are given access to a system or account.

You can set up 2FA on your Linux server by first installing an authentication package. Depending on which package you choose, the additional layer of security can be anywhere from a security key to biometric authentication factors.

Work with a trusted endpoint management company

You can always manually check your Linux server for any vulnerabilities. Still, it may be more cost-effective to work with a trusted endpoint management company like Ninjaone to monitor, manage, and support your Linux devices from a single pane of glass.

Minimalism is as important to Linux server security as action

No amount of actions after the fact can replace making good choices from the start — in today’s cloud-connected world, one’s choices of Linux server distribution and remote management solutions matter more than ever. NinjaOne’s embedded Linux remote management software gives you the freedom to use any endpoint OS platform while remaining secure and patched.

It would be safe to say that the majority of SME companies’ tech infrastructure includes some Linux servers as part of their critical infrastructure. If you have a website, chances are that it’s running on a Linux server; if you use online banking, odds are it depends on the security of dozens of Linux servers. Given that even Android itself is (grossly oversimplified) a Linux kernel with some GUI stuff on top if you’re reading this on a smartphone, then the issue of Linux security may literally be closer to your heart than you might think.