• Published:
  • Updated:
  • Written by:

CISS.2024.hardened.webstack

Overview

A) Preliminary remarks

The discussed CISS.2024.hardened.webstack script is a digitally signed, self-verifying shell script for setting up an all-inclusive LAMP environment that relies on the most secure ciphers and follows the latest best practices in webserver hardening to achieve a quadruple A+ rating at

incl. HSTS Preload preparation and HTTP/3 and QUIC support. Letsencrypt CA has been replaced by the Norwegian CA buypass.no, which supports 180-day free TLS certificates.

The script will configure nginx to use a php upstream handler to forward incoming http requests by default. But you can easily choose this setup to use it as a reverse proxy by simply changing the location blocks to forward the traffic to a local port for example for some docker containers.

A complete series of detailed discussions of all the settings implemented against the background of best practices and the latest recommendations of the IT security community will be started to give you a deep dive into the matter.

1) Features of the script

  • The script and all modules and assets downloaded by my scripts are cryptographically signed with my private OpenPGP key, see the public key fingerprint here, which resides in an HSM. All signing operations are performed on an air-gapped system. Help me move to the next level of security: a room-gapped signing environment. All cryptographic verification operations can be disabled for easy and convenient script editing.
  • Self verification of all downloaded assets. Can be disabled for easy and convenient script editing.
  • If there are any errors in the verification process of cryptographically signed scripts, assets and modules, the script will automatically fail immediately.
  • Advanced error trap function. Returns exit code, line number and module name both on screen and in the error log file.
  • Detailed logs available at /root/webstack/logs, including logging of user input (without logging of any credentials to be clear) and coloured log file.
  • All files touched by the script are backed up to .bak. No files are overwritten: see bash environment settings below.
  • Select the trust level of the signing key I have provided.
  • For security reasons, the bash history is also deleted after the setup routine.
  • Take advantage of the following bash error handling options:
set -o errexit   # Exit if a command fails.
set -o nounset   # Exit if an unset variable is used.
set -o pipefail  # Exit if a pipeline fails.
set -o noclobber # Prevent output redirection ">", ">&", "<>" from overwriting existing files.
set +o history   # Temporarily turn off history, to avoid sensitive information leakage.

2) Last test runs

Script VersionProviderDateNotes
V4.8.128Netcup Root G1110.07.2024E) 1) 2) 3) 4) 5) 6)
  • E) Locale en_US.UTF-8 tested.
  • G) Locale de_DE.UTF-8 tested.
  • 1) acme.sh succesfully retrieved domain.tld buypass.no cert.
  • 2) acme.sh succesfully retrieved phpmyadmin.domain.tld buypass.no cert
  • 3) Docker, Docker Compose installation succesful.
  • 4) nginx auth installation succesful.
  • 5) PHPMyAdmin installation succesful.
  • 6) Updated fail2ban jails: checked.

3) Features

  • This script will retrieve all RSA-4096 bit leaf certificates for your-domain.tld and phpmyadmin.your-domain.tld with a validity of 180 days.
  • nginx setup – only the strongest TLS-1.2 and TLS-1.3 ciphers are selected, no DHE ciphers are supported:
    • ECDHE-ECDSA-AES256-GCM-SHA384
    • ECDHE-ECDSA-CHACHA20-POLY1305
    • ECDHE-RSA-AES256-GCM-SHA384
    • ECDHE-RSA-CHACHA20-POLY1305.
  • nginx setup – only the strongest elliptic curves are chosen:
    • X448
    • secp521r1
    • secp384r1
  • nginx setup – all major http headers are pre-selected to ensure maximum security against common web attacks.
    • add_header Cross-Origin-Embedder-Policy "require-corp";
    • add_header Cross-Origin-Opener-Policy "same-origin";
    • add_header Cross-Origin-Resource-Policy "same-origin";
    • add_header Expect-CT "max-age=86400, enforce";
    • add_header Permissions-Policy "interest-cohort=()";
    • add_header Referrer-Policy "same-origin" always;
    • add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload";
    • add_header X-Content-Type-Options "nosniff";
    • add_header X-Frame-Options "SAMEORIGIN";
    • add_header X-Xss-Protection "1; mode=block";
    • add_header Content-Security-Policy "default-src 'none'; connect-src 'self'; font-src 'self' data:; form-action 'self'; frame-src 'self'; frame-ancestors 'none'; img-src 'self' data:; script-src 'self'; style-src 'self' 'unsafe-inline'; base-uri 'none'; block-all-mixed-content;";
  • Hardened nginx fail2ban: including the following jails and preconfigured bantime of 1 day.
    • http-auth
    • bad-request
    • botsearch
    • directory-listing
  • Pre-configured http-auth setup and default enable when PHPMyAdmin setup is selected.
  • Hardened nginx service via /etc/systemd/system/nginx.service.d/override.conf:
    • KillMode=mixed
    • AmbientCapabilities=CAP_NET_BIND_SERVICE
    • ProtectSystem=true
    • ProtectHome=true
    • SystemCallFilter=@system-service
    • NoNewPrivileges=true
    • ProtectKernelTunables=true
    • ProtectKernelModules=true
    • ProtectKernelLogs=true
    • ProtectControlGroups=true
    • RestrictSUIDSGID=true
    • KeyringMode=private
    • ProtectClock=true
    • RestrictRealtime=true
    • PrivateDevices=true
    • PrivateTmp=true
    • PrivateIPC=true
    • ProtectHostname=true
    • ProtectProc=invisible
    • ProtectSystem=true
    • CapabilityBoundingSet=~CAP_BLOCK_SUSPEND
    • CapabilityBoundingSet=~CAP_LEASE
    • CapabilityBoundingSet=~CAP_SYS_PACCT
    • CapabilityBoundingSet=~CAP_SYS_TTY_CONFIG
  • Hardened PHPMyAdmin installation if you choose to install it.
    • /* Custom Security Configuration */
    • $cfg['TempDir'] = '/var/lib/phpmyadmin/tmp';
    • $cfg['LoginCookieValidity'] = 1440;
    • $cfg['LoginCookieRecall'] = false;
    • $cfg['AllowUserDropDatabase'] = false;
    • $cfg['ShowServerInfo'] = true;
    • $cfg['ForceSSL'] = true;
  • PHP settings are memory optimized.
  • The following PHP extensions are installed:
    • apcu
    • bcmath
    • bz2
    • cli
    • common
    • ctype
    • curl
    • fileinfo
    • fpm
    • gd
    • gmp
    • ldap
    • iconv
    • igbinary
    • imagick
    • intl
    • mbstring
    • mysql
    • opcache
    • pdo
    • readline
    • redis
    • smbclient
    • tokenizer
    • xml
    • zip
  • Redis listens only on localhost and REDISPASS is installed by default.
  • The following components will be installed:
a) Online audit resutls

Once installed, the web server will receive a quadruple A+ rating from the following online audit tools:

AuditorResult
Qualys. SSL Labs
mozilla Observatory
Security Headers
immuniweb SSL Security Test

Please check out Online Audits for more online tests.

B) Main Installation

1) Prerequisites

For an apparent setup procedure, it is recommended to install this script on a freshly hardened Debian Bookworm server according to this CISS.2024.vps.bookworm.hardening Tutorial.

  • Root Server (recommended: netcup Root VPS | Hetzner shared VPS | Hetzner dedicated VPS | own Proxmox installation).
  • Fully hardened Debian Bookworm installation (recommended: CISS.2024.vps.bookworm.hardening).
  • fail2ban and ufw installation (recommended).
  • Fully configured FQDN already pointing to the IPv4 and IPv6 of the server in question.
  • Make sure that your-domain.tld and if PHPMyAdmin setup is selected the domain phpmyadmin.your-domain.tld is set up properly.
  • Do not forget (e.g. Hetzner Firewall) to open 80/tcp, 443/tcp and 443/udp in your firewall. Otherwise buypass.no CA could not proof the ownership of your-domain.tld and if selected phpmyadmin.your-domain.tld.
  • The port 443/udp is needed for Quic protocol, so keep in mind to open this protocol in your firewall as well.
  • DNSSEC secured domain is recommended.
  • reverseDNS setup properly, especially if you want to set up an MTA.

Feel free to prepare /root/server-scripts/webstack.conf in advance before editing all the variables on the CLI using nano editor.

mkdir /root/server-scripts
cd /root/server-scripts
wget --https-only https://cendev.eu/marc.weidner/CISS.2024.hardened.webstack/raw/branch/main/webstack.conf

Edit /root/server-scripts/webstack.conf to suit your needs.

###########################################################################################
# Debian Bookworm Hardening Webstack Setup Script Conf File           V4.8.128.2024.07.10 #
###########################################################################################
# Copyright (c) 2019 - 2024, Marc Weidner, Centurion Intelligence Consulting Agency       #
# https://coresecret.eu/                                                                  #
# Licensed under the EUROPEAN UNION PUBLIC LICENCE v. 1.2 https://eupl.eu/1.2/en/         #
###########################################################################################
# https://keys.openpgp.org/vks/v1/by-fingerprint/A6D46A56AE17A185AB0F6DB77095A8A13CBE0FA3 #
# Fingerprint A6D4 6A56 AE17 A185 AB0F 6DB7 7095 A8A1 3CBE 0FA3 ## valid till: 01.01.2031 #
###########################################################################################
# Please change the variables according to your needs                                     #
###########################################################################################

ACMEEMAIL="acme.client@domain.tld"
CURRENTTIMEZONE="Europe/Lisbon"
DOCKERCOMPOSEVERS="v2.28.1"
DOCKERINST="Yes"
DOCKERUSER="dockeruser"
HOSTNAME="FQDN"
IPV4="123.123.123.123"
IPV6="AAAA::FFFF"
MARIADBVERS="11.4"
NGINXAUTHUSER="security"
PHPMAINST="Yes"
PHPMAUSER="phpmauser"
PHPMAVERS="5.2.1"
PHPVERS="8.3"
SELFCHECK="Yes"
TRUSTKEY="Yes"
TRUSTLEVEL="5"
VPSPROVIDER="Netcup"

###########################################################################################
# ! No changes below this line !                                                          #
###########################################################################################
CONFVERS="V4.8.128.2024.07.10"
MARIADBROOTPASSWORD=$(pwgen -n 32 -B -c -s -1)
NGINXAUTHPASS=$(pwgen -n 32 -B -c -s -1)
PHPMABLOWFISHSECRET=$(pwgen -n 32 -B -c -s -1)
PHPMAPMAPASSWD=$(pwgen -n 32 -B -c -s -1)
PHPMAUSERPASSWD=$(pwgen -n 32 -B -c -s -1)
PHPTIMEZONE=$(sed 's|/|\\/|g' <<< $CURRENTTIMEZONE)
REDISPASS=$(pwgen -n 32 -B -c -s -1)
SLEEPTIMER="1"
a) webstack.conf values

All variables must be entered in double quotes.

ACMEEMAIL

Please specify the email to setup the CA account via acme.sh.

CURRENTTIMEZONE

Allowed values are:
"Europe/Lisbon" or any other specified Timezone (TZ) identifier: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones

DOCKERCOMPOSEVERS

Please specify the Docker Compose version to install.

DOCKERINST

Please select if youwant to install Docker and Docker Compose. Allowed values are:
"Yes" or "No"

DOCKERUSER

Please choose the username that should be added to the ‚docker‘ group.

HOSTNAME

Please specify the desired FQDN ot the server. Allowed values are:
any valid FQDN: https://en.wikipedia.org/wiki/Fully_qualified_domain_name

IPV4

Please provide the IPv4 of the server in question.

IPV6

As a good practice the following IPv6 convention is used and suggested:
Consider your IPv4 to be: 123.123.123.123
Consider your IPv6 Prefix to be: 2a10:ABCD:1111:FFFF::1/64
Then the server IPv6 is as follows: 2a10:ABCD:1111:FFFF:123:123:123:123/128

MARIADBVERS

Please specify the Maria DB version to install.

NGINXAUTHUSER

Please specify the username to set up the nginx web auth module. The password is generated automatically by the script and will be displayed at the end of the script.

PHPVERS

Please specify the PHP version to install.

PHPMAINST

Please select if you want to install PHPMyAdmin.

PHPMAVERS

Please specify the PHPMyAdmin version to install.

PHPMAUSER

Please select the username you will use to login to PHPMyAdmin. The password is generated automatically by the script and will be displayed at the end of the script.

SELFCHECK

Please specify if you want the script to self-verify, or not to self-verify the script and avoid a script verification failure due to an invalid signature or for easy and convenient script editing. Allowed values are:
"Yes" or "No"

TRUSTKEY

Please specify if you want to trust my ed25519 Gitea Signing Key. All my used keys reside in HSM like NitroKey and signing operations are done via an airgapped system. Allowed values are:
"Yes" or "No"

TRUSTLEVEL

Please specify the trust level of the signing key. PGP Trustlevel:

1 = I don't know or won't say
2 = I do NOT trust
3 = I trust marginally
4 = I trust fully
5 = I trust ultimately

Allowed values are:
"5" or "4" or "3"

VPSPROVIDER

Please select the VPS provider you are using. Or use „Generic“ for a basic hardening of your generic server or VPS after a freshly installed Debian Bookworm environment including the CISS partition scheme. Allowed values are:
"Contabo" or "Hetzner" or "Netcup" or "Generic"

SLEEPTIMER

This variable is used to set the clear screen routine after each module and is measured in seconds.

2) Quick Start

Start the setup using the following commands:

apt install wget --no-install-recommends -y
mkdir /root/server-scripts
cd /root/server-scripts
wget --https-only https://cendev.eu/marc.weidner/CISS.2024.hardened.webstack/raw/branch/main/webstack.sh
chmod 700 webstack.sh
./webstack.sh

3) Setup routine

a) Overview

The setup script performs basic initialization, loads all the functions needed to start the script, and installs all the software required to run the next modules. The first dialogue that appears is for you to select which NIC to use. If only one NIC is found, this dialogue is arbitrary.

The next prompt will prepare you to edit the .conf file. Use this stop to manually edit the .conf file, and perhaps transfer it using your preferred editing method. Then press ENTER to continue.

A nano editor will open and you can either confirm and/or edit the variables. Press CTRL-O to quit nano.

A summary of all variables read is displayed.

Now comes the most important part of the script. My public signing key

Fingerprint A6D4 6A56 AE17 A185 AB0F 6DB7 7095 A8A1 3CBE 0FA3

is installed. Please confirm the fingerprint provided here with the fingerprint expected by the script and provided by the downloaded file.

THEY MUST ALL MATCH EXACTLY.

Please confirm that the fingerprint matches by entering „Yes“ and pressing ENTER.

Please ignore the error that appears, these messages are only displayed because the script is trying to check if you have already installed my keys. If you have not, the grep command will return an error which the script will ignore for this reason only.

If the fingerprints do not match, enter „No“ and press ENTER to continue. The setup will fail immediately and discard the downloaded key. Please contact me at security@coresecret.eu with the subject PGP Key Fingerprint mismatch and include screenshots if possible. You do not need to provide your real name. See also my security policy.

If you prepared the server using the CISS.2024.vps.bookworm.hardening script, the keys are already installed and you will receive the following confirmation:

While acme.sh is running, you will see the following error, which is the same error described earlier in the key confirmation routine:

The script tries to check if you already have an acmeuser. If you do not, the grep command will return an error, which the script will ignore for this reason only.

The Docker install script is called by this script and the following erroneous behavior occurs. During the execution of the Docker setup and after completion, the newly spawned bash process terminates at a prompt. If this happens, do nothing else, then type exit at the prompt and press ENTER. You will immediately return to the main webstack install routine.

The final summary screen gives you information about the login credentials, where and how to view the log file from the CLI, the password file which should be deleted asap.

C) Ressources

1) Online audit tools

D) Changelog

DateInfo
2024.07.11Initial Version

Please consider a donation in support of my free services.

Search: