Question: How to: File/Folder Integrity Monitoring With AIDE (WIP)

Answer:
#### Why

WIP

#### How It Works

WIP

#### Goals

WIP

#### References

- https://aide.github.io/
- https://www.hiroom2.com/2017/06/09/debian-8-file-integrity-check-with-aide/
- https://blog.rapid7.com/2017/06/30/how-to-install-and-configure-aide-on-ubuntu-linux/
- https://www.stephenrlang.com/2016/03/using-aide-for-file-integrity-monitoring-fim-on-ubuntu/
- https://www.howtoforge.com/how-to-configure-the-aide-advanced-intrusion-detection-environment-file-integrity-scanner-for-your-website
- https://www.tecmint.com/check-integrity-of-file-and-directory-using-aide-in-linux/
- https://www.cyberciti.biz/faq/debian-ubuntu-linux-software-integrity-checking-with-aide/
- https://github.com/imthenachoman/How-To-Secure-A-Linux-Server/issues/83

#### Steps

1. Install AIDE.

    On Debian based systems:

    ``` bash
    sudo apt install aide aide-common
    ```

1. Make a backup of AIDE's defaults file:

    ``` bash
    sudo cp -p /etc/default/aide /etc/default/aide-COPY-$(date +"%Y%m%d%H%M%S")
    ```

1. Go through `/etc/default/aide` and set AIDE's defaults per your requirements. If you want AIDE to run daily and e-mail you, be sure to set `CRON_DAILY_RUN` to `yes`.

1. Make a backup of AIDE's configuration files:

    ``` bash
    sudo cp -pr /etc/aide /etc/aide-COPY-$(date +"%Y%m%d%H%M%S")
    ```

1. On Debian based systems:

    - AIDE's configuration files are in `/etc/aide/aide.conf.d/`.
    - You'll want to go through AIDE's documentation and the configuration files in to set them per your requirements.
    - If you want new settings, to monitor a new folder for example, you'll want to add them to `/etc/aide/aide.conf` or `/etc/aide/aide.conf.d/`.
    - Take a backup of the stock configuration files: `sudo cp -pr /etc/aide /etc/aide-COPY-$(date +"%Y%m%d%H%M%S")`.

1. Create a new database, and install it.

    On Debian based systems:

    ``` bash
    sudo aideinit
    ```

    > ```
    > Running aide --init...
    > Start timestamp: 2019-04-01 21:23:37 -0400 (AIDE 0.16)
    > AIDE initialized database at /var/lib/aide/aide.db.new
    > Verbose level: 6
    >
    > Number of entries:      25973
    >
    > ---------------------------------------------------
    > The attributes of the (uncompressed) database(s):
    > ---------------------------------------------------
    >
    > /var/lib/aide/aide.db.new
    >   RMD160   : moyQ1YskQQbidX+Lusv3g2wf1gQ=
    >   TIGER    : 7WoOgCrXzSpDrlO6I3PyXPj1gRiaMSeo
    >   SHA256   : gVx8Fp7r3800WF2aeXl+/KHCzfGsNi7O
    >              g16VTPpIfYQ=
    >   SHA512   : GYfa0DJwWgMLl4Goo5VFVOhu4BphXCo3
    >              rZnk49PYztwu50XjaAvsVuTjJY5uIYrG
    >              tV+jt3ELvwFzGefq4ZBNMg==
    >   CRC32    : /cusZw==
    >   HAVAL    : E/i5ceF3YTjwenBfyxHEsy9Kzu35VTf7
    >              CPGQSW4tl14=
    >   GOST     : n5Ityzxey9/1jIs7LMc08SULF1sLBFUc
    >              aMv7Oby604A=
    >
    >
    > End timestamp: 2019-04-01 21:24:45 -0400 (run time: 1m 8s)
    > ```

1. Test everything works with no changes.

    On Debian based systems:

    ``` bash
    sudo aide.wrapper --check
    ```

    > ```
    > Start timestamp: 2019-04-01 21:24:45 -0400 (AIDE 0.16)
    > AIDE found NO differences between database and filesystem. Looks okay!!
    > Verbose level: 6
    >
    > Number of entries:      25973
    >
    > ---------------------------------------------------
    > The attributes of the (uncompressed) database(s):
    > ---------------------------------------------------
    >
    > /var/lib/aide/aide.db
    >   RMD160   : moyQ1YskQQbidX+Lusv3g2wf1gQ=
    >   TIGER    : 7WoOgCrXzSpDrlO6I3PyXPj1gRiaMSeo
    >   SHA256   : gVx8Fp7r3800WF2aeXl+/KHCzfGsNi7O
    >              g16VTPpIfYQ=
    >   SHA512   : GYfa0DJwWgMLl4Goo5VFVOhu4BphXCo3
    >              rZnk49PYztwu50XjaAvsVuTjJY5uIYrG
    >              tV+jt3ELvwFzGefq4ZBNMg==
    >   CRC32    : /cusZw==
    >   HAVAL    : E/i5ceF3YTjwenBfyxHEsy9Kzu35VTf7
    >              CPGQSW4tl14=
    >   GOST     : n5Ityzxey9/1jIs7LMc08SULF1sLBFUc
    >              aMv7Oby604A=
    >
    >
    > End timestamp: 2019-04-01 21:26:03 -0400 (run time: 1m 18s)
    > ```

1. Test everything works after making some changes.

    On Debian based systems:

    ``` bash
    sudo touch /etc/test.sh
    sudo touch /root/test.sh

    sudo aide.wrapper --check

    sudo rm /etc/test.sh
    sudo rm /root/test.sh

    sudo aideinit -y -f
    ```

    > ```
    > Start timestamp: 2019-04-01 21:37:37 -0400 (AIDE 0.16)
    > AIDE found differences between database and filesystem!!
    > Verbose level: 6
    >
    > Summary:
    >   Total number of entries:      25972
    >   Added entries:                2
    >   Removed entries:              0
    >   Changed entries:              1
    >
    > ---------------------------------------------------
    > Added entries:
    > ---------------------------------------------------
    >
    > f++++++++++++++++: /etc/test.sh
    > f+++++++++


Question: How to: Anti-Virus Scanning With ClamAV (WIP)

Answer:
#### Why

WIP

#### How It Works

- ClamAV is a virus scanner
- ClamAV-Freshclam is a service that keeps the virus definitions updated
- ClamAV-Daemon keeps the `clamd` process running to make scanning faster

#### Goals

WIP

#### Notes

- These instructions **do not** tell you how to enable the ClamAV daemon service to ensure `clamd` is running all the time. `clamd` is only if you're running a mail server and does not provide real-time monitoring of files. Instead, you'd want to scan files manually or on a schedule.

#### References

- https://www.clamav.net/documents/installation-on-debian-and-ubuntu-linux-distributions
- https://wiki.debian.org/ClamAV
- https://www.osradar.com/install-clamav-debian-9-ubuntu-18/
- https://www.lisenet.com/2014/automate-clamav-to-perform-daily-system-scan-and-send-email-notifications-on-linux/
- https://www.howtoforge.com/tutorial/configure-clamav-to-scan-and-notify-virus-and-malware/
- https://serverfault.com/questions/741299/is-there-a-way-to-keep-clamav-updated-on-debian-8
- https://askubuntu.com/questions/250290/how-do-i-scan-for-viruses-with-clamav
- https://ngothang.com/how-to-install-clamav-and-configure-daily-scanning-on-centos/

#### Steps

1. Install ClamAV.

    On Debian based systems:

    ``` bash
    sudo apt install clamav clamav-freshclam clamav-daemon
    ```

1. Make a backup of `clamav-freshclam`'s configuration file `/etc/clamav/freshclam.conf`:

    ``` bash
    sudo cp --archive /etc/clamav/freshclam.conf /etc/clamav/freshclam.conf-COPY-$(date +"%Y%m%d%H%M%S")
    ```

1. `clamav-freshclam`'s default settings are probably good enough but if you want to change them, you can either edit the file `/etc/clamav/freshclam.conf` or use `dpkg-reconfigure`:

    ``` bash
    sudo dpkg-reconfigure clamav-freshclam
    ```

    **Note**: The default settings will update the definitions 24 times in a day. To change the interval, check the `Checks` setting in `/etc/clamav/freshclam.conf` or use `dpkg-reconfigure`.

1. Start the `clamav-freshclam` service:

    ``` bash
    sudo service clamav-freshclam start
    ```

1. You can make sure `clamav-freshclam` running:

    ``` bash
    sudo service clamav-freshclam status
    ```

    > ```
    > ● clamav-freshclam.service - ClamAV virus database updater
    >    Loaded: loaded (/lib/systemd/system/clamav-freshclam.service; enabled; vendor preset: enabled)   Active: active (running) since Sat 2019-03-16 22:57:07 EDT; 2min 13s ago
    >      Docs: man:freshclam(1)
    >            man:freshclam.conf(5)
    >            https://www.clamav.net/documents
    >  Main PID: 1288 (freshclam)
    >    CGroup: /system.slice/clamav-freshclam.service
    >            └─1288 /usr/bin/freshclam -d --foreground=true
    >
    > Mar 16 22:57:08 host freshclam[1288]: Sat Mar 16 22:57:08 2019 -> ^Local version: 0.100.2 Recommended version: 0.101.1
    > Mar 16 22:57:08 host freshclam[1288]: Sat Mar 16 22:57:08 2019 -> DON'T PANIC! Read https://www.clamav.net/documents/upgrading-clamav
    > Mar 16 22:57:15 host freshclam[1288]: Sat Mar 16 22:57:15 2019 -> Downloading main.cvd [100%]
    > Mar 16 22:57:38 host freshclam[1288]: Sat Mar 16 22:57:38 2019 -> main.cvd updated (version: 58, sigs: 4566249, f-level: 60, builder: sigmgr)
    > Mar 16 22:57:40 host freshclam[1288]: Sat Mar 16 22:57:40 2019 -> Downloading daily.cvd [100%]
    > Mar 16 22:58:13 host freshclam[1288]: Sat Mar 16 22:58:13 2019 -> daily.cvd updated (version: 25390, sigs: 1520006, f-level: 63, builder: raynman)
    > Mar 16 22:58:14 host freshclam[1288]: Sat Mar 16 22:58:14 2019 -> Downloading bytecode.cvd [100%]
    > Mar 16 22:58:16 host freshclam[1288]: Sat Mar 16 22:58:16 2019 -> bytecode.cvd updated (version: 328, sigs: 94, f-level: 63, builder: neo)
    > Mar 16 22:58:24 host freshclam[1288]: Sat Mar 16 22:58:24 2019 -> Database updated (6086349 signatures) from db.local.clamav.net (IP: 104.16.219.84)
    > Mar 16 22:58:24 host freshclam[1288]: Sat Mar 16 22:58:24 2019 -> ^Clamd was NOT notified: Can't connect to clamd through /var/run/clamav/clamd.ctl: No such file or directory
    > ```

    **Note**: Don't worry about that `Local version` line. Check https://serverfault.com/questions/741299/is-there-a-way-to-keep-clamav-updated-on-debian-8 for more details.

1. Make a backup of `clamav-daemon`'s configuration file `/etc/clamav/clamd.conf`:

    ``` bash
    sudo cp --archive /etc/clamav/clamd.conf /etc/clamav/clamd.conf-COPY-$(date +"%Y%m%d%H%M%S")
    ```

1. You can change `clamav-daemon`'s settings by editing the file `/etc/clamav/clamd.conf` or useing `dpkg-reconfigure`:

    ``` bash
    sudo dpkg-reconfigure clamav-daemon
    ```

#### Scanning Files/Folders

- To scan files/folders use the `clamscan` program.
- `clamscan` runs as the user it is executed as so it needs read permissions to the files/folders it is scanning.
- Using `clamscan` as `root` is dangerous because if a file is in fact a virus there is risk that it could


Question: How to: Rootkit Detection With Rkhunter (WIP)

Answer:
#### Why

WIP

#### How It Works

WIP

#### Goals

WIP

#### References

- http://rkhunter.sourceforge.net/
- https://www.cyberciti.biz/faq/howto-check-linux-rootkist-with-detectors-software/
- https://www.tecmint.com/install-rootkit-hunter-scan-for-rootkits-backdoors-in-linux/

#### Steps

1. Install Rkhunter.

    On Debian based systems:

    ``` bash
    sudo apt install rkhunter
    ```

1. Make a backup of rkhunter' defaults file:

    ``` bash
    sudo cp -p /etc/default/rkhunter /etc/default/rkhunter-COPY-$(date +"%Y%m%d%H%M%S")
    ```

1. rkhunter's configuration file is `/etc/rkhunter.conf`. Instead of making changes to it, create and use the file `/etc/rkhunter.conf.local` instead:

    ``` bash
    sudo cp -p /etc/rkhunter.conf /etc/rkhunter.conf.local
    ```

1. Go through the configuration file `/etc/rkhunter.conf.local` and set to your requirements. My recommendations:

    |Setting|Note|
    |--|--|
    |`UPDATE_MIRRORS=1`||
    |`MIRRORS_MODE=0`||
    |`MAIL-ON-WARNING=root`||
    |`COPY_LOG_ON_ERROR=1`|to save a copy of the log if there is an error|
    |`PKGMGR=...`|set to the appropriate value per the documentation|
    |`PHALANX2_DIRTEST=1`|read the documentation for why|
    |`WEB_CMD=""`|this is to address an issue with the Debian package that disables the ability for rkhunter to self-update.|
    |`USE_LOCKING=1`|to prevent issues with rkhunter running multiple times|
    |`SHOW_SUMMARY_WARNINGS_NUMBER=1`|to see the actual number of warnings found|

1. You want rkhunter to run every day and e-mail you the result. You can write your own script or check https://www.tecmint.com/install-rootkit-hunter-scan-for-rootkits-backdoors-in-linux/ for a sample cron script you can use.

    On Debian based system, rkhunter comes with cron scripts. To enable them check `/etc/default/rkhunter` or use `dpkg-reconfigure` and say `Yes` to all of the questions:

    ``` bash
    sudo dpkg-reconfigure rkhunter
    ```

1. After you've finished with all of the changes, make sure all the settings are valid:

    ``` bash
    sudo rkhunter -C
    ```

1. Update rkhunter and its database:

    ``` bash
    sudo rkhunter --versioncheck
    sudo rkhunter --update
    sudo rkhunter --propupd
    ```

1. If you want to do a manual scan and see the output:

    ``` bash
    sudo rkhunter --check
    ```

([Table of Contents](#table-of-contents))


Question: How to: Rootkit Detection With chrootkit (WIP)

Answer:
#### Why

WIP

#### How It Works

WIP

#### Goals

WIP

#### References

- http://www.chkrootkit.org/
- https://www.cyberciti.biz/faq/howto-check-linux-rootkist-with-detectors-software/
- https://askubuntu.com/questions/258658/eth0-packet-sniffer-sbin-dhclient

#### Steps

1. Install chkrootkit.

    On Debian based systems:

    ``` bash
    sudo apt install chkrootkit
    ```

1. Do a manual scan:

    ``` bash
    sudo chkrootkit
    ```

    > ```
    > ROOTDIR is `/'
    > Checking `amd'...                                           not found
    > Checking `basename'...                                      not infected
    > Checking `biff'...                                          not found
    > Checking `chfn'...                                          not infected
    > Checking `chsh'...                                          not infected
    > ...
    > Checking `scalper'...                                       not infected
    > Checking `slapper'...                                       not infected
    > Checking `z2'...                                            chklastlog: nothing deleted
    > Checking `chkutmp'...                                       chkutmp: nothing deleted
    > Checking `OSX_RSPLUG'...                                    not infected
    > ```

1. Make a backup of chkrootkit's configuration file `/etc/chkrootkit.conf`:

    ``` bash
    sudo cp --archive /etc/chkrootkit.conf /etc/chkrootkit.conf-COPY-$(date +"%Y%m%d%H%M%S")
    ```

1. You want chkrootkit to run every day and e-mail you the result.

    On Debian based system, chkrootkit comes with cron scripts. To enable them check `/etc/chkrootkit.conf` or use `dpkg-reconfigure` and say `Yes` to the first question:

    ``` bash
    sudo dpkg-reconfigure chkrootkit
    ```

([Table of Contents](#table-of-contents))


Question: How to: logwatch - system log analyzer and reporter

Answer:
#### Why

Your server will be generating a lot of logs that may contain important information. Unless you plan on checking your server everyday, you'll want a way to get e-mail summary of your server's logs. To accomplish this we'll use [logwatch](https://sourceforge.net/projects/logwatch/).

#### How It Works

logwatch scans system log files and summarizes them. You can run it directly from the command line or schedule it to run on a recurring schedule. logwatch uses service files to know how to read/summarize a log file. You can see all of the stock service files in `/usr/share/logwatch/scripts/services`.

logwatch's configuration file `/usr/share/logwatch/default.conf/logwatch.conf` specifies default options. You can override them via command line arguments.

#### Goals

- Logwatch configured to send a daily e-mail summary of all of the server's status and logs

#### Notes

- Your server will need to be able to send e-mails for this to work
- The below steps will result in logwatch running every day. If you want to change the schedule, modify the cronjob to your liking. You'll also want to change the `range` option to cover your recurrence window. See https://www.badpenguin.org/configure-logwatch-for-weekly-email-and-html-output-format for an example.
- If logwatch fails to deliver mail due to the e-mail having long lines please check https://blog.dhampir.no/content/exim4-line-length-in-debian-stretch-mail-delivery-failed-returning-message-to-sender as documented in [issue #29](https://github.com/imthenachoman/How-To-Secure-A-Linux-Server/issues/29). If you followed [Gmail and Exim4 As MTA With Implicit TLS](#gmail-and-exim4-as-mta-with-implicit-tls) then we already took care of this in step #7.

#### References

- Thanks to [amacheema](https://github.com/amacheema) for fixing some issues with the steps and letting me know of a long line bug with exim4 as documented in [issue #29](https://github.com/imthenachoman/How-To-Secure-A-Linux-Server/issues/29).
- https://sourceforge.net/projects/logwatch/
- https://www.digitalocean.com/community/tutorials/how-to-install-and-use-logwatch-log-analyzer-and-reporter-on-a-vps

#### Steps

1. Install logwatch.

    On Debian based systems:

    ``` bash
    sudo apt install logwatch
    ```

1. To see a sample of what logwatch collects you can run it directly:

    ``` bash
    sudo /usr/sbin/logwatch --output stdout --format text --range yesterday --service all
    ```

    > ```
    >
    >  ################### Logwatch 7.4.3 (12/07/16) ####################
    >         Processing Initiated: Mon Mar  4 00:05:50 2019
    >         Date Range Processed: yesterday
    >                               ( 2019-Mar-03 )
    >                               Period is day.
    >         Detail Level of Output: 5
    >         Type of Output/Format: stdout / text
    >         Logfiles for Host: host
    >  ##################################################################
    >
    >  --------------------- Cron Begin ------------------------
    > ...
    > ...
    >  ---------------------- Disk Space End -------------------------
    >
    >
    >  ###################### Logwatch End #########################
    > ```

1. Go through logwatch's self-documented configuration file `/usr/share/logwatch/default.conf/logwatch.conf` before continuing. There is no need to change anything here but pay special attention to the `Output`, `Format`, `MailTo`, `Range`, and `Service` as those are the ones we'll be using. For our purposes, instead of specifying our options in the configuration file, we will pass them as command line arguments in the daily cron job that executes logwatch. That way, if the configuration file is ever modified (e.g. during an update), our options will still be there.

1. Make a backup of logwatch's daily cron file `/etc/cron.daily/00logwatch` and unset the execute bit:

    ``` bash
    sudo cp --archive /etc/cron.daily/00logwatch /etc/cron.daily/00logwatch-COPY-$(date +"%Y%m%d%H%M%S")
    sudo chmod -x /etc/cron.daily/00logwatch-COPY*
    ```

1. By default, logwatch outputs to `stdout`. Since the goal is to get a daily e-mail, we need to change the output type that logwatch uses to send e-mail instead. We could do this through the configuration file above, but that would apply to every time it is run -- even when we run it manually and want to see the output to the screen. Instead, we'll change the cron job that executes logwatch to send e-mail. This way, when run manually, we'll still get output to `stdout` and when run by cron, it'll send an e-mail. We'll also make sure it checks for all services, and change the output format to html so it's easier to read regardless of what the configuration file says. In the file `/etc/cron.daily/00logwatch` find the execute line and change it to:

    ```
    /usr/sbin/logwatch --output mail --format html --mailto root --range yesterday --service all
    ```

    > ```
    > #!/bin/bash
    >
    > #Check if remove


Question: How to: ss - Seeing Ports Your Server Is Listening On

Answer:
#### Why

Ports are how applications, services, and processes communicate with each other -- either locally within your server or with other devices on the network. When you have an application or service (like SSH or Apache) running on your server, they listen for requests on specific ports.

Obviously we don't want your server listening on ports we don't know about. We'll use `ss` to see all the ports that services are listening on. This will help us track down and stop rogue, potentially dangerous, services.

#### Goals

- find out non-localhost what ports are open and listening for connections

#### References

- https://www.reddit.com/r/linux/comments/arx7st/howtosecurealinuxserver_an_evolving_howto_guide/egrib6o/
- https://www.reddit.com/r/linux/comments/arx7st/howtosecurealinuxserver_an_evolving_howto_guide/egs1rev/
- https://www.tecmint.com/find-open-ports-in-linux/
- `man ss`

#### Steps

1. To see the all the ports listening for traffic:

    ``` bash
    sudo ss -lntup
    ```

    > ```
    > Netid  State      Recv-Q Send-Q     Local Address:Port     Peer Address:Port
    > udp    UNCONN     0      0                      *:68                  *:*        users:(("dhclient",pid=389,fd=6))
    > tcp    LISTEN     0      128                    *:22                  *:*        users:(("sshd",pid=4390,fd=3))
    > tcp    LISTEN     0      128                   :::22                 :::*        users:(("sshd",pid=4390,fd=4))
    > ```

    **Switch Explanations**:
    - `l` = display listening sockets
    - `n` = do not try to resolve service names
    - `t` = display TCP sockets
    - `u` = display UDP sockets
    - `p` = show process information

1. If you see anything suspicious, like a port you're not aware of or a process you don't know, investigate and remediate as necessary.

([Table of Contents](#table-of-contents))


Question: How to: Lynis - Linux Security Auditing

Answer:
#### Why

From [https://cisofy.com/lynis/](https://cisofy.com/lynis/):

> Lynis is a battle-tested security tool for systems running Linux, macOS, or Unix-based operating system. It performs an extensive health scan of your systems to support system hardening and compliance testing.

#### Goals

- Lynis installed

#### Notes

- CISOFY offers packages for many distributions. Check https://packages.cisofy.com/ for distribution specific installation instructions.

#### References

- https://cisofy.com/documentation/lynis/get-started/
- https://packages.cisofy.com/community/#debian-ubuntu
- https://thelinuxcode.com/audit-lynis-ubuntu-server/
- https://www.vultr.com/docs/install-lynis-on-debian-8

#### Steps

1. Install lynis. https://cisofy.com/lynis/#installation has detailed instructions on how to install it for your distribution.

    On Debian based systems, using CISOFY's community software repository:

    ``` bash
    sudo apt install apt-transport-https ca-certificates host
    sudo wget -O - https://packages.cisofy.com/keys/cisofy-software-public.key | sudo apt-key add -
    sudo echo "deb https://packages.cisofy.com/community/lynis/deb/ stable main" | sudo tee /etc/apt/sources.list.d/cisofy-lynis.list
    sudo apt update
    sudo apt install lynis host
    ```

1. Update it:

    ``` bash
    sudo lynis update info
    ```

1. Run a security audit:

    ``` bash
    sudo lynis audit system
    ```

    This will scan your server, report its audit findings, and at the end it will give you suggestions. Spend some time going through the output and address gaps as necessary.

([Table of Contents](#table-of-contents))


Question: How to: OSSEC - Host Intrusion Detection

Answer:
#### Why
From [https://github.com/ossec/ossec-hids](https://github.com/ossec/ossec-hids)
> OSSEC is a full platform to monitor and control your systems. It mixes together all the aspects of HIDS (host-based intrusion detection), log monitoring and SIM/SIEM together in a simple, powerful and open source solution.

#### Goals

- OSSEC-HIDS installed

#### References

- https://www.ossec.net/docs/

#### Steps

1. Install OSSEC-HIDS from sources
    ```bash
    sudo apt install -y libz-dev libssl-dev libpcre2-dev build-essential libsystemd-dev
    wget https://github.com/ossec/ossec-hids/archive/3.7.0.tar.gz
    tar xzf 3.7.0.tar.gz
    cd ossec-hids-3.7.0/
    sudo ./install.sh
    ```

1. Useful commands:

**Agent information**

   ```bash
    sudo /var/ossec/bin/agent_control -i
```
`AGENT_ID` by default is `000`, to be sure the command `sudo /var/ossec/bin/agent_control -l` can be used.

**Run integrity/rootkit checking**

OSSEC by default run rootkit check each 2 hours.

   ```bash
    sudo /var/ossec/bin/agent_control -u
-r
   ```

**Alerts**

- All:
    ```bash
    tail -f /var/ossec/logs/alerts/alerts.log
    ```
- Integrity check:
    ```bash
    sudo cat /var/ossec/logs/alerts/alerts.log | grep -A4  -i integrity
    ```
- Rootkit check:
    ```bash
     sudo cat /var/ossec/logs/alerts/alerts.log | grep -A4  "rootcheck,"
    ```

([Table of Contents](#table-of-contents))


Question: How to: Proceed At Your Own Risk

Answer:
This sections cover things that are high risk because there is a possibility they can make your system unusable, or are considered unnecessary by many because the risks outweigh any rewards.

**!! PROCEED AT YOUR OWN RISK !!**
!! PROCEED AT YOUR OWN RISK !!
([Table of Contents](#table-of-contents))


Question: How to: Password Protect GRUB

Answer:
!! PROCEED AT YOUR OWN RISK !!
#### Why

If a bad actor has physical access to your server, they could use GRUB to gain unauthorized access to your system.

#### Why Not

If you forget the password, you'll have to go through [some work](https://www.cyberciti.biz/tips/howto-recovering-grub-boot-loader-password.html) to recover the password.

#### Goals

- auto boot the default Debian install and require a password for anything else

#### Notes

- This will only protect GRUB and anything behind it like your operating systems. Check your motherboard's documentation for password protecting your BIOS to prevent a bad actor from circumventing GRUB.

#### References

- https://selivan.github.io/2017/12/21/grub2-password-for-all-but-default-menu-entries.html
- https://help.ubuntu.com/community/Grub2/Passwords
- https://computingforgeeks.com/how-to-protect-grub-with-password-on-debian-ubuntu-and-kali-linux/
- `man grub`
- `man grub-mkpasswd-pbkdf2`

#### Steps

1. Create a [Password-Based Key Derivation Function 2 (PBKDF2)](https://en.wikipedia.org/wiki/PBKDF2) hash of your password:

    ``` bash
    grub-mkpasswd-pbkdf2 -c 100000
    ```

   The below output is from using `password` as the password:

    > ```
    > Enter password:
    > Reenter password:
    > PBKDF2 hash of your password is grub.pbkdf2.sha512.100000.2812C233DFC899EFC3D5991D8CA74068C99D6D786A54F603E9A1EFE7BAEDDB6AA89672F92589FAF98DB9364143E7A1156C9936328971A02A483A84C3D028C4FF.C255442F9C98E1F3C500C373FE195DCF16C56EEBDC55ABDD332DD36A92865FA8FC4C90433757D743776AB186BD3AE5580F63EF445472CC1D151FA03906D08A6D
    > ```

1. Copy everything **after** `PBKDF2 hash of your password is `, **starting from and including** `grub.pbkdf2.sha512...` to the end. You'll need this in the next step.

1. The `update-grub` program uses scripts to generate configuration files it will use for GRUB's settings.  Create the file `/etc/grub.d/01_password` and add the below code after replacing `[hash]` with the hash you copied from the first step. This tells `update-grub` to use this username and password for GRUB.

    ``` bash
    #!/bin/sh
    set -e

    cat << EOF
    set superusers="grub"
    password_pbkdf2 grub [hash]
    EOF
    ```

    For example:

    > ``` bash
    > #!/bin/sh
    > set -e
    >
    > cat << EOF
    > set superusers="grub"
    > password_pbkdf2 grub grub.pbkdf2.sha512.100000.2812C233DFC899EFC3D5991D8CA74068C99D6D786A54F603E9A1EFE7BAEDDB6AA89672F92589FAF98DB9364143E7A1156C9936328971A02A483A84C3D028C4FF.C255442F9C98E1F3C500C373FE195DCF16C56EEBDC55ABDD332DD36A92865FA8FC4C90433757D743776AB186BD3AE5580F63EF445472CC1D151FA03906D08A6D
    > EOF
    > ```

1. Set the file's execute bit so `update-grub` includes it when it updates GRUB's configuration:

   ``` bash
   sudo chmod a+x /etc/grub.d/01_password
   ```

1. Make a backup of GRUB's configuration file `/etc/grub.d/10_linux` that we'll be modifying and unset the execute bit so `update-grub` doesn't try to run it:

    ``` bash
    sudo cp --archive /etc/grub.d/10_linux /etc/grub.d/10_linux-COPY-$(date +"%Y%m%d%H%M%S")
    sudo chmod a-x /etc/grub.d/10_linux.*
    ```

1. To make the default Debian install unrestricted (**without** the password) while keeping everything else restricted (**with** the password) modify `/etc/grub.d/10_linux` and add `--unrestricted` to the `CLASS` variable.

    [For the lazy](#editing-configuration-files---for-the-lazy):

    ``` bash
    sudo sed -i -r -e "/^CLASS=/ a CLASS=\"\${CLASS} --unrestricted\"         # added by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")" /etc/grub.d/10_linux
    ```

1. Update GRUB with `update-grub`:

    ``` bash
    sudo update-grub
    ```
([Table of Contents](#table-of-contents))


Question: How to: Change Default umask

Answer:
!! PROCEED AT YOUR OWN RISK !!
#### Why

umask controls the **default** permissions of files/folders when they are created. Insecure file/folder permissions give other accounts potentially unauthorized access to your data. This may include the ability to make configuration changes.

- For **non-root** accounts, there is no need for other accounts to get any access to the account's files/folders **by default**.
- For the **root** account, there is no need for the file/folder primary group or other accounts to have any access to **root**'s files/folders **by default**.

When and if other accounts need access to a file/folder, you want to explicitly grant it using a combination of file/folder permissions and primary group.

#### Why Not

Changing the default umask can create unexpected problems. For example, if you set umask to `0077` for **root**, then **non-root** accounts **will not** have access to application configuration files/folders in `/etc/` which could break applications that do not run with **root** privileges.

#### How It Works

In order to explain how umask works I'd have to explain how Linux file/folder permissions work. As that is a rather complicated question, I will defer you to the references below for further reading.

#### Goals

- set default umask for **non-root** accounts to **0027**
- set default umask for the **root** account to **0077**

#### Notes

- umask is a Bash built-in which means a user can change their own umask setting.

#### References

- https://www.linuxnix.com/umask-define-linuxunix/
- https://serverfault.com/questions/818783/which-umask-is-more-secure-in-linux-022-or-027
- https://www.cyberciti.biz/tips/understanding-linux-unix-umask-value-usage.html
- `man umask`

#### Steps

1. Make a backup of files we'll be editing:

    ``` bash
    sudo cp --archive /etc/profile /etc/profile-COPY-$(date +"%Y%m%d%H%M%S")
    sudo cp --archive /etc/bash.bashrc /etc/bash.bashrc-COPY-$(date +"%Y%m%d%H%M%S")
    sudo cp --archive /etc/login.defs /etc/login.defs-COPY-$(date +"%Y%m%d%H%M%S")
    sudo cp --archive /root/.bashrc /root/.bashrc-COPY-$(date +"%Y%m%d%H%M%S")
    ```

1. Set default umask for **non-root** accounts to **0027** by adding this line to `/etc/profile` and `/etc/bash.bashrc`:

    ```
    umask 0027
    ```

    [For the lazy](#editing-configuration-files---for-the-lazy):

    ``` bash
    echo -e "\numask 0027         # added by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")" | sudo tee -a /etc/profile /etc/bash.bashrc
    ```

1. We also need to add this line to `/etc/login.defs`:

    ```
    UMASK 0027
    ```

    [For the lazy](#editing-configuration-files---for-the-lazy):

    ``` bash
    echo -e "\nUMASK 0027         # added by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")" | sudo tee -a /etc/login.defs
    ```

1. Set default umask for the **root** account to **0077** by adding this line to `/root/.bashrc`:

    ```
    umask 0077
    ```

    [For the lazy](#editing-configuration-files---for-the-lazy):

    ``` bash
    echo -e "\numask 0077         # added by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")" | sudo tee -a /root/.bashrc
    ```
([Table of Contents](#table-of-contents))


Question: How to: Orphaned Software

Answer:
!! PROCEED AT YOUR OWN RISK !!
#### Why

As you use your system, and you install and uninstall software, you'll eventually end up with orphaned, or unused software/packages/libraries. You don't need to remove them, but if you don't need them, why keep them? When security is a priority, anything not explicitly needed is a potential security threat. You want to keep your server as trimmed and lean as possible.

#### Notes

- Each distribution manages software/packages/libraries differently so how you find and remove orphaned packages will be different. So far I only have steps for Debian based systems.

#### Debian Based Systems

On Debian based systems, you can use [deborphan](http://freshmeat.sourceforge.net/projects/deborphan/) to find orphaned packages.

#####
Why Not

Keep in mind, deborphan finds packages that have **no package dependencies**. That does not mean they are not used. You could very well have a package you use every day that has no dependencies that you wouldn't want to remove. And, if deborphan gets anything wrong, then removing critical packages may break your system.

##### Steps

1. Install deborphan.

    ``` bash
    sudo apt install deborphan
    ```

1. Run deborphan as **root** to see a list of orphaned packages:

    ``` bash
    sudo deborphan
    ```

    > ```
    > libxapian30
    > libpipeline1
    > ```

1. [Assuming you want to remove all of the packages deborphan finds](#orphaned-software-why-not), you can pass it's output to `apt` to remove them:

    ``` bash
    sudo apt --autoremove purge $(deborphan)
    ```
([Table of Contents](#table-of-contents))


Question: How to: The Simple way with MSMTP

Answer:
(#msmtp-alternative)
#### Why

Well I will SIMPLIFY this method, to only output email using Google Mail account (and others). True Simple! :)

    ``` bash
    #!/bin/bash
    ###### PLEASE .... EDIT IT...
    USEREMAIL="usernameemail"
    DOMPROV="gmail.com"
    PWDEMAIL="passwordStrong"  ## ATTENTION DONT USE Special Chars.. like as SPACE # and some others not all. Feel free to test ;)
    MAILPROV="smtp.google.com:583"
    MYMAIL="$USRMAIL@$DOMPROV"
    USERLOC="root"
    #######
    apt install -y msmtp
        ln -s /usr/bin/msmtp /usr/sbin/sendmail
    #wget http://www.cacert.org/revoke.crl -O /etc/ssl/certs/revoke.crl
    #chmod 644 /etc/ssl/certs/revoke.crl
    touch /root/.msmtprc
    cat <
.msmtprc
    defaults
    account gmail
    host $MAILPROV
    port $MAILPORT
    #proxy_host 127.0.0.1
    #proxy_port 9001
    from $MYEMAIL
    timeout off
    protocol smtp
    #auto_from [(on|off)]
    #from envelope_from
    #maildomain [domain]
    auth on
    user $USRMAIL
    passwordeval "gpg -q --for-your-eyes-only --no-tty -d /root/msmtp-mail.gpg"
    #passwordeval "gpg --quiet --for-your-eyes-only --no-tty --decrypt /root/msmtp-mail.gpg"
    tls on
    tls_starttls on
    tls_trust_file /etc/ssl/certs/ca-certificates.crt
    #tls_crl_file /etc/ssl/certs/revoke.crl
    #tls_fingerprint [fingerprint]
    #tls_key_file [file]
    #tls_cert_file [file]
    tls_certcheck on
    tls_force_sslv3 on
    tls_min_dh_prime_bits 512
    #tls_priorities [priorities]
    #dsn_notify (off|condition)
    #dsn_return (off|amount)
    #domain argument
    #keepbcc off
    logfile /var/log/mail.log
    syslog on
    account default : gmail
    EOF
    chmod 0400 /root/.msmtprc

       ## In testing .. auto command
    # echo -e "1\n4096\n\ny\n$MYUSRMAIL\n$MYEMAIL\nmy key\nO\n$PWDMAIL\n$PWDMAIL\n" | gpg --full-generate-key
    ##
    gpg --full-generate-key
    gpg --output revoke.asc --gen-revoke $MYEMAIL
    echo -e "$PWDEMAIL\n" | gpg -e -o /root/msmtp-mail.gpg --recipient $MYEMAIL
    echo "export GPG_TTY=\$(tty)" >> .baschrc
    chmod 400 msmtp-mail.gpg

    echo "Hello there" | msmtp --debug $MYEMAIL
    echo"######################
    ## MSMTP Configured ##
    ######################"
    ```

DONE!! ;)
([Table of Contents](#table-of-contents))


Question: How to: Gmail and Exim4 As MTA With Implicit TLS

Answer:
#### Why

Unless you're planning on setting up your own mail server, you'll need a way to send e-mails from your server. This will be important for system alerts/messages.

You can use any Gmail account. I recommend you create one specific for this server. That way if your server **is** compromised, the bad-actor won't have any passwords for your primary account. Granted, if you have 2FA/MFA enabled, and you use an app password, there isn't much a bad-actor can do with just the app password, but why take the risk?

There are many guides on-line that cover how to configure Gmail as MTA using STARTTLS including a [previous version of this guide](https://github.com/imthenachoman/How-To-Secure-A-Linux-Server/tree/cc5edcae1cf846dd250e76b121e721d836481d2f#configure-gmail-as-mta). With STARTTLS, an initial **unencrypted** connection is made and then upgraded to an encrypted TLS or SSL connection. Instead, with the approach outlined below, an encrypted TLS connection is made from the start.

Also, as discussed in [issue #29](https://github.com/imthenachoman/How-To-Secure-A-Linux-Server/issues/29) and [here](https://blog.dhampir.no/content/exim4-line-length-in-debian-stretch-mail-delivery-failed-returning-message-to-sender), exim4 will fail for messages with long lines. We'll fix this in this section too.

** **IMPORTANT** ** As mentioned in [#106](https://github.com/imthenachoman/How-To-Secure-A-Linux-Server/issues/106), Google no longer lets you use your account's password for authentication. You have to enable 2FA and then use an app-password.

#### Goals

- `mail` configured to send e-mails from your server using [Gmail](https://mail.google.com/)
- long line support for exim4

#### References

- Thanks to [remyabel](https://github.com/remyabel) for figuring out how to get this to work with TLS as documented in [issue #24](https://github.com/imthenachoman/How-To-Secure-A-Linux-Server/issues/24) and [pull request #26](https://github.com/imthenachoman/How-To-Secure-A-Linux-Server/pull/26).
- https://wiki.debian.org/Exim
- https://wiki.debian.org/GmailAndExim4
- https://www.exim.org/exim-html-current/doc/html/spec_html/ch-encrypted_smtp_connections_using_tlsssl.html
- https://php.quicoto.com/setup-exim4-to-use-gmail-in-ubuntu/
- https://www.fastmail.com/help/technical/ssltlsstarttls.html
- exim4 fails for messages with long lines - [issue #29](https://github.com/imthenachoman/How-To-Secure-A-Linux-Server/issues/29) and https://blog.dhampir.no/content/exim4-line-length-in-debian-stretch-mail-delivery-failed-returning-message-to-sender
- https://github.com/imthenachoman/How-To-Secure-A-Linux-Server/issues/106

#### Steps

1. Install exim4. You will also need openssl and ca-certificates.

    On Debian based systems:

    ``` bash
    sudo apt install exim4 openssl ca-certificates
    ```

1. Configure exim4:

    For Debian based systems:
    ``` bash
    sudo dpkg-reconfigure exim4-config
    ```

    You'll be prompted with some questions:

    |Prompt|Answer|
    |--:|--|
    |General type of mail configuration|`mail sent by smarthost; no local mail`|
    |System mail name|`localhost`|
    |IP-addresses to listen on for incoming SMTP connections|`127.0.0.1; ::1`|
    |Other destinations for which mail is accepted|(default)|
    |Visible domain name for local users|`localhost`|
    |IP address or host name of the outgoing smarthost|`smtp.gmail.com::465`|
    |Keep number of DNS-queries minimal (Dial-on-Demand)?|`No`|
    |Split configuration into small files?|`No`|

1. Make a backup of `/etc/exim4/passwd.client`:

    ``` bash
    sudo cp --archive /etc/exim4/passwd.client /etc/exim4/passwd.client-COPY-$(date +"%Y%m%d%H%M%S")
    ```

1. Add a line like this to `/etc/exim4/passwd.client`

    ```
    smtp.gmail.com:yourAccount@gmail.com:yourPassword
    *.google.com:yourAccount@gmail.com:yourPassword
    ```

    **Notes**:
    - Replace `yourAccount@gmail.com` and `yourPassword` with your details. If you have 2FA/MFA enabled on your Gmail then you'll need to create and use an app password here.
    - Always check `host smtp.gmail.com` for the most up-to-date domains to list.

1. This file has your Gmail password so we need to lock it down:

    ``` bash
    sudo chown root:Debian-exim /etc/exim4/passwd.client
    sudo chmod 640 /etc/exim4/passwd.client
    ```

1. The next step is to create an TLS certificate that exim4 will use to make the encrypted connection to `smtp.gmail.com`. You can use your own certificate, like one from [Let's Encrypt](https://letsencrypt.org/), or create one yourself using openssl. We will use a script that comes with exim4 that calls openssl to make our certificate:

    ``` bash
    sudo bash /usr/share/doc/exim4-base/examples/exim-gencert
    ```

    > ```
    > [*] Creating a self signed SSL certificate for Exim!
    >     This may be sufficient to establish encrypted connections but for
    >     secure identification you need to buy a real certificate!
    >
    >     Please enter


Question: How to: Separate iptables Log File

Answer:
#### Why

There will come a time when you'll need to look through your iptables logs. Having all the iptables logs go to their own file will make it a lot easier to find what you're looking for.

#### References

- https://blog.shadypixel.com/log-iptables-messages-to-a-separate-file-with-rsyslog/
- https://gist.github.com/netson/c45b2dc4e835761fbccc
- https://www.rsyslog.com/doc/v8-stable/configuration/actions.html

#### Steps

1. The first step is by telling your firewall to prefix all log entries with some unique string. If you're using iptables directly, you would do something like `--log-prefix "[IPTABLES] "` for all the rules. We took care of this in [step 4 of installing psad](#psad_step4).

1. After you've added a prefix to the firewall logs, we need to tell rsyslog to send those lines to its own file. Do this by creating the file `/etc/rsyslog.d/10-iptables.conf` and adding this:

    ```
    :msg, contains, "[IPTABLES] " /var/log/iptables.log
    & stop
    ```

    If you're expecting a lot if data being logged by your firewall, prefix the filename with a `-` ["to omit syncing the file after every logging"](https://www.rsyslog.com/doc/v8-stable/configuration/actions.html#regular-file). For example:

    ```
    :msg, contains, "[IPTABLES] " -/var/log/iptables.log
    & stop
    ```

    **Note**: Remember to change the prefix to whatever you use.

    [For the lazy](#editing-configuration-files---for-the-lazy):

    ``` bash
    cat << EOF | sudo tee /etc/rsyslog.d/10-iptables.conf
    :msg, contains, "[IPTABLES] " /var/log/iptables.log
    & stop
    EOF
    ```

1. Since we're logging firewall messages to a different file, we need to tell psad where the new file is. Edit `/etc/psad/psad.conf` and set `IPT_SYSLOG_FILE` to the path of the log file. For example:

    ```
    IPT_SYSLOG_FILE /var/log/iptables.log;
    ```

    **Note**: Remember to change the prefix to whatever you use.

    [For the lazy](#editing-configuration-files---for-the-lazy):

    ``` bash
    sudo sed -i -r -e "s/^(IPT_SYSLOG_FILE\s+)([^;]+)(;)$/# \1\2\3       # commented by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")\n\1\/var\/log\/iptables.log\3       # added by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")/" /etc/psad/psad.conf
    ```

1. Restart psad and rsyslog to activate the changes (or reboot):

    ``` bash
    sudo psad -R
    sudo psad --sig-update
    sudo psad -H
    sudo service rsyslog restart
    ```

1. The last thing we have to do is tell logrotate to rotate the new log file so it doesn't get to big and fill up our disk. Create the file `/etc/logrotate.d/iptables` and add this:

    ```
    /var/log/iptables.log
    {
        rotate 7
        daily
        missingok
        notifempty
        delaycompress
        compress
        postrotate
            invoke-rc.d rsyslog rotate > /dev/null
        endscript
    }
    ```

    [For the lazy](#editing-configuration-files---for-the-lazy):

    ``` bash
    cat << EOF | sudo tee /etc/logrotate.d/iptables
    /var/log/iptables.log
    {
        rotate 7
        daily
        missingok
        notifempty
        delaycompress
        compress
        postrotate
            invoke-rc.d rsyslog rotate > /dev/null
        endscript
    }
    EOF
    ```

([Table of Contents](#table-of-contents))


Question: How to: Contacting Me

Answer:
For any questions, comments, concerns, feedback, or issues, submit a [new issue](https://github.com/imthenachoman/How-To-Secure-A-Linux-Server/issues/new).

([Table of Contents](#table-of-contents))


Question: How to: Helpful Links

Answer:
- [https://github.com/pratiktri/server_init_harden](https://github.com/pratiktri/server_init_harden) - Bash script that automates few of the tasks that you need to perform on a new Linux server to give it basic amount security.

([Table of Contents](#table-of-contents))


Question: How to: Acknowledgments

Answer:
- https://www.reddit.com/r/linuxquestions/comments/aopzl7/new_guide_created_by_me_how_to_secure_a_linux/
- https://www.reddit.com/r/selfhosted/comments/aoxd4l/new_guide_created_by_me_how_to_secure_a_linux/
- https://news.ycombinator.com/item?id=19177435#19178618
- https://www.reddit.com/r/linuxadmin/comments/arx7xo/howtosecurealinuxserver_an_evolving_howto_guide/
- https://www.reddit.com/r/linux/comments/arx7st/howtosecurealinuxserver_an_evolving_howto_guide/
- https://github.com/moltenbit/How-To-Secure-A-Linux-Server-With-Ansible

([Table of Contents](#table-of-contents))


Question: How to: License and Copyright

Answer:
[![CC-BY-SA](https://i.creativecommons.org/l/by-sa/4.0/88x31.png)](http://creativecommons.org/licenses/by-sa/4.0/)

[How To Secure A Linux Server](https://github.com/imthenachoman/How-To-Secure-A-Linux-Server) by [Anchal Nigam](https://github.com/imthenachoman) is licensed under [Creative Commons Attribution-ShareAlike 4.0 International License](http://creativecommons.org/licenses/by-sa/4.0).

See [LICENSE](LICENSE.txt) for the full license.

([Table of Contents](#table-of-contents))


Question: How to: Table of Contents

Answer:
1. [Getting Started with Self-Hosting](https://github.com/mikeroyal/Self-Hosting-Guide#getting-started-with-self-hosting)

    - [Tools for Self-Hosting](https://github.com/mikeroyal/Self-Hosting-Guide#tools-for-self-hosting)
       * [Containers](https://github.com/mikeroyal/Self-Hosting-Guide#containers)
       * [CI/CD](https://github.com/mikeroyal/Self-Hosting-Guide#cicd)
       * [Development](https://github.com/mikeroyal/Self-Hosting-Guide#development)
       * [Web servers](#web-servers)
       * [Large language models (LLMs)](#llms)
       * [ChatGPT Chatbots](#chatgpt)
       * [Automation](#automation)
       * [Configuration Management](#Configuration-Management)
       * [Cloud Storage](#cloud-storage)
       * [Cloud](https://github.com/mikeroyal/Self-Hosting-Guide#Cloud)
        * [Linode](#Linode)
	* [Nextcloud](#Nextcloud)
	* [DigitalOcean](#DigitalOcean)
        * [Back4app Web Deployment](#back4app-web-deployment)
	* [MinIO Object Storage](#MinIO-Object-Storage)
       * [Databases](#Databases)
         - [SQL](#SQL)
         - [NoSQL](#NoSQL)
       * [Remote Access](https://github.com/mikeroyal/Self-Hosting-Guide#Remote-Access)
       * [Virtualization](https://github.com/mikeroyal/Self-Hosting-Guide#Virtualization)
       * [Password Management](https://github.com/mikeroyal/Self-Hosting-Guide#password-management)
       * [SSH](#ssh)
       * [VPN](#vpn)
       * [LDAP(Lightweight Directory Access Protocol)](#ldap)
       * [Log Management](#log-management)
       * [DNS](#dns)
       * [Network Tools](https://github.com/mikeroyal/Self-Hosting-Guide#network-tools)
       * [Service Discovery](#service-discovery)
       * [Security](#security)
       * [Troubleshooting](#troubleshooting)
       * [Monitoring](https://github.com/mikeroyal/Self-Hosting-Guide#monitoring)
       * [Dashboards](#Dashboards)
       * [Analytics](#Analytics)
       * [Search](#Search)
       * [Notifications](#Notifications)
       * [RSS](#RSS)
       * [Websites/Blogs](#WebsitesBlogs)
       * [Social](#Social)
       * [Nostr](#nostr)
       * [iMessage](#imessage)
       * [Communications](https://github.com/mikeroyal/Self-Hosting-Guide#communications)
       * [Business Management](https://github.com/mikeroyal/Self-Hosting-Guide#business-management)
       * [Collaboration & Synchronization](https://github.com/mikeroyal/Self-Hosting-Guide#Collaboration--Synchronization)
       * [Encryption](#Encryption)
       * [Backups](https://github.com/mikeroyal/Self-Hosting-Guide#backups)
       * [Snapshots Management/System Recovery](snapshots-managementsystem-recovery)
       * [Archiving](#archiving)
       * [Home Server](https://github.com/mikeroyal/Self-Hosting-Guide#home-server)
       * [Media Server](https://github.com/mikeroyal/Self-Hosting-Guide#media-server)
       * [Smart Home Automation](#Smart-Home-Automation)
       * [Voice Assistants](#Voice-Assistants)
       * [Video Surveillance](#Video-Surveillance)
       * [Text-To-Speech Synthesis (TTS)](#Text-To-Speech-Synthesis-TTS)
       * [Video and Audio Processing](#Video-and-Audio-Processing)
       * [Podcasting](#Podcasting)
       * [Audiobooks](#Audiobooks)
       * [Health](#Health)
       * [Gardening](#gardening)
       * [Maps](https://github.com/mikeroyal/Self-Hosting-Guide#maps)
       * [Bookmarks](#Bookmarks)
       * [Photos](https://github.com/mikeroyal/Self-Hosting-Guide#photos)
       * [Pastebins](#pastebins)
       * [Note-Taking](#Note-Taking)
       * [Time Monitoring](#time-monitoring)
       * [Wikis](#wikis)
       * [Gaming](https://github.com/mikeroyal/Self-Hosting-Guide#gaming)
       * [Foundations/Projects](https://github.com/mikeroyal/Self-Hosting-Guide#foundationsprojects)

    - [System Hardware](#System-Hardware)
    - [Operating Systems](#Operating-Systems)
    - [Storage](https://github.com/mikeroyal/Self-Hosting-Guide#storage)
    - [File systems](https://github.com/mikeroyal/Self-Hosting-Guide#file-systems)
    - [Books](https://github.com/mikeroyal/Self-Hosting-Guide#books)
    - [Podcasts](https://github.com/mikeroyal/Self-Hosting-Guide#podcasts)
    - [YouTube Channels](https://github.com/mikeroyal/Self-Hosting-Guide#youtube-channels)
    - [Tutorials & Resources](https://github.com/mikeroyal/Self-Hosting-Guide#tutorials--resources)
    - [Useful Subreddits to Follow](https://github.com/mikeroyal/Self-Hosting-Guide#subreddits)

2. [WireGuard](https://github.com/mikeroyal/Self-Hosting-Guide#wireguard)
     * [What is WireGuard?](#what-is-wireguard)
     * [What is Tailscale?](#what-is-tailscale)
     * [What is Netmaker?](#what-is-netmaker)
     * [WireGuard Tools](#wireguard-tools)
     * [Setting up WireGuard with PiVPN](#setting-up-wireguard-with-pivpn)
     * [Setting up WireGuard on Unraid](#setting-up-wireguard-on-unraid)
     * [Setting up WireGuard on pfSense](#setting-up-wireguard-on-pfsense)
     * [Setting up WireGuard on OpenWRT](#setting-up-wireguard-on-openwrt)
     * [Setting up WireGuard on


Question: How to: Getting Started with Self-Hosting

Answer:
[Back to the Top](https://github.com/mikeroyal/Self-Hosting-Guide#table-of-contents)

[Self-Hosting](https://www.reddit.com/r/selfhosted/) is the practice of locally hosting(on premises & private web servers) and managing software applications by a person or organization instead of monthly subscriptions from [Software as a service (SaaS) providers](https://azure.microsoft.com/en-us/overview/what-is-saas/).

Most self-hosted software can be installed using [Docker](https://en.wikipedia.org/wiki/Docker_(software)), a packaging system which allows software to bundle their configuration and dependencies and isolate them from your operating system.  Software using docker can be installed using the command line or via graphical interfaces such as [Portainer](https://github.com/portainer/portainer).  Software is installed with Docker by downloading an image file containing the application, then creating a copy that sets up its own dependencies and configuration within what is called a container.  Without containers you would often need to install different versions of the same programming languages or tools to satisfy the dependencies for the software you want to use which can get complicated.


Question: How to: Tools for Self-Hosting

Answer: [Back to the Top](https://github.com/mikeroyal/Self-Hosting-Guide#table-of-contents)


Question: How to: Containers

Answer:
[Back to the Top](#table-of-contents)

**Container** is a standard unit of software that packages up code and all its dependencies(including CPU, memory, file storage, and network connections) so the application runs quickly and reliably from one computing environment to another.

  * [Application Container Security Guide | NIST (PDF)](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-190.pdf)

**Container Image** is a lightweight, standalone, executable package of software that includes everything needed to run an application such as the code, runtime, system tools, system libraries, and settings.

**Best places to get Container Images:**

 * [DockerHub Container Images](https://hub.docker.com/search?image_filter=official&q=&type=image)
 * [LinuxServer.io Container Images](https://fleet.linuxserver.io/)
 * [Quay Container Images](https://quay.io/search)

[Docker Compose](https://github.com/docker/compose) is a tool that was developed to help define and share multi-container applications. With Compose, we can create a YAML file to define the services and with a single command, can spin everything up or tear it all down.

[Docker Include](https://docs.docker.com/compose/compose-file/14-include/) is a Compose application can declare dependency on another Compose application. This is useful if you want to reuse other Compose files. Also, if you need to factor out parts of your application model into separate Compose files so they can be managed separately or shared with others.

[Kompose](https://kompose.io/) is a conversion tool for Docker Compose to container orchestrators such as [Kubernetes](https://kubernetes.io/) or [OpenShift](https://openshift.com/).

[SwarmKit](https://github.com/moby/swarmkit) is a toolkit for orchestrating distributed systems at any scale. It includes primitives for node discovery, raft-based consensus, task scheduling and more.

[Containerd](https://containerd.io/) is a daemon that manages the complete container lifecycle of its host system, from image transfer and storage to container execution and supervision to low-level storage to network attachments and beyond. It is available for Linux and Windows.

[ContainersSSH](https://containerssh.io/) is an SSH Server that Launches Containers in Kubernetes and Docker on demand.

[Podman](https://podman.io/) is a daemonless, open source, Linux native tool designed to make it easy to find, run, build, share and deploy applications using Open Containers Initiative (OCI) Containers and Container Images. Podman provides a command line interface (CLI) familiar to anyone who has used the Docker Container Engine.

[Lima](https://github.com/lima-vm/lima) is a tool that launches Linux virtual machines with automatic file sharing and port forwarding (similar to WSL2), and [containerd](https://containerd.io/). It's a great free and open-source alternative for [Docker Desktop](https://www.docker.com/products/docker-desktop).

[Colima](https://github.com/abiosoft/colima) is a container runtimes on macOS (and Linux) with minimal setup.

[Portainer Community Edition](https://github.com/portainer/portainer) is a lightweight service delivery platform for containerized applications that can be used to manage Docker, Swarm, Kubernetes and ACI environments. It is designed to be as simple to deploy as it is to use.

[Yacht](https://github.com/SelfhostedPro/Yacht) is a container management UI with a focus on templates and 1-click deployments.

[Kitematic](https://kitematic.com/) is a simple application for managing Docker containers on Mac, Linux and Windows letting you control your app containers from a graphical user interface (GUI).

[HashiCorp Nomad](https://www.nomadproject.io/) is a simple and flexible scheduler and orchestrator to deploy and manage containers and non-containerized applications across on-premises and clouds at scale.

[Open Container Initiative](https://opencontainers.org/about/overview/) is an open governance structure for the express purpose of creating open industry standards around container formats and runtimes.

[OpenNebula](https://opennebula.io/)  is an open source platform delivering a simple but feature-rich and flexible solution to build and manage enterprise clouds for virtualized services, containerized applications and serverless computing.

[Buildah](https://buildah.io/) is a command line tool to build Open Container Initiative (OCI) images. It can be used with Docker, Podman, Kubernetes.

[Red Hat Universal Base Images (UBI)](https://developers.redhat.com/products/rhel/ubi) is a tool that offers a way to build your container images on a foundation of Red Hat Enterprise Linux software. They are OCI-compliant, container-based, operating system images with complementary runtime languages and packages that are freely redistributable. Easily find UBI images in the Red Hat container catalog, and they are buildable and deployable anywhere.

[Red Hat Quay](https://quay.io/) is a project that Builds, Stores,


Question: How to: Development

Answer:
[Back to the Top](#table-of-contents)

[Proxmox VE(Virtual Environment)](https://www.proxmox.com/en/proxmox-ve) is an open-source platform for enterprise virtualization. It has a built-in web interface that you can use to easily manage VMs and containers, software-defined storage and networking, high-availability clustering, and multiple out-of-the-box tools on a single solution.

[Terraform provider plugin for Proxmox](https://github.com/Telmate/terraform-provider-proxmox) is a  Terraform provider for the [Proxmox virtualization platform](https://pve.proxmox.com/pve-docs/) and exposes Terraform resources to provision QEMU VMs and LXC Containers.

[OTF](https://github.com/leg100/otf) is an open source alternative to Terraform Enterprise. Includes SSO, team management, agents, and as many applies as you can throw hardware at.

[Semaphore UI](https://github.com/ansible-semaphore/semaphore) is a modern UI for Ansible. It lets you easily run Ansible playbooks, get notifications about fails, control access to deployment system.

[APITable](https://apitable.com/) is an API-oriented low-code platform for building collaborative apps and better than all other Airtable open-source alternatives.

[Chisel Kubernetes Operator](https://github.com/FyraLabs/chisel-operator/) is a Kubernetes operator for Chisel. It allows you to use Chisel as a LoadBalancer provider for your Kubernetes cluster, similar to [inlets-operator](https://github.com/inlets/inlets-operator).

[Docker-pgautoupgrade](https://github.com/pgautoupgrade/docker-pgautoupgrade) is a PostgreSQL Docker container that automatically upgrades your database. It's whole purpose in life is to automatically detect the version of PostgreSQL used in the existing PostgreSQL data directory, and automatically upgrade it (if needed) to the required version of PostgreSQL.

[IT-Tools](https://it-tools.tech/) is a collection of handy online tools for developers, with great UX.

[Lazygit](https://github.com/jesseduffield/lazygit) is a simple terminal UI for git commands, written in Go with the [gocui](https://github.com/jroimartin/gocui) library.

[LazyDocker](https://github.com/jesseduffield/lazydocker) is a  simple terminal UI for both docker and docker-compose, written in Go with the [gocui](https://github.com/jroimartin/gocui) library.

[Code-Server](https://github.com/coder/code-server) is Visual Studio Code running on a remote server, accessible through the browser.

[Turbopilot](https://github.com/ravenscroftj/turbopilot) is an open source large-language-model based code completion engine that runs locally on your CPU.

[Self-Hosted Sentry nightly](https://develop.sentry.dev/self-hosted/) is an official bootstrap for running your own Sentry with Docker. Sentry, feature-complete and packaged up for low-volume deployments and proofs-of-concept.

[Visual Studio Live Share](https://visualstudio.microsoft.com/services/live-share/) is a service/extension that enables you to collaboratively edit and debug with others in real time, regardless of the programming languages you're using or app types you're building. You can instantly and securely share your current project, start a joint debugging session, share terminal instances, forward localhost web apps, have voice calls, and more.

[GistPad](https://marketplace.visualstudio.com/items?itemName=vsls-contrib.gistfs) is a Visual Studio Code extension that allows you to edit GitHub Gists and repositories from the comfort of your favorite editor. You can open, create, delete, fork and star gists and repositories, and then seamlessly begin editing files as if they were local, without ever cloning, pushing or pulling anything.

[Live Server](https://marketplace.visualstudio.com/items?itemName=ritwickdey.LiveServer) is an extension for Visual Studio Code that launches a development local Server with live reload feature for static & dynamic pages.

[Gitea](https://gittea.dev/) is a community managed painless self-hosted Git service.

[Act](https://github.com/nektos/act) is a a tool to run your GitHub Actions locally.

[Act runner](https://gitea.com/gitea/act_runner) is a runner for Gitea based on [act](https://gitea.com/gitea/act).

[GitLab](https://about.gitlab.com/) is an open source end-to-end software development platform with built-in version control, issue tracking, code review, CI/CD, and more. Self-host GitLab on your own servers, in a container, or on a cloud provider.

[Bonobo Git Server](https://bonobogitserver.com/) - Set up your own self hosted git server on IIS for Windows. Manage users and have full control over your repositories with a nice user friendly graphical interface.

[Fossil](https://www.fossil-scm.org/index.html/doc/trunk/www/index.wiki) - Distributed version control system featuring wiki and bug tracker.

[Gerrit](https://www.gerritcodereview.com/) - A code review and project management tool for Git based projects.

[Gitblit](https://www.gitblit.com/) - Pure Java stack for managing, viewing, and servi


Question: How to: Web servers

Answer:
[Back to The Top](#table-of-contents)

**Web servers**

[Apache](https://httpd.apache.org/) - Most popular web server.

[OpenResty Manager](https://om.uusec.com/) - The easiest using, powerful and beautiful OpenResty Manager(Nginx Enhanced Version), open source alternative to OpenResty Edge.

[Beakon](https://github.com/RealDudePerson/beakon) - A self-host location sharing webserver. Beakon aims to leak as little data as possible and uses mostly self-contained libraries and local database files. Where possible, it will reference local files and not reach out over any network.

[Caddy](https://caddyserver.com/) - The HTTP/2 Web Server with Fully Managed TLS.

[Cherokee](https://cherokee-project.com/) - Lightweight, high-performance web server/reverse proxy.

[Lighttpd](https://www.lighttpd.net/) - Web server more optimized for speed-critical environments.

[Nginx](https://nginx.org/) - Reverse proxy, load balancer, HTTP cache, and web server.

[uWSGI](https://github.com/unbit/uwsgi/) - The uWSGI project aims at developing a full stack for building hosting services.

**Web Performance**

[HAProxy](https://www.haproxy.org/) - Software based load Balancing, SSL offloading and performance optimization, compression, and general web routing.

[Squid](https://www.squid-cache.org/) - Caching proxy for the web supporting HTTP, HTTPS, FTP, and more.

[Traefik](https://traefik.io/) - Taefik is a modern HTTP reverse proxy and load balancer made to deploy microservices with ease.

[Varnish](https://www.varnish-cache.org/) - HTTP based web application accelerator focusing on optimizing caching and compression.


Question: How to: Running Locally on Windows, MacOS, and Linux:

Answer:
**1. Clone Project Repo**

```bash
git clone https://github.com/mckaywrigley/chatbot-ui.git
```

**2. Install Dependencies**

```bash
npm i
```

**3. Provide OpenAI API Key**

Create a .env.local file in the root of the repo with your **[OpenAI API Key](https://platform.openai.com/account/api-keys)**:

```bash
OPENAI_API_KEY=YOUR_KEY
```

* **You can set `OPENAI_API_HOST` where access to the official OpenAI host is restricted or unavailable, allowing users to configure an alternative host for their specific needs.**

* **Additionally, if you have multiple OpenAI Organizations, you can set `OPENAI_ORGANIZATION` to specify one.**

**4. Run App**

```bash
npm run dev
```

**You done you should be able to start chatting with ChatGPT!**
Chatbot UI
[MiniGPT-4](https://minigpt-4.github.io/) is an enhancing Vision-language Understanding with Advanced Large Language Models

**Launching Demo Locally**

Try out the demo [demo.py](https://github.com/Vision-CAIR/MiniGPT-4/blob/main/demo.py) on your local machine by running

```python demo.py --cfg-path eval_configs/minigpt4_eval.yaml  --gpu-id 0```

Here, the demo loads Vicuna as 8 bit by default to save some GPU memory usage. Besides, the default beam search width is 1. Under this setting, the **demo cost about 23G GPU memory**. If you have a more powerful GPU with larger GPU memory, you can run the model in 16 bit by setting low_resource to False in the config file [minigpt4_eval.yaml](https://github.com/Vision-CAIR/MiniGPT-4/blob/main/eval_configs/minigpt4_eval.yaml) and use a larger beam search width.
MiniGPT-4 Demo
[GPT4All](https://github.com/nomic-ai/gpt4all) is an ecosystem of open-source chatbots trained on a massive collections of clean assistant data including code, stories and dialogue based on [LLaMa](https://github.com/facebookresearch/llama).
[GPT4All UI](https://github.com/nomic-ai/gpt4all-ui) is a Flask web application that provides a chat UI for interacting with the GPT4All chatbot.
[Alpaca.cpp](https://github.com/antimatter15/alpaca.cpp) is a fast ChatGPT-like model locally on your device. It combines the [LLaMA foundation model](https://github.com/facebookresearch/llama) with an [open reproduction](https://github.com/tloen/alpaca-lora) of [Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca) a fine-tuning of the base model to obey instructions (akin to the [RLHF](https://huggingface.co/blog/rlhf) used to train ChatGPT) and a set of modifications to [llama.cpp](https://github.com/ggerganov/llama.cpp) to add a chat interface.

[llama.cpp](https://github.com/ggerganov/llama.cpp) is a Port of Facebook's LLaMA model in C/C++.

[Serge](https://github.com/serge-chat/serge) is a web interface for chatting with Alpaca through llama.cpp. Fully self-hosted & dockerized, with an easy to use API.

[OpenPlayground](https://github.com/nat/openplayground) is a playfround for running ChatGPT-like models locally on your device.

[Vicuna](https://vicuna.lmsys.org/) is an open source chatbot trained by fine tuning LLaMA. It apparently achieves more than 90% quality of chatgpt and costs $300 to train.

[Yeagar ai](https://github.com/yeagerai/yeagerai-agent) is a Langchain Agent creator designed to help you build, prototype, and deploy AI-powered agents with ease.

[LocalAI](https://localai.io/) is a self-hosted, community-driven, local OpenAI-compatible API. Drop-in replacement for OpenAI running LLMs on consumer-grade hardware with no GPU required. It's an API to run ggml compatible models: llama, gpt4all, rwkv, whisper, vicuna, koala, gpt4all-j, cerebras, falcon, dolly, starcoder, and many others.

[DoctorGPT](https://github.com/ingyamilmolinar/doctorgpt) is a lightweight self-contained binary that monitors your application logs for problems and diagnoses them.

[HttpGPT](https://github.com/lucoiso/UEHttpGPT/releases) is an Unreal Engine 5 plugin that facilitates integration with OpenAI's GPT based services (ChatGPT and DALL-E) through asynchronous REST requests, making it easy for developers to communicate with these services. It also includes Editor Tools to integrate Chat GPT and DALL-E image generation directly in the Engine.


Question: How to: Automation

Answer:
[Back to the Top](#table-of-contents)

[Accelerated Text](https://github.com/accelerated-text/accelerated-text) - Automatically generate multiple natural language descriptions of your data varying in wording and structure.

[Activepieces](https://www.activepieces.com) - No-code business automation tool like Zapier or Tray. For example, you can send a Slack notification for each new Trello card.

[ActiveWorkflow](https://github.com/automaticmode/active_workflow) - An intelligent process and workflow automation platform based on software agents.

[Alltube](https://github.com/Rudloff/alltube) - Web GUI for youtube-dl, a program to download videos and audio from more than 100 websites.

[AmIUnique](https://amiunique.org/) - Learn how identifiable you are on the Internet (browser fingerprinting tool).

[Automatisch](https://automatisch.io) - Business automation tool that lets you connect different services like Twitter, Slack, and more to automate your business processes (Open source Zapier alternative).

[Baserow](https://baserow.io/) - Open source online database tool and Airtable alternative. Create your own database without technical experience.

[betanin](https://github.com/sentriz/betanin) - Music organization man-in-the-middle of your torrent client and music player. Based on beets.io, similar to Sonarr and Radarr.

[ChiefOnboarding](https://chiefonboarding.com) - Employee onboarding platform that allows you to provision user accounts and create sequences with todo items, resources, text/email/Slack messages, and more! Available as a web portal and Slack bot.

[Datasette](https://datasette.io/) - An open source multi-tool for exploring and publishing data, easy import and export and database management.

[Eonza](https://www.eonza.org) - Eonza is used to create scripts and automate tasks on servers or VPS hosting. Manage your servers from any browser on any device.

[Exadel CompreFace](https://exadel.com/solutions/compreface/) - Face recognition system that provides REST API for face recognition, face detection, and other face services, and is easily deployed with docker. There are SDKs for Python and JavaScript languages. Can be used without prior machine learning skills.

[feed2toot](https://feed2toot.readthedocs.io/en/latest/) - Feed2toot parses a RSS feed, extracts the last entries and sends them to Mastodon.

[feedmixer](https://github.com/cristoper/feedmixer) - FeedMixer is a WSGI (Python3) micro web service which takes a list of feed URLs and returns a new feed consisting of the most recent n entries from each given feed(Returns Atom, RSS, or JSON).

[Headphones](https://github.com/rembo10/headphones) - Automated music downloader for NZB and Torrent, written in Python. It supports SABnzbd, NZBget, Transmission, µTorrent, Deluge and Blackhole.

[Healthchecks](https://healthchecks.io/) - Django app which listens for pings and sends alerts when pings are late.

[HRConvert2](https://github.com/zelon88/HRConvert2) - Drag-and-drop file conversion server with session based authentication, automatic temporary file maintenance, and logging capability.

[Huginn](https://github.com/huginn/huginn) - Allows you to build agents that monitor and act on your behalf.

[Kibitzr](https://kibitzr.github.io) - Lightweight personal web assistant with powerful integrations.

[Krayin](https://krayincrm.com/) - Free and Opensource Laravel CRM Application.

[Leon](https://getleon.ai) - Open-source personal assistant who can live on your server.

[Lidarr](https://lidarr.audio/) - Lidarr is a music collection manager for Usenet and BitTorrent users.

[Matchering](https://github.com/sergree/matchering) - A containerized web app for automated music mastering. An open-source alternative to LANDR, eMastered, and MajorDecibel.

[Medusa](https://pymedusa.com/) - Automatic Video Library Manager for TV Shows. It watches for new episodes of your favorite shows, and when they are posted it does its magic. ([Source Code](https://github.com/pymedusa/Medusa)) `GPL-3.0` `Python`

[MeTube](https://github.com/alexta69/metube) - Web GUI for youtube-dl, with playlist support. Allows downloading videos from dozens of websites. `AGPL-3.0` `Python/Nodejs/Docker`

[Nautobot](https://github.com/nautobot/nautobot) is a Network Source of Truth and Network Automation Platform built as a web application atop the Django Python framework with a PostgreSQL or MySQL database.

[nefarious](https://github.com/lardbit/nefarious) - Web application that automates downloading Movies and TV Shows.

[NocoDB](https://www.nocodb.com/) - No-code platform that turns any database into a smart spreadsheet. It can be considered as an Airtable or Smartsheet alternative.

[OliveTin](https://github.com/OliveTin/OliveTin) - OliveTin is a web interface for running Linux shell commands.

[Patrowl](https://github.com/Patrowl/PatrowlManager) - Open Source, Smart and Scalable Security Operations Orchestration Platform.

[Podgrab](https://github.com/akhilrex/po


Question: How to: Configuration Management

Answer:
[Back to The Top](#table-of-contents)

[Ansible](https://www.ansible.com/) -  is a tool  is a powerful, agentless tool that works everywhere and with everything. When you add in proven enterprise engineering and support from Red Hat that's written in Python.

[Ansible.Ai](https://ansible.ai/) is an AI for Ansible Content Development tool to automate in your IT infrastructure and it will generate syntactically correct playbook to help you get there.

[CFEngine](https://cfengine.com/) - is a Lightweight agent system where the configuration state is specified via a declarative language.

[mgmt](https://github.com/purpleidea/mgmt) - is a next generation config management written in Go.

[Pallet](https://palletops.com/) - is a Infrastructure definition, configuration and management via a Clojure DSL.

[Puppet](https://puppetlabs.com/) - is an automated administrative engine for your Linux, Unix, and Windows systems, performs administrative tasks (such as adding users, installing packages, and updating server configurations) based on a centralized specification.

[Chef](https://www.opscode.com/chef/) - is a powerful automation platform that transforms infrastructure into code automating how infrastructure is configured, deployed and managed across any environment.

[(R)?ex](https://www.rexify.org/) - is a friendly automation framework to any combinations of local and remote execution, push and pull style of management, or imperative and declarative approach.

[Salt](https://www.saltstack.com/) -  is an event-driven automation tool and framework to deploy, configure, and manage complex IT systems. It automates common infrastructure administration tasks and ensure that all the components of your infrastructure are operating in a consistent desired state.

[Fleek](https://getfleek.dev/) is an all-in-one management system for everything you need to be productive on your computer.


Question: How to: Cloud Storage

Answer:
[Back to The Top](#table-of-contents)

[Swift](https://docs.openstack.org/developer/swift/) - A highly available, distributed, eventually consistent object/blob store.

[Syncthing](https://syncthing.net/) - Open Source system for private, encrypted and authenticated distribution of data.

[git-annex assistant](https://git-annex.branchable.com/assistant/) - A synchronized folder on each of your MacOS and Linux computers, Android devices, removable drives, NAS appliances, and cloud services.

[NextCloud](https://nextcloud.com) - Provides access to your files via the web.

[ownCloud](https://owncloud.org) - Provides universal access to your files via the web, your computer or your mobile devices.

[Seafile](https://seafile.com) - Another Open Source Cloud Storage solution.

[SparkleShare](https://sparkleshare.org/) - Provides cloud storage and file synchronization services. By default, it uses Git as a storage backend.


Question: How to: Back4app Web Deployment

Answer:
[Back4app Web Deployment](https://www.back4app.com/web-deployment-platform) is a Container as a Service (CaaS) provider platform that allows the dev teams to build and deploy containerized applications with no downtime. You can simply connect it to a GitHub repository and publish the code within seconds.

  * [Back4app Web Deployment Platform Pricing](https://www.back4app.com/pricing/container-as-a-service)

  * [Back4app GitHub](https://github.com/back4app)

  * [Back4app Tutorials](https://www.back4app.com/tutorials)


Question: How to: MinIO Object Storage

Answer:
[Back to the Top](#table-of-contents)
[MinIO](https://min.io/download) is a High Performance Object Storage released under GNU Affero General Public License v3.0. It is API compatible with [Amazon S3 cloud storage service](https://aws.amazon.com/s3/). Use MinIO to build high performance infrastructure for machine learning, analytics and application data workloads. It's one of the fastest object storage platforms globally, with a read/write speed of **183GB/s-171GB/s** if you use standard hardware. It can function as the main storage tier for many workloads like **Spark, TensorFlow, Presto, Hadoop HDFS, and H2O.**
MinIO UI
**Run the following command to run the latest stable image of MinIO as a container using an ephemeral data volume:**


Question: How to: Binary Download for MacOS

Answer:
```
wget https://dl.min.io/server/minio/release/darwin-amd64/minio
chmod +x minio
./minio server /data
```


Question: How to: Install from Source

Answer:
Use the following commands to compile and run a standalone MinIO server from source. Source installation is only intended for developers and advanced users. If you do not have a working Golang environment, please follow [How to install Golang](https://golang.org/doc/install). The minimum version required is [go1.19](https://golang.org/dl/#stable).

```go install github.com/minio/minio@latest```

**After you install MinIO:**

The MinIO deployment starts using default root credentials ```minioadmin:minioadmin```. You can test the deployment using the MinIO Console, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to ```http://127.0.0.1:9000``` and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.

When you run Minio you will be issued a key and a secret. These are used by the client or the web front-end to connect securely. I found my codes by typing in ```docker logs minio```.

```
Created minio configuration file at /root/.minio

Endpoint:  http://172.17.0.2:9000  http://127.0.0.1:9000
AccessKey: accessCode
SecretKey: secretCode
Region:    us-west-1
SQS ARNs:
Browser Access:
   http://172.17.0.2:9000  http://127.0.0.1:9000

Command-line Access: https://docs.minio.io/docs/minio-client-quickstart-guide
   $ mc config host add myminio http://172.17.0.2:9000 accessCode secretCode

Object API (Amazon S3 compatible):
   Go:         https://docs.minio.io/docs/golang-client-quickstart-guide
   Java:       https://docs.minio.io/docs/java-client-quickstart-guide
   Python:     https://docs.minio.io/docs/python-client-quickstart-guide
   JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide

Drive Capacity: 50 GiB Free, 70 GiB Total
```

If you'd like to learn more then most of the Minio client commands support a help flag or give info on the command line:

```
NAME:
  mc - Minio Client for cloud storage and filesystems.

USAGE:
  mc [FLAGS] COMMAND [COMMAND FLAGS | -h] [ARGUMENTS...]

COMMANDS:
  ls       List files and folders.
  mb       Make a bucket or a folder.
  cat      Display file and object contents.
  pipe     Redirect STDIN to an object or file or STDOUT.
  share    Generate URL for sharing.
  cp       Copy files and objects.
  mirror   Mirror buckets and folders.
  diff     Show differences between two folders or buckets.
  rm       Remove files and objects.
  events   Manage object notifications.
  watch    Watch for files and objects events.
  policy   Manage anonymous access to objects.
  session  Manage saved sessions for cp and mirror commands.
  config   Manage mc configuration file.
  update   Check for new mc update.
  version  Print version info.
  help, h  Shows a list of commands or help for one command
```


Question: How to: Advanced options

Answer:
You can have your client point to multiple Minio servers, which is really neat especially if you're working on a distributed team.

Minio's test-server called "play" is already configured in the default client, you can see all the servers you have configured with mc config host list.

**To upload the photo to Minio's "play" S3 server just type in:**

```# mc mb play/somebucketname```

```# mc cp ~/Downloads/IMG_2016120-25.jpg play/somebucketname```

**Recursive uploads:**

**If you want to test something larger out you could try uploading your entire Downloads photo, and then you should use the --recursive flag to make sure nothing's missed:**

```# mc cp --recursive ~/Downloads/IMG_2016120-25.jpg myminio/photos```


Question: How to: Remote Access

Answer:
[Back to the Top](#table-of-contents)

[FreeRDP](https://github.com/FreeRDP/FreeRDP) is a free remote desktop protocol library and clients.

[Rustdesk](https://rustdesk.com/) is an open source virtual/remote desktop infrastructure for everyone. Display and control your PC (Windows, macOS, and Linux) and Android devices.

[TinyPilot](https://tinypilotkvm.com/) is a tool that enables KVM over IP letting you control any computer remotely.

[X2Go](https://wiki.x2go.org/) is open source remote desktop software for Linux that uses a modified NX 3 protocol. It gives remote access to a Linux system's GUI.

[Apache Guacamole](https://guacamole.apache.org/) is a clientless remote desktop gateway. It supports standard protocols like VNC, RDP, and SSH.

[Remmina](https://remmina.org/) is a Remote access screen and file sharing to your desktop. It has Remote Access Protocol Plugins for [RDP](https://remmina.org/remmina-rdp/), [SSH](https://remmina.org/remmina-ssh/), [SPICE](https://remmina.org/remmina-spice/), [VNC](https://remmina.org/remmina-vnc/), [X2Go](https://remmina.org/remmina-x2go/), [HTTP/HTTPS](https://remmina.org/remmina-www/).

[Remotely](https://github.com/immense/Remotely) is a  remote control and remote scripting solution, built with .NET 6, Blazor, SignalR Core, and WebRTC.

[P2P Remote Desktop](https://github.com/miroslavpejic85/p2p) is a portable, no configuration or installation needed remote desktop tool.

[Cloudflare Tunnel](https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/install-and-setup/tunnel-guide) is a tunneling daemon that proxies traffic from the Cloudflare network to your origins. This daemon sits between Cloudflare network and your origin (a webserver). This attracts client requests and sends them to you via this daemon, without requiring you to poke holes on your firewall and your origin(webserver) can remain as closed as possible.

[WireGuard®](https://www.wireguard.com/) is a straight-forward, fast and modern VPN that utilizes state-of-the-art cryptography. It aims to be faster, simpler, leaner, and more useful than IPsec while avoiding the massive headache. WireGuard is designed as a general-purpose VPN for running on embedded interfaces and super computers alike, fit for many circumstances. It's cross-platform (Windows, macOS, BSD, iOS, Android) and widely deployable.

[NetBird](https://netbird.io/) is an open-source VPN management platform built on top of WireGuard® making it easy to create secure private networks for your organization or home.

[Tailscale](https://github.com/tailscale) is a WireGuard-based app that makes secure, private networks easy for teams of any scale. It works like an overlay network between the computers of your networks using all kinds of NAT traversal sorcery.

[Headscale](https://github.com/juanfont/headscale) is an open source, self-hosted implementation of the Tailscale coordination server.

[MeshCentral](https://meshcentral.com/) is a full computer management web site. It can run your own web server to remotely manage and control computers on a local network or anywhere on the internet. Once you get the server started, create device group and download and install an agent on each computer you want to manage.

[VNC Viewer](https://www.realvnc.com/en/connect/download/viewer/) is a free remote desktop application that use can use on your iPhone, iPad, Mac, Windows and Linux computers from anywhere in the world.

[TightVNC](https://www.tightvnc.com/) is a free remote desktop application. It can see the desktop of a remote machine and control it with your local mouse and keyboard, just like you would do it sitting in the front of that computer.

[KRDC](https://apps.kde.org/krdc/) is a client application that allows you to view or even control the desktop session on another machine that is running a compatible server. VNC and RDP is supported.

[Krfb Desktop Sharing](https://apps.kde.org/krfb/) is a server application that allows you to share your current session with a user on another machine, who can use a VNC client to view or even control the desktop.

[wayvnc](https://github.com/any1/wayvnc) is a VNC server for wlroots-based Wayland compositors (no_entry Gnome, KDE and Weston are not supported). It attaches to a running Wayland session, creates virtual input devices, and exposes a single display via the RFB protocol.

[Waypipe](https://gitlab.freedesktop.org/mstoeckl/waypipe/) is a proxy for Wayland clients. It forwards Wayland messages and serializes changes to shared memory buffers over a single socket.


Question: How to: Virtualization

Answer:
[Back to the Top](#table-of-contents)

[HVM (Hardware Virtual Machine)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/virtualization_types.html) is a virtualization type that provides the ability to run an operating system directly on top of a virtual machine without any modification, as if it were run on the bare-metal hardware.

[PV(ParaVirtualization)](https://wiki.xenproject.org/wiki/Paravirtualization_(PV)) is an efficient and lightweight virtualization technique introduced by the Xen Project team, later adopted by other virtualization solutions. PV does not require virtualization extensions from the host CPU and thus enables virtualization on hardware architectures that do not support Hardware-assisted virtualization.

[Network functions virtualization (NFV)](https://www.vmware.com/topics/glossary/content/network-functions-virtualization-nfv) is the replacement of network appliance hardware with virtual machines. The virtual machines use a hypervisor to run networking software and processes such as routing and load balancing. NFV allows for the separation of communication services from dedicated hardware, such as routers and firewalls. This separation means network operations can provide new services dynamically and without installing new hardware. Deploying network components with network functions virtualization only takes hours compared to months like with traditional networking solutions.

[Software Defined Networking (SDN)](https://www.vmware.com/topics/glossary/content/software-defined-networking) is an approach to networking that uses software-based controllers or application programming interfaces (APIs) to communicate with underlying hardware infrastructure and direct traffic on a network. This model differs from that of traditional networks, which use dedicated hardware devices (routers and switches) to control network traffic.

[Virtualized Infrastructure Manager (VIM)](https://www.cisco.com/c/en/us/td/docs/net_mgmt/network_function_virtualization_Infrastructure/3_2_2/install_guide/Cisco_VIM_Install_Guide_3_2_2/Cisco_VIM_Install_Guide_3_2_2_chapter_00.html) is a service delivery and reduce costs with high performance lifecycle management Manage the full lifecycle of the software and hardware comprising your NFV infrastructure (NFVI), and maintaining a live inventory and allocation plan of both physical and virtual resources.

[Management and Orchestration(MANO)](https://www.etsi.org/technologies/open-source-mano) is an ETSI-hosted initiative to develop an Open Source NFV Management and Orchestration (MANO) software stack aligned with ETSI NFV. Two of the key components of the ETSI NFV architectural framework are the NFV Orchestrator and VNF Manager, known as NFV MANO.

[Magma](https://www.magmacore.org/) is an open source software platform that gives network operators an open, flexible and extendable mobile core network solution. Their mission is to connect the world to a faster network by enabling service providers to build cost-effective and extensible carrier-grade networks. Magma is 3GPP generation (2G, 3G, 4G or upcoming 5G networks) and access network agnostic (cellular or WiFi). It can flexibly support a radio access network with minimal development and deployment effort.

[OpenRAN](https://open-ran.org/) is an intelligent Radio Access Network(RAN) integrated on general purpose platforms with open interface between software defined functions. Open RANecosystem enables enormous flexibility and interoperability with a complete openess to multi-vendor deployments.

[Open vSwitch(OVS)](https://www.openvswitch.org/)is an open source production quality, multilayer virtual switch licensed under the open source Apache 2.0 license. It is designed to enable massive network automation through programmatic extension, while still supporting standard management interfaces and protocols (NetFlow, sFlow, IPFIX, RSPAN, CLI, LACP, 802.1ag).

[Edge](https://www.ibm.com/cloud/what-is-edge-computing) is a distributed computing framework that brings enterprise applications closer to data sources such as IoT devices or local edge servers. This proximity to data at its source can deliver strong business benefits, including faster insights, improved response times and better bandwidth availability.

[Multi-access edge computing (MEC)](https://www.etsi.org/technologies/multi-access-edge-computing) is an Industry Specification Group (ISG) within ETSI to create a standardized, open environment which will allow the efficient and seamless integration of applications from vendors, service providers, and third-parties across multi-vendor Multi-access Edge Computing platforms.

[Virtualized network functions(VNFs)](https://www.juniper.net/documentation/en_US/cso4.1/topics/concept/nsd-vnf-overview.html) is a software application used in a Network Functions Virtualization (NFV) implementation that has well defined interfaces, and provides one or more component networking functions in a defined way. For example,


Question: How to: Password Management

Answer:
[Back to the Top](#table-of-contents)

[Bitwarden](https://bitwarden.com/host/) is a free and open-source password management service that stores sensitive information such as website credentials in an encrypted vault.

[Bitwarden Server](https://github.com/bitwarden/server) is a project contains the APIs, database, and other core infrastructure items needed for the "backend" of all bitwarden client applications. Checkout [Bitwarden's self-hosted release repository](https://github.com/bitwarden/self-host).

[Vaultwarden](https://github.com/dani-garcia/vaultwarden) is an unofficial Bitwarden compatible server written in Rust, formerly known as bitwarden_rs.

[Passbolt](https://www.passbolt.com/) is an open-source/self-hosted password manager for teams. It allows you to securely share and store credentials. For instance, the wifi password of your office, the administrator password of a router or your organization's social media account passwords, all of them can be secured using passbolt.

[KeePassXC](https://keepassxc.org/) is a modern, secure, and open-source password manager that stores and manages your most sensitive information. You can run KeePassXC on Windows, macOS, and Linux systems. It saves many different types of information, such as usernames, passwords, URLs, attachments, and notes in an offline, encrypted file that can be stored in any location, including private and public cloud solutions.

[AuthPass.app](https://authpass.app/) is an Open-Source Password Manager for mobile and desktop that is Keepass 2.x (kdbx 3.x) compatible.

[pass](https://www.passwordstore.org/) is an open-source unix-based password utilitiy with various [gui clients](https://www.passwordstore.org/#other)


Question: How to: Log Management

Answer:
[Back to The Top](#table-of-contents)

[Echofish](https://echothrust.github.io/echofish/) - A web based real-time event log aggregation, analysis, monitoring and management system.

[Fluentd](https://www.fluentd.org/) - Log Collector and Shipper.

[Flume](https://flume.apache.org/) - Distributed log collection and aggregation system.

[Graylog2](https://graylog2.org/) - Pluggable Log and Event Analysis Server with Alerting options.

[Heka](https://hekad.readthedocs.org/en/latest/) - Stream processing system which may be used for log aggregation.

[Elasticsearch](https://www.elasticsearch.org/) - A Lucene Based Document store mainly used for log indexing, storage and analysis.

[Kibana](https://www.elasticsearch.org/overview/kibana/) - Visualize logs and time-stamped data.

[Logstash](https://logstash.net/) - Tool for managing events and logs.

[Octopussy](https://www.octopussy.pm) - Log Management Solution (Visualize/Alert/Report).


Question: How to: Service Discovery

Answer:
[Back to The Top](#table-of-contents)

[Consul](http://www.consul.io/)  is a tool for service discovery, monitoring and configuration. [Install Consul on Self-Hosted Kubernetes Clusters](https://github.com/hashicorp/consul/blob/main/website/content/docs/k8s/platforms/self-hosted-kubernetes.mdx).

[Linkerd](https://linkerd.io/) is an ultralight, security-first service mesh for Kubernetes. Linkerd adds critical security, observability, and reliability features to your Kubernetes stack with no code change required.

[Doozerd](https://github.com/ha/doozerd) is a highly-available, completely consistent store for small amounts of extremely important data.

[Admiral](https://github.com/istio-ecosystem/admiral) is a tool for for service discovery that provides automatic configuration and service discovery for multicluster Istio service mesh.

[ScaleCube](https://github.com/scalecube/scalecube-services) is a library that simplifies the development of reactive and distributed applications by providing an embeddable microservices library. It connects distributed microservices in a way that resembles a fabric when viewed collectively. It greatly simplifies and streamlines asynchronous programming and provides a tool-set for managing microservices architecture.

[DPS(dns-proxy-server)](https://github.com/mageddo/dns-proxy-server) is a lightweight end user (Developers, Server Administrators) DNS server tool for service discovery, which make it easy to develop in systems where one hostname can solve to different IPs based on the configured environment, so you can:

 * Solve hostnames from local configuration database.
 * Solve hostnames from docker containers using docker hostname option or HOSTNAMES env.
 * Solve hostnames from a list of configured remote DNS servers(as a proxy) if no answer of two above  .
 * Graphic interface to Create/List/Update/Delete A/CNAME records.
 * Solve host machine IP using host.docker hostname.
 * Access container by its container name / service name.
 * Specify from which network solve container IP.

[ZooKeeper](http://zookeeper.apache.org/)  is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services.


Question: How to: Troubleshooting

Answer:
[Back to The Top](#table-of-contents)

[NETworkManager](https://github.com/BornToBeRoot/NETworkManager) - A powerful tool for managing networks and troubleshoot network problems. It contains features like a WiFi analyzer, IP scanner, port scanner, ping monitor, traceroute, DNS lookup or a LLDP/CDP capture.

[Wireshark](https://www.wireshark.org/) - The world's foremost network protocol analyzer.

[Selfspy](https://github.com/selfspy/selfspy) is a daemon for Unix/X11, MacOS (thanks to @ljos) and Windows (thanks to @Foxboron), that continuously monitors and stores what you are doing on your computer. This way, you can get all sorts of nifty statistics and reminders on what you have been up to.

[Cilium](https://github.com/cilium/cilium) - A networking, observability, and security solution with an eBPF-based dataplane. It provides a simple flat Layer 3 network with the ability to span multiple clusters in either a native routing or overlay mode.

[Netshoot](https://github.com/nicolaka/netshoot) - A  Docker + Kubernetes network trouble-shooting swiss-army container.

[Kubevious](https://kubevious.io/) - A suite of app-centric assurance, validation, and introspection products for Kubernetes. It helps running modern Kubernetes applications without disasters and costly outages by continuously validating application manifests, cluster state, and configuration.

[HOMER](https://github.com/sipcapture/homer) - A robust, carrier-grade, scalable Packet and Event capture system and VoiP/RTC Monitoring Application based on the HEP/EEP protocol and ready to process & store insane amounts of signaling, rtc events, logs and statistics with instant search, end-to-end analysis and drill-down capabilities.

[mitmproxy](https://mitmproxy.org/) - A Python tool used for intercepting, viewing and modifying network traffic. Invaluable in troubleshooting certain problems.

[Sysdig](https://www.sysdig.org/) - Capture system state and activity from a running Linux instance, then save, filter and analyze.

[Sysdig Inspect](https://github.com/draios/sysdig-inspect) - A powerful opensource interface for container troubleshooting and security investigation.