Skip to content

Instantly share code, notes, and snippets.

@justinhartman
Last active January 17, 2024 00:46
Show Gist options
  • Save justinhartman/73c9266fbb5b01d1d9031263c695be66 to your computer and use it in GitHub Desktop.
Save justinhartman/73c9266fbb5b01d1d9031263c695be66 to your computer and use it in GitHub Desktop.

Revisions

  1. justinhartman revised this gist Aug 31, 2021. 5 changed files with 37 additions and 22 deletions.
    1 change: 1 addition & 0 deletions 03_install-lemp.md
    Original file line number Diff line number Diff line change
    @@ -42,6 +42,7 @@ $ sudo apt -y install lsb-release apt-transport-https ca-certificates wget
    $ wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
    $ echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/php.list
    $ sudo apt update
    $ sudo apt install curl unzip zip
    $ sudo apt -y install php8.0-{fpm,cli,apc,bcmath,bz2,curl,gd,imagick,intl,memcached,mbstring,mysql,redis,xml,zip}
    $ sudo systemctl start php8.0-fpm
    $ sudo systemctl enable php8.0-fpm
    6 changes: 5 additions & 1 deletion 04_configure-php-nginx.md
    Original file line number Diff line number Diff line change
    @@ -81,6 +81,10 @@ emergency_restart_interval=1m
    process_control_timeout=10s
    ```

    ```console
    $ sudo systemctl reload php8.0-fpm
    ```

    ## Configure nginx

    ```console
    @@ -90,7 +94,7 @@ $ sudo nano /etc/nginx/nginx.conf
    Change these values:

    ```nginx
    client_max_body_size 128M;
    client_max_body_size 256M;
    keepalive_timeout 2;
    server_tokens off;
    ```
    27 changes: 20 additions & 7 deletions 06_fine-tune-mysql.md
    Original file line number Diff line number Diff line change
    @@ -23,6 +23,19 @@ $ ulimit -Hn
    65535
    ```

    Alternatively, you could set system-wide file and process limits with the following configuration:

    ```ini
    # /etc/security/limits.conf
    # Added to increase server limit of 1024 per process.
    * soft nofile 102400
    * hard nofile 102400
    * soft nproc 10240
    * hard nproc 10240
    ```

    **NB: Your milage may vary so please only use the above with caution!**

    ## Setting Swappiness on Linux for MariaDB

    ```console
    @@ -61,20 +74,20 @@ $ sudo nano /etc/mysql/my.cnf
    ```ini
    # /etc/mysql/my.cnf
    [mysqld]
    open_files_limit = 102400
    innodb_buffer_pool_size = 3G
    # innodb_buffer_pool_instances = 4 (default is 1)
    innodb_flush_log_at_trx_commit = 0
    innodb_flush_method = O_DIRECT

    innodb_log_file_size = 256M
    max_connections = 500
    # (50 + max_connections/5)
    back_log = 150
    max_allowed_packet = 32M
    innodb_log_file_size = 1G
    max_connections = 5000
    # back_log = (max_connections/5) + 50
    back_log = 1050
    max_allowed_packet = 256M
    # Disable query cache
    query_cache_type = OFF
    query_cache_size = 0
    slow_query_log = ON
    long_query_time = 1
    # slow_query_log_file = /var/lib/mysql/hostname-slow.log
    slow_query_log_file = /var/lib/mysql/slow.log
    ```
    9 changes: 6 additions & 3 deletions 11_install-redis-memcached.md
    Original file line number Diff line number Diff line change
    @@ -4,6 +4,8 @@

    ```console
    $ sudo apt install redis-server
    $ sudo systemctl start redis-server
    $ sudo systemctl enable redis-server
    ```

    ### Edit Configuration
    @@ -24,15 +26,16 @@ bind 127.0.0.1 ::1
    ### Restart Redis

    ```console
    $ sudo systemctl restart redis
    $ sudo systemctl status redis
    $ sudo systemctl restart redis-server
    $ sudo systemctl status redis-server
    ```

    ## Install Memcached

    ```console
    $ sudo apt install memcached libmemcached-tools
    $ sudo systemctl status memcached
    $ sudo systemctl start memcached
    $ sudo systemctl enable memcached
    ```

    ### Edit Configuration
    16 changes: 5 additions & 11 deletions 12_install-composer-node-npm.md
    Original file line number Diff line number Diff line change
    @@ -16,14 +16,8 @@ $ php -r "unlink('composer-setup.php');"
    Composer will be installed at `/usr/local/bin/composer` and you can test the installation as follows:

    ```console
    $ /usr/local/bin/composer -v
    ______
    / ____/___ ____ ___ ____ ____ ________ _____
    / / / __ \/ __ `__ \/ __ \/ __ \/ ___/ _ \/ ___/
    / /___/ /_/ / / / / / / /_/ / /_/ (__ ) __/ /
    \____/\____/_/ /_/ /_/ .___/\____/____/\___/_/
    /_/
    Composer version 2.0.8 2020-12-03 17:20:38
    $ /usr/local/bin/composer -V
    Composer version 2.1.6 2021-08-19 17:11:08
    ```

    If you get an error output stating `Installer corrupt`, it means the version of composer has been updated and you'll need a new hash. Visit the official [getcomposer.org download page][compser] to get the latest `hash_file` string and/or install via that method.
    @@ -43,9 +37,9 @@ Test the installation:

    ```console
    $ node -v
    v12.20.0
    v12.22.5
    $ npm -v
    6.14.8
    6.14.14
    ```

    ### Install Yarn
    @@ -62,7 +56,7 @@ You can test it out with:

    ```console
    $ yarn -v
    1.22.10
    1.22.5
    ```

    ### Optional
  2. justinhartman revised this gist Aug 26, 2021. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion 05_fine-tune-php-fpm.md
    Original file line number Diff line number Diff line change
    @@ -98,7 +98,7 @@ pm.max_children = 222
    pm.start_servers = 16
    ; CPU * 2 = 8
    pm.min_spare_servers = 8
    ; Same as children
    ; Same as start_servers
    pm.max_spare_servers = 16
    pm.max_requests = 500
    ```
  3. justinhartman revised this gist Aug 26, 2021. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion 05_fine-tune-php-fpm.md
    Original file line number Diff line number Diff line change
    @@ -82,7 +82,7 @@ $ sudo nano /etc/php/7.4/fpm/pool.d/www.conf
    [www]
    user = www-data
    group = www-data
    # listen = /run/php/php7.4-fpm.sock
    ; listen = /run/php/php7.4-fpm.sock
    listen = /run/php/php8.0-fpm.sock
    listen.owner = www-data
    listen.group = www-data
  4. justinhartman revised this gist Aug 26, 2021. 3 changed files with 7 additions and 7 deletions.
    6 changes: 3 additions & 3 deletions 02_general.md
    Original file line number Diff line number Diff line change
    @@ -1,6 +1,6 @@
    # General Setup

    These are a series of specific commands to configure the Ubuntu server for our environment. You may not need to do these.
    These are a series of specific commands to configure the Debian server for our environment. You may not need to do these.

    ```console
    $ sudo apt-get install locales-all
    @@ -21,7 +21,7 @@ export LC_TERMINAL=en_GB.UTF-8
    ## Change Timezone

    ```console
    $ sudo cp /usr/share/zoneinfo/Africa/Johannesburg /etc/localtime # set time
    $ sudo cp /usr/share/zoneinfo/Europe/London /etc/localtime # set time
    ```

    ## Change Hostname
    @@ -37,6 +37,6 @@ $ hostnamectl
    Boot ID: dae700b6e5bb44fcb5d4683f05f30919
    Virtualization: vmware
    Operating System: Debian GNU/Linux 10 (buster)
    Kernel: Linux 4.19.0-12-amd64
    Kernel: Linux 4.19.0-17-amd64
    Architecture: x86-64
    ```
    6 changes: 3 additions & 3 deletions 03_install-lemp.md
    Original file line number Diff line number Diff line change
    @@ -34,7 +34,7 @@ mysql> exit

    ## Step 3 - Install PHP-FPM

    _I've chosen to install PHP 8.0 but you can change any of the commands from `php7.4` to `php8.0` if needed. For example any reference to `php7.4-fpm` can be adjusted simply to `php8.0-fpm`._
    _I've chosen to install PHP 8.0 but you can change any of the commands from `php8.0` to `php7.4` as needed. For example any reference to `php8.0-fpm` can be adjusted simply to `php7.4-fpm`._

    ```console
    $ sudo apt-get update
    @@ -43,6 +43,6 @@ $ wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
    $ echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/php.list
    $ sudo apt update
    $ sudo apt -y install php8.0-{fpm,cli,apc,bcmath,bz2,curl,gd,imagick,intl,memcached,mbstring,mysql,redis,xml,zip}
    $ sudo systemctl start php8.0-fpm # or replace with php7.4-fpm
    $ sudo systemctl enable php8.0-fpm # php7.4-fpm
    $ sudo systemctl start php8.0-fpm
    $ sudo systemctl enable php8.0-fpm
    ```
    2 changes: 1 addition & 1 deletion 04_configure-php-nginx.md
    Original file line number Diff line number Diff line change
    @@ -20,7 +20,7 @@ Change these values:
    cgi.fix_pathinfo=0
    post_max_size=256M
    upload_max_filesize=256M
    date.timezone=Africa/Johannesburg
    date.timezone=Europe/London
    [opcache]
    opcache.memory_consumption=256
    opcache.max_accelerated_files=50000
  5. justinhartman revised this gist Aug 1, 2021. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion 07_remove_systemd_limits.md
    Original file line number Diff line number Diff line change
    @@ -50,7 +50,7 @@ TasksMax=infinity
    ## Nginx

    ```console
    $ sudo nano /lib/systemd/system/redis-server.service
    $ sudo nano /lib/systemd/system/nginx.service
    [Service]
    TasksMax=infinity
    ```
  6. justinhartman revised this gist Aug 1, 2021. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion 01_introduction.md
    Original file line number Diff line number Diff line change
    @@ -27,7 +27,7 @@ These documents take you through each step of the configuration from:
    [config]: #file-04_configure-php-nginx-md
    [fpm-tune]: #file-05_fine-tune-php-fpm-md
    [mysql]: #file-06_fine-tune-mysql-md
    [systemd]: #file-07_remove-systemd-process-limits-md
    [systemd]: #file-07_remove_systemd_limits-md
    [vhost]: #file-08_new-vhost-md
    [ssl]: #file-09_setup-ssl-md
    [ssh]: #file-10_ssh-remote-login-md
  7. justinhartman revised this gist Aug 1, 2021. 8 changed files with 88 additions and 6 deletions.
    12 changes: 7 additions & 5 deletions 01_introduction.md
    Original file line number Diff line number Diff line change
    @@ -15,6 +15,7 @@ These documents take you through each step of the configuration from:
    1. [Confiure PHP and Nginx][config]
    1. [How to fine-tune PHP-FPM][fpm-tune]
    1. [Fine-tune MySQL/MariaDB for Performance][mysql]
    1. [Remove systemd process limits][systemd]
    1. [Create a New Nginx Host][vhost]
    1. [Setup Let's Encrypt SSL certificates][ssl]
    1. [Setup Remote SSH Login with sudo][ssh]
    @@ -26,8 +27,9 @@ These documents take you through each step of the configuration from:
    [config]: #file-04_configure-php-nginx-md
    [fpm-tune]: #file-05_fine-tune-php-fpm-md
    [mysql]: #file-06_fine-tune-mysql-md
    [vhost]: #file-07_new-vhost-md
    [ssl]: #file-08_setup-ssl-md
    [ssh]: #file-09_ssh-remote-login-md
    [redis]: #file-10_install-redis-memcached-md
    [composer]: #file-11_install-composer-node-npm-md
    [systemd]: #file-07_remove-systemd-process-limits-md
    [vhost]: #file-08_new-vhost-md
    [ssl]: #file-09_setup-ssl-md
    [ssh]: #file-10_ssh-remote-login-md
    [redis]: #file-11_install-redis-memcached-md
    [composer]: #file-12_install-composer-node-npm-md
    4 changes: 3 additions & 1 deletion 06_fine-tune-mysql.md
    Original file line number Diff line number Diff line change
    @@ -10,6 +10,8 @@ To increase this, edit `/etc/security/limits.conf` and specify or add the follow
    # Added to increase server limit of 1024 per process.
    mysql soft nofile 65535
    mysql hard nofile 65535
    # Uncomment this if ulimit doesn't change from 1024
    #* soft nofile 65535
    ```

    This requires a system restart. Once rebooted, check the limit with:
    @@ -75,4 +77,4 @@ query_cache_size = 0
    slow_query_log = ON
    long_query_time = 1
    # slow_query_log_file = /var/lib/mysql/hostname-slow.log
    ```
    ```
    78 changes: 78 additions & 0 deletions 07_remove_systemd_limits.md
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,78 @@
    # Remove systemd process limits

    There are now also some issues with `systemd` limiting file processes. Have a look at the following:

    ```console
    $ sudo systemctl status redis-server
    ● redis-server.service - Advanced key-value store
    Loaded: loaded (/lib/systemd/system/redis-server.service; enabled; vendor preset: enabled)
    Active: active (running) since Sun 2021-08-01 23:24:48 BST; 13min ago
    Docs: http://redis.io/documentation,
    man:redis-server(1)
    Main PID: 571 (redis-server)
    Tasks: 4 (limit: 4915)
    Memory: 9.7M
    CGroup: /system.slice/redis-server.service
    └─571 /usr/bin/redis-server 127.0.0.1:6379
    ```

    If you see the `Tasks: 4 (limit: 4915)` has a limit set of 4915. This is how the Linux Kernel works and it defaults to 15%,
    which equals `4915` with the kernel's defaults on the host.

    In order to check what your system is capable of it's a rather easy calculation.

    ```console
    $ more /proc/sys/kernel/pid_max
    32768
    ```

    You now take this number and work out 15% of the value, i.e. `32768*15/100 = 4915.2` and you can see how this is calculated.

    That said, you can override this for certain Debian-based services and here's how we change them for those services.

    ## MariaDB

    ```console
    $ sudo nano /etc/systemd/system/mysqld.service
    [Service]
    TasksMax=infinity
    LimitNOFILE=65535
    ```

    ## PHP

    ```console
    $ sudo nano /lib/systemd/system/php8.0-fpm.service
    [Service]
    TasksMax=infinity
    ```

    ## Nginx

    ```console
    $ sudo nano /lib/systemd/system/redis-server.service
    [Service]
    TasksMax=infinity
    ```

    ## Redis

    ```console
    $ sudo nano /lib/systemd/system/redis-server.service
    [Service]
    TasksMax=infinity
    ```

    ## Memcached

    ```console
    $ sudo nano /lib/systemd/system/memcached.service
    [Service]
    TasksMax=infinity
    ```

    ## Restarting systemctl

    When you're done making these changes you have to restart systemctl with `sudo systemctl daemon-reload` in order for the file
    limits to be removed.

    File renamed without changes.
    File renamed without changes.
    File renamed without changes.
  8. justinhartman revised this gist Mar 24, 2021. 1 changed file with 4 additions and 2 deletions.
    6 changes: 4 additions & 2 deletions 11_install-composer-node-npm.md
    Original file line number Diff line number Diff line change
    @@ -50,10 +50,12 @@ $ npm -v

    ### Install Yarn

    With NVM installed you can install `yarn` with the following command:
    To install the Yarn package manager, run:

    ```console
    $ npm install -g yarn
    $ curl -sL https://dl.yarnpkg.com/debian/pubkey.gpg | gpg --dearmor | sudo tee /usr/share/keyrings/yarnkey.gpg >/dev/null
    $ echo "deb [signed-by=/usr/share/keyrings/yarnkey.gpg] https://dl.yarnpkg.com/debian stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
    $ sudo apt-get update && sudo apt-get install yarn
    ```

    You can test it out with:
  9. justinhartman revised this gist Feb 17, 2021. 1 changed file with 97 additions and 2 deletions.
    99 changes: 97 additions & 2 deletions 08_setup-ssl.md
    Original file line number Diff line number Diff line change
    @@ -70,10 +70,105 @@ $ sudo nginx -t && sudo systemctl reload nginx

    ## Renew Certificates

    Configure Certbot to reload NGINX when it successfully renews certificates. Run the following commands:
    You will now need to create a post hook script for certbot so that it can reload Nginx once a domain has been renewed. Luckily I have created a script for you to do this and you can [access the script over here](https://git.io/JtMvt). Setting up this script as is a simple 3 step process.

    ### Step 1

    First create the `nginx-reload.sh` post hook file:

    ```console
    $ sudo nano /etc/letsencrypt/renewal-hooks/post/nginx-reload.sh
    ```

    ### Step 2

    You are going to want to copy the [code of the script by clicking here](https://git.io/JtMvt) and head on over and paste it in your `/etc/letsencrypt/renewal-hooks/post/nginx-reload.sh` file.

    ### Step 3

    Once you have the file saved, make it executable and reload nginx.

    ```console
    $ echo -e '#!/bin/bash\nnginx -t &>/dev/null && systemctl reload nginx' | sudo tee /etc/letsencrypt/renewal-hooks/post/nginx-reload.sh
    $ sudo chmod a+x /etc/letsencrypt/renewal-hooks/post/nginx-reload.sh
    $ sudo nginx -t && sudo systemctl reload nginx
    ```

    Done. The script is now setup to begin working automatically in the background. Certbot runs a cronjob twice daily and if a certificate is renewed your newly created post hook script will be executed and run.

    ### Success Output

    Executing the `nginx-reload.sh` file without any errors in any of your `nginx` config or virtual host files will output the following success messages to your console:

    ```console
    $ sudo /etc/letsencrypt/renewal-hooks/post/nginx-reload.sh
    Removing old log file.
    Success: Reloading nginx server.
    ```

    Similarly, when the `certbot` cronjob executes and runs this post hook script the following successful output will be seen.

    ```console
    $ sudo certbot renew
    Saving debug log to /var/log/letsencrypt/letsencrypt.log
    Processing /etc/letsencrypt/renewal/example.com.conf
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Plugins selected: Authenticator webroot, Installer None
    Renewing an existing certificate
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    new certificate deployed without reload, fullchain is
    /etc/letsencrypt/live/example.com/fullchain.pem
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Congratulations, all renewals succeeded. The following certs have been renewed:
    /etc/letsencrypt/live/example.com/fullchain.pem (success)
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Running post-hook command: /etc/letsencrypt/renewal-hooks/post/nginx-reload.sh
    Output from nginx-reload.sh:
    Removing old log file.
    Success: Reloading nginx server.
    ```

    ### Error Output

    Executing the `nginx-reload.sh` file with an error in either your `nginx` config or virtual hosts files will output the following error and log file:

    ```console
    $ sudo /etc/letsencrypt/renewal-hooks/post/nginx-reload.sh
    Removing old log file.
    Fail: Error with config file.
    Aborting nginx restart.
    Log file output:
    nginx: [emerg] "worker_connections" directive is not allowed here in /etc/nginx/nginx.conf:7
    nginx: configuration file /etc/nginx/nginx.conf test failed
    ```

    Similarly, when the `certbot` cronjob executes and runs this post hook script the following output will be seen.

    ```console
    $ sudo certbot renew
    Saving debug log to /var/log/letsencrypt/letsencrypt.log
    Processing /etc/letsencrypt/renewal/example.com.conf
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Plugins selected: Authenticator webroot, Installer None
    Renewing an existing certificate
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    new certificate deployed without reload, fullchain is
    /etc/letsencrypt/live/example.com/fullchain.pem
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Congratulations, all renewals succeeded. The following certs have been renewed:
    /etc/letsencrypt/live/example.com/fullchain.pem (success)
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Running post-hook command: /etc/letsencrypt/renewal-hooks/post/nginx-reload.sh
    Output from nginx-reload.sh:
    Removing old log file.
    Fail: Error with config file.
    Aborting nginx restart.
    Log file output:
    nginx: [emerg] "worker_connections" directive is not allowed here in /etc/nginx/nginx.conf:7
    nginx: configuration file /etc/nginx/nginx.conf test failed

    Hook command "/etc/letsencrypt/renewal-hooks/post/nginx-reload.sh" returned error code 1
    ```
  10. justinhartman revised this gist Feb 17, 2021. 6 changed files with 26 additions and 5 deletions.
    4 changes: 2 additions & 2 deletions 03_install-lemp.md
    Original file line number Diff line number Diff line change
    @@ -43,6 +43,6 @@ $ wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
    $ echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/php.list
    $ sudo apt update
    $ sudo apt -y install php8.0-{fpm,cli,apc,bcmath,bz2,curl,gd,imagick,intl,memcached,mbstring,mysql,redis,xml,zip}
    $ sudo systemctl start php8.0-fpm
    $ sudo systemctl enable php8.0-fpm
    $ sudo systemctl start php8.0-fpm # or replace with php7.4-fpm
    $ sudo systemctl enable php8.0-fpm # php7.4-fpm
    ```
    12 changes: 12 additions & 0 deletions 04_configure-php-nginx.md
    Original file line number Diff line number Diff line change
    @@ -8,6 +8,12 @@ These are the different configuration files for setting up the servers.
    $ sudo nano /etc/php/8.0/fpm/php.ini
    ```

    OR for PHP 7.4:

    ```console
    $ sudo nano /etc/php/7.4/fpm/php.ini
    ```

    Change these values:

    ```ini
    @@ -63,6 +69,12 @@ $ sudo systemctl reload php8.0-fpm
    $ sudo nano /etc/php/8.0/fpm/php-fpm.conf
    ```

    OR for PHP 7.4:

    ```console
    $ sudo nano /etc/php/7.4/fpm/php-fpm.conf
    ```

    ```nginx
    emergency_restart_threshold=10
    emergency_restart_interval=1m
    9 changes: 8 additions & 1 deletion 05_fine-tune-php-fpm.md
    Original file line number Diff line number Diff line change
    @@ -72,10 +72,17 @@ Change the `www` pool so it now has the following settings.
    $ sudo nano /etc/php/8.0/fpm/pool.d/www.conf
    ```

    OR for PHP 7.4:

    ```console
    $ sudo nano /etc/php/7.4/fpm/pool.d/www.conf
    ```

    ```ini
    [www]
    user = www-data
    group = www-data
    # listen = /run/php/php7.4-fpm.sock
    listen = /run/php/php8.0-fpm.sock
    listen.owner = www-data
    listen.group = www-data
    @@ -99,7 +106,7 @@ pm.max_requests = 500
    Reload PHP:

    ```console
    $ sudo systemctl reload php8.0-fpm
    $ sudo systemctl reload php8.0-fpm # or replace with php7.4-fpm
    ```

    [tuning]: https://tideways.com/profiler/blog/an-introduction-to-php-fpm-tuning
    1 change: 1 addition & 0 deletions 06_fine-tune-mysql.md
    Original file line number Diff line number Diff line change
    @@ -57,6 +57,7 @@ $ sudo nano /etc/mysql/my.cnf
    ```

    ```ini
    # /etc/mysql/my.cnf
    [mysqld]
    innodb_buffer_pool_size = 3G
    # innodb_buffer_pool_instances = 4 (default is 1)
    1 change: 1 addition & 0 deletions 07_new-vhost.md
    Original file line number Diff line number Diff line change
    @@ -19,6 +19,7 @@ server {
    location ~ \.php$ {
    include snippets/fastcgi-php.conf;
    # fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
    fastcgi_pass unix:/var/run/php/php8.0-fpm.sock;
    }
    4 changes: 2 additions & 2 deletions 08_setup-ssl.md
    Original file line number Diff line number Diff line change
    @@ -70,10 +70,10 @@ $ sudo nginx -t && sudo systemctl reload nginx

    ## Renew Certificates

    Configure Certbot to reload NGINX when it successfully renews certificates:
    Configure Certbot to reload NGINX when it successfully renews certificates. Run the following commands:

    ```console
    $ echo -e '#!/bin/bash\nnginx -t && systemctl reload nginx' | sudo tee /etc/letsencrypt/renewal-hooks/post/nginx-reload.sh
    $ echo -e '#!/bin/bash\nnginx -t &>/dev/null && systemctl reload nginx' | sudo tee /etc/letsencrypt/renewal-hooks/post/nginx-reload.sh
    $ sudo chmod a+x /etc/letsencrypt/renewal-hooks/post/nginx-reload.sh
    $ sudo nginx -t && sudo systemctl reload nginx
    ```
  11. justinhartman revised this gist Feb 17, 2021. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion 03_install-lemp.md
    Original file line number Diff line number Diff line change
    @@ -34,7 +34,7 @@ mysql> exit

    ## Step 3 - Install PHP-FPM

    _I've chosen to install PHP 8.0 but you can change any of the commands from `php8.0` to `php8.0` if needed. For example any reference to `php8.0-fpm` can be adjusted simply to `php8.0-fpm`._
    _I've chosen to install PHP 8.0 but you can change any of the commands from `php7.4` to `php8.0` if needed. For example any reference to `php7.4-fpm` can be adjusted simply to `php8.0-fpm`._

    ```console
    $ sudo apt-get update
  12. justinhartman revised this gist Feb 16, 2021. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion 04_configure-php-nginx.md
    Original file line number Diff line number Diff line change
    @@ -17,7 +17,7 @@ upload_max_filesize=256M
    date.timezone=Africa/Johannesburg
    [opcache]
    opcache.memory_consumption=256
    opcache.max_accelerated_files=100000
    opcache.max_accelerated_files=50000
    opcache.validate_timestamps=0
    opcache.revalidate_freq=120
    opcache.error_log=/var/log/opcache-errors.log
  13. justinhartman revised this gist Feb 16, 2021. 4 changed files with 17 additions and 15 deletions.
    8 changes: 5 additions & 3 deletions 03_install-lemp.md
    Original file line number Diff line number Diff line change
    @@ -34,13 +34,15 @@ mysql> exit

    ## Step 3 - Install PHP-FPM

    _I've chosen to install PHP 8.0 but you can change any of the commands from `php8.0` to `php8.0` if needed. For example any reference to `php8.0-fpm` can be adjusted simply to `php8.0-fpm`._

    ```console
    $ sudo apt-get update
    $ sudo apt -y install lsb-release apt-transport-https ca-certificates wget
    $ wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
    $ echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/php.list
    $ sudo apt update
    $ sudo apt -y install php7.4-{fpm,cli,apc,bcmath,bz2,curl,gd,imagick,intl,memcached,mbstring,mysql,redis,xml,zip}
    $ sudo systemctl start php7.4-fpm
    $ sudo systemctl enable php7.4-fpm
    $ sudo apt -y install php8.0-{fpm,cli,apc,bcmath,bz2,curl,gd,imagick,intl,memcached,mbstring,mysql,redis,xml,zip}
    $ sudo systemctl start php8.0-fpm
    $ sudo systemctl enable php8.0-fpm
    ```
    14 changes: 7 additions & 7 deletions 04_configure-php-nginx.md
    Original file line number Diff line number Diff line change
    @@ -5,7 +5,7 @@ These are the different configuration files for setting up the servers.
    ## Configure php.ini

    ```console
    $ sudo nano /etc/php/7.4/fpm/php.ini
    $ sudo nano /etc/php/8.0/fpm/php.ini
    ```

    Change these values:
    @@ -54,19 +54,19 @@ apc.lazy_functions=0
    ```

    ```console
    $ sudo systemctl reload php7.4-fpm
    $ sudo systemctl reload php8.0-fpm
    ```

    ## Configure php-fpm.conf

    ```console
    $ sudo nano /etc/php/7.4/fpm/php-fpm.conf
    $ sudo nano /etc/php/8.0/fpm/php-fpm.conf
    ```

    ```nginx
    emergency_restart_threshold 10
    emergency_restart_interval 1m
    process_control_timeout 10s
    emergency_restart_threshold=10
    emergency_restart_interval=1m
    process_control_timeout=10s
    ```

    ## Configure nginx
    @@ -100,7 +100,7 @@ server {
    index index.php index.html index.htm index.nginx-debian.html;
    location ~ \.php$ {
    include snippets/fastcgi-php.conf;
    fastcgi_pass unix:/run/php/php7.4-fpm.sock;
    fastcgi_pass unix:/run/php/php8.0-fpm.sock;
    }
    }
    ```
    8 changes: 4 additions & 4 deletions 05_fine-tune-php-fpm.md
    Original file line number Diff line number Diff line change
    @@ -45,7 +45,7 @@ You're now going to want to download the [ps_mem.py][psmem] file and run the com
    $ cd ~/
    $ wget https://raw.githubusercontent.com/pixelb/ps_mem/master/ps_mem.py
    $ sudo python ps_mem.py | grep php-fpm
    20: 24.8 MiB + 9.0 MiB = 33.7 MiB php-fpm7.4 (3)
    20: 24.8 MiB + 9.0 MiB = 33.7 MiB php-fpm8.0 (3)
    ```

    **Work It Out**
    @@ -69,14 +69,14 @@ The other calculations are now easy because they are all based on how many CPU C
    Change the `www` pool so it now has the following settings.

    ```console
    $ sudo nano /etc/php/7.4/fpm/pool.d/www.conf
    $ sudo nano /etc/php/8.0/fpm/pool.d/www.conf
    ```

    ```ini
    [www]
    user = www-data
    group = www-data
    listen = /run/php/php7.4-fpm.sock
    listen = /run/php/php8.0-fpm.sock
    listen.owner = www-data
    listen.group = www-data

    @@ -99,7 +99,7 @@ pm.max_requests = 500
    Reload PHP:

    ```console
    $ sudo systemctl reload php7.4-fpm
    $ sudo systemctl reload php8.0-fpm
    ```

    [tuning]: https://tideways.com/profiler/blog/an-introduction-to-php-fpm-tuning
    2 changes: 1 addition & 1 deletion 07_new-vhost.md
    Original file line number Diff line number Diff line change
    @@ -19,7 +19,7 @@ server {
    location ~ \.php$ {
    include snippets/fastcgi-php.conf;
    fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
    fastcgi_pass unix:/var/run/php/php8.0-fpm.sock;
    }
    location ~ /\.ht {
  14. justinhartman revised this gist Feb 16, 2021. 1 changed file with 17 additions and 0 deletions.
    17 changes: 17 additions & 0 deletions 11_install-composer-node-npm.md
    Original file line number Diff line number Diff line change
    @@ -48,6 +48,23 @@ $ npm -v
    6.14.8
    ```

    ### Install Yarn

    With NVM installed you can install `yarn` with the following command:

    ```console
    $ npm install -g yarn
    ```

    You can test it out with:

    ```console
    $ yarn -v
    1.22.10
    ```

    ### Optional

    Optionally, you can install the Debian `build-essential` packages as certain node packages require the source code of certain packages when building them at runtime. You probably won't need to do this but I've included it here for those who think they will need it moving forward.

    ```composer
  15. justinhartman revised this gist Jan 26, 2021. 3 changed files with 5 additions and 3 deletions.
    2 changes: 1 addition & 1 deletion 03_install-lemp.md
    Original file line number Diff line number Diff line change
    @@ -40,7 +40,7 @@ $ sudo apt -y install lsb-release apt-transport-https ca-certificates wget
    $ wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
    $ echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/php.list
    $ sudo apt update
    $ sudo apt -y install php7.4-{fpm,cli,apc,bcmath,bz2,curl,gd,imagick,intl,memcached,mbstring,mysql,xml,zip}
    $ sudo apt -y install php7.4-{fpm,cli,apc,bcmath,bz2,curl,gd,imagick,intl,memcached,mbstring,mysql,redis,xml,zip}
    $ sudo systemctl start php7.4-fpm
    $ sudo systemctl enable php7.4-fpm
    ```
    5 changes: 3 additions & 2 deletions 04_configure-php-nginx.md
    Original file line number Diff line number Diff line change
    @@ -17,8 +17,9 @@ upload_max_filesize=256M
    date.timezone=Africa/Johannesburg
    [opcache]
    opcache.memory_consumption=256
    opcache.max_accelerated_files=10000
    opcache.validate_timestamps=120
    opcache.max_accelerated_files=100000
    opcache.validate_timestamps=0
    opcache.revalidate_freq=120
    opcache.error_log=/var/log/opcache-errors.log
    [apc]
    apc.shm_size=128M
    1 change: 1 addition & 0 deletions 11_install-composer-node-npm.md
    Original file line number Diff line number Diff line change
    @@ -36,6 +36,7 @@ Debian 10 comes with Node 10 and the following instructions will install Node 12
    $ cd ~
    $ curl -sL https://deb.nodesource.com/setup_12.x -o nodesource_setup.sh
    $ sudo bash nodesource_setup.sh
    $ sudo apt-get install -y nodejs
    ```

    Test the installation:
  16. justinhartman revised this gist Dec 16, 2020. 2 changed files with 8 additions and 4 deletions.
    10 changes: 7 additions & 3 deletions 01_introduction.md
    Original file line number Diff line number Diff line change
    @@ -1,13 +1,17 @@
    # How To Install Linux, Nginx, MySQL, PHP on a Debian 10 Virtual Machine
    # How To Install Nginx, MySQL, PHP, Redis and more on a Debian 10 Virtual Machine

    This series of documents will configure and setup a Linux, Nginx, MySQL, and
    This series of documents will configure and setup a Nginx, MySQL, and
    PHP (LEMP) server on a pretty _feature-rich_ VM running Debian. I used a
    VM with 8GB RAM and 4 CPU Cores so YMML with some of the settings below.

    This will also install other useful packages and configurations like,
    Redis, Memcached, Node, NPM, Composer, and a fully automated SSL
    service using certbot for Let's Encrypt.

    These documents take you through each step of the configuration from:

    1. [General Setup][gen]
    1. [Install Linux, Nginx, MySQL, and PHP][lemp]
    1. [Install Nginx, MySQL, and PHP (LEMP)][lemp]
    1. [Confiure PHP and Nginx][config]
    1. [How to fine-tune PHP-FPM][fpm-tune]
    1. [Fine-tune MySQL/MariaDB for Performance][mysql]
    2 changes: 1 addition & 1 deletion 03_install-lemp.md
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    # Install Linux, Nginx, MySQL, and PHP
    # Install Nginx, MySQL, and PHP (LEMP)

    There are three steps to install the LEMP server and then there are some configuration changes which you need to do.

  17. justinhartman revised this gist Dec 16, 2020. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion 03_install-lemp.md
    Original file line number Diff line number Diff line change
    @@ -40,7 +40,7 @@ $ sudo apt -y install lsb-release apt-transport-https ca-certificates wget
    $ wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
    $ echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/php.list
    $ sudo apt update
    $ sudo apt -y install php7.4-{fpm,cli,apc,bcmath,bz2,curl,gd,imagick,intl,mbstring,mysql,xml,zip}
    $ sudo apt -y install php7.4-{fpm,cli,apc,bcmath,bz2,curl,gd,imagick,intl,memcached,mbstring,mysql,xml,zip}
    $ sudo systemctl start php7.4-fpm
    $ sudo systemctl enable php7.4-fpm
    ```
  18. justinhartman revised this gist Dec 11, 2020. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion 01_introduction.md
    Original file line number Diff line number Diff line change
    @@ -10,12 +10,12 @@ These documents take you through each step of the configuration from:
    1. [Install Linux, Nginx, MySQL, and PHP][lemp]
    1. [Confiure PHP and Nginx][config]
    1. [How to fine-tune PHP-FPM][fpm-tune]
    1. [Fine-tune MySQL/MariaDB for Performance][mysql]
    1. [Create a New Nginx Host][vhost]
    1. [Setup Let's Encrypt SSL certificates][ssl]
    1. [Setup Remote SSH Login with sudo][ssh]
    1. [Install Redis & Memcached Servers][redis]
    1. [Install Composer & Node/NPM][composer]
    1. [Fine-tune MySQL/MariaDB for Performance][mysql]

    [gen]: #file-02_general-md
    [lemp]: #file-03_install-lemp-md
  19. justinhartman revised this gist Dec 11, 2020. 11 changed files with 10 additions and 10 deletions.
    20 changes: 10 additions & 10 deletions 0_introduction.md → 01_introduction.md
    Original file line number Diff line number Diff line change
    @@ -17,13 +17,13 @@ These documents take you through each step of the configuration from:
    1. [Install Composer & Node/NPM][composer]
    1. [Fine-tune MySQL/MariaDB for Performance][mysql]

    [gen]: #file-1_general-md
    [lemp]: #file-2_install-lemp-md
    [config]: #file-3_configure-php-nginx-md
    [fpm-tune]: #file-4_fine-tune-php-fpm-md
    [mysql]: #file-10_fine-tune-mysql-md
    [vhost]: #file-5_new-vhost-md
    [ssl]: #file-6_setup-ssl-md
    [ssh]: #file-7_ssh-remote-login-md
    [redis]: #file-8_install-redis-memcached-md
    [composer]: #file-9_install-composer-node-npm-md
    [gen]: #file-02_general-md
    [lemp]: #file-03_install-lemp-md
    [config]: #file-04_configure-php-nginx-md
    [fpm-tune]: #file-05_fine-tune-php-fpm-md
    [mysql]: #file-06_fine-tune-mysql-md
    [vhost]: #file-07_new-vhost-md
    [ssl]: #file-08_setup-ssl-md
    [ssh]: #file-09_ssh-remote-login-md
    [redis]: #file-10_install-redis-memcached-md
    [composer]: #file-11_install-composer-node-npm-md
    File renamed without changes.
    File renamed without changes.
    File renamed without changes.
    File renamed without changes.
    File renamed without changes.
    File renamed without changes.
    File renamed without changes.
    File renamed without changes.
  20. justinhartman revised this gist Dec 11, 2020. 2 changed files with 79 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions 0_introduction.md
    Original file line number Diff line number Diff line change
    @@ -15,11 +15,13 @@ These documents take you through each step of the configuration from:
    1. [Setup Remote SSH Login with sudo][ssh]
    1. [Install Redis & Memcached Servers][redis]
    1. [Install Composer & Node/NPM][composer]
    1. [Fine-tune MySQL/MariaDB for Performance][mysql]

    [gen]: #file-1_general-md
    [lemp]: #file-2_install-lemp-md
    [config]: #file-3_configure-php-nginx-md
    [fpm-tune]: #file-4_fine-tune-php-fpm-md
    [mysql]: #file-10_fine-tune-mysql-md
    [vhost]: #file-5_new-vhost-md
    [ssl]: #file-6_setup-ssl-md
    [ssh]: #file-7_ssh-remote-login-md
    77 changes: 77 additions & 0 deletions 10_fine-tune-mysql.md
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,77 @@
    # Fine-tune MySQL/MariaDB for Performance

    ## Increase Open Files Limit

    To ensure good server performance, the total number of client connections, database files, and log files must not exceed the maximum file descriptor limit on the operating system `ulimit -n`. Linux systems limit the number of file descriptors that any one process may open to 1,024 per process. On active database servers (especially production ones) it can easily reach the default system limit.

    To increase this, edit `/etc/security/limits.conf` and specify or add the following:

    ```ini
    # Added to increase server limit of 1024 per process.
    mysql soft nofile 65535
    mysql hard nofile 65535
    ```

    This requires a system restart. Once rebooted, check the limit with:

    ```console
    $ ulimit -Sn
    65535
    $ ulimit -Hn
    65535
    ```

    ## Setting Swappiness on Linux for MariaDB

    ```console
    $ sudo sysctl -w vm.swappiness=1
    $ sudo nano /etc/sysctl.conf
    ```

    Add this line to the file: `vm.swappiness=1`

    ## Filesystem Optimisations for MariaDB

    Most database setups do not need to record file access time. You might want to disable this when mounting the volume into the system. To do this, edit your file `/etc/fstab`. For example, on a volume named /dev/sda1, this how it looks like

    ```config
    /dev/sda1 / ext4 noatime,errors=remount-ro 0 1
    ```

    ## Tuneup MariaDB To Utilise Memory Efficiently

    You have these settings to consideration when fine-tuning your server:

    - `innodb_buffer_pool_size`: You can set this from 70-80% of the total available memory on a dedicated database server with only or primarily InnoDB tables. If set to 2 GB or more, you will probably want to adjust `innodb_buffer_pool_instances` as well. You can set this dynamically if you are using MariaDB >= 10.2.2 version.
    - `innodb_flush_log_at_trx_commit`: This setting offers a significant trade-off between performance and reliability. When set to 0, database performance is greatly increased. However, up to 1 second of transactions can be lost in a crash. The default value for this setting is 1.
    - `innodb_flush_method`: Set this to O_DIRECT in order to avoid double buffering data.
    - `innodb_log_file_size`: If you are preferring more performance gains especially during and handling your InnoDB transactions, setting the variable innodb_log_file_size to a larger value such as 5Gib or even 10GiB is reasonable. Increasing means that the larger transactions can run without needing to perform disk I/O before committing.
    - `max_connections`: If you are application requires a lot of concurrent connections, you can start setting this to 500.
    - `max_allowed_packet`: The max_allowed_packet directive defines the maximum size of packet that can be sent. If the client sends more data than max_allowed_packet size, the socket will be closed. Ideally, especially on most application demands today, you can start setting this to 512M.
    - `back_log`: Increase if you expect short bursts of connections. Cannot be set higher than the operating system limit. To calculate use `(50 + max_connections/5)` to get total value.
    - Query Cache: Preferably you should disable query cache in all of your MariaDB setup. You need to ensure that `query_cache_type=OFF` and `query_cache_size=0` to complete disable query cache. Unlike MySQL, MariaDB is still completely supporting query cache and do not have any plans on withdrawing its support to use query cache.
    - Turn on `slow_query_log` and se the `long_query_time` to 1 second to log queries to `/var/lib/mysql/hostname-slow.log`.

    ```console
    $ sudo nano /etc/mysql/my.cnf
    ```

    ```ini
    [mysqld]
    innodb_buffer_pool_size = 3G
    # innodb_buffer_pool_instances = 4 (default is 1)
    innodb_flush_log_at_trx_commit = 0
    innodb_flush_method = O_DIRECT

    innodb_log_file_size = 256M
    max_connections = 500
    # (50 + max_connections/5)
    back_log = 150
    max_allowed_packet = 32M
    # Disable query cache
    query_cache_type = OFF
    query_cache_size = 0
    slow_query_log = ON
    long_query_time = 1
    # slow_query_log_file = /var/lib/mysql/hostname-slow.log
    ```
  21. justinhartman revised this gist Dec 8, 2020. 8 changed files with 178 additions and 146 deletions.
    12 changes: 6 additions & 6 deletions 0_introduction.md
    Original file line number Diff line number Diff line change
    @@ -12,16 +12,16 @@ These documents take you through each step of the configuration from:
    1. [How to fine-tune PHP-FPM][fpm-tune]
    1. [Create a New Nginx Host][vhost]
    1. [Setup Let's Encrypt SSL certificates][ssl]
    1. [Setup SFTP server][sftp]
    1. [Install Redis Server][redis]
    1. [Download and Configure phpMyAdmin][pma]
    1. [Setup Remote SSH Login with sudo][ssh]
    1. [Install Redis & Memcached Servers][redis]
    1. [Install Composer & Node/NPM][composer]

    [gen]: #file-1_general-md
    [lemp]: #file-2_install-lemp-md
    [config]: #file-3_configure-php-nginx-md
    [fpm-tune]: #file-4_fine-tune-php-fpm-md
    [vhost]: #file-5_new-vhost-md
    [ssl]: #file-6_setup-ssl-md
    [sftp]: #file-7_sftp-setup-md
    [redis]: #file-8_install-redis-md
    [pma]: #file-9_phpmyadmin-md
    [ssh]: #file-7_ssh-remote-login-md
    [redis]: #file-8_install-redis-memcached-md
    [composer]: #file-9_install-composer-node-npm-md
    2 changes: 1 addition & 1 deletion 2_install-lemp.md
    Original file line number Diff line number Diff line change
    @@ -40,7 +40,7 @@ $ sudo apt -y install lsb-release apt-transport-https ca-certificates wget
    $ wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
    $ echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/php.list
    $ sudo apt update
    $ sudo apt -y install php7.4-{fpm,cli,apc,bcmath,bz2,curl,gd,imagick,intl,mbstring,mysql,zip}
    $ sudo apt -y install php7.4-{fpm,cli,apc,bcmath,bz2,curl,gd,imagick,intl,mbstring,mysql,xml,zip}
    $ sudo systemctl start php7.4-fpm
    $ sudo systemctl enable php7.4-fpm
    ```
    40 changes: 0 additions & 40 deletions 7_sftp-setup.md
    Original file line number Diff line number Diff line change
    @@ -1,40 +0,0 @@
    # Setup SFTP server

    This uses public/private SSH key for authentication. Your user will only be able to write files in /var/www.

    ```console
    $ sudo apt-get install openssh-server
    $ sudo addgroup sftp
    $ sudo useradd -m FTPUSER -g sftp
    $ sudo passwd FTPUSER
    $ sudo adduser FTPUSER www-data # add user to group they can write to www folder
    $ sudo chmod -R g+w /var/www # set group write permissions
    $ sudo mkdir -p /home/FTPUSER/.ssh
    $ sudo nano /home/FTPUSER/.ssh/authorized_keys # copy public key from computer and paste here
    $ sudo chmod -R go= /home/FTPUSER/.ssh
    $ sudo chown -R FTPUSER:sftp /home/FTPUSER/.ssh
    $ sudo nano /etc/ssh/sshd_config
    ```

    ```nginx
    PubkeyAuthentication yes
    Match group sftp
    X11Forwarding no
    AllowAgentForwarding no
    AllowTcpForwarding no
    ForceCommand internal-sftp
    ```

    ```console
    $ sudo service ssh restart
    ```

    Test it out using your private key.

    ```console
    $ sftp -i ~/.ssh/id_rsa [email protected]
    Connected to 140.88.106.229.
    sftp> exit
    ```

    > **TODO**: figure out why `ChrootDirectory /var/www/` in `/etc/ssh/sshd_config` creates a broken_pipe issue when connecting to the SFTP server. This has been disabled until I can solve this! :scream_cat:
    57 changes: 57 additions & 0 deletions 7_ssh-remote-login.md
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,57 @@
    # Setup Remote SSH Login with sudo

    This uses public/private SSH key for authentication.

    ```console
    $ sudo apt-get install openssh-server
    $ sudo useradd -m USERNAME -g www-data # add user to group they can write to www folder
    $ sudo passwd USERNAME # set password for login
    $ sudo chmod -R g+w /var/www # set group write permissions
    $ sudo mkdir -p /home/USERNAME/.ssh
    $ sudo nano /home/USERNAME/.ssh/authorized_keys # copy your `~/.ssh/id_rsa.pub` key from computer and paste here
    $ sudo chmod -R go= /home/USERNAME/.ssh
    $ sudo chown -R USERNAME:www-data /home/USERNAME
    $ sudo usermod -a -G sudo USERNAME # optionally, add user to sudoers file
    $ sudo service ssh restart
    ```

    Now login to the SSH server with your private key and test:

    ```console
    $ ssh -i ~/.ssh/id_rsa [email protected]
    Linux server.com 4.19.0-13-amd64 #1 SMP Debian 4.19.160-2 (2020-11-28) x86_64

    The programs included with the Debian GNU/Linux system are free software;
    the exact distribution terms for each program are described in the
    individual files in /usr/share/doc/*/copyright.

    Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
    permitted by applicable law.
    Last login: Tue Dec 8 23:12:02 2020 from 104.245.101.122
    USERNAME@server:~#
    ```

    If all goes well you won't be prompted for a password and you will just login straight away. If you are asked for a password you have done something wrong. Try again.

    ## Turn off sudo password authentication

    With the above configured you will still be prompted for your password when running a `sudo` command on the server. To disable the prompting of passwords, do the following.

    ```console
    $ sudo visudo
    ```

    Add your username to the end of the file so it looks similar to:

    ```bash
    # Allow members of group sudo to execute any command
    %sudo ALL=(ALL:ALL) ALL
    #includedir /etc/sudoers.d
    USERNAME ALL=(ALL) NOPASSWD: ALL
    ```

    **NB:** replace `USERNAME` with the username you created, don't leave it as is. Then save the file and restart the `ssh` server.

    ```console
    $ sudo service ssh restart
    ```
    57 changes: 57 additions & 0 deletions 8_install-redis-memcached.md
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,57 @@
    # Install and Setup Redis & Memcached Servers

    ## Install Redis

    ```console
    $ sudo apt install redis-server
    ```

    ### Edit Configuration

    ```console
    $ sudo nano /etc/redis/redis.conf
    ```

    Change these settings:

    ```ini
    supervised systemd # use Debian's systemd
    bind 127.0.0.1 ::1
    # If you want to secure your Redis server set a password below.
    # requirepass change-this-password
    ```

    ### Restart Redis

    ```console
    $ sudo systemctl restart redis
    $ sudo systemctl status redis
    ```

    ## Install Memcached

    ```console
    $ sudo apt install memcached libmemcached-tools
    $ sudo systemctl status memcached
    ```

    ### Edit Configuration

    ```console
    $ sudo nano /etc/memcached.conf
    ```

    Change at least this setting:

    ```ini
    # Start with a cap of 64 megs of memory. It's reasonable, and the daemon default
    # Note that the daemon will grow to this size, but does not start out holding this much
    # memory
    -m 256
    ```

    ### Restart Memcached

    ```console
    $ sudo systemctl restart memcached
    ```
    29 changes: 0 additions & 29 deletions 8_install-redis.md
    Original file line number Diff line number Diff line change
    @@ -1,29 +0,0 @@
    # Install and Setup Redis Server

    ## Install Redis

    ```console
    $ sudo apt install redis-server
    ```

    ## Edit Configuration

    ```console
    $ sudo nano /etc/redis/redis.conf
    ```

    Change these settings:

    ```ini
    supervised systemd # use Debian's systemd
    bind 127.0.0.1 ::1
    # If you want to secure your Redis server set a password below.
    # requirepass change-this-password
    ```

    ## Restart Redis

    ```console
    $ sudo systemctl restart redis
    $ sudo systemctl status redis
    ```
    57 changes: 57 additions & 0 deletions 9_install-composer-node-npm.md
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,57 @@
    # Installing Composer and Node/NPM

    This section covers the instructions to installing [Composer][compser] and [Node.js and NPM][node] on your server.

    ## Install Composer

    ```console
    $ sudo apt install curl git unzip zip
    $ cd ~
    $ php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
    $ php -r "if (hash_file('sha384', 'composer-setup.php') === '756890a4488ce9024fc62c56153228907f1545c228516cbf63f885e036d37e9a59d27d63f46af1d4d07ee0f76181c7d3') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;"
    $ sudo php composer-setup.php --install-dir=/usr/local/bin --filename=composer
    $ php -r "unlink('composer-setup.php');"
    ```

    Composer will be installed at `/usr/local/bin/composer` and you can test the installation as follows:

    ```console
    $ /usr/local/bin/composer -v
    ______
    / ____/___ ____ ___ ____ ____ ________ _____
    / / / __ \/ __ `__ \/ __ \/ __ \/ ___/ _ \/ ___/
    / /___/ /_/ / / / / / / /_/ / /_/ (__ ) __/ /
    \____/\____/_/ /_/ /_/ .___/\____/____/\___/_/
    /_/
    Composer version 2.0.8 2020-12-03 17:20:38
    ```

    If you get an error output stating `Installer corrupt`, it means the version of composer has been updated and you'll need a new hash. Visit the official [getcomposer.org download page][compser] to get the latest `hash_file` string and/or install via that method.

    ## Install Node & NPM

    Debian 10 comes with Node 10 and the following instructions will install Node 12 and NPM 6 by installing from source. If you would like to install Node 14 simply replace `setup_12.x` in the command below with `setup_14.x`. If that's not high enough and you'd like Node 15, replace the command with `setup_15.x` instead.

    ```console
    $ cd ~
    $ curl -sL https://deb.nodesource.com/setup_12.x -o nodesource_setup.sh
    $ sudo bash nodesource_setup.sh
    ```

    Test the installation:

    ```console
    $ node -v
    v12.20.0
    $ npm -v
    6.14.8
    ```

    Optionally, you can install the Debian `build-essential` packages as certain node packages require the source code of certain packages when building them at runtime. You probably won't need to do this but I've included it here for those who think they will need it moving forward.

    ```composer
    $ sudo apt install build-essential
    ```

    [compser]: https://getcomposer.org/download/
    [node]: https://nodejs.org/
    70 changes: 0 additions & 70 deletions 9_phpmyadmin.md
    Original file line number Diff line number Diff line change
    @@ -1,70 +0,0 @@
    # Download and Configure phpMyAdmin

    ```console
    $ cd /var/www/html
    $ sudo -u www-data mkdir phpmyadmin
    $ sudo -u www-data wget https://files.phpmyadmin.net/phpMyAdmin/5.0.4/phpMyAdmin-5.0.4-english.tar.gz
    $ sudo -u www-data tar -zxf phpMyAdmin-5.0.4-english.tar.gz -C phpmyadmin --strip-components 1
    $ sudo -u www-data rm phpMyAdmin-5.0.4-english.tar.gz
    ```

    1. Open phpMyAdmin in a browser and log in as root.
    1. Create a database called `phpmyadmin`
    1. Create a user called `pma` and set the *host* to `localhost`.
    1. Open the SQL dump file in `/var/www/html/phpmyadmin/create_tables.sql`.
    1. In phpMyAdmin, select the `phpmyadmin` database and click on the *SQL* tab.
    1. Copy/paste the entire text from `create_tables.sql` into the text box, and run the query.
    1. Open the `config.inc.php` file in the phpMyAdmin install directory, and add the following.

    ```php
    /* Blowfish needed for cookie based authentication. MUST to be 32 chars long. */
    $cfg['blowfish_secret'] = '12345678901234567890123456789012';
    $cfg['DefaultConnectionCollation'] = 'utf8mb4_unicode_ci';
    /* Servers configuration */
    $i = 0;
    /* First server */
    $i++;
    /* Authentication type */
    $cfg['Servers'][$i]['auth_type'] = 'cookie';
    /* Server parameters */
    $cfg['Servers'][$i]['host'] = 'localhost';
    $cfg['Servers'][$i]['compress'] = false;
    $cfg['Servers'][$i]['AllowNoPassword'] = false;
    /* Hide system dbs */
    $cfg['Servers'][$i]['hide_db'] = '^(information_schema|mysql|performance_schema|sys)$';
    /* phpMyAdmin configuration storage settings. */
    $cfg['Servers'][$i]['controluser'] = 'phpmyadmin';
    $cfg['Servers'][$i]['controlpass'] = 'ADD_DB_PASSWORD';
    /* phpMyAdmin storage database and tables */
    $cfg['Servers'][$i]['pmadb'] = 'phpmyadmin';
    $cfg['Servers'][$i]['bookmarktable'] = 'pma__bookmark';
    $cfg['Servers'][$i]['relation'] = 'pma__relation';
    $cfg['Servers'][$i]['table_info'] = 'pma__table_info';
    $cfg['Servers'][$i]['table_coords'] = 'pma__table_coords';
    $cfg['Servers'][$i]['pdf_pages'] = 'pma__pdf_pages';
    $cfg['Servers'][$i]['column_info'] = 'pma__column_info';
    $cfg['Servers'][$i]['history'] = 'pma__history';
    $cfg['Servers'][$i]['table_uiprefs'] = 'pma__table_uiprefs';
    $cfg['Servers'][$i]['tracking'] = 'pma__tracking';
    $cfg['Servers'][$i]['userconfig'] = 'pma__userconfig';
    $cfg['Servers'][$i]['recent'] = 'pma__recent';
    $cfg['Servers'][$i]['favorite'] = 'pma__favorite';
    $cfg['Servers'][$i]['users'] = 'pma__users';
    $cfg['Servers'][$i]['usergroups'] = 'pma__usergroups';
    $cfg['Servers'][$i]['navigationhiding'] = 'pma__navigationhiding';
    $cfg['Servers'][$i]['savedsearches'] = 'pma__savedsearches';
    $cfg['Servers'][$i]['central_columns'] = 'pma__central_columns';
    $cfg['Servers'][$i]['designer_settings'] = 'pma__designer_settings';
    $cfg['Servers'][$i]['export_templates'] = 'pma__export_templates';
    /* Directories for saving/loading files from server */
    $cfg['UploadDir'] = '';
    $cfg['SaveDir'] = '';
    /* General config */
    $cfg['RowActionType'] = 'icons';
    $cfg['ShowAll'] = true;
    $cfg['MaxRows'] = 50;
    $cfg['QueryHistoryDB'] = true;
    $cfg['QueryHistoryMax'] = 100;
    $cfg['SendErrorReports'] = 'never';
    $cfg['Console']['DarkTheme'] = true;
    ```
  22. justinhartman revised this gist Dec 8, 2020. 1 changed file with 9 additions and 9 deletions.
    18 changes: 9 additions & 9 deletions 4_fine-tune-php-fpm.md
    Original file line number Diff line number Diff line change
    @@ -1,6 +1,6 @@
    # How to fine-tune PHP-FPM

    This section outlines how to go about configuring your php-fpm `www.conf` pool file. It may seem complicated and overwhelming and you can totally skip this if you want to. That said, overlooking this step is like buying a sports car and never going above 20 mph; you're going to kill the engine by not using the car correctly. You really do want to maximise your server as much as possible to avoid bottlenecks and crashes.
    This section outlines how to go about configuring your php-fpm `www.conf` pool file. It may seem complicated and overwhelming and you can totally skip this if you want to. That said, overlooking this step is like buying a sports car and never going above 20 mph; you're going to kill the engine by not using the car correctly. You really do want to maximise your server as much as possible to avoid bottlenecks and crashes later on.

    ## An Overview

    @@ -21,11 +21,11 @@ Here are the settings and calculations. The `Value` column has details on how to

    ### How to calculate max_children

    Let's assume, for this example, you have a server with **8GB RAM** and **4 CPU Cores**. With this in mind we can now continue to work this formula out:
    Let's assume, for this example, you have a server with **8GB RAM** and **4 CPU Cores**. With this in mind we can continue to work this formula out:

    `(Total RAM - Memory used on server) / Process Size = max_children`

    First let's run some commands on the server to get the data we need for the calculation.
    First let's run some commands on the server to get the data we need for the formula.

    **Memory Usage**

    @@ -50,13 +50,13 @@ $ sudo python ps_mem.py | grep php-fpm

    **Work It Out**

    We've not got everything we need so now we can apply this info to the calculation as follows.
    We've got everything we need to apply these values to the formula as follows:

    - `Total RAM` = `7800` (for 7.8GB) this is the `total` column value when running `sudo free -hl`
    - `Memory Used` = `331` this is the `used` column value when running `sudo free -hl`
    - `Process Size` = `33.7` which is the total after running `sudo python ps_mem.py | grep php-fpm`
    - `Total RAM` = `7800` is the `total` column value (`7.8Gi`) when running `sudo free -hl`
    - `Memory Used` = `331` is the `used` column value (`331Mi`) when running `sudo free -hl`
    - `Process Size` = `33.7` is the total value (`33.7 MiB`) after executing `sudo python ps_mem.py | grep php-fpm`

    Now you pull this all together and run the formula of `(Total RAM - Memory used on server) / Process Size` it will give you something like this:
    Pull this all together and run the formula of `(Total RAM - Memory used on server) / Process Size` which will give you something similar to this:

    > **(7800 - 331) / 33.7 = 221.63**
    @@ -86,7 +86,7 @@ listen.group = www-data
    pm = dynamic
    ; (Mem - Used) / ps_mem.py = max_children
    ; (7800 - 331) / 33.7 = 221.63
    pm.max_children = 220
    pm.max_children = 222
    ; CPU * 4 = 16
    pm.start_servers = 16
    ; CPU * 2 = 8
  23. justinhartman revised this gist Dec 8, 2020. 1 changed file with 8 additions and 2 deletions.
    10 changes: 8 additions & 2 deletions 4_fine-tune-php-fpm.md
    Original file line number Diff line number Diff line change
    @@ -21,14 +21,16 @@ Here are the settings and calculations. The `Value` column has details on how to

    ### How to calculate max_children

    If you have a server with **8GB RAM** and **4 CPU Cores** this is how you work this formula out:
    Let's assume, for this example, you have a server with **8GB RAM** and **4 CPU Cores**. With this in mind we can now continue to work this formula out:

    `(Total RAM - Memory used on server) / Process Size = max_children`

    First let's run some commands on the server to get the data we need.
    First let's run some commands on the server to get the data we need for the calculation.

    **Memory Usage**

    We need to know how much `total` memory we have as well as how much of the memory is `used`. The rest is irrelevant.

    ```console
    $ sudo free -hl
    total used free shared buff/cache available
    @@ -37,6 +39,8 @@ Mem: 7.8Gi 331Mi 6.0Gi 27Mi 1.5Gi 7.1Gi

    **Process Memory**

    You're now going to want to download the [ps_mem.py][psmem] file and run the command. When done we are only interested in the total amount the sum calculates; i.e. the total amount in `MiB` after the `=` sign.

    ```console
    $ cd ~/
    $ wget https://raw.githubusercontent.com/pixelb/ps_mem/master/ps_mem.py
    @@ -46,6 +50,8 @@ $ sudo python ps_mem.py | grep php-fpm

    **Work It Out**

    We've not got everything we need so now we can apply this info to the calculation as follows.

    - `Total RAM` = `7800` (for 7.8GB) this is the `total` column value when running `sudo free -hl`
    - `Memory Used` = `331` this is the `used` column value when running `sudo free -hl`
    - `Process Size` = `33.7` which is the total after running `sudo python ps_mem.py | grep php-fpm`
  24. justinhartman revised this gist Dec 8, 2020. 1 changed file with 31 additions and 8 deletions.
    39 changes: 31 additions & 8 deletions 4_fine-tune-php-fpm.md
    Original file line number Diff line number Diff line change
    @@ -1,12 +1,12 @@
    # How to fine-tune PHP-FPM

    This section outlines how to go about configuring your php-fpm `www.conf` pool file.
    This section outlines how to go about configuring your php-fpm `www.conf` pool file. It may seem complicated and overwhelming and you can totally skip this if you want to. That said, overlooking this step is like buying a sports car and never going above 20 mph; you're going to kill the engine by not using the car correctly. You really do want to maximise your server as much as possible to avoid bottlenecks and crashes.

    ## An Overview

    You are probably going to want to read [this article on FPM tuning][tuning] which I used to determine the settings below. The article will help you scientifically calculate the exact setting you should use for `max_children`, `start_servers`, `min_spare_servers`, and `max_spare_servers`. The most complicated one to calculate is `max_children` because there is a formula he uses to do this which is `(Total RAM - Memory used on server) / Process Size = max_children`. I've tried my best to give you the succinct version of what to calculate, as well as how, but depending on your level of skill, you may need to read the article fully.
    You are probably going to want to read [this article on FPM tuning][tuning] which I used to determine the settings below. The article will help you scientifically calculate the exact settings you should use for `max_children`, `start_servers`, `min_spare_servers`, and `max_spare_servers`. The most complicated one to calculate is `max_children` because there is a formula he uses to do this which is `(Total RAM - Memory used on server) / Process Size = max_children`. I've tried my best to give you the succinct version of what to calculate, as well as how, but depending on your level of skill, you may need to read the article fully.

    There is also reference to a script called `ps_mem.py` which you can [download directly from here][psmem], or you can run `wget https://raw.githubusercontent.com/pixelb/ps_mem/master/ps_mem.py` from your server to download the file there instead. This script will give you the total `Process Size` needed in the `max_children` formula above and is an amazing tool for helping you determine your resources available.
    There is also reference to a script called [`ps_mem.py`][psmem] which, when run on your server, will give you the total `Process Size` needed in the `max_children` formula above.

    ## Calculations

    @@ -21,19 +21,40 @@ Here are the settings and calculations. The `Value` column has details on how to

    ### How to calculate max_children

    If you have a server with **8GB RAM** and **4 CPU Cores** this is how you work this out.
    If you have a server with **8GB RAM** and **4 CPU Cores** this is how you work this formula out:

    `(Total RAM - Memory used on server) / Process Size = max_children`

    - `Total RAM` = `7800` (for 7.8GB) run `sudo free -hl` from the command line to get this
    - `Memory Used` = Run `sudo free -hl` from the command line to get the total used memory (e.g. 331MB)
    - `Process Size` = Run the [ps_mem.py][psmem] script by executing `sudo python ps_mem.py | grep php-fpm`. This would output something similar to this `24.8 MiB + 9.0 MiB = 33.7 MiB`. The value of `Process Size` is therefor `33.7`.
    First let's run some commands on the server to get the data we need.

    **Memory Usage**

    ```console
    $ sudo free -hl
    total used free shared buff/cache available
    Mem: 7.8Gi 331Mi 6.0Gi 27Mi 1.5Gi 7.1Gi
    ```

    **Process Memory**

    ```console
    $ cd ~/
    $ wget https://raw.githubusercontent.com/pixelb/ps_mem/master/ps_mem.py
    $ sudo python ps_mem.py | grep php-fpm
    20: 24.8 MiB + 9.0 MiB = 33.7 MiB php-fpm7.4 (3)
    ```

    **Work It Out**

    - `Total RAM` = `7800` (for 7.8GB) this is the `total` column value when running `sudo free -hl`
    - `Memory Used` = `331` this is the `used` column value when running `sudo free -hl`
    - `Process Size` = `33.7` which is the total after running `sudo python ps_mem.py | grep php-fpm`

    Now you pull this all together and run the formula of `(Total RAM - Memory used on server) / Process Size` it will give you something like this:

    > **(7800 - 331) / 33.7 = 221.63**
    Therefore: `max_children = 222`
    Therefore, we now know that `max_children = 222` 👍🏿

    The other calculations are now easy because they are all based on how many CPU Cores your have. Open up the `www.conf` file and begin editing with your newly calculated values.

    @@ -69,6 +90,8 @@ pm.max_spare_servers = 16
    pm.max_requests = 500
    ```

    Reload PHP:

    ```console
    $ sudo systemctl reload php7.4-fpm
    ```
  25. justinhartman revised this gist Dec 8, 2020. 2 changed files with 4 additions and 4 deletions.
    3 changes: 0 additions & 3 deletions 3_configure-php-nginx.md
    Original file line number Diff line number Diff line change
    @@ -112,6 +112,3 @@ nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
    nginx: configuration file /etc/nginx/nginx.conf test is successful
    $ sudo systemctl reload nginx
    ```

    [tuning]: https://tideways.com/profiler/blog/an-introduction-to-php-fpm-tuning
    [psmem]: https://raw.githubusercontent.com/pixelb/ps_mem/master/ps_mem.py
    5 changes: 4 additions & 1 deletion 4_fine-tune-php-fpm.md
    Original file line number Diff line number Diff line change
    @@ -71,4 +71,7 @@ pm.max_requests = 500

    ```console
    $ sudo systemctl reload php7.4-fpm
    ```
    ```

    [tuning]: https://tideways.com/profiler/blog/an-introduction-to-php-fpm-tuning
    [psmem]: https://raw.githubusercontent.com/pixelb/ps_mem/master/ps_mem.py
  26. justinhartman revised this gist Dec 8, 2020. 8 changed files with 85 additions and 80 deletions.
    14 changes: 8 additions & 6 deletions 0_introduction.md
    Original file line number Diff line number Diff line change
    @@ -9,7 +9,8 @@ These documents take you through each step of the configuration from:
    1. [General Setup][gen]
    1. [Install Linux, Nginx, MySQL, and PHP][lemp]
    1. [Confiure PHP and Nginx][config]
    1. [Create a new nginx vhost][vhost]
    1. [How to fine-tune PHP-FPM][fpm-tune]
    1. [Create a New Nginx Host][vhost]
    1. [Setup Let's Encrypt SSL certificates][ssl]
    1. [Setup SFTP server][sftp]
    1. [Install Redis Server][redis]
    @@ -18,8 +19,9 @@ These documents take you through each step of the configuration from:
    [gen]: #file-1_general-md
    [lemp]: #file-2_install-lemp-md
    [config]: #file-3_configure-php-nginx-md
    [vhost]: #file-4_new-vhost-md
    [ssl]: #file-5_setup-ssl-md
    [sftp]: #file-6_sftp-setup-md
    [redis]: #file-7_install-redis-md
    [pma]: #file-8_phpmyadmin-md
    [fpm-tune]: #file-4_fine-tune-php-fpm-md
    [vhost]: #file-5_new-vhost-md
    [ssl]: #file-6_setup-ssl-md
    [sftp]: #file-7_sftp-setup-md
    [redis]: #file-8_install-redis-md
    [pma]: #file-9_phpmyadmin-md
    73 changes: 0 additions & 73 deletions 3_configure-php-nginx.md
    Original file line number Diff line number Diff line change
    @@ -68,79 +68,6 @@ emergency_restart_interval 1m
    process_control_timeout 10s
    ```

    ## Configure php-fpm www.conf

    ### An Overview

    You are probably going to want to read [this article on FPM tuning][tuning] which I used to determine the settings below. The article will help you scientifically calculate the exact setting you should use for `max_children`, `start_servers`, `min_spare_servers`, and `max_spare_servers`. The most complicated one to calculate is `max_children` because there is a formula he uses to do this which is `(Total RAM - Memory used on server) / Process Size = max_children`. I've tried my best to give you the succinct version of what to calculate, as well as how, but depending on your level of skill, you may need to read the article fully.

    There is also reference to a script called `ps_mem.py` which you can [download directly from here][psmem], or you can run `wget https://raw.githubusercontent.com/pixelb/ps_mem/master/ps_mem.py` from your server to download the file there instead. This script will give you the total `Process Size` needed in the `max_children` formula above and is an amazing tool for helping you determine your resources available.

    ### Calculations

    Here are the settings and calculations. The `Value` column has details on how to calculate the `Setting` value.

    | Setting | Value |
    |:---------------------|:---------------------------------------------------------|
    | `max_children` | (`Total RAM` - `Memory` used on server) / `Process Size` |
    | `start_servers` | Number of CPU cores x 4 |
    | `min_spare_servers` | Number of CPU cores x 2 |
    | `max_spare_servers` | Equal to same value as `start_servers` |

    #### How to calculate max_children

    If you have a server with **8GB RAM** and **4 CPU Cores** this is how you work this out.

    `(Total RAM - Memory used on server) / Process Size = max_children`

    - `Total RAM` = `7800` (for 7.8GB) run `sudo free -hl` from the command line to get this
    - `Memory Used` = Run `sudo free -hl` from the command line to get the total used memory (e.g. 331MB)
    - `Process Size` = Run the [ps_mem.py][psmem] script by executing `sudo python ps_mem.py | grep php-fpm`. This would output something similar to this `24.8 MiB + 9.0 MiB = 33.7 MiB`. The value of `Process Size` is therefor `33.7`.

    Now you pull this all together and run the formula of `(Total RAM - Memory used on server) / Process Size` it will give you something like this:

    > **(7800 - 331) / 33.7 = 221.63**
    Therefore: `max_children = 222`

    The other calculations are now easy because they are all based on how many CPU Cores your have. Open up the `www.conf` file and begin editing with your newly calculated values.

    ### Edit the Settings

    Change the `www` pool so it now has the following settings.

    ```console
    $ sudo nano /etc/php/7.4/fpm/pool.d/www.conf
    ```

    ```ini
    [www]
    user = www-data
    group = www-data
    listen = /run/php/php7.4-fpm.sock
    listen.owner = www-data
    listen.group = www-data

    ; Server Resources: 8G RAM | 4x CPU Cores
    ; `free -hl` output: Total 7.8Gi Used 331Mi Free 5.8Gi
    ; `ps_mem.py` output: 24.8 MiB + 9.0 MiB = 33.7 MiB
    pm = dynamic
    ; (Mem - Used) / ps_mem.py = max_children
    ; (7800 - 331) / 33.7 = 221.63
    pm.max_children = 220
    ; CPU * 4 = 16
    pm.start_servers = 16
    ; CPU * 2 = 8
    pm.min_spare_servers = 8
    ; Same as children
    pm.max_spare_servers = 16
    pm.max_requests = 500
    ```

    ```console
    $ sudo systemctl reload php7.4-fpm
    ```

    ## Configure nginx

    ```console
    74 changes: 74 additions & 0 deletions 4_fine-tune-php-fpm.md
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,74 @@
    # How to fine-tune PHP-FPM

    This section outlines how to go about configuring your php-fpm `www.conf` pool file.

    ## An Overview

    You are probably going to want to read [this article on FPM tuning][tuning] which I used to determine the settings below. The article will help you scientifically calculate the exact setting you should use for `max_children`, `start_servers`, `min_spare_servers`, and `max_spare_servers`. The most complicated one to calculate is `max_children` because there is a formula he uses to do this which is `(Total RAM - Memory used on server) / Process Size = max_children`. I've tried my best to give you the succinct version of what to calculate, as well as how, but depending on your level of skill, you may need to read the article fully.

    There is also reference to a script called `ps_mem.py` which you can [download directly from here][psmem], or you can run `wget https://raw.githubusercontent.com/pixelb/ps_mem/master/ps_mem.py` from your server to download the file there instead. This script will give you the total `Process Size` needed in the `max_children` formula above and is an amazing tool for helping you determine your resources available.

    ## Calculations

    Here are the settings and calculations. The `Value` column has details on how to calculate the `Setting` value.

    | Setting | Value |
    |:---------------------|:---------------------------------------------------------|
    | `max_children` | (`Total RAM` - `Memory` used on server) / `Process Size` |
    | `start_servers` | Number of CPU cores x 4 |
    | `min_spare_servers` | Number of CPU cores x 2 |
    | `max_spare_servers` | Equal to same value as `start_servers` |

    ### How to calculate max_children

    If you have a server with **8GB RAM** and **4 CPU Cores** this is how you work this out.

    `(Total RAM - Memory used on server) / Process Size = max_children`

    - `Total RAM` = `7800` (for 7.8GB) run `sudo free -hl` from the command line to get this
    - `Memory Used` = Run `sudo free -hl` from the command line to get the total used memory (e.g. 331MB)
    - `Process Size` = Run the [ps_mem.py][psmem] script by executing `sudo python ps_mem.py | grep php-fpm`. This would output something similar to this `24.8 MiB + 9.0 MiB = 33.7 MiB`. The value of `Process Size` is therefor `33.7`.

    Now you pull this all together and run the formula of `(Total RAM - Memory used on server) / Process Size` it will give you something like this:

    > **(7800 - 331) / 33.7 = 221.63**
    Therefore: `max_children = 222`

    The other calculations are now easy because they are all based on how many CPU Cores your have. Open up the `www.conf` file and begin editing with your newly calculated values.

    ## Edit the Settings

    Change the `www` pool so it now has the following settings.

    ```console
    $ sudo nano /etc/php/7.4/fpm/pool.d/www.conf
    ```

    ```ini
    [www]
    user = www-data
    group = www-data
    listen = /run/php/php7.4-fpm.sock
    listen.owner = www-data
    listen.group = www-data

    ; Server Resources: 8G RAM | 4x CPU Cores
    ; `free -hl` output: Total 7.8Gi Used 331Mi Free 5.8Gi
    ; `ps_mem.py` output: 24.8 MiB + 9.0 MiB = 33.7 MiB
    pm = dynamic
    ; (Mem - Used) / ps_mem.py = max_children
    ; (7800 - 331) / 33.7 = 221.63
    pm.max_children = 220
    ; CPU * 4 = 16
    pm.start_servers = 16
    ; CPU * 2 = 8
    pm.min_spare_servers = 8
    ; Same as children
    pm.max_spare_servers = 16
    pm.max_requests = 500
    ```

    ```console
    $ sudo systemctl reload php7.4-fpm
    ```
    4 changes: 3 additions & 1 deletion 4_new-vhost.md → 5_new-vhost.md
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,6 @@
    # Create a new nginx vhost
    # Create a New Nginx Host

    It's time to setup your domain on nginx and this is how you go about enabling it and ensuring PHP-FPM is working for the host.

    ```console
    $ sudo nano /etc/nginx/sites-available/example.com
    File renamed without changes.
    File renamed without changes.
    File renamed without changes.
    File renamed without changes.
  27. justinhartman revised this gist Dec 8, 2020. 1 changed file with 2 additions and 2 deletions.
    4 changes: 2 additions & 2 deletions 3_configure-php-nginx.md
    Original file line number Diff line number Diff line change
    @@ -72,9 +72,9 @@ process_control_timeout 10s

    ### An Overview

    You are probably going to want to read [this article on FPM tuning][tuning] which I used to determine the settings below. The article will help you scientifically calculate the exact setting you should use for `max_children`, `start_servers`, `min_spare_servers`, and `max_spare_servers`. The most complicated one to calculate is `max_children` because there is a formula he uses to do this which is `(Total RAM - Memory used on server) / Process Size = max_children`. I've tried my best to give you the succinct version of what to calculate and how but depending on your level of skill you may need to read the article fully.
    You are probably going to want to read [this article on FPM tuning][tuning] which I used to determine the settings below. The article will help you scientifically calculate the exact setting you should use for `max_children`, `start_servers`, `min_spare_servers`, and `max_spare_servers`. The most complicated one to calculate is `max_children` because there is a formula he uses to do this which is `(Total RAM - Memory used on server) / Process Size = max_children`. I've tried my best to give you the succinct version of what to calculate, as well as how, but depending on your level of skill, you may need to read the article fully.

    There is also reference to a script called `ps_mem.py` which you can [download directly from here][psmem]. This script will give you the total `Process Size` needed in the `max_children` formula above.
    There is also reference to a script called `ps_mem.py` which you can [download directly from here][psmem], or you can run `wget https://raw.githubusercontent.com/pixelb/ps_mem/master/ps_mem.py` from your server to download the file there instead. This script will give you the total `Process Size` needed in the `max_children` formula above and is an amazing tool for helping you determine your resources available.

    ### Calculations

  28. justinhartman revised this gist Dec 8, 2020. 5 changed files with 53 additions and 27 deletions.
    2 changes: 1 addition & 1 deletion 0_introduction.md
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    # How To Install Linux, Nginx, MySQL, PHP on an Ubuntu Azure Virtual Machine
    # How To Install Linux, Nginx, MySQL, PHP on a Debian 10 Virtual Machine

    This series of documents will configure and setup a Linux, Nginx, MySQL, and
    PHP (LEMP) server on a pretty _feature-rich_ VM running Debian. I used a
    2 changes: 1 addition & 1 deletion 2_install-lemp.md
    Original file line number Diff line number Diff line change
    @@ -4,7 +4,7 @@ There are three steps to install the LEMP server and then there are some configu

    ## Step 1 - Install Nginx

    ```
    ```console
    $ sudo apt update
    $ sudo apt install nginx
    $ sudo systemctl start nginx
    72 changes: 49 additions & 23 deletions 3_configure-php-nginx.md
    Original file line number Diff line number Diff line change
    @@ -62,20 +62,57 @@ $ sudo systemctl reload php7.4-fpm
    $ sudo nano /etc/php/7.4/fpm/php-fpm.conf
    ```

    ```ini
    ```nginx
    emergency_restart_threshold 10
    emergency_restart_interval 1m
    process_control_timeout 10s
    ```

    ## Configure php-fpm www.conf

    ### An Overview

    You are probably going to want to read [this article on FPM tuning][tuning] which I used to determine the settings below. The article will help you scientifically calculate the exact setting you should use for `max_children`, `start_servers`, `min_spare_servers`, and `max_spare_servers`. The most complicated one to calculate is `max_children` because there is a formula he uses to do this which is `(Total RAM - Memory used on server) / Process Size = max_children`. I've tried my best to give you the succinct version of what to calculate and how but depending on your level of skill you may need to read the article fully.

    There is also reference to a script called `ps_mem.py` which you can [download directly from here][psmem]. This script will give you the total `Process Size` needed in the `max_children` formula above.

    ### Calculations

    Here are the settings and calculations. The `Value` column has details on how to calculate the `Setting` value.

    | Setting | Value |
    |:---------------------|:---------------------------------------------------------|
    | `max_children` | (`Total RAM` - `Memory` used on server) / `Process Size` |
    | `start_servers` | Number of CPU cores x 4 |
    | `min_spare_servers` | Number of CPU cores x 2 |
    | `max_spare_servers` | Equal to same value as `start_servers` |

    #### How to calculate max_children

    If you have a server with **8GB RAM** and **4 CPU Cores** this is how you work this out.

    `(Total RAM - Memory used on server) / Process Size = max_children`

    - `Total RAM` = `7800` (for 7.8GB) run `sudo free -hl` from the command line to get this
    - `Memory Used` = Run `sudo free -hl` from the command line to get the total used memory (e.g. 331MB)
    - `Process Size` = Run the [ps_mem.py][psmem] script by executing `sudo python ps_mem.py | grep php-fpm`. This would output something similar to this `24.8 MiB + 9.0 MiB = 33.7 MiB`. The value of `Process Size` is therefor `33.7`.

    Now you pull this all together and run the formula of `(Total RAM - Memory used on server) / Process Size` it will give you something like this:

    > **(7800 - 331) / 33.7 = 221.63**
    Therefore: `max_children = 222`

    The other calculations are now easy because they are all based on how many CPU Cores your have. Open up the `www.conf` file and begin editing with your newly calculated values.

    ### Edit the Settings

    Change the `www` pool so it now has the following settings.

    ```console
    $ sudo nano /etc/php/7.4/fpm/pool.d/www.conf
    ```

    Change so the `www` pool has the following:

    ```ini
    [www]
    user = www-data
    @@ -84,26 +121,12 @@ listen = /run/php/php7.4-fpm.sock
    listen.owner = www-data
    listen.group = www-data

    ; How to Calculate the settings.
    ; Read this article for details on full calculations + ps_mem.py script:
    ; https://tideways.com/profiler/blog/an-introduction-to-php-fpm-tuning
    ; wget https://raw.githubusercontent.com/pixelb/ps_mem/master/ps_mem.py
    ;
    ; | Setting | Value |
    ; |:-------------------|:----------------------------------------------------|
    ; | max_children | (Total RAM - Memory used on server) / process size |
    ; | | use 'free -hl' to get memory used |
    ; | start_servers | Number of CPU cores x 4 |
    ; | min_spare_servers | Number of CPU cores x 2 |
    ; | max_spare_servers | Same as start_servers |
    ;
    ; Server Resources: 7.77G | 331MB used | 4x CPU Cores
    ; sudo python ps_mem.py | grep php-fpm
    ; Used Memory: 24.8 MiB + 9.0 MiB = 33.7 MiB
    ;
    ; Server Resources: 8G RAM | 4x CPU Cores
    ; `free -hl` output: Total 7.8Gi Used 331Mi Free 5.8Gi
    ; `ps_mem.py` output: 24.8 MiB + 9.0 MiB = 33.7 MiB
    pm = dynamic
    ; (Mem - Used) / ps_mem.py = Total Children
    ; (7770 - 331) / 33.7 = 220.74
    ; (Mem - Used) / ps_mem.py = max_children
    ; (7800 - 331) / 33.7 = 221.63
    pm.max_children = 220
    ; CPU * 4 = 16
    pm.start_servers = 16
    @@ -161,4 +184,7 @@ $ sudo nginx -t
    nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
    nginx: configuration file /etc/nginx/nginx.conf test is successful
    $ sudo systemctl reload nginx
    ```
    ```

    [tuning]: https://tideways.com/profiler/blog/an-introduction-to-php-fpm-tuning
    [psmem]: https://raw.githubusercontent.com/pixelb/ps_mem/master/ps_mem.py
    2 changes: 1 addition & 1 deletion 6_sftp-setup.md
    Original file line number Diff line number Diff line change
    @@ -16,7 +16,7 @@ $ sudo chown -R FTPUSER:sftp /home/FTPUSER/.ssh
    $ sudo nano /etc/ssh/sshd_config
    ```

    ```conf
    ```nginx
    PubkeyAuthentication yes
    Match group sftp
    X11Forwarding no
    2 changes: 1 addition & 1 deletion 7_install-redis.md
    Original file line number Diff line number Diff line change
    @@ -14,7 +14,7 @@ $ sudo nano /etc/redis/redis.conf

    Change these settings:

    ```nginx
    ```ini
    supervised systemd # use Debian's systemd
    bind 127.0.0.1 ::1
    # If you want to secure your Redis server set a password below.
  29. justinhartman revised this gist Dec 8, 2020. 8 changed files with 173 additions and 101 deletions.
    14 changes: 8 additions & 6 deletions 0_introduction.md
    Original file line number Diff line number Diff line change
    @@ -8,16 +8,18 @@ These documents take you through each step of the configuration from:

    1. [General Setup][gen]
    1. [Install Linux, Nginx, MySQL, and PHP][lemp]
    1. [Confiure PHP and Nginx][config]
    1. [Create a new nginx vhost][vhost]
    1. [Setup Let's Encrypt SSL certificates][ssl]
    1. [Setup SFTP server][sftp]
    1. [Install Redis Server][redis]
    1. [Download and Configure phpMyAdmin][pma]
    1. [Install Redis][redis]

    [gen]: #file-1_general-md
    [lemp]: #file-2_install-lemp-md
    [vhost]: #file-3_new-vhost-md
    [ssl]: #file-4_setup-ssl-md
    [sftp]: #file-5_sftp-setup-md
    [pma]: #file-6_phpmyadmin-md
    [redis]: #file-7_install-redis-md
    [config]: #file-3_configure-php-nginx-md
    [vhost]: #file-4_new-vhost-md
    [ssl]: #file-5_setup-ssl-md
    [sftp]: #file-6_sftp-setup-md
    [redis]: #file-7_install-redis-md
    [pma]: #file-8_phpmyadmin-md
    94 changes: 0 additions & 94 deletions 2_install-lemp.md
    Original file line number Diff line number Diff line change
    @@ -44,97 +44,3 @@ $ sudo apt -y install php7.4-{fpm,cli,apc,bcmath,bz2,curl,gd,imagick,intl,mbstri
    $ sudo systemctl start php7.4-fpm
    $ sudo systemctl enable php7.4-fpm
    ```

    ### Configure php.ini

    ```console
    $ sudo nano /etc/php/7.4/fpm/php.ini
    ```

    Change these values:

    ```ini
    cgi.fix_pathinfo=0
    post_max_size=256M
    upload_max_filesize=256M
    date.timezone=Africa/Johannesburg
    [opcache]
    opcache.memory_consumption=256
    opcache.max_accelerated_files=10000
    opcache.validate_timestamps=120
    opcache.error_log=/var/log/opcache-errors.log
    [apc]
    apc.shm_size=128M
    apc.stat=1
    ; Relative to the number of cached files (you may need to watch your stats for
    ; a day or two to find out a good number).
    apc.num_files_hint=7000
    ; The number of seconds a cache entry is allowed to idle in a slot before APC
    ; dumps the cache.
    apc.ttl=7200
    apc.user_ttl=7200
    apc.include_once_override=0
    ; Allow 2 seconds after a file is created before it is cached to prevent users
    ; from seeing half-written/weird pages.
    apc.file_update_protection=2
    ; Ignore files
    apc.filters = "/var/www/html/apc.php"
    apc.cache_by_default=1
    apc.use_request_time=1
    apc.slam_defense=0
    apc.stat_ctime=0
    apc.canonicalize=1
    apc.write_lock=1
    apc.report_autofilter=0
    apc.rfc1867=0
    apc.rfc1867_prefix=upload_
    apc.rfc1867_name=APC_UPLOAD_PROGRESS
    apc.rfc1867_freq=0
    apc.rfc1867_ttl=3600
    apc.lazy_classes=0
    apc.lazy_functions=0
    ```

    ```console
    $ sudo systemctl reload php7.4-fpm
    $ sudo systemctl reload nginx
    ```

    ### Configure nginx

    ```console
    $ sudo nano /etc/nginx/nginx.conf
    ```

    Change these values:

    ```nginx
    client_max_body_size 128M;
    keepalive_timeout 2;
    server_tokens off;
    ```

    Edit default host to enable PHP-FPM.

    ```console
    $ sudo nano /etc/nginx/sites-available/default
    ```

    Change these values:

    ```nginx
    index index.php index.html index.htm index.nginx-debian.html;
    location ~ \.php$ {
    include snippets/fastcgi-php.conf;
    fastcgi_pass unix:/run/php/php7.4-fpm.sock;
    }
    ```

    Test the output and reload.

    ```console
    $ sudo nginx -t
    nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
    nginx: configuration file /etc/nginx/nginx.conf test is successful
    $ sudo systemctl reload nginx
    ```
    164 changes: 164 additions & 0 deletions 3_configure-php-nginx.md
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,164 @@
    # Configure PHP-FPM and Nginx

    These are the different configuration files for setting up the servers.

    ## Configure php.ini

    ```console
    $ sudo nano /etc/php/7.4/fpm/php.ini
    ```

    Change these values:

    ```ini
    cgi.fix_pathinfo=0
    post_max_size=256M
    upload_max_filesize=256M
    date.timezone=Africa/Johannesburg
    [opcache]
    opcache.memory_consumption=256
    opcache.max_accelerated_files=10000
    opcache.validate_timestamps=120
    opcache.error_log=/var/log/opcache-errors.log
    [apc]
    apc.shm_size=128M
    apc.stat=1
    ; Relative to the number of cached files (you may need to watch your stats for
    ; a day or two to find out a good number).
    apc.num_files_hint=7000
    ; The number of seconds a cache entry is allowed to idle in a slot before APC
    ; dumps the cache.
    apc.ttl=7200
    apc.user_ttl=7200
    apc.include_once_override=0
    ; Allow 2 seconds after a file is created before it is cached to prevent users
    ; from seeing half-written/weird pages.
    apc.file_update_protection=2
    ; Ignore files
    apc.filters = "/var/www/html/apc.php"
    apc.cache_by_default=1
    apc.use_request_time=1
    apc.slam_defense=0
    apc.stat_ctime=0
    apc.canonicalize=1
    apc.write_lock=1
    apc.report_autofilter=0
    apc.rfc1867=0
    apc.rfc1867_prefix=upload_
    apc.rfc1867_name=APC_UPLOAD_PROGRESS
    apc.rfc1867_freq=0
    apc.rfc1867_ttl=3600
    apc.lazy_classes=0
    apc.lazy_functions=0
    ```

    ```console
    $ sudo systemctl reload php7.4-fpm
    ```

    ## Configure php-fpm.conf

    ```console
    $ sudo nano /etc/php/7.4/fpm/php-fpm.conf
    ```

    ```ini
    emergency_restart_threshold 10
    emergency_restart_interval 1m
    process_control_timeout 10s
    ```

    ## Configure php-fpm www.conf

    ```console
    $ sudo nano /etc/php/7.4/fpm/pool.d/www.conf
    ```

    Change so the `www` pool has the following:

    ```ini
    [www]
    user = www-data
    group = www-data
    listen = /run/php/php7.4-fpm.sock
    listen.owner = www-data
    listen.group = www-data

    ; How to Calculate the settings.
    ; Read this article for details on full calculations + ps_mem.py script:
    ; https://tideways.com/profiler/blog/an-introduction-to-php-fpm-tuning
    ; wget https://raw.githubusercontent.com/pixelb/ps_mem/master/ps_mem.py
    ;
    ; | Setting | Value |
    ; |:-------------------|:----------------------------------------------------|
    ; | max_children | (Total RAM - Memory used on server) / process size |
    ; | | use 'free -hl' to get memory used |
    ; | start_servers | Number of CPU cores x 4 |
    ; | min_spare_servers | Number of CPU cores x 2 |
    ; | max_spare_servers | Same as start_servers |
    ;
    ; Server Resources: 7.77G | 331MB used | 4x CPU Cores
    ; sudo python ps_mem.py | grep php-fpm
    ; Used Memory: 24.8 MiB + 9.0 MiB = 33.7 MiB
    ;
    pm = dynamic
    ; (Mem - Used) / ps_mem.py = Total Children
    ; (7770 - 331) / 33.7 = 220.74
    pm.max_children = 220
    ; CPU * 4 = 16
    pm.start_servers = 16
    ; CPU * 2 = 8
    pm.min_spare_servers = 8
    ; Same as children
    pm.max_spare_servers = 16
    pm.max_requests = 500
    ```

    ```console
    $ sudo systemctl reload php7.4-fpm
    ```

    ## Configure nginx

    ```console
    $ sudo nano /etc/nginx/nginx.conf
    ```

    Change these values:

    ```nginx
    client_max_body_size 128M;
    keepalive_timeout 2;
    server_tokens off;
    ```

    ```console
    $ sudo systemctl reload nginx
    ```

    ## Edit default host to enable PHP-FPM

    ```console
    $ sudo nano /etc/nginx/sites-available/default
    ```

    Change only these values in the host:

    ```nginx
    server {
    index index.php index.html index.htm index.nginx-debian.html;
    location ~ \.php$ {
    include snippets/fastcgi-php.conf;
    fastcgi_pass unix:/run/php/php7.4-fpm.sock;
    }
    }
    ```

    Test the output and reload.

    ```console
    $ sudo nginx -t
    nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
    nginx: configuration file /etc/nginx/nginx.conf test is successful
    $ sudo systemctl reload nginx
    ```
    File renamed without changes.
    File renamed without changes.
    File renamed without changes.
    2 changes: 1 addition & 1 deletion 7_install-redis.md
    Original file line number Diff line number Diff line change
    @@ -14,7 +14,7 @@ $ sudo nano /etc/redis/redis.conf

    Change these settings:

    ```env
    ```nginx
    supervised systemd # use Debian's systemd
    bind 127.0.0.1 ::1
    # If you want to secure your Redis server set a password below.
    File renamed without changes.
  30. justinhartman revised this gist Dec 8, 2020. 1 changed file with 2 additions and 2 deletions.
    4 changes: 2 additions & 2 deletions 7_install-redis.md
    Original file line number Diff line number Diff line change
    @@ -14,7 +14,7 @@ $ sudo nano /etc/redis/redis.conf

    Change these settings:

    ```config
    ```env
    supervised systemd # use Debian's systemd
    bind 127.0.0.1 ::1
    # If you want to secure your Redis server set a password below.
    @@ -23,7 +23,7 @@ bind 127.0.0.1 ::1

    ## Restart Redis

    ```redis
    ```console
    $ sudo systemctl restart redis
    $ sudo systemctl status redis
    ```