Using install.sh during deployments

When is the install.sh executed?

The install.sh is run to install the projects on each application instance to complete either the deployment of a new release or the provisioning of a new instance. Since the configuration of the necessary steps is to be done by the project, the install.sh must be part of the source code.

To install by install.sh include the following steps:

  1. Configure the application with the appropriate endpoints of the environment

  2. Installation of crons

  3. Execute necessary PHP commands

The steps can be adapted as desired to the requirements of individual projects.

Background information

  • The install.sh is run as www-data user. All file operations are therefore executed in the context of the web server in order to avoid access problems.

  • The install.sh must exit successfully (exitcode 0), otherwise the deployment will be canceled for the affected instance. This is why you should append " || exit $?" for all commands that are critical to the deployment (usually all of them), thereby exiting the deployment with the errorcode of the failed command. If this is not specified, the deployment or the provisioning continues even in the event of an unsuccessful command, making it much harder to debug.

  • The execution of the install.sh always takes place in the root directory of the repository

What to avoid in your install.sh

Clearing you central cache

Each new instance will run the install.sh and flush your cache. So if autoscaling is starting new instance each of them will flush the cache, which works against the autoscaling which was triggered by high cpu load. Instead use deployment hooks for flushing caches.

Creating database dumps/snapshots

Each instance during deployment will create a dump which will result in multiple dumps at the same time and or several dumps during autoscaling. Instead use deployment hooks for creating snapshots/dumps.

Creating cronjobs without “run-once-per-role” on autoscaling systems

The install.sh is run on each instance during deployment and on each new instance that is created via autoscaling. If you create cronjobs without “run-once-per-role” each instance will have the same cronjob. So your cronjob will be executed multiple times which might cause harm to your application.

Examples

In all examples you will notice that we use some form of template (.dist) and replace the application config with this file after all placeholders have been replaced in the template. This is most resilient way to create application configs. As you can have a config file for local testing in your release and the Root360 deployment just overwrites it via install.sh. So you can use the same release for local testing and for the Root360 Platform.

Generic example

An example install.sh might look like this

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 #! /bin/bash # inject environment variables (e.g. db/redis/ses endpoints) to .env envsubst < .env.dist > .env || exit $? # install CRON if role "backend" is installed if [[ "${ROLE}" == "backend" ]] then # cron example echo "* * * * * cd /var/www/ && /usr/bin/php cron.php > /dev/null 2>&1" >> project-crontab # cron example with custom logging echo "* * * * * date >> /var/log/application/cron.log && cd /var/www/ && /usr/bin/php cron.php >> /var/log/application/cron.log 2>&1" >> project-crontab # cron example to register dynamic application log files stored in a dedicated log folder echo "* * * * * /usr/local/bin/check-log-registration /var/www/${ROLE}/logs/" >> project-crontab # cron example to run a command on only one instance per role echo "* * * * * /usr/local/bin/run-once-per-role.sh 'web' /var/www/web/public/bin/artisan cron:start" >> project-crontab crontab project-crontab || exit $? rm project-crontab fi # register custom application log file register-log -k "/var/www/${ROLE}/log_dir/custom_application.log"

 

Shopware6 example for .env.dist and install.sh

.env.dist

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 ## Database Config DB_USER=$DATABASE_USER DB_PASSWORD=$DATABASE_PASSWORD DB_HOST=$DATABASE_HOST DB_NAME=$DATABASE_NAME DB_PORT=$DATABASE_PORT ## Elasticsearch Config ES_ENABLED=true ES_AWS=true ES_VERSION=6.8.0 ES_CLUSTER=MASTER ES_MASTER_HOST=https://$ELASTICSEARCH_HOST:%ELASTICSEARCH_PORT% ## Redis Config REDIS=true REDIS_SESSION_HOST=$REDIS_HOST REDIS_SESSION_PORT=$REDIS_PORT

install.sh

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 #! /bin/bash # inject environment variables (e.g. db/redis/ses endpoints) to .env envsubst < .env.aws > public/.env || exit $? if [[ "${ROLE}" == "backend" ]] then echo "*/15 * * * * /usr/local/bin/check-log-registration /var/www/backend/public/var/log backend" >> project-crontab crontab project-crontab || exit $? rm project-crontab fi if [[ "${ROLE}" == "frontend" ]] then echo "*/15 * * * * /usr/local/bin/check-log-registration /var/www/frontend/public/var/log frontend" >> project-crontab crontab project-crontab || exit $? rm project-crontab fi cd ./public || exit $? # password protection for stage environments including ip whitelisting if [[ "${ENV}" == "prod" ]] && [[ "${ROLE}" == "backend" ]] then echo " <RequireAny> AuthType Basic AuthName \"Protected Area\" AuthUserFile /var/www/${ROLE}/public/.htpasswd AuthType Basic # Whitelisted IP Require ip [insert IP Adress] Require valid-user </RequireAny> " >> .htaccess || exit $? fi

 

 

Magento2 Example for dist.env.php and install.sh

dist.env.php

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 <?php return [ 'backend' => [ 'frontName' => 'backend' ], 'crypt' => [ 'key' => '$MAGENTO_CRYPT' ], 'db' => [ 'table_prefix' => '', 'connection' => [ 'default' => [ 'host' => '$DATABASE_HOST', 'dbname' => '$DATABASE_NAME', 'username' => '$DATABASE_USER', 'password' => '$DATABASE_PASSWORD', 'model' => 'mysql4', 'engine' => 'innodb', 'initStatements' => 'SET NAMES utf8;', 'active' => '1', 'driver_options' => [ 1014 => false ] ] ] ], 'x-frame-options' => 'SAMEORIGIN', 'MAGE_MODE' => 'production', 'session' => [ 'save' => 'redis', 'redis' => [ 'host' => '$REDIS_HOST', 'port' => '$REDIS_PORT', 'password' => '', 'timeout' => '2.5', 'persistent_identifier' => '', 'database' => '1', 'compression_threshold' => '2048', 'compression_library' => 'gzip', 'log_level' => '1', 'max_concurrency' => '32', 'break_after_frontend' => '5', 'break_after_adminhtml' => '30', 'first_lifetime' => '600', 'bot_first_lifetime' => '60', 'bot_lifetime' => '7200', 'disable_locking' => '0', 'min_lifetime' => '60', 'max_lifetime' => '2592000', 'sentinel_master' => '', 'sentinel_servers' => '', 'sentinel_connect_retries' => '5', 'sentinel_verify_master' => '0' ] ] ];

install.sh

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 #! /bin/bash # # Write application configuration # inject environment variables (e.g. db/redis/ses endpoints) to env.php envsubst < dist.env.php > app/etc/env.php || exit $? # Reset the crontab on every system crontab -r if [[ "${ROLE}" == "backend" ]] then # Install magento cron on backend php bin/magento cron:install || exit $? fi # Add cron for log rotation on prod if [[ "${ENV}" == "prod" ]] then (crontab -l 2>/dev/null; echo "0 * * * * /usr/local/bin/check-log-registration /var/www/${ROLE}/var/log/") | crontab - fi

 

 

TYPO3 example for dist.env.php and install.sh

dist.env.php

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 <?php $GLOBALS['TYPO3_CONF_VARS']['DB']['Connections'] = [ 'Default' => [ 'charset' => 'utf8', 'dbname' => '$DATABASE_NAME', 'driver' => 'mysqli', 'host' => '$DATABASE_HOST', 'password' => '$DATABASE_PASSWORD', 'port' => $DATABASE_PORT, 'unix_socket' => '', 'user' => '$DATABASE_USER' ] ]; $GLOBALS['TYPO3_CONF_VARS']['SYS']['session'] = [ 'BE' => [ 'backend' => \TYPO3\CMS\Core\Session\Backend\RedisSessionBackend::class, 'options' => [ 'hostname' => '$REDIS_HOST', 'database' => '1', 'port' => $REDIS_PORT ] ], 'FE' => [ 'backend' => \TYPO3\CMS\Core\Session\Backend\RedisSessionBackend::class, 'options' => [ 'hostname' => '$REDIS_HOST', 'database' => '2', 'port' => $REDIS_PORT ] ] ];

install.sh

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 #! /bin/bash # # Write application configuration # inject environment variables (e.g. db/redis/ses endpoints) to env.php envsubst < dist.env.php > app/etc.env.php || exit $? # Reset the crontab on every system crontab -r if [[ "${ROLE}" == "backend" ]] then # Install typo3 cron on backend echo "Install crontab" echo "* * * * * cd /var/www/typo3; 'typo3' /usr/bin/php -d memory_limit=4G /var/www/typo3/vendor/bin/typo3 scheduler:run" | crontab fi # Add cron for log rotation on prod if [[ "${ENV}" == "prod" ]] then (crontab -l 2>/dev/null; echo "0 * * * * /usr/local/bin/check-log-registration /var/www/${ROLE}/var/log/") | crontab - fi

 

See Scripts Snippets for an explanation of the used scripts register-logcheck-log-registration and run-once-per-role.sh.

Explanation of envsubst

The envsubst program will replace placeholder variables with the values from the appropriate environment variables for the role. The result is written into the configuration file used. In the example, a template configuration file .env.dist is used to provide the placeholders. The environment variables for each project can be taken by executing sudo get-application-env on the respective application instance.

1 2 # example .env.dist DATABASE_URL="mysql://$DATABASE_USER:$DATABASE_PASSWORD@$DATABASE_HOST:$DATABASE_PORT/$DATABASE_NAME"

Explanation of "Installing CRONs"

The example shows the possibility to install a CRON. This installation is installed in the example only for the "backend" role. With this methodology, they allow all commands to be applied to different roles.

Explanation of "Running Required PHP Commands"

For Shopware, for example, you may want the attributes to be regenerated. The following command should be added:

1 php -d bin / console sw: generate: attributes ||  exit $?

Related tutorials

Related components