Additional Configuration Options

Advanced morpheus.rb Settings

Morpheus allows for additional advanced customizations to the morpheus.rb file located in /etc/morpheus/morpheus.rb. Below is a list of the supported items available in the morpheus.rb file.

appliance_url 'https://morpheus.appliance-url.com'
  # The appliance_url must not contain a trailing `/`.
  # Appending alternate port to appliance_url is supported. ie 'https://morpheus.appliance-url.com:8443'
  # The appliance_url cannot exceed 64 characters

app['encrypted_key_suffix'] = 'suffix'

ui['http_client_connect_timeout'] = 10000  #in seconds
ui['http_client_connect_timeout'] = 600000 #in seconds
ui['kerberos_config'] = nil
ui['kerberos_login_config'] = nil
ui['log_dir'] = '/var/log/morpheus/morpheus-ui'
ui['max_memory_mb'] = nil
ui['memory_alloc_arena_max'] = 2
ui['memory_map_max'] = 65536
ui['memory_map_threshold'] = 131072
ui['memory_top_pad'] = 131072
ui['memory_trim_threshold'] = 131072
ui['vm_images_cdn_url'] = 'https://morpheus-images.morpheusdata.com'

mysql['enable'] = true
mysql['host'] = {'127.0.0.1' => 3306}
mysql['log_dir'] = '/var/log/morpheus/mysql'
mysql['max_active'] = 100
mysql['morpheus_db_user'] = 'morpheus'
mysql['morpheus_db'] = 'morpheus'
mysql['mysql_url_overide'] = 'jdbc:mysql://10.30.20.10:3306,10.30.20.11:3306,10.30.20.12:3306/morpheusdb?autoReconnect=true&useUnicode=true&characterEncoding=utf8&failOverReadOnly=false&useSSL=false'
mysql['tmp_dir'] = '/tmp/mysql'

logging['svlogd_num'] = 30 # keep 30 rotated log files
logging['svlogd_size'] = 209715200 # 200 MB in bytes
logging['svlogd_timeout'] = 86400 # rotate after 24 hours in seconds

rabbitmq['enable'] = true
rabbitmq['heartbeat'] = nil
rabbitmq['host'] = '127.0.0.1'
rabbitmq['log_dir'] = '/var/log/morpheus/rabbitmq'
rabbitmq['nodename'] = 'rabbit@localhost'
rabbitmq['port'] = '5672'
rabbitmq['queue_user'] = 'queue_user'
rabbitmq['stomp_port'] = 61613
rabbitmq['vhost'] = 'morpheus'

elasticsearch['log_dir'] = '/var/log/morpheus/elaseticsearch'
elasticsearch['enable'] = true
elasticsearch['es_hosts'] = {'127.0.0.1' => 9200}
elasticsearch['host'] = "127.0.0.1"
elasticsearch['memory_alloc_arena_max'] = 2
elasticsearch['memory_map_max'] = 65536
elasticsearch['memory_map_threshold'] = 131072
elasticsearch['memory_top_pad'] = 131072
elasticsearch['memory_trim_threshold'] = 131072
elasticsearch['open_files'] = 204800
elasticsearch['replica_count'] = 1

nginx['cache_max_size'] = '5000m'
nginx['enable'] = true
nginx['loading_pages']['failure_page_h1'] = 'Morpheus Server Error'
nginx['loading_pages']['failure_page_h2'] = 'Please contact your system administrator for assistance.'
nginx['loading_pages']['failure_page_title'] = 'Morpheus Server Error'
nginx['loading_pages']['iteration_time'] = 10000 # milliseconds
nginx['loading_pages']['loading_page_h1'] = 'Morpheus is Loading...'
nginx['loading_pages']['loading_page_h2'] = 'please wait'
nginx['loading_pages']['loading_page_title'] = 'Morpheus Loading'
nginx['loading_pages']['max_loops'] = 60 # seconds
nginx['loading_pages']['timeout_page'] = '/timeout.html'
nginx['loading_pages']['timout_page_h1'] = 'Timeout waiting for Morpheus to load, click below to try again.'
nginx['loading_pages']['timout_page_title'] = 'Morpheus timeout, please try again...'
nginx['ssl_ciphers'] = "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4"
nginx['ssl_company_name'] = "Morpheus, LLC"
nginx['ssl_country_name'] = "US"
nginx['ssl_email_address'] = "personal@email.com"
nginx['ssl_locality_name'] = "San Mateo"
nginx['ssl_organizational_unit_name'] = "DevOps"
nginx['ssl_protocols'] = "TLSv1 TLSv1.1 TLSv1.2"
nginx['ssl_session_cache'] = "builtin:1000  shared:SSL:10m"
nginx['ssl_session_timeout'] = "5m"
nginx['ssl_state_name'] = "CA"
nginx['worker_connections'] = 10240
nginx['workers'] = integer calculated from number of cpus

repo['repo_host_url'] = 'https://downloads.morpheusdata.com'

Note

elasticsearch[‘replica_count’] settings only apply to local Elasticsearch and not an external cluster. The user must set the replica count in the code for each index. The setting in morpheus.rb is only the cluster default and only applies to the all-in-one appliance. If the cluster is external, the user must set the default on their Elasticsearch config file.

Offline Installations and Upgrades

For customers that have an appliance behind a firewall/proxy that does not allow downloads from our Amazon download site, you can add the supplemental package to add the needed packages the standard Morpheus installer would have downloaded.

Offline Installation Requirements

  • NTP should be correctly configured and the server is able to connect to the NTP server in the ntp.conf file
  • The OS package repositories should be configured to use local LAN repository servers or the server should be able to receive packages from the configured repositories
  • The standard Morpheus and supplemental packages must be downloaded from another system and transferred to the Morpheus Appliance server
  • The supplemental package is additive, the full installer is also required

Note

The supplemental package is linked 1-to-1 to the appliance release. For example the supplemental package for 4.2.1-1 should be used with the appliance package 4.2.1-1

Offline Install

Ubuntu/Debian

  1. Download both the regular Morpheus Appliance package and the Supplemental packages on to the appliance server:

    wget http://example_url/morpheus-appliance_version_amd64.deb
    wget http://example_url/morpheus-appliance-supplemental_version_all.deb
    
  2. Install the both the Appliance package AND the Supplemental package.

    sudo dpkg -i morpheus-appliance_version_amd64.deb
    sudo dpkg -i morpheus-appliance-supplemental_version_all.deb
    
  3. Set the Morpheus UI appliance url (if needed, hostname will be automatically set).

    # edit appliance_url to resolvable url (if not configured correctly by default)
    
    sudo vi /etc/morpheus/morpheus.rb
    
  4. Reconfigure the appliance to install required packages

    sudo morpheus-ctl reconfigure
    

The Chef run should complete successfully. There is a small pause when Chef runs the resource remote_file[package_name] action create while Chef verifies the checksum. After the reconfigure is complete, the morpheus-ui will start and be up in a few minutes.

Note

Tail the morpheus log file located at /var/log/morpheus/morpheus-ui/current with the command morpheus-ctl tail morpheus-ui and look for the Morpheus ascii logo to know when the morpheus-ui is up.

CentOS/RHEL

  1. Download both the regular Morpheus Appliance package and the matching Supplemental package on to the Appliance server:

    wget http://example_url/morpheus-appliance_package_url.noarch.rpm
    wget http://example_url/morpheus-appliance_package_supplemental_url.noarch.rpm
    
  2. Install the both the Appliance package AND the Supplemental package.

    sudo rpm -i morpheus-appliance_package_url.noarch.rpm
    sudo rpm -i morpheus-appliance_package_supplemental_url.noarch.rpm
    
  3. Set the Morpheus UI appliance url (if needed, hostname will be automatically set).

    #Edit appliance_url to resolvable url (if not configured correctly by default)
    
    sudo vi /etc/morpheus/morpheus.rb
    
  4. Reconfigure the appliance to install required packages

    sudo morpheus-ctl reconfigure
    

The Chef run should complete successfully. There is a small pause when Chef runs the resource remote_file[package_name] action create while Chef verifies the checksum. After the reconfigure is complete, the morpheus-ui will start and be up in a few minutes.

Note

Tail the morpheus-ui log file with morpheus-ctl tail morpheus-ui and look for the Morpheus ascii logo to know when the morpheus-ui is up.

Proxies

Overview

In many situations , companies deploy virtual machines in proxy restricted environments for things such as PCI Compliance, or just general security. As a result of this Morpheus provides out of the box support for proxy connectivity. Proxy authentication support is also provided with both Basic Authentication capabilities as well as NTLM for Windows Proxy environments. Morpheus is even able to configure virtual machines it provisions to utilize these proxies by setting up the operating systems proxy settings directly (restricted to cloud-init based Linux platforms for now, but can also be done on windows based platforms in a different manner).

To get started with Proxies, it may first be important to configure the Morpheus appliance itself to have access to proxy communication for downloading service catalog images. To configure this, visit the Admin -> Settings page where a section labeled “Proxy Settings” is located. Fill in the relevant connection info needed to utilize the proxy. It may also be advised to ensure that the Linux environment’s http_proxy, https_proxy, and no_proxy are set appropriately.

Defining Proxies

Proxies can be used in a few different contexts and optionally scoped to specific networks with which one may be provisioning into or on a cloud integration as a whole. To configure a Proxy for use by the provisioning engines within Morpheus we must go to Infrastructure -> Networks -> Proxies. Here we can create records representing connection information for various proxies. This includes the host ip address, proxy port, and any credentials (if necessary) needed to utilize the proxy. Now that these proxies are defined we can use them in various contexts.

Cloud Communication

When morpheus needs to connect to various cloud APIs to issue provisioning commands or to sync in existing environments, we need to ensure that those api endpoints are accessible by the appliance. In some cases the appliance may be behind a proxy when it comes to public cloud access like Azure and AWS. To configure the cloud integration to utilize aa proxy, when adding or editing a cloud there is a setting called “API Proxy” under “Advanced Options”. This is where the proxy of choice can be selected to instruct the Provisioning engine how to communicate with the public cloud. Simply adjust this setting and the cloud should start being able to receive/issue instructions.

Provisioning with Proxies

Proxy configurations can vary from operating system to operating system and in some cases it is necessary for these to be configured in the blueprints as a prerequisite. In other cases it can also be configured automatically. Mostly with the use of cloud-init (which all of our out of the box service catalog utilizes on all clouds). When editing/creating a cloud there is a setting for “Provisioning Proxy” in “Provisioning Options”. If this proxy is set, Morpheus will automatically apply these proxy settings to the guest operating system.

Overriding proxy settings can also be done on the Network record. Networks (or subnets) can be configured in Infrastructure -> Networks or on the Networks tab of the relevant Cloud detail page. Here, a proxy can also be assigned as well as additional options like the No Proxy rules for proxy exceptions.

Docker

When provisioning Docker based hosts within a Proxy environment it is up to the user to configure the docker hosts proxy configuration manually. There are workflows that can be configured via the Automation engine to make this automatic when creating docker based hosts. Please see documentation on Docker and proxies for specific information.

Proxy setups can vary widely from company to company, and it may be advised to contact support for help configuring morpheus to work in the proxy environment.

SSL Certificates

By default Morpheus generates a Self-Signed SSL Certificate. The Self-Signed SSL Certificate can be replaced with a Trusted CA Signed SSL Certificate.

Trusted CA Signed SSL Certificate Implementation

  1. Copy the private key and certificate to /etc/morpheus/ssl/your_fqdn_name.key and /etc/morpheus/ssl/your_fqdn_name.crt respectively.

  2. Edit the configuration file /etc/morpheus/morpheus.rb and add the following entries:

    nginx['ssl_certificate'] = 'path to the certificate file'
    nginx['ssl_server_key'] = 'path to the server key file'
    

    Note

    Both files should be owned by root and only readable by root, also if the server certificate is signed by an intermediate then you should include the signing chain inside the certificate file. The key file needs to be decrypted for Morpheus to install it properly.

  3. Next simply reconfigure the appliance and restart nginx:

    sudo morpheus-ctl reconfigure
    sudo morpheus-ctl restart nginx
    

Self-Signed SSL Certificate Regeneration

When Morpheus is deployed it generates a 10 year Self-Signed SSL Certificate. Below details the process to regenerate the Certificate and Key files.

Regenerate both the Certificate and Key

  1. Delete the certificate and key files in /etc/morpheus/ssl/.
  2. Run Reconfigure morpheus-ctl reconfigure.
  3. Restart NGINX morpheus-ctl restart nginx.

Regenerate only the Certificate

  1. Delete the certificate file in /etc/morpheus/ssl/.
  2. Run Reconfigure morpheus-ctl reconfigure.
  3. Restart NGINX morpheus-ctl restart nginx.

Import Trusted Certificates

Important

The following applies to upgrades after modifying the java keystore.

Steps to import trusted certificates to Morpheus after an upgrade.

  1. Obtain the full SSL certificate chain in PEM format.

  2. Copy them to each appliance and place them in the /etc/morpheus/ssl/trusted_certificates directory.

  3. Run morpheus-ctl reconfigure on each appliance, note you don’t need to stop Morpheus before you run this.

  4. Run the following command as root:

    export PATH=/opt/morpheus/sbin:/opt/morpheus/sbin:/opt/morpheus/embedded/sbin:/opt/morpheus/embedded/bin:$PATH
    
  5. Run the following command for each certificate in the chain, adjusting the file and alias name as needed. Answer yes for the root certificate when asked it you want to trust it.

    /opt/morpheus/embedded/java/jre/bin/keytool -import -keystore /opt/morpheus/embedded/java/jre/lib/security/cacerts -trustcacerts -file /etc/morpheus/ssl/trusted_certs/root_ca.pem -alias some_alias -storepass changeit
    
  6. Verify by running:

    openssl s_client -connect host:port -showcerts -tls1_2``
    
  7. You should get an output similar to:

    New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
    Server public key is 2048 bit
    Secure Renegotiation IS supported
    No ALPN negotiated
    SSL-Session:
    Protocol : TLSv1.2
    Cipher  : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 5D9E820E4FF2A73A9977BA663E6029AA5415FEE85F49D8B1E541F5997C8E1FB2
    Session-ID-ctx:
    Master-Key: 29EEC2E7750C659AECB9942902D9A87B824E571522812B718420FC08F8D2ACE68CB16EC812A7D90B12A86D1970FFD81C
    Key-Arg  : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1547219217
    Timeout  : 7200 (sec)
    Verify return code: 0 (ok) #<----------------
    
  8. If the certificates are installed correctly you should see Verify return code: 0 (ok). If they were not installed correctly then you will see a return similar to: Verify return code: 21 (unable to verify the first certificate)

  9. Repeat for all App Nodes

Data Encryption

By default, Morpheus encrypts sensitive data in the Database using AES encryption mode with GCM. Passwords and other strings in Morpheus Appliance configuration files such as morpheus-secrets.json and morpheus.rb are in plain text as they are only accessible by root.

Passwords and other strings in Morpheus Appliance configuration files can be set to an encrypted string using the Morpheus crypto utility to generate ENC strings and then using ENC(string) as the value in the configuration file.

Additionally a custom Encryption Key Suffix can be set in the morpheus.rb configuration file. This suffix will be combined with a system string to generate a SHA-256 hash, which is used to generate the AES encryption key.

Generate ENC Strings for morpheus-secrets.json

System generated passwords are set in /etc/morpheus/morpheus-secrets.json. These entries can be updated to ENC strings with the following steps:

  1. On the Morpheus appliance, run morpheus-ctl get-crypto-string migrate which will output ENC() strings for the passwords in morpheus-secrets.json
  2. Update the desired password strings in the morpheus-secrets.json config file with the matching ENC() string.
  3. Save morpheus-secrets.json
  4. Run morpheus-ctl reconfigure

Generate ENC Strings for custom morpheus.rb entries

ENC() strings can be generated for sensitive data set in morpheus.rb, such as the password to an external service.

To generate ENC(0) strings for morpheus.rb entries:

  1. On the Morpheus appliance, run morpheus-ctl get-crypto-string string $clear_text '$suffix' which will output strings for the passwords in morpheus-secrets.json

    • Replace $clear_text with the string to be encrypted
    • If a suffix is defined in morpheus.rb (as described in the next section), replace $suffix with your suffix.

    Note

    It is advisable to disable bash history logging by running unset HISTFILE before running the morphesu-ctl get-crypto-string command and then set HISTFILE=$HOME/.bash_history to reenable.

  2. Update the desired password strings in the morpheus.rb config file with the matching string output, using ENC($output) format

    • Example: mysql['morpheus_password'] = 'ENC($ZI5DnaO0quhxKe$kDFD+U2ZeJUuYiNC$F1+czPNyo+3lAdq7V0gcrWwHnkINYqr13cUGrDVyog==)'
  3. Save morpheus.rb

  4. Run morpheus-ctl reconfigure

Encrypted Key Suffix

A custom Encryption Key Suffix can be set in the morpheus.rb configuration file. This suffix will be combined with a system string to generate a SHA-256 Hash, which is used in the generation of the system AES encryption key.

Danger

Setting a custom Encryption Key Suffix affects all data encrypted by Morpheus, including database and cypher data. Encryption Key Suffix is required in the event data needs to be migrated or recovered. Once the Encryption Key Suffix is set, data cannot be recovered without it. Store any Encryption Key Suffix externally where it can be referenced for future scenarios.

Important

The Encryption Key Suffix cannot be changed or removed after being set. Changing or removing an existing Encryption Key Suffix will prevent data access. If an existing suffix is altered in the morpheus.rb file, it must be restore to its original value.

  1. Add app['encrypted_key_suffix'] = '$suffix' to /etc/morpheus/morpheus.rb, replacing $suffix with your suffix.

    Danger

    Once an Encryption Key Suffix is set and applied via reconfigure, it cannot be altered or removed and data cannot be migrated or recovered without it.

  2. Run morpheus-ctl reconfigure

    • Reconfigure will generate a new encryption key using the suffix and set (ENC) values for service password in application.yml