Server setup

This document describe how the server must be setup for production use.

Note

Replace customer-infra by the true name of your infrastructure.

Common Requirements

  • Apache 2 with:
    • mod_rewrite. To enable mod_rewrite on Debian based systems, use a2enmod rewrite
    • mod_expires. To enable mod_expires on Debian based systems, use a2enmod expires
    • mod_headers. To enable mod_headers on Debian based systems, use a2enmod headers
    • mod_fcgid. To enable mod_fcgid on Debian based systems install the libapache2-mod-fcgid package (aptitude install libapache2-mod-fcgid) and on openSuse, install apache2-mod_fcgid.
    • mod_filter. To enable mod_filter on Debian based systems, use a2enmod filter
    • mod_deflate. To enable mod_deflate on Debian based systems, use a2enmod deflate
    • mod_proxy, mod_proxy_ajp and mod_proxy_http. To enable mod_proxy, mod_proxy_ajp and mod_proxy_http on Debian based systems, use a2enmod proxy, a2enmod proxy_ajp and a2enmod proxy_http.
    • mod_wsgi. This is most likely in a separate package named like python3-mod_wsgi. To enable mod_wsgi on Debian based systems, use a2enmod wsgi.
  • Python 3.4 or above with virtualenv capabilities (probably in the python3-venv package or included with your Python 3 install)
  • nodejs 4.0 or above
  • GDAL 2.0 or above with Python3 bindings
  • Sphinx search 2.2 or above for the search features (package commonly named sphinx on most distributions, on Debian system, use sphinxsearch in Jessie backports)
  • A WMS/WMTS server. For instance MapServer. The version depends on your needs and the Map files you will write but we recommend the last version.
  • tomcat 8.0 or above to deploy the print component
  • Bash 4 or above to launch the tasks
  • git 2.0 or above to get the code
  • proj the cartographic projection software
  • A database system with GIS support. Currently, we only support PostgreSQL with PostGIS for production usages.

Requirements for a usage with a virtualenv

The following libraries to correctly create the python venv of the API:

  • geos
  • geos-devel
  • postgresql-devel
  • libxml2-devel
  • libxslt-devel

On debian based system, use this list:

  • libgeos-c1
  • libgeos-dev
  • libxml2-dev
  • libxslt-dev

Requirements without a virtualenv

To view the full list of Python 3 packages necessary for the API, take a look at this file.

Production configurations

Review the production configurations. The script containing the production values for deployment is located in customer-infra/config/config.dist.sh. You can view an example here.

You should also check that the configuration for the vhost are correct (domain, HTTPS certificates). This will be located in customer-infra/config/dist/_common.dist.toml. Be aware that these values can be overridden in portal specific files. The section looks like this:

[vhost]
api_proxy = 'http://localhost:9080'
# If this is empty, the configuration will not use HTTPS
certificate_file = ''
certificate_chain_file = ''
domain = 'geoportal.local'
certificate_key_file = ''
# Keep trailing slash!
ows_path = '~/geoportal-infras/cgi-bin/'
print_proxy = 'ajp://localhost:8009'
# On dev, infra_dir is used instead. No trailing slash!
prod_data_dir = '/var/lib/geoportal/data'
server_name = '{portal}.{domain}'

sudo

You will need sudo to launch some commands with the user used to deploy the portals. Your /etc/sudoers file must contains the lines below. Replace USER by the user defined by $PROD_USER. See the section about Production configurations of this document to learn more about $PROD_USER.

USER ALL=(ALL) NOPASSWD: /usr/bin/systemctl restart httpd.service  # Or /bin/systemctl restart apache2.service on Debian based system
USER ALL=(ALL) NOPASSWD: /usr/bin/systemctl reload httpd.service  # Or /bin/systemctl reload apache2.service on Debian based system
USER ALL=(ALL) NOPASSWD: /usr/bin/systemctl restart searchd@customer-infra.service # Or /bin/systemctl restart searchd@customer-infra.service on Debian based system
USER ALL=(ALL) NOPASSWD: /usr/bin/systemctl restart tomcat.service # Or /bin/systemctl restart tomcat.service on Debian based system
USER ALL=(ALL) NOPASSWD: /bin/systemctl restart tomcat8.service  # Debian based system only, in addition to the previous line.
USER ALL=(ALL) NOPASSWD: /usr/bin/indexer --verbose --rotate --config /etc/sphinx/customer-infra.conf --all --quiet
USER ALL=(ALL) NOPASSWD: /usr/bin/indexer --verbose --rotate --config /etc/sphinx/customer-infra.conf --all
USER ALL=(ALL) NOPASSWD: /usr/bin/indexer --verbose --rotate --config /etc/sphinx/sigeom-infra.conf --quiet *
USER ALL=(ALL) NOPASSWD: /usr/bin/indexer --verbose --rotate --config /etc/sphinx/sigeom-infra.conf *
USER ALL=(ALL) NOPASSWD: /usr/sbin/apachectl -t

Production scripts

In order to be sure that tomcat, apache, search can restart and that a reindex can be triggered, we invite you to create scripts available in the PATH of the user that will do the deployment. These scripts are:

  • sudo_tomcat_restart. It may contain:

    sudo /usr/bin/systemctl restart tomcat.service
    
  • sudo_apache_restart. It may contain:

    sudo /usr/bin/systemctl restart httpd.service
    
  • sudo_apache_reload. It may contain:

    sudo /usr/bin/systemctl reload httpd.service
    
  • sudo_search_restart. It may contain:

    sudo /usr/bin/systemctl restart searchd@customer-infra.service
    
  • sudo_search_reindex. It may contain:

    # If launched without arguments, rebuild all indexes, otherwise, rebuild indexes only for the
    # specified portals.
    if [[ -z "${1:-}" ]]; then
        sudo /usr/bin/indexer --verbose --rotate --config /etc/sphinx/sigeom-infra.conf --all
    else
        # Don't put quotes around $@ for the loop to work correctly.
        for portal in $@; do
            indexes=()
            pushd "/home/jenselme/prod/${portal}/search" > /dev/null
                for index in $(grep -E --only-matching --no-filename "^index (${portal}_[^{]+)$" *.conf |
                        cut -d ' ' -f 2 |
                        grep -v "${portal}_locations"); do
                    indexes+=("${index}")
                done
            popd > /dev/null
            echo "++++++ Reindexing ${portal} ++++++"
            sudo /usr/bin/indexer --verbose --rotate --config /etc/sphinx/sigeom-infra.conf "${indexes[@]}"
            echo "++++++ Done ++++++"
        done
    fi
    
  • sudo_tomcat_copyconf. It may contain:

    set -u
    set -e
    
    MFP_PRINT_APPS='/srv/tomcat/webapps/print-ioda-infra/print-apps'
    SOURCE_APP="/home/geop/ioda-infra/prod/$1/print"
    
    mkdir -p "${MFP_PRINT_APPS}/$1"
    /usr/bin/cp -av ${SOURCE_APP}/* "${MFP_PRINT_APPS}/$1/"
    

vhosts

Include all files in $PROD_GIT_REPOS_LOCATION/vhosts.d in your apache configuration. This can be done be editing /etc/httpd/conf/httpd.conf or /etc/apache2/apache2.conf depending on your system and appending this line at the end of the file: IncludeOptional $PROD_GIT_REPOS_LOCATION/vhosts.d/*.conf. See the section about Production configurations of this document to learn more about $PROD_GIT_REPOS_LOCATION.

Print

Printing a map relies on MapFish Print a Java servlet developed by Camptocamp SA.

You can either build it from scratch from the source or use our last build. Once you have the WAR, do the following actions as root:

  • Copy the WAR in your tomcat webapps folder (eg /usr/share/tomcat/webapps, /srv/tomcat/webapps/ or /var/lib/tomcat8/webapps) under the name print-customer-infra.war.

  • Start tomcat: systemctl start tomcat

    Attention

    On Debian based systems, the target is named tomcat8

  • Go to the tomcat webapps folder.

  • Check that print-customer-infra.war is correctly deployed.

  • Create the print-customer-infra/print-apps directory and make it owned by tomcat: mkdir print-customer-infra/print-apps && chown tomcat:tomcat print-customer-infra/print-apps.

    Attention

    On Debian based systems, the correct user is tomcat8.

  • Check and correct permissions on <tomcat-webapps>/print-customer-infra/print-apps:

    • Check that with the user defined by $PROD_USER you can access this directory. If ls <tomcat-webapps>/print-customer-infra/print-apps runed with $PROD_USER returns successfuly, you are good to go. If not, correct the permissions to give it read and execute access on all folders on the path.
    • Setup ACL to give the user write permissions to the directory (don’t use standard unix permissions, it breaks tomcat’s expectations): setfacl -m u:<user>:rwx print-customer-infra/print-apps.
  • Check that tomcat has an AJP connector defined on port 8009 in /etc/tomcat/server.xml. If not, add the line below in the <Service name="Catalina"> section:

    <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
    
  • Restart tomcat systemctl resart tomcat

  • Enable tomcat systemctl enable tomcat

API

  1. On the production server, either:

    • Clone the api for the first deploy: git clone https://github.com/ioda-net/geo-api3.git
    • Update the API: git pull

    Note

    Depending on who you are, you may:

    • get the code of the API from another location
    • need to switch to a custom branch.
  2. Update the configuration of the API for this deployment. To do this, create a file named geo-api3/config/config.<branchname>.toml and override any values necessary from the geo-api3/config/config.dist.toml config file.

  3. Override any commands necessary in geo-api3/config/config.dist.sh by creating a geo-api3/config/config.sh file.

  4. Deploy the API: on the production server, in the geo-api3 folder, launch: ./manuel deploy

  5. Add a new vhost for the API. It should look like the vhost below. Adapt the user names and file paths to match those defined in the geo-api3/config/config.<branchname>.toml.

    <VirtualHost *:80>
        ServerAdmin webmaster@geoportal.local
        ServerName api.geoportal.local
    
        DocumentRoot /srv/www/htdocs
        HostnameLookups Off
        UseCanonicalName Off
    
        # configures the footer on server-generated documents
        ServerSignature On
    
        <Directory "/srv/www/htdocs">
            Options Indexes FollowSymLinks Multiviews
            AllowOverride None
            Require all granted
            DirectoryIndex index.html
        </Directory>
    
        Redirect permanent / https://api.geoportal.local/
    
    </VirtualHost>
    
    
    <IfDefine SSL>
    <IfDefine !NOSSL>
    
    ##
    ## SSL Virtual Host Context
    ##
    
    <VirtualHost api.geoportal.local:443>
    
            #  General setup for the virtual host
            ServerName api.geoportal.local:443
            ServerAdmin webmaster@geoportal.local
    
            SSLEngine on
            SSLProtocol all -SSLv2 -SSLv3
            SSLCipherSuite HIGH:MEDIUM:!aNULL:!MD5
            SSLHonorCipherOrder on
    
            SSLCertificateFile /etc/ssl/ssl.crt/geoportal.local.crt
            SSLCertificateKeyFile /etc/ssl/ssl.key/geoportal.local.key
            SSLCertificateChainFile /etc/ssl/ssl.crt/positivessl-ca-bundle.crt
    
            #   Access Control:
            #   With SSLRequire you can do per-directory access control based
            #   on arbitrary complex boolean expressions containing server
            #   variable checks and other lookup directives.  The syntax is a
            #   mixture between C and Perl.  See the mod_ssl documentation
            #   for more details.
            <Location />
                SSLRequire (    %{SSL_CIPHER} !~ m/^(EXP|NULL)/  )
            </Location>
    
            ExpiresActive on
    
            AddOutputFilterByType DEFLATE text/css
            AddOutputFilterByType DEFLATE application/x-javascript application/javascript application/json application/xml
            AddOutputFilterByType DEFLATE text/html text/plain text/xml
    
            # uncomment this if you need HTTP authentication/authorization to work (with
            # repoze.who or any other security toolkit), see the Apache mod_wsgi FAQ to
            # understand why mod_wsgi doesn't pass the user credentials to the WSGI
            # application by default.
            # http://code.google.com/p/modwsgi/wiki/FrequentlyAskedQuestions#Access_Control_Mechanisms
            WSGIPassAuthorization On
    
            WSGIDaemonProcess api display-name=geo-api3 user=wwwrun threads=32 python-path=/home/geop/geo-api3/lib/python3.4/site-packages:/usr/lib64/python3.4/site-package::/usr/lib/python3.4/site-package
            # define the path to the WSGI app
            WSGIScriptAlias / /home/geop/geo-api3/parts/wsgi/production.wsgi
    
            # assign the WSGI app instance the process group defined above, we put the WSGI
            # app instance in the global application group so it is always executed within
            # the main interpreter
            <Directory /home/geop/geo-api3/parts/wsgi>
                    # Only the front vhost from localhost can access the api.
                    Require local
                    WSGIProcessGroup api
                    WSGIApplicationGroup geo-api3
            </Directory>
    
            # kml file downloads
            AliasMatch ^//kml/(map\.geo\.admin\.ch_KML_\d+\.kml)$ /tmp/$1
    </VirtualHost>
    
    </IfDefine>
    </IfDefine>