Getting Started

Estimated time to complete: 3/4 hours.

Requirements

  • Apache 2 with:

    • mod_rewrite. To enable mod_rewrite on Debian based systems, use a2enmod rewrite
    • mod_expires. To enable mod_expires on Debian based systems, use a2enmod expires
    • mod_headers. To enable mod_headers on Debian based systems, use a2enmod headers
    • mod_filter. To enable mod_filter on Debian based systems, use a2enmode filter
    • mod_deflate. To enable mod_deflate on Debian based systems, use a2enmod deflate
    • mod_fcgid. To enable mod_fcgid on Debian based systems install the libapache2-mod-fcgid package (aptitude install libapache2-mod-fcgid) and on openSuse, install apache2-mod_fcgid.
    • mod_proxy, mod_proxy_ajp and mod_proxy_http. To enable mod_proxy, mod_proxy_ajp and mod_proxy_http on Debian based systems, use a2enmod proxy, a2enmod proxy_ajp and a2enmod proxy_http.
  • Python 3.4 or above with virtualenv capabilities (probably in the python3-venv package or included with your Python 3 install)

  • nodejs 4.0 or above

  • MapServer 6.4.3+ (it will not work with 6.4.1) or 7.0.1+ (package commonly named mapserver on most distributions, on Debian based system, use cgi-mapserver and mapserver-bin)

  • GDAL 2.0 or above with Python3 bindings

  • Sphinx search 2.2 or above for the search features (package commonly named sphinx on most distributions, on Debian system, use sphinxsearch in Jessie backports)

  • tomcat 8.0 or above to deploy the print component

  • Bash 4 or above to launch the tasks

  • git 2.0 or above to get the code

  • proj the cartographic projection software

  • sudo to launch some commands with your normal user. Your /etc/sudoers file must contains the following lines (edit it with visudo):

    USER ALL=(ALL) NOPASSWD: /usr/bin/systemctl restart httpd.service  # Or /bin/systemctl restart apache2.service on Debian based system
    USER ALL=(ALL) NOPASSWD: /usr/bin/systemctl restart searchd@customer-infra.service # Or /bin/systemctl restart searchd@customer-infra.service on Debian based system
    USER ALL=(ALL) NOPASSWD: /usr/bin/systemctl restart tomcat.service # Or /bin/systemctl restart tomcat.service on Debian based system
    USER ALL=(ALL) NOPASSWD: /bin/systemctl restart tomcat8.service  # Debian based system only, in addition to the previous line.
    USER ALL=(ALL) NOPASSWD: /usr/bin/indexer --verbose --rotate --config /etc/sphinx/customer-infra.conf --all --quiet
    USER ALL=(ALL) NOPASSWD: /usr/bin/indexer --verbose --rotate --config /etc/sphinx/customer-infra.conf --all
    USER ALL=(ALL) NOPASSWD: /usr/sbin/apachectl -t
    
  • The following libraries to correctly create the Python venv:

    • geos
    • geos-devel
    • postgresql-devel
    • libxml2-devel
    • libxslt-devel
    • python3-devel
    • gcc

    On debian based system, use this list:

    • libgeos-c1
    • libgeos-dev
    • libxml2-dev
    • libxslt-dev
    • python3-dev
    • gcc

Before we start

All tasks are launched here with manuel, a task runner written in Bash. To enable autocompletion in a Bash shell, source the geo-infra/manuel.autocomplete.bash file. Completion is also available for zsh.

To launch manuel without always appending ./ copy geo-infra/manuel to your ~/bin folder.

To get help about any task (description of what it does and its arguments), use manuel help TASK. For instance, manuel help help.

Setup the portal

  • Clone all the repositories listed below in the same folder [1]:

    • The main infrastructure directory: git clone https://github.com/ioda-net/geo-infra.git
    • The API: git clone https://github.com/ioda-net/geo-api3.git
    • The frontend: git clone https://github.com/ioda-net/geo-front3.git
    • The sample customer infra: git clone https://github.com/ioda-net/customer-infra.git
  • Switch to customer-infra:

    • Download the ShapeFiles for Swisstzerland from geofabrik and uncompress it in customer-infra/data:

      • wget http://download.geofabrik.de/europe/switzerland-latest.shp.zip
      • unzip -d data/osm-switzerland switzerland-latest.shp.zip
      • You should now have a customer-infra/data/osm-swisstzeland folder containing various ShapeFiles.
    • Add the symlink to the fonts:

      ln -s /usr/share/fonts/liberation/LiberationSans-Regular.ttf data/LiberationSans-Regular.ttf
      

      or on Debian system:

      ln -s /usr/share/fonts/truetype/liberation/LiberationSans-Regular.ttf data/LiberationSans-Regular.ttf
      
    • Update the configuration: in order to correctly test customer-infra for development, you only need to create customer-infra/config/dev/_common.dev.toml with the content below.

    [vhost]
    ows_path = '/path/to/folder/containing/symlinks/to/mapserver/executable/'
    

    In this folder, create a symlink to your mapserver executable named after the portal. For instance, using the values of config/dist/_common.dist.toml, we have ln -s /usr/bin/mapserv ~/geoportal-infras/cgi-bin/demo.

  • Switch to geo-front3:

    • Launch npm install to install the node modules.
  • Switch to geo-infra:

    • Install the python dependencies listed in requires.txt. You can install them globally with sudo pip install -r requires.txt or in a venv. If you use a version of Python below 3.5, you’ll also need glob2. You can install it this way: sudo pip install glob2. To install the dependencies in a venv, follow the steps below:
      • Create it: python3 -m venv .venv
      • Activate it: source .venv/bin/activate This must be done once before launching any command with manuel.
      • Install the deps: pip install -r requires.txt
      • Install glob2 if needed: pip install glob2
    • Include all files in customer-infra/dev/vhosts.d in your apache configuration. This can be done be editing /etc/httpd/conf/httpd.conf or /etc/apache2/apache2.conf depending on your system and appending this line at the end of the file: IncludeOptional /path/to/infra/dir/customer-infra/dev/vhosts.d/*.conf.
    • Create the vhost: ./manuel vhost demo.
    • Create a symlink named mapserv to your MapServer executable in cgi-bin. Eg: ln -s /usr/bin/mapserv cgi-bin/mapserv.
    • Generate the utility files: ./manuel dev demo.
    • Generate the frontend: ./manuel front dev demo.
    • Add demo.geoportal.local to your /etc/hosts.
    • Open http://demo.geoportal.local.
    • You should see a portal. It should be similar to the image below. Please note that only map navigation is functionnal at this stage. The features that rely on the API (searches, QR code, shortener, get features) and print, will be enabled in the sections below.
    Demo portal home page

    Demo portal home page

[1]They can be arranged differently later if you override the proper values in geo-infra/config/config.sh

API

Switch to geo-api3:

  • Update the configuration. To do that, create a file named config/config.devel.toml and adapt the content below to your use case:

    default_epsg = 2056
    
    [db]
    type = 'sqlite'
    file_path = 'customer_infra.sqlite'
    staging = ''
    
    [raster]
    # Optionnal. If you have bt for altitude, put their path here. If you don't, height and profile related features won't work.
    # It must contain a bt folder with the bt.
    # dtm_base_path = '/var/lib/geoportal/data/'
    
    [search]
    port = 9314
    [search.origins_to_ranks]
    places = 10
    
    [shortener]
    allowed_domains = ['geoportal.local']
    allowed_hosts = ['localhost']
    
    [storage]
    kml = '/tmp'
    
    [waitress]
    # This must be coherent with vhost.api_proxy from customer-infra. 9080 is the default value from _common.dist.toml
    port = 9080
    
    • Download the sample sqlite database file into the geo-api3 folder:
      • wget https://docs.geoportal.xyz/data/getting-started/customer_infra.sqlite
    • Create the proper venv with ./manuel venv
    • Update the ini files used by Pyramid: ./manuel ini-files.
    • Launch the API: ./manuel serve.

    Attention

    If the command fails due to ImportError: No module named 'osgeo', check that the osgeo module from system install is available in the PYTHONPATH specificied in config/config.dist.sh. If not, create a config/config.sh with the correct value for PYTHONPATH. Eg for Debian, put this value:

    export PYTHONPATH=".venv/lib/python${PYTHON_VERSION}/site-packages:/usr/lib/python3/dist-packages:$(pwd)"
    
    • If you go to the portal, the QR codes, short link and features identification should work as expected. Search will be enable in the next section.
    Portal page with QR code and short link

    Portal page with QR code and short link

Attention

If you test the get features capability, you will only get a very basic presentation: it is what MapServer returns to the GetFeatures request. You improve this if you with access to the database. See the relevant section of the documentation.

Print

Switch to geo-infra:

  • Download the print WAR:

    • wget https://docs.geoportal.xyz/data/getting-started/print.war
  • Do the following actions as root:

    • Copy the WAR in your tomcat webapps folder (eg /usr/share/tomcat/webapps, /srv/tomcat/webapps/ or /var/lib/tomcat8/webapps) under the name print-customer-infra.war.

    • Start tomcat: systemctl start tomcat

      Attention

      On Debian based systems, the target is named tomcat8

    • Go to the tomcat webapps folder.

    • Check that print-customer-infra.war is correctly deployed.

    • Create the print-customer-infra/print-apps directory and make it owned by tomcat:

      • mkdir print-customer-infra/print-apps
      • chown tomcat:tomcat print-customer-infra/print-apps

      Attention

      On Debian based systems, the correct user is tomcat8.

    • Check and correct permissions on <tomcat-webapps>/print-customer-infra/print-apps:

      • Check that with the user you are using to run ./manuel in geo-infra you can access this directory. If ls <tomcat-webapps>/print-customer-infra/print-apps returns successfuly, you are good to go. If not, correct the permissions to give it read and execute access on all folders on the path.
      • Setup ACL to give the user write permissions to the directory (don’t use standard unix permissions, it breaks tomcat’s expectations): setfacl -m u:<user>:rwx print-customer-infra/print-apps.
    • Check that tomcat has an AJP connector defined on port 8009 in /etc/tomcat/server.xml. If not, add the line below in the <Service name="Catalina"> section:

      <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
      
    • Restart tomcat systemctl resart tomcat

  • Go to geo-infra

  • Export your value of MPF_APP_FOLDER to allow manuel to copy it to the correct location. For instance: export MFP_APP_FOLDER="/usr/share/tomcat/webapps/print-customer-infra/print-apps/"

  • Copy print configuration: ./manuel tomcat-copy-conf "dev" demo

  • Check that you have a demo folder in <tomcat-webapps>/print-customer-infra/print-apps.

  • If so, try to print in the portal. It should work as expected.

  • Unset the MFP_APP_FOLDER variable: unset MFP_APP_FOLDER

Switch to customer-infra:

  • In config/config.sh add a line like this: set-var MFP_APP_FOLDER "/usr/share/tomcat/webapps/print-customer-infra/print-apps/" that corresponds to the correct path to MapFish Print. It will now be used automatically by the scripts.

Conclusion

Everything should work correctly now. You can now rebuild the configuration of the portal with ./manuel dev demo or rebuild everything with ./manuel dev-full demo.