API

The API is written in Python 3 with the Pyramid web framework.

Setup

Requirements

  • The following libraries are needed to correctly create the python venv:

    • geos
    • geos-devel
    • postgresql-devel
    • libxml2-devel
    • libxslt-devel
    • python3-devel
    • gcc

    On debian based system, use this list:

    • libgeos-c1
    • libgeos-dev
    • libxml2-dev
    • libxslt-dev
    • python3-dev
    • gcc
  • GDAL 2.0 or above with Python3 bindings

  • Python 3.4 or above with virtualenv capabilities (probably in the python3-venv package or included in your Python 3 installation)

  • If you don’t want to use a venv as suggested below to install the Python dependencies, you will find the list of packages to install on your system with their minimal version in the requirements.txt.

Configuration

The configuration is written in the TOML format. It is loaded like this:

  1. config/config.dist.toml
  2. config/config.<branchname>.toml optional

You can customize the configuration for the branch you are on in config/config.<branchname>.toml. The keys used in it, will override any values loaded from config/config.dist.toml. Without this configuration file, configuration will only come from config/config.dist.toml.

Create the venv

  • Create the proper venv with ./manuel venv
  • Update the ini files used by Pyramid: ./manuel ini-files. You can check that the values in production.ini (contains database related configuration) and development.ini (imports the production.ini configuration file and contains development specific values) are correct.

Serve

To launch pserve with the development configuration: manuel serve

Attention

If the command fails due to ImportError: No module named 'osgeo', check that the osgeo module from system install is available in the PYTHONPATH specificied in config/config.dist.sh. If not, create a config/config.sh with the correct value for PYTHONPATH. Eg for Debian, put this value:

export PYTHONPATH=".venv/lib/python${PYTHON_VERSION}/site-packages:/usr/lib/python3/dist-packages:$(pwd)"

Deploy

Use manuel, on the production server: manuel deploy.

Search keywords

In order to add a keyword, you must edit geo-api3/chsdi/customers/utils/search.py. If you edit the file on devel, then the keywords will be used for all customer (on next merge). If you edit it in a customer specific branch, then it will only be available for this customer.

To add a keyword, you must add a SearchKeywords namedtuple to the SEARCH_KEYWORDS tuple, like this:

SEARCH_KEYWORDS = (
    SearchKeywords(
        keywords=('addresse', 'adresse', 'indirizzo', 'address'),
        filter_keys=['places']
    ),
)

Note

The SearchKeywords namedtuple has two members:

  • keywords: the list of keywords that the user can use.
  • filter_keys: the list of index names associated with these keywords.

Tests

  • To launch all the tests, use: manuel test
  • To launch only some tests, pass the proper arguments to manuel test. You can pass it as many files and options recognized by nose as you want. For instance:
manuel test chsdi/tests/integration/test_file_storage.py

Lint

Use manuel lint.

Launch with uWSGI and Unix sockets

Note

this is still a work in progress. Use the standard WSGI and proxy pass on production.

Apache Configuration

Replace:

ProxyPass /api http://localhost:9090 connectiontimeout=5 timeout=180
ProxyPassReverse /api http://localhost:9090

By

<Location /api>
    Options FollowSymLinks Indexes
    SetHandler uwsgi-handler
    uWSGISocket /run/uwsgi/geo-api3.sock
</Location>

uWSGI Configuration

In your /etc/uwsgi.ini:

[uwsgi]
pidfile = /run/uwsgi/uwsgi.pid
emperor = /etc/uwsgi.d
stats = /run/uwsgi/stats.sock
emperor-tyrant = true
plugins = python3

Adapt your config.<branchname>.toml to get something like this in uwsgi.ini (generated with manuel ini-files):

[uwsgi]
chmod-socket = 666
chown-socket = uwsgi:uwsgi
chdir = /home/jenselme/Work/geo-api3
home = /home/jenselme/Work/geo-api3/.venv
gid = uwsgi
uid = uwsgi
ini-paste = /home/jenselme/Work/geo-api3/production.ini
master = 1
plugins = python3
processes = 4
pythonpath = .venv/lib/python3.5/site-packages
pythonpath = /usr/lib64/python3.5/site-packages
pythonpath = /home/jenselme/Work/geo-api3
socket = /run/uwsgi/geo-api3.sock

Note

Your production.ini and uwsgi.ini must be owned by the user uwsgi and by the group uwsgi.