Data access

OpenBioMaps provides several ways to query, view, and download project data. The records and fields available through these interfaces depend on the project’s access rules and on the permissions of the current user.

This page provides an overview of the available data retrieval methods and the mechanisms used to control access to project data.

Retrieving data

Data can be retrieved through the web application, downloaded in supported file formats, or accessed from external applications.

File download

Query results can be downloaded in a variety of formats. Depending on the settings in the project module, the available formats may include:

  • text and structured data: CSV and JSON;

  • spreadsheets: ODS, XLS and XLSX;

  • spatial data with text attributes: ESRI Shapefile, KML, GPX and SQLite.

The ESRI Shapefile export may consist of several associated files, including .shp, .dbf, .cpg, .prj and .shx files.

Users can download images associated with data records individually, whilst project admins can download them in bulk via the administrator interface.

Administrators can export the text fields of data tables in CSV format via the database table management page.

Data export can also be subject to individual authorisation requests using the Export module.

Web queries

The web application provides tools for filtering and retrieving accessible records. Depending on the project configuration, users can perform:

  • attribute-based queries using text, lists, dates, and other configured fields;

  • spatial queries using geometries selected or drawn on the map; or

  • combined spatial and attribute-based queries.

For an overview of the query interfaces, see User interfaces.

External applications

Project data can also be accessed from external applications:

  • the OpenBioMaps API can be used by scripts, the OpenBioMaps R package, and other API clients;

  • an authorised SQL connection can provide direct database access for applications such as QGIS; and

  • supported client applications can provide their own query and download interfaces.

Access through an external application is subject to the project’s access rules and the permissions associated with the authenticated user or connection.

For more information, see:

Controlling access to data

OpenBioMaps can control access at several levels:

  • project-level settings define the default access and modification policy;

  • row-level rules control access to individual records; and

  • column-level rules control which fields can be viewed or downloaded.

The effective permissions may therefore depend on several settings. Project administrators should test the resulting access with users belonging to different groups, as well as without authentication where public access is enabled.

Project-level access

The default project-level access settings are defined in the local_vars.php.inc configuration file:

define('ACC_LEVEL', 'group'); // Can be set to 'public' or 'login'.
define('MOD_LEVEL', 'group');

ACC_LEVEL defines the default level at which project data can be accessed. The documented values are:

public

Data are publicly accessible, subject to any more specific access rules.

login

Data are accessible to authenticated users, subject to any more specific access rules.

group

Access is controlled through project groups and additional access rules.

MOD_LEVEL defines the default level at which data can be modified. It uses a similar access model.

Setting MOD_LEVEL to public allows data to be modified without requiring the user to sign in. This setting should only be used when unauthenticated modification is explicitly intended and its security implications have been considered.

Row-level access

When ACC_LEVEL or MOD_LEVEL is set to group, access to individual records can be controlled through a project-specific *_rules table. Here, * represents the name or prefix used by the project.

A rules table is associated with a data table. A data record is linked to its corresponding rule through the obm_id value in the data table and the row_id value in the rules table.

In a project using group-level access, records without a corresponding entry in the rules table are available only to project hosts.

The rules-table functionality can be configured in the project administration interface under Project administration > Functions > Create access rules. This interface can be used to create or update the trigger function and to enable or disable it.

When enabled, the trigger maintains the rules table after records are created, modified, or deleted.

Assigning read and write groups

Read and write access can be assigned to individual records through the group-related fields of the rules table.

These values can be populated automatically by the rules-table trigger. The assigned groups may be derived from the access settings of the upload form used to create the record. Information about completed uploads and their configured owner and group values is stored in the system.uploadings table.

Regenerating a rules table

A rules table can also be regenerated manually. The following examples use abc as the data table and abc_rules as its rules table.

The following statements recreate rules without assigning read or write groups:

DELETE FROM abc_rules
WHERE data_table = 'abc';

INSERT INTO abc_rules (row_id, sensitivity, data_table)
SELECT obm_id, 'sensitive', 'abc'
FROM abc;

The following statements derive the group and owner values from the corresponding entry in system.uploadings:

DELETE FROM abc_rules
WHERE data_table = 'abc';

INSERT INTO abc_rules (row_id, sensitivity, data_table, read, write)
SELECT a.obm_id, 'sensitive', 'abc', s."group", s.owner
FROM abc AS a
LEFT JOIN system.uploadings AS s
    ON s.id = a.obm_uploading_id;

These examples must be adapted to the actual table names, schema, column types, and access policy of the project. Administrators should back up the existing rules table and verify the generated permissions before using the statements in a production database.

Sensitivity settings

The sensitivity field in the rules table affects the public availability of a record in a project using group-level access.

The documented values include:

sensitive

The record can be read or modified only by members of the groups specified by the applicable access rules.

restricted

This value currently has the same documented meaning as sensitive.

no-geom

The record may be accessible at the public level, but its geometry is not displayed publicly.

only-owner

Only the project owner can access the record.

Column-level access

Access can be further controlled for individual database fields by using the allowed_columns module. This module determines which columns can be viewed or downloaded by public users or specified user groups.

In a project where ACC_LEVEL is set to group, the module can be used to make selected fields accessible even when the project does not otherwise provide general access to every field. It can also restrict the visible fields of records that are accessible through row-level rules.

This makes it possible, for example, to allow users to discover that a record exists while exposing only an approved subset of its fields.

How access rules interact

If only group-level project access is configured and no more specific rules grant access, project data are available only to the administrative role that is allowed to bypass those restrictions.

A rules table adds row-level control, allowing different records to be made available to different groups. The allowed_columns module adds column-level control, allowing only selected fields of an accessible record to be viewed or downloaded.

Where several rules apply, the effective permissions are determined by their combined project-, row-, and column-level restrictions. Administrators should not assume that a broader rule automatically overrides a more specific restriction.