User Guide

No token available. Code chunks will not be evaluated.

Introduction

The dtmapi package provides functions to interact with the Displacement Tracking Matrix (DTM) API. This vignette demonstrates how to use the package’s functions to fetch data from the API.

Please note that despite the overarching term DTM API, there are, in fact, two distinct APIs which are currently available. These are the Baseline API (which is what simply “the DTM API” USED to refer to) and the Humanitarian Needs Assessment (HNA) API.

Reflecting this, this R package comes with a set of functions that are clearly named so as to distinguish between these two APIs. Those prefixed with baseline_ and those prefixed with hna_.

The functions covered include:

baseline_get_countries()
baseline_get_operations()
baseline_get_admin0()
baseline_get_admin1()
baseline_get_admin2()
hna_get_countries()
hna_get_catalog()
hna_get_admin2()

Notice that as a consequence of there being two distinct APIs, there are also two distinct subscription key-related function pairs. This will become clear in their respective technical documentations.

Install Package

The latest version of the dtmapi package can be installed directly from GitHub using the following command (make sure the remotes package is installed):

remotes::install_github("Displacement-Tracking-Matrix/dtmapi-R")

Load Package

After installation, load the package using library():

library(dtmapi)

Setting the Subscription Keys

After creating a subscription key on https://dtm-apim-portal.iom.int, that key needs to be given to R. The package has two sets of functions, each with its own subscription key: the baseline_ functions use the BASELINE_DTM_SUBSCRIPTION_KEY environment variable, and the hna_ functions use the HNA_DTM_SUBSCRIPTION_KEY environment variable. There are three main ways to set these, as described below. The examples use the Baseline key; the HNA key works the same way, with hna_set_subscription_key() and HNA_DTM_SUBSCRIPTION_KEY in place of baseline_set_subscription_key() and BASELINE_DTM_SUBSCRIPTION_KEY.

1. Using `baseline_set_subscription_key()`: interactive, safe option

By calling baseline_set_subscription_key(), after which the user is prompted to input the key into a pop-up field.

baseline_set_subscription_key()

Note that if the environment is not interactive, it is likely for the user to receive an error message instead, such as in the case of using Google Colab. In such cases see the other options described further below.

2. Using `baseline_set_subscription_key()`: the non-interactive, discouraged option

If the interactive option above fails for the user, such that there is an error instead of a pop-up, the user may rely on passing their key directly into baseline_set_subscription_key() by naming the key parameter.

# Specifying `key` is required for non-interactive use.
baseline_set_subscription_key(key = "mysubscriptionkey")

This is basically using baseline_set_subscription_key() as a wrapper for setting up the environment variable BASELINE_DTM_SUBSCRIPTION_KEY using base R’s Sys.setenv(), as shown below:

Sys.setenv("BASELINE_DTM_SUBSCRIPTION_KEY" = "mysubscriptionkey")

Please note that by doing this in, say, a Google Colab script, .R script, or Jupyter notebook, you are leaving a secret (your subscription key) exposed and available for anyone to read with ease. This is why this option is not recommended, but it will allow you to get things working faster if you are not familiar with managing secrets in a non-interactive context.

3. Setting up the environment variable with good secrets management

If one would like to avoid re-inputting the subscription key for each R session, and they would like to set the subscription key in a way that is both non-interactive and safer, then this becomes a matter of knowing the right secrets management practices depending on where R is being used (e.g. whether that is in a local desktop, a cloud-based programming environment, a server, etc.).

In the vast majority of cases, the approach is the fundamentally the same: anything that directly stores your key should be separate from the rest of your code, and your code accesses the key through an environment variable. That is the main principle to the most common way to manage your secrets. What differs by where R is being used is how this principle is applied. Below, the user guide will mostly focus on how this is handled when using a code editor such as VSCode, RStudio, or Positron, as this is the most common case. If these do not reflect your circumstance or use-case, then researching how to manage secrets generally for that circumstance should suffice, as the only thing specific about doing so for the dtmapi package is just the names of the environment variables, BASELINE_DTM_SUBSCRIPTION_KEY and HNA_DTM_SUBSCRIPTION_KEY.

Using a code editor such as VSCode or RStudio on a desktop:

Use a .Renviron file to set the keys. You can create it from scratch, and its contents should look something like this:

BASELINE_DTM_SUBSCRIPTION_KEY = "mysubscriptionkey"
HNA_DTM_SUBSCRIPTION_KEY = "myothersubscriptionkey"
Some_Other_Env_Var = "something else if desired"

For this to work robustly, your .Renviron should be in the same working directory that you would like to use for your R sessions (i.e. it should be in the same location or folder that you would like your R sessions to use as a default). You can see which folder that currently is by running:

getwd()

This is important to know because, if your R session can start in the same working directory that the .Renviron file is located in, then the environment variables specified there are automatically loaded for your R session!

Usually, your choice of working directory is best managed by creating an R project or equivalent (such as VS Code workspaces). That is to say, one should have a dedicated folder for their specific project, the R sessions they run should make that folder the working directory, and .Renviron should be placed there as well. For more information, depending on whether you use RStudio, VSCode, or Positron, see: Using RStudio Projects, What is a VS Code Workspace?, or Launching Positron in a workspace.

If one is unfamiliar and struggling to create the .Renviron file, then one quick way to proceed with editing the .Renviron file or placing it in their project directory (if they have one) is by calling the following line of code (make sure the usethis package is installed):

usethis::edit_r_environ("project")

Or, if one is not using an R project or equivalent, then this is an option as well, though using the general user directory is not recommended as it is likely to get lost:

usethis::edit_r_environ("user")

Remember: NEVER SHARE THE .RENVIRON FILE.

Get All Countries

The baseline_get_countries() function retrieves a list of all countries from the DTM API.

# Fetch all countries
countries_df <- baseline_get_countries()

# Display the first few rows of the data frame
head(countries_df)

Get All Operations

The baseline_get_operations() function retrieves a list of all operations from the DTM API.

# Fetch all operations
operations_df <- baseline_get_operations()

# Display the first few rows of the data frame
head(operations_df)

Get IDP Data at Admin Level 0

The baseline_get_admin0() function retrieves Internally Displaced Persons (IDP) data aggregated at the country level.

# Fetch IDP data at Admin Level 0
idp_admin0_df <-
  baseline_get_admin0(admin0_name = "Ethiopia",
                      from_round_number = 0,
                      to_round_number = 10)

# Display the first few rows of the data frame
head(idp_admin0_df)

Get IDP Data at Admin Level 1

The baseline_get_admin1() function retrieves IDP data aggregated at Admin Level 1.

# Fetch IDP data at Admin Level 1
idp_admin1_df <- baseline_get_admin1(
  admin0_name = "Sudan",
  from_reporting_date = "2020-01-01",
  to_reporting_date = "2024-08-15"
)

# Display the first few rows of the data frame
head(idp_admin1_df)

Get IDP Data at Admin Level 2

The baseline_get_admin2() function retrieves IDP data aggregated at Admin Level 2.

# Fetch IDP data at Admin Level 2
idp_admin2_df <-
  baseline_get_admin2(operation = "Displacement due to conflict",
                      admin0_name = "Lebanon")

# Display the first few rows of the data frame
head(idp_admin2_df)

Get All Countries (HNA)

The hna_get_countries() function retrieves a list of all countries covered by the DTM Humanitarian Needs Assessment (HNA) API.

# Fetch all HNA countries
hna_countries_df <- hna_get_countries()

# Display the first few rows of the data frame
head(hna_countries_df)

Get the HNA Data Catalog

The hna_get_catalog() function retrieves the HNA data catalog, which provides information on the indicators available in the HNA API, including their name, description, and data type.

# Fetch the HNA data catalog
hna_catalog_df <- hna_get_catalog()

# Display the first few rows of the data frame
head(hna_catalog_df)

Get HNA Data at Admin Level 2

The hna_get_admin2() function retrieves HNA data at Admin Level 2. Unlike the other functions, it returns a list of two elements: data, a data frame holding the requested data, and pagination, a list containing metadata on pagination.

# Fetch HNA data at Admin Level 2
hna_admin2 <-
  hna_get_admin2(admin0_name = "Nigeria",
                 year = 2025)

# Display the first few rows of the data
head(hna_admin2$data)

# Display the pagination metadata
print(hna_admin2$pagination)

Function Arguments

Here are the descriptions for the arguments used in the functions of the dtmapi package to get data.

Baseline Arguments

For the baseline_ functions that get IDP data, at least one of the following parameters must be provided: operation, admin0_name, or admin0_pcode.

operation: Optional; Name of the DTM operation for which the data was collected.
admin0_name: Optional; Name of the country where the data was collected.
admin0_pcode: Optional; Country code (ISO 3166-1 alpha-3).
admin1_name: Optional; Name of level 1 administrative boundaries.
admin1_pcode: Optional; Place code of level 1 administrative boundaries.
admin2_name: Optional; Name of level 2 administrative boundaries.
admin2_pcode: Optional; Place code of level 2 administrative boundaries.
from_reporting_date: Optional; Start date for the reporting period (format: ‘YYYY-MM-DD’).
to_reporting_date: Optional; End date for the reporting period (format: ‘YYYY-MM-DD’).
from_round_number: Optional; Starting round number for the data collection range.
to_round_number: Optional; Ending round number for the data collection range.

HNA Arguments

For hna_get_admin2(), at least one of the following parameters must be provided: admin0_name or admin0_pcode.

admin0_name: Optional; Name of the country where the data was collected.
admin0_pcode: Optional; Country code (ISO 3166-1 alpha-3).
population_group: Optional; A specific subpopulation (e.g. internally displaced persons).
year: Optional; Year of data collection.
page: Optional; Pagination, to fetch a certain chunk of the data.