Data Sources

Data sources supply the rows a page group turns into pages. A group can have several; add them on its Data sources page with Add data source, pick a Type, fill in the fields that type shows, and save. Rank reads the sources in order: when two sources hold the same slug, the first one wins.

Every source also has a Cache duration (seconds); see Loading and refreshing the data for what it does and why a new or changed source needs a reload before any page resolves.

The data a source must yield

Every source resolves to a list of rows, each an object of named fields. A row’s slug field is the part of the URL that replaces {slug} in the pattern, so every row that should become a page needs one:

[
  {
    "slug": "new-york",
    "name": "New York",
    "state": "NY",
    "population": "8.3M"
  },
  {
    "slug": "los-angeles",
    "name": "Los Angeles",
    "state": "CA",
    "population": "3.9M"
  }
]

Slugs should be unique: within one source only the first row with a given slug is kept. The other fields are what the mappings write into the page. Notion rows without a slug property take the slug from their title. Any other row without a slug stays in the data but becomes no page and no sitemap entry.

Source types

JSON URL, CSV URL and REST API fetch only addresses WordPress accepts as safe: an http or https address on port 80, 443 or 8080, or on the site’s own port for the site’s own host. Another host that resolves into a private or local network fails the reload unless the http_request_host_is_external filter allows it.

JSON URL

Fetches a JSON array from a URL. Use it for any service that publishes its data as JSON.

• URL: the address of the JSON document

JSON file

Reads a JSON array from a file inside the active theme’s directory, so the data can live in version control beside the theme. Large files are read one row at a time, and a reload of an unchanged file only renews the cached rows’ freshness.

• File path: the path relative to the theme directory, such as data/cities.json. A path outside the theme is refused.

CSV URL

Fetches a CSV file from a URL and turns each line into a row.

• URL: the address of the CSV file
• Delimiter: the column separator, a comma by default
• Use the first row as headings: when on, the first line names the fields; when off, the fields are numbered

CSV file

Reads a CSV file inside the active theme’s directory; otherwise it works like CSV URL.

• File path: the path relative to the theme directory, such as data/products.csv
• Delimiter and Use the first row as headings, as above

REST API

Requests an API with a chosen method and authentication, and finds the list inside the response.

• URL: the endpoint
• Method: GET or POST
• Authentication: None, Bearer token (sent as Authorization: Bearer <token>) or API key (sent in a header you name)
• Authentication token: the token or key, required unless authentication is None. A saved token is never shown or returned again; leave the field blank to keep it, or type a new one to replace it. Agent Sync leaves tokens and request headers out of the page group files it writes to the theme and keeps the saved ones when it reads a file back.
• API key header: the header that carries an API key, X-API-Key by default
• JSON path: an optional dot path to the list inside the response, such as data.items. A path that does not exist fails the reload.

Google Sheets

Reads one tab of a Google Sheet; each row becomes a row of data, named by the header row.

• Spreadsheet ID: the sheet’s ID, the long string between /d/ and /edit in its address. The full address works too.
• Sheet name: the tab to read, Sheet1 by default
• Use the Google Sheets API: off, Rank downloads the tab through Google’s public CSV export; on, it reads the tab through the Google Sheets API with the key from Settings

Both ways reach only a sheet Google serves without signing in, so share it as Anyone with the link (viewer). An API key identifies a Google Cloud project and grants no access to private files, and Rank has no Google sign-in, so a private sheet cannot be read either way.

Notion

Queries a Notion database; each entry becomes a row, named by its properties.

• Database ID: the database’s ID, or its full address

The Notion integration token from Settings must be stored first, and the integration must be shared with the database. Rank turns these property types into plain values; other types are left out:

• title and rich_text: plain text
• number: the number
• select: the option’s name
• multi_select: the option names, comma-separated
• url and email: the string
• checkbox: true or false
• date: the start date as an ISO string

Loading and refreshing the data

Rank never reads a data source while it serves a page. It stores each source’s rows in its own indexed table, {prefix}sleek_rank_items, and the generated pages, the sitemap, the preview and the Data view all read that store. A source whose rows have not been stored yet resolves no pages and adds nothing to the sitemap.

Nothing fills the store when you create a group or change a source. After saving, load the data with one of these:

• Reload data on the group’s Data sources page or its Data view
• wp sleek rank reload <id>, or wp sleek rank reload --all for every published group
• wp sleek rank cache sync --group=<id>, or wp sleek rank cache sync for every published group

A reload fetches every source of the group in order and replaces each source’s rows in one step, so a failed fetch leaves the previous rows in place. The first source that fails stops the reload with its error; the sources before it are already reloaded. The result shows as a notice, and under Reload data Rank says how many rows it skipped because an earlier row has the same slug.

The stored rows belong to the source’s exact configuration, its cache duration included. Changing any field of a source points the group at a new, empty cache, so reload after every change. Rows that no page group refers to any more are removed by Rank’s twice-daily cleanup, and a minute after a group is saved or deleted.

Cache duration (seconds), 3600 by default, sets how long the stored rows count as fresh. Once they are older, pages keep being served from them while Rank schedules one WP-Cron refresh of that source, between 30 seconds and five and a half minutes later. The scheduled event names the source by its cache key only; the refresh reads the source’s settings, its token included, from the page group when it runs. Without a working WP-Cron, stale rows are refreshed only by a reload or wp sleek rank cache sync. A duration of 0 keeps the rows until you reload.

The Data view lists the stored rows of the group fifty at a time, with every field of the page’s rows as a column; its search matches slugs and field values across every row, and its slug links open the row’s preview. The page and the search are part of the address, so a link opens the same rows. When the rows cannot be read, the table’s card says so with Retry. wp sleek rank items <id> prints the same rows, and wp sleek rank cache status reports the active caches and the expired rows waiting for cleanup (see CLI).

Configuring sources from WP-CLI or REST

wp sleek rank page-group create and update take the sources as a JSON array through --data-sources, and the PageGroups_create and PageGroups_update operations take the same array as dataSources. Each entry has a type, a config object and a cacheDuration:

The URL-based types and Google Sheets and Notion also accept timeout, the seconds a request may take before it fails, 30 by default. The workspace does not show headers or timeout.

A plugin can add its own source type through the sleekRank/register/dataSources action (see Hooks). The workspace editor offers only the seven built-in types, so configure a custom type through WP-CLI or REST.