📍 Dataset Distributions, APIs & Hypermedia Links

📍 Dataset Distributions, APIs & Hypermedia Links#

This recipe explains how to describe dataset download locations, alternative formats, interactive APIs, documentation, summary statistics, and frequencies using standard JSON Hyper-Schema and RFC 8288 Web Linking (links) enriched with minimal fair: discovery annotations.

1. The Challenge: Bridging Data Locations to FAIR & CDIF#

In FAIR data discovery and CDIF (Cross-Domain Interoperability Framework), finding a dataset’s metadata is only half the journey. Automated AI agents, data pipelines, and researchers must also locate and access the physical data files or query services:

Multiple Formats: Providing the same data in both Parquet (for analytical queries) and CSV.gz (for broad accessibility).
API Endpoints vs. File Downloads: Differentiating between streaming static file downloads and calling parameterized REST/OpenAPI services.
Documentation & Materials: Linking variable codebooks, questionnaires, data dictionaries, and methodology reports.
Pre-computed Statistics: Linking descriptive summary metrics or categorical frequency tables (as standalone data files or FAIR schemas).
Safety Guardrails: Knowing the file size (fair:byteSize) and verifying cryptographic bit-level integrity (fair:checksum) before and after download.

Rather than inventing a new custom distribution vocabulary, FAIR Data JSON Schema builds directly on native JSON Hyper-Schema links and the official Link Relations Controlled Vocabulary (/cv/link-relations-v1).

2. The Solution: Enriched `links`#

Minimal `fair:` Keywords on Link Objects#

Keyword	Type	Description	CDIF / DCAT Alignment
`href` (Standard)	`string`	Target URI or RFC 6570 URI template.	`dcat:downloadURL` or `dcat:accessService`
`rel` (Standard)	`string`	Link relation type from Link Relations CV.	Relation / distribution role
`targetMediaType` (Standard)	`string`	MIME type of the payload (e.g. `application/vnd.apache.parquet`).	`dcat:mediaType`
`targetSchema` (Standard)	`object`/`string`	Schema validating the target resource or OpenAPI spec.	`dcat:accessService`
`fair:byteSize`	`integer`	Physical file size in bytes (minimum 0).	`dcat:byteSize` / `schema:contentSize`
`fair:checksum`	`object`	Cryptographic hash (`algorithm`, `value`) verifying file integrity.	`spdx:checksum`
`fair:compressFormat`	`string`	Container compression MIME type (e.g. `application/gzip`).	`dcat:compressFormat`

3. Complete Recipe Example#

Here is a complete dataset schema with multi-format downloads, an OpenAPI query service, documentation, summary statistics, and frequencies:

{
  "$schema": "https://highvaluedata.net/fair-data-schema/dev",
  "$id": "https://example.org/schemas/nhanes-2024-dietary",
  "title": "2024 National Health and Nutrition Examination Survey — Dietary Data",
  "fair:resourceType": "dataset",
  "fair:license": "CC0-1.0",
  "fair:licenseRef": "https://creativecommons.org/publicdomain/zero/1.0/",
  "links": [
    {
      "rel": "enclosure",
      "title": "Direct Parquet File Download (Optimized for columnar queries)",
      "href": "https://example.org/downloads/nhanes_2024_dietary.parquet",
      "targetMediaType": "application/vnd.apache.parquet",
      "fair:byteSize": 18500000,
      "fair:checksum": {
        "algorithm": "SHA256",
        "value": "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08"
      }
    },
    {
      "rel": "alternate",
      "title": "Compressed CSV Distribution",
      "href": "https://example.org/downloads/nhanes_2024_dietary.csv.gz",
      "targetMediaType": "text/csv",
      "fair:compressFormat": "application/gzip",
      "fair:byteSize": 45200000,
      "fair:checksum": {
        "algorithm": "SHA256",
        "value": "5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8"
      }
    },
    {
      "rel": "service",
      "title": "Interactive Query API (OpenAPI 3.1)",
      "href": "https://api.example.org/v1/dietary{?respondent_id,food_code,limit}",
      "targetMediaType": "application/json",
      "description": "Dynamic REST API supporting parameterized filtering."
    },
    {
      "rel": "documentation",
      "title": "Variable Codebook & Survey Questionnaire (PDF)",
      "href": "https://example.org/docs/nhanes_2024_documentation.pdf",
      "targetMediaType": "application/pdf"
    },
    {
      "rel": "summary-statistics",
      "title": "Pre-computed Nutrient Univariates & Summary Metrics",
      "href": "https://example.org/stats/nhanes_2024_dietary_summary.json",
      "targetMediaType": "application/json"
    },
    {
      "rel": "frequencies",
      "title": "Dietary Food Code Frequency Distributions",
      "href": "https://example.org/stats/nhanes_2024_food_frequencies.json",
      "targetMediaType": "application/json"
    },
    {
      "rel": "about",
      "title": "NHANES Study Portal Landing Page",
      "href": "https://doi.org/10.5281/zenodo.1234567",
      "targetMediaType": "text/html"
    }
  ],
  "type": "object",
  "properties": {
    "SEQN": { "type": "integer", "title": "Respondent Sequence Number" },
    "DR1TKCAL": {
      "type": "number",
      "title": "Energy Intake (kcal)",
      "fair:unit": "kcal",
      "fair:measurementUnitRef": "http://qudt.org/vocab/unit/KiloCAL"
    }
  }
}

4. Differentiating Distribution Types#

Distribution Type	Link Relation (`rel`)	`targetMediaType`	How AI Agents & Exporters Interpret It
Direct File Stream	`enclosure` / `alternate`	`application/vnd.apache.parquet`, `text/csv`	Streams raw bytes, checks `fair:byteSize` quota, and validates `fair:checksum`. Exported as `dcat:downloadURL`.
Interactive Query API	`service`	`application/json`, `application/vnd.oai.openapi+json`	Calls parameterized URI template or inspects `targetSchema` for dynamic tool calling. Exported as `dcat:DataService`.
Landing Page / Web UI	`about`	`text/html`	Navigates to user portal or terms-of-use agreement. Exported as `dcat:accessURL` or `schema:url`.
Documentation & Materials	`documentation`	`application/pdf`, `text/html`, `application/json`	Ingests variable codebooks, questionnaires, data dictionaries, or methodology reports.
Summary Statistics	`summary-statistics`	`application/json`, `text/csv`	Accesses pre-aggregated univariates and descriptive metrics (can be a data file or schema).
Frequency Tables	`frequencies`	`application/json`, `text/csv`	Accesses pre-aggregated categorical frequency distributions (can be a data file or schema).

5. Python Pydantic Model Authoring#

You can author and validate schemas with distributions directly in Python:

from models import ChecksumObject, DatasetSchema, LinkDescriptionObject

schema = DatasetSchema(
    id="https://example.org/schemas/nhanes-2024",
    title="2024 National Health Survey",
    links=[
        LinkDescriptionObject(
            rel="enclosure",
            title="Direct Parquet Download",
            href="https://example.org/downloads/nhanes_2024.parquet",
            target_media_type="application/vnd.apache.parquet",
            fair_byte_size=18500000,
            fair_checksum=ChecksumObject(
                algorithm="SHA256",
                value="9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08",
            ),
        ),
        LinkDescriptionObject(
            rel="service",
            title="Interactive Query API",
            href="https://api.example.org/v1/nhanes{?year,region}",
            target_media_type="application/json",
        ),
    ],
)

print(schema.to_json(indent=2))