# Persistent Identifiers & Discovery (PIDs, DIDs, GUIDs & Scoped IDs)

Findability and Discovery are fundamental pillars of the **FAIR Principles** (F1: (Meta)data are assigned a globally unique and persistent identifier) and **CDIF (Cross-Domain Interoperability Framework)**.

This guide explains how to document primary canonical identifiers (`$id`, `$anchor`), web-resolvable persistent identifiers (DOIs, w3ids, DIDs, ARKs, Handles), random synthetic identifiers (GUIDs/UUIDs), and secondary identifiers across administrative uniqueness scopes (`global`, `regional`, `national`, `institutional`, `project`, `local`, `other`).

---

## 1. Primary Identifiers: `$id` and `$anchor`

Standard JSON Schema Draft 2020-12 uses `$id` to declare the canonical base URI of a schema object. In FAIR Data JSON Schema, `$id` serves as the primary **Persistent Identifier (PID)** or **Decentralized Identifier (DID)** for a dataset or data product.

### Persistent Web URIs as `$id`

Whenever possible, use a web-resolvable persistent URI as the `$id` of your root dataset schema:

```json
{
  "$schema": "https://highvaluedata.net/fair-data-schema",
  "$id": "https://doi.org/10.5281/zenodo.1234567",
  "title": "Global Longitudinal Demographic Survey (2024)"
}
```

Other valid persistent `$id` schemes include:

- **w3ids**: `https://w3id.org/my-project/dataset/v1`
- **ARKs**: `https://n2t.net/ark:/12345/fk36108`
- **Handles**: `https://hdl.handle.net/10222/3456`
- **Decentralized Identifiers (DIDs)**: `did:example:123456789abcdefghi`

### Fragment Identifiers (`$anchor`)

To reference specific tables or individual variables within a larger schema document, use `$anchor`. Combining `$id` with `$anchor` produces a globally resolvable URI fragment PID:

```json
"properties": {
  "age": {
    "$anchor": "var_age",
    "type": "integer",
    "fair:label": "Respondent Age"
  }
}
```

The resulting PID for the `age` variable is `https://doi.org/10.5281/zenodo.1234567#var_age`.

### Benefits of Root Anchors (`$anchor` at Root Level)

While `$id` provides the canonical base URI for the document, defining a root `$anchor` (e.g., `"$anchor": "dataset_demographics_2024"`) provides key advantages:

- **Decoupling Opaque PIDs (DOIs) from Readable Slugs**: PIDs like DOIs (`https://doi.org/10.5281/zenodo.1234567`) are numeric and opaque. A root `$anchor` creates a human-readable alias URI (`https://doi.org/10.5281/zenodo.1234567#dataset_demographics_2024`), making `$ref` statements in external schemas self-documenting.
- **Bundled Catalogs & Multi-Schema Documents**: When multiple dataset schemas are packaged into a single master file (e.g., inside `$defs` or an API catalog payload), sub-schemas share the master file's `$id`. The root `$anchor` allows tools to target and pull out that specific dataset schema from the bundle via `$ref`.
- **Offline & Local Registry Resolution**: If schemas are resolved locally (where `https://doi.org/...` cannot be fetched over the network), internal references using `#dataset_demographics_2024` remain valid within the local schema scope regardless of base URI rewriting.
- **Uniform Identifier Hierarchy**: Establishes a consistent naming pattern across the entire data product hierarchy:
  - **Dataset Root**: `https://doi.org/10.5281/zenodo.1234567#dataset_demographics_2024`
  - **Table / Resource**: `https://doi.org/10.5281/zenodo.1234567#table_respondents`
  - **Variable / Property**: `https://doi.org/10.5281/zenodo.1234567#var_age`

---

## 2. Resolvable PIDs vs. Random GUIDs / UUIDs

Data stewards and software systems deal with two main classes of identifiers:

| Identifier Category          | Examples                    | Resolution Mechanism           | Primary Use Case                                                   |
| :--------------------------- | :-------------------------- | :----------------------------- | :----------------------------------------------------------------- |
| **Resolvable PIDs**          | DOI, w3id, DID, ARK, Handle | Direct HTTP GET / DID resolver | Canonical dataset identification, publishing, external citation    |
| **Random / Synthetic GUIDs** | UUID v4 (`urn:uuid:...`)    | Non-resolvable (offline hash)  | Local database keys, transaction logs, immutable payload snapshots |

### Resolvable PIDs

Resolvable PIDs provide active resolution services that redirect human users to dataset landing pages and machine agents (LLMs, FAIR tools, MCP servers) to machine-readable JSON Schema payloads.

### Random Identifiers (GUIDs / UUIDs)

Random synthetic identifiers like RFC 4122 UUID v4 (`urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6`) guarantee mathematical uniqueness (`global` scope) without relying on a centralized domain authority. They are ideal for tagging offline data extracts, local primary keys, or runtime message payloads.

---

## 3. Secondary Identifiers (`fair:identifiers`)

Datasets and variables often carry legacy accession numbers, institutional database keys, or project-level grant numbers. The `fair:identifiers` annotation keyword follows **progressive disclosure**: start with simple string shorthands and add richer metadata only when needed.

### 🟢 MVP Shorthand: Simple String List (Option A)

For 90% of everyday datasets, simply list your identifier strings:

```json
"fair:identifiers": [
  "10.5281/zenodo.1234567",
  "ICPSR-38492",
  "urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6"
]
```

### 🟢 MVP Shorthand: Clean 2-Key Objects (Option B)

Add light category or scope context using clean key aliases (`type` and `scope`):

```json
"fair:identifiers": [
  { "identifier": "10.5281/zenodo.1234567", "type": "DOI" },
  { "identifier": "ICPSR-38492", "type": "Accession", "scope": "institutional" },
  { "identifier": "GRANT-2024-EU-8842", "type": "Custom", "scope": "project" }
]
```

### 🔵 Advanced Stewardship: Extended Objects (Optional)

When institutional data stewards require active resolution links, formal scheme references, or historical notes, expand to full objects:

```json
"fair:identifiers": [
  {
    "identifier": "10.5281/zenodo.1234567",
    "identifierType": "DOI",
    "identifierTypeRef": "https://highvaluedata.net/fair-data-schema/cv/identifier-types-v1#DOI",
    "uniquenessScope": "global",
    "scheme": "DataCite",
    "isResolvable": true,
    "resolverUrl": "https://doi.org/",
    "description": "Canonical DOI published on Zenodo."
  }
]
```

### Uniqueness Scopes

The `uniquenessScope` property specifies the administrative or spatial scope within which the identifier is guaranteed unique:

- **`global`**: Globally unique across all systems worldwide (DOIs, w3ids, DIDs, UUIDs).
- **`regional`**: Unique within a supranational geopolitical region (e.g. Eurostat series keys, EU NUTS codes).
- **`national`**: Unique within a national statistical agency or country (e.g. US Census ID, INSEE commune code).
- **`subnational`**: Unique within a subnational jurisdiction, state, province, canton, city, municipality, or district (e.g. US State FIPS code, city parcel ID).
- **`institutional`**: Unique within a specific university, data bank, or repository (e.g. ICPSR study number, Dataverse accession).
- **`project`**: Unique within a research project or grant (e.g. `PROJ-2024-VAR01`).
- **`local`**: Unique only within a single table, database schema, or runtime instance (e.g. primary key integer).
- **`other`**: Custom or unlisted administrative uniqueness scope.

Controlled vocabulary reference: [Identifier Scopes CV](https://highvaluedata.net/fair-data-schema/cv/identifier-scopes-v1).
Controlled vocabulary reference: [Identifier Types CV](https://highvaluedata.net/fair-data-schema/cv/identifier-types-v1).

---

## Primer on Identifiers

Understanding the core properties, trade-offs, and ecosystem tooling of digital identifiers helps data stewards and software engineers choose the right identifier strategy for datasets, tables, and variables.

### 📌 1. Persistent Identifiers (PIDs)

- **Concept**: A long-lasting, stable digital reference to a resource that remains constant even if the hosting institution, web server URL, or backend database changes over time.
- **How it Works**: PIDs rely on institutional commitment and central resolution infrastructure (e.g. DataCite, Crossref, ORCID, ROR) to update target URLs behind the scenes without breaking citations.
- **Key Standards & Schemes**: Digital Object Identifiers (`DOI`), Archival Resource Keys (`ARK`), Handles (`Handle`), Permanent Web URLs (`w3id`).
- **Primary Benefits**: Ensures long-term research reproducibility, FAIR Findability (F1), and citation integrity across decades.
- **Drawbacks & Challenges**: Some persistent identifiers may involve registration and fees (e.g. formal DataCite or Crossref memberships). All PIDs require ongoing commitment to keep target redirection URLs updated.
- **Common Tools & Services**:
  - **Free / Open Access / Self-Hosted**: Zenodo, Figshare, Harvard Dataverse, ORCID, ROR Registry, `w3id.org`, N2T.net (ARK resolver), self-hosted PURLs/ARKs.
  - **Fee-Based / Institutional Membership Registrars**: DataCite, Crossref, EZID (requires institutional subscription/account for direct minting).

### 🌐 2. Resolvable Identifiers

- **Concept**: An identifier that can be actively dereferenced over a network protocol (HTTP/HTTPS, DNS, or DID resolution protocol) to retrieve metadata, documentation, or payload data.
- **How it Works**: When a human browser or machine agent (LLM, MCP server, API tool) sends an HTTP GET request (or content negotiation header `Accept: application/schema+json`), the resolver returns machine-actionable metadata.
- **Key Standards & Schemes**: Web PIDs (`https://doi.org/...`, `https://w3id.org/...`), HTTP URIs, resolvable DIDs (`did:web:...`).
- **Primary Benefits**: Enables automated machine-to-machine discovery, API tool calling, and programmatic FAIR data access without human intervention.
- **Drawbacks & Challenges**: Subject to network outages, resolver server downtime, or link rot if web domain registrations expire or HTTP redirection rules break.
- **Common Tools & Services**: `w3id.org` (community redirector), Handle.net, DOI.org, `purl.org`, cURL / HTTP client libraries with content negotiation support (`httpx`, `fetch`).

### 🔐 3. Decentralized Identifiers (DIDs)

- **Concept**: A W3C standard for globally unique, self-sovereign digital identifiers that operate without relying on a centralized registration authority, DNS provider, or single database operator.
- **How it Works**: DIDs use public-key cryptography, distributed ledgers, or peer-to-peer networks to cryptographically verify data authenticity, origin, and ownership.
- **Key Standards & Schemes**: W3C DID Specification (`did:key:...`, `did:ion:...`, `did:web:...`).
- **Primary Benefits**: Enables tamper-proof, cryptographically verifiable data products across decentralized ecosystems and multi-party research networks.
- **Drawbacks & Challenges**: Higher technical complexity, evolving standards across different DID methods (`did:key` vs `did:web` vs `did:ion`), and potential blockchain gas fees or indexer latency depending on the underlying DID method.
- **Common Tools & Services**: W3C Universal Resolver (`resolver.identity.foundation`), Veramo Framework, ION / Sidetree, Spruce ID, `did-cli`, DIF (Decentralized Identity Foundation) SDKs.

### 🎲 4. Random / Synthetic Identifiers (GUIDs & UUIDs)

- **Concept**: Mathematically generated, non-semantic strings created algorithmically without central coordination, designed to guarantee collision prevention.
- **How it Works**: Uses 128-bit pseudo-random generators (e.g. RFC 4122 UUID v4) to ensure that the probability of generating identical strings across different systems worldwide is effectively zero.
- **Key Standards & Schemes**: Universally Unique Identifiers (`UUID` v4), Globally Unique Identifiers (`GUID`).
- **Primary Benefits**: Fast, offline generation for internal primary keys, payload hashes, or message transaction logs without needing web domain setup or registrar fees.
- **Drawbacks & Challenges**: **Typically non-resolvable over HTTP natively**—a raw UUID (`f81d4fae-7dec-11d0-a765-00a0c91e6bf6`) cannot be pasted into a browser to view metadata unless wrapped in a resolver URL. Opaque and non-semantic (contains no human title or domain context).
- **Common Tools & Services**: `uuidgen` CLI tool, Python `uuid` module, JavaScript `crypto.randomUUID()`, PostgreSQL `gen_random_uuid()`, Rust `uuid` crate.

---

## CDIF Metadata Publication, Signposting & RDA PID Kernel Alignment

The [Cross-Domain Interoperability Framework (CDIF) Handbook on Publishing Metadata](https://cross-domain-interoperability-framework.github.io/cdifbook/metadata/publishingmetadata) defines mechanisms for making digital objects discoverable by search engines, machine crawlers, and autonomous agents across domains.

### 🌐 1. Signposting & FAIR Digital Object (FDO) Discovery (IETF RFC 8288)

CDIF recommends [Signposting](https://signposting.org/)—using typed HTTP web links ([IETF RFC 8288](https://datatracker.ietf.org/doc/html/rfc8288)) and IANA-registered link relation types—to allow software agents to discover a resource's metadata when resolving its PID URI (`$id`).

When hosting or serving FAIR Data JSON Schemas over HTTP, web servers and API endpoints should return typed Signposting link headers:

```http
HTTP/1.1 200 OK
Content-Type: application/schema+json
Link: <https://doi.org/10.5281/zenodo.1234567>; rel="cite-as",
      <https://highvaluedata.net/fair-data-schema/dev/schema.json>; rel="describedby"; type="application/schema+json",
      <https://spdx.org/licenses/CC-BY-4.0>; rel="license",
      <https://ror.org/007qwym43>; rel="author"
```

In human-readable landing pages, Signposting links can also be embedded directly in HTML `<head>` tags:

```html
<link rel="cite-as" href="https://doi.org/10.5281/zenodo.1234567" />
<link rel="describedby" href="https://example.org/dataset-schema.json" type="application/schema+json" />
<link rel="license" href="https://spdx.org/licenses/CC-BY-4.0" />
```

### 📋 2. RDA PID Kernel & FAIR Digital Object (FDOF) Attribute Mapping

The Research Data Alliance (RDA) Recommendation on [PID Kernel Information](https://doi.org/10.15497/RDA00031) and the FAIR Digital Object Framework (FDOF) specify a minimal metadata record returned when resolving a PID. The table below illustrates how FAIR Data JSON Schema keywords map to RDA PID Kernel attributes and CDIF schema.org recommendations:

| RDA PID Kernel Attribute | CDIF schema.org Element | FAIR Data JSON Schema Keyword | Mapping & Implementation Notes |
| :--- | :--- | :--- | :--- |
| **FDO Canonical PID** | `@id` | `$id` | Primary web-resolvable PID / base URI (`DOI`, `w3id`, `ARK`, `DID`). |
| **FDO Secondary PIDs** | `identifier` | `fair:identifiers` | List of secondary, institutional accessions, or legacy database keys. |
| **FDO Resource Type** | `@type` | `fair:resourceType` | Object role (`dataset`, `data-product`, `variable`). |
| **FDO Creator / Agent** | `creator` | `fair:contributors` (`role="Creator"`) | Agent responsible for creating the object / registering the PID. |
| **FDO Responsible Org** | `provider` / `publisher` | `fair:contributors` (`role="Provider"`) | Responsible organization holding custody (supports ROR URIs). |
| **Rights & License** | `license` | `fair:licenseRef` / `fair:license` | SPDX license URI or license terms string. |
| **Bit Checksum / Integrity** | `spdx:checksum` / `etag` | `fair:checksum` (or via `fair:version`) | Hash digest (e.g. SHA-256) verifying payload bit-sequence integrity. |
| **Persistency Policy** | `publishingPrinciples` | `fair:version` / `fair:description` | Maintenance intentions and archiving policies. |