Skip to content

Metadata Fetcher

Look up a book in Google Books, Open Library and Project Gutenberg, rank the candidates by how well they match, and run it in Node or as a Docker HTTP service.

@prose-reader/metadata-fetcher fetches a book's metadata from online catalogs. archive-reader reads what the file itself carries; this reads what the world knows about it.

It takes a compact FetchMetadataInput, asks a list of pluggable providers, scores every rich catalog candidate against that input, and returns the ranked alternatives plus the per-provider detail behind them:

import { resolveArchive } from "@prose-reader/archive-reader"
import {
createGoogleBooksProvider,
createOpenLibraryProvider,
createProjectGutenbergProvider,
fetchMetadata,
metadataInputFromResolvedArchive,
} from "@prose-reader/metadata-fetcher"
const resolved = await resolveArchive(archive)
const input = metadataInputFromResolvedArchive(resolved)
const fetched = await fetchMetadata(input, {
providers: [
createProjectGutenbergProvider(),
createGoogleBooksProvider({ apiKey: "your-api-key" }),
createOpenLibraryProvider(),
],
})
const match = fetched.matches.find(({ accepted }) => accepted)
match?.metadata.cover?.uri // what the best accepted candidate says
match?.score // …and how sure we are it is this book

Nothing here needs an archive: hand-built terms work just as well, which is what a "search by title" form or a filename-derived guess produces.

const fetched = await fetchMetadata(
{ title: "Dune", authors: ["Frank Herbert"] },
{ providers: [createOpenLibraryProvider()] },
)

The input is owned by the fetcher and contains exactly the fields its providers and matcher understand:

type KnownMetadataIdentifierScheme =
| "ISBN"
| "GTIN"
| "DOI"
| "GoogleBooks"
| "OpenLibrary"
| "ProjectGutenberg"
| "UUID"
| "URN"
| "URL"
| "Unknown"
type MetadataIdentifierScheme =
| KnownMetadataIdentifierScheme
| (string & {}) // custom catalog/application scheme
type MetadataIdentifier = {
value: string
scheme: MetadataIdentifierScheme
}
type FetchMetadataInput = {
title?: string
authors?: ReadonlyArray<string>
identifiers?: ReadonlyArray<MetadataIdentifier>
series?: string
publisher?: string
publishedYear?: number
languages?: ReadonlyArray<string>
numberOfPages?: number
}

Some fields start a request (title, authors, series, identifiers); others disambiguate the candidates that come back (publisher, publishedYear, languages, page count). ISBN, GTIN and catalog ids all use the same identifier shape: { value: "9780441013593", scheme: "ISBN" } and { value: "zyTCAlFPjgYC", scheme: "GoogleBooks" }. The known schemes give TypeScript autocomplete, while any consumer-defined string remains valid. Unknown explicitly preserves a source identifier whose namespace cannot be classified; URL means a validated absolute HTTP(S) identifier; UUID and URN are what an EPUB's urn:uuid: unique identifier and other URNs resolve to. Reading a specific identifier back out of an input or a candidate belongs to archive-reader: identifierValue accepts both the ISBN and GTIN schemes for either, reads a catalog id out of the reference URL that carries it, and normalizes the authored value — that is what the providers and the matcher use. Fields with no input role — covers, descriptions, subjects, rights, layout and format-specific metadata — are deliberately absent.

metadataInputFromResolvedArchive is the explicit bridge from archive-reader. It only requires the metadata projection, so callers do not need to resolve reading order, TOC or sources for a lookup:

const resolved = await resolveArchive(archive, { include: ["metadata"] })
const input = metadataInputFromResolvedArchive(resolved)

The adapter selects authors from contributors, the first series, and the edition publisher/year with original-publication data as fallback. It forwards every identifier with its scheme and removes only archive-specific details such as unique.

fetchMetadata returns a plain-JSON entity — structured-clone-able, persistable, cacheable — the metadata-fetching equivalent of ResolvedArchive:

type FetchedMetadata = {
version: number
/** every match across every provider, ranked best-first */
matches: MetadataMatch[]
/** per-provider detail, keyed by provider id */
sources: Record<string, { provider: { id, name }; matches: MetadataMatch[] }>
/** providers whose search threw */
/** providers that dropped out, with the HTTP status when there was one */
failedProviders: { id: string; status?: number }[]
}

version is the schema version, currently 5, bumped only when the shape or meaning of existing fields changes incompatibly — a MetadataMatch included. Every candidate uses the explicit publication.original / publication.edition vocabulary.

The same value is exported as FETCHED_METADATA_VERSION, for a consumer that persists part of the entity, or builds that shape without calling fetchMetadata, and still wants to tag what it stores:

import {
DEFAULT_METADATA_MATCH_MIN_SCORE,
FETCHED_METADATA_VERSION,
scoreMetadataCandidate,
} from "@prose-reader/metadata-fetcher"
// keep only the best accepted match, with the schema it was written in
const [best] = fetched.matches.filter(({ accepted }) => accepted)
if (best) db.put(bookId, { version: FETCHED_METADATA_VERSION, match: best })
// a migration scoring records stored before metadata-fetcher, offline, writes
// the same shape — and has no fetchMetadata result to read the version from,
// so it accepts by the threshold fetchMetadata uses when given no minScore
const { score, signals } = scoreMetadataCandidate(input, legacy.metadata)
db.put(bookId, {
version: FETCHED_METADATA_VERSION,
match: {
providerId: legacy.providerId,
score,
signals,
accepted: score >= DEFAULT_METADATA_MATCH_MIN_SCORE,
metadata: legacy.metadata,
},
})

matches are deliberately not consolidated into a single metadata record. A score says how strongly a candidate resembles the input; it does not prove that several accepted candidates describe the same edition. Keeping the candidates separate avoids silently mixing fields from competing records and leaves selection policy with the consumer.

sources groups those same matches by provider. Unlike archive sources, the key space is open: providers are pluggable, so the keys are whatever providers you passed.

A match is a candidate plus the evidence for it:

type MetadataMatch = {
providerId: string
/** 0–1 aggregate confidence */
score: number
/** every field comparison behind the score */
signals: {
field: "isbn" | "gtin" | "identifiers" | "title" | "authors" |
"series" | "publisher" | "publishedYear" | "languages" |
"numberOfPages"
score: number
weight: number
/** the two values compared, rendered for display */
query: string
candidate: string
}[]
/** reached minScore */
accepted: boolean
metadata: ResolvedMetadata
id?: string
url?: string
raw?: unknown
}

Scoring is done by the package, identically for every provider — a provider never grades its own homework, so scores stay comparable across catalogs and a match stays explainable to a user ("same title and author, different edition publisher"):

fetched.matches[0]?.signals
// [
// { field: "title", score: 1, weight: 0.8, query: "Dune", candidate: "Dune" },
// { field: "authors", score: 1, weight: 0.5, query: "Frank Herbert", … },
// { field: "publisher", score: 0.2, weight: 0.15, query: "Ace", candidate: "Chilton Books" },
// ]

The rules:

  • Only comparable fields count. A field is compared when both sides state it. What the book doesn't know can neither raise nor lower a candidate, so a sparse EPUB and a rich one land on the same 0–1 scale. A candidate with nothing in common to compare scores 0.

  • The aggregate is a weighted average of those fields, with weights expressing intent rather than statistics (exported as METADATA_MATCH_WEIGHTS):

    Field Weight
    isbn, gtin 1 identifiers are the book
    title 0.8 what a human recognizes it by
    identifiers 0.6 other identifier systems, compared within the namespace both sides state
    authors 0.5 compared as people
    series 0.3
    publishedYear 0.2 year proximity against the candidate's publication data
    publisher, languages, numberOfPages 0.15 publication details — they must never sink a convincing match alone
  • Confirmed identity settles it. An agreeing ISBN or GTIN pins the score to 1 whatever else disagrees. A contradicting ISBN or GTIN scores 0, which sinks a plausible-looking wrong edition. ISBN-10 and ISBN-13 of the same book compare equal (toIsbn13 is exported).

  • Other identifiers are compared within a namespace both sides state. A catalog id is read the way identifierValue reads it, so a Gutenberg URL and a ProjectGutenberg number, or a Google Books link and a GoogleBooks volume id, are one namespace; any other stated scheme is its own. A UUID is read the same way, so its bare and urn:uuid: spellings, in either case, are one value, and any other URN is scoped by the namespace it names and compared as RFC 8141 compares URNs, whatever the case of that namespace or of a percent-encoded octet: a urn:nbn: and a urn:issn: identifier are never compared. Ids compare in the spelling their namespace canonicalizes to, so a Google Books volume id is case-sensitive, while a DOI compares without case, as the DOI system defines it. A shared identifier pins the score to 1. A different id in the same namespace is a disagreement, scored 0 at its weight rather than pinned, since one edition can sit under several records of a catalog. Lists with no namespace in common are not compared at all, and add no identifiers signal: an EPUB 3 urn:uuid and a catalog's own id are different id spaces, not a contradiction. An untyped value that is neither a URN nor a known catalog's URL (an Unknown string, a publisher's web page) has no namespace, and nor has any value whose scheme is blank, a URN included, so neither is ever compared, even against an equal value of another scheme.

  • Comparisons are fuzzy where the world is. Titles compare on character bigrams, against both the candidate's main title and that title composed with its subtitle entry — a book states Dune: A Novel in one dc:title where a catalog splits it in two, and only the comparison string is composed, never the returned metadata. The subtitle asymmetry is repaired on top (Dune ≡ Dune: a novel, but Dune: Book One ≠ Dune: Messiah). An explicit conflicting volume, part, or book number makes the title score 0 (Vol. I ≠ vol. 2) — in any compared form, so a bare Irina cannot launder the candidate's own Vol. 1 against a query for Vol. 2, and a number the query states in its title is also compared against the candidate's series position, where a catalog keeps the same fact. Names compare on their token set (Herbert, Frank ≡ Frank Herbert); diacritics and punctuation are folded (Les Misérables ≡ Les Miserables); languages compare on their primary subtag (en-US ≡ en).

  • Flat publication evidence uses the best candidate value. publisher and publishedYear compare against both the candidate's original and edition publication details when present; the closest value supplies the single signal.

  • A rating is never evidence. aggregateRating is returned but deliberately not scored: how much a crowd liked a book says nothing about whether it is this book, and a popular edition must not outrank the right one.

minScore (default 0.5, exported as DEFAULT_METADATA_MATCH_MIN_SCORE) decides which matches are accepted. Rejected matches are kept and ranked, not dropped: "the catalog found three books, none convincing" is an answer, and it is exactly what a "did you mean?" picker renders.

if (!fetched.matches.some(({ accepted }) => accepted)) {
showPicker(fetched.matches) // nothing convincing — let the user choose
}
const fetched = await fetchMetadata(input, {
providers: [
createProjectGutenbergProvider(),
createGoogleBooksProvider({ apiKey: "your-api-key" }),
createOpenLibraryProvider(),
],
limit: 5,
minScore: 0.5,
includeRaw: false,
signal: controller.signal,
})
Option Default
providers — the catalogs to ask, queried concurrently. Their order is the tie-break precedence: equally-scored matches rank in declaration order
limit 5 hard cap on the matches kept per provider, best-scoring first, and the page-size hint passed to each provider
minScore 0.5 score a match must reach to be accepted; the default is exported as DEFAULT_METADATA_MATCH_MIN_SCORE, for a consumer scoring candidates itself with scoreMetadataCandidate
includeRaw false keep each provider's own record on match.raw. Off by default: provenance most consumers don't need, and it can dwarf the normalized entity they persist
signal — cancellation, forwarded to every provider

Mirrors resolveArchive: a provider that throws — a rate limit, an outage, a network error, a malformed response — never fails the fetch. Providers are asked concurrently and each is isolated, so one failing costs you that catalog and nothing else. It is logged through the debug Report and listed in failedProviders, the only trace left:

// don't persist "we found nothing" when we simply couldn't ask
if (fetched.failedProviders.length === 0) cache.set(bookId, fetched)
// rate limited rather than broken: worth asking again, later
const throttled = fetched.failedProviders.filter(({ status }) => status === 429)

Each entry carries the catalog's HTTP status when the failure was a response — 429 (ask again later), 5xx (the catalog is down), 4xx (we asked wrong) — and no status when it wasn't one, which is itself the answer: a network error, an unparseable payload, a bug. Three states are distinguishable: a provider in sources answered, one in failedProviders threw, one in neither was never asked (narrowed out by the providers option).

fetchMetadata itself does not retry or interpret Retry-After; providers own that policy because only they know which operations are idempotent and which failures their catalog treats as transient. The built-in providers share one:

  • Each request makes at most three attempts.
  • A network failure, a rate limit (429) and an outage (500, 502, 503, 504) are retried with abortable exponential backoff and jitter, bounded at 3 seconds per wait. Cancelling through signal interrupts a wait.
  • When the catalog sends Retry-After, the provider waits exactly that long instead. A Retry-After beyond the 3-second bound is not waited out: the provider gives up at once, since an earlier retry would only be refused again.
  • Client errors, such as an invalid key, are not retried.

A provider that exhausts its retries, or gives up on a long Retry-After, is reported here normally, with its status.

Cancelling through signal is the one exception: it rejects, since the caller asked for it.

The globally ranked matches list makes the simplest policy explicit: select its first accepted entry. A UI can instead present several alternatives, prefer a provider, or require a confirmed identifier before applying anything.

const match = fetched.matches.find(({ accepted }) => accepted)
if (match !== undefined) {
importMetadata(match.metadata)
}

Each candidate remains a ResolvedMetadata, so it can be handled with the same application code as metadata read from the publication. The fetcher does not decide how local and remote values should be reconciled.

aggregateRating is each provider's own crowd, normalized to the shared 0–5 scale — Google's mean and Open Library's are different populations of readers, not two measurements of one number, so a picker showing both should attribute them (match.providerId) rather than average them. It is also the most volatile thing in a persisted FetchedMetadata: a title, an ISBN and a page count do not move, a rating does. Refresh it on its own schedule, or treat a cached one as indicative.

Provider Import Catalog Notes
createProjectGutenbergProvider @prose-reader/metadata-fetcher Project Gutenberg official per-eBook RDF; exact Gutenberg identifiers only, no key
createGoogleBooksProvider @prose-reader/metadata-fetcher Google Books broad edition metadata and covers; API key required
createOpenLibraryProvider @prose-reader/metadata-fetcher Open Library free, no key; books broadly, comics and manga much less so
import { createProjectGutenbergProvider } from "@prose-reader/metadata-fetcher"
const provider = createProjectGutenbergProvider({
userAgent: "MyReader/1.0 (contact@example.com)",
})
Option Default
baseUrl https://www.gutenberg.org catalog origin; override for a mirror or stub
fetch the global one for tests, a custom agent, or a caching layer
userAgent — optional identifying user agent

Lookup strategy, exactly one request, retried under the built-in providers' retry policy: the provider recognizes an official Gutenberg URL in identifiers (including /ebooks/78139, /files/78139/…, /cache/epub/78139/…, and the older /78139 form), or a numeric identifier whose scheme is ProjectGutenberg. It then requests Gutenberg's official per-eBook RDF record at /cache/epub/{id}/pg{id}.rdf. A title or author alone never triggers the provider, so it neither crawls the website nor returns a fuzzy Gutenberg hit. Arbitrary URLs and identifiers with another authored scheme are ignored.

The confirmed candidate carries { value: "78139", scheme: "ProjectGutenberg" }, which the matcher compares with the book's URL as the same Gutenberg id: the match is decisive even when the catalog and embedded titles differ. A missing RDF record returns no candidate; a failing or malformed response is reported through failedProviders like any other provider failure.

Mapping (per-eBook RDF → ResolvedMetadata; same exhaustive-destructure check as the other providers):

Project Gutenberg RDF Resolved
pgterms:ebook@rdf:about identifiers numeric scheme ProjectGutenberg, which the matcher compares with a Gutenberg URL the book states
dcterms:title title, titles
dcterms:creator, marcrel:* contributors common MARC relators normalize to roles such as author, translator, editor, and illustrator; unknown codes remain verbatim
dcterms:issued publication.edition.date Gutenberg ebook release date, parsed through day precision when available; it is not the work's original publication date
dcterms:publisher publication.edition.publisher the publisher of the Gutenberg edition
pgterms:marc260 publication.original.date, publication.original.publisher original print statement; only one unambiguous $c year and $b publisher are promoted
dcterms:rights rights
dcterms:language languages RDF values are already BCP 47
LCSH dcterms:subject, pgterms:bookshelf subjects deduped and capped at 25; Library of Congress classification codes are excluded
pgterms:marc520, then dcterms:description description catalog summary preferred over a general note
medium/small cover in dcterms:hasFormat cover medium preferred; absolute HTTP(S) url, confidence: "derived"

The response is parsed and normalized in memory. The provider has no database, cache, file writes, or other persistence.

import { createGoogleBooksProvider } from "@prose-reader/metadata-fetcher"
const provider = createGoogleBooksProvider({
apiKey: "your-api-key",
})
Option Default
apiKey — required application API key
baseUrl https://www.googleapis.com/books/v1 API root; override for a stub
fetch the global one for tests, a custom agent, or a caching layer

Lookup strategy, from most exact to broadest: an authored GoogleBooks identifier (or official Google Books/API URL), an ISBN identifier, then title plus first author. When the author makes a title query too narrow, the provider tries title alone. Search requests ask only for books, preserve Google's relevance order, and never exceed 40 results — the API's maximum. A query with none of these terms returns no candidates without making a request.

Title and author reach intitle:/inauthor: unquoted. A quoted phrase makes Google demand the catalog title contain it verbatim, which a title carrying a volume marker, a scanner tag or a translated subtitle never does — and with no query relaxation to retry with, the lookup would report nothing for a book Google holds. Unquoted, the catalog ranks loosely and the matcher decides, which is where precision belongs.

Unquoted also means Google parses the term as syntax rather than reading it literally, so anything in it that Google would take as an instruction is neutralized first: quotes and backslashes (they unbalance the query), colons (Dune inauthor:Asimov would search authors) and a hyphen starting a word (BLAME! -Master Edition- would demand books without "Master"). A term left with nothing searchable counts as absent, so a title that is only query syntax makes no request.

For the same reason a title search asks Google for four times limit: those candidates arrive in Google's relevance order rather than the matcher's, so the edition the caller described can sit below the ones Google finds most popular. fetchMetadata scores the wider page and trims to limit afterwards. Exact identifier and ISBN lookups need no such room and ask for limit.

To replace Oboku's googleVolumeId, pass it directly:

const input = {
title: "Dune",
identifiers: [{ value: googleVolumeId, scheme: "GoogleBooks" }],
}

An official Google Books website/API URL with scheme URL is recognized too. Every candidate carries its volume id as { value: id, scheme: "GoogleBooks" }, which the matcher compares with either spelling of the book's id, so an exact lookup is decisive, and metadataInputFromResolvedArchive can forward the id unchanged for a later refresh. A missing volume id falls through to ISBN/title when available. ISBN searches likewise retain the queried ISBN identifier even when a sparse Google record omits industryIdentifiers.

Each request follows the built-in providers' retry policy: rate limits and outages are retried, honouring Retry-After, and client errors such as an invalid key are not. When the last attempt fails, or when a successful exact response is malformed, the provider is reported through failedProviders normally.

Mapping (Google Books Volume → ResolvedMetadata; the resolver names every parsed field and asserts the rest is empty, so a field added to the parser fails the build until someone decides what becomes of it):

Google Books Resolved
id identifiers scheme GoogleBooks; also id on the match
volumeInfo.title titles verbatim; the main title (first entry)
subtitle titles its own entry, typed subtitle — never appended to the main title
seriesInfo.volumeSeries belongsTo.series seriesId as a GoogleBooks identifier, orderNumber as position. Google never states the series name, and bookDisplayNumber / shortSeriesBookTitle are display strings by its own definition, so neither is mapped
authors contributors role author
publisher, publishedDate publication.edition partial dates retain the available year/month/day
description description Google may provide simple HTML formatting
industryIdentifiers identifiers ISBN schemes normalize to ISBN; every announced identifier is retained
pageCount numberOfPages
averageRating, ratingsCount aggregateRating Google's mean is already on the shared 0–5 scale (it rounds to a half star); a count alone states no rating, and a mean over zero ratings is dropped
categories subjects deduped and capped at 25
language languages Google's best ISO 639-1 language
imageLinks cover the largest linked size, or the thumbnail asked at 1,280 px wide; HTTPS, without the page curl. See Covers
canonicalVolumeLink, then infoLink match url upgraded to HTTPS; a stable Google Books URL is synthesized from id when absent

Google-specific values without a cross-format home (access and sale data, reading-mode flags, and so on) remain available on match.raw when includeRaw is enabled.

A Google Books match carries the largest cover the provider can link without another request, and without the page curl Google draws on its covers.

Google's search links only two small sizes, at most about 128 px wide. A volume's own record also links large (800 px wide) and extraLarge when Google holds the publisher's cover, but only a lookup by GoogleBooks id reads that record. So the provider picks:

  1. extraLarge or large, when the record links them.
  2. Else thumbnail, with fife=w1280 added. Google's image server then serves the same cover at most 1,280 px wide.
  3. Else the largest other link.

Each link goes over HTTPS and loses edge=curl, which serves the same image at the same size without the curl. The provider keeps every other parameter as Google wrote it. In particular, it never changes zoom: for a volume Google knows only from a library catalog, another zoom serves Google's "image not available" picture, or another page of a scanned book.

The fife size has these side effects, which no API reports:

  • A publisher cover found by search stops at 1,280 px wide. A smaller source is enlarged to that width.
  • A catalog-only volume has no larger image, so its thumbnail comes back enlarged to about 300 px, with no new detail.
  • A scanned library book can show its title page instead of the cover its thumbnail shows.
  • Google documents none of fife, zoom or edge. If Google changes how it serves them, covers can come back small or wrong.
import { createOpenLibraryProvider } from "@prose-reader/metadata-fetcher"
const provider = createOpenLibraryProvider({
userAgent: "MyReader/1.0 (contact@example.com)",
})
Option Default
baseUrl https://openlibrary.org API origin
coversBaseUrl https://covers.openlibrary.org cover service origin
fetch the global one for tests, a custom agent, or a caching layer
userAgent — Open Library's API etiquette asks for an identifying one (app name + contact) and throttles anonymous traffic harder

Lookup strategy, at most four requests: an ISBN search when the book states one — the catalog then verifies the identity for us; an exact id_project_gutenberg search when an identifier is an official Project Gutenberg URL or a numeric ProjectGutenberg identifier; then a precise title (+ first author) search when those identifiers are unknown to Open Library or absent. When that precise search is empty, a free-text title-and-author query handles records whose work title and subtitle were indexed separately. The Gutenberg crosswalk is this provider's own, because id_project_gutenberg is a field of Open Library's search index. A query with none of those terms yields no candidates rather than a fishing expedition. Each request follows the built-in providers' retry policy. Open Library throttles anonymous and fast traffic with 429, which the policy retries.

Mapping (search.json doc → ResolvedMetadata; same exhaustive-destructure check as the other providers):

Open Library Resolved
title titles verbatim; the main title (first entry)
subtitle titles its own entry, typed subtitle — the matcher composes the two when it compares, so nothing is gained by joining them here
author_name contributors role author
first_publish_year publication.original.date.year
language languages MARC 21 → BCP 47 (eng → en); unknown codes pass through, und/mul/zxx are dropped
subject subjects capped at the first 25 — a popular work carries hundreds, most of them long-tail noise
number_of_pages_median numberOfPages
ratings_average, ratings_count aggregateRating the catalog's raw 0–5 mean of the work's ratings, unrounded. ratings_sortable is deliberately not mapped: it is Open Library's internal ranking key, not a rating it states
cover_i cover absolute url on the cover service, confidence: "derived"
id_project_gutenberg identifiers scheme ProjectGutenberg, which the matcher compares with a Gutenberg URL the book states
key identifiers scheme OpenLibrary, e.g. /works/OL893415W; also id and url on the match

An ISBN identifier is added only when the search was an ISBN lookup, and then it uses the queried value: the API answered "this work has that ISBN", which is a fact about the record. A title-search hit describes a work, whose editions each have their own ISBN, so picking one would be fabrication.

Only the mapped fields are requested (fields=), which is the difference between a few hundred bytes per hit and a few hundred kilobytes.

A Docker image wraps this package in a small HTTP API, so metadata lookups don't have to happen inside your JavaScript app — and so you can try a provider against a real catalog with curl:

For repository development, copy apps/metadata-fetcher-api/.env.example to apps/metadata-fetcher-api/.env, set GOOGLE_BOOKS_API_KEY, then run pnpm run start:metadata-fetcher from the repository root. Compose loads that file automatically and it remains ignored by Git.

Terminal window
docker run -p 6382:6382 \
-e GOOGLE_BOOKS_API_KEY="your-api-key" \
-e OPEN_LIBRARY_USER_AGENT="MyApp/1.0 (me@example.com)" \
mbret/prose-metadata-fetcher-api:latest
curl "http://localhost:6382/metadata?isbn=9780441013593"

The image is published with every release, tagged with the version and latest, for linux/amd64 and linux/arm64.

Route
GET /health liveness, plus the providers the deployment exposes
GET /metadata?title=&author=&isbn=… human-friendly lookup
POST /metadata body is a FetchMetadataInput, options on the query string

Both metadata routes answer with the FetchedMetadata entity verbatim. The options above (limit, minScore, includeRaw, plus a providers filter) are query parameters, and the deployment defaults are environment variables.

The service does no per-client rate limiting: who a client is, which address is real and how often each may call depend on your infrastructure and your own provider quotas, so that limit belongs to the proxy or gateway in front of it. It does bound its own capacity — MAX_CONCURRENT_LOOKUPS (default 32) caps the lookups one process runs at once, and a lookup beyond it answers 503 with Retry-After. This package has no limit of its own either. A built-in provider retries a catalog's 429 briefly, as its retry policy describes; one that persists is reported in failedProviders, and backing off further is the caller's decision.

For development, pnpm run start:metadata-fetcher starts the same image with the source bind-mounted: editing the express app or this package restarts it — node runs both as TypeScript, so nothing is rebuilt. It also serves a playground at / — enter metadata manually or choose an EPUB/CBZ/ZIP file, then inspect each candidate, its score and the signals behind it. In both cases the playground posts the canonical FetchMetadataInput JSON to /metadata, so the browser's network inspector shows the exact input an SDK consumer would provide. Uploaded files have no application-level size limit: archive-reader resolves them in memory and discards them before that compact input is posted, with no disk or browser storage. The playground is development-only: the production image sets NODE_ENV=production, and the route is then never registered.

See the app's README for the full reference.

A provider is three things: a stable id, a display name, and a search returning normalized candidates. It never scores its own results — that stays in the package, identically for everyone.

import { type MetadataProvider } from "@prose-reader/metadata-fetcher"
const myProvider: MetadataProvider = {
id: "myCatalog",
name: "My Catalog",
search: async (input, { limit, signal }) => {
if (input.title === undefined) return []
const response = await fetch(
`https://example.com/search?q=${encodeURIComponent(input.title)}` +
`&author=${encodeURIComponent(input.authors?.[0] ?? "")}`,
{ signal },
)
const { results } = await response.json()
return results.slice(0, limit).map((result) => ({
id: result.id,
url: `https://example.com/book/${result.id}`,
raw: result,
metadata: {
title: result.name,
identifiers: [{ value: result.isbn, scheme: "ISBN" }],
contributors: result.authors.map((name) => ({
name,
roles: ["author"],
})),
},
}))
},
}

The asymmetry is intentional: providers receive the small FetchMetadataInput and return rich ResolvedMetadata candidates. The input exposes only lookup and matching evidence; results retain covers, descriptions, subjects, contributors and every other useful catalog value.

Helpers cover archive conversion and common input checks:

import {
hasSearchTerms,
metadataInputFromResolvedArchive,
} from "@prose-reader/metadata-fetcher"
const input = metadataInputFromResolvedArchive(resolved)
// is there anything to go on at all? publication details or a page count
// narrow a search but cannot start one, so they don't count
if (!hasSearchTerms(input)) return

Rules of thumb:

  • Return an empty list, don't throw, when the input carries nothing you can search on. Throwing is for actual failures — they land in failedProviders.

  • Throw MetadataProviderResponseError when the catalog answers with a failing status, so the status reaches the caller instead of dying in a log line. Any error carrying a numeric status works too, which covers HTTP clients that throw their own:

    import { MetadataProviderResponseError } from "@prose-reader/metadata-fetcher"
    if (!response.ok) {
    throw new MetadataProviderResponseError(response.status, "My Catalog search failed")
    }
  • Use retryWithBackoff when an idempotent provider request has known transient failures. attempts counts the initial call, and cancellation interrupts the next delay. Pass the same signal to the request when it should cancel an active call too. The helper deliberately leaves the retry predicate to the provider. requestedDelayMs lets a failure name its own delay, such as a rate limit's Retry-After, in place of the backoff; a delay longer than maxDelayMs rethrows at once instead of retrying early. parseRetryAfterHeader reads the header, the error carries it as retryAfterMs, and responseErrorRetryAfterMs reads it back from any error carrying one. Cancel the body of a response you do not read: Node's fetch keeps its connection busy until garbage collection, and failed attempts would pile up sockets:

    import {
    MetadataProviderResponseError,
    parseRetryAfterHeader,
    responseErrorRetryAfterMs,
    responseErrorStatus,
    retryWithBackoff,
    } from "@prose-reader/metadata-fetcher"
    const response = await retryWithBackoff(
    async () => {
    const response = await fetch(url, { signal })
    if (!response.ok) {
    await response.body?.cancel()
    throw new MetadataProviderResponseError(response.status, "My Catalog search failed", {
    retryAfterMs: parseRetryAfterHeader(response.headers.get("Retry-After")),
    })
    }
    return response
    },
    {
    attempts: 3,
    initialDelayMs: 1_000,
    maxDelayMs: 5_000,
    signal,
    shouldRetry: (error) => [429, 503].includes(responseErrorStatus(error) ?? 0),
    requestedDelayMs: responseErrorRetryAfterMs,
    },
    )
  • Forward signal to every request you make.

  • Treat limit as a page-size hint; the fetch caps the ranked result anyway.

  • Normalize honestly. Populate a field only when the record actually states it — a fabricated value is a signal the matcher will score, and a wrong ISBN identifier is the single most damaging thing you can emit.

  • raw is yours: put your parsed record there for provenance and provider-specific fields with no home in the vocabulary. It is kept only when the consumer asks for it.

The matching primitives are exported too (scoreMetadataCandidate, titleSimilarity, personNameSimilarity, textSimilarity, normalizeForComparison) if you want to rank something of your own by the same rules — scoreMetadataCandidate(input, candidate) takes a FetchMetadataInput and a rich ResolvedMetadata candidate. Accept by DEFAULT_METADATA_MATCH_MIN_SCORE to draw the line where fetchMetadata draws it by default.