Skip to content

API — Getting Started

The API service is a NestJS application that acts as the primary HTTP interface for the entire system. It exposes five functional areas — content management, analytics, real-time event streaming, a fetch trigger, and authentication — all under the /api prefix on port 3001. It reads from the same PostgreSQL database that the consumer writes to, and it produces Kafka messages for two purposes: requeueing content for re-classification and signalling the producer to start a new fetch run.

Prerequisites

  • PostgreSQL — the primary data store, shared with the consumer service. The API connects with TypeORM but runs with synchronize: false, meaning the schema is never auto-managed. Tables must already exist before the service starts, either from a prior migration or from the consumer having run first.
  • Kafka — required for the requeue endpoint, the fetch trigger, and the SSE bridge. The API is configured with allowAutoTopicCreation: false, so all three topics (raw_content_aa, fetch_content_aa, processed_content_aa) must exist before startup.
  • better-auth secret — an arbitrary string used to sign and verify magic link tokens. In production this must be a strong secret; the default dev-secret-change-in-production is intentionally weak.

Configuration keys

  • PORT — HTTP port the service listens on. Defaults to 3001.
  • DB_HOST — PostgreSQL host. Defaults to localhost.
  • DB_PORT — PostgreSQL port. Defaults to 5432.
  • DB_USER — Database user. Defaults to postgres.
  • DB_PASS — Database password. Defaults to postgres.
  • DB_NAME — Database name. Defaults to aan_db.
  • KAFKA_BROKERS — Comma-separated list of broker addresses. Defaults to localhost:9092.
  • BETTER_AUTH_SECRET — Secret used by the better-auth library to sign session tokens. Defaults to dev-secret-change-in-production.
  • BASE_URL — Public base URL of this service, used by better-auth when constructing magic link URLs. Defaults to http://localhost:3001.

What the API exposes

  • /api/auth/* — Handled entirely by the better-auth adapter. Covers magic link request, verification, and session management. No custom controller is involved; the adapter intercepts these routes directly.
  • /api/contents — The main content management surface. Supports paginated listing with optional filters (date range, category, partial title match), fetching a single item along with its full model prediction history, retrieving the pending classification queue, patching the human-assigned category on an item, and requeueing a batch of items back to Kafka.
  • /api/analytics — Six read-only endpoints that aggregate data from the shared database: category distribution across all stored content, daily processed counts over the last 30 days, median model confidence score, a human-vs-model category agreement rate, and separate analytics feeds for the producer and consumer services.
  • /api/trigger — A single POST endpoint that emits a timestamped signal to the fetch_content_aa Kafka topic, instructing the producer to start a new AA API fetch run immediately.
  • /api/events — A Server-Sent Events endpoint. Clients that connect receive processed_content events in real time as the consumer finishes classifying articles.

Authentication notes

Authentication is powered by the better-auth library with the magic link plugin enabled. When a user requests a link, better-auth generates a signed URL. In the current implementation, that URL is printed to stdout rather than sent by email — this is a development-mode stub. The trusted origin is http://localhost:3000, which corresponds to the dashboard. The BETTER_AUTH_SECRET env var must be consistent across restarts since it is used to verify existing sessions.

The content and analytics routes currently have no authentication guards applied. Any client that can reach the service can call them.

Data storage overview

The API does not maintain its own separate schema. It reads from the contents, content_results, consumer_analytics, and producer_analytics tables, all of which are written to by the consumer service. The only writes the API performs are: patching the kategori column on a content row (manual category override) and emitting Kafka messages for requeue and trigger operations. Classification results are never written by the API.

Operational notes & troubleshooting

  • Body parsing — bodyParser is disabled globally at the NestJS application level. This is required so the better-auth adapter can read the raw request stream on auth routes. Controllers that need a JSON body (such as the requeue and category patch endpoints) parse the body manually.
  • CORS — Enabled with credentials: true and origin: true, allowing the dashboard to make authenticated cross-origin requests.
  • Kafka topic auto-creation — Disabled. If a topic does not exist when the service starts, Kafka operations will fail silently or throw. Ensure all three topics are created beforehand.
  • SSE consumer group — The SSE bridge runs as KafkaJS consumer group api-sse-group and subscribes to processed_content_aa from the current offset (not from the beginning). Clients that connect after an event was emitted will not receive that event retroactively.
  • No HTTP exposure on other services — The API is the only service in the stack that exposes an HTTP port. The consumer, producer, and model services communicate exclusively over Kafka and gRPC.