Data Ingestion Platform

Compliant crawling, data quality, and external APIs in one platform.

Review sources, respect robots rules, preview schedules, simulate crawling, deduplicate records, and manage Elasticsearch templates through a single official site, API layer, and web console.

Core Delivery

Three entry points, one platform

The public introduction pages, external API guide, and React web system are deployed under the same IIS site and subdomain for a clear user and integration experience.

External API integration dashboard

External API integration

Use health, catalog, source, job, run, simulate, parser rule, quality, and compliance endpoints with consistent response envelopes.

Data governance pipeline

Data governance workflow

Connect source review, legal checks, schedule preview, deduplication, and quality rules into an auditable operating flow.

Web console dashboard

Web system

The React console gives operators and engineers shared views for sources, jobs, runs, alerts, and Elasticsearch templates.

Public APIs12+Status, catalog, simulation, and governance
ES Templates10Tenant, source, job, run, and record indexes
Workflows6Review, schedule, crawl, clean, alert, audit
Single Subdomain1Website, API, docs, and web console

Central Member Auth

Member login is centralized at loginapi.sun-bd.com. This project stores no member credentials.

After users sign in through the central member API, this platform only consumes the returned access token. The API validates RS256 signatures and access token type through JWKS. Local registration, login, logout, password changes, and profile editing are not implemented in this project.

  • Protected platform APIs require Authorization: Bearer accessToken from the central member API.
  • React and WinForms clients use only the returned token and do not store user names or passwords.
  • Refresh, logout, and permission reads are handled through the central member integration flow.
Platform console after central member authentication