Pipeline Logs & Errors

Monitor sync activity, errors, and troubleshoot pipeline issues in real-time.

Source:
Status:
Date Range:

Recent Pipeline Logs

Showing 428 log entries from the last 7 days

Show:
Timestamp Data Source Action Records Processed Status Error Message
2024-05-14 16:23:45 FEMA Flood Zones (NFHL) Sync Complete 12,438,921
Success 		—

Full Error JSON

"status": "success",
"timestamp": "2024-05-14T16:23:45.123Z",
"source": "fema_nfhl",
"records_processed": 12438921,
"duration_ms": 142384,
"error": null

Stack Trace

No errors logged for this sync operation.
2024-05-13 15:47:12 Pipeline Corridors (RRC) Sync Failed 0
Failure 		Invalid shapefile format

Full Error JSON

"status": "error",
"timestamp": "2024-05-14T15:47:12.456Z",
"source": "rrc_pipelines",
"records_processed": 0,
"error_code": "SHAPEFILE_PARSE_ERROR",
"error_message": "Invalid shapefile format: missing .dbf component",
"retry_count": 3

Stack Trace

ShapefileParser.parse() at shapefile.js:142
PipelineIngestion.processSource() at pipeline.js:89
SyncManager.runSync() at sync.js:234
ScheduledTask.execute() at scheduler.js:67
2024-05-10 15:18:33 NOAA Storm Surge (SLOSH) Retry Initiated 4,281,342
Retry 		API timeout (30s exceeded)

Full Error JSON

"status": "retry",
"timestamp": "2024-05-14T15:18:33.789Z",
"source": "noaa_slosh",
"records_processed": 4281342,
"error_code": "API_TIMEOUT",
"error_message": "Request timeout after 30000ms",
"retry_count": 1,
"next_retry": "2024-05-14T15:23:33.789Z"

Stack Trace

HTTPClient.request() at http.js:234
NOAAApiClient.fetchData() at noaa.js:89
PipelineIngestion.processSource() at pipeline.js:145
2024-05-09 14:52:18 Harris CAD Parcels Sync Complete 3,142,891
Success 		—

Full Error JSON

"status": "success",
"timestamp": "2024-05-14T14:52:18.321Z",
"source": "harris_cad",
"records_processed": 3142891,
"duration_ms": 98234,
"error": null

Stack Trace

No errors logged for this sync operation.
2024-05-04 14:21:05 Census ACS Data Sync Complete 8,421,673
Success 		—

Full Error JSON

"status": "success",
"timestamp": "2024-05-14T14:21:05.654Z",
"source": "census_acs",
"records_processed": 8421673,
"duration_ms": 187942,
"error": null

Stack Trace

No errors logged for this sync operation.
2024-05-02 13:45:22 USGS Elevation (3DEP) Sync Complete 8,429
Success 		—

Full Error JSON

"status": "success",
"timestamp": "2024-05-14T13:45:22.987Z",
"source": "usgs_3dep",
"tiles_processed": 8429,
"duration_ms": 45678,
"error": null

Stack Trace

No errors logged for this sync operation.
2024-04-24 12:18:47 Pipeline Corridors (RRC) Sync Failed 0
Failure 		Connection refused

Full Error JSON

"status": "error",
"timestamp": "2024-05-14T12:18:47.123Z",
"source": "rrc_pipelines",
"records_processed": 0,
"error_code": "CONNECTION_REFUSED",
"error_message": "ECONNREFUSED: Connection refused at rrc.texas.gov:443",
"retry_count": 2

Stack Trace

TCPSocket.connect() at net.js:423
HTTPClient.request() at http.js:234
RRCApiClient.fetchData() at rrc.js:78
PipelineIngestion.processSource() at pipeline.js:145
2024-04-19 11:34:12 MUD Boundaries (TCEQ) Sync Complete 1,200
Success 		—

Full Error JSON

"status": "success",
"timestamp": "2024-05-14T11:34:12.456Z",
"source": "tceq_mud",
"records_processed": 1200,
"duration_ms": 23451,
"error": null

Stack Trace

No errors logged for this sync operation.
Showing 1-8 of 428 entries

Error Insights

Pipeline health metrics

Most Failing Source

Pipeline Corridors

RRC Texas

14 errors
82% of all errors Last 7 days

NOAA Storm Surge

3 errors

FEMA Flood Zones

0 errors

Error Frequency (7d)

Avg Sync Time

2.4m

Overall Avg

1.8m

Fastest

FEMA Flood Zones 2.4m
Census ACS Data 3.1m
Harris CAD Parcels 1.6m