68 lines
2.6 KiB
Markdown
68 lines
2.6 KiB
Markdown
# MARS-956: Exporting Historical data for ODS: C2D MPEC (delta)
|
|
|
|
## Overview
|
|
|
|
**Purpose**: One-time export of historical C2D MPEC delta data from operational database (OU_C2D) to DATA bucket as CSV files.
|
|
|
|
**Approach**: Use DATA_EXPORTER export functionality EXPORT_TABLE_DATA for bulk data movement with file registration.
|
|
|
|
**Input**: Old tables in OU_C2D operational database
|
|
**Output**: CSV files in DATA bucket
|
|
**Mapping**: Structure must match new ODS template tables
|
|
|
|
## Tables to Export
|
|
|
|
| Source Table (OU_C2D) | Target Location (DATA) | Export Type | Time Dependency |
|
|
|------------------------|-------------------------|-------------|------------------|
|
|
| `MPEC_ADMIN` | `mrds_data_dev/DATA/C2D/C2D_MPEC_ADMIN` | CSV to DATA | Sync with REL_02 |
|
|
| `MPEC_CONTENT` | `mrds_data_dev/DATA/C2D/C2D_MPEC_CONTENT` | CSV to DATA | Sync with REL_02 |
|
|
| `MPEC_CONTENT_CRITERION` | `mrds_data_dev/DATA/C2D/C2D_MPEC_CONTENT_CRITERION` | CSV to DATA | Sync with REL_02 |
|
|
|
|
## Export Strategy
|
|
|
|
- **Format**: CSV files in DATA bucket
|
|
- **Reason**: Complete history of delta records needed for all queries
|
|
- **Method**: `DATA_EXPORTER.EXPORT_TABLE_DATA` procedure
|
|
- **Bucket Area**: `'DATA'`
|
|
- **Folder Structure**: `'DATA/C2D/{TABLE_NAME}'`
|
|
- **File Registration**: Files registered in A_SOURCE_FILE_RECEIVED table
|
|
|
|
## Installation Steps
|
|
|
|
1. Run master install script: `@install_mars956.sql`
|
|
2. Verify exports completed successfully
|
|
3. Confirm CSV files created in DATA bucket with expected structure
|
|
|
|
## Files Structure
|
|
|
|
```
|
|
MARS-956/
|
|
├── README.md # This file
|
|
├── install_mars956.sql # Master installation script
|
|
├── 01_MARS_956_export_c2d_mpec_data.sql # Export procedures execution
|
|
├── track_package_versions.sql # Universal version tracking
|
|
├── verify_packages_version.sql # Universal version verification
|
|
└── rollback_mars956.sql # Rollback script (if needed)
|
|
```
|
|
|
|
## Prerequisites
|
|
|
|
- OU_C2D schema access for source tables
|
|
- DATA_EXPORTER package v2.7.5+ deployed (with pRegisterExport support)
|
|
- DEF_CRED_ARN credentials configured
|
|
- DATA bucket accessible
|
|
|
|
## Post-Installation Verification
|
|
|
|
1. Check export completion in A_PROCESS_LOG
|
|
2. Verify CSV files created in DATA bucket
|
|
3. Validate file structure matches template tables
|
|
4. Confirm row counts match source tables
|
|
5. Check file registration in A_SOURCE_FILE_RECEIVED table
|
|
|
|
## Notes
|
|
|
|
- This is a **one-time** data migration
|
|
- No package modifications required (uses existing DATA_EXPORTER)
|
|
- Export timing critical - must sync with REL_02 deployment
|
|
- Complete history required for delta queries |