MARS-956: Exporting Historical data for ODS: C2D MPEC (delta)

Overview

Purpose: One-time export of historical C2D MPEC delta data from operational database (OU_C2D) to DATA bucket as CSV files.

Approach: Use DATA_EXPORTER export functionality EXPORT_TABLE_DATA for bulk data movement with file registration.

Input: Old tables in OU_C2D operational database
Output: CSV files in DATA bucket
Mapping: Structure must match new ODS template tables

Tables to Export

Source Table (OU_C2D)	Target Location (DATA)	Export Type	Time Dependency
`MPEC_ADMIN`	`mrds_data_dev/DATA/C2D/C2D_MPEC_ADMIN`	CSV to DATA	Sync with REL_02
`MPEC_CONTENT`	`mrds_data_dev/DATA/C2D/C2D_MPEC_CONTENT`	CSV to DATA	Sync with REL_02
`MPEC_CONTENT_CRITERION`	`mrds_data_dev/DATA/C2D/C2D_MPEC_CONTENT_CRITERION`	CSV to DATA	Sync with REL_02

Export Strategy

Format: CSV files in DATA bucket
Reason: Complete history of delta records needed for all queries
Method: DATA_EXPORTER.EXPORT_TABLE_DATA procedure
Bucket Area: 'DATA'
Folder Structure: 'DATA/C2D/{TABLE_NAME}'
File Registration: Files registered in A_SOURCE_FILE_RECEIVED table

Installation Steps

Run master install script: @install_mars956.sql
Verify exports completed successfully
Confirm CSV files created in DATA bucket with expected structure

Files Structure

MARS-956/
├── README.md                           # This file
├── install_mars956.sql                 # Master installation script
├── 01_MARS_956_export_c2d_mpec_data.sql # Export procedures execution
├── track_package_versions.sql          # Universal version tracking
├── verify_packages_version.sql         # Universal version verification
└── rollback_mars956.sql               # Rollback script (if needed)

Prerequisites

OU_C2D schema access for source tables
DATA_EXPORTER package v2.7.5+ deployed (with pRegisterExport support)
DEF_CRED_ARN credentials configured
DATA bucket accessible

Post-Installation Verification

Check export completion in A_PROCESS_LOG
Verify CSV files created in DATA bucket
Validate file structure matches template tables
Confirm row counts match source tables
Check file registration in A_SOURCE_FILE_RECEIVED table

Notes

This is a one-time data migration
No package modifications required (uses existing DATA_EXPORTER)
Export timing critical - must sync with REL_02 deployment
Complete history required for delta queries

2.6 KiB Raw Blame History