admin/mars
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Refactor export scripts to use DATA_EXPORTER.EXPORT_TABLE_DATA procedure, update folder structure to DATA/C2D, and add file registration validation in README and validation script.

Browse Source
This commit is contained in:
Grzegorz Michalski
2026-02-11 21:22:37 +01:00
parent 641af7415f
commit f8213b76ab
3 changed files with 83 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
MARS_Packages/REL02_POST/MARS-956/README.md
View File

@@ -4,7 +4,7 @@
**Purpose**: One-time export of historical C2D MPEC delta data from operational database (OU_C2D) to DATA bucket as CSV files.
**Approach**: Use FILE_MANAGER export functionality EXPORT_TABLE_DATA_BY_DATE for bulk data movement.
**Approach**: Use DATA_EXPORTER export functionality EXPORT_TABLE_DATA for bulk data movement with file registration.
**Input**: Old tables in OU_C2D operational database
**Output**: CSV files in DATA bucket
@@ -13,18 +13,19 @@
## Tables to Export
| Source Table (OU_C2D) | Target Location (DATA) | Export Type | Time Dependency |
|------------------------|-------------------------|-------------|-----------------|
| `MPEC_ADMIN` | `mrds_data_dev/ODS/C2D/C2D_MPEC_ADMIN` | CSV to DATA | Sync with REL_02 |
| `MPEC_CONTENT` | `mrds_data_dev/ODS/C2D/C2D_MPEC_CONTENT` | CSV to DATA | Sync with REL_02 |
| `MPEC_CONTENT_CRITERION` | `mrds_data_dev/ODS/C2D/C2D_MPEC_CONTENT_CRITERION` | CSV to DATA | Sync with REL_02 |
|------------------------|-------------------------|-------------|------------------|
| `MPEC_ADMIN` | `mrds_data_dev/DATA/C2D/C2D_MPEC_ADMIN` | CSV to DATA | Sync with REL_02 |
| `MPEC_CONTENT` | `mrds_data_dev/DATA/C2D/C2D_MPEC_CONTENT` | CSV to DATA | Sync with REL_02 |
| `MPEC_CONTENT_CRITERION` | `mrds_data_dev/DATA/C2D/C2D_MPEC_CONTENT_CRITERION` | CSV to DATA | Sync with REL_02 |
## Export Strategy
- **Format**: CSV files in DATA bucket
- **Reason**: Complete history of delta records needed for all queries
- **Method**: `DATA_EXPORTER.EXPORT_TABLE_DATA_BY_DATE` procedure
- **Method**: `DATA_EXPORTER.EXPORT_TABLE_DATA` procedure
- **Bucket Area**: `'DATA'`
- **Folder Structure**: `'ODS/C2D/{TABLE_NAME}'`
- **Folder Structure**: `'DATA/C2D/{TABLE_NAME}'`
- **File Registration**: Files registered in A_SOURCE_FILE_RECEIVED table
## Installation Steps
@@ -47,7 +48,7 @@ MARS-956/
## Prerequisites
- OU_C2D schema access for source tables
- DATA_EXPORTER package v2.7.4+ deployed
- DATA_EXPORTER package v2.7.5+ deployed (with pRegisterExport support)
- DEF_CRED_ARN credentials configured
- DATA bucket accessible
@@ -57,6 +58,7 @@ MARS-956/
2. Verify CSV files created in DATA bucket
3. Validate file structure matches template tables
4. Confirm row counts match source tables
5. Check file registration in A_SOURCE_FILE_RECEIVED table
## Notes