admin/mars
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Refactor MARS-1005 scripts for OU_TOP legacy data export and rollback

Browse Source 
- Updated SQL scripts to verify data integrity for 6 OU_TOP.LEGACY_* tables instead of 3 C2D MPEC tables.
- Modified rollback script to delete exported CSV files from ODS/TOP/ bucket paths.
- Enhanced verification script to check for remaining files and cloud bucket contents specific to MARS-1005.
- Adjusted install script to reflect changes in target tables and their corresponding paths in the ODS bucket.
- Updated README to include instructions for the new MARS-1005 installation and rollback processes.
This commit is contained in:
Grzegorz Michalski
2026-03-06 14:34:12 +01:00
parent 73e99b6e76
commit b81e524351
6 changed files with 423 additions and 343 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
MARS_Packages/REL03/MARS-1005/99_MARS_1005_verify_rollback.sql
View File

@@ -20,7 +20,7 @@ BEGIN
SELECT COUNT(*)
INTO vRemainingFiles
FROM CT_MRDS.A_SOURCE_FILE_RECEIVED
WHERE SOURCE_FILE_NAME LIKE '200%' -- ETL keys 2001-2010
WHERE PROCESS_NAME = 'MARS-1005'
AND RECEPTION_DATE >= SYSDATE - 7; -- Last week
IF vRemainingFiles = 0 THEN
@@ -33,7 +33,7 @@ BEGIN
SELECT SUBSTR(SOURCE_FILE_NAME, 1, 50) AS FILE_NAME,
TO_CHAR(RECEPTION_DATE, 'YYYY-MM-DD HH24:MI:SS') AS RECEIVED_TIME
FROM CT_MRDS.A_SOURCE_FILE_RECEIVED
WHERE SOURCE_FILE_NAME LIKE '200%'
WHERE PROCESS_NAME = 'MARS-1005'
AND RECEPTION_DATE >= SYSDATE - 7
) LOOP
DBMS_OUTPUT.PUT_LINE(' Remaining: ' || rec.FILE_NAME);
@@ -96,7 +96,7 @@ BEGIN
credential_name => vCredentialName,
location_uri => vDataBucketUri
))
WHERE object_name LIKE 'ODS/C2D/C2D_MPEC_%'
WHERE object_name LIKE 'ODS/TOP/%'
) LOOP
vCloudFileCount := vCloudFileCount + 1;
IF vCloudFileCount <= 3 THEN -- Show first 3 files
@@ -105,11 +105,11 @@ BEGIN
END LOOP;
IF vCloudFileCount = 0 THEN
DBMS_OUTPUT.PUT_LINE('SUCCESS: No C2D MPEC files found in cloud bucket');
DBMS_OUTPUT.PUT_LINE('SUCCESS: No TOP files found in cloud bucket - clean');
ELSE
DBMS_OUTPUT.PUT_LINE('INFO: ' || vCloudFileCount || ' C2D MPEC files still in cloud bucket');
DBMS_OUTPUT.PUT_LINE('INFO: ' || vCloudFileCount || ' TOP file(s) still in cloud bucket');
DBMS_OUTPUT.PUT_LINE(' Note: Cloud files are not automatically deleted by rollback');
DBMS_OUTPUT.PUT_LINE(' Manual deletion required if needed');
DBMS_OUTPUT.PUT_LINE(' Run 90_MARS_1005_rollback_delete_csv_files.sql to remove them');
END IF;
EXCEPTION
@@ -167,7 +167,7 @@ BEGIN
SELECT COUNT(*)
INTO vRemainingFiles
FROM CT_MRDS.A_SOURCE_FILE_RECEIVED
WHERE SOURCE_FILE_NAME LIKE '200%'
WHERE PROCESS_NAME = 'MARS-1005'
AND RECEPTION_DATE >= SYSDATE - 7;
-- Count remaining process logs