admin/mars
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add FILE_ARCHIVER package and enhance A_SOURCE_FILE_CONFIG table

Browse Source 
- Created new FILE_ARCHIVER package (v3.2.1) for managing file archival processes, including procedures for archiving, restoring, and purging files.
- Added versioning and build information to the package with detailed version history.
- Enhanced A_SOURCE_FILE_CONFIG table by adding ARCHIVE_ENABLED and KEEP_IN_TRASH columns to control archival participation and TRASH retention policy.
- Implemented constraints and comments for new columns to ensure data integrity and provide clarity on their usage.
This commit is contained in:
Grzegorz Michalski
2026-02-11 18:24:18 +01:00
parent b96becd8ef
commit 26aba08759
13 changed files with 1792 additions and 89 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
MARS_Packages/REL01_ADDITIONS/MARS-828/install_mars828.sql
View File

@@ -35,9 +35,9 @@ PROMPT =========================================================================
PROMPT MARS-828 Installation Starting
PROMPT ============================================================================
PROMPT Package: CT_MRDS.FILE_ARCHIVER
PROMPT Change: Enhanced archival strategies (MINIMUM_AGE_MONTHS, HYBRID) + TRASH retention control
PROMPT Purpose: Flexible archival policies per data source with file retention management
PROMPT Steps: 9 (DDL, Trigger, Statuses, Package v3.2.0, Verify, Track, Configure)
PROMPT Change: Enhanced archival strategies (MINIMUM_AGE_MONTHS, HYBRID) + TRASH retention + Selective archiving
PROMPT Purpose: Flexible archival policies per data source with file retention and config-based control
PROMPT Steps: 9 (DDL, Trigger, Statuses, Package v3.3.0, Verify, Track, Configure)
PROMPT Timestamp:
SELECT TO_CHAR(SYSDATE, 'YYYY-MM-DD HH24:MI:SS') AS install_start FROM DUAL;
PROMPT ============================================================================
@@ -55,8 +55,8 @@ WHENEVER SQLERROR CONTINUE
-- Installation steps
PROMPT
PROMPT Step 1/9: Adding archival strategy columns to A_SOURCE_FILE_CONFIG
PROMPT ===================================================================
PROMPT Step 1/9: Adding archival strategy and config columns to A_SOURCE_FILE_CONFIG
PROMPT =============================================================================
@@01_MARS_828_install_add_archival_strategy_columns.sql
PROMPT
@@ -70,12 +70,12 @@ PROMPT =====================================================================
@@07_MARS_828_install_add_trash_retention_statuses.sql
PROMPT
PROMPT Step 4/9: Deploying FILE_ARCHIVER Package Specification v3.2.0
PROMPT Step 4/9: Deploying FILE_ARCHIVER Package Specification v3.3.0
PROMPT ================================================================
@@03_MARS_828_install_CT_MRDS_FILE_ARCHIVER_SPEC.sql
PROMPT
PROMPT Step 5/9: Deploying FILE_ARCHIVER Package Body v3.2.0
PROMPT Step 5/9: Deploying FILE_ARCHIVER Package Body v3.3.0
PROMPT ======================================================
@@04_MARS_828_install_CT_MRDS_FILE_ARCHIVER_BODY.sql
@@ -108,21 +108,21 @@ SELECT TO_CHAR(SYSDATE, 'YYYY-MM-DD HH24:MI:SS') AS install_end FROM DUAL;
PROMPT
PROMPT Installation Summary:
PROMPT - Package: CT_MRDS.FILE_ARCHIVER
PROMPT - Version: 3.2.0 (includes TRASH folder retention control)
PROMPT - Version: 3.3.0 (includes selective archiving and config-based TRASH policy)
PROMPT - Strategies: THRESHOLD_BASED (default), MINIMUM_AGE_MONTHS (0=current month), HYBRID
PROMPT - TRASH Retention: pKeepInTrash parameter (DEFAULT TRUE for safety)
PROMPT - Selective Archiving: ARCHIVE_ENABLED column (Y=archive, N=skip)
PROMPT - TRASH Policy: KEEP_IN_TRASH column (Y=keep files, N=delete immediately)
PROMPT * Default: ARCHIVE_ENABLED='Y', KEEP_IN_TRASH='N' (archiving enabled, immediate deletion)
PROMPT * TRASH is a subfolder in DATA bucket (e.g., TRASH/LM/TABLE_NAME)
PROMPT * Files kept in TRASH by default for compliance and rollback capability
PROMPT * No more pKeepInTrash parameter - policy from config only
PROMPT - New Procedure: ARCHIVE_ALL_FOR_SOURCE(pSourceKey) for batch processing
PROMPT - TRASH Management: RESTORE_FILE_FROM_TRASH, PURGE_TRASH_FOLDER (3-level granularity)
PROMPT - New Statuses: ARCHIVED_AND_TRASHED, ARCHIVED_AND_PURGED
PROMPT - Backward Compatible: Yes (default THRESHOLD_BASED + TRASH retention preserved)
PROMPT - Backward Compatible: Yes (default THRESHOLD_BASED, existing behavior preserved)
PROMPT - Configured Tables: 25 Release 01 tables (19 LM + 6 CSDB)
PROMPT - Includes Fixes:
PROMPT * v3.1.1: ORA-01422 for multiple parquet files
PROMPT * v3.1.2: PARTITION_YEAR/PARTITION_MONTH assignments
PROMPT * v3.1.2: Export query circular dependency
PROMPT * v3.2.0: TRASH retention control (pKeepInTrash parameter)
PROMPT - Includes All Fixes from v3.0.0 through v3.2.1
PROMPT
PROMPT Note: Incremental patches (v3.1.0->v3.1.1->v3.1.2->v3.2.0) available in patches/
PROMPT Note: Incremental patches available in patches/ directory
PROMPT
PROMPT Log file: &_filename
PROMPT ============================================================================