- Introduced CT_MRDS.DATA_EXPORTER package to facilitate data exports in CSV and Parquet formats.
- Implemented support for cloud storage integration via Oracle Cloud Infrastructure (OCI).
- Added versioning and detailed version history for tracking changes and improvements.
- Included main export procedures: EXPORT_TABLE_DATA, EXPORT_TABLE_DATA_BY_DATE, and EXPORT_TABLE_DATA_TO_CSV_BY_DATE.
- Enhanced parallel processing capabilities for improved performance during data exports.
- Dropped A_WORKFLOW_HISTORY_KEY column from A_SOURCE_FILE_RECEIVED with data preservation.
- Removed unnecessary checks for column existence during rollback.
- Updated A_SOURCE_FILE_CONFIG, A_TABLE_STAT, and A_TABLE_STAT_HIST to their pre-MARS-1409 structures, excluding new columns added in MARS-1409.
- Adjusted FILE_ARCHIVER package to reflect changes in statistics handling and archival triggers.
- Revised rollback script to ensure proper order of operations for restoring previous versions of packages and tables.
- Bump package version to 3.6.3 and update build date.
- Add new parameters: pIsArchiveEnabled, pIsKeepInTrash, pArchivalStrategy, pMinimumAgeMonths to ADD_SOURCE_FILE_CONFIG.
- Include pIsWorkflowSuccessRequired parameter to control workflow success requirement for archival.
- Update version history to reflect changes.
feat(A_SOURCE_FILE_CONFIG): Modify table structure to include new archival control flags
- Add IS_WORKFLOW_SUCCESS_REQUIRED column to A_SOURCE_FILE_CONFIG for workflow bypass functionality.
- Update constraints and comments for new columns.
- Ensure backward compatibility with default values.
fix(A_TABLE_STAT, A_TABLE_STAT_HIST): Extend table structures to accommodate new workflow success tracking
- Add IS_WORKFLOW_SUCCESS_REQUIRED column to both A_TABLE_STAT and A_TABLE_STAT_HIST.
- Update comments to clarify the purpose of new columns.
docs(FILE_ARCHIVER_Guide): Revise documentation to reflect new archival features and configurations
- Document new IS_WORKFLOW_SUCCESS_REQUIRED flag and its implications for archival processes.
- Update examples and configurations to align with recent changes in the database schema.
- Ensure clarity on archival strategies and their configurations.
- Added .gitignore to exclude temporary folders.
- Created SQL script to update existing A_WORKFLOW_HISTORY_KEY in A_SOURCE_FILE_RECEIVED.
- Implemented rollback script to clear backfilled A_WORKFLOW_HISTORY_KEY values.
- Added README.md for installation and usage instructions.
- Developed master installation and rollback scripts for MARS-1409 post-hook.
- Verified installation and rollback processes with detailed checks.
- Updated trigger logic to manage workflow history updates.
- Ensured proper version tracking and verification for related packages.
- Added checks for existing columns before adding or dropping A_WORKFLOW_HISTORY_KEY in relevant scripts to prevent errors.
- Updated rollback scripts to ensure proper restoration of previous states, including recompilation of dependent packages.
- Introduced a diagnostic script to assess the status of workflow keys against ODS tables, providing detailed reporting on discrepancies.
- Adjusted trigger definitions to accommodate new workflow names and ensure correct handling of workflow history.
- Modified master rollback script to streamline the rollback process and improve clarity in step descriptions.
- Added script to delete exported CSV files from DATA bucket.
- Enhanced rollback file registrations script with improved error handling and success messages.
- Updated process logs cleanup script to include error handling and success messages.
- Modified rollback verification script for clearer success and warning messages.
- Removed outdated README.md file and updated installation script to reflect schema changes.
- Adjusted rollback master script to streamline execution steps and improve clarity.
- Updated README.md to enhance clarity on the one-time export process, including detailed prerequisites, data structure, and validation steps.
- Revised install_mars956.sql to streamline installation commands and improve logging mechanisms.
- Modified rollback_mars956.sql to include user confirmation before proceeding with deletions and to clarify rollback steps.
- Removed unnecessary tracking and validation scripts (track_package_versions.sql, validate_export.sql, verify_packages_version.sql) to simplify the package structure.
- Enhanced error handling and validation checks throughout the installation and rollback processes.
