Commit Graph

127 Commits

Author SHA1 Message Date
Grzegorz Michalski
a9a51f19be Rename ARCH_FILE_NAME to ARCH_PATH in FILE_ARCHIVER for clarity and consistency in directory URI handling 2026-02-16 11:01:01 +01:00
Grzegorz Michalski
fc6304c60b wk 2026-02-16 10:50:38 +01:00
Grzegorz Michalski
912b7a6466 Remove recent export activity logging from C2D MPEC historical data export script 2026-02-13 21:06:15 +01:00
Grzegorz Michalski
b6f6446232 Add author information to MARS-828 installation script 2026-02-13 21:05:09 +01:00
Grzegorz Michalski
aa2f2f13f9 Update author information to Grzegorz Michalski in MARS-835 SQL scripts 2026-02-13 20:16:05 +01:00
Grzegorz Michalski
5c77d42d9a Update author information to Grzegorz Michalski in MARS-956 SQL scripts and remove pre-check script from installation process 2026-02-13 19:42:02 +01:00
Grzegorz Michalski
d370b9c9ef Update DATA_EXPORTER package to version 2.10.0 with multi-file support for DBMS_CLOUD.EXPORT_DATA, preventing orphaned files in rollback processes. 2026-02-13 19:08:54 +01:00
Grzegorz Michalski
c5d39fc01d Add ENV_MANAGER package with comprehensive error handling and logging features
- Introduced ENV_MANAGER package for managing environment configurations and error handling.
- Implemented detailed error codes and messages for various scenarios, enhancing debugging capabilities.
- Added logging procedures to track process events and errors, with configurable logging levels.
- Included version management functions to track package versioning and changes.
- Established a structure for documenting functions and procedures for better maintainability.
2026-02-13 13:56:36 +01:00
Grzegorz Michalski
80916ea302 Refactor SQL scripts to replace MRDS_LOADER.cloud_wrapper.list_objects with DBMS_CLOUD.LIST_OBJECTS for improved compatibility 2026-02-13 12:39:03 +01:00
Grzegorz Michalski
a4ab56e96d Enhance MARS-835 rollback process with new scripts for file registration deletion and manual cleanup, and add process tracking identifier for exports 2026-02-13 12:36:45 +01:00
Grzegorz Michalski
92c5261215 Revert "Add process tracking identifier to MARS-835 export and rollback scripts"
This reverts commit 2c40f091e0.
2026-02-13 09:45:49 +01:00
Grzegorz Michalski
b34c942a8f Revert "Enhance MARS-826 Rollback Scripts and Export Procedures"
This reverts commit d6c34085f7.
2026-02-13 09:24:02 +01:00
Grzegorz Michalski
d175179ddc Update DATA_EXPORTER package version to 2.9.0 and enhance version history for process tracking improvements 2026-02-13 09:21:06 +01:00
Grzegorz Michalski
d8a5edc3b1 Update step prompts for MARS-835-PREHOOK installation script to reflect correct sequence of operations 2026-02-13 09:00:42 +01:00
Grzegorz Michalski
d6c34085f7 Enhance MARS-826 Rollback Scripts and Export Procedures
- Added parameters `pRegisterExport` and `pProcessName` to export procedures for better tracking.
- Created a new rollback script `90_MARS_826_rollback_file_registrations.sql` to delete file registration records from `A_SOURCE_FILE_RECEIVED`.
- Updated existing rollback scripts for ADHOC_ADJ, BALANCESHEET, CSM_ADJ, STANDING_FACILITY, MRR_IND_CURRENT_ACCOUNT, FORECAST, QR_ADJ, and TTS tables to improve file deletion logic and output messages.
- Ensured all rollback scripts provide informative output regarding the number of files deleted or skipped.
2026-02-13 09:00:13 +01:00
Grzegorz Michalski
2c40f091e0 Add process tracking identifier to MARS-835 export and rollback scripts 2026-02-13 08:50:56 +01:00
Grzegorz Michalski
255044f23b Enhance MARS-956 export and rollback scripts with process tracking identifier 2026-02-13 08:39:39 +01:00
Grzegorz Michalski
0053ecc556 Add MARS-835-PREHOOK scripts to update and rollback A_SOURCE_FILE_RECEIVED table structure 2026-02-13 08:39:03 +01:00
Grzegorz Michalski
3b7ce7de97 Implement rollback functionality for MARS-956:
- Added script to delete exported CSV files from DATA bucket.
- Enhanced rollback file registrations script with improved error handling and success messages.
- Updated process logs cleanup script to include error handling and success messages.
- Modified rollback verification script for clearer success and warning messages.
- Removed outdated README.md file and updated installation script to reflect schema changes.
- Adjusted rollback master script to streamline execution steps and improve clarity.
2026-02-13 07:50:54 +01:00
Grzegorz Michalski
a3ec31341b Update DATA_EXPORTER package to version 2.8.1, fix query in EXPORT_TABLE_DATA to ensure single file output, and update version history. 2026-02-12 16:31:00 +01:00
Grzegorz Michalski
6039d6bce4 Update DATA_EXPORTER package to version 2.8.0, refactor EXPORT_TABLE_DATA to export to a single CSV file, and add pFileName and pMaxFileSize parameters. 2026-02-12 09:56:57 +01:00
Grzegorz Michalski
a7d286b1e6 Refactor MARS-956 package for C2D MPEC data export
- Updated README.md to enhance clarity on the one-time export process, including detailed prerequisites, data structure, and validation steps.
- Revised install_mars956.sql to streamline installation commands and improve logging mechanisms.
- Modified rollback_mars956.sql to include user confirmation before proceeding with deletions and to clarify rollback steps.
- Removed unnecessary tracking and validation scripts (track_package_versions.sql, validate_export.sql, verify_packages_version.sql) to simplify the package structure.
- Enhanced error handling and validation checks throughout the installation and rollback processes.
2026-02-12 09:56:37 +01:00
Grzegorz Michalski
f2bcdb3a76 ignore 2026-02-12 09:28:54 +01:00
Grzegorz Michalski
15dafbff2a zachowanie wersj 2.7.5 2026-02-12 09:27:57 +01:00
Grzegorz Michalski
3261aa1a6d wk 2026-02-11 23:05:51 +01:00
Grzegorz Michalski
1327410880 Fix URI construction in ARCHIVE_TABLE_DATA procedure to include 'ARCHIVE' directory for correct file path. 2026-02-11 21:53:31 +01:00
Grzegorz Michalski
f4e9fb9cb2 Merge branch 'develop' 2026-02-11 21:32:47 +01:00
Grzegorz Michalski
f8213b76ab Refactor export scripts to use DATA_EXPORTER.EXPORT_TABLE_DATA procedure, update folder structure to DATA/C2D, and add file registration validation in README and validation script. 2026-02-11 21:22:37 +01:00
Grzegorz Michalski
641af7415f Update DATA_EXPORTER package to version 2.7.5, adding pRegisterExport parameter to EXPORT_TABLE_DATA procedure for file registration in A_SOURCE_FILE_RECEIVED. 2026-02-11 20:32:21 +01:00
Grzegorz Michalski
c1c3890a1a pierwsza wersja paczki MARS-956 2026-02-11 20:24:45 +01:00
Grzegorz Michalski
63ed05930e Enhance EXPORT_TABLE_DATA procedure by adding pTemplateTableName parameter for flexible column order and date formatting. Update version history for clarity on recent changes. 2026-02-11 19:46:28 +01:00
Grzegorz Michalski
5320db627b przygotowanie pod nowe zmiany 2026-02-11 19:04:06 +01:00
Grzegorz Michalski
3171ff2ddf Update DATA_EXPORTER package to version 2.7.4, enhancing filename storage and metadata retrieval logic for exported files. 2026-02-11 18:27:44 +01:00
Grzegorz Michalski
26aba08759 Add FILE_ARCHIVER package and enhance A_SOURCE_FILE_CONFIG table
- Created new FILE_ARCHIVER package (v3.2.1) for managing file archival processes, including procedures for archiving, restoring, and purging files.
- Added versioning and build information to the package with detailed version history.
- Enhanced A_SOURCE_FILE_CONFIG table by adding ARCHIVE_ENABLED and KEEP_IN_TRASH columns to control archival participation and TRASH retention policy.
- Implemented constraints and comments for new columns to ensure data integrity and provide clarity on their usage.
2026-02-11 18:24:18 +01:00
Grzegorz Michalski
d237c2d7aa pRegisterExport 2026-02-11 09:54:04 +01:00
Grzegorz Michalski
96e3e2f845 pRegisterExport 2026-02-11 09:53:53 +01:00
Grzegorz Michalski
3eb39091de dodanie: pRegisterExport => TRUE 2026-02-11 09:49:28 +01:00
Grzegorz Michalski
113b9f55e3 deprecated version 2.8.0 2026-02-11 09:45:11 +01:00
Grzegorz Michalski
b96becd8ef Add autonomous transaction pragma to ARCHIVE_TABLE_DATA and GATHER_TABLE_STAT procedures for improved transaction handling 2026-02-11 07:36:52 +01:00
Grzegorz Michalski
b1bb24e487 Refactor file retrieval logic in ARCHIVE_TABLE_DATA and GATHER_TABLE_STAT procedures to simplify queries and enhance clarity 2026-02-10 14:03:52 +01:00
Grzegorz Michalski
86535048fc Add TRASH folder management procedures for restoring and purging files 2026-02-10 13:53:12 +01:00
Grzegorz Michalski
9ffad718fe Update FILE_ARCHIVER package to version 3.2.1, fixing status update from ARCHIVED to ARCHIVED_AND_TRASHED for files moved to TRASH folder 2026-02-10 13:18:17 +01:00
Grzegorz Michalski
6c8b22eac9 Add DATA_EXPORTER package and update installation scripts for export registration
- Created new package CT_MRDS.DATA_EXPORTER (v2.6.3) for comprehensive data export capabilities, including CSV and Parquet formats with OCI integration.
- Implemented version history tracking and enhanced error handling for export processes.
- Updated installation scripts to include pRegisterExport parameter for registering exports in A_WORKFLOW_HISTORY and A_SOURCE_FILE_RECEIVED.
2026-02-10 09:33:46 +01:00
Grzegorz Michalski
d443c4f07d Update FILE_ARCHIVER package to version 3.2.0 with TRASH retention control and new statuses 2026-02-10 08:23:06 +01:00
Grzegorz Michalski
cdd9dff32d aktualizacja dokumentacji w związku z TRASH i nowymi statusami plików. 2026-02-10 08:21:51 +01:00
Grzegorz Michalski
70909ba8c4 Enhance FILE_ARCHIVER package to include ODS bucket URI retrieval and update insert statements for A_TABLE_STAT_HIST and A_TABLE_STAT 2026-02-09 13:42:50 +01:00
Grzegorz Michalski
88325f0b8c Refactor FILE_ARCHIVER package to use CT_MRDS.ENV_MANAGER consistently for logging and parameter formatting 2026-02-09 11:04:33 +01:00
Grzegorz Michalski
dc8031262c konfiguracja 2026-02-09 10:22:38 +01:00
Grzegorz Michalski
fc95b12c9e Remove test scenarios for archival strategies from test_archival_strategies.sql 2026-02-09 09:40:31 +01:00
Grzegorz Michalski
23b9962128 Update ARCHIVE_TABLE_DATA procedure to use GROUP BY instead of DISTINCT to resolve ORA-22950 issue 2026-02-09 09:36:05 +01:00