feat(MARS-1057): Upgrade FILE_MANAGER to v3.5.0 with area filter functionality
- Updated install script to reflect new version and features. - Added pArea parameter to CREATE_EXTERNAL_TABLES_SET and CREATE_EXTERNAL_TABLES_BATCH procedures for selective table creation (INBOX/ODS/ARCHIVE/ALL). - Enhanced grant preservation logic based on the pArea parameter. - Updated FILE_MANAGER_ODS package to support new functionality. - Modified rollback script to remove new features and revert to previous versions.
This commit is contained in:
@@ -1,16 +1,16 @@
|
|||||||
--=============================================================================================================================
|
--=============================================================================================================================
|
||||||
-- MARS-1057: Install FILE_MANAGER Package Specification v3.4.0
|
-- MARS-1057: Install FILE_MANAGER Package Specification v3.5.0
|
||||||
--=============================================================================================================================
|
--=============================================================================================================================
|
||||||
-- Purpose: Deploy FILE_MANAGER package specification with new batch external table creation procedures
|
-- Purpose: Deploy FILE_MANAGER package specification with batch external table creation and area filter functionality
|
||||||
-- Author: Grzegorz Michalski
|
-- Author: Grzegorz Michalski
|
||||||
-- Date: 2025-11-27
|
-- Date: 2026-02-18
|
||||||
-- Related: MARS-1057 Batch External Table Creation
|
-- Related: MARS-1057 Batch External Table Creation + Area Filter Enhancement
|
||||||
--=============================================================================================================================
|
--=============================================================================================================================
|
||||||
|
|
||||||
SET SERVEROUTPUT ON
|
SET SERVEROUTPUT ON
|
||||||
|
|
||||||
PROMPT ========================================================================
|
PROMPT ========================================================================
|
||||||
PROMPT Installing FILE_MANAGER Package Specification v3.4.0
|
PROMPT Installing FILE_MANAGER Package Specification v3.5.0
|
||||||
PROMPT ========================================================================
|
PROMPT ========================================================================
|
||||||
|
|
||||||
@@new_version/FILE_MANAGER.pkg
|
@@new_version/FILE_MANAGER.pkg
|
||||||
|
|||||||
@@ -1,16 +1,16 @@
|
|||||||
--=============================================================================================================================
|
--=============================================================================================================================
|
||||||
-- MARS-1057: Install FILE_MANAGER Package Body v3.4.0
|
-- MARS-1057: Install FILE_MANAGER Package Body v3.5.0
|
||||||
--=============================================================================================================================
|
--=============================================================================================================================
|
||||||
-- Purpose: Deploy FILE_MANAGER package body with implementation of batch external table creation procedures
|
-- Purpose: Deploy FILE_MANAGER package body with batch external table creation and area filter functionality
|
||||||
-- Author: Grzegorz Michalski
|
-- Author: Grzegorz Michalski
|
||||||
-- Date: 2025-11-27
|
-- Date: 2026-02-18
|
||||||
-- Related: MARS-1057 Batch External Table Creation
|
-- Related: MARS-1057 Batch External Table Creation + Area Filter Enhancement
|
||||||
--=============================================================================================================================
|
--=============================================================================================================================
|
||||||
|
|
||||||
SET SERVEROUTPUT ON
|
SET SERVEROUTPUT ON
|
||||||
|
|
||||||
PROMPT ========================================================================
|
PROMPT ========================================================================
|
||||||
PROMPT Installing FILE_MANAGER Package Body v3.4.0
|
PROMPT Installing FILE_MANAGER Package Body v3.5.0
|
||||||
PROMPT ========================================================================
|
PROMPT ========================================================================
|
||||||
|
|
||||||
@@new_version/FILE_MANAGER.pkb
|
@@new_version/FILE_MANAGER.pkb
|
||||||
|
|||||||
@@ -1,17 +1,17 @@
|
|||||||
-- ===================================================================
|
-- ===================================================================
|
||||||
-- MARS-1057: Install ODS.FILE_MANAGER_ODS Package Specification
|
-- MARS-1057: Install ODS.FILE_MANAGER_ODS Package Specification
|
||||||
-- ===================================================================
|
-- ===================================================================
|
||||||
-- Purpose: Deploy FILE_MANAGER_ODS package specification v2.2.0
|
-- Purpose: Deploy FILE_MANAGER_ODS package specification v2.4.0
|
||||||
-- Author: Grzegorz Michalski
|
-- Author: Grzegorz Michalski
|
||||||
-- Date: 2025-11-27
|
-- Date: 2026-02-18
|
||||||
-- Package: ODS.FILE_MANAGER_ODS
|
-- Package: ODS.FILE_MANAGER_ODS
|
||||||
-- Version: 2.1.0 -> 2.2.0
|
-- Version: 2.1.0 -> 2.4.0
|
||||||
-- Changes: Added CREATE_EXTERNAL_TABLES_SET and CREATE_EXTERNAL_TABLES_BATCH wrappers
|
-- Changes: Added CREATE_EXTERNAL_TABLES_SET and CREATE_EXTERNAL_TABLES_BATCH wrappers with pArea and pRestoreGrants parameters
|
||||||
|
|
||||||
SET ECHO ON
|
SET ECHO ON
|
||||||
SET DEFINE OFF
|
SET DEFINE OFF
|
||||||
|
|
||||||
PROMPT Installing ODS.FILE_MANAGER_ODS Package Specification v2.2.0...
|
PROMPT Installing ODS.FILE_MANAGER_ODS Package Specification v2.4.0...
|
||||||
|
|
||||||
@@new_version/FILE_MANAGER_ODS.pkg
|
@@new_version/FILE_MANAGER_ODS.pkg
|
||||||
|
|
||||||
|
|||||||
@@ -1,17 +1,17 @@
|
|||||||
-- ===================================================================
|
-- ===================================================================
|
||||||
-- MARS-1057: Install ODS.FILE_MANAGER_ODS Package Body
|
-- MARS-1057: Install ODS.FILE_MANAGER_ODS Package Body
|
||||||
-- ===================================================================
|
-- ===================================================================
|
||||||
-- Purpose: Deploy FILE_MANAGER_ODS package body v2.2.0
|
-- Purpose: Deploy FILE_MANAGER_ODS package body v2.4.0
|
||||||
-- Author: Grzegorz Michalski
|
-- Author: Grzegorz Michalski
|
||||||
-- Date: 2025-11-27
|
-- Date: 2026-02-18
|
||||||
-- Package: ODS.FILE_MANAGER_ODS
|
-- Package: ODS.FILE_MANAGER_ODS
|
||||||
-- Version: 2.1.0 -> 2.2.0
|
-- Version: 2.1.0 -> 2.4.0
|
||||||
-- Changes: Added CREATE_EXTERNAL_TABLES_SET and CREATE_EXTERNAL_TABLES_BATCH wrapper implementations
|
-- Changes: Added CREATE_EXTERNAL_TABLES_SET and CREATE_EXTERNAL_TABLES_BATCH wrapper implementations with pArea and pRestoreGrants parameters
|
||||||
|
|
||||||
SET ECHO ON
|
SET ECHO ON
|
||||||
SET DEFINE OFF
|
SET DEFINE OFF
|
||||||
|
|
||||||
PROMPT Installing ODS.FILE_MANAGER_ODS Package Body v2.2.0...
|
PROMPT Installing ODS.FILE_MANAGER_ODS Package Body v2.4.0...
|
||||||
|
|
||||||
@@new_version/FILE_MANAGER_ODS.pkb
|
@@new_version/FILE_MANAGER_ODS.pkb
|
||||||
|
|
||||||
|
|||||||
@@ -1,19 +1,19 @@
|
|||||||
--=============================================================================================================================
|
--=============================================================================================================================
|
||||||
-- MARS-1057: Rollback FILE_MANAGER Package Specification to v3.3.0
|
-- MARS-1057: Rollback FILE_MANAGER Package Specification to v3.3.0
|
||||||
--=============================================================================================================================
|
--=============================================================================================================================
|
||||||
-- Purpose: Restore FILE_MANAGER package specification to version before MARS-1057 changes
|
-- Purpose: Restore FILE_MANAGER package specification to version before MARS-1057 changes (from v3.5.0 to v3.3.0)
|
||||||
-- Author: Grzegorz Michalski
|
-- Author: Grzegorz Michalski
|
||||||
-- Date: 2025-11-27
|
-- Date: 2026-02-18
|
||||||
-- Related: MARS-1057 Batch External Table Creation (ROLLBACK)
|
-- Related: MARS-1057 Batch External Table Creation + Area Filter (ROLLBACK)
|
||||||
--=============================================================================================================================
|
--=============================================================================================================================
|
||||||
|
|
||||||
SET SERVEROUTPUT ON
|
SET SERVEROUTPUT ON
|
||||||
|
|
||||||
PROMPT ========================================================================
|
PROMPT ========================================================================
|
||||||
PROMPT Rolling back FILE_MANAGER Package Specification to v3.3.0
|
PROMPT Rolling back FILE_MANAGER Package Specification from v3.5.0 to v3.3.0
|
||||||
PROMPT ========================================================================
|
PROMPT ========================================================================
|
||||||
|
|
||||||
@@current_version/FILE_MANAGER.pkg
|
@@rollback_version/FILE_MANAGER.pkg
|
||||||
|
|
||||||
-- Verify compilation status (check specific schema when installing as ADMIN)
|
-- Verify compilation status (check specific schema when installing as ADMIN)
|
||||||
SELECT object_name, object_type, status
|
SELECT object_name, object_type, status
|
||||||
|
|||||||
@@ -1,19 +1,19 @@
|
|||||||
--=============================================================================================================================
|
--=============================================================================================================================
|
||||||
-- MARS-1057: Rollback FILE_MANAGER Package Body to v3.3.0
|
-- MARS-1057: Rollback FILE_MANAGER Package Body to v3.3.0
|
||||||
--=============================================================================================================================
|
--=============================================================================================================================
|
||||||
-- Purpose: Restore FILE_MANAGER package body to version before MARS-1057 changes
|
-- Purpose: Restore FILE_MANAGER package body to version before MARS-1057 changes (from v3.5.0 to v3.3.0)
|
||||||
-- Author: Grzegorz Michalski
|
-- Author: Grzegorz Michalski
|
||||||
-- Date: 2025-11-27
|
-- Date: 2026-02-18
|
||||||
-- Related: MARS-1057 Batch External Table Creation (ROLLBACK)
|
-- Related: MARS-1057 Batch External Table Creation + Area Filter (ROLLBACK)
|
||||||
--=============================================================================================================================
|
--=============================================================================================================================
|
||||||
|
|
||||||
SET SERVEROUTPUT ON
|
SET SERVEROUTPUT ON
|
||||||
|
|
||||||
PROMPT ========================================================================
|
PROMPT ========================================================================
|
||||||
PROMPT Rolling back FILE_MANAGER Package Body to v3.3.0
|
PROMPT Rolling back FILE_MANAGER Package Body from v3.5.0 to v3.3.0
|
||||||
PROMPT ========================================================================
|
PROMPT ========================================================================
|
||||||
|
|
||||||
@@current_version/FILE_MANAGER.pkb
|
@@rollback_version/FILE_MANAGER.pkb
|
||||||
|
|
||||||
-- Verify compilation status (check specific schema when installing as ADMIN)
|
-- Verify compilation status (check specific schema when installing as ADMIN)
|
||||||
SELECT object_name, object_type, status
|
SELECT object_name, object_type, status
|
||||||
|
|||||||
@@ -3,9 +3,9 @@
|
|||||||
-- ===================================================================
|
-- ===================================================================
|
||||||
-- Purpose: Restore FILE_MANAGER_ODS package body to v2.1.0
|
-- Purpose: Restore FILE_MANAGER_ODS package body to v2.1.0
|
||||||
-- Author: Grzegorz Michalski
|
-- Author: Grzegorz Michalski
|
||||||
-- Date: 2025-11-27
|
-- Date: 2026-02-18
|
||||||
-- Package: ODS.FILE_MANAGER_ODS
|
-- Package: ODS.FILE_MANAGER_ODS
|
||||||
-- Version: 2.2.0 -> 2.1.0 (rollback)
|
-- Version: 2.4.0 -> 2.1.0 (rollback)
|
||||||
|
|
||||||
SET ECHO ON
|
SET ECHO ON
|
||||||
SET DEFINE OFF
|
SET DEFINE OFF
|
||||||
|
|||||||
@@ -3,9 +3,9 @@
|
|||||||
-- ===================================================================
|
-- ===================================================================
|
||||||
-- Purpose: Restore FILE_MANAGER_ODS package specification to v2.1.0
|
-- Purpose: Restore FILE_MANAGER_ODS package specification to v2.1.0
|
||||||
-- Author: Grzegorz Michalski
|
-- Author: Grzegorz Michalski
|
||||||
-- Date: 2025-11-27
|
-- Date: 2026-02-18
|
||||||
-- Package: ODS.FILE_MANAGER_ODS
|
-- Package: ODS.FILE_MANAGER_ODS
|
||||||
-- Version: 2.2.0 -> 2.1.0 (rollback)
|
-- Version: 2.4.0 -> 2.1.0 (rollback)
|
||||||
|
|
||||||
SET ECHO ON
|
SET ECHO ON
|
||||||
SET DEFINE OFF
|
SET DEFINE OFF
|
||||||
|
|||||||
@@ -1,376 +0,0 @@
|
|||||||
# MARS-1057: Batch External Table Creation
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
This MARS package adds batch external table creation capabilities to the FILE_MANAGER and FILE_MANAGER_ODS packages, enabling automatic creation of external table sets (INBOX, ODS, ARCHIVE) based on A_SOURCE_FILE_CONFIG metadata.
|
|
||||||
|
|
||||||
**Jira Issue:** MARS-1057
|
|
||||||
**Package Version:** FILE_MANAGER 3.4.0, FILE_MANAGER_ODS 2.2.0
|
|
||||||
**Author:** Grzegorz Michalski
|
|
||||||
**Date:** 2025-11-27
|
|
||||||
|
|
||||||
## Contents
|
|
||||||
- `install_mars1057.sql` - Master installation script with SPOOL logging
|
|
||||||
- `rollback_mars1057.sql` - Master rollback script
|
|
||||||
- `01_MARS_1057_install_CT_MRDS_FILE_MANAGER_SPEC.sql` - Install FILE_MANAGER package specification
|
|
||||||
- `02_MARS_1057_install_CT_MRDS_FILE_MANAGER_BODY.sql` - Install FILE_MANAGER package body
|
|
||||||
- `03_MARS_1057_install_ODS_FILE_MANAGER_ODS_SPEC.sql` - Install FILE_MANAGER_ODS package specification
|
|
||||||
- `04_MARS_1057_install_ODS_FILE_MANAGER_ODS_BODY.sql` - Install FILE_MANAGER_ODS package body
|
|
||||||
- `91_MARS_1057_rollback_CT_MRDS_FILE_MANAGER_SPEC.sql` - Rollback FILE_MANAGER package specification
|
|
||||||
- `92_MARS_1057_rollback_CT_MRDS_FILE_MANAGER_BODY.sql` - Rollback FILE_MANAGER package body
|
|
||||||
- `93_MARS_1057_rollback_ODS_FILE_MANAGER_ODS_BODY.sql` - Rollback FILE_MANAGER_ODS package body
|
|
||||||
- `94_MARS_1057_rollback_ODS_FILE_MANAGER_ODS_SPEC.sql` - Rollback FILE_MANAGER_ODS package specification
|
|
||||||
- `track_package_versions.sql` - Universal package version tracking
|
|
||||||
- `verify_packages_version.sql` - Universal package verification
|
|
||||||
- `rollback_version/` - FILE_MANAGER v3.3.0, FILE_MANAGER_ODS v2.1.0 (before MARS-1057)
|
|
||||||
- `new_version/` - FILE_MANAGER v3.4.0, FILE_MANAGER_ODS v2.2.0 (after MARS-1057)
|
|
||||||
- `.gitignore` - Git exclusions for temporary files
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
- Oracle Database 23ai
|
|
||||||
- FILE_MANAGER package v3.3.0 installed
|
|
||||||
- FILE_MANAGER_ODS package v2.1.0 installed
|
|
||||||
- ENV_MANAGER package v3.1.0+ with version tracking
|
|
||||||
- ADMIN user access for deployment
|
|
||||||
- ODS schema with necessary privileges
|
|
||||||
|
|
||||||
## New Features
|
|
||||||
|
|
||||||
### 1. CREATE_EXTERNAL_TABLES_SET
|
|
||||||
Creates a complete set of 3 external tables (INBOX, ODS, ARCHIVE) for a single configuration from A_SOURCE_FILE_CONFIG.
|
|
||||||
|
|
||||||
**Signature:**
|
|
||||||
```sql
|
|
||||||
PROCEDURE CREATE_EXTERNAL_TABLES_SET (
|
|
||||||
pSourceFileConfigKey IN NUMBER,
|
|
||||||
pRecreate IN BOOLEAN DEFAULT FALSE
|
|
||||||
);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Example:**
|
|
||||||
```sql
|
|
||||||
BEGIN
|
|
||||||
FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET(
|
|
||||||
pSourceFileConfigKey => 123,
|
|
||||||
pRecreate => FALSE
|
|
||||||
);
|
|
||||||
END;
|
|
||||||
/
|
|
||||||
```
|
|
||||||
|
|
||||||
**Features:**
|
|
||||||
- Automatic table naming: `{TABLE_ID}_{INBOX|ODS|ARCHIVE}`
|
|
||||||
- Official path patterns compliance
|
|
||||||
- Optional drop and recreate
|
|
||||||
- Full ENV_MANAGER logging
|
|
||||||
- Error handling with detailed messages
|
|
||||||
|
|
||||||
### 2. CREATE_EXTERNAL_TABLES_BATCH
|
|
||||||
Creates external table sets for multiple configurations based on filter criteria.
|
|
||||||
|
|
||||||
**Signature:**
|
|
||||||
```sql
|
|
||||||
PROCEDURE CREATE_EXTERNAL_TABLES_BATCH (
|
|
||||||
pSourceKey IN VARCHAR2 DEFAULT NULL,
|
|
||||||
pSourceFileId IN VARCHAR2 DEFAULT NULL,
|
|
||||||
pTableId IN VARCHAR2 DEFAULT NULL,
|
|
||||||
pRecreate IN BOOLEAN DEFAULT FALSE
|
|
||||||
);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Examples:**
|
|
||||||
```sql
|
|
||||||
-- All external tables for C2D source
|
|
||||||
BEGIN
|
|
||||||
FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
|
|
||||||
pSourceKey => 'C2D'
|
|
||||||
);
|
|
||||||
END;
|
|
||||||
/
|
|
||||||
|
|
||||||
-- Recreate all external tables
|
|
||||||
BEGIN
|
|
||||||
FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
|
|
||||||
pRecreate => TRUE
|
|
||||||
);
|
|
||||||
END;
|
|
||||||
/
|
|
||||||
|
|
||||||
-- Specific table across all sources
|
|
||||||
BEGIN
|
|
||||||
FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
|
|
||||||
pTableId => 'A_UC_DISSEM_METADATA_LOADS'
|
|
||||||
);
|
|
||||||
END;
|
|
||||||
/
|
|
||||||
```
|
|
||||||
|
|
||||||
**Features:**
|
|
||||||
- Filters only INPUT type files
|
|
||||||
- Continues processing on errors
|
|
||||||
- Returns summary (Total/Processed/Failed)
|
|
||||||
- Comprehensive logging
|
|
||||||
|
|
||||||
### 3. FILE_MANAGER_ODS Wrapper Procedures
|
|
||||||
The ODS.FILE_MANAGER_ODS package now includes DEFINER rights wrappers for the new batch procedures.
|
|
||||||
|
|
||||||
**New Wrappers:**
|
|
||||||
```sql
|
|
||||||
-- Create single external table set via ODS wrapper
|
|
||||||
EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_SET(
|
|
||||||
pSourceFileConfigKey => 123,
|
|
||||||
pRecreate => FALSE
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Create batch external tables via ODS wrapper
|
|
||||||
EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_BATCH(
|
|
||||||
pSourceKey => 'C2D',
|
|
||||||
pRecreate => FALSE
|
|
||||||
);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Benefits:**
|
|
||||||
- Objects created in ODS schema regardless of execution user
|
|
||||||
- Same functionality as CT_MRDS.FILE_MANAGER with DEFINER rights
|
|
||||||
- Full logging and error handling
|
|
||||||
- Consistent API with original package
|
|
||||||
|
|
||||||
## Installation
|
|
||||||
|
|
||||||
### Option 1: Master Script (Recommended)
|
|
||||||
```powershell
|
|
||||||
# IMPORTANT: Execute as ADMIN user
|
|
||||||
Get-Content "MARS_Packages/REL03/MARS-1057/install_mars1057.sql" | sql "ADMIN/password@service"
|
|
||||||
|
|
||||||
# Log file created: log/INSTALL_MARS_1057_<PDB>_<timestamp>.log
|
|
||||||
```
|
|
||||||
|
|
||||||
**Installation Steps:**
|
|
||||||
1. Install FILE_MANAGER package specification v3.4.0
|
|
||||||
2. Install FILE_MANAGER package body v3.4.0
|
|
||||||
3. Install FILE_MANAGER_ODS package specification v2.2.0
|
|
||||||
4. Install FILE_MANAGER_ODS package body v2.2.0
|
|
||||||
5. Track versions in A_PACKAGE_VERSION_TRACKING
|
|
||||||
6. Verify all tracked packages for untracked changes
|
|
||||||
|
|
||||||
### Option 2: Individual Scripts
|
|
||||||
```powershell
|
|
||||||
# IMPORTANT: Execute as ADMIN user
|
|
||||||
Get-Content "01_MARS_1057_install_CT_MRDS_FILE_MANAGER_SPEC.sql" | sql "ADMIN/password@service"
|
|
||||||
Get-Content "02_MARS_1057_install_CT_MRDS_FILE_MANAGER_BODY.sql" | sql "ADMIN/password@service"
|
|
||||||
Get-Content "03_MARS_1057_install_ODS_FILE_MANAGER_ODS_SPEC.sql" | sql "ADMIN/password@service"
|
|
||||||
Get-Content "04_MARS_1057_install_ODS_FILE_MANAGER_ODS_BODY.sql" | sql "ADMIN/password@service"
|
|
||||||
Get-Content "track_package_versions.sql" | sql "ADMIN/password@service"
|
|
||||||
Get-Content "verify_packages_version.sql" | sql "ADMIN/password@service"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Verification
|
|
||||||
```sql
|
|
||||||
-- Check FILE_MANAGER version
|
|
||||||
SELECT CT_MRDS.FILE_MANAGER.GET_VERSION() FROM DUAL;
|
|
||||||
-- Expected: 3.4.0
|
|
||||||
|
|
||||||
-- Check FILE_MANAGER_ODS version
|
|
||||||
SELECT ODS.FILE_MANAGER_ODS.GET_VERSION() FROM DUAL;
|
|
||||||
-- Expected: 2.2.0
|
|
||||||
|
|
||||||
-- Check for errors (ADMIN user checks specific schema)
|
|
||||||
SELECT * FROM ALL_ERRORS
|
|
||||||
WHERE OWNER IN ('CT_MRDS', 'ODS')
|
|
||||||
AND NAME IN ('FILE_MANAGER', 'FILE_MANAGER_ODS');
|
|
||||||
-- Expected: No rows
|
|
||||||
|
|
||||||
-- Verify new FILE_MANAGER procedures exist
|
|
||||||
SELECT procedure_name
|
|
||||||
FROM ALL_PROCEDURES
|
|
||||||
WHERE OWNER = 'CT_MRDS'
|
|
||||||
AND object_name = 'FILE_MANAGER'
|
|
||||||
AND procedure_name IN ('CREATE_EXTERNAL_TABLES_SET', 'CREATE_EXTERNAL_TABLES_BATCH');
|
|
||||||
-- Expected: 2 rows
|
|
||||||
|
|
||||||
-- Verify new FILE_MANAGER_ODS procedures exist
|
|
||||||
SELECT procedure_name
|
|
||||||
FROM ALL_PROCEDURES
|
|
||||||
WHERE OWNER = 'ODS'
|
|
||||||
AND object_name = 'FILE_MANAGER_ODS'
|
|
||||||
AND procedure_name IN ('CREATE_EXTERNAL_TABLES_SET', 'CREATE_EXTERNAL_TABLES_BATCH');
|
|
||||||
-- Expected: 2 rows
|
|
||||||
|
|
||||||
-- Check for untracked changes
|
|
||||||
SELECT CT_MRDS.ENV_MANAGER.CHECK_PACKAGE_CHANGES('CT_MRDS', 'FILE_MANAGER') FROM DUAL;
|
|
||||||
-- Expected: OK: Package CT_MRDS.FILE_MANAGER has not changed.
|
|
||||||
|
|
||||||
SELECT CT_MRDS.ENV_MANAGER.CHECK_PACKAGE_CHANGES('ODS', 'FILE_MANAGER_ODS') FROM DUAL;
|
|
||||||
-- Expected: OK: Package ODS.FILE_MANAGER_ODS has not changed.
|
|
||||||
```
|
|
||||||
|
|
||||||
## Rollback
|
|
||||||
```powershell
|
|
||||||
# IMPORTANT: Execute as ADMIN user
|
|
||||||
Get-Content "MARS_Packages/REL03/MARS-1057/rollback_mars1057.sql" | sql "ADMIN/password@service"
|
|
||||||
```
|
|
||||||
|
|
||||||
**Rollback restores:**
|
|
||||||
- FILE_MANAGER package specification v3.3.0
|
|
||||||
- FILE_MANAGER package body v3.3.0
|
|
||||||
- FILE_MANAGER_ODS package specification v2.1.0
|
|
||||||
- FILE_MANAGER_ODS package body v2.1.0
|
|
||||||
- Removes CREATE_EXTERNAL_TABLES_SET procedures from both packages
|
|
||||||
- Removes CREATE_EXTERNAL_TABLES_BATCH procedures from both packages
|
|
||||||
|
|
||||||
## Expected Changes
|
|
||||||
- **FILE_MANAGER package**: v3.3.0 → v3.4.0
|
|
||||||
- **FILE_MANAGER_ODS package**: v2.1.0 → v2.2.0
|
|
||||||
- **New procedures in FILE_MANAGER**: CREATE_EXTERNAL_TABLES_SET, CREATE_EXTERNAL_TABLES_BATCH
|
|
||||||
- **New procedures in FILE_MANAGER_ODS**: CREATE_EXTERNAL_TABLES_SET, CREATE_EXTERNAL_TABLES_BATCH
|
|
||||||
- **FILE_MANAGER SPEC size**: +3.3 KB (declaration and documentation)
|
|
||||||
- **FILE_MANAGER BODY size**: +8.6 KB (implementation with logging)
|
|
||||||
- **FILE_MANAGER_ODS SPEC size**: +2.8 KB (wrapper declarations)
|
|
||||||
- **FILE_MANAGER_ODS BODY size**: +4.5 KB (wrapper implementations)
|
|
||||||
|
|
||||||
## Testing
|
|
||||||
|
|
||||||
### Test 1: Create Single Set (CT_MRDS.FILE_MANAGER)
|
|
||||||
```sql
|
|
||||||
BEGIN
|
|
||||||
CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET(
|
|
||||||
pSourceFileConfigKey => 123
|
|
||||||
);
|
|
||||||
END;
|
|
||||||
/
|
|
||||||
|
|
||||||
-- Verify tables created
|
|
||||||
SELECT table_name
|
|
||||||
FROM ALL_TABLES
|
|
||||||
WHERE OWNER = 'ODS'
|
|
||||||
AND (table_name LIKE '%_INBOX'
|
|
||||||
OR table_name LIKE '%_ODS'
|
|
||||||
OR table_name LIKE '%_ARCHIVE');
|
|
||||||
```
|
|
||||||
|
|
||||||
### Test 2: Create Single Set (ODS.FILE_MANAGER_ODS)
|
|
||||||
```sql
|
|
||||||
BEGIN
|
|
||||||
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_SET(
|
|
||||||
pSourceFileConfigKey => 123,
|
|
||||||
pRecreate => TRUE
|
|
||||||
);
|
|
||||||
END;
|
|
||||||
/
|
|
||||||
```
|
|
||||||
|
|
||||||
### Test 3: Batch Creation (CT_MRDS.FILE_MANAGER)
|
|
||||||
```sql
|
|
||||||
BEGIN
|
|
||||||
FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET(
|
|
||||||
pSourceFileConfigKey => 123
|
|
||||||
);
|
|
||||||
END;
|
|
||||||
/
|
|
||||||
|
|
||||||
-- Verify tables created
|
|
||||||
SELECT table_name
|
|
||||||
FROM ALL_TABLES
|
|
||||||
WHERE OWNER = 'ODS'
|
|
||||||
AND table_name LIKE '%_INBOX'
|
|
||||||
OR table_name LIKE '%_ODS'
|
|
||||||
OR table_name LIKE '%_ARCHIVE';
|
|
||||||
```
|
|
||||||
|
|
||||||
### Test 2: Batch Creation
|
|
||||||
```sql
|
|
||||||
BEGIN
|
|
||||||
FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
|
|
||||||
pSourceKey => 'C2D'
|
|
||||||
);
|
|
||||||
END;
|
|
||||||
/
|
|
||||||
|
|
||||||
-- Check process log for results
|
|
||||||
SELECT *
|
|
||||||
FROM CT_MRDS.A_PROCESS_LOG
|
|
||||||
WHERE LOG_TIMESTAMP > SYSDATE - INTERVAL '1' HOUR
|
|
||||||
AND PROCEDURE_NAME LIKE '%CREATE_EXTERNAL_TABLES%'
|
|
||||||
ORDER BY LOG_TIMESTAMP DESC;
|
|
||||||
```
|
|
||||||
|
|
||||||
### Test 3: Recreate Existing Tables
|
|
||||||
```sql
|
|
||||||
BEGIN
|
|
||||||
FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET(
|
|
||||||
pSourceFileConfigKey => 123,
|
|
||||||
pRecreate => TRUE
|
|
||||||
);
|
|
||||||
END;
|
|
||||||
/
|
|
||||||
```
|
|
||||||
|
|
||||||
## Usage Examples
|
|
||||||
|
|
||||||
### Example 1: Setup All External Tables for New Source
|
|
||||||
```sql
|
|
||||||
-- 1. Add source configuration
|
|
||||||
CALL FILE_MANAGER.ADD_SOURCE('LM', 'Liquidity Management');
|
|
||||||
|
|
||||||
-- 2. Add file configurations
|
|
||||||
CALL FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
|
||||||
pSourceKey => 'LM',
|
|
||||||
pSourceFileType => 'INPUT',
|
|
||||||
pSourceFileId => 'STANDING_FACILITIES',
|
|
||||||
pSourceFileDesc => 'Standing Facilities Data',
|
|
||||||
pSourceFileNamePattern => 'SF_*.csv',
|
|
||||||
pTableId => 'STANDING_FACILITIES',
|
|
||||||
pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES',
|
|
||||||
pEncoding => 'UTF8'
|
|
||||||
);
|
|
||||||
|
|
||||||
-- 3. Create all external tables for this source
|
|
||||||
BEGIN
|
|
||||||
FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
|
|
||||||
pSourceKey => 'LM'
|
|
||||||
);
|
|
||||||
END;
|
|
||||||
/
|
|
||||||
```
|
|
||||||
|
|
||||||
### Example 2: Recreate All External Tables
|
|
||||||
```sql
|
|
||||||
-- Useful after bucket URI changes or template table modifications
|
|
||||||
BEGIN
|
|
||||||
FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
|
|
||||||
pRecreate => TRUE
|
|
||||||
);
|
|
||||||
END;
|
|
||||||
/
|
|
||||||
```
|
|
||||||
|
|
||||||
### Example 3: Create Tables for Specific File Type
|
|
||||||
```sql
|
|
||||||
BEGIN
|
|
||||||
FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
|
|
||||||
pSourceFileId => 'UC_DISSEM'
|
|
||||||
);
|
|
||||||
END;
|
|
||||||
/
|
|
||||||
```
|
|
||||||
|
|
||||||
## Known Issues
|
|
||||||
None
|
|
||||||
|
|
||||||
## Dependencies
|
|
||||||
- **CT_MRDS.ENV_MANAGER** - Logging and error handling
|
|
||||||
- **ODS.FILE_MANAGER_ODS** - AUTHID DEFINER wrapper for external table creation
|
|
||||||
- **CT_MRDS.A_SOURCE_FILE_CONFIG** - Source file configuration metadata
|
|
||||||
- **CT_ET_TEMPLATES schema** - Template table definitions
|
|
||||||
|
|
||||||
## Related
|
|
||||||
- **MARS-1056** - VARCHAR2 CHAR/BYTE semantics fix
|
|
||||||
- **MARS-1049** - CSV encoding support
|
|
||||||
- **Package Deployment Guide** - Standard deployment procedures
|
|
||||||
- **Tables Setup Guide** - External table configuration guide
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
- All installations must be executed as ADMIN user
|
|
||||||
- Use `ALL_*` views instead of `USER_*` views for verification
|
|
||||||
- Master scripts include SPOOL logging for audit trail
|
|
||||||
- ACCEPT validation prevents accidental execution
|
|
||||||
- Follows official path patterns: INBOX (3-level), ODS (2-level), ARCHIVE (2-level)
|
|
||||||
@@ -1,10 +1,11 @@
|
|||||||
-- ===================================================================
|
-- ===================================================================
|
||||||
-- MARS-1057 INSTALL SCRIPT: Batch External Table Creation
|
-- MARS-1057 INSTALL SCRIPT: Batch External Table Creation
|
||||||
-- ===================================================================
|
-- ===================================================================
|
||||||
-- Purpose: Install FILE_MANAGER v3.4.0 with batch external table creation procedures
|
-- Purpose: Install FILE_MANAGER v3.5.0 with batch external table creation procedures
|
||||||
|
-- and area filter functionality
|
||||||
-- Author: Grzegorz Michalski
|
-- Author: Grzegorz Michalski
|
||||||
-- Date: 2025-11-27
|
-- Date: 2026-02-18
|
||||||
-- Version: 3.4.0
|
-- Version: 3.5.0
|
||||||
|
|
||||||
-- Dynamic spool file generation (using SYS_CONTEXT - no DBA privileges required)
|
-- Dynamic spool file generation (using SYS_CONTEXT - no DBA privileges required)
|
||||||
-- Log files are automatically created in log/ subdirectory
|
-- Log files are automatically created in log/ subdirectory
|
||||||
@@ -29,20 +30,28 @@ SET PAUSE OFF
|
|||||||
-- ALTER SESSION SET CURRENT_SCHEMA = CT_MRDS;
|
-- ALTER SESSION SET CURRENT_SCHEMA = CT_MRDS;
|
||||||
|
|
||||||
PROMPT =========================================================================
|
PROMPT =========================================================================
|
||||||
PROMPT MARS-1057: Batch External Table Creation
|
PROMPT MARS-1057: Batch External Table Creation + Area Filter
|
||||||
PROMPT =========================================================================
|
PROMPT =========================================================================
|
||||||
PROMPT
|
PROMPT
|
||||||
PROMPT This script will install FILE_MANAGER package v3.4.0 with new features:
|
PROMPT This script will install FILE_MANAGER package v3.5.0 with new features:
|
||||||
PROMPT - CREATE_EXTERNAL_TABLES_SET procedure
|
PROMPT - CREATE_EXTERNAL_TABLES_SET procedure
|
||||||
PROMPT - CREATE_EXTERNAL_TABLES_BATCH procedure
|
PROMPT - CREATE_EXTERNAL_TABLES_BATCH procedure
|
||||||
|
PROMPT - pArea parameter for selective table creation (INBOX/ODS/ARCHIVE/ALL)
|
||||||
|
PROMPT - pRestoreGrants parameter for grant preservation during recreate
|
||||||
PROMPT
|
PROMPT
|
||||||
PROMPT And FILE_MANAGER_ODS package v2.2.0 with wrapper procedures.
|
PROMPT And FILE_MANAGER_ODS package v2.4.0 with wrapper procedures.
|
||||||
PROMPT
|
PROMPT
|
||||||
PROMPT Changes:
|
PROMPT Changes:
|
||||||
PROMPT - FILE_MANAGER package specification (3.3.0 -> 3.4.0)
|
PROMPT - FILE_MANAGER package specification (3.3.0 -> 3.5.0)
|
||||||
PROMPT - FILE_MANAGER package body (3.3.0 -> 3.4.0)
|
PROMPT - FILE_MANAGER package body (3.3.0 -> 3.5.0)
|
||||||
PROMPT - FILE_MANAGER_ODS package specification (2.1.0 -> 2.2.0)
|
PROMPT - FILE_MANAGER_ODS package specification (2.1.0 -> 2.4.0)
|
||||||
PROMPT - FILE_MANAGER_ODS package body (2.1.0 -> 2.2.0)
|
PROMPT - FILE_MANAGER_ODS package body (2.1.0 -> 2.4.0)
|
||||||
|
PROMPT
|
||||||
|
PROMPT New functionality:
|
||||||
|
PROMPT - pArea => 'INBOX' : creates only INBOX table
|
||||||
|
PROMPT - pArea => 'ODS' : creates only ODS table
|
||||||
|
PROMPT - pArea => 'ARCHIVE' : creates only ARCHIVE table
|
||||||
|
PROMPT - pArea => 'ALL' : creates all tables (default)
|
||||||
PROMPT
|
PROMPT
|
||||||
PROMPT Expected Duration: 1-2 minutes
|
PROMPT Expected Duration: 1-2 minutes
|
||||||
PROMPT =========================================================================
|
PROMPT =========================================================================
|
||||||
@@ -60,25 +69,25 @@ WHENEVER SQLERROR CONTINUE
|
|||||||
|
|
||||||
PROMPT
|
PROMPT
|
||||||
PROMPT =========================================================================
|
PROMPT =========================================================================
|
||||||
PROMPT Step 1: Install FILE_MANAGER Package Specification v3.4.0
|
PROMPT Step 1: Install FILE_MANAGER Package Specification v3.5.0
|
||||||
PROMPT =========================================================================
|
PROMPT =========================================================================
|
||||||
@@01_MARS_1057_install_CT_MRDS_FILE_MANAGER_SPEC.sql
|
@@01_MARS_1057_install_CT_MRDS_FILE_MANAGER_SPEC.sql
|
||||||
|
|
||||||
PROMPT
|
PROMPT
|
||||||
PROMPT =========================================================================
|
PROMPT =========================================================================
|
||||||
PROMPT Step 2: Install FILE_MANAGER Package Body v3.4.0
|
PROMPT Step 2: Install FILE_MANAGER Package Body v3.5.0
|
||||||
PROMPT =========================================================================
|
PROMPT =========================================================================
|
||||||
@@02_MARS_1057_install_CT_MRDS_FILE_MANAGER_BODY.sql
|
@@02_MARS_1057_install_CT_MRDS_FILE_MANAGER_BODY.sql
|
||||||
|
|
||||||
PROMPT
|
PROMPT
|
||||||
PROMPT =========================================================================
|
PROMPT =========================================================================
|
||||||
PROMPT Step 3: Install FILE_MANAGER_ODS Package Specification v2.2.0
|
PROMPT Step 3: Install FILE_MANAGER_ODS Package Specification v2.4.0
|
||||||
PROMPT =========================================================================
|
PROMPT =========================================================================
|
||||||
@@03_MARS_1057_install_ODS_FILE_MANAGER_ODS_SPEC.sql
|
@@03_MARS_1057_install_ODS_FILE_MANAGER_ODS_SPEC.sql
|
||||||
|
|
||||||
PROMPT
|
PROMPT
|
||||||
PROMPT =========================================================================
|
PROMPT =========================================================================
|
||||||
PROMPT Step 4: Install FILE_MANAGER_ODS Package Body v2.2.0
|
PROMPT Step 4: Install FILE_MANAGER_ODS Package Body v2.4.0
|
||||||
PROMPT =========================================================================
|
PROMPT =========================================================================
|
||||||
@@04_MARS_1057_install_ODS_FILE_MANAGER_ODS_BODY.sql
|
@@04_MARS_1057_install_ODS_FILE_MANAGER_ODS_BODY.sql
|
||||||
|
|
||||||
|
|||||||
@@ -1969,7 +1969,8 @@ AS
|
|||||||
PROCEDURE CREATE_EXTERNAL_TABLES_SET (
|
PROCEDURE CREATE_EXTERNAL_TABLES_SET (
|
||||||
pSourceFileConfigKey IN NUMBER,
|
pSourceFileConfigKey IN NUMBER,
|
||||||
pRecreate IN BOOLEAN DEFAULT FALSE,
|
pRecreate IN BOOLEAN DEFAULT FALSE,
|
||||||
pRestoreGrants IN BOOLEAN DEFAULT TRUE
|
pRestoreGrants IN BOOLEAN DEFAULT TRUE,
|
||||||
|
pArea IN VARCHAR2 DEFAULT 'ALL'
|
||||||
)
|
)
|
||||||
IS
|
IS
|
||||||
-- Type for storing grant information
|
-- Type for storing grant information
|
||||||
@@ -2001,6 +2002,7 @@ AS
|
|||||||
|
|
||||||
vTableExists NUMBER;
|
vTableExists NUMBER;
|
||||||
vParameters VARCHAR2(4000);
|
vParameters VARCHAR2(4000);
|
||||||
|
vAreaUpper VARCHAR2(20);
|
||||||
|
|
||||||
-- Nested procedure to save table grants before DROP
|
-- Nested procedure to save table grants before DROP
|
||||||
PROCEDURE SAVE_GRANTS(pTableName VARCHAR2, pGrantList OUT tGrantList) IS
|
PROCEDURE SAVE_GRANTS(pTableName VARCHAR2, pGrantList OUT tGrantList) IS
|
||||||
@@ -2104,10 +2106,19 @@ AS
|
|||||||
END DROP_IF_EXISTS;
|
END DROP_IF_EXISTS;
|
||||||
|
|
||||||
BEGIN
|
BEGIN
|
||||||
|
-- Validate and normalize pArea parameter
|
||||||
|
vAreaUpper := UPPER(TRIM(pArea));
|
||||||
|
|
||||||
|
IF vAreaUpper NOT IN ('INBOX', 'ODS', 'ARCHIVE', 'ALL') THEN
|
||||||
|
vgMsgTmp := 'Invalid pArea parameter: ''' || pArea || '''. Must be one of: INBOX, ODS, ARCHIVE, ALL';
|
||||||
|
RAISE_APPLICATION_ERROR(-20010, vgMsgTmp);
|
||||||
|
END IF;
|
||||||
|
|
||||||
vParameters := ENV_MANAGER.FORMAT_PARAMETERS(SYS.ODCIVARCHAR2LIST(
|
vParameters := ENV_MANAGER.FORMAT_PARAMETERS(SYS.ODCIVARCHAR2LIST(
|
||||||
'pSourceFileConfigKey => ' || NVL(TO_CHAR(pSourceFileConfigKey), 'NULL'),
|
'pSourceFileConfigKey => ' || NVL(TO_CHAR(pSourceFileConfigKey), 'NULL'),
|
||||||
'pRecreate => ' || CASE WHEN pRecreate THEN 'TRUE' ELSE 'FALSE' END,
|
'pRecreate => ' || CASE WHEN pRecreate THEN 'TRUE' ELSE 'FALSE' END,
|
||||||
'pRestoreGrants => ' || CASE WHEN pRestoreGrants THEN 'TRUE' ELSE 'FALSE' END
|
'pRestoreGrants => ' || CASE WHEN pRestoreGrants THEN 'TRUE' ELSE 'FALSE' END,
|
||||||
|
'pArea => ''' || vAreaUpper || ''''
|
||||||
));
|
));
|
||||||
ENV_MANAGER.LOG_PROCESS_EVENT('Start CREATE_EXTERNAL_TABLES_SET', 'INFO', vParameters);
|
ENV_MANAGER.LOG_PROCESS_EVENT('Start CREATE_EXTERNAL_TABLES_SET', 'INFO', vParameters);
|
||||||
|
|
||||||
@@ -2140,7 +2151,8 @@ AS
|
|||||||
vArchivePrefix := 'ARCHIVE/' || vSourceKey || '/' || vTableId;
|
vArchivePrefix := 'ARCHIVE/' || vSourceKey || '/' || vTableId;
|
||||||
|
|
||||||
ENV_MANAGER.LOG_PROCESS_EVENT(
|
ENV_MANAGER.LOG_PROCESS_EVENT(
|
||||||
'Creating external tables for: ' || vSourceKey || '/' || vSourceFileId || '/' || vTableId,
|
'Creating external tables for: ' || vSourceKey || '/' || vSourceFileId || '/' || vTableId ||
|
||||||
|
' (Area: ' || vAreaUpper || ')',
|
||||||
'INFO'
|
'INFO'
|
||||||
);
|
);
|
||||||
|
|
||||||
@@ -2149,60 +2161,93 @@ AS
|
|||||||
-- Save grants before dropping tables (if pRestoreGrants = TRUE)
|
-- Save grants before dropping tables (if pRestoreGrants = TRUE)
|
||||||
IF pRestoreGrants THEN
|
IF pRestoreGrants THEN
|
||||||
ENV_MANAGER.LOG_PROCESS_EVENT('Saving grants before dropping tables...', 'INFO');
|
ENV_MANAGER.LOG_PROCESS_EVENT('Saving grants before dropping tables...', 'INFO');
|
||||||
SAVE_GRANTS(vInboxTableName, vInboxGrants);
|
|
||||||
SAVE_GRANTS(vOdsTableName, vOdsGrants);
|
IF vAreaUpper IN ('INBOX', 'ALL') THEN
|
||||||
SAVE_GRANTS(vArchiveTableName, vArchiveGrants);
|
SAVE_GRANTS(vInboxTableName, vInboxGrants);
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
IF vAreaUpper IN ('ODS', 'ALL') THEN
|
||||||
|
SAVE_GRANTS(vOdsTableName, vOdsGrants);
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
IF vAreaUpper IN ('ARCHIVE', 'ALL') THEN
|
||||||
|
SAVE_GRANTS(vArchiveTableName, vArchiveGrants);
|
||||||
|
END IF;
|
||||||
END IF;
|
END IF;
|
||||||
|
|
||||||
-- Drop existing tables
|
-- Drop existing tables based on pArea
|
||||||
DROP_IF_EXISTS(vInboxTableName);
|
IF vAreaUpper IN ('INBOX', 'ALL') THEN
|
||||||
DROP_IF_EXISTS(vOdsTableName);
|
DROP_IF_EXISTS(vInboxTableName);
|
||||||
DROP_IF_EXISTS(vArchiveTableName);
|
END IF;
|
||||||
|
|
||||||
|
IF vAreaUpper IN ('ODS', 'ALL') THEN
|
||||||
|
DROP_IF_EXISTS(vOdsTableName);
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
IF vAreaUpper IN ('ARCHIVE', 'ALL') THEN
|
||||||
|
DROP_IF_EXISTS(vArchiveTableName);
|
||||||
|
END IF;
|
||||||
END IF;
|
END IF;
|
||||||
|
|
||||||
-- 5. Create INBOX external table
|
-- 5. Create INBOX external table (if requested)
|
||||||
ENV_MANAGER.LOG_PROCESS_EVENT('Creating INBOX external table: ' || vInboxTableName, 'INFO');
|
IF vAreaUpper IN ('INBOX', 'ALL') THEN
|
||||||
CREATE_EXTERNAL_TABLE(
|
ENV_MANAGER.LOG_PROCESS_EVENT('Creating INBOX external table: ' || vInboxTableName, 'INFO');
|
||||||
pTableName => vInboxTableName,
|
CREATE_EXTERNAL_TABLE(
|
||||||
pTemplateTableName => vTemplateTableName,
|
pTableName => vInboxTableName,
|
||||||
pPrefix => vInboxPrefix,
|
pTemplateTableName => vTemplateTableName,
|
||||||
pBucketUri => ENV_MANAGER.gvInboxBucketUri,
|
pPrefix => vInboxPrefix,
|
||||||
pDelimiter => vDelimiter,
|
pBucketUri => ENV_MANAGER.gvInboxBucketUri,
|
||||||
pEncoding => vEncoding
|
pDelimiter => vDelimiter,
|
||||||
);
|
pEncoding => vEncoding
|
||||||
|
);
|
||||||
|
END IF;
|
||||||
|
|
||||||
-- 6. Create ODS external table
|
-- 6. Create ODS external table (if requested)
|
||||||
ENV_MANAGER.LOG_PROCESS_EVENT('Creating ODS external table: ' || vOdsTableName, 'INFO');
|
IF vAreaUpper IN ('ODS', 'ALL') THEN
|
||||||
CREATE_EXTERNAL_TABLE(
|
ENV_MANAGER.LOG_PROCESS_EVENT('Creating ODS external table: ' || vOdsTableName, 'INFO');
|
||||||
pTableName => vOdsTableName,
|
CREATE_EXTERNAL_TABLE(
|
||||||
pTemplateTableName => vTemplateTableName,
|
pTableName => vOdsTableName,
|
||||||
pPrefix => vOdsPrefix,
|
pTemplateTableName => vTemplateTableName,
|
||||||
pBucketUri => ENV_MANAGER.gvDataBucketUri,
|
pPrefix => vOdsPrefix,
|
||||||
pDelimiter => vDelimiter,
|
pBucketUri => ENV_MANAGER.gvDataBucketUri,
|
||||||
pEncoding => vEncoding
|
pDelimiter => vDelimiter,
|
||||||
);
|
pEncoding => vEncoding
|
||||||
|
);
|
||||||
|
END IF;
|
||||||
|
|
||||||
-- 7. Create ARCHIVE external table
|
-- 7. Create ARCHIVE external table (if requested)
|
||||||
ENV_MANAGER.LOG_PROCESS_EVENT('Creating ARCHIVE external table: ' || vArchiveTableName, 'INFO');
|
IF vAreaUpper IN ('ARCHIVE', 'ALL') THEN
|
||||||
CREATE_EXTERNAL_TABLE(
|
ENV_MANAGER.LOG_PROCESS_EVENT('Creating ARCHIVE external table: ' || vArchiveTableName, 'INFO');
|
||||||
pTableName => vArchiveTableName,
|
CREATE_EXTERNAL_TABLE(
|
||||||
pTemplateTableName => vTemplateTableName,
|
pTableName => vArchiveTableName,
|
||||||
pPrefix => vArchivePrefix,
|
pTemplateTableName => vTemplateTableName,
|
||||||
pBucketUri => ENV_MANAGER.gvArchiveBucketUri,
|
pPrefix => vArchivePrefix,
|
||||||
pDelimiter => vDelimiter,
|
pBucketUri => ENV_MANAGER.gvArchiveBucketUri,
|
||||||
pEncoding => vEncoding
|
pDelimiter => vDelimiter,
|
||||||
);
|
pEncoding => vEncoding
|
||||||
|
);
|
||||||
|
END IF;
|
||||||
|
|
||||||
-- 8. Restore grants after creating tables (if pRecreate = TRUE and pRestoreGrants = TRUE)
|
-- 8. Restore grants after creating tables (if pRecreate = TRUE and pRestoreGrants = TRUE)
|
||||||
IF pRecreate AND pRestoreGrants THEN
|
IF pRecreate AND pRestoreGrants THEN
|
||||||
ENV_MANAGER.LOG_PROCESS_EVENT('Restoring grants after creating tables...', 'INFO');
|
ENV_MANAGER.LOG_PROCESS_EVENT('Restoring grants after creating tables...', 'INFO');
|
||||||
RESTORE_GRANTS(vInboxTableName, vInboxGrants);
|
|
||||||
RESTORE_GRANTS(vOdsTableName, vOdsGrants);
|
IF vAreaUpper IN ('INBOX', 'ALL') THEN
|
||||||
RESTORE_GRANTS(vArchiveTableName, vArchiveGrants);
|
RESTORE_GRANTS(vInboxTableName, vInboxGrants);
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
IF vAreaUpper IN ('ODS', 'ALL') THEN
|
||||||
|
RESTORE_GRANTS(vOdsTableName, vOdsGrants);
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
IF vAreaUpper IN ('ARCHIVE', 'ALL') THEN
|
||||||
|
RESTORE_GRANTS(vArchiveTableName, vArchiveGrants);
|
||||||
|
END IF;
|
||||||
END IF;
|
END IF;
|
||||||
|
|
||||||
ENV_MANAGER.LOG_PROCESS_EVENT(
|
ENV_MANAGER.LOG_PROCESS_EVENT(
|
||||||
'End CREATE_EXTERNAL_TABLES_SET - Successfully created all 3 external tables for config: ' || pSourceFileConfigKey,
|
'End CREATE_EXTERNAL_TABLES_SET - Successfully created external tables for config: ' ||
|
||||||
|
pSourceFileConfigKey || ' (Area: ' || vAreaUpper || ')',
|
||||||
'INFO',
|
'INFO',
|
||||||
vParameters
|
vParameters
|
||||||
);
|
);
|
||||||
@@ -2221,20 +2266,31 @@ AS
|
|||||||
pSourceFileId IN VARCHAR2 DEFAULT NULL,
|
pSourceFileId IN VARCHAR2 DEFAULT NULL,
|
||||||
pTableId IN VARCHAR2 DEFAULT NULL,
|
pTableId IN VARCHAR2 DEFAULT NULL,
|
||||||
pRecreate IN BOOLEAN DEFAULT FALSE,
|
pRecreate IN BOOLEAN DEFAULT FALSE,
|
||||||
pRestoreGrants IN BOOLEAN DEFAULT TRUE
|
pRestoreGrants IN BOOLEAN DEFAULT TRUE,
|
||||||
|
pArea IN VARCHAR2 DEFAULT 'ALL'
|
||||||
)
|
)
|
||||||
IS
|
IS
|
||||||
vCount NUMBER := 0;
|
vCount NUMBER := 0;
|
||||||
vProcessed NUMBER := 0;
|
vProcessed NUMBER := 0;
|
||||||
vFailed NUMBER := 0;
|
vFailed NUMBER := 0;
|
||||||
vParameters VARCHAR2(4000);
|
vParameters VARCHAR2(4000);
|
||||||
|
vAreaUpper VARCHAR2(20);
|
||||||
BEGIN
|
BEGIN
|
||||||
|
-- Validate and normalize pArea parameter
|
||||||
|
vAreaUpper := UPPER(TRIM(pArea));
|
||||||
|
|
||||||
|
IF vAreaUpper NOT IN ('INBOX', 'ODS', 'ARCHIVE', 'ALL') THEN
|
||||||
|
vgMsgTmp := 'Invalid pArea parameter: ''' || pArea || '''. Must be one of: INBOX, ODS, ARCHIVE, ALL';
|
||||||
|
RAISE_APPLICATION_ERROR(-20010, vgMsgTmp);
|
||||||
|
END IF;
|
||||||
|
|
||||||
vParameters := ENV_MANAGER.FORMAT_PARAMETERS(SYS.ODCIVARCHAR2LIST(
|
vParameters := ENV_MANAGER.FORMAT_PARAMETERS(SYS.ODCIVARCHAR2LIST(
|
||||||
'pSourceKey => ''' || NVL(pSourceKey, 'NULL') || '''',
|
'pSourceKey => ''' || NVL(pSourceKey, 'NULL') || '''',
|
||||||
'pSourceFileId => ''' || NVL(pSourceFileId, 'NULL') || '''',
|
'pSourceFileId => ''' || NVL(pSourceFileId, 'NULL') || '''',
|
||||||
'pTableId => ''' || NVL(pTableId, 'NULL') || '''',
|
'pTableId => ''' || NVL(pTableId, 'NULL') || '''',
|
||||||
'pRecreate => ' || CASE WHEN pRecreate THEN 'TRUE' ELSE 'FALSE' END,
|
'pRecreate => ' || CASE WHEN pRecreate THEN 'TRUE' ELSE 'FALSE' END,
|
||||||
'pRestoreGrants => ' || CASE WHEN pRestoreGrants THEN 'TRUE' ELSE 'FALSE' END
|
'pRestoreGrants => ' || CASE WHEN pRestoreGrants THEN 'TRUE' ELSE 'FALSE' END,
|
||||||
|
'pArea => ''' || vAreaUpper || ''''
|
||||||
));
|
));
|
||||||
|
|
||||||
ENV_MANAGER.LOG_PROCESS_EVENT('Start CREATE_EXTERNAL_TABLES_BATCH', 'INFO', vParameters);
|
ENV_MANAGER.LOG_PROCESS_EVENT('Start CREATE_EXTERNAL_TABLES_BATCH', 'INFO', vParameters);
|
||||||
@@ -2254,15 +2310,16 @@ AS
|
|||||||
BEGIN
|
BEGIN
|
||||||
ENV_MANAGER.LOG_PROCESS_EVENT(
|
ENV_MANAGER.LOG_PROCESS_EVENT(
|
||||||
'Creating external tables set for: ' || rec.A_SOURCE_KEY || '/' ||
|
'Creating external tables set for: ' || rec.A_SOURCE_KEY || '/' ||
|
||||||
rec.SOURCE_FILE_ID || '/' || rec.TABLE_ID,
|
rec.SOURCE_FILE_ID || '/' || rec.TABLE_ID || ' (Area: ' || vAreaUpper || ')',
|
||||||
'INFO'
|
'INFO'
|
||||||
);
|
);
|
||||||
|
|
||||||
-- Call procedure to create set of 3 tables
|
-- Call procedure to create set of tables (based on pArea)
|
||||||
CREATE_EXTERNAL_TABLES_SET(
|
CREATE_EXTERNAL_TABLES_SET(
|
||||||
pSourceFileConfigKey => rec.A_SOURCE_FILE_CONFIG_KEY,
|
pSourceFileConfigKey => rec.A_SOURCE_FILE_CONFIG_KEY,
|
||||||
pRecreate => pRecreate,
|
pRecreate => pRecreate,
|
||||||
pRestoreGrants => pRestoreGrants
|
pRestoreGrants => pRestoreGrants,
|
||||||
|
pArea => vAreaUpper
|
||||||
);
|
);
|
||||||
|
|
||||||
vProcessed := vProcessed + 1;
|
vProcessed := vProcessed + 1;
|
||||||
@@ -2281,7 +2338,7 @@ AS
|
|||||||
|
|
||||||
ENV_MANAGER.LOG_PROCESS_EVENT(
|
ENV_MANAGER.LOG_PROCESS_EVENT(
|
||||||
'End CREATE_EXTERNAL_TABLES_BATCH - Total: ' || vCount ||
|
'End CREATE_EXTERNAL_TABLES_BATCH - Total: ' || vCount ||
|
||||||
', Processed: ' || vProcessed || ', Failed: ' || vFailed,
|
', Processed: ' || vProcessed || ', Failed: ' || vFailed || ' (Area: ' || vAreaUpper || ')',
|
||||||
'INFO',
|
'INFO',
|
||||||
vParameters
|
vParameters
|
||||||
);
|
);
|
||||||
|
|||||||
@@ -17,12 +17,13 @@ AS
|
|||||||
**/
|
**/
|
||||||
|
|
||||||
-- Package Version Information (Semantic Versioning: MAJOR.MINOR.PATCH)
|
-- Package Version Information (Semantic Versioning: MAJOR.MINOR.PATCH)
|
||||||
PACKAGE_VERSION CONSTANT VARCHAR2(10) := '3.4.0';
|
PACKAGE_VERSION CONSTANT VARCHAR2(10) := '3.5.0';
|
||||||
PACKAGE_BUILD_DATE CONSTANT VARCHAR2(20) := '2025-11-27 14:00:00';
|
PACKAGE_BUILD_DATE CONSTANT VARCHAR2(20) := '2026-02-18 16:00:00';
|
||||||
PACKAGE_AUTHOR CONSTANT VARCHAR2(100) := 'Grzegorz Michalski';
|
PACKAGE_AUTHOR CONSTANT VARCHAR2(100) := 'Grzegorz Michalski';
|
||||||
|
|
||||||
-- Version History (Latest changes first)
|
-- Version History (Latest changes first)
|
||||||
VERSION_HISTORY CONSTANT VARCHAR2(4000) :=
|
VERSION_HISTORY CONSTANT VARCHAR2(4000) :=
|
||||||
|
'3.5.0 (2026-02-18): MARS-1057 - Added pArea parameter for selective table creation (INBOX/ODS/ARCHIVE/ALL)' || CHR(13)||CHR(10) ||
|
||||||
'3.4.0 (2025-11-27): MARS-1057 - Added CREATE_EXTERNAL_TABLES_SET and CREATE_EXTERNAL_TABLES_BATCH procedures for batch external table creation' || CHR(13)||CHR(10) ||
|
'3.4.0 (2025-11-27): MARS-1057 - Added CREATE_EXTERNAL_TABLES_SET and CREATE_EXTERNAL_TABLES_BATCH procedures for batch external table creation' || CHR(13)||CHR(10) ||
|
||||||
'3.3.1 (2025-11-27): MARS-1046 - Fixed ISO 8601 datetime format parsing with milliseconds and timezone (e.g., 2012-03-02T14:16:23.798+01:00)' || CHR(13)||CHR(10) ||
|
'3.3.1 (2025-11-27): MARS-1046 - Fixed ISO 8601 datetime format parsing with milliseconds and timezone (e.g., 2012-03-02T14:16:23.798+01:00)' || CHR(13)||CHR(10) ||
|
||||||
'3.3.0 (2025-11-26): MARS-1056 - Fixed VARCHAR2 definitions in GENERATE_EXTERNAL_TABLE_PARAMS to preserve CHAR/BYTE semantics from template tables' || CHR(13)||CHR(10) ||
|
'3.3.0 (2025-11-26): MARS-1056 - Fixed VARCHAR2 definitions in GENERATE_EXTERNAL_TABLE_PARAMS to preserve CHAR/BYTE semantics from template tables' || CHR(13)||CHR(10) ||
|
||||||
@@ -602,38 +603,50 @@ AS
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* @name CREATE_EXTERNAL_TABLES_SET
|
* @name CREATE_EXTERNAL_TABLES_SET
|
||||||
* @desc Creates a complete set of 3 external tables (INBOX, ODS, ARCHIVE) for a single configuration
|
* @desc Creates a complete set of external tables for a single configuration from A_SOURCE_FILE_CONFIG table.
|
||||||
* from A_SOURCE_FILE_CONFIG table. Automatically generates table names and paths following
|
* Automatically generates table names and paths following official path patterns.
|
||||||
* official path patterns. Optionally drops and recreates existing tables.
|
* Optionally drops and recreates existing tables. If pRestoreGrants is TRUE, saves and restores table grants.
|
||||||
* If pRestoreGrants is TRUE, saves and restores table grants during recreate operation.
|
* The pArea parameter allows selective table creation.
|
||||||
* @param pSourceFileConfigKey - Primary key from A_SOURCE_FILE_CONFIG table
|
* @param pSourceFileConfigKey - Primary key from A_SOURCE_FILE_CONFIG table
|
||||||
* @param pRecreate - If TRUE, drops existing tables before creating new ones; if FALSE, fails if tables exist
|
* @param pRecreate - If TRUE, drops existing tables before creating new ones; if FALSE, fails if tables exist
|
||||||
* @param pRestoreGrants - If TRUE, saves grants before DROP and restores after CREATE (only when pRecreate=TRUE)
|
* @param pRestoreGrants - If TRUE, saves grants before DROP and restores after CREATE (only when pRecreate=TRUE)
|
||||||
* Uses DBA_TAB_PRIVS - requires SELECT ANY DICTIONARY or SELECT ON DBA_TAB_PRIVS privilege
|
* Uses DBA_TAB_PRIVS - requires SELECT ANY DICTIONARY or SELECT ON DBA_TAB_PRIVS privilege
|
||||||
* @example BEGIN
|
* @param pArea - Specifies which tables to create: 'INBOX', 'ODS', 'ARCHIVE', or 'ALL' (default)
|
||||||
|
* 'INBOX' - creates only INBOX table
|
||||||
|
* 'ODS' - creates only ODS table
|
||||||
|
* 'ARCHIVE' - creates only ARCHIVE table
|
||||||
|
* 'ALL' - creates all three tables (default)
|
||||||
|
* @example -- Create only INBOX table
|
||||||
|
* BEGIN
|
||||||
|
* FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET(
|
||||||
|
* pSourceFileConfigKey => 123,
|
||||||
|
* pArea => 'INBOX'
|
||||||
|
* );
|
||||||
|
* END;
|
||||||
|
*
|
||||||
|
* -- Create all tables with grant preservation
|
||||||
|
* BEGIN
|
||||||
* FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET(
|
* FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET(
|
||||||
* pSourceFileConfigKey => 123,
|
* pSourceFileConfigKey => 123,
|
||||||
* pRecreate => TRUE,
|
* pRecreate => TRUE,
|
||||||
* pRestoreGrants => TRUE
|
* pRestoreGrants => TRUE,
|
||||||
|
* pArea => 'ALL'
|
||||||
* );
|
* );
|
||||||
* END;
|
* END;
|
||||||
* @ex_rslt Creates three external tables in ODS schema:
|
* @ex_rslt Creates external table(s) in ODS schema based on pArea parameter
|
||||||
* - C2D_A_UC_DISSEM_METADATA_LOADS_INBOX
|
|
||||||
* - C2D_A_UC_DISSEM_METADATA_LOADS_ODS
|
|
||||||
* - C2D_A_UC_DISSEM_METADATA_LOADS_ARCHIVE
|
|
||||||
* Preserves existing grants if pRestoreGrants = TRUE
|
|
||||||
**/
|
**/
|
||||||
PROCEDURE CREATE_EXTERNAL_TABLES_SET (
|
PROCEDURE CREATE_EXTERNAL_TABLES_SET (
|
||||||
pSourceFileConfigKey IN NUMBER,
|
pSourceFileConfigKey IN NUMBER,
|
||||||
pRecreate IN BOOLEAN DEFAULT FALSE,
|
pRecreate IN BOOLEAN DEFAULT FALSE,
|
||||||
pRestoreGrants IN BOOLEAN DEFAULT TRUE
|
pRestoreGrants IN BOOLEAN DEFAULT TRUE,
|
||||||
|
pArea IN VARCHAR2 DEFAULT 'ALL'
|
||||||
);
|
);
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @name CREATE_EXTERNAL_TABLES_BATCH
|
* @name CREATE_EXTERNAL_TABLES_BATCH
|
||||||
* @desc Creates external table sets for multiple configurations based on filter criteria.
|
* @desc Creates external table sets for multiple configurations based on filter criteria.
|
||||||
* Processes only INPUT type files from A_SOURCE_FILE_CONFIG. Creates 3 tables per configuration
|
* Processes only INPUT type files from A_SOURCE_FILE_CONFIG. Creates tables based on pArea parameter
|
||||||
* (INBOX, ODS, ARCHIVE). Continues processing even if individual sets fail.
|
* (INBOX, ODS, ARCHIVE, or ALL). Continues processing even if individual sets fail.
|
||||||
* If pRestoreGrants is TRUE, saves and restores table grants during recreate operations.
|
* If pRestoreGrants is TRUE, saves and restores table grants during recreate operations.
|
||||||
* @param pSourceKey - Filter by A_SOURCE_KEY (NULL = all sources)
|
* @param pSourceKey - Filter by A_SOURCE_KEY (NULL = all sources)
|
||||||
* @param pSourceFileId - Filter by SOURCE_FILE_ID (NULL = all file types)
|
* @param pSourceFileId - Filter by SOURCE_FILE_ID (NULL = all file types)
|
||||||
@@ -641,20 +654,21 @@ AS
|
|||||||
* @param pRecreate - If TRUE, drops and recreates existing tables; if FALSE, skips if tables exist
|
* @param pRecreate - If TRUE, drops and recreates existing tables; if FALSE, skips if tables exist
|
||||||
* @param pRestoreGrants - If TRUE, saves grants before DROP and restores after CREATE (only when pRecreate=TRUE)
|
* @param pRestoreGrants - If TRUE, saves grants before DROP and restores after CREATE (only when pRecreate=TRUE)
|
||||||
* Uses DBA_TAB_PRIVS - requires SELECT ANY DICTIONARY or SELECT ON DBA_TAB_PRIVS privilege
|
* Uses DBA_TAB_PRIVS - requires SELECT ANY DICTIONARY or SELECT ON DBA_TAB_PRIVS privilege
|
||||||
* @example -- Create all external tables for C2D source with grant preservation
|
* @param pArea - Specifies which tables to create: 'INBOX', 'ODS', 'ARCHIVE', or 'ALL' (default)
|
||||||
|
* @example -- Create only INBOX tables for C2D source
|
||||||
* BEGIN
|
* BEGIN
|
||||||
* FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
|
* FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
|
||||||
* pSourceKey => 'C2D',
|
* pSourceKey => 'C2D',
|
||||||
* pRecreate => TRUE,
|
* pArea => 'INBOX'
|
||||||
* pRestoreGrants => TRUE
|
|
||||||
* );
|
* );
|
||||||
* END;
|
* END;
|
||||||
*
|
*
|
||||||
* -- Recreate all external tables without grant preservation
|
* -- Create all external tables for all sources with grant preservation
|
||||||
* BEGIN
|
* BEGIN
|
||||||
* FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
|
* FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
|
||||||
* pRecreate => TRUE,
|
* pRecreate => TRUE,
|
||||||
* pRestoreGrants => FALSE
|
* pRestoreGrants => TRUE,
|
||||||
|
* pArea => 'ALL'
|
||||||
* );
|
* );
|
||||||
* END;
|
* END;
|
||||||
* @ex_rslt Returns summary: Total: 10, Processed: 9, Failed: 1
|
* @ex_rslt Returns summary: Total: 10, Processed: 9, Failed: 1
|
||||||
@@ -664,7 +678,8 @@ AS
|
|||||||
pSourceFileId IN VARCHAR2 DEFAULT NULL,
|
pSourceFileId IN VARCHAR2 DEFAULT NULL,
|
||||||
pTableId IN VARCHAR2 DEFAULT NULL,
|
pTableId IN VARCHAR2 DEFAULT NULL,
|
||||||
pRecreate IN BOOLEAN DEFAULT FALSE,
|
pRecreate IN BOOLEAN DEFAULT FALSE,
|
||||||
pRestoreGrants IN BOOLEAN DEFAULT TRUE
|
pRestoreGrants IN BOOLEAN DEFAULT TRUE,
|
||||||
|
pArea IN VARCHAR2 DEFAULT 'ALL'
|
||||||
);
|
);
|
||||||
|
|
||||||
---------------------------------------------------------------------------------------------------------------------------
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
|||||||
@@ -64,7 +64,8 @@ AS
|
|||||||
PROCEDURE CREATE_EXTERNAL_TABLES_SET (
|
PROCEDURE CREATE_EXTERNAL_TABLES_SET (
|
||||||
pSourceFileConfigKey IN NUMBER,
|
pSourceFileConfigKey IN NUMBER,
|
||||||
pRecreate IN BOOLEAN DEFAULT FALSE,
|
pRecreate IN BOOLEAN DEFAULT FALSE,
|
||||||
pRestoreGrants IN BOOLEAN DEFAULT TRUE
|
pRestoreGrants IN BOOLEAN DEFAULT TRUE,
|
||||||
|
pArea IN VARCHAR2 DEFAULT 'ALL'
|
||||||
)
|
)
|
||||||
IS
|
IS
|
||||||
vParameters CT_MRDS.A_PROCESS_LOG.PROCEDURE_PARAMETERS%TYPE;
|
vParameters CT_MRDS.A_PROCESS_LOG.PROCEDURE_PARAMETERS%TYPE;
|
||||||
@@ -80,7 +81,8 @@ AS
|
|||||||
SYS.ODCIVARCHAR2LIST(
|
SYS.ODCIVARCHAR2LIST(
|
||||||
'pSourceFileConfigKey => ' || pSourceFileConfigKey,
|
'pSourceFileConfigKey => ' || pSourceFileConfigKey,
|
||||||
'pRecreate => ' || vRecreateStr,
|
'pRecreate => ' || vRecreateStr,
|
||||||
'pRestoreGrants => ' || vRestoreGrantsStr
|
'pRestoreGrants => ' || vRestoreGrantsStr,
|
||||||
|
'pArea => ''' || pArea || ''''
|
||||||
)
|
)
|
||||||
);
|
);
|
||||||
|
|
||||||
@@ -92,7 +94,8 @@ AS
|
|||||||
CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET(
|
CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET(
|
||||||
pSourceFileConfigKey => pSourceFileConfigKey,
|
pSourceFileConfigKey => pSourceFileConfigKey,
|
||||||
pRecreate => pRecreate,
|
pRecreate => pRecreate,
|
||||||
pRestoreGrants => pRestoreGrants
|
pRestoreGrants => pRestoreGrants,
|
||||||
|
pArea => pArea
|
||||||
);
|
);
|
||||||
|
|
||||||
CT_MRDS.ENV_MANAGER.LOG_PROCESS_EVENT('End FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_SET', 'INFO', vParameters);
|
CT_MRDS.ENV_MANAGER.LOG_PROCESS_EVENT('End FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_SET', 'INFO', vParameters);
|
||||||
@@ -117,7 +120,8 @@ AS
|
|||||||
pSourceFileId IN VARCHAR2 DEFAULT NULL,
|
pSourceFileId IN VARCHAR2 DEFAULT NULL,
|
||||||
pTableId IN VARCHAR2 DEFAULT NULL,
|
pTableId IN VARCHAR2 DEFAULT NULL,
|
||||||
pRecreate IN BOOLEAN DEFAULT FALSE,
|
pRecreate IN BOOLEAN DEFAULT FALSE,
|
||||||
pRestoreGrants IN BOOLEAN DEFAULT TRUE
|
pRestoreGrants IN BOOLEAN DEFAULT TRUE,
|
||||||
|
pArea IN VARCHAR2 DEFAULT 'ALL'
|
||||||
)
|
)
|
||||||
IS
|
IS
|
||||||
vParameters CT_MRDS.A_PROCESS_LOG.PROCEDURE_PARAMETERS%TYPE;
|
vParameters CT_MRDS.A_PROCESS_LOG.PROCEDURE_PARAMETERS%TYPE;
|
||||||
@@ -135,7 +139,8 @@ AS
|
|||||||
'pSourceFileId => ''' || NVL(pSourceFileId, 'NULL') || '''',
|
'pSourceFileId => ''' || NVL(pSourceFileId, 'NULL') || '''',
|
||||||
'pTableId => ''' || NVL(pTableId, 'NULL') || '''',
|
'pTableId => ''' || NVL(pTableId, 'NULL') || '''',
|
||||||
'pRecreate => ' || vRecreateStr,
|
'pRecreate => ' || vRecreateStr,
|
||||||
'pRestoreGrants => ' || vRestoreGrantsStr
|
'pRestoreGrants => ' || vRestoreGrantsStr,
|
||||||
|
'pArea => ''' || pArea || ''''
|
||||||
)
|
)
|
||||||
);
|
);
|
||||||
|
|
||||||
@@ -149,7 +154,8 @@ AS
|
|||||||
pSourceFileId => pSourceFileId,
|
pSourceFileId => pSourceFileId,
|
||||||
pTableId => pTableId,
|
pTableId => pTableId,
|
||||||
pRecreate => pRecreate,
|
pRecreate => pRecreate,
|
||||||
pRestoreGrants => pRestoreGrants
|
pRestoreGrants => pRestoreGrants,
|
||||||
|
pArea => pArea
|
||||||
);
|
);
|
||||||
|
|
||||||
CT_MRDS.ENV_MANAGER.LOG_PROCESS_EVENT('End FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_BATCH', 'INFO', vParameters);
|
CT_MRDS.ENV_MANAGER.LOG_PROCESS_EVENT('End FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_BATCH', 'INFO', vParameters);
|
||||||
|
|||||||
@@ -16,12 +16,13 @@ AS
|
|||||||
*/
|
*/
|
||||||
|
|
||||||
-- Package Version Information (Semantic Versioning: MAJOR.MINOR.PATCH)
|
-- Package Version Information (Semantic Versioning: MAJOR.MINOR.PATCH)
|
||||||
PACKAGE_VERSION CONSTANT VARCHAR2(10) := '2.3.0';
|
PACKAGE_VERSION CONSTANT VARCHAR2(10) := '2.4.0';
|
||||||
PACKAGE_BUILD_DATE CONSTANT VARCHAR2(20) := '2026-02-18 13:30:00';
|
PACKAGE_BUILD_DATE CONSTANT VARCHAR2(20) := '2026-02-18 16:00:00';
|
||||||
PACKAGE_AUTHOR CONSTANT VARCHAR2(100) := 'Grzegorz Michalski';
|
PACKAGE_AUTHOR CONSTANT VARCHAR2(100) := 'Grzegorz Michalski';
|
||||||
|
|
||||||
-- Version History (Latest changes first)
|
-- Version History (Latest changes first)
|
||||||
VERSION_HISTORY CONSTANT VARCHAR2(4000) :=
|
VERSION_HISTORY CONSTANT VARCHAR2(4000) :=
|
||||||
|
'2.4.0 (2026-02-18): MARS-1057 - Added pArea parameter for selective table creation (INBOX/ODS/ARCHIVE/ALL)' || CHR(13)||CHR(10) ||
|
||||||
'2.3.0 (2026-02-18): MARS-1057 - Added pRestoreGrants parameter support for grant preservation during table recreate' || CHR(13)||CHR(10) ||
|
'2.3.0 (2026-02-18): MARS-1057 - Added pRestoreGrants parameter support for grant preservation during table recreate' || CHR(13)||CHR(10) ||
|
||||||
'2.2.0 (2025-11-27): MARS-1057 - Added CREATE_EXTERNAL_TABLES_SET and CREATE_EXTERNAL_TABLES_BATCH wrappers for batch external table creation' || CHR(13)||CHR(10) ||
|
'2.2.0 (2025-11-27): MARS-1057 - Added CREATE_EXTERNAL_TABLES_SET and CREATE_EXTERNAL_TABLES_BATCH wrappers for batch external table creation' || CHR(13)||CHR(10) ||
|
||||||
'2.1.0 (2025-11-24): MARS-1049 - Added pEncoding parameter support for CSV character set specification' || CHR(13)||CHR(10) ||
|
'2.1.0 (2025-11-24): MARS-1049 - Added pEncoding parameter support for CSV character set specification' || CHR(13)||CHR(10) ||
|
||||||
@@ -64,38 +65,43 @@ AS
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* @name CREATE_EXTERNAL_TABLES_SET
|
* @name CREATE_EXTERNAL_TABLES_SET
|
||||||
* @desc Creates a complete set of 3 external tables (INBOX, ODS, ARCHIVE) for a single configuration
|
* @desc Creates a complete set of external tables for a single configuration from A_SOURCE_FILE_CONFIG table.
|
||||||
* from A_SOURCE_FILE_CONFIG table. Automatically generates table names and paths following
|
* Automatically generates table names and paths following official path patterns.
|
||||||
* official path patterns. Wrapper for CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET
|
* Wrapper for CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET with DEFINER rights.
|
||||||
* with DEFINER rights to ensure objects are created in ODS schema.
|
|
||||||
* If pRestoreGrants is TRUE, saves and restores table grants during recreate operation.
|
* If pRestoreGrants is TRUE, saves and restores table grants during recreate operation.
|
||||||
|
* The pArea parameter allows selective table creation.
|
||||||
* @param pSourceFileConfigKey - Primary key from A_SOURCE_FILE_CONFIG table
|
* @param pSourceFileConfigKey - Primary key from A_SOURCE_FILE_CONFIG table
|
||||||
* @param pRecreate - If TRUE, drops existing tables before creating new ones; if FALSE, fails if tables exist
|
* @param pRecreate - If TRUE, drops existing tables before creating new ones; if FALSE, fails if tables exist
|
||||||
* @param pRestoreGrants - If TRUE, saves grants before DROP and restores after CREATE (only when pRecreate=TRUE)
|
* @param pRestoreGrants - If TRUE, saves grants before DROP and restores after CREATE (only when pRecreate=TRUE)
|
||||||
* Uses DBA_TAB_PRIVS - requires SELECT ANY DICTIONARY or SELECT ON DBA_TAB_PRIVS privilege
|
* Uses DBA_TAB_PRIVS - requires SELECT ANY DICTIONARY or SELECT ON DBA_TAB_PRIVS privilege
|
||||||
* @example EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_SET(
|
* @param pArea - Specifies which tables to create: 'INBOX', 'ODS', 'ARCHIVE', or 'ALL' (default)
|
||||||
|
* @example -- Create only INBOX table
|
||||||
|
* EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_SET(
|
||||||
|
* pSourceFileConfigKey => 123,
|
||||||
|
* pArea => 'INBOX'
|
||||||
|
* );
|
||||||
|
*
|
||||||
|
* -- Create all tables with grant preservation
|
||||||
|
* EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_SET(
|
||||||
* pSourceFileConfigKey => 123,
|
* pSourceFileConfigKey => 123,
|
||||||
* pRecreate => TRUE,
|
* pRecreate => TRUE,
|
||||||
* pRestoreGrants => TRUE
|
* pRestoreGrants => TRUE,
|
||||||
|
* pArea => 'ALL'
|
||||||
* );
|
* );
|
||||||
* @ex_rslt Creates three external tables in ODS schema:
|
* @ex_rslt Creates external table(s) in ODS schema based on pArea parameter
|
||||||
* - C2D_A_UC_DISSEM_METADATA_LOADS_INBOX
|
|
||||||
* - C2D_A_UC_DISSEM_METADATA_LOADS_ODS
|
|
||||||
* - C2D_A_UC_DISSEM_METADATA_LOADS_ARCHIVE
|
|
||||||
* Preserves existing grants if pRestoreGrants = TRUE
|
|
||||||
*/
|
*/
|
||||||
PROCEDURE CREATE_EXTERNAL_TABLES_SET (
|
PROCEDURE CREATE_EXTERNAL_TABLES_SET (
|
||||||
pSourceFileConfigKey IN NUMBER,
|
pSourceFileConfigKey IN NUMBER,
|
||||||
pRecreate IN BOOLEAN DEFAULT FALSE,
|
pRecreate IN BOOLEAN DEFAULT FALSE,
|
||||||
pRestoreGrants IN BOOLEAN DEFAULT TRUE
|
pRestoreGrants IN BOOLEAN DEFAULT TRUE,
|
||||||
|
pArea IN VARCHAR2 DEFAULT 'ALL'
|
||||||
);
|
);
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @name CREATE_EXTERNAL_TABLES_BATCH
|
* @name CREATE_EXTERNAL_TABLES_BATCH
|
||||||
* @desc Creates external table sets for multiple configurations based on filter criteria.
|
* @desc Creates external table sets for multiple configurations based on filter criteria.
|
||||||
* Processes only INPUT type files from A_SOURCE_FILE_CONFIG. Creates 3 tables per configuration
|
* Processes only INPUT type files from A_SOURCE_FILE_CONFIG. Creates tables based on pArea parameter.
|
||||||
* (INBOX, ODS, ARCHIVE). Wrapper for CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH
|
* Wrapper for CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH with DEFINER rights.
|
||||||
* with DEFINER rights to ensure objects are created in ODS schema.
|
|
||||||
* If pRestoreGrants is TRUE, saves and restores table grants during recreate operations.
|
* If pRestoreGrants is TRUE, saves and restores table grants during recreate operations.
|
||||||
* @param pSourceKey - Filter by A_SOURCE_KEY (NULL = all sources)
|
* @param pSourceKey - Filter by A_SOURCE_KEY (NULL = all sources)
|
||||||
* @param pSourceFileId - Filter by SOURCE_FILE_ID (NULL = all file types)
|
* @param pSourceFileId - Filter by SOURCE_FILE_ID (NULL = all file types)
|
||||||
@@ -103,17 +109,18 @@ AS
|
|||||||
* @param pRecreate - If TRUE, drops and recreates existing tables; if FALSE, skips if tables exist
|
* @param pRecreate - If TRUE, drops and recreates existing tables; if FALSE, skips if tables exist
|
||||||
* @param pRestoreGrants - If TRUE, saves grants before DROP and restores after CREATE (only when pRecreate=TRUE)
|
* @param pRestoreGrants - If TRUE, saves grants before DROP and restores after CREATE (only when pRecreate=TRUE)
|
||||||
* Uses DBA_TAB_PRIVS - requires SELECT ANY DICTIONARY or SELECT ON DBA_TAB_PRIVS privilege
|
* Uses DBA_TAB_PRIVS - requires SELECT ANY DICTIONARY or SELECT ON DBA_TAB_PRIVS privilege
|
||||||
* @example -- Create all external tables for C2D source with grant preservation
|
* @param pArea - Specifies which tables to create: 'INBOX', 'ODS', 'ARCHIVE', or 'ALL' (default)
|
||||||
|
* @example -- Create only INBOX tables for C2D source
|
||||||
* EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_BATCH(
|
* EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_BATCH(
|
||||||
* pSourceKey => 'C2D',
|
* pSourceKey => 'C2D',
|
||||||
* pRecreate => TRUE,
|
* pArea => 'INBOX'
|
||||||
* pRestoreGrants => TRUE
|
|
||||||
* );
|
* );
|
||||||
*
|
*
|
||||||
* -- Recreate all external tables without grant preservation
|
* -- Create all tables with grant preservation
|
||||||
* EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_BATCH(
|
* EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_BATCH(
|
||||||
* pRecreate => TRUE,
|
* pRecreate => TRUE,
|
||||||
* pRestoreGrants => FALSE
|
* pRestoreGrants => TRUE,
|
||||||
|
* pArea => 'ALL'
|
||||||
* );
|
* );
|
||||||
* @ex_rslt Returns summary: Total: 10, Processed: 9, Failed: 1
|
* @ex_rslt Returns summary: Total: 10, Processed: 9, Failed: 1
|
||||||
*/
|
*/
|
||||||
@@ -122,7 +129,8 @@ AS
|
|||||||
pSourceFileId IN VARCHAR2 DEFAULT NULL,
|
pSourceFileId IN VARCHAR2 DEFAULT NULL,
|
||||||
pTableId IN VARCHAR2 DEFAULT NULL,
|
pTableId IN VARCHAR2 DEFAULT NULL,
|
||||||
pRecreate IN BOOLEAN DEFAULT FALSE,
|
pRecreate IN BOOLEAN DEFAULT FALSE,
|
||||||
pRestoreGrants IN BOOLEAN DEFAULT TRUE
|
pRestoreGrants IN BOOLEAN DEFAULT TRUE,
|
||||||
|
pArea IN VARCHAR2 DEFAULT 'ALL'
|
||||||
);
|
);
|
||||||
|
|
||||||
---------------------------------------------------------------------------------------------------------------------------
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
|||||||
@@ -1,9 +1,9 @@
|
|||||||
-- ===================================================================
|
-- ===================================================================
|
||||||
-- MARS-1057 ROLLBACK SCRIPT: Batch External Table Creation
|
-- MARS-1057 ROLLBACK SCRIPT: Batch External Table Creation
|
||||||
-- ===================================================================
|
-- ===================================================================
|
||||||
-- Purpose: Rollback FILE_MANAGER package to v3.3.0 (remove batch external table procedures)
|
-- Purpose: Rollback FILE_MANAGER package to v3.3.0 (remove batch external table procedures and area filter)
|
||||||
-- Author: Grzegorz Michalski
|
-- Author: Grzegorz Michalski
|
||||||
-- Date: 2025-11-27
|
-- Date: 2026-02-18
|
||||||
|
|
||||||
-- Dynamic spool file generation (using SYS_CONTEXT - no DBA privileges required)
|
-- Dynamic spool file generation (using SYS_CONTEXT - no DBA privileges required)
|
||||||
-- Log files are automatically created in log/ subdirectory
|
-- Log files are automatically created in log/ subdirectory
|
||||||
@@ -25,17 +25,19 @@ SET SERVEROUTPUT ON SIZE UNLIMITED
|
|||||||
SET PAUSE OFF
|
SET PAUSE OFF
|
||||||
|
|
||||||
PROMPT =========================================================================
|
PROMPT =========================================================================
|
||||||
PROMPT MARS-1057: Rollback Batch External Table Creation
|
PROMPT MARS-1057: Rollback Batch External Table Creation + Area Filter
|
||||||
PROMPT =========================================================================
|
PROMPT =========================================================================
|
||||||
PROMPT WARNING: This will reverse all changes from MARS-1057 installation!
|
PROMPT WARNING: This will reverse all changes from MARS-1057 installation!
|
||||||
PROMPT
|
PROMPT
|
||||||
PROMPT Changes to be rolled back:
|
PROMPT Changes to be rolled back:
|
||||||
PROMPT - FILE_MANAGER package specification (3.4.0 -> 3.3.0)
|
PROMPT - FILE_MANAGER package specification (3.5.0 -> 3.3.0)
|
||||||
PROMPT - FILE_MANAGER package body (3.4.0 -> 3.3.0)
|
PROMPT - FILE_MANAGER package body (3.5.0 -> 3.3.0)
|
||||||
PROMPT - FILE_MANAGER_ODS package specification (2.2.0 -> 2.1.0)
|
PROMPT - FILE_MANAGER_ODS package specification (2.4.0 -> 2.1.0)
|
||||||
PROMPT - FILE_MANAGER_ODS package body (2.2.0 -> 2.1.0)
|
PROMPT - FILE_MANAGER_ODS package body (2.4.0 -> 2.1.0)
|
||||||
PROMPT - Remove CREATE_EXTERNAL_TABLES_SET procedures
|
PROMPT - Remove CREATE_EXTERNAL_TABLES_SET procedures
|
||||||
PROMPT - Remove CREATE_EXTERNAL_TABLES_BATCH procedures
|
PROMPT - Remove CREATE_EXTERNAL_TABLES_BATCH procedures
|
||||||
|
PROMPT - Remove pArea parameter (selective table creation)
|
||||||
|
PROMPT - Remove pRestoreGrants parameter (grant preservation)
|
||||||
PROMPT =========================================================================
|
PROMPT =========================================================================
|
||||||
|
|
||||||
-- Confirm rollback with user
|
-- Confirm rollback with user
|
||||||
|
|||||||
Reference in New Issue
Block a user