feat(MARS-1057): Upgrade FILE_MANAGER to v3.5.0 with area filter functionality

- Updated install script to reflect new version and features.
- Added pArea parameter to CREATE_EXTERNAL_TABLES_SET and CREATE_EXTERNAL_TABLES_BATCH procedures for selective table creation (INBOX/ODS/ARCHIVE/ALL).
- Enhanced grant preservation logic based on the pArea parameter.
- Updated FILE_MANAGER_ODS package to support new functionality.
- Modified rollback script to remove new features and revert to previous versions.
This commit is contained in:
Grzegorz Michalski
2026-02-18 14:03:11 +01:00
parent 204616252a
commit 2b116c0256
15 changed files with 253 additions and 532 deletions

View File

@@ -1,16 +1,16 @@
--============================================================================================================================= --=============================================================================================================================
-- MARS-1057: Install FILE_MANAGER Package Specification v3.4.0 -- MARS-1057: Install FILE_MANAGER Package Specification v3.5.0
--============================================================================================================================= --=============================================================================================================================
-- Purpose: Deploy FILE_MANAGER package specification with new batch external table creation procedures -- Purpose: Deploy FILE_MANAGER package specification with batch external table creation and area filter functionality
-- Author: Grzegorz Michalski -- Author: Grzegorz Michalski
-- Date: 2025-11-27 -- Date: 2026-02-18
-- Related: MARS-1057 Batch External Table Creation -- Related: MARS-1057 Batch External Table Creation + Area Filter Enhancement
--============================================================================================================================= --=============================================================================================================================
SET SERVEROUTPUT ON SET SERVEROUTPUT ON
PROMPT ======================================================================== PROMPT ========================================================================
PROMPT Installing FILE_MANAGER Package Specification v3.4.0 PROMPT Installing FILE_MANAGER Package Specification v3.5.0
PROMPT ======================================================================== PROMPT ========================================================================
@@new_version/FILE_MANAGER.pkg @@new_version/FILE_MANAGER.pkg

View File

@@ -1,16 +1,16 @@
--============================================================================================================================= --=============================================================================================================================
-- MARS-1057: Install FILE_MANAGER Package Body v3.4.0 -- MARS-1057: Install FILE_MANAGER Package Body v3.5.0
--============================================================================================================================= --=============================================================================================================================
-- Purpose: Deploy FILE_MANAGER package body with implementation of batch external table creation procedures -- Purpose: Deploy FILE_MANAGER package body with batch external table creation and area filter functionality
-- Author: Grzegorz Michalski -- Author: Grzegorz Michalski
-- Date: 2025-11-27 -- Date: 2026-02-18
-- Related: MARS-1057 Batch External Table Creation -- Related: MARS-1057 Batch External Table Creation + Area Filter Enhancement
--============================================================================================================================= --=============================================================================================================================
SET SERVEROUTPUT ON SET SERVEROUTPUT ON
PROMPT ======================================================================== PROMPT ========================================================================
PROMPT Installing FILE_MANAGER Package Body v3.4.0 PROMPT Installing FILE_MANAGER Package Body v3.5.0
PROMPT ======================================================================== PROMPT ========================================================================
@@new_version/FILE_MANAGER.pkb @@new_version/FILE_MANAGER.pkb

View File

@@ -1,17 +1,17 @@
-- =================================================================== -- ===================================================================
-- MARS-1057: Install ODS.FILE_MANAGER_ODS Package Specification -- MARS-1057: Install ODS.FILE_MANAGER_ODS Package Specification
-- =================================================================== -- ===================================================================
-- Purpose: Deploy FILE_MANAGER_ODS package specification v2.2.0 -- Purpose: Deploy FILE_MANAGER_ODS package specification v2.4.0
-- Author: Grzegorz Michalski -- Author: Grzegorz Michalski
-- Date: 2025-11-27 -- Date: 2026-02-18
-- Package: ODS.FILE_MANAGER_ODS -- Package: ODS.FILE_MANAGER_ODS
-- Version: 2.1.0 -> 2.2.0 -- Version: 2.1.0 -> 2.4.0
-- Changes: Added CREATE_EXTERNAL_TABLES_SET and CREATE_EXTERNAL_TABLES_BATCH wrappers -- Changes: Added CREATE_EXTERNAL_TABLES_SET and CREATE_EXTERNAL_TABLES_BATCH wrappers with pArea and pRestoreGrants parameters
SET ECHO ON SET ECHO ON
SET DEFINE OFF SET DEFINE OFF
PROMPT Installing ODS.FILE_MANAGER_ODS Package Specification v2.2.0... PROMPT Installing ODS.FILE_MANAGER_ODS Package Specification v2.4.0...
@@new_version/FILE_MANAGER_ODS.pkg @@new_version/FILE_MANAGER_ODS.pkg

View File

@@ -1,17 +1,17 @@
-- =================================================================== -- ===================================================================
-- MARS-1057: Install ODS.FILE_MANAGER_ODS Package Body -- MARS-1057: Install ODS.FILE_MANAGER_ODS Package Body
-- =================================================================== -- ===================================================================
-- Purpose: Deploy FILE_MANAGER_ODS package body v2.2.0 -- Purpose: Deploy FILE_MANAGER_ODS package body v2.4.0
-- Author: Grzegorz Michalski -- Author: Grzegorz Michalski
-- Date: 2025-11-27 -- Date: 2026-02-18
-- Package: ODS.FILE_MANAGER_ODS -- Package: ODS.FILE_MANAGER_ODS
-- Version: 2.1.0 -> 2.2.0 -- Version: 2.1.0 -> 2.4.0
-- Changes: Added CREATE_EXTERNAL_TABLES_SET and CREATE_EXTERNAL_TABLES_BATCH wrapper implementations -- Changes: Added CREATE_EXTERNAL_TABLES_SET and CREATE_EXTERNAL_TABLES_BATCH wrapper implementations with pArea and pRestoreGrants parameters
SET ECHO ON SET ECHO ON
SET DEFINE OFF SET DEFINE OFF
PROMPT Installing ODS.FILE_MANAGER_ODS Package Body v2.2.0... PROMPT Installing ODS.FILE_MANAGER_ODS Package Body v2.4.0...
@@new_version/FILE_MANAGER_ODS.pkb @@new_version/FILE_MANAGER_ODS.pkb

View File

@@ -1,19 +1,19 @@
--============================================================================================================================= --=============================================================================================================================
-- MARS-1057: Rollback FILE_MANAGER Package Specification to v3.3.0 -- MARS-1057: Rollback FILE_MANAGER Package Specification to v3.3.0
--============================================================================================================================= --=============================================================================================================================
-- Purpose: Restore FILE_MANAGER package specification to version before MARS-1057 changes -- Purpose: Restore FILE_MANAGER package specification to version before MARS-1057 changes (from v3.5.0 to v3.3.0)
-- Author: Grzegorz Michalski -- Author: Grzegorz Michalski
-- Date: 2025-11-27 -- Date: 2026-02-18
-- Related: MARS-1057 Batch External Table Creation (ROLLBACK) -- Related: MARS-1057 Batch External Table Creation + Area Filter (ROLLBACK)
--============================================================================================================================= --=============================================================================================================================
SET SERVEROUTPUT ON SET SERVEROUTPUT ON
PROMPT ======================================================================== PROMPT ========================================================================
PROMPT Rolling back FILE_MANAGER Package Specification to v3.3.0 PROMPT Rolling back FILE_MANAGER Package Specification from v3.5.0 to v3.3.0
PROMPT ======================================================================== PROMPT ========================================================================
@@current_version/FILE_MANAGER.pkg @@rollback_version/FILE_MANAGER.pkg
-- Verify compilation status (check specific schema when installing as ADMIN) -- Verify compilation status (check specific schema when installing as ADMIN)
SELECT object_name, object_type, status SELECT object_name, object_type, status

View File

@@ -1,19 +1,19 @@
--============================================================================================================================= --=============================================================================================================================
-- MARS-1057: Rollback FILE_MANAGER Package Body to v3.3.0 -- MARS-1057: Rollback FILE_MANAGER Package Body to v3.3.0
--============================================================================================================================= --=============================================================================================================================
-- Purpose: Restore FILE_MANAGER package body to version before MARS-1057 changes -- Purpose: Restore FILE_MANAGER package body to version before MARS-1057 changes (from v3.5.0 to v3.3.0)
-- Author: Grzegorz Michalski -- Author: Grzegorz Michalski
-- Date: 2025-11-27 -- Date: 2026-02-18
-- Related: MARS-1057 Batch External Table Creation (ROLLBACK) -- Related: MARS-1057 Batch External Table Creation + Area Filter (ROLLBACK)
--============================================================================================================================= --=============================================================================================================================
SET SERVEROUTPUT ON SET SERVEROUTPUT ON
PROMPT ======================================================================== PROMPT ========================================================================
PROMPT Rolling back FILE_MANAGER Package Body to v3.3.0 PROMPT Rolling back FILE_MANAGER Package Body from v3.5.0 to v3.3.0
PROMPT ======================================================================== PROMPT ========================================================================
@@current_version/FILE_MANAGER.pkb @@rollback_version/FILE_MANAGER.pkb
-- Verify compilation status (check specific schema when installing as ADMIN) -- Verify compilation status (check specific schema when installing as ADMIN)
SELECT object_name, object_type, status SELECT object_name, object_type, status

View File

@@ -3,9 +3,9 @@
-- =================================================================== -- ===================================================================
-- Purpose: Restore FILE_MANAGER_ODS package body to v2.1.0 -- Purpose: Restore FILE_MANAGER_ODS package body to v2.1.0
-- Author: Grzegorz Michalski -- Author: Grzegorz Michalski
-- Date: 2025-11-27 -- Date: 2026-02-18
-- Package: ODS.FILE_MANAGER_ODS -- Package: ODS.FILE_MANAGER_ODS
-- Version: 2.2.0 -> 2.1.0 (rollback) -- Version: 2.4.0 -> 2.1.0 (rollback)
SET ECHO ON SET ECHO ON
SET DEFINE OFF SET DEFINE OFF

View File

@@ -3,9 +3,9 @@
-- =================================================================== -- ===================================================================
-- Purpose: Restore FILE_MANAGER_ODS package specification to v2.1.0 -- Purpose: Restore FILE_MANAGER_ODS package specification to v2.1.0
-- Author: Grzegorz Michalski -- Author: Grzegorz Michalski
-- Date: 2025-11-27 -- Date: 2026-02-18
-- Package: ODS.FILE_MANAGER_ODS -- Package: ODS.FILE_MANAGER_ODS
-- Version: 2.2.0 -> 2.1.0 (rollback) -- Version: 2.4.0 -> 2.1.0 (rollback)
SET ECHO ON SET ECHO ON
SET DEFINE OFF SET DEFINE OFF

View File

@@ -1,376 +0,0 @@
# MARS-1057: Batch External Table Creation
## Overview
This MARS package adds batch external table creation capabilities to the FILE_MANAGER and FILE_MANAGER_ODS packages, enabling automatic creation of external table sets (INBOX, ODS, ARCHIVE) based on A_SOURCE_FILE_CONFIG metadata.
**Jira Issue:** MARS-1057
**Package Version:** FILE_MANAGER 3.4.0, FILE_MANAGER_ODS 2.2.0
**Author:** Grzegorz Michalski
**Date:** 2025-11-27
## Contents
- `install_mars1057.sql` - Master installation script with SPOOL logging
- `rollback_mars1057.sql` - Master rollback script
- `01_MARS_1057_install_CT_MRDS_FILE_MANAGER_SPEC.sql` - Install FILE_MANAGER package specification
- `02_MARS_1057_install_CT_MRDS_FILE_MANAGER_BODY.sql` - Install FILE_MANAGER package body
- `03_MARS_1057_install_ODS_FILE_MANAGER_ODS_SPEC.sql` - Install FILE_MANAGER_ODS package specification
- `04_MARS_1057_install_ODS_FILE_MANAGER_ODS_BODY.sql` - Install FILE_MANAGER_ODS package body
- `91_MARS_1057_rollback_CT_MRDS_FILE_MANAGER_SPEC.sql` - Rollback FILE_MANAGER package specification
- `92_MARS_1057_rollback_CT_MRDS_FILE_MANAGER_BODY.sql` - Rollback FILE_MANAGER package body
- `93_MARS_1057_rollback_ODS_FILE_MANAGER_ODS_BODY.sql` - Rollback FILE_MANAGER_ODS package body
- `94_MARS_1057_rollback_ODS_FILE_MANAGER_ODS_SPEC.sql` - Rollback FILE_MANAGER_ODS package specification
- `track_package_versions.sql` - Universal package version tracking
- `verify_packages_version.sql` - Universal package verification
- `rollback_version/` - FILE_MANAGER v3.3.0, FILE_MANAGER_ODS v2.1.0 (before MARS-1057)
- `new_version/` - FILE_MANAGER v3.4.0, FILE_MANAGER_ODS v2.2.0 (after MARS-1057)
- `.gitignore` - Git exclusions for temporary files
## Prerequisites
- Oracle Database 23ai
- FILE_MANAGER package v3.3.0 installed
- FILE_MANAGER_ODS package v2.1.0 installed
- ENV_MANAGER package v3.1.0+ with version tracking
- ADMIN user access for deployment
- ODS schema with necessary privileges
## New Features
### 1. CREATE_EXTERNAL_TABLES_SET
Creates a complete set of 3 external tables (INBOX, ODS, ARCHIVE) for a single configuration from A_SOURCE_FILE_CONFIG.
**Signature:**
```sql
PROCEDURE CREATE_EXTERNAL_TABLES_SET (
pSourceFileConfigKey IN NUMBER,
pRecreate IN BOOLEAN DEFAULT FALSE
);
```
**Example:**
```sql
BEGIN
FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET(
pSourceFileConfigKey => 123,
pRecreate => FALSE
);
END;
/
```
**Features:**
- Automatic table naming: `{TABLE_ID}_{INBOX|ODS|ARCHIVE}`
- Official path patterns compliance
- Optional drop and recreate
- Full ENV_MANAGER logging
- Error handling with detailed messages
### 2. CREATE_EXTERNAL_TABLES_BATCH
Creates external table sets for multiple configurations based on filter criteria.
**Signature:**
```sql
PROCEDURE CREATE_EXTERNAL_TABLES_BATCH (
pSourceKey IN VARCHAR2 DEFAULT NULL,
pSourceFileId IN VARCHAR2 DEFAULT NULL,
pTableId IN VARCHAR2 DEFAULT NULL,
pRecreate IN BOOLEAN DEFAULT FALSE
);
```
**Examples:**
```sql
-- All external tables for C2D source
BEGIN
FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
pSourceKey => 'C2D'
);
END;
/
-- Recreate all external tables
BEGIN
FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
pRecreate => TRUE
);
END;
/
-- Specific table across all sources
BEGIN
FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
pTableId => 'A_UC_DISSEM_METADATA_LOADS'
);
END;
/
```
**Features:**
- Filters only INPUT type files
- Continues processing on errors
- Returns summary (Total/Processed/Failed)
- Comprehensive logging
### 3. FILE_MANAGER_ODS Wrapper Procedures
The ODS.FILE_MANAGER_ODS package now includes DEFINER rights wrappers for the new batch procedures.
**New Wrappers:**
```sql
-- Create single external table set via ODS wrapper
EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_SET(
pSourceFileConfigKey => 123,
pRecreate => FALSE
);
-- Create batch external tables via ODS wrapper
EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_BATCH(
pSourceKey => 'C2D',
pRecreate => FALSE
);
```
**Benefits:**
- Objects created in ODS schema regardless of execution user
- Same functionality as CT_MRDS.FILE_MANAGER with DEFINER rights
- Full logging and error handling
- Consistent API with original package
## Installation
### Option 1: Master Script (Recommended)
```powershell
# IMPORTANT: Execute as ADMIN user
Get-Content "MARS_Packages/REL03/MARS-1057/install_mars1057.sql" | sql "ADMIN/password@service"
# Log file created: log/INSTALL_MARS_1057_<PDB>_<timestamp>.log
```
**Installation Steps:**
1. Install FILE_MANAGER package specification v3.4.0
2. Install FILE_MANAGER package body v3.4.0
3. Install FILE_MANAGER_ODS package specification v2.2.0
4. Install FILE_MANAGER_ODS package body v2.2.0
5. Track versions in A_PACKAGE_VERSION_TRACKING
6. Verify all tracked packages for untracked changes
### Option 2: Individual Scripts
```powershell
# IMPORTANT: Execute as ADMIN user
Get-Content "01_MARS_1057_install_CT_MRDS_FILE_MANAGER_SPEC.sql" | sql "ADMIN/password@service"
Get-Content "02_MARS_1057_install_CT_MRDS_FILE_MANAGER_BODY.sql" | sql "ADMIN/password@service"
Get-Content "03_MARS_1057_install_ODS_FILE_MANAGER_ODS_SPEC.sql" | sql "ADMIN/password@service"
Get-Content "04_MARS_1057_install_ODS_FILE_MANAGER_ODS_BODY.sql" | sql "ADMIN/password@service"
Get-Content "track_package_versions.sql" | sql "ADMIN/password@service"
Get-Content "verify_packages_version.sql" | sql "ADMIN/password@service"
```
## Verification
```sql
-- Check FILE_MANAGER version
SELECT CT_MRDS.FILE_MANAGER.GET_VERSION() FROM DUAL;
-- Expected: 3.4.0
-- Check FILE_MANAGER_ODS version
SELECT ODS.FILE_MANAGER_ODS.GET_VERSION() FROM DUAL;
-- Expected: 2.2.0
-- Check for errors (ADMIN user checks specific schema)
SELECT * FROM ALL_ERRORS
WHERE OWNER IN ('CT_MRDS', 'ODS')
AND NAME IN ('FILE_MANAGER', 'FILE_MANAGER_ODS');
-- Expected: No rows
-- Verify new FILE_MANAGER procedures exist
SELECT procedure_name
FROM ALL_PROCEDURES
WHERE OWNER = 'CT_MRDS'
AND object_name = 'FILE_MANAGER'
AND procedure_name IN ('CREATE_EXTERNAL_TABLES_SET', 'CREATE_EXTERNAL_TABLES_BATCH');
-- Expected: 2 rows
-- Verify new FILE_MANAGER_ODS procedures exist
SELECT procedure_name
FROM ALL_PROCEDURES
WHERE OWNER = 'ODS'
AND object_name = 'FILE_MANAGER_ODS'
AND procedure_name IN ('CREATE_EXTERNAL_TABLES_SET', 'CREATE_EXTERNAL_TABLES_BATCH');
-- Expected: 2 rows
-- Check for untracked changes
SELECT CT_MRDS.ENV_MANAGER.CHECK_PACKAGE_CHANGES('CT_MRDS', 'FILE_MANAGER') FROM DUAL;
-- Expected: OK: Package CT_MRDS.FILE_MANAGER has not changed.
SELECT CT_MRDS.ENV_MANAGER.CHECK_PACKAGE_CHANGES('ODS', 'FILE_MANAGER_ODS') FROM DUAL;
-- Expected: OK: Package ODS.FILE_MANAGER_ODS has not changed.
```
## Rollback
```powershell
# IMPORTANT: Execute as ADMIN user
Get-Content "MARS_Packages/REL03/MARS-1057/rollback_mars1057.sql" | sql "ADMIN/password@service"
```
**Rollback restores:**
- FILE_MANAGER package specification v3.3.0
- FILE_MANAGER package body v3.3.0
- FILE_MANAGER_ODS package specification v2.1.0
- FILE_MANAGER_ODS package body v2.1.0
- Removes CREATE_EXTERNAL_TABLES_SET procedures from both packages
- Removes CREATE_EXTERNAL_TABLES_BATCH procedures from both packages
## Expected Changes
- **FILE_MANAGER package**: v3.3.0 → v3.4.0
- **FILE_MANAGER_ODS package**: v2.1.0 → v2.2.0
- **New procedures in FILE_MANAGER**: CREATE_EXTERNAL_TABLES_SET, CREATE_EXTERNAL_TABLES_BATCH
- **New procedures in FILE_MANAGER_ODS**: CREATE_EXTERNAL_TABLES_SET, CREATE_EXTERNAL_TABLES_BATCH
- **FILE_MANAGER SPEC size**: +3.3 KB (declaration and documentation)
- **FILE_MANAGER BODY size**: +8.6 KB (implementation with logging)
- **FILE_MANAGER_ODS SPEC size**: +2.8 KB (wrapper declarations)
- **FILE_MANAGER_ODS BODY size**: +4.5 KB (wrapper implementations)
## Testing
### Test 1: Create Single Set (CT_MRDS.FILE_MANAGER)
```sql
BEGIN
CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET(
pSourceFileConfigKey => 123
);
END;
/
-- Verify tables created
SELECT table_name
FROM ALL_TABLES
WHERE OWNER = 'ODS'
AND (table_name LIKE '%_INBOX'
OR table_name LIKE '%_ODS'
OR table_name LIKE '%_ARCHIVE');
```
### Test 2: Create Single Set (ODS.FILE_MANAGER_ODS)
```sql
BEGIN
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_SET(
pSourceFileConfigKey => 123,
pRecreate => TRUE
);
END;
/
```
### Test 3: Batch Creation (CT_MRDS.FILE_MANAGER)
```sql
BEGIN
FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET(
pSourceFileConfigKey => 123
);
END;
/
-- Verify tables created
SELECT table_name
FROM ALL_TABLES
WHERE OWNER = 'ODS'
AND table_name LIKE '%_INBOX'
OR table_name LIKE '%_ODS'
OR table_name LIKE '%_ARCHIVE';
```
### Test 2: Batch Creation
```sql
BEGIN
FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
pSourceKey => 'C2D'
);
END;
/
-- Check process log for results
SELECT *
FROM CT_MRDS.A_PROCESS_LOG
WHERE LOG_TIMESTAMP > SYSDATE - INTERVAL '1' HOUR
AND PROCEDURE_NAME LIKE '%CREATE_EXTERNAL_TABLES%'
ORDER BY LOG_TIMESTAMP DESC;
```
### Test 3: Recreate Existing Tables
```sql
BEGIN
FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET(
pSourceFileConfigKey => 123,
pRecreate => TRUE
);
END;
/
```
## Usage Examples
### Example 1: Setup All External Tables for New Source
```sql
-- 1. Add source configuration
CALL FILE_MANAGER.ADD_SOURCE('LM', 'Liquidity Management');
-- 2. Add file configurations
CALL FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
pSourceKey => 'LM',
pSourceFileType => 'INPUT',
pSourceFileId => 'STANDING_FACILITIES',
pSourceFileDesc => 'Standing Facilities Data',
pSourceFileNamePattern => 'SF_*.csv',
pTableId => 'STANDING_FACILITIES',
pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES',
pEncoding => 'UTF8'
);
-- 3. Create all external tables for this source
BEGIN
FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
pSourceKey => 'LM'
);
END;
/
```
### Example 2: Recreate All External Tables
```sql
-- Useful after bucket URI changes or template table modifications
BEGIN
FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
pRecreate => TRUE
);
END;
/
```
### Example 3: Create Tables for Specific File Type
```sql
BEGIN
FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
pSourceFileId => 'UC_DISSEM'
);
END;
/
```
## Known Issues
None
## Dependencies
- **CT_MRDS.ENV_MANAGER** - Logging and error handling
- **ODS.FILE_MANAGER_ODS** - AUTHID DEFINER wrapper for external table creation
- **CT_MRDS.A_SOURCE_FILE_CONFIG** - Source file configuration metadata
- **CT_ET_TEMPLATES schema** - Template table definitions
## Related
- **MARS-1056** - VARCHAR2 CHAR/BYTE semantics fix
- **MARS-1049** - CSV encoding support
- **Package Deployment Guide** - Standard deployment procedures
- **Tables Setup Guide** - External table configuration guide
## Notes
- All installations must be executed as ADMIN user
- Use `ALL_*` views instead of `USER_*` views for verification
- Master scripts include SPOOL logging for audit trail
- ACCEPT validation prevents accidental execution
- Follows official path patterns: INBOX (3-level), ODS (2-level), ARCHIVE (2-level)

View File

@@ -1,10 +1,11 @@
-- =================================================================== -- ===================================================================
-- MARS-1057 INSTALL SCRIPT: Batch External Table Creation -- MARS-1057 INSTALL SCRIPT: Batch External Table Creation
-- =================================================================== -- ===================================================================
-- Purpose: Install FILE_MANAGER v3.4.0 with batch external table creation procedures -- Purpose: Install FILE_MANAGER v3.5.0 with batch external table creation procedures
-- and area filter functionality
-- Author: Grzegorz Michalski -- Author: Grzegorz Michalski
-- Date: 2025-11-27 -- Date: 2026-02-18
-- Version: 3.4.0 -- Version: 3.5.0
-- Dynamic spool file generation (using SYS_CONTEXT - no DBA privileges required) -- Dynamic spool file generation (using SYS_CONTEXT - no DBA privileges required)
-- Log files are automatically created in log/ subdirectory -- Log files are automatically created in log/ subdirectory
@@ -29,20 +30,28 @@ SET PAUSE OFF
-- ALTER SESSION SET CURRENT_SCHEMA = CT_MRDS; -- ALTER SESSION SET CURRENT_SCHEMA = CT_MRDS;
PROMPT ========================================================================= PROMPT =========================================================================
PROMPT MARS-1057: Batch External Table Creation PROMPT MARS-1057: Batch External Table Creation + Area Filter
PROMPT ========================================================================= PROMPT =========================================================================
PROMPT PROMPT
PROMPT This script will install FILE_MANAGER package v3.4.0 with new features: PROMPT This script will install FILE_MANAGER package v3.5.0 with new features:
PROMPT - CREATE_EXTERNAL_TABLES_SET procedure PROMPT - CREATE_EXTERNAL_TABLES_SET procedure
PROMPT - CREATE_EXTERNAL_TABLES_BATCH procedure PROMPT - CREATE_EXTERNAL_TABLES_BATCH procedure
PROMPT - pArea parameter for selective table creation (INBOX/ODS/ARCHIVE/ALL)
PROMPT - pRestoreGrants parameter for grant preservation during recreate
PROMPT PROMPT
PROMPT And FILE_MANAGER_ODS package v2.2.0 with wrapper procedures. PROMPT And FILE_MANAGER_ODS package v2.4.0 with wrapper procedures.
PROMPT PROMPT
PROMPT Changes: PROMPT Changes:
PROMPT - FILE_MANAGER package specification (3.3.0 -> 3.4.0) PROMPT - FILE_MANAGER package specification (3.3.0 -> 3.5.0)
PROMPT - FILE_MANAGER package body (3.3.0 -> 3.4.0) PROMPT - FILE_MANAGER package body (3.3.0 -> 3.5.0)
PROMPT - FILE_MANAGER_ODS package specification (2.1.0 -> 2.2.0) PROMPT - FILE_MANAGER_ODS package specification (2.1.0 -> 2.4.0)
PROMPT - FILE_MANAGER_ODS package body (2.1.0 -> 2.2.0) PROMPT - FILE_MANAGER_ODS package body (2.1.0 -> 2.4.0)
PROMPT
PROMPT New functionality:
PROMPT - pArea => 'INBOX' : creates only INBOX table
PROMPT - pArea => 'ODS' : creates only ODS table
PROMPT - pArea => 'ARCHIVE' : creates only ARCHIVE table
PROMPT - pArea => 'ALL' : creates all tables (default)
PROMPT PROMPT
PROMPT Expected Duration: 1-2 minutes PROMPT Expected Duration: 1-2 minutes
PROMPT ========================================================================= PROMPT =========================================================================
@@ -60,25 +69,25 @@ WHENEVER SQLERROR CONTINUE
PROMPT PROMPT
PROMPT ========================================================================= PROMPT =========================================================================
PROMPT Step 1: Install FILE_MANAGER Package Specification v3.4.0 PROMPT Step 1: Install FILE_MANAGER Package Specification v3.5.0
PROMPT ========================================================================= PROMPT =========================================================================
@@01_MARS_1057_install_CT_MRDS_FILE_MANAGER_SPEC.sql @@01_MARS_1057_install_CT_MRDS_FILE_MANAGER_SPEC.sql
PROMPT PROMPT
PROMPT ========================================================================= PROMPT =========================================================================
PROMPT Step 2: Install FILE_MANAGER Package Body v3.4.0 PROMPT Step 2: Install FILE_MANAGER Package Body v3.5.0
PROMPT ========================================================================= PROMPT =========================================================================
@@02_MARS_1057_install_CT_MRDS_FILE_MANAGER_BODY.sql @@02_MARS_1057_install_CT_MRDS_FILE_MANAGER_BODY.sql
PROMPT PROMPT
PROMPT ========================================================================= PROMPT =========================================================================
PROMPT Step 3: Install FILE_MANAGER_ODS Package Specification v2.2.0 PROMPT Step 3: Install FILE_MANAGER_ODS Package Specification v2.4.0
PROMPT ========================================================================= PROMPT =========================================================================
@@03_MARS_1057_install_ODS_FILE_MANAGER_ODS_SPEC.sql @@03_MARS_1057_install_ODS_FILE_MANAGER_ODS_SPEC.sql
PROMPT PROMPT
PROMPT ========================================================================= PROMPT =========================================================================
PROMPT Step 4: Install FILE_MANAGER_ODS Package Body v2.2.0 PROMPT Step 4: Install FILE_MANAGER_ODS Package Body v2.4.0
PROMPT ========================================================================= PROMPT =========================================================================
@@04_MARS_1057_install_ODS_FILE_MANAGER_ODS_BODY.sql @@04_MARS_1057_install_ODS_FILE_MANAGER_ODS_BODY.sql

View File

@@ -1969,7 +1969,8 @@ AS
PROCEDURE CREATE_EXTERNAL_TABLES_SET ( PROCEDURE CREATE_EXTERNAL_TABLES_SET (
pSourceFileConfigKey IN NUMBER, pSourceFileConfigKey IN NUMBER,
pRecreate IN BOOLEAN DEFAULT FALSE, pRecreate IN BOOLEAN DEFAULT FALSE,
pRestoreGrants IN BOOLEAN DEFAULT TRUE pRestoreGrants IN BOOLEAN DEFAULT TRUE,
pArea IN VARCHAR2 DEFAULT 'ALL'
) )
IS IS
-- Type for storing grant information -- Type for storing grant information
@@ -2001,6 +2002,7 @@ AS
vTableExists NUMBER; vTableExists NUMBER;
vParameters VARCHAR2(4000); vParameters VARCHAR2(4000);
vAreaUpper VARCHAR2(20);
-- Nested procedure to save table grants before DROP -- Nested procedure to save table grants before DROP
PROCEDURE SAVE_GRANTS(pTableName VARCHAR2, pGrantList OUT tGrantList) IS PROCEDURE SAVE_GRANTS(pTableName VARCHAR2, pGrantList OUT tGrantList) IS
@@ -2104,10 +2106,19 @@ AS
END DROP_IF_EXISTS; END DROP_IF_EXISTS;
BEGIN BEGIN
-- Validate and normalize pArea parameter
vAreaUpper := UPPER(TRIM(pArea));
IF vAreaUpper NOT IN ('INBOX', 'ODS', 'ARCHIVE', 'ALL') THEN
vgMsgTmp := 'Invalid pArea parameter: ''' || pArea || '''. Must be one of: INBOX, ODS, ARCHIVE, ALL';
RAISE_APPLICATION_ERROR(-20010, vgMsgTmp);
END IF;
vParameters := ENV_MANAGER.FORMAT_PARAMETERS(SYS.ODCIVARCHAR2LIST( vParameters := ENV_MANAGER.FORMAT_PARAMETERS(SYS.ODCIVARCHAR2LIST(
'pSourceFileConfigKey => ' || NVL(TO_CHAR(pSourceFileConfigKey), 'NULL'), 'pSourceFileConfigKey => ' || NVL(TO_CHAR(pSourceFileConfigKey), 'NULL'),
'pRecreate => ' || CASE WHEN pRecreate THEN 'TRUE' ELSE 'FALSE' END, 'pRecreate => ' || CASE WHEN pRecreate THEN 'TRUE' ELSE 'FALSE' END,
'pRestoreGrants => ' || CASE WHEN pRestoreGrants THEN 'TRUE' ELSE 'FALSE' END 'pRestoreGrants => ' || CASE WHEN pRestoreGrants THEN 'TRUE' ELSE 'FALSE' END,
'pArea => ''' || vAreaUpper || ''''
)); ));
ENV_MANAGER.LOG_PROCESS_EVENT('Start CREATE_EXTERNAL_TABLES_SET', 'INFO', vParameters); ENV_MANAGER.LOG_PROCESS_EVENT('Start CREATE_EXTERNAL_TABLES_SET', 'INFO', vParameters);
@@ -2140,7 +2151,8 @@ AS
vArchivePrefix := 'ARCHIVE/' || vSourceKey || '/' || vTableId; vArchivePrefix := 'ARCHIVE/' || vSourceKey || '/' || vTableId;
ENV_MANAGER.LOG_PROCESS_EVENT( ENV_MANAGER.LOG_PROCESS_EVENT(
'Creating external tables for: ' || vSourceKey || '/' || vSourceFileId || '/' || vTableId, 'Creating external tables for: ' || vSourceKey || '/' || vSourceFileId || '/' || vTableId ||
' (Area: ' || vAreaUpper || ')',
'INFO' 'INFO'
); );
@@ -2149,60 +2161,93 @@ AS
-- Save grants before dropping tables (if pRestoreGrants = TRUE) -- Save grants before dropping tables (if pRestoreGrants = TRUE)
IF pRestoreGrants THEN IF pRestoreGrants THEN
ENV_MANAGER.LOG_PROCESS_EVENT('Saving grants before dropping tables...', 'INFO'); ENV_MANAGER.LOG_PROCESS_EVENT('Saving grants before dropping tables...', 'INFO');
SAVE_GRANTS(vInboxTableName, vInboxGrants);
SAVE_GRANTS(vOdsTableName, vOdsGrants); IF vAreaUpper IN ('INBOX', 'ALL') THEN
SAVE_GRANTS(vArchiveTableName, vArchiveGrants); SAVE_GRANTS(vInboxTableName, vInboxGrants);
END IF;
IF vAreaUpper IN ('ODS', 'ALL') THEN
SAVE_GRANTS(vOdsTableName, vOdsGrants);
END IF;
IF vAreaUpper IN ('ARCHIVE', 'ALL') THEN
SAVE_GRANTS(vArchiveTableName, vArchiveGrants);
END IF;
END IF; END IF;
-- Drop existing tables -- Drop existing tables based on pArea
DROP_IF_EXISTS(vInboxTableName); IF vAreaUpper IN ('INBOX', 'ALL') THEN
DROP_IF_EXISTS(vOdsTableName); DROP_IF_EXISTS(vInboxTableName);
DROP_IF_EXISTS(vArchiveTableName); END IF;
IF vAreaUpper IN ('ODS', 'ALL') THEN
DROP_IF_EXISTS(vOdsTableName);
END IF;
IF vAreaUpper IN ('ARCHIVE', 'ALL') THEN
DROP_IF_EXISTS(vArchiveTableName);
END IF;
END IF; END IF;
-- 5. Create INBOX external table -- 5. Create INBOX external table (if requested)
ENV_MANAGER.LOG_PROCESS_EVENT('Creating INBOX external table: ' || vInboxTableName, 'INFO'); IF vAreaUpper IN ('INBOX', 'ALL') THEN
CREATE_EXTERNAL_TABLE( ENV_MANAGER.LOG_PROCESS_EVENT('Creating INBOX external table: ' || vInboxTableName, 'INFO');
pTableName => vInboxTableName, CREATE_EXTERNAL_TABLE(
pTemplateTableName => vTemplateTableName, pTableName => vInboxTableName,
pPrefix => vInboxPrefix, pTemplateTableName => vTemplateTableName,
pBucketUri => ENV_MANAGER.gvInboxBucketUri, pPrefix => vInboxPrefix,
pDelimiter => vDelimiter, pBucketUri => ENV_MANAGER.gvInboxBucketUri,
pEncoding => vEncoding pDelimiter => vDelimiter,
); pEncoding => vEncoding
);
END IF;
-- 6. Create ODS external table -- 6. Create ODS external table (if requested)
ENV_MANAGER.LOG_PROCESS_EVENT('Creating ODS external table: ' || vOdsTableName, 'INFO'); IF vAreaUpper IN ('ODS', 'ALL') THEN
CREATE_EXTERNAL_TABLE( ENV_MANAGER.LOG_PROCESS_EVENT('Creating ODS external table: ' || vOdsTableName, 'INFO');
pTableName => vOdsTableName, CREATE_EXTERNAL_TABLE(
pTemplateTableName => vTemplateTableName, pTableName => vOdsTableName,
pPrefix => vOdsPrefix, pTemplateTableName => vTemplateTableName,
pBucketUri => ENV_MANAGER.gvDataBucketUri, pPrefix => vOdsPrefix,
pDelimiter => vDelimiter, pBucketUri => ENV_MANAGER.gvDataBucketUri,
pEncoding => vEncoding pDelimiter => vDelimiter,
); pEncoding => vEncoding
);
END IF;
-- 7. Create ARCHIVE external table -- 7. Create ARCHIVE external table (if requested)
ENV_MANAGER.LOG_PROCESS_EVENT('Creating ARCHIVE external table: ' || vArchiveTableName, 'INFO'); IF vAreaUpper IN ('ARCHIVE', 'ALL') THEN
CREATE_EXTERNAL_TABLE( ENV_MANAGER.LOG_PROCESS_EVENT('Creating ARCHIVE external table: ' || vArchiveTableName, 'INFO');
pTableName => vArchiveTableName, CREATE_EXTERNAL_TABLE(
pTemplateTableName => vTemplateTableName, pTableName => vArchiveTableName,
pPrefix => vArchivePrefix, pTemplateTableName => vTemplateTableName,
pBucketUri => ENV_MANAGER.gvArchiveBucketUri, pPrefix => vArchivePrefix,
pDelimiter => vDelimiter, pBucketUri => ENV_MANAGER.gvArchiveBucketUri,
pEncoding => vEncoding pDelimiter => vDelimiter,
); pEncoding => vEncoding
);
END IF;
-- 8. Restore grants after creating tables (if pRecreate = TRUE and pRestoreGrants = TRUE) -- 8. Restore grants after creating tables (if pRecreate = TRUE and pRestoreGrants = TRUE)
IF pRecreate AND pRestoreGrants THEN IF pRecreate AND pRestoreGrants THEN
ENV_MANAGER.LOG_PROCESS_EVENT('Restoring grants after creating tables...', 'INFO'); ENV_MANAGER.LOG_PROCESS_EVENT('Restoring grants after creating tables...', 'INFO');
RESTORE_GRANTS(vInboxTableName, vInboxGrants);
RESTORE_GRANTS(vOdsTableName, vOdsGrants); IF vAreaUpper IN ('INBOX', 'ALL') THEN
RESTORE_GRANTS(vArchiveTableName, vArchiveGrants); RESTORE_GRANTS(vInboxTableName, vInboxGrants);
END IF;
IF vAreaUpper IN ('ODS', 'ALL') THEN
RESTORE_GRANTS(vOdsTableName, vOdsGrants);
END IF;
IF vAreaUpper IN ('ARCHIVE', 'ALL') THEN
RESTORE_GRANTS(vArchiveTableName, vArchiveGrants);
END IF;
END IF; END IF;
ENV_MANAGER.LOG_PROCESS_EVENT( ENV_MANAGER.LOG_PROCESS_EVENT(
'End CREATE_EXTERNAL_TABLES_SET - Successfully created all 3 external tables for config: ' || pSourceFileConfigKey, 'End CREATE_EXTERNAL_TABLES_SET - Successfully created external tables for config: ' ||
pSourceFileConfigKey || ' (Area: ' || vAreaUpper || ')',
'INFO', 'INFO',
vParameters vParameters
); );
@@ -2221,20 +2266,31 @@ AS
pSourceFileId IN VARCHAR2 DEFAULT NULL, pSourceFileId IN VARCHAR2 DEFAULT NULL,
pTableId IN VARCHAR2 DEFAULT NULL, pTableId IN VARCHAR2 DEFAULT NULL,
pRecreate IN BOOLEAN DEFAULT FALSE, pRecreate IN BOOLEAN DEFAULT FALSE,
pRestoreGrants IN BOOLEAN DEFAULT TRUE pRestoreGrants IN BOOLEAN DEFAULT TRUE,
pArea IN VARCHAR2 DEFAULT 'ALL'
) )
IS IS
vCount NUMBER := 0; vCount NUMBER := 0;
vProcessed NUMBER := 0; vProcessed NUMBER := 0;
vFailed NUMBER := 0; vFailed NUMBER := 0;
vParameters VARCHAR2(4000); vParameters VARCHAR2(4000);
vAreaUpper VARCHAR2(20);
BEGIN BEGIN
-- Validate and normalize pArea parameter
vAreaUpper := UPPER(TRIM(pArea));
IF vAreaUpper NOT IN ('INBOX', 'ODS', 'ARCHIVE', 'ALL') THEN
vgMsgTmp := 'Invalid pArea parameter: ''' || pArea || '''. Must be one of: INBOX, ODS, ARCHIVE, ALL';
RAISE_APPLICATION_ERROR(-20010, vgMsgTmp);
END IF;
vParameters := ENV_MANAGER.FORMAT_PARAMETERS(SYS.ODCIVARCHAR2LIST( vParameters := ENV_MANAGER.FORMAT_PARAMETERS(SYS.ODCIVARCHAR2LIST(
'pSourceKey => ''' || NVL(pSourceKey, 'NULL') || '''', 'pSourceKey => ''' || NVL(pSourceKey, 'NULL') || '''',
'pSourceFileId => ''' || NVL(pSourceFileId, 'NULL') || '''', 'pSourceFileId => ''' || NVL(pSourceFileId, 'NULL') || '''',
'pTableId => ''' || NVL(pTableId, 'NULL') || '''', 'pTableId => ''' || NVL(pTableId, 'NULL') || '''',
'pRecreate => ' || CASE WHEN pRecreate THEN 'TRUE' ELSE 'FALSE' END, 'pRecreate => ' || CASE WHEN pRecreate THEN 'TRUE' ELSE 'FALSE' END,
'pRestoreGrants => ' || CASE WHEN pRestoreGrants THEN 'TRUE' ELSE 'FALSE' END 'pRestoreGrants => ' || CASE WHEN pRestoreGrants THEN 'TRUE' ELSE 'FALSE' END,
'pArea => ''' || vAreaUpper || ''''
)); ));
ENV_MANAGER.LOG_PROCESS_EVENT('Start CREATE_EXTERNAL_TABLES_BATCH', 'INFO', vParameters); ENV_MANAGER.LOG_PROCESS_EVENT('Start CREATE_EXTERNAL_TABLES_BATCH', 'INFO', vParameters);
@@ -2254,15 +2310,16 @@ AS
BEGIN BEGIN
ENV_MANAGER.LOG_PROCESS_EVENT( ENV_MANAGER.LOG_PROCESS_EVENT(
'Creating external tables set for: ' || rec.A_SOURCE_KEY || '/' || 'Creating external tables set for: ' || rec.A_SOURCE_KEY || '/' ||
rec.SOURCE_FILE_ID || '/' || rec.TABLE_ID, rec.SOURCE_FILE_ID || '/' || rec.TABLE_ID || ' (Area: ' || vAreaUpper || ')',
'INFO' 'INFO'
); );
-- Call procedure to create set of 3 tables -- Call procedure to create set of tables (based on pArea)
CREATE_EXTERNAL_TABLES_SET( CREATE_EXTERNAL_TABLES_SET(
pSourceFileConfigKey => rec.A_SOURCE_FILE_CONFIG_KEY, pSourceFileConfigKey => rec.A_SOURCE_FILE_CONFIG_KEY,
pRecreate => pRecreate, pRecreate => pRecreate,
pRestoreGrants => pRestoreGrants pRestoreGrants => pRestoreGrants,
pArea => vAreaUpper
); );
vProcessed := vProcessed + 1; vProcessed := vProcessed + 1;
@@ -2281,7 +2338,7 @@ AS
ENV_MANAGER.LOG_PROCESS_EVENT( ENV_MANAGER.LOG_PROCESS_EVENT(
'End CREATE_EXTERNAL_TABLES_BATCH - Total: ' || vCount || 'End CREATE_EXTERNAL_TABLES_BATCH - Total: ' || vCount ||
', Processed: ' || vProcessed || ', Failed: ' || vFailed, ', Processed: ' || vProcessed || ', Failed: ' || vFailed || ' (Area: ' || vAreaUpper || ')',
'INFO', 'INFO',
vParameters vParameters
); );

View File

@@ -17,12 +17,13 @@ AS
**/ **/
-- Package Version Information (Semantic Versioning: MAJOR.MINOR.PATCH) -- Package Version Information (Semantic Versioning: MAJOR.MINOR.PATCH)
PACKAGE_VERSION CONSTANT VARCHAR2(10) := '3.4.0'; PACKAGE_VERSION CONSTANT VARCHAR2(10) := '3.5.0';
PACKAGE_BUILD_DATE CONSTANT VARCHAR2(20) := '2025-11-27 14:00:00'; PACKAGE_BUILD_DATE CONSTANT VARCHAR2(20) := '2026-02-18 16:00:00';
PACKAGE_AUTHOR CONSTANT VARCHAR2(100) := 'Grzegorz Michalski'; PACKAGE_AUTHOR CONSTANT VARCHAR2(100) := 'Grzegorz Michalski';
-- Version History (Latest changes first) -- Version History (Latest changes first)
VERSION_HISTORY CONSTANT VARCHAR2(4000) := VERSION_HISTORY CONSTANT VARCHAR2(4000) :=
'3.5.0 (2026-02-18): MARS-1057 - Added pArea parameter for selective table creation (INBOX/ODS/ARCHIVE/ALL)' || CHR(13)||CHR(10) ||
'3.4.0 (2025-11-27): MARS-1057 - Added CREATE_EXTERNAL_TABLES_SET and CREATE_EXTERNAL_TABLES_BATCH procedures for batch external table creation' || CHR(13)||CHR(10) || '3.4.0 (2025-11-27): MARS-1057 - Added CREATE_EXTERNAL_TABLES_SET and CREATE_EXTERNAL_TABLES_BATCH procedures for batch external table creation' || CHR(13)||CHR(10) ||
'3.3.1 (2025-11-27): MARS-1046 - Fixed ISO 8601 datetime format parsing with milliseconds and timezone (e.g., 2012-03-02T14:16:23.798+01:00)' || CHR(13)||CHR(10) || '3.3.1 (2025-11-27): MARS-1046 - Fixed ISO 8601 datetime format parsing with milliseconds and timezone (e.g., 2012-03-02T14:16:23.798+01:00)' || CHR(13)||CHR(10) ||
'3.3.0 (2025-11-26): MARS-1056 - Fixed VARCHAR2 definitions in GENERATE_EXTERNAL_TABLE_PARAMS to preserve CHAR/BYTE semantics from template tables' || CHR(13)||CHR(10) || '3.3.0 (2025-11-26): MARS-1056 - Fixed VARCHAR2 definitions in GENERATE_EXTERNAL_TABLE_PARAMS to preserve CHAR/BYTE semantics from template tables' || CHR(13)||CHR(10) ||
@@ -602,38 +603,50 @@ AS
/** /**
* @name CREATE_EXTERNAL_TABLES_SET * @name CREATE_EXTERNAL_TABLES_SET
* @desc Creates a complete set of 3 external tables (INBOX, ODS, ARCHIVE) for a single configuration * @desc Creates a complete set of external tables for a single configuration from A_SOURCE_FILE_CONFIG table.
* from A_SOURCE_FILE_CONFIG table. Automatically generates table names and paths following * Automatically generates table names and paths following official path patterns.
* official path patterns. Optionally drops and recreates existing tables. * Optionally drops and recreates existing tables. If pRestoreGrants is TRUE, saves and restores table grants.
* If pRestoreGrants is TRUE, saves and restores table grants during recreate operation. * The pArea parameter allows selective table creation.
* @param pSourceFileConfigKey - Primary key from A_SOURCE_FILE_CONFIG table * @param pSourceFileConfigKey - Primary key from A_SOURCE_FILE_CONFIG table
* @param pRecreate - If TRUE, drops existing tables before creating new ones; if FALSE, fails if tables exist * @param pRecreate - If TRUE, drops existing tables before creating new ones; if FALSE, fails if tables exist
* @param pRestoreGrants - If TRUE, saves grants before DROP and restores after CREATE (only when pRecreate=TRUE) * @param pRestoreGrants - If TRUE, saves grants before DROP and restores after CREATE (only when pRecreate=TRUE)
* Uses DBA_TAB_PRIVS - requires SELECT ANY DICTIONARY or SELECT ON DBA_TAB_PRIVS privilege * Uses DBA_TAB_PRIVS - requires SELECT ANY DICTIONARY or SELECT ON DBA_TAB_PRIVS privilege
* @example BEGIN * @param pArea - Specifies which tables to create: 'INBOX', 'ODS', 'ARCHIVE', or 'ALL' (default)
* 'INBOX' - creates only INBOX table
* 'ODS' - creates only ODS table
* 'ARCHIVE' - creates only ARCHIVE table
* 'ALL' - creates all three tables (default)
* @example -- Create only INBOX table
* BEGIN
* FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET(
* pSourceFileConfigKey => 123,
* pArea => 'INBOX'
* );
* END;
*
* -- Create all tables with grant preservation
* BEGIN
* FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET( * FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET(
* pSourceFileConfigKey => 123, * pSourceFileConfigKey => 123,
* pRecreate => TRUE, * pRecreate => TRUE,
* pRestoreGrants => TRUE * pRestoreGrants => TRUE,
* pArea => 'ALL'
* ); * );
* END; * END;
* @ex_rslt Creates three external tables in ODS schema: * @ex_rslt Creates external table(s) in ODS schema based on pArea parameter
* - C2D_A_UC_DISSEM_METADATA_LOADS_INBOX
* - C2D_A_UC_DISSEM_METADATA_LOADS_ODS
* - C2D_A_UC_DISSEM_METADATA_LOADS_ARCHIVE
* Preserves existing grants if pRestoreGrants = TRUE
**/ **/
PROCEDURE CREATE_EXTERNAL_TABLES_SET ( PROCEDURE CREATE_EXTERNAL_TABLES_SET (
pSourceFileConfigKey IN NUMBER, pSourceFileConfigKey IN NUMBER,
pRecreate IN BOOLEAN DEFAULT FALSE, pRecreate IN BOOLEAN DEFAULT FALSE,
pRestoreGrants IN BOOLEAN DEFAULT TRUE pRestoreGrants IN BOOLEAN DEFAULT TRUE,
pArea IN VARCHAR2 DEFAULT 'ALL'
); );
/** /**
* @name CREATE_EXTERNAL_TABLES_BATCH * @name CREATE_EXTERNAL_TABLES_BATCH
* @desc Creates external table sets for multiple configurations based on filter criteria. * @desc Creates external table sets for multiple configurations based on filter criteria.
* Processes only INPUT type files from A_SOURCE_FILE_CONFIG. Creates 3 tables per configuration * Processes only INPUT type files from A_SOURCE_FILE_CONFIG. Creates tables based on pArea parameter
* (INBOX, ODS, ARCHIVE). Continues processing even if individual sets fail. * (INBOX, ODS, ARCHIVE, or ALL). Continues processing even if individual sets fail.
* If pRestoreGrants is TRUE, saves and restores table grants during recreate operations. * If pRestoreGrants is TRUE, saves and restores table grants during recreate operations.
* @param pSourceKey - Filter by A_SOURCE_KEY (NULL = all sources) * @param pSourceKey - Filter by A_SOURCE_KEY (NULL = all sources)
* @param pSourceFileId - Filter by SOURCE_FILE_ID (NULL = all file types) * @param pSourceFileId - Filter by SOURCE_FILE_ID (NULL = all file types)
@@ -641,20 +654,21 @@ AS
* @param pRecreate - If TRUE, drops and recreates existing tables; if FALSE, skips if tables exist * @param pRecreate - If TRUE, drops and recreates existing tables; if FALSE, skips if tables exist
* @param pRestoreGrants - If TRUE, saves grants before DROP and restores after CREATE (only when pRecreate=TRUE) * @param pRestoreGrants - If TRUE, saves grants before DROP and restores after CREATE (only when pRecreate=TRUE)
* Uses DBA_TAB_PRIVS - requires SELECT ANY DICTIONARY or SELECT ON DBA_TAB_PRIVS privilege * Uses DBA_TAB_PRIVS - requires SELECT ANY DICTIONARY or SELECT ON DBA_TAB_PRIVS privilege
* @example -- Create all external tables for C2D source with grant preservation * @param pArea - Specifies which tables to create: 'INBOX', 'ODS', 'ARCHIVE', or 'ALL' (default)
* @example -- Create only INBOX tables for C2D source
* BEGIN * BEGIN
* FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH( * FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
* pSourceKey => 'C2D', * pSourceKey => 'C2D',
* pRecreate => TRUE, * pArea => 'INBOX'
* pRestoreGrants => TRUE
* ); * );
* END; * END;
* *
* -- Recreate all external tables without grant preservation * -- Create all external tables for all sources with grant preservation
* BEGIN * BEGIN
* FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH( * FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH(
* pRecreate => TRUE, * pRecreate => TRUE,
* pRestoreGrants => FALSE * pRestoreGrants => TRUE,
* pArea => 'ALL'
* ); * );
* END; * END;
* @ex_rslt Returns summary: Total: 10, Processed: 9, Failed: 1 * @ex_rslt Returns summary: Total: 10, Processed: 9, Failed: 1
@@ -664,7 +678,8 @@ AS
pSourceFileId IN VARCHAR2 DEFAULT NULL, pSourceFileId IN VARCHAR2 DEFAULT NULL,
pTableId IN VARCHAR2 DEFAULT NULL, pTableId IN VARCHAR2 DEFAULT NULL,
pRecreate IN BOOLEAN DEFAULT FALSE, pRecreate IN BOOLEAN DEFAULT FALSE,
pRestoreGrants IN BOOLEAN DEFAULT TRUE pRestoreGrants IN BOOLEAN DEFAULT TRUE,
pArea IN VARCHAR2 DEFAULT 'ALL'
); );
--------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------

View File

@@ -64,7 +64,8 @@ AS
PROCEDURE CREATE_EXTERNAL_TABLES_SET ( PROCEDURE CREATE_EXTERNAL_TABLES_SET (
pSourceFileConfigKey IN NUMBER, pSourceFileConfigKey IN NUMBER,
pRecreate IN BOOLEAN DEFAULT FALSE, pRecreate IN BOOLEAN DEFAULT FALSE,
pRestoreGrants IN BOOLEAN DEFAULT TRUE pRestoreGrants IN BOOLEAN DEFAULT TRUE,
pArea IN VARCHAR2 DEFAULT 'ALL'
) )
IS IS
vParameters CT_MRDS.A_PROCESS_LOG.PROCEDURE_PARAMETERS%TYPE; vParameters CT_MRDS.A_PROCESS_LOG.PROCEDURE_PARAMETERS%TYPE;
@@ -80,7 +81,8 @@ AS
SYS.ODCIVARCHAR2LIST( SYS.ODCIVARCHAR2LIST(
'pSourceFileConfigKey => ' || pSourceFileConfigKey, 'pSourceFileConfigKey => ' || pSourceFileConfigKey,
'pRecreate => ' || vRecreateStr, 'pRecreate => ' || vRecreateStr,
'pRestoreGrants => ' || vRestoreGrantsStr 'pRestoreGrants => ' || vRestoreGrantsStr,
'pArea => ''' || pArea || ''''
) )
); );
@@ -92,7 +94,8 @@ AS
CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET( CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET(
pSourceFileConfigKey => pSourceFileConfigKey, pSourceFileConfigKey => pSourceFileConfigKey,
pRecreate => pRecreate, pRecreate => pRecreate,
pRestoreGrants => pRestoreGrants pRestoreGrants => pRestoreGrants,
pArea => pArea
); );
CT_MRDS.ENV_MANAGER.LOG_PROCESS_EVENT('End FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_SET', 'INFO', vParameters); CT_MRDS.ENV_MANAGER.LOG_PROCESS_EVENT('End FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_SET', 'INFO', vParameters);
@@ -117,7 +120,8 @@ AS
pSourceFileId IN VARCHAR2 DEFAULT NULL, pSourceFileId IN VARCHAR2 DEFAULT NULL,
pTableId IN VARCHAR2 DEFAULT NULL, pTableId IN VARCHAR2 DEFAULT NULL,
pRecreate IN BOOLEAN DEFAULT FALSE, pRecreate IN BOOLEAN DEFAULT FALSE,
pRestoreGrants IN BOOLEAN DEFAULT TRUE pRestoreGrants IN BOOLEAN DEFAULT TRUE,
pArea IN VARCHAR2 DEFAULT 'ALL'
) )
IS IS
vParameters CT_MRDS.A_PROCESS_LOG.PROCEDURE_PARAMETERS%TYPE; vParameters CT_MRDS.A_PROCESS_LOG.PROCEDURE_PARAMETERS%TYPE;
@@ -135,7 +139,8 @@ AS
'pSourceFileId => ''' || NVL(pSourceFileId, 'NULL') || '''', 'pSourceFileId => ''' || NVL(pSourceFileId, 'NULL') || '''',
'pTableId => ''' || NVL(pTableId, 'NULL') || '''', 'pTableId => ''' || NVL(pTableId, 'NULL') || '''',
'pRecreate => ' || vRecreateStr, 'pRecreate => ' || vRecreateStr,
'pRestoreGrants => ' || vRestoreGrantsStr 'pRestoreGrants => ' || vRestoreGrantsStr,
'pArea => ''' || pArea || ''''
) )
); );
@@ -149,7 +154,8 @@ AS
pSourceFileId => pSourceFileId, pSourceFileId => pSourceFileId,
pTableId => pTableId, pTableId => pTableId,
pRecreate => pRecreate, pRecreate => pRecreate,
pRestoreGrants => pRestoreGrants pRestoreGrants => pRestoreGrants,
pArea => pArea
); );
CT_MRDS.ENV_MANAGER.LOG_PROCESS_EVENT('End FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_BATCH', 'INFO', vParameters); CT_MRDS.ENV_MANAGER.LOG_PROCESS_EVENT('End FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_BATCH', 'INFO', vParameters);

View File

@@ -16,12 +16,13 @@ AS
*/ */
-- Package Version Information (Semantic Versioning: MAJOR.MINOR.PATCH) -- Package Version Information (Semantic Versioning: MAJOR.MINOR.PATCH)
PACKAGE_VERSION CONSTANT VARCHAR2(10) := '2.3.0'; PACKAGE_VERSION CONSTANT VARCHAR2(10) := '2.4.0';
PACKAGE_BUILD_DATE CONSTANT VARCHAR2(20) := '2026-02-18 13:30:00'; PACKAGE_BUILD_DATE CONSTANT VARCHAR2(20) := '2026-02-18 16:00:00';
PACKAGE_AUTHOR CONSTANT VARCHAR2(100) := 'Grzegorz Michalski'; PACKAGE_AUTHOR CONSTANT VARCHAR2(100) := 'Grzegorz Michalski';
-- Version History (Latest changes first) -- Version History (Latest changes first)
VERSION_HISTORY CONSTANT VARCHAR2(4000) := VERSION_HISTORY CONSTANT VARCHAR2(4000) :=
'2.4.0 (2026-02-18): MARS-1057 - Added pArea parameter for selective table creation (INBOX/ODS/ARCHIVE/ALL)' || CHR(13)||CHR(10) ||
'2.3.0 (2026-02-18): MARS-1057 - Added pRestoreGrants parameter support for grant preservation during table recreate' || CHR(13)||CHR(10) || '2.3.0 (2026-02-18): MARS-1057 - Added pRestoreGrants parameter support for grant preservation during table recreate' || CHR(13)||CHR(10) ||
'2.2.0 (2025-11-27): MARS-1057 - Added CREATE_EXTERNAL_TABLES_SET and CREATE_EXTERNAL_TABLES_BATCH wrappers for batch external table creation' || CHR(13)||CHR(10) || '2.2.0 (2025-11-27): MARS-1057 - Added CREATE_EXTERNAL_TABLES_SET and CREATE_EXTERNAL_TABLES_BATCH wrappers for batch external table creation' || CHR(13)||CHR(10) ||
'2.1.0 (2025-11-24): MARS-1049 - Added pEncoding parameter support for CSV character set specification' || CHR(13)||CHR(10) || '2.1.0 (2025-11-24): MARS-1049 - Added pEncoding parameter support for CSV character set specification' || CHR(13)||CHR(10) ||
@@ -64,38 +65,43 @@ AS
/** /**
* @name CREATE_EXTERNAL_TABLES_SET * @name CREATE_EXTERNAL_TABLES_SET
* @desc Creates a complete set of 3 external tables (INBOX, ODS, ARCHIVE) for a single configuration * @desc Creates a complete set of external tables for a single configuration from A_SOURCE_FILE_CONFIG table.
* from A_SOURCE_FILE_CONFIG table. Automatically generates table names and paths following * Automatically generates table names and paths following official path patterns.
* official path patterns. Wrapper for CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET * Wrapper for CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLES_SET with DEFINER rights.
* with DEFINER rights to ensure objects are created in ODS schema.
* If pRestoreGrants is TRUE, saves and restores table grants during recreate operation. * If pRestoreGrants is TRUE, saves and restores table grants during recreate operation.
* The pArea parameter allows selective table creation.
* @param pSourceFileConfigKey - Primary key from A_SOURCE_FILE_CONFIG table * @param pSourceFileConfigKey - Primary key from A_SOURCE_FILE_CONFIG table
* @param pRecreate - If TRUE, drops existing tables before creating new ones; if FALSE, fails if tables exist * @param pRecreate - If TRUE, drops existing tables before creating new ones; if FALSE, fails if tables exist
* @param pRestoreGrants - If TRUE, saves grants before DROP and restores after CREATE (only when pRecreate=TRUE) * @param pRestoreGrants - If TRUE, saves grants before DROP and restores after CREATE (only when pRecreate=TRUE)
* Uses DBA_TAB_PRIVS - requires SELECT ANY DICTIONARY or SELECT ON DBA_TAB_PRIVS privilege * Uses DBA_TAB_PRIVS - requires SELECT ANY DICTIONARY or SELECT ON DBA_TAB_PRIVS privilege
* @example EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_SET( * @param pArea - Specifies which tables to create: 'INBOX', 'ODS', 'ARCHIVE', or 'ALL' (default)
* @example -- Create only INBOX table
* EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_SET(
* pSourceFileConfigKey => 123,
* pArea => 'INBOX'
* );
*
* -- Create all tables with grant preservation
* EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_SET(
* pSourceFileConfigKey => 123, * pSourceFileConfigKey => 123,
* pRecreate => TRUE, * pRecreate => TRUE,
* pRestoreGrants => TRUE * pRestoreGrants => TRUE,
* pArea => 'ALL'
* ); * );
* @ex_rslt Creates three external tables in ODS schema: * @ex_rslt Creates external table(s) in ODS schema based on pArea parameter
* - C2D_A_UC_DISSEM_METADATA_LOADS_INBOX
* - C2D_A_UC_DISSEM_METADATA_LOADS_ODS
* - C2D_A_UC_DISSEM_METADATA_LOADS_ARCHIVE
* Preserves existing grants if pRestoreGrants = TRUE
*/ */
PROCEDURE CREATE_EXTERNAL_TABLES_SET ( PROCEDURE CREATE_EXTERNAL_TABLES_SET (
pSourceFileConfigKey IN NUMBER, pSourceFileConfigKey IN NUMBER,
pRecreate IN BOOLEAN DEFAULT FALSE, pRecreate IN BOOLEAN DEFAULT FALSE,
pRestoreGrants IN BOOLEAN DEFAULT TRUE pRestoreGrants IN BOOLEAN DEFAULT TRUE,
pArea IN VARCHAR2 DEFAULT 'ALL'
); );
/** /**
* @name CREATE_EXTERNAL_TABLES_BATCH * @name CREATE_EXTERNAL_TABLES_BATCH
* @desc Creates external table sets for multiple configurations based on filter criteria. * @desc Creates external table sets for multiple configurations based on filter criteria.
* Processes only INPUT type files from A_SOURCE_FILE_CONFIG. Creates 3 tables per configuration * Processes only INPUT type files from A_SOURCE_FILE_CONFIG. Creates tables based on pArea parameter.
* (INBOX, ODS, ARCHIVE). Wrapper for CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH * Wrapper for CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLES_BATCH with DEFINER rights.
* with DEFINER rights to ensure objects are created in ODS schema.
* If pRestoreGrants is TRUE, saves and restores table grants during recreate operations. * If pRestoreGrants is TRUE, saves and restores table grants during recreate operations.
* @param pSourceKey - Filter by A_SOURCE_KEY (NULL = all sources) * @param pSourceKey - Filter by A_SOURCE_KEY (NULL = all sources)
* @param pSourceFileId - Filter by SOURCE_FILE_ID (NULL = all file types) * @param pSourceFileId - Filter by SOURCE_FILE_ID (NULL = all file types)
@@ -103,17 +109,18 @@ AS
* @param pRecreate - If TRUE, drops and recreates existing tables; if FALSE, skips if tables exist * @param pRecreate - If TRUE, drops and recreates existing tables; if FALSE, skips if tables exist
* @param pRestoreGrants - If TRUE, saves grants before DROP and restores after CREATE (only when pRecreate=TRUE) * @param pRestoreGrants - If TRUE, saves grants before DROP and restores after CREATE (only when pRecreate=TRUE)
* Uses DBA_TAB_PRIVS - requires SELECT ANY DICTIONARY or SELECT ON DBA_TAB_PRIVS privilege * Uses DBA_TAB_PRIVS - requires SELECT ANY DICTIONARY or SELECT ON DBA_TAB_PRIVS privilege
* @example -- Create all external tables for C2D source with grant preservation * @param pArea - Specifies which tables to create: 'INBOX', 'ODS', 'ARCHIVE', or 'ALL' (default)
* @example -- Create only INBOX tables for C2D source
* EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_BATCH( * EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_BATCH(
* pSourceKey => 'C2D', * pSourceKey => 'C2D',
* pRecreate => TRUE, * pArea => 'INBOX'
* pRestoreGrants => TRUE
* ); * );
* *
* -- Recreate all external tables without grant preservation * -- Create all tables with grant preservation
* EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_BATCH( * EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLES_BATCH(
* pRecreate => TRUE, * pRecreate => TRUE,
* pRestoreGrants => FALSE * pRestoreGrants => TRUE,
* pArea => 'ALL'
* ); * );
* @ex_rslt Returns summary: Total: 10, Processed: 9, Failed: 1 * @ex_rslt Returns summary: Total: 10, Processed: 9, Failed: 1
*/ */
@@ -122,7 +129,8 @@ AS
pSourceFileId IN VARCHAR2 DEFAULT NULL, pSourceFileId IN VARCHAR2 DEFAULT NULL,
pTableId IN VARCHAR2 DEFAULT NULL, pTableId IN VARCHAR2 DEFAULT NULL,
pRecreate IN BOOLEAN DEFAULT FALSE, pRecreate IN BOOLEAN DEFAULT FALSE,
pRestoreGrants IN BOOLEAN DEFAULT TRUE pRestoreGrants IN BOOLEAN DEFAULT TRUE,
pArea IN VARCHAR2 DEFAULT 'ALL'
); );
--------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------

View File

@@ -1,9 +1,9 @@
-- =================================================================== -- ===================================================================
-- MARS-1057 ROLLBACK SCRIPT: Batch External Table Creation -- MARS-1057 ROLLBACK SCRIPT: Batch External Table Creation
-- =================================================================== -- ===================================================================
-- Purpose: Rollback FILE_MANAGER package to v3.3.0 (remove batch external table procedures) -- Purpose: Rollback FILE_MANAGER package to v3.3.0 (remove batch external table procedures and area filter)
-- Author: Grzegorz Michalski -- Author: Grzegorz Michalski
-- Date: 2025-11-27 -- Date: 2026-02-18
-- Dynamic spool file generation (using SYS_CONTEXT - no DBA privileges required) -- Dynamic spool file generation (using SYS_CONTEXT - no DBA privileges required)
-- Log files are automatically created in log/ subdirectory -- Log files are automatically created in log/ subdirectory
@@ -25,17 +25,19 @@ SET SERVEROUTPUT ON SIZE UNLIMITED
SET PAUSE OFF SET PAUSE OFF
PROMPT ========================================================================= PROMPT =========================================================================
PROMPT MARS-1057: Rollback Batch External Table Creation PROMPT MARS-1057: Rollback Batch External Table Creation + Area Filter
PROMPT ========================================================================= PROMPT =========================================================================
PROMPT WARNING: This will reverse all changes from MARS-1057 installation! PROMPT WARNING: This will reverse all changes from MARS-1057 installation!
PROMPT PROMPT
PROMPT Changes to be rolled back: PROMPT Changes to be rolled back:
PROMPT - FILE_MANAGER package specification (3.4.0 -> 3.3.0) PROMPT - FILE_MANAGER package specification (3.5.0 -> 3.3.0)
PROMPT - FILE_MANAGER package body (3.4.0 -> 3.3.0) PROMPT - FILE_MANAGER package body (3.5.0 -> 3.3.0)
PROMPT - FILE_MANAGER_ODS package specification (2.2.0 -> 2.1.0) PROMPT - FILE_MANAGER_ODS package specification (2.4.0 -> 2.1.0)
PROMPT - FILE_MANAGER_ODS package body (2.2.0 -> 2.1.0) PROMPT - FILE_MANAGER_ODS package body (2.4.0 -> 2.1.0)
PROMPT - Remove CREATE_EXTERNAL_TABLES_SET procedures PROMPT - Remove CREATE_EXTERNAL_TABLES_SET procedures
PROMPT - Remove CREATE_EXTERNAL_TABLES_BATCH procedures PROMPT - Remove CREATE_EXTERNAL_TABLES_BATCH procedures
PROMPT - Remove pArea parameter (selective table creation)
PROMPT - Remove pRestoreGrants parameter (grant preservation)
PROMPT ========================================================================= PROMPT =========================================================================
-- Confirm rollback with user -- Confirm rollback with user