Init
This commit is contained in:
974
.github/copilot-instructions.md
vendored
Normal file
974
.github/copilot-instructions.md
vendored
Normal file
@@ -0,0 +1,974 @@
|
|||||||
|
# GitHub Copilot Instructions - Oracle Database File Management System
|
||||||
|
|
||||||
|
## Critical Development Workflow
|
||||||
|
|
||||||
|
### **CODE-FIRST DEVELOPMENT PRINCIPLE**
|
||||||
|
**MANDATORY**: Always prepare code changes in files BEFORE deploying to database.
|
||||||
|
|
||||||
|
**Correct Workflow:**
|
||||||
|
1. **ANALYZE** current code structure and identify required changes
|
||||||
|
2. **UPDATE STRUCTURE DEFINITIONS** in source files (.sql table/view/index definitions)
|
||||||
|
3. **PREPARE MIGRATION SCRIPTS** (ALTER TABLE, CREATE INDEX, etc.) for existing databases
|
||||||
|
4. **MODIFY PACKAGE CODE** (.pkb, .pks) to utilize new structures
|
||||||
|
5. **VALIDATE** syntax and logic in all code files
|
||||||
|
6. **TEST** changes through file deployment scripts
|
||||||
|
7. **DEPLOY** migration scripts first, then updated packages to database
|
||||||
|
|
||||||
|
**NEVER:**
|
||||||
|
- Execute DDL/DML directly on database without corresponding file updates
|
||||||
|
- Create database objects without updating source control files
|
||||||
|
- Modify database structures ad-hoc through SQL commands
|
||||||
|
- Skip creation of migration scripts for existing environments
|
||||||
|
|
||||||
|
**Two-Tier File Management:**
|
||||||
|
- **Structure Definitions** - Complete CREATE statements in `database/{schema}/tables/`
|
||||||
|
- **Migration Scripts** - Incremental ALTER statements in `database/migrations/`
|
||||||
|
|
||||||
|
**File-First Approach Benefits:**
|
||||||
|
- Version control integrity maintained
|
||||||
|
- Reproducible deployments across environments
|
||||||
|
- Code review capability for all changes
|
||||||
|
- Rollback capability through source control
|
||||||
|
- Team collaboration on structured changes
|
||||||
|
- Clear separation between baseline and incremental changes
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Project Architecture Overview
|
||||||
|
|
||||||
|
This Oracle database system provides enterprise-grade file processing and data export capabilities for Oracle Cloud Infrastructure (OCI). The architecture centers around multi-schema data pipelines with bucket-based storage management and secure cloud wrapper functionality.
|
||||||
|
|
||||||
|
### Dual System Architecture
|
||||||
|
- **File Management System** (Primary) - Enterprise data export and processing capabilities
|
||||||
|
- **Cloud Wrapper System** (Foundation) - Secure DBMS_CLOUD access wrapper for controlled cloud operations
|
||||||
|
|
||||||
|
### Core Components
|
||||||
|
|
||||||
|
**Primary Packages:**
|
||||||
|
- `CT_MRDS.FILE_MANAGER` - Main file processing and validation orchestration package (AUTHID CURRENT_USER)
|
||||||
|
- `CT_MRDS.DATA_EXPORTER` - Data export operations with Hive-style partitioning support (separated from FILE_MANAGER)
|
||||||
|
- `CT_MRDS.ENV_MANAGER` - Environment configuration and error management
|
||||||
|
- `CT_MRDS.FILE_ARCHIVER` - Data archiving and archival operations
|
||||||
|
- `ODS.FILE_MANAGER_ODS` - Wrapper package with DEFINER rights for external table creation (RECOMMENDED)
|
||||||
|
- `MRDS_LOADER.cloud_wrapper` - Secure wrapper around DBMS_CLOUD functions
|
||||||
|
- `MRDS_LOADER.DATA_REPLICATOR` - Cloud data replication and export operations
|
||||||
|
|
||||||
|
**Package Architecture Recommendation:**
|
||||||
|
- **RECOMMENDED**: Use `ODS.FILE_MANAGER_ODS` (AUTHID DEFINER) - works from any user context
|
||||||
|
- **ALTERNATIVE**: Use `CT_MRDS.FILE_MANAGER` (AUTHID CURRENT_USER) - requires ODS user context for CREATE_EXTERNAL_TABLE
|
||||||
|
- **DATA_EXPORTER**: Export procedures moved from FILE_MANAGER to dedicated DATA_EXPORTER package for better modularity
|
||||||
|
|
||||||
|
**Schema Architecture:**
|
||||||
|
- **CT_MRDS** - Core file management and processing schema
|
||||||
|
- **CT_ODS** - Operational Data Store with load history tracking
|
||||||
|
- **ODS** - External table creation schema with DEFINER privileges
|
||||||
|
- **OU_TOP** - Business data schema (e.g., AGGREGATED_ALLOTMENT table)
|
||||||
|
- **MRDS_LOADER** - Cloud operations schema with DBMS_CLOUD access and package ownership
|
||||||
|
- **CT_ET_TEMPLATES** - Template table definitions
|
||||||
|
- **ADMIN** - Database administrator with full privileges (REQUIRED for schema creation)
|
||||||
|
- **MICHALZ** - Standard development user with execute privileges on cloud_wrapper
|
||||||
|
|
||||||
|
### Critical Architecture Pattern
|
||||||
|
**Privilege Separation Model**: MRDS_LOADER owns cloud_wrapper package with DBMS_CLOUD access, other users get EXECUTE grants for controlled cloud operations.
|
||||||
|
|
||||||
|
### File Processing Configuration
|
||||||
|
**Core Configuration Procedures:**
|
||||||
|
- `FILE_MANAGER.ADD_SOURCE` - Register new source systems (e.g., 'C2D', 'LM', 'TOP')
|
||||||
|
- `FILE_MANAGER.ADD_SOURCE_FILE_CONFIG` - Configure file processing rules and naming patterns
|
||||||
|
- `FILE_MANAGER.ADD_COLUMN_DATE_FORMAT` - Set up date format handling for specific columns
|
||||||
|
- `FILE_MANAGER.DELETE_SOURCE_CASCADE` - Safe removal of source systems with cascade delete
|
||||||
|
- `FILE_MANAGER.ANALYZE_VALIDATION_ERRORS` - Analyze file validation failures with detailed reporting
|
||||||
|
|
||||||
|
**Official Path Patterns (MANDATORY):**
|
||||||
|
- **INBOX**: `'INBOX/{SOURCE}/{SOURCE_FILE_ID}/{TABLE_NAME}/'` (3-level path)
|
||||||
|
- **ODS**: `'ODS/{SOURCE}/{TABLE_NAME}/'` (2-level path, no SOURCE_FILE_ID)
|
||||||
|
- **ARCHIVE**: `'ARCHIVE/{SOURCE}/{TABLE_NAME}/'` (2-level path, no SOURCE_FILE_ID)
|
||||||
|
|
||||||
|
**File Processing Workflow:**
|
||||||
|
- `FILE_MANAGER.PROCESS_SOURCE_FILE` - Main umbrella procedure (6-step validation workflow)
|
||||||
|
- Status tracking: RECEIVED → VALIDATED → READY_FOR_INGESTION → INGESTED → ARCHIVED
|
||||||
|
- Automatic file movement: INBOX → ODS/DATA → ARCHIVE (with Hive-style partitioning)
|
||||||
|
|
||||||
|
## Data Export System
|
||||||
|
|
||||||
|
### Daily Data Flow Workflow
|
||||||
|
|
||||||
|
**1. Data Ingestion (INBOX)**
|
||||||
|
Daily data files arrive in the INBOX bucket area where they undergo validation:
|
||||||
|
```sql
|
||||||
|
-- File validation and processing (umbrella procedure with 6 steps)
|
||||||
|
FILE_MANAGER.PROCESS_SOURCE_FILE(
|
||||||
|
pSourceFileReceivedName => 'INBOX/C2D/UC_DISSEM/A_UC_DISSEM_METADATA_LOADS/data_file.csv'
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
**PROCESS_SOURCE_FILE Workflow (6 Steps):**
|
||||||
|
1. **REGISTER_SOURCE_FILE_RECEIVED** - Register file and extract metadata
|
||||||
|
2. **CREATE_EXTERNAL_TABLE** - Create temporary external table for validation
|
||||||
|
3. **VALIDATE_SOURCE_FILE_RECEIVED** - Comprehensive data validation (column count, data types, business rules)
|
||||||
|
4. **DROP_EXTERNAL_TABLE** - Clean up temporary external table
|
||||||
|
5. **MOVE_FILE** - Relocate from INBOX to ODS bucket
|
||||||
|
6. **SET_SOURCE_FILE_RECEIVED_STATUS** - Update status to 'READY_FOR_INGESTION'
|
||||||
|
|
||||||
|
**Status Progression:** RECEIVED → VALIDATED → READY_FOR_INGESTION → INGESTED → ARCHIVED
|
||||||
|
|
||||||
|
- Files are validated for structure, format, and business rules using template tables
|
||||||
|
- Successfully validated files are moved to ODS/DATA bucket area and prepared for Airflow+DBT processing
|
||||||
|
- Failed validation results in error logging with detailed analysis via `ANALYZE_VALIDATION_ERRORS()` function
|
||||||
|
- Comprehensive error handling for column mismatches, file access issues, and configuration problems
|
||||||
|
|
||||||
|
**2. Data Processing (ODS/DATA)**
|
||||||
|
Validated files are processed and stored in the operational data area for further operations.
|
||||||
|
|
||||||
|
**3. Legacy Data Migration (DATA_EXPORTER)**
|
||||||
|
The DATA_EXPORTER package handles migration of data from legacy systems with three export scenarios:
|
||||||
|
|
||||||
|
**Scenario 1: Complete DATA Export (CSV)**
|
||||||
|
- All data exported to DATA bucket area in CSV format
|
||||||
|
- No implicit partitioning applied
|
||||||
|
- Used for simple data migrations
|
||||||
|
|
||||||
|
**Scenario 2: Split Export (DATA + ARCHIVE)**
|
||||||
|
- Partial data to DATA bucket area (CSV format)
|
||||||
|
- Partial data to ARCHIVE bucket area (Parquet with Hive-style partitioning)
|
||||||
|
- Requires two separate export operations
|
||||||
|
|
||||||
|
**Scenario 3: Complete ARCHIVE Export (Parquet)**
|
||||||
|
- All data exported to ARCHIVE bucket area
|
||||||
|
- Parquet format with Hive-style partitioning (PARTITION_YEAR=/PARTITION_MONTH=)
|
||||||
|
- Used for long-term storage and analytics
|
||||||
|
|
||||||
|
### Bucket Area Mapping
|
||||||
|
The system uses logical bucket areas that map to physical OCI buckets via `GET_BUCKET_URI()`:
|
||||||
|
- `'INBOX'` → gvInboxBucketUri (incoming data validation)
|
||||||
|
- `'ODS'` → gvDataBucketUri (operational data processing)
|
||||||
|
- `'DATA'` → gvDataBucketUri (processed data - CSV format)
|
||||||
|
- `'ARCHIVE'` → gvArchiveBucketUri (archived data - Parquet with partitioning)
|
||||||
|
|
||||||
|
### Export Procedures
|
||||||
|
|
||||||
|
**Scenario 1: Complete DATA Export (CSV Format)**
|
||||||
|
```sql
|
||||||
|
-- Export all data to DATA bucket area in CSV format without partitioning
|
||||||
|
FILE_MANAGER.EXPORT_TABLE_DATA_TO_CSV_BY_DATE(
|
||||||
|
pSchemaName => 'OU_TOP',
|
||||||
|
pTableName => 'AGGREGATED_ALLOTMENT',
|
||||||
|
pKeyColumnName => 'A_WORKFLOW_HISTORY_KEY',
|
||||||
|
pBucketArea => 'DATA',
|
||||||
|
pFolderName => 'legacy_migration',
|
||||||
|
pMinDate => DATE '2024-01-01',
|
||||||
|
pMaxDate => SYSDATE
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
**Scenario 2: Split Export (DATA CSV + ARCHIVE Parquet)**
|
||||||
|
```sql
|
||||||
|
-- Part 1: Export recent data to DATA bucket (CSV format)
|
||||||
|
FILE_MANAGER.EXPORT_TABLE_DATA_TO_CSV_BY_DATE(
|
||||||
|
pSchemaName => 'OU_TOP',
|
||||||
|
pTableName => 'AGGREGATED_ALLOTMENT',
|
||||||
|
pKeyColumnName => 'A_WORKFLOW_HISTORY_KEY',
|
||||||
|
pBucketArea => 'DATA',
|
||||||
|
pFolderName => 'current_data',
|
||||||
|
pMinDate => DATE '2024-01-01',
|
||||||
|
pMaxDate => SYSDATE
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Part 2: Export historical data to ARCHIVE bucket (Parquet with partitioning)
|
||||||
|
DATA_EXPORTER.EXPORT_TABLE_DATA_BY_DATE(
|
||||||
|
pSchemaName => 'OU_TOP',
|
||||||
|
pTableName => 'AGGREGATED_ALLOTMENT',
|
||||||
|
pKeyColumnName => 'A_WORKFLOW_HISTORY_KEY',
|
||||||
|
pBucketArea => 'ARCHIVE',
|
||||||
|
pFolderName => 'historical_data',
|
||||||
|
pMinDate => DATE '2020-01-01',
|
||||||
|
pMaxDate => DATE '2023-12-31'
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
**Scenario 3: Complete ARCHIVE Export (Parquet with Partitioning)**
|
||||||
|
```sql
|
||||||
|
-- Export all data to ARCHIVE bucket with Hive-style partitioning
|
||||||
|
DATA_EXPORTER.EXPORT_TABLE_DATA_BY_DATE(
|
||||||
|
pSchemaName => 'OU_TOP',
|
||||||
|
pTableName => 'AGGREGATED_ALLOTMENT',
|
||||||
|
pKeyColumnName => 'A_WORKFLOW_HISTORY_KEY',
|
||||||
|
pBucketArea => 'ARCHIVE',
|
||||||
|
pFolderName => 'complete_export',
|
||||||
|
pMinDate => DATE '2020-01-01',
|
||||||
|
pMaxDate => SYSDATE
|
||||||
|
);
|
||||||
|
-- Creates: complete_export/PARTITION_YEAR=2024/PARTITION_MONTH=01/*.parquet
|
||||||
|
```
|
||||||
|
|
||||||
|
**Daily Processing Workflow**
|
||||||
|
```sql
|
||||||
|
-- Standard daily file processing procedure
|
||||||
|
FILE_MANAGER.EXPORT_TABLE_DATA_BY_DATE(
|
||||||
|
pSchemaName => 'CT_MRDS',
|
||||||
|
pTableName => 'MY_TABLE',
|
||||||
|
pBucketArea => 'DATA',
|
||||||
|
pFolderName => 'daily_processing'
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
## Critical Implementation Patterns
|
||||||
|
|
||||||
|
### Error Handling Architecture
|
||||||
|
The system uses centralized error management through `ENV_MANAGER`:
|
||||||
|
|
||||||
|
**CRITICAL**: All error handling modifications must be implemented in source files first.
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- CORRECT: Use constant concatenation to avoid duplication
|
||||||
|
-- UPDATE FILE: database/CT_MRDS/packages/ENV_MANAGER.sql
|
||||||
|
FUNCTION GET_BUCKET_URI(pBucketName VARCHAR2) RETURN VARCHAR2 IS
|
||||||
|
BEGIN
|
||||||
|
CASE pBucketName
|
||||||
|
WHEN 'INBOX' THEN RETURN ENV_MANAGER.gvInboxBucketUri;
|
||||||
|
-- ... other cases
|
||||||
|
ELSE
|
||||||
|
RAISE_APPLICATION_ERROR(ENV_MANAGER.CODE_INVALID_BUCKET_AREA,
|
||||||
|
ENV_MANAGER.MSG_INVALID_BUCKET_AREA || ' Provided: ''' || pBucketName || '''');
|
||||||
|
END CASE;
|
||||||
|
END;
|
||||||
|
|
||||||
|
-- WRONG: Never modify global MSG_ variables (causes duplication in same session)
|
||||||
|
-- ENV_MANAGER.MSG_INVALID_BUCKET_AREA := ENV_MANAGER.MSG_INVALID_BUCKET_AREA || '...';
|
||||||
|
|
||||||
|
-- Structured exception handling with local variables
|
||||||
|
EXCEPTION
|
||||||
|
WHEN ENV_MANAGER.ERR_TABLE_NOT_EXISTS THEN
|
||||||
|
vgMsgTmp := ENV_MANAGER.MSG_TABLE_NOT_EXISTS || ': ' || vTableName;
|
||||||
|
ENV_MANAGER.LOG_PROCESS_EVENT(vgMsgTmp, 'ERROR', vParameters);
|
||||||
|
RAISE_APPLICATION_ERROR(ENV_MANAGER.CODE_TABLE_NOT_EXISTS, vgMsgTmp);
|
||||||
|
```
|
||||||
|
|
||||||
|
**Error Message Pattern**: Always use `MSG_*` constants for fresh messages, never modify global variables.
|
||||||
|
|
||||||
|
### AUTHID Patterns
|
||||||
|
- `AUTHID CURRENT_USER` - Functions execute with caller's privileges (FILE_MANAGER)
|
||||||
|
- `AUTHID DEFINER` - Objects created in package owner schema (FILE_MANAGER_ODS)
|
||||||
|
|
||||||
|
### Date Format Configuration System
|
||||||
|
|
||||||
|
**Three-Tier Hierarchical Date Format Resolution** (Confirmed through GET_DATE_FORMAT function analysis):
|
||||||
|
|
||||||
|
1. **Column-Specific Configuration** - First priority lookup in `A_COLUMN_DATE_FORMAT` table:
|
||||||
|
```sql
|
||||||
|
SELECT DATE_FORMAT FROM A_COLUMN_DATE_FORMAT
|
||||||
|
WHERE TEMPLATE_TABLE_NAME = 'CT_ET_TEMPLATES.TABLE_NAME'
|
||||||
|
AND COLUMN_NAME = 'SPECIFIC_COLUMN'
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **DEFAULT Column Fallback** - Second priority using special DEFAULT entry:
|
||||||
|
```sql
|
||||||
|
SELECT DATE_FORMAT FROM A_COLUMN_DATE_FORMAT
|
||||||
|
WHERE TEMPLATE_TABLE_NAME = 'CT_ET_TEMPLATES.TABLE_NAME'
|
||||||
|
AND COLUMN_NAME = 'DEFAULT'
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Global Fallback** - Final fallback to ENV_MANAGER global variable:
|
||||||
|
```sql
|
||||||
|
-- Falls back to ENV_MANAGER.gvDefaultDateFormat ('DD/MM/YYYY HH24:MI:SS')
|
||||||
|
-- Configured via A_FILE_MANAGER_CONFIG table with CONFIG_VARIABLE = 'DefaultDateFormat'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Configuration Examples:**
|
||||||
|
```sql
|
||||||
|
-- Column-specific date format
|
||||||
|
CALL FILE_MANAGER.ADD_COLUMN_DATE_FORMAT(
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.C2D_A_UC_DISSEM_METADATA_LOADS',
|
||||||
|
pColumnName => 'SNAPSHOT_DATE',
|
||||||
|
pDateFormat => 'YYYY-MM-DD HH24:MI:SS'
|
||||||
|
);
|
||||||
|
|
||||||
|
-- DEFAULT fallback for entire template table
|
||||||
|
CALL FILE_MANAGER.ADD_COLUMN_DATE_FORMAT(
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.C2D_A_UC_DISSEM_METADATA_LOADS',
|
||||||
|
pColumnName => 'DEFAULT',
|
||||||
|
pDateFormat => 'MM/DD/YYYY'
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Global configuration (via A_FILE_MANAGER_CONFIG)
|
||||||
|
INSERT INTO A_FILE_MANAGER_CONFIG (ENVIRONMENT_ID, CONFIG_VARIABLE, CONFIG_VARIABLE_VALUE)
|
||||||
|
VALUES ('dev', 'DefaultDateFormat', 'DD-MON-YYYY HH24:MI:SS');
|
||||||
|
```
|
||||||
|
|
||||||
|
### Date Filtering Integration
|
||||||
|
Export procedures integrate with `CT_ODS.A_LOAD_HISTORY` for temporal partitioning:
|
||||||
|
```sql
|
||||||
|
-- Date-based partitioning query pattern
|
||||||
|
SELECT DISTINCT TO_CHAR(L.LOAD_START,'YYYY') AS YR, TO_CHAR(L.LOAD_START,'MM') AS MN
|
||||||
|
FROM target_table T, CT_ODS.A_LOAD_HISTORY L
|
||||||
|
WHERE T.A_WORKFLOW_HISTORY_KEY = L.A_WORKFLOW_HISTORY_KEY
|
||||||
|
AND L.LOAD_START >= :pMinDate AND L.LOAD_START < :pMaxDate
|
||||||
|
```
|
||||||
|
|
||||||
|
### Hive-Style Partitioning Support
|
||||||
|
**CONFIRMED**: DBMS_CLOUD.EXPORT_DATA **FULLY SUPPORTS** Hive-style partitioning with key=value format:
|
||||||
|
```
|
||||||
|
folder/PARTITION_YEAR=2025/PARTITION_MONTH=09/file.parquet
|
||||||
|
```
|
||||||
|
Export creates directory structure compatible with big data tools (Spark, Hive, etc.).
|
||||||
|
|
||||||
|
### Database Triggers
|
||||||
|
**A_SOURCE_FILE_CONFIG_KEY Auto-Generation**:
|
||||||
|
- `TRG_BI_A_SOURCE_FILE_CONFIG_CHECK` trigger automatically generates primary keys
|
||||||
|
- Uses `A_SOURCE_FILE_CONFIG_KEY_SEQ.NEXTVAL` sequence
|
||||||
|
- Validates CONTAINER relationships and business rules
|
||||||
|
|
||||||
|
### External Table Management
|
||||||
|
**Template Tables and External Tables Setup:**
|
||||||
|
- **Template Tables**: Located in `CT_ET_TEMPLATES` schema, define structure for external tables
|
||||||
|
- **CRITICAL**: Template tables can ONLY be created in `CT_ET_TEMPLATES` schema by ADMIN or CT_ET_TEMPLATES user
|
||||||
|
- **MRDS_LOADER Limitation**: Cannot create template tables - must use existing ones or request ADMIN to create them
|
||||||
|
- **External Table Creation**: Use `ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE` (recommended) or `CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLE`
|
||||||
|
- **Storage Locations**: INBOX (incoming), ODS (operational), ARCHIVE (historical with partitioning)
|
||||||
|
- **Validation Features**: Automatic CSV column validation, excess column detection, detailed error reporting
|
||||||
|
|
||||||
|
**MRDS_LOADER External Table Creation Pattern:**
|
||||||
|
- **RECOMMENDED APPROACH**: Use MRDS_LOADER connection with `ODS.FILE_MANAGER_ODS` package to create external tables in ODS schema
|
||||||
|
- **Required Prerequisites**:
|
||||||
|
- Template table must exist in CT_ET_TEMPLATES (created by ADMIN)
|
||||||
|
- MRDS_LOADER must have comprehensive privileges to CT_MRDS schema objects
|
||||||
|
- ODS schema must have access to CT_MRDS.ENV_MANAGER for logging operations
|
||||||
|
- **Privilege Requirements for MRDS_LOADER**:
|
||||||
|
```sql
|
||||||
|
-- CT_MRDS schema access (essential for FILE_MANAGER_ODS operations)
|
||||||
|
GRANT SELECT, INSERT, UPDATE, DELETE ON CT_MRDS.A_PROCESS_LOG TO MRDS_LOADER;
|
||||||
|
GRANT EXECUTE ON CT_MRDS.ENV_MANAGER TO MRDS_LOADER;
|
||||||
|
GRANT EXECUTE ON ODS.FILE_MANAGER_ODS TO MRDS_LOADER;
|
||||||
|
```
|
||||||
|
- **Privilege Requirements for ODS Schema**:
|
||||||
|
```sql
|
||||||
|
-- Required for FILE_MANAGER_ODS to access logging and configuration
|
||||||
|
GRANT SELECT, INSERT, UPDATE, DELETE ON CT_MRDS.A_PROCESS_LOG TO ODS;
|
||||||
|
GRANT EXECUTE ON CT_MRDS.ENV_MANAGER TO ODS;
|
||||||
|
```
|
||||||
|
|
||||||
|
**Configuration Examples:**
|
||||||
|
```sql
|
||||||
|
-- Register source system
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE(pSourceKey => 'C2D', pSourceName => 'Central Bank Data');
|
||||||
|
|
||||||
|
-- Configure file processing
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'C2D', pSourceFileType => 'INPUT', pSourceFileId => 'UC_DISSEM',
|
||||||
|
pSourceFileDesc => 'UC DISSEM Metadata', pSourceFileNamePattern => 'UC_NMA_DISSEM-*.csv',
|
||||||
|
pTableId => 'A_UC_DISSEM_METADATA_LOADS', pTemplateTableName => 'CT_ET_TEMPLATES.C2D_A_UC_DISSEM_METADATA_LOADS'
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Configure date formats
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_COLUMN_DATE_FORMAT(
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.C2D_A_UC_DISSEM_METADATA_LOADS',
|
||||||
|
pColumnName => 'SNAPSHOT_DATE', pDateFormat => 'YYYY-MM-DD HH24:MI:SS'
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
**Complete External Table Creation Example (STANDING_FACILITIES):**
|
||||||
|
```sql
|
||||||
|
-- Step 1: Create template table (as ADMIN)
|
||||||
|
CREATE TABLE CT_ET_TEMPLATES.LM_STANDING_FACILITIES (
|
||||||
|
A_KEY NUMBER NOT NULL,
|
||||||
|
REV_NUMBER NUMBER,
|
||||||
|
REF_DATE DATE,
|
||||||
|
FREE_TEXT VARCHAR2(1000),
|
||||||
|
MLF_BS_TOTAL NUMBER,
|
||||||
|
DF_BS_TOTAL NUMBER,
|
||||||
|
MLF_SF_TOTAL NUMBER,
|
||||||
|
DF_SF_TOTAL NUMBER
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Step 2: Configure date format (as CT_MRDS or via MRDS_LOADER with privileges)
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_COLUMN_DATE_FORMAT(
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES',
|
||||||
|
pColumnName => 'REF_DATE',
|
||||||
|
pDateFormat => 'DD/MM/YYYY'
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Step 3: Create external tables using MRDS_LOADER connection
|
||||||
|
-- (Requires all prerequisite privileges for MRDS_LOADER and ODS schemas)
|
||||||
|
-- Following OFFICIAL PATH PATTERNS
|
||||||
|
|
||||||
|
-- INBOX table (CSV format for incoming files) - 3-level path
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_STANDING_FACILITIES_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES',
|
||||||
|
pPrefix => 'INBOX/LM/STANDING_FACILITIES/STANDING_FACILITIES',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ODS table (CSV format for operational data) - 2-level path
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_STANDING_FACILITIES_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES',
|
||||||
|
pPrefix => 'ODS/LM/STANDING_FACILITIES',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ARCHIVE table (Parquet format for historical data) - 2-level path
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_STANDING_FACILITIES_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES',
|
||||||
|
pPrefix => 'ARCHIVE/LM/STANDING_FACILITIES',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- Result: Three external tables created in ODS schema:
|
||||||
|
-- ODS.LM_STANDING_FACILITIES_INBOX (CSV)
|
||||||
|
-- ODS.LM_STANDING_FACILITIES_ODS (CSV)
|
||||||
|
-- ODS.LM_STANDING_FACILITIES_ARCHIVE (Parquet)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Development Workflows
|
||||||
|
|
||||||
|
### Database Operations Approaches
|
||||||
|
|
||||||
|
#### **MCP Server Approach (Primary - Recommended)**
|
||||||
|
Use MCP SQLcl server for most database operations:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Connect to specific schema
|
||||||
|
mcp_sqlcl_connect("ADMIN@ggmichalski_high")
|
||||||
|
mcp_sqlcl_connect("CT_MRDS@ggmichalski_high")
|
||||||
|
|
||||||
|
-- Execute SQL directly
|
||||||
|
mcp_sqlcl_run-sql("SELECT USER FROM DUAL")
|
||||||
|
mcp_sqlcl_run-sql("CREATE OR REPLACE PACKAGE...")
|
||||||
|
|
||||||
|
-- List available connections
|
||||||
|
mcp_sqlcl_list-connections()
|
||||||
|
```
|
||||||
|
|
||||||
|
#### **Traditional SQLcl Approach (Secondary)**
|
||||||
|
Use direct SQLcl commands for complex scripts:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# Single script execution pattern
|
||||||
|
Get-Content "path/to/script.sql" | sql "USERNAME/PASSWORD@SERVICE"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Package Deployment
|
||||||
|
```powershell
|
||||||
|
# Core packages deployment sequence using source DDL files
|
||||||
|
Get-Content "MARS_Packages\mrds_elt-dev-database\mrds_elt-dev-database\database\CT_MRDS\packages\ENV_MANAGER.sql" | sql "CT_MRDS/Cloudpass#34@ggmichalski_high"
|
||||||
|
Get-Content "MARS_Packages\mrds_elt-dev-database\mrds_elt-dev-database\database\CT_MRDS\packages\FILE_MANAGER.sql" | sql "CT_MRDS/Cloudpass#34@ggmichalski_high"
|
||||||
|
Get-Content "MARS_Packages\mrds_elt-dev-database\mrds_elt-dev-database\database\ODS\packages\FILE_MANAGER_ODS.sql" | sql "ODS/Cloudpass#34@ggmichalski_high"
|
||||||
|
|
||||||
|
# MARS issue deployment using release packages
|
||||||
|
Get-Content "MARS_Packages\REL01\MARS-846\*.sql" | sql "CT_MRDS/Cloudpass#34@ggmichalski_high"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Archive Creation
|
||||||
|
```powershell
|
||||||
|
# Create encrypted archive of core packages from source DDL
|
||||||
|
7z a -pMojeSuperHaslo -mhe=on FM_arch.7z MARS_Packages\mrds_elt-dev-database\mrds_elt-dev-database\database\CT_MRDS\packages\FILE_MANAGER.sql MARS_Packages\mrds_elt-dev-database\mrds_elt-dev-database\database\CT_MRDS\packages\FILE_ARCHIVER.sql MARS_Packages\mrds_elt-dev-database\mrds_elt-dev-database\database\CT_MRDS\packages\ENV_MANAGER.sql
|
||||||
|
|
||||||
|
# Create archive of specific MARS release
|
||||||
|
7z a -pMojeSuperHaslo -mhe=on MARS_846_arch.7z MARS_Packages\REL01\MARS-846\*.sql
|
||||||
|
```
|
||||||
|
|
||||||
|
### PowerShell Integration
|
||||||
|
Use the `OracleHelper` module for batch operations:
|
||||||
|
```powershell
|
||||||
|
Import-Module ".\powershell_modules\OracleHelper\OracleHelper.psm1" -Force
|
||||||
|
Invoke-OracleScript -ScriptPath "database\CT_MRDS\packages\FILE_MANAGER.sql" -User "CT_MRDS"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Development File Structure Guidelines
|
||||||
|
|
||||||
|
**Source Code Organization:**
|
||||||
|
```
|
||||||
|
MARS_Packages/
|
||||||
|
├── mrds_elt-dev-database/
|
||||||
|
│ └── mrds_elt-dev-database/
|
||||||
|
│ └── database/ # SOURCE DDL FILES (baseline definitions)
|
||||||
|
│ ├── CT_MRDS/
|
||||||
|
│ │ ├── packages/
|
||||||
|
│ │ │ ├── ENV_MANAGER.sql # Environment management
|
||||||
|
│ │ │ ├── FILE_MANAGER.sql # Main file processing
|
||||||
|
│ │ │ ├── DATA_EXPORTER.sql # Data export operations
|
||||||
|
│ │ │ └── FILE_ARCHIVER.sql # Archival operations
|
||||||
|
│ │ ├── tables/
|
||||||
|
│ │ │ ├── A_SOURCE_FILE_CONFIG.sql # Source configuration (CREATE TABLE)
|
||||||
|
│ │ │ ├── A_PROCESS_LOG.sql # Process logging (CREATE TABLE)
|
||||||
|
│ │ │ └── A_FILE_MANAGER_CONFIG.sql # Environment config (CREATE TABLE)
|
||||||
|
│ │ ├── views/
|
||||||
|
│ │ │ └── V_SOURCE_FILE_STATUS.sql # View definitions (CREATE VIEW)
|
||||||
|
│ │ ├── indexes/
|
||||||
|
│ │ │ └── IDX_SOURCE_FILE_CONFIG.sql # Index definitions (CREATE INDEX)
|
||||||
|
│ │ └── triggers/
|
||||||
|
│ │ └── TRG_BI_A_SOURCE_FILE_CONFIG.sql
|
||||||
|
│ ├── ODS/
|
||||||
|
│ │ ├── FILE_MANAGER_ODS.pkb # DEFINER wrapper body
|
||||||
|
│ │ ├── FILE_MANAGER_ODS.pkg # DEFINER wrapper spec
|
||||||
|
│ │ ├── CREATE_USER.sql # User creation scripts
|
||||||
|
│ │ └── CREATE_ROLE.sql # Role creation scripts
|
||||||
|
│ └── CT_ET_TEMPLATES/
|
||||||
|
│ └── tables/
|
||||||
|
│ ├── CSDB_DEBT.sql # Template tables
|
||||||
|
│ ├── CSDB_DEBT_DAILY.sql
|
||||||
|
│ └── [other_templates].sql
|
||||||
|
├── REL01/ # RELEASE PACKAGES (deployment scripts)
|
||||||
|
│ ├── MARS-846/
|
||||||
|
│ │ ├── 01_MARS_846_install_*.sql
|
||||||
|
│ │ ├── 02_MARS_846_*.sql
|
||||||
|
│ │ ├── 91_MARS_846_rollback_*.sql
|
||||||
|
│ │ └── README.md
|
||||||
|
│ ├── MARS-1011/
|
||||||
|
│ └── [other_issues]/
|
||||||
|
├── REL02/ # NEXT RELEASE
|
||||||
|
│ ├── MARS-949/
|
||||||
|
│ ├── MARS-954/
|
||||||
|
│ └── [other_issues]/
|
||||||
|
└── migrations/ # MIGRATION SCRIPTS (for existing environments)
|
||||||
|
├── 001_add_characterset_columns.sql # ALTER TABLE scripts
|
||||||
|
├── 002_create_config_indexes.sql # New indexes
|
||||||
|
├── 003_update_default_values.sql # Data updates
|
||||||
|
└── rollback/
|
||||||
|
├── 001_rollback_characterset.sql # Rollback scripts
|
||||||
|
└── 002_rollback_indexes.sql
|
||||||
|
```
|
||||||
|
|
||||||
|
**Mandatory File Updates for Structural Changes:**
|
||||||
|
|
||||||
|
**1. Structure Definition Updates:**
|
||||||
|
- **New Tables**: Create complete `CREATE TABLE` in `MARS_Packages/mrds_elt-dev-database/mrds_elt-dev-database/database/{schema}/tables/{table_name}.sql`
|
||||||
|
- **Table Modifications**: Update existing table definition files with new structure
|
||||||
|
- **New Indexes**: Add `CREATE INDEX` statements in `database/{schema}/indexes/`
|
||||||
|
- **View Changes**: Update view definitions in `database/{schema}/views/`
|
||||||
|
|
||||||
|
**2. Migration Script Creation:**
|
||||||
|
- **Existing Environment Changes**: Create `ALTER TABLE` scripts in `MARS_Packages/migrations/`
|
||||||
|
- **Data Migration**: Prepare `INSERT/UPDATE` scripts for configuration changes
|
||||||
|
- **Rollback Scripts**: Create corresponding rollback scripts in `MARS_Packages/migrations/rollback/`
|
||||||
|
- **Deployment Order**: Number migration scripts sequentially (001_, 002_, etc.)
|
||||||
|
|
||||||
|
**3. Release Package Creation:**
|
||||||
|
- **MARS Issue Packages**: Create deployment scripts in `MARS_Packages/{RELEASE}/{MARS_ISSUE}/`
|
||||||
|
- **Installation Scripts**: Numbered install scripts (01_, 02_, etc.)
|
||||||
|
- **Rollback Scripts**: Numbered rollback scripts (91_, 92_, etc.)
|
||||||
|
- **Documentation**: Include README.md with deployment instructions
|
||||||
|
|
||||||
|
**4. Package Code Updates:**
|
||||||
|
- **Package Modifications**: Update .pks (spec) and .pkb (body) files in source `database/{schema}/packages/`
|
||||||
|
- **Wrapper Packages**: Update related wrapper packages if applicable
|
||||||
|
- **Documentation**: Update confluence/ directory with changes
|
||||||
|
|
||||||
|
**Change Management Process:**
|
||||||
|
```sql
|
||||||
|
-- 1. PREPARE STRUCTURE DEFINITIONS: Update baseline table definitions
|
||||||
|
-- File: MARS_Packages/mrds_elt-dev-database/mrds_elt-dev-database/database/CT_MRDS/tables/A_SOURCE_FILE_CONFIG.sql
|
||||||
|
CREATE TABLE A_SOURCE_FILE_CONFIG (
|
||||||
|
A_SOURCE_FILE_CONFIG_KEY NUMBER NOT NULL,
|
||||||
|
-- ... existing columns ...
|
||||||
|
CHARACTERSET VARCHAR2(50) DEFAULT 'WE8MSWIN1252', -- NEW COLUMN
|
||||||
|
DELIMITER VARCHAR2(10) DEFAULT ',', -- NEW COLUMN
|
||||||
|
CONSTRAINT PK_A_SOURCE_FILE_CONFIG PRIMARY KEY (A_SOURCE_FILE_CONFIG_KEY)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- 2. PREPARE MIGRATION SCRIPT: For existing environments
|
||||||
|
-- File: MARS_Packages/migrations/001_add_characterset_columns.sql
|
||||||
|
ALTER TABLE CT_MRDS.A_SOURCE_FILE_CONFIG ADD (
|
||||||
|
CHARACTERSET VARCHAR2(50) DEFAULT 'WE8MSWIN1252',
|
||||||
|
DELIMITER VARCHAR2(10) DEFAULT ','
|
||||||
|
);
|
||||||
|
COMMENT ON COLUMN CT_MRDS.A_SOURCE_FILE_CONFIG.CHARACTERSET IS 'File encoding';
|
||||||
|
|
||||||
|
-- 3. CREATE RELEASE PACKAGE: For MARS issue deployment
|
||||||
|
-- File: MARS_Packages/REL02/MARS-XXXX/01_MARS_XXXX_add_characterset_columns.sql
|
||||||
|
-- (Copy migration script content with proper header and logging)
|
||||||
|
|
||||||
|
-- 4. PREPARE ROLLBACK SCRIPT: For emergency rollback
|
||||||
|
-- File: MARS_Packages/REL02/MARS-XXXX/91_MARS_XXXX_rollback_characterset.sql
|
||||||
|
ALTER TABLE CT_MRDS.A_SOURCE_FILE_CONFIG DROP (CHARACTERSET, DELIMITER);
|
||||||
|
|
||||||
|
-- 5. DEPLOY: Execute MARS package scripts
|
||||||
|
Get-Content "MARS_Packages/REL02/MARS-XXXX/01_MARS_XXXX_add_characterset_columns.sql" | sql "CT_MRDS/password@service"
|
||||||
|
Get-Content "MARS_Packages/REL02/MARS-XXXX/02_MARS_XXXX_update_packages.sql" | sql "CT_MRDS/password@service"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Cloud Storage Integration
|
||||||
|
|
||||||
|
#### Available Object Storage Buckets
|
||||||
|
- **obucket** - Primary bucket for data operations (contains test files and parquet exports)
|
||||||
|
- **data** - Data storage bucket
|
||||||
|
- **archive** - Archived data bucket
|
||||||
|
- **inbox** - Incoming data bucket
|
||||||
|
|
||||||
|
#### OCI CLI Operations
|
||||||
|
```powershell
|
||||||
|
# Object Storage operations
|
||||||
|
oci os object put --bucket-name obucket --file "data.csv" --name "import/data.csv"
|
||||||
|
oci os object get --bucket-name obucket --name "export/results.parquet" --file "results.parquet"
|
||||||
|
oci os object list --bucket-name obucket --prefix "folder/"
|
||||||
|
|
||||||
|
# Namespace: frtgjxu7zl7c
|
||||||
|
# Region: eu-frankfurt-1 (Frankfurt)
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Credential Management Pattern
|
||||||
|
Cloud operations use `DEF_CRED_ARN` credential pointing to Oracle Cloud Storage bucket: `https://objectstorage.eu-frankfurt-1.oraclecloud.com/n/frtgjxu7zl7c/b/obucket/`
|
||||||
|
|
||||||
|
## File Organization Conventions
|
||||||
|
|
||||||
|
### Storage Hierarchy
|
||||||
|
```
|
||||||
|
OCI Bucket Structure (OFFICIAL PATTERNS):
|
||||||
|
INBOX/{SOURCE}/{SOURCE_FILE_ID}/{TABLE_NAME}/ -- 3-level (incoming validation)
|
||||||
|
ODS/{SOURCE}/{TABLE_NAME}/ -- 2-level (operational data)
|
||||||
|
ARCHIVE/{SOURCE}/{TABLE_NAME}/PARTITION_YEAR={YYYY}/PARTITION_MONTH={MM}/ -- 2-level + partitioning
|
||||||
|
```
|
||||||
|
|
||||||
|
### Documentation Location
|
||||||
|
- Technical guides → `confluence/` directory
|
||||||
|
- Package documentation → Auto-generated to `confluence/`
|
||||||
|
- Setup procedures → `confluence/Tables_setup.md`
|
||||||
|
|
||||||
|
#### Documentation Generation Workflow
|
||||||
|
```powershell
|
||||||
|
# Generate package documentation using GET_PACKAGE_DOCUMENTATION function
|
||||||
|
echo "SET PAGESIZE 0`nSET LINESIZE 32000`nSET LONG 1000000`nSELECT GET_PACKAGE_DOCUMENTATION('PACKAGE_NAME', 'SCHEMA_NAME') FROM DUAL;" | sql "CT_MRDS/Cloudpass#34@ggmichalski_high" > confluence/package_name_documentation.md
|
||||||
|
```
|
||||||
|
|
||||||
|
**GET_PACKAGE_DOCUMENTATION Function:**
|
||||||
|
- **Purpose**: Automatically generates comprehensive markdown documentation from Oracle packages
|
||||||
|
- **Features**: Extracts procedural metadata, function signatures, parameter information, embedded comments
|
||||||
|
- **Usage**: `GET_PACKAGE_DOCUMENTATION('FILE_MANAGER', 'CT_MRDS')` returns CLOB with formatted markdown
|
||||||
|
- **Output**: Complete package documentation including usage examples and return types
|
||||||
|
|
||||||
|
## Testing and Validation
|
||||||
|
|
||||||
|
### SQL Package Testing Infrastructure
|
||||||
|
|
||||||
|
**Standard Package Testing Pattern with @install:**
|
||||||
|
```sql
|
||||||
|
-- Master install script pattern (executes sub-scripts using @)
|
||||||
|
-- Example: install_mars1049.sql
|
||||||
|
@@01_MARS_1049_install_CT_MRDS_ADD_ENCODING_COLUMN.sql
|
||||||
|
@@02_MARS_1049_install_CT_MRDS_FILE_MANAGER_SPEC.sql
|
||||||
|
@@03_MARS_1049_install_CT_MRDS_FILE_MANAGER_BODY.sql
|
||||||
|
@@04_MARS_1049_install_ODS_FILE_MANAGER_ODS_SPEC.sql
|
||||||
|
@@05_MARS_1049_install_ODS_FILE_MANAGER_ODS_BODY.sql
|
||||||
|
@@06_MARS_1049_verify_encoding_functionality.sql
|
||||||
|
@@07_MARS_1049_track_version.sql
|
||||||
|
|
||||||
|
-- Usage with PowerShell/SQLcl:
|
||||||
|
sql 'CT_MRDS/Cloudpass#34@ggmichalski_high' '@install_mars1049.sql'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Package Verification Standards:**
|
||||||
|
```sql
|
||||||
|
-- Standard verification script pattern (XX_verify_*.sql)
|
||||||
|
-- Check compilation status
|
||||||
|
SELECT object_name, object_type, status
|
||||||
|
FROM user_objects
|
||||||
|
WHERE object_name = 'PACKAGE_NAME'
|
||||||
|
AND object_type IN ('PACKAGE', 'PACKAGE BODY');
|
||||||
|
|
||||||
|
-- Verify package version
|
||||||
|
SELECT SCHEMA.PACKAGE_NAME.GET_VERSION() AS VERSION FROM DUAL;
|
||||||
|
|
||||||
|
-- Check build information
|
||||||
|
SELECT SCHEMA.PACKAGE_NAME.GET_BUILD_INFO() AS BUILD_INFO FROM DUAL;
|
||||||
|
|
||||||
|
-- Verify functionality (package-specific tests)
|
||||||
|
-- Example: Test new encoding functionality
|
||||||
|
CALL FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(..., pEncoding => 'WE8MSWIN1252');
|
||||||
|
```
|
||||||
|
|
||||||
|
**Testing Workflow Integration:**
|
||||||
|
- **Phase 1**: Structure Changes (`01_*_install_*_TABLE.sql`)
|
||||||
|
- **Phase 2**: Package Specification (`02_*_install_*_SPEC.sql`)
|
||||||
|
- **Phase 3**: Package Body (`03_*_install_*_BODY.sql`)
|
||||||
|
- **Phase 4**: Verification (`04_*_verify_*.sql`)
|
||||||
|
- **Phase 5**: Version Tracking (`05_*_track_*.sql`)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
### Key Test Files
|
||||||
|
- `MARS_Packages/REL*/MARS-*/install_mars*.sql` - Master installation scripts with @include pattern
|
||||||
|
- `MARS_Packages/REL*/MARS-*/rollback_mars*.sql` - Complete rollback testing
|
||||||
|
|
||||||
|
|
||||||
|
### Common Operations
|
||||||
|
```sql
|
||||||
|
-- Test bucket area validation
|
||||||
|
SELECT GET_BUCKET_URI('DATA') FROM DUAL; -- Returns: https://objectstorage.eu-frankfurt-1.oraclecloud.com/n/frtgjxu7zl7c/b/data/o/
|
||||||
|
|
||||||
|
-- Validate export procedures with Hive-style partitioning
|
||||||
|
BEGIN
|
||||||
|
DATA_EXPORTER.EXPORT_TABLE_DATA_BY_DATE(
|
||||||
|
pSchemaName => 'OU_TOP',
|
||||||
|
pTableName => 'AGGREGATED_ALLOTMENT',
|
||||||
|
pKeyColumnName => 'A_WORKFLOW_HISTORY_KEY',
|
||||||
|
pBucketArea => 'DATA',
|
||||||
|
pFolderName => 'test_export',
|
||||||
|
pMinDate => DATE '2025-09-01',
|
||||||
|
pMaxDate => DATE '2025-09-30'
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
-- Creates files in: test_export/PARTITION_YEAR=2025/PARTITION_MONTH=09/*.parquet
|
||||||
|
|
||||||
|
-- List cloud storage files using cloud_wrapper
|
||||||
|
SELECT object_name FROM TABLE(MRDS_LOADER.cloud_wrapper.list_objects(
|
||||||
|
credential_name => 'DEF_CRED_ARN',
|
||||||
|
location_uri => 'https://objectstorage.eu-frankfurt-1.oraclecloud.com/n/frtgjxu7zl7c/b/data/'
|
||||||
|
)) WHERE object_name LIKE 'test_export%';
|
||||||
|
```
|
||||||
|
|
||||||
|
## Connection Management
|
||||||
|
|
||||||
|
### User Credentials
|
||||||
|
- **Password**: All users use `Cloudpass#34`
|
||||||
|
- **Service**: All connect via `ggmichalski_high` service
|
||||||
|
- **Available Users**: ADMIN, MRDS_LOADER, MICHALZ, CT_MRDS, CT_ODS, ODS, CT_ET_TEMPLATES, OU_TOP
|
||||||
|
|
||||||
|
### SQLcl Connection Commands
|
||||||
|
```sql
|
||||||
|
-- Connect to database as specific user
|
||||||
|
connect ADMIN/Cloudpass#34@ggmichalski_high
|
||||||
|
connect CT_MRDS/Cloudpass#34@ggmichalski_high
|
||||||
|
connect MRDS_LOADER/Cloudpass#34@ggmichalski_high
|
||||||
|
|
||||||
|
-- Connection management
|
||||||
|
connmgr list -- List all saved connections
|
||||||
|
connmgr test ODS@ggmichalski_high -- Test specific connection
|
||||||
|
```
|
||||||
|
|
||||||
|
## Critical Implementation Patterns
|
||||||
|
|
||||||
|
### MCP Server Integration
|
||||||
|
The system supports Model Context Protocol (MCP) servers for enhanced database operations:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Use MCP SQLcl tools for database operations
|
||||||
|
mcp_sqlcl_connect("CT_MRDS@ggmichalski_high")
|
||||||
|
mcp_sqlcl_run-sql("EXEC FILE_MANAGER.EXPORT_TABLE_DATA(...)")
|
||||||
|
mcp_pylance_mcp_s_pylanceRunCodeSnippet() -- For Python validation
|
||||||
|
```
|
||||||
|
|
||||||
|
### Configuration Management Pattern
|
||||||
|
Environment variables are centrally managed via `A_FILE_MANAGER_CONFIG` table.
|
||||||
|
|
||||||
|
**MANDATORY**: Update configuration table definition files before database changes.
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- UPDATE FILE: database/CT_MRDS/tables/A_FILE_MANAGER_CONFIG.sql
|
||||||
|
-- INCLUDE: Initial data setup with correct camelCase format
|
||||||
|
|
||||||
|
CREATE TABLE A_FILE_MANAGER_CONFIG (
|
||||||
|
ENVIRONMENT_ID VARCHAR2(50) NOT NULL,
|
||||||
|
CONFIG_VARIABLE VARCHAR2(100) NOT NULL,
|
||||||
|
CONFIG_VARIABLE_VALUE VARCHAR2(4000),
|
||||||
|
CONSTRAINT PK_A_FILE_MANAGER_CONFIG PRIMARY KEY (ENVIRONMENT_ID, CONFIG_VARIABLE)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Configuration variables loaded at package initialization
|
||||||
|
INSERT INTO A_FILE_MANAGER_CONFIG (ENVIRONMENT_ID, CONFIG_VARIABLE, CONFIG_VARIABLE_VALUE)
|
||||||
|
VALUES ('default', 'EnvironmentID', 'dev');
|
||||||
|
INSERT INTO A_FILE_MANAGER_CONFIG (ENVIRONMENT_ID, CONFIG_VARIABLE, CONFIG_VARIABLE_VALUE)
|
||||||
|
VALUES ('dev', 'Region', 'eu-frankfurt-1');
|
||||||
|
INSERT INTO A_FILE_MANAGER_CONFIG (ENVIRONMENT_ID, CONFIG_VARIABLE, CONFIG_VARIABLE_VALUE)
|
||||||
|
VALUES ('dev', 'NameSpace', 'frtgjxu7zl7c');
|
||||||
|
INSERT INTO A_FILE_MANAGER_CONFIG (ENVIRONMENT_ID, CONFIG_VARIABLE, CONFIG_VARIABLE_VALUE)
|
||||||
|
VALUES ('dev', 'InboxBucketName', 'inbox');
|
||||||
|
INSERT INTO A_FILE_MANAGER_CONFIG (ENVIRONMENT_ID, CONFIG_VARIABLE, CONFIG_VARIABLE_VALUE)
|
||||||
|
VALUES ('dev', 'DataBucketName', 'data');
|
||||||
|
INSERT INTO A_FILE_MANAGER_CONFIG (ENVIRONMENT_ID, CONFIG_VARIABLE, CONFIG_VARIABLE_VALUE)
|
||||||
|
VALUES ('dev', 'ArchiveBucketName', 'archive');
|
||||||
|
```
|
||||||
|
|
||||||
|
**CRITICAL**: Variable names are case-sensitive - use exact camelCase format as shown above.
|
||||||
|
|
||||||
|
### Parameter Logging Pattern
|
||||||
|
All procedures use standardized parameter logging:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Standard parameter formatting for logging
|
||||||
|
vParameters := ENV_MANAGER.FORMAT_PARAMETERS(SYS.ODCIVARCHAR2LIST(
|
||||||
|
'pSchemaName => '''||nvl(pSchemaName,'NULL')||'''',
|
||||||
|
'pTableName => '''||nvl(pTableName,'NULL')||''''
|
||||||
|
));
|
||||||
|
ENV_MANAGER.LOG_PROCESS_EVENT('Start','INFO', vParameters);
|
||||||
|
```
|
||||||
|
|
||||||
|
### Cloud Wrapper Integration
|
||||||
|
```sql
|
||||||
|
-- PIPELINED function pattern (for list_objects)
|
||||||
|
FUNCTION list_objects(...) RETURN DBMS_CLOUD_TYPES.LIST_OBJECT_RET_TAB PIPELINED IS
|
||||||
|
BEGIN
|
||||||
|
FOR rec IN (SELECT * FROM TABLE(DBMS_CLOUD.LIST_OBJECTS(...))) LOOP
|
||||||
|
PIPE ROW(rec);
|
||||||
|
END LOOP;
|
||||||
|
EXCEPTION
|
||||||
|
WHEN OTHERS THEN
|
||||||
|
RAISE_APPLICATION_ERROR(-20005, 'Error listing objects: ' || SQLERRM);
|
||||||
|
END;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Resource Principal Configuration
|
||||||
|
```sql
|
||||||
|
-- Enable Resource Principal (execute as ADMIN)
|
||||||
|
EXEC DBMS_CLOUD_ADMIN.ENABLE_RESOURCE_PRINCIPAL(username => 'CT_MRDS');
|
||||||
|
EXEC DBMS_CLOUD_ADMIN.ENABLE_RESOURCE_PRINCIPAL(username => 'MRDS_LOADER');
|
||||||
|
|
||||||
|
-- Test Resource Principal access
|
||||||
|
SELECT object_name FROM DBMS_CLOUD.LIST_OBJECTS(
|
||||||
|
credential_name => 'OCI$RESOURCE_PRINCIPAL',
|
||||||
|
location_uri => 'https://objectstorage.eu-frankfurt-1.oraclecloud.com/n/frtgjxu7zl7c/b/data/'
|
||||||
|
) WHERE ROWNUM <= 5;
|
||||||
|
```
|
||||||
|
|
||||||
|
## Debugging and Troubleshooting
|
||||||
|
|
||||||
|
### Package Compilation Validation
|
||||||
|
```sql
|
||||||
|
-- Check for compilation errors after changes
|
||||||
|
SELECT * FROM USER_ERRORS WHERE TYPE = 'PACKAGE' AND NAME = 'FILE_MANAGER';
|
||||||
|
```
|
||||||
|
|
||||||
|
### MCP Pylance Integration
|
||||||
|
For Python-related Oracle operations, use Pylance MCP tools:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Validate Python code syntax before database integration
|
||||||
|
mcp_pylance_mcp_s_pylanceSyntaxErrors(code, pythonVersion)
|
||||||
|
-- Check workspace Python environment
|
||||||
|
mcp_pylance_mcp_s_pylancePythonEnvironments(workspaceRoot)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Archive and Backup Patterns
|
||||||
|
```powershell
|
||||||
|
# Standard backup command for all core packages
|
||||||
|
7z a -pMojeSuperHaslo -mhe=on FM_arch.7z database\CT_MRDS\packages\FILE_MANAGER.sql database\CT_MRDS\packages\FILE_ARCHIVER.sql database\CT_MRDS\packages\ENV_MANAGER.sql database\ODS\packages\FILE_MANAGER_ODS.sql
|
||||||
|
```
|
||||||
|
|
||||||
|
## Key Development Principles
|
||||||
|
|
||||||
|
When working with this system:
|
||||||
|
|
||||||
|
### **PRIMARY PRINCIPLE: CODE-FIRST DEVELOPMENT**
|
||||||
|
1. **ALWAYS update source files BEFORE database changes** - maintain version control integrity
|
||||||
|
2. **Prepare complete file modifications** before any database deployment
|
||||||
|
3. **Validate changes in files** before executing on database
|
||||||
|
4. **Document structural changes** in corresponding .sql files
|
||||||
|
|
||||||
|
### **SYSTEM-SPECIFIC PRINCIPLES**
|
||||||
|
5. **Always use MSG_* constants** for error messages to prevent session-level duplication
|
||||||
|
6. **Follow the privilege separation model** - MRDS_LOADER owns cloud operations, others get EXECUTE grants
|
||||||
|
7. **Use standardized parameter logging** with FORMAT_PARAMETERS for all procedures
|
||||||
|
8. **Test bucket area mappings** before deploying export procedures
|
||||||
|
9. **Leverage MCP tools** for database operations and Python validation when available
|
||||||
|
6. **Hive-style partitioning is fully supported** - DBMS_CLOUD.EXPORT_DATA creates PARTITION_YEAR=/PARTITION_MONTH= structure
|
||||||
|
7. **Use triggers for auto-key generation** - TRG_BI_A_SOURCE_FILE_CONFIG_CHECK handles A_SOURCE_FILE_CONFIG_KEY
|
||||||
|
8. **Follow configuration sequence** - ADD_SOURCE → ADD_SOURCE_FILE_CONFIG → ADD_COLUMN_DATE_FORMAT
|
||||||
|
9. **Use ODS.FILE_MANAGER_ODS package** for external table creation (AUTHID DEFINER) over CT_MRDS.FILE_MANAGER (AUTHID CURRENT_USER)
|
||||||
|
10. **Implement comprehensive validation** - Use PROCESS_SOURCE_FILE for complete 6-step file validation workflow
|
||||||
|
11. **Utilize ANALYZE_VALIDATION_ERRORS** for detailed file validation error analysis and troubleshooting
|
||||||
|
12. **Safe configuration removal** - Use DELETE_SOURCE_CASCADE for complete source system cleanup with referential integrity
|
||||||
|
13. **Template table restriction** - Template tables can ONLY be created in CT_ET_TEMPLATES schema by ADMIN or CT_ET_TEMPLATES user, not by MRDS_LOADER
|
||||||
|
14. **MRDS_LOADER external table creation** - Use MRDS_LOADER connection with ODS.FILE_MANAGER_ODS package for creating external tables in ODS schema (requires proper privilege setup for both MRDS_LOADER and ODS schemas)
|
||||||
|
15. **Cross-schema privilege requirements** - When using FILE_MANAGER_ODS package, ensure both MRDS_LOADER and ODS have sufficient privileges to CT_MRDS schema objects (especially A_PROCESS_LOG and ENV_MANAGER)
|
||||||
|
16. **MANDATORY PATH PATTERNS** - Always use official patterns: INBOX (3-level), ODS (2-level), ARCHIVE (2-level) - no simplified versions allowed
|
||||||
|
17. **Path pattern compliance** - INBOX requires SOURCE_FILE_ID, while ODS and ARCHIVE omit it for simplified access
|
||||||
|
|
||||||
|
### **Common File Modification Scenarios**
|
||||||
|
|
||||||
|
**Adding New Columns to Configuration Tables:**
|
||||||
|
```sql
|
||||||
|
-- 1. UPDATE STRUCTURE DEFINITION: database/CT_MRDS/tables/A_SOURCE_FILE_CONFIG.sql
|
||||||
|
-- Modify the complete CREATE TABLE statement to include new columns
|
||||||
|
CREATE TABLE A_SOURCE_FILE_CONFIG (
|
||||||
|
A_SOURCE_FILE_CONFIG_KEY NUMBER NOT NULL,
|
||||||
|
-- ... existing columns ...
|
||||||
|
NEW_COLUMN VARCHAR2(50) DEFAULT 'default_value', -- ADD THIS LINE
|
||||||
|
CONSTRAINT PK_A_SOURCE_FILE_CONFIG PRIMARY KEY (A_SOURCE_FILE_CONFIG_KEY)
|
||||||
|
);
|
||||||
|
COMMENT ON COLUMN A_SOURCE_FILE_CONFIG.NEW_COLUMN IS 'Description';
|
||||||
|
|
||||||
|
-- 2. CREATE MIGRATION SCRIPT: database/migrations/XXX_add_new_column.sql
|
||||||
|
ALTER TABLE CT_MRDS.A_SOURCE_FILE_CONFIG ADD (
|
||||||
|
NEW_COLUMN VARCHAR2(50) DEFAULT 'default_value'
|
||||||
|
);
|
||||||
|
COMMENT ON COLUMN CT_MRDS.A_SOURCE_FILE_CONFIG.NEW_COLUMN IS 'Description';
|
||||||
|
|
||||||
|
-- 3. CREATE ROLLBACK SCRIPT: database/migrations/rollback/XXX_rollback_new_column.sql
|
||||||
|
ALTER TABLE CT_MRDS.A_SOURCE_FILE_CONFIG DROP COLUMN NEW_COLUMN;
|
||||||
|
|
||||||
|
-- 4. UPDATE PACKAGE CODE: database/CT_MRDS/packages/FILE_MANAGER.sql
|
||||||
|
-- Add logic to utilize new configuration column
|
||||||
|
|
||||||
|
-- 5. DEPLOY: Execute migration first, then updated packages
|
||||||
|
```
|
||||||
|
|
||||||
|
**Modifying Package Procedures:**
|
||||||
|
```sql
|
||||||
|
-- 1. UPDATE PACKAGE DEFINITION: database/CT_MRDS/packages/FILE_MANAGER.sql
|
||||||
|
-- Modify both .pks (specification) and .pkb (body) sections in the same file
|
||||||
|
|
||||||
|
-- 2. UPDATE WRAPPER PACKAGES: database/ODS/packages/FILE_MANAGER_ODS.sql
|
||||||
|
-- Update related wrapper packages if applicable
|
||||||
|
|
||||||
|
-- 3. DEPLOY: Execute updated package files
|
||||||
|
Get-Content "database/CT_MRDS/packages/FILE_MANAGER.sql" | sql "CT_MRDS/password@service"
|
||||||
|
Get-Content "database/ODS/packages/FILE_MANAGER_ODS.sql" | sql "ODS/password@service"
|
||||||
|
|
||||||
|
-- 4. UPDATE DOCUMENTATION: confluence/ directory
|
||||||
|
```
|
||||||
|
|
||||||
|
**Adding New Template Tables:**
|
||||||
|
```sql
|
||||||
|
-- 1. CREATE STRUCTURE DEFINITION: database/CT_ET_TEMPLATES/tables/NEW_TEMPLATE.sql
|
||||||
|
CREATE TABLE CT_ET_TEMPLATES.NEW_TEMPLATE (
|
||||||
|
-- Column definitions matching expected CSV structure
|
||||||
|
COL1 VARCHAR2(50),
|
||||||
|
COL2 NUMBER,
|
||||||
|
COL3 DATE
|
||||||
|
);
|
||||||
|
|
||||||
|
-- 2. CREATE MIGRATION SCRIPT: database/migrations/XXX_create_new_template.sql
|
||||||
|
-- (Same CREATE TABLE statement for deployment)
|
||||||
|
|
||||||
|
-- 3. UPDATE CONFIGURATION: Add new source file config entries
|
||||||
|
-- 4. UPDATE EXTERNAL TABLE SCRIPTS: If needed for this template
|
||||||
|
```
|
||||||
|
|
||||||
|
## Code Style Guidelines
|
||||||
|
|
||||||
|
**Text Formatting**:
|
||||||
|
- **NO EMOJI/ICONS**: Do not use emoji, Unicode symbols, or decorative icons in code, comments, documentation, or console output
|
||||||
|
- Use plain text with clear, professional formatting
|
||||||
|
- Prefer descriptive text labels over visual symbols
|
||||||
|
- Example: Use "ERROR:" instead of "🚨", "LOCATION:" instead of "📍"
|
||||||
|
|
||||||
|
**Line Break Formatting**:
|
||||||
|
- **Use cgBL constant**: Always use the predefined constant `cgBL CONSTANT VARCHAR2(2) := CHR(13)||CHR(10);` for line breaks in error messages and logging
|
||||||
|
- Replace `CHR(10)` with `cgBL` for consistent CRLF formatting across Windows environments
|
||||||
|
- Exception: Keep `CHR(10)` in REGEXP_COUNT functions where it's required for line counting operations
|
||||||
|
|
||||||
|
**Professional Output**:
|
||||||
|
- Keep all logging, error messages, and documentation strictly text-based
|
||||||
|
- Use standard ASCII characters and proper formatting
|
||||||
|
- Maintain enterprise-grade professional appearance
|
||||||
|
|
||||||
|
**Oracle DDL File Standards**:
|
||||||
|
- **MANDATORY**: All Oracle DDL files (.pkg, .pkb, .sql for packages/triggers) must end with forward slash `/`
|
||||||
|
- **Placement**: Add `/` after final `END;` statement with blank line separation
|
||||||
|
- **Example**: `END;` followed by blank line, then `/` on separate line
|
||||||
|
- **Scope**: Applies to all packages (specification and body), triggers, and PL/SQL blocks
|
||||||
|
- **Purpose**: Ensures proper Oracle SQL*Plus and SQLcl termination of PL/SQL blocks
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
The system is designed for enterprise-scale data processing with proper logging, error management, and OCI integration.
|
||||||
29
.gitignore
vendored
Normal file
29
.gitignore
vendored
Normal file
@@ -0,0 +1,29 @@
|
|||||||
|
# Temporary files and directories
|
||||||
|
tmp/
|
||||||
|
|
||||||
|
# Issues directory
|
||||||
|
issues/
|
||||||
|
|
||||||
|
# IDE and editor files
|
||||||
|
.idea/
|
||||||
|
*.swp
|
||||||
|
*.swo
|
||||||
|
*~
|
||||||
|
|
||||||
|
# OS generated files
|
||||||
|
.DS_Store
|
||||||
|
.DS_Store?
|
||||||
|
._*
|
||||||
|
.Spotlight-V100
|
||||||
|
.Trashes
|
||||||
|
ehthumbs.db
|
||||||
|
Thumbs.db
|
||||||
|
|
||||||
|
MARS_Packages/REL01/MARS-1056/confluence/
|
||||||
|
MARS_Packages/REL01/MARS-1056/log/
|
||||||
|
MARS_Packages/REL01/MARS-1046/confluence/
|
||||||
|
MARS_Packages/REL01/MARS-1046/log/
|
||||||
|
MARS_Packages/REL01/MARS-1046/test/
|
||||||
|
|
||||||
|
MARS_Packages/REL01/MARS-1096/log/
|
||||||
|
|
||||||
127
.vscode/settings.json
vendored
Normal file
127
.vscode/settings.json
vendored
Normal file
@@ -0,0 +1,127 @@
|
|||||||
|
{
|
||||||
|
"files.associations": {
|
||||||
|
"*.sql": "oracle-sql",
|
||||||
|
"*.pkg": "oracle-sql",
|
||||||
|
"*.pkb": "oracle-sql"
|
||||||
|
},
|
||||||
|
"[oracle-sql]": {
|
||||||
|
"editor.defaultFormatter": "Oracle.sql-developer"
|
||||||
|
},
|
||||||
|
"java.home": "C:\\apps\\Amazon Corretto\\jdk17.0.11_9",
|
||||||
|
"oracle.developer.javaHome": "C:\\apps\\Amazon Corretto\\jdk17.0.11_9",
|
||||||
|
"java.configuration.runtimes": [
|
||||||
|
{
|
||||||
|
"name": "JavaSE-17",
|
||||||
|
"path": "C:\\apps\\Amazon Corretto\\jdk17.0.11_9"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Oracle-JDK",
|
||||||
|
"path": "C:\\apps\\sqldeveloper\\jdk\\jre"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"java.import.gradle.java.home": "C:\\apps\\Amazon Corretto\\jdk17.0.11_9",
|
||||||
|
"oracle.developer.sqlcl.path": "sql",
|
||||||
|
"oracle.developer.enableSQLCL": true,
|
||||||
|
"oracle.developer.enableLanguageServer": true,
|
||||||
|
"oracle.developer.walletLocation": "c:\\_git\\OracleAI\\oracledb1\\Wallet_ggmichalski",
|
||||||
|
"oracle.developer.sqlDeveloperHome": "C:\\apps\\sqldeveloper",
|
||||||
|
"oracle.developer.enableJDBCDrivers": true,
|
||||||
|
"oracle.developer.debug.enabled": true,
|
||||||
|
"oracle.developer.logging.level": "ALL",
|
||||||
|
"oracle.developer.startup.timeout": 120000,
|
||||||
|
"oracle.developer.jvm.maxMemory": "2g",
|
||||||
|
"oracle.developer.jvm.initialMemory": "512m",
|
||||||
|
"oracle.developer.connectionTimeout": 60000,
|
||||||
|
"oracle.developer.tnsAdminLocation": "c:\\_git\\OracleAI\\oracledb1\\Wallet_ggmichalski",
|
||||||
|
"oracle.developer.oracleHome": "",
|
||||||
|
"oracle.developer.disableOracleHomeDetection": true,
|
||||||
|
"oracle.developer.queryTimeout": 300000,
|
||||||
|
"oracle.developer.useWalletOnly": true,
|
||||||
|
"oracle.developer.ignoreSystemOracleHome": true,
|
||||||
|
"oracle.developer.overrideSystemVariables": true,
|
||||||
|
"oracle.developer.forceWalletConnections": true,
|
||||||
|
"oracle.developer.bypassTnsLookup": true,
|
||||||
|
"terminal.integrated.env.windows": {
|
||||||
|
"TNS_ADMIN": "c:\\_git\\OracleAI\\oracledb1\\Wallet_ggmichalski",
|
||||||
|
"ORACLE_HOME": "",
|
||||||
|
"ORACLE_BASE": "",
|
||||||
|
"ORA_JAVA_HOME": "C:\\apps\\Amazon Corretto\\jdk17.0.11_9"
|
||||||
|
},
|
||||||
|
"oracledevtools.download.otherPlatforms": true,
|
||||||
|
"oracledevtools.bookmarkFileLocation": "",
|
||||||
|
"oracle.connectionConfiguration": {
|
||||||
|
"defaultConnection": "ADMIN@ggmichalski_high",
|
||||||
|
"connections": [
|
||||||
|
{
|
||||||
|
"name": "ADMIN@ggmichalski_high",
|
||||||
|
"username": "ADMIN",
|
||||||
|
"password": "Cloudpass#34",
|
||||||
|
"connectionString": "ggmichalski_high",
|
||||||
|
"walletLocation": "c:\\_git\\OracleAI\\oracledb1\\Wallet_ggmichalski"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "MICHALZ@ggmichalski_high",
|
||||||
|
"username": "MICHALZ",
|
||||||
|
"password": "Cloudpass#34",
|
||||||
|
"connectionString": "ggmichalski_high",
|
||||||
|
"walletLocation": "c:\\_git\\OracleAI\\oracledb1\\Wallet_ggmichalski"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "MRDS_LOADER@ggmichalski_high",
|
||||||
|
"username": "MRDS_LOADER",
|
||||||
|
"password": "Cloudpass#34",
|
||||||
|
"connectionString": "ggmichalski_high",
|
||||||
|
"walletLocation": "c:\\_git\\OracleAI\\oracledb1\\Wallet_ggmichalski"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "CT_MRDS@ggmichalski_high",
|
||||||
|
"username": "CT_MRDS",
|
||||||
|
"password": "Cloudpass#34",
|
||||||
|
"connectionString": "ggmichalski_high",
|
||||||
|
"walletLocation": "c:\\_git\\OracleAI\\oracledb1\\Wallet_ggmichalski"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "ODS@ggmichalski_high",
|
||||||
|
"username": "ODS",
|
||||||
|
"password": "Cloudpass#34",
|
||||||
|
"connectionString": "ggmichalski_high",
|
||||||
|
"walletLocation": "c:\\_git\\OracleAI\\oracledb1\\Wallet_ggmichalski"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "CT_ODS@ggmichalski_high",
|
||||||
|
"username": "CT_ODS",
|
||||||
|
"password": "Cloudpass#34",
|
||||||
|
"connectionString": "ggmichalski_high",
|
||||||
|
"walletLocation": "c:\\_git\\OracleAI\\oracledb1\\Wallet_ggmichalski"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "CT_ET_TEMPLATES@ggmichalski_high",
|
||||||
|
"username": "CT_ET_TEMPLATES",
|
||||||
|
"password": "Cloudpass#34",
|
||||||
|
"connectionString": "ggmichalski_high",
|
||||||
|
"walletLocation": "c:\\_git\\OracleAI\\oracledb1\\Wallet_ggmichalski"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "OU_TOP@ggmichalski_high",
|
||||||
|
"username": "OU_TOP",
|
||||||
|
"password": "Cloudpass#34",
|
||||||
|
"connectionString": "ggmichalski_high",
|
||||||
|
"walletLocation": "c:\\_git\\OracleAI\\oracledb1\\Wallet_ggmichalski"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "OU_LM@ggmichalski_high",
|
||||||
|
"username": "OU_LM",
|
||||||
|
"password": "Cloudpass#34",
|
||||||
|
"connectionString": "ggmichalski_high",
|
||||||
|
"walletLocation": "c:\\_git\\OracleAI\\oracledb1\\Wallet_ggmichalski"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "OU_MRR@ggmichalski_high",
|
||||||
|
"username": "OU_MRR",
|
||||||
|
"password": "Cloudpass#34",
|
||||||
|
"connectionString": "ggmichalski_high",
|
||||||
|
"walletLocation": "c:\\_git\\OracleAI\\oracledb1\\Wallet_ggmichalski"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
SET SERVEROUTPUT ON;
|
||||||
|
SET ECHO ON;
|
||||||
|
|
||||||
|
prompt ========================================
|
||||||
|
prompt Installing WORKFLOW_MANAGER Package Specification
|
||||||
|
prompt MARS-1011: Update WORKFLOW_MANAGER v1.7.1
|
||||||
|
prompt ========================================
|
||||||
|
|
||||||
|
@@WORKFLOW_MANAGER_SPEC.sql
|
||||||
|
|
||||||
|
prompt ========================================
|
||||||
|
prompt WORKFLOW_MANAGER Package Specification installed successfully
|
||||||
|
prompt ========================================
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
SET SERVEROUTPUT ON;
|
||||||
|
SET ECHO ON;
|
||||||
|
|
||||||
|
prompt ========================================
|
||||||
|
prompt Installing WORKFLOW_MANAGER Package Body
|
||||||
|
prompt MARS-1011: Update WORKFLOW_MANAGER v1.7.1
|
||||||
|
prompt Adding SERVICE_NAME='ODS' to INIT_TASK function
|
||||||
|
prompt ========================================
|
||||||
|
|
||||||
|
@@WORKFLOW_MANAGER_BODY.sql
|
||||||
|
|
||||||
|
prompt ========================================
|
||||||
|
prompt WORKFLOW_MANAGER Package Body installed successfully
|
||||||
|
prompt ========================================
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
SET SERVEROUTPUT ON;
|
||||||
|
SET ECHO ON;
|
||||||
|
|
||||||
|
prompt ========================================
|
||||||
|
prompt Verifying WORKFLOW_MANAGER Package Installation
|
||||||
|
prompt MARS-1011: Update WORKFLOW_MANAGER v1.7.1
|
||||||
|
prompt ========================================
|
||||||
|
|
||||||
|
prompt Checking package compilation status...
|
||||||
|
SELECT object_name, object_type, status
|
||||||
|
FROM user_objects
|
||||||
|
WHERE object_name = 'WORKFLOW_MANAGER'
|
||||||
|
AND object_type IN ('PACKAGE', 'PACKAGE BODY');
|
||||||
|
|
||||||
|
prompt Checking package version...
|
||||||
|
SELECT CT_MRDS.WORKFLOW_MANAGER.GET_VERSION() AS VERSION FROM DUAL;
|
||||||
|
|
||||||
|
prompt Checking build info...
|
||||||
|
SELECT CT_MRDS.WORKFLOW_MANAGER.GET_BUILD_INFO() AS BUILD_INFO FROM DUAL;
|
||||||
|
|
||||||
|
prompt ========================================
|
||||||
|
prompt WORKFLOW_MANAGER Package verification completed
|
||||||
|
prompt Expected version: 1.7.1
|
||||||
|
prompt ========================================
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
SET SERVEROUTPUT ON;
|
||||||
|
SET ECHO ON;
|
||||||
|
|
||||||
|
prompt ========================================
|
||||||
|
prompt Tracking WORKFLOW_MANAGER Package Version
|
||||||
|
prompt MARS-1011: Record version 1.7.1 in tracking system
|
||||||
|
prompt ========================================
|
||||||
|
|
||||||
|
BEGIN
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Tracking WORKFLOW_MANAGER version...');
|
||||||
|
CT_MRDS.ENV_MANAGER.TRACK_PACKAGE_VERSION(
|
||||||
|
pPackageOwner => 'CT_MRDS',
|
||||||
|
pPackageName => 'WORKFLOW_MANAGER',
|
||||||
|
pPackageVersion => CT_MRDS.WORKFLOW_MANAGER.PACKAGE_VERSION,
|
||||||
|
pPackageBuildDate => CT_MRDS.WORKFLOW_MANAGER.PACKAGE_BUILD_DATE,
|
||||||
|
pPackageAuthor => CT_MRDS.WORKFLOW_MANAGER.PACKAGE_AUTHOR
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('WORKFLOW_MANAGER version tracked successfully.');
|
||||||
|
|
||||||
|
-- Display current version info
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Current Version: ' || CT_MRDS.WORKFLOW_MANAGER.GET_VERSION());
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Build Info: ' || CT_MRDS.WORKFLOW_MANAGER.GET_BUILD_INFO());
|
||||||
|
EXCEPTION
|
||||||
|
WHEN OTHERS THEN
|
||||||
|
DBMS_OUTPUT.PUT_LINE('ERROR tracking WORKFLOW_MANAGER: ' || SQLERRM);
|
||||||
|
RAISE;
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
prompt ========================================
|
||||||
|
prompt WORKFLOW_MANAGER Package version tracking completed
|
||||||
|
prompt ========================================
|
||||||
@@ -0,0 +1,201 @@
|
|||||||
|
SET SERVEROUTPUT ON;
|
||||||
|
SET ECHO ON;
|
||||||
|
|
||||||
|
prompt ========================================
|
||||||
|
prompt Rollback: WORKFLOW_MANAGER Package Body
|
||||||
|
prompt MARS-1011: Rollback to version 1.7.0 (Remove SERVICE_NAME from INIT_TASK)
|
||||||
|
prompt ========================================
|
||||||
|
|
||||||
|
-- Create package body with INIT_TASK without SERVICE_NAME (rollback to v1.7.0 behavior)
|
||||||
|
CREATE OR REPLACE PACKAGE BODY CT_MRDS.WORKFLOW_MANAGER
|
||||||
|
IS
|
||||||
|
|
||||||
|
FUNCTION INIT_WORKFLOW(pServiceName IN VARCHAR2, pWorkflowRunId IN VARCHAR2, pWorkflowName in VARCHAR2)
|
||||||
|
RETURN NUMBER
|
||||||
|
IS
|
||||||
|
vWorkflowHistoryKey NUMBER;
|
||||||
|
BEGIN
|
||||||
|
vWorkflowHistoryKey := A_WORKFLOW_HISTORY_KEY_SEQ.NEXTVAL;
|
||||||
|
INSERT INTO CT_MRDS.A_WORKFLOW_HISTORY (SERVICE_NAME, A_WORKFLOW_HISTORY_KEY, ORCHESTRATION_RUN_ID,
|
||||||
|
WORKFLOW_NAME, WORKFLOW_START, WORKFLOW_SUCCESSFUL)
|
||||||
|
VALUES (pServiceName, vWorkflowHistoryKey, pWorkflowRunId,
|
||||||
|
pWorkflowName, SYSTIMESTAMP, vpRunningStatus);
|
||||||
|
|
||||||
|
return vWorkflowHistoryKey;
|
||||||
|
END INIT_WORKFLOW;
|
||||||
|
|
||||||
|
PROCEDURE FINALISE_WORKFLOW(pWorkflowHistoryKey IN NUMBER, pWorkflowStatus IN VARCHAR2)
|
||||||
|
IS
|
||||||
|
BEGIN
|
||||||
|
FINALISE_WORKFLOW(pWorkflowHistoryKey, NULL, pWorkflowStatus);
|
||||||
|
END;
|
||||||
|
|
||||||
|
PROCEDURE FINALISE_WORKFLOW(pWorkflowHistoryKey IN NUMBER, pServiceName IN VARCHAR2, pWorkflowStatus IN VARCHAR2)
|
||||||
|
IS
|
||||||
|
BEGIN
|
||||||
|
UPDATE CT_MRDS.A_WORKFLOW_HISTORY SET WORKFLOW_SUCCESSFUL = pWorkflowStatus,
|
||||||
|
WORKFLOW_END = SYSTIMESTAMP
|
||||||
|
WHERE A_WORKFLOW_HISTORY_KEY = pWorkflowHistoryKey
|
||||||
|
AND SERVICE_NAME = NVL(pServiceName,SERVICE_NAME);
|
||||||
|
END FINALISE_WORKFLOW;
|
||||||
|
|
||||||
|
-- ROLLBACK: INIT_TASK without SERVICE_NAME (version 1.7.0 behavior)
|
||||||
|
FUNCTION INIT_TASK(pTaskRunId IN VARCHAR2, pTaskName in VARCHAR2, pWorkflowHistoryKey IN NUMBER)
|
||||||
|
RETURN NUMBER
|
||||||
|
IS
|
||||||
|
vTaskHistoryKey NUMBER;
|
||||||
|
BEGIN
|
||||||
|
vTaskHistoryKey := A_TASK_HISTORY_KEY_SEQ.NEXTVAL;
|
||||||
|
INSERT INTO CT_MRDS.A_TASK_HISTORY (A_TASK_HISTORY_KEY, TASK_RUN_ID, A_WORKFLOW_HISTORY_KEY,
|
||||||
|
TASK_NAME, TASK_START, TASK_SUCCESSFUL)
|
||||||
|
VALUES (vTaskHistoryKey, pTaskRunId, pWorkflowHistoryKey,
|
||||||
|
pTaskName, SYSTIMESTAMP, vpRunningStatus);
|
||||||
|
|
||||||
|
return vTaskHistoryKey;
|
||||||
|
END INIT_TASK;
|
||||||
|
|
||||||
|
PROCEDURE FINALISE_TASK(pTaskHistoryKey IN NUMBER, pTaskStatus IN VARCHAR2)
|
||||||
|
IS
|
||||||
|
BEGIN
|
||||||
|
UPDATE CT_MRDS.A_TASK_HISTORY SET TASK_SUCCESSFUL = pTaskStatus,
|
||||||
|
TASK_END = SYSTIMESTAMP
|
||||||
|
WHERE A_TASK_HISTORY_KEY = pTaskHistoryKey;
|
||||||
|
END FINALISE_TASK;
|
||||||
|
|
||||||
|
-- Other functions remain the same as current version
|
||||||
|
FUNCTION STORE_TASK_SOURCE_INFO_INTERNAL(pTaskHistoryKey IN NUMBER, pSourceName IN VARCHAR2, pNumRows IN NUMBER)
|
||||||
|
RETURN NUMBER
|
||||||
|
IS
|
||||||
|
vTaskHistorySourceKey NUMBER;
|
||||||
|
BEGIN
|
||||||
|
vTaskHistorySourceKey := A_TASK_HISTORY_SOURCE_KEY_SEQ.NEXTVAL;
|
||||||
|
INSERT INTO CT_MRDS.A_TASK_HISTORY_SOURCE (A_TASK_HISTORY_SOURCE_KEY, A_TASK_HISTORY_KEY,
|
||||||
|
SOURCE_NAME, ROW_COUNT)
|
||||||
|
VALUES (vTaskHistorySourceKey, pTaskHistoryKey,
|
||||||
|
pSourceName, pNumRows);
|
||||||
|
RETURN vTaskHistorySourceKey;
|
||||||
|
END;
|
||||||
|
|
||||||
|
FUNCTION STORE_TASK_SOURCE_INFO(pTaskHistoryKey IN NUMBER, pSourceName IN VARCHAR2, pNumRows IN NUMBER)
|
||||||
|
RETURN NUMBER
|
||||||
|
IS
|
||||||
|
vTaskHistorySourceKey NUMBER;
|
||||||
|
BEGIN
|
||||||
|
vTaskHistorySourceKey := STORE_TASK_SOURCE_INFO_INTERNAL(pTaskHistoryKey, pSourceName, pNumRows);
|
||||||
|
COMMIT;
|
||||||
|
RETURN vTaskHistorySourceKey;
|
||||||
|
END;
|
||||||
|
|
||||||
|
FUNCTION STORE_TASK_TARGET_INFO_INTERNAL(pTaskHistoryKey IN NUMBER, pSourceName IN VARCHAR2,
|
||||||
|
pNumRowsApplied IN NUMBER, pNumRowsRejected IN NUMBER,
|
||||||
|
pLoadSuccessful IN VARCHAR2, pServiceName IN VARCHAR2)
|
||||||
|
RETURN NUMBER
|
||||||
|
IS
|
||||||
|
vTaskHistoryTargetKey NUMBER;
|
||||||
|
BEGIN
|
||||||
|
vTaskHistoryTargetKey := A_TASK_HISTORY_TARGET_KEY_SEQ.NEXTVAL;
|
||||||
|
INSERT INTO CT_MRDS.A_TASK_HISTORY_TARGET (A_TASK_HISTORY_TARGET_KEY, A_TASK_HISTORY_KEY,
|
||||||
|
TARGET_NAME, ROW_COUNT_APPLIED, ROW_COUNT_REJECTED,
|
||||||
|
LOAD_SUCCESSFUL, SERVICE_NAME)
|
||||||
|
VALUES (vTaskHistoryTargetKey, pTaskHistoryKey,
|
||||||
|
pSourceName, pNumRowsApplied, pNumRowsRejected,
|
||||||
|
pLoadSuccessful, pServiceName);
|
||||||
|
RETURN vTaskHistoryTargetKey;
|
||||||
|
END;
|
||||||
|
|
||||||
|
FUNCTION STORE_TASK_TARGET_INFO(pTaskHistoryKey IN NUMBER, pSourceName IN VARCHAR2,
|
||||||
|
pNumRowsApplied IN NUMBER, pNumRowsRejected IN NUMBER,
|
||||||
|
pLoadSuccessful IN VARCHAR2, pServiceName IN VARCHAR2)
|
||||||
|
RETURN NUMBER
|
||||||
|
IS
|
||||||
|
vTaskHistoryTargetKey NUMBER;
|
||||||
|
BEGIN
|
||||||
|
vTaskHistoryTargetKey := STORE_TASK_TARGET_INFO_INTERNAL(
|
||||||
|
pTaskHistoryKey, pSourceName, pNumRowsApplied,
|
||||||
|
pNumRowsRejected, pLoadSuccessful, pServiceName
|
||||||
|
);
|
||||||
|
COMMIT;
|
||||||
|
RETURN vTaskHistoryTargetKey;
|
||||||
|
END;
|
||||||
|
|
||||||
|
FUNCTION FINALISE_TASK_WITH_TARGET_INFO(pTaskHistoryKey IN NUMBER, pTaskStatus IN VARCHAR2,
|
||||||
|
pTargetName IN VARCHAR2, pNumRowsApplied IN NUMBER,
|
||||||
|
pNumRowsRejected IN NUMBER, pLoadSuccessful IN VARCHAR2,
|
||||||
|
pServiceName IN VARCHAR2)
|
||||||
|
RETURN NUMBER
|
||||||
|
IS
|
||||||
|
vTaskHistoryTargetKey NUMBER;
|
||||||
|
BEGIN
|
||||||
|
vTaskHistoryTargetKey := STORE_TASK_TARGET_INFO_INTERNAL(
|
||||||
|
pTaskHistoryKey => pTaskHistoryKey,
|
||||||
|
pSourceName => pTargetName,
|
||||||
|
pNumRowsApplied => pNumRowsApplied,
|
||||||
|
pNumRowsRejected => pNumRowsRejected,
|
||||||
|
pLoadSuccessful => pLoadSuccessful,
|
||||||
|
pServiceName => pServiceName
|
||||||
|
);
|
||||||
|
|
||||||
|
FINALISE_TASK(pTaskHistoryKey, pTaskStatus);
|
||||||
|
COMMIT;
|
||||||
|
RETURN vTaskHistoryTargetKey;
|
||||||
|
END;
|
||||||
|
|
||||||
|
PROCEDURE SET_WORKFLOW_PROPERTY(
|
||||||
|
pWorkflowHistoryKey IN NUMBER
|
||||||
|
,pServiceName IN VARCHAR2
|
||||||
|
,pProperty IN VARCHAR2
|
||||||
|
,pValue IN VARCHAR2
|
||||||
|
) IS
|
||||||
|
BEGIN
|
||||||
|
INSERT INTO CT_MRDS.A_WORKFLOW_HISTORY_PROPERTY (A_WORKFLOW_HISTORY_KEY, SERVICE_NAME, PROPERTY, VALUE)
|
||||||
|
VALUES (pWorkflowHistoryKey, pServiceName, pProperty, pValue);
|
||||||
|
END;
|
||||||
|
|
||||||
|
FUNCTION GET_WORKFLOW_PROPERTY(
|
||||||
|
pWorkflowHistoryKey IN NUMBER
|
||||||
|
,pServiceName IN VARCHAR2
|
||||||
|
,pProperty IN VARCHAR2
|
||||||
|
) RETURN VARCHAR2
|
||||||
|
IS
|
||||||
|
vValue CT_MRDS.A_WORKFLOW_HISTORY_PROPERTY.VALUE%TYPE;
|
||||||
|
BEGIN
|
||||||
|
SELECT VALUE
|
||||||
|
INTO vValue
|
||||||
|
FROM CT_MRDS.A_WORKFLOW_HISTORY_PROPERTY
|
||||||
|
WHERE A_WORKFLOW_HISTORY_KEY = pWorkflowHistoryKey
|
||||||
|
AND SERVICE_NAME = pServiceName
|
||||||
|
AND PROPERTY = pProperty;
|
||||||
|
|
||||||
|
RETURN vValue;
|
||||||
|
END;
|
||||||
|
|
||||||
|
FUNCTION GET_VERSION RETURN VARCHAR2 IS
|
||||||
|
BEGIN
|
||||||
|
RETURN PACKAGE_VERSION;
|
||||||
|
END GET_VERSION;
|
||||||
|
|
||||||
|
FUNCTION GET_BUILD_INFO RETURN VARCHAR2 IS
|
||||||
|
BEGIN
|
||||||
|
RETURN ENV_MANAGER.GET_PACKAGE_VERSION_INFO(
|
||||||
|
pPackageName => 'WORKFLOW_MANAGER',
|
||||||
|
pVersion => PACKAGE_VERSION,
|
||||||
|
pBuildDate => PACKAGE_BUILD_DATE,
|
||||||
|
pAuthor => PACKAGE_AUTHOR
|
||||||
|
);
|
||||||
|
END GET_BUILD_INFO;
|
||||||
|
|
||||||
|
FUNCTION GET_VERSION_HISTORY RETURN VARCHAR2 IS
|
||||||
|
BEGIN
|
||||||
|
RETURN ENV_MANAGER.FORMAT_VERSION_HISTORY(
|
||||||
|
pPackageName => 'WORKFLOW_MANAGER',
|
||||||
|
pVersionHistory => VERSION_HISTORY
|
||||||
|
);
|
||||||
|
END GET_VERSION_HISTORY;
|
||||||
|
|
||||||
|
END WORKFLOW_MANAGER;
|
||||||
|
/
|
||||||
|
|
||||||
|
prompt ========================================
|
||||||
|
prompt WORKFLOW_MANAGER Package Body rolled back to v1.7.0
|
||||||
|
prompt INIT_TASK function no longer includes SERVICE_NAME
|
||||||
|
prompt ========================================
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
SET SERVEROUTPUT ON;
|
||||||
|
SET ECHO ON;
|
||||||
|
|
||||||
|
prompt ========================================
|
||||||
|
prompt Rollback: WORKFLOW_MANAGER Package Specification
|
||||||
|
prompt MARS-1011: Rollback to version 1.7.0 (2025-10-31)
|
||||||
|
prompt ========================================
|
||||||
|
|
||||||
|
-- Rollback WORKFLOW_MANAGER specification to version 1.7.0
|
||||||
|
CREATE OR REPLACE PACKAGE CT_MRDS.WORKFLOW_MANAGER
|
||||||
|
IS
|
||||||
|
|
||||||
|
-- Package Version Information (Semantic Versioning: MAJOR.MINOR.PATCH)
|
||||||
|
PACKAGE_VERSION CONSTANT VARCHAR2(10) := '1.7.0';
|
||||||
|
PACKAGE_BUILD_DATE CONSTANT VARCHAR2(20) := '2025-10-31 16:00:00';
|
||||||
|
PACKAGE_AUTHOR CONSTANT VARCHAR2(100) := 'Grzegorz Michalski';
|
||||||
|
|
||||||
|
-- Version History (Latest changes first)
|
||||||
|
VERSION_HISTORY CONSTANT VARCHAR2(4000) :=
|
||||||
|
'1.7.0 (2025-10-31): Added FINALISE_TASK_WITH_TARGET_INFO function for atomic task completion with target info' || CHR(13)||CHR(10) ||
|
||||||
|
'1.6.0 (2025-10-31): Converted STORE_TASK_SOURCE_INFO and STORE_TASK_TARGET_INFO from procedures to functions returning sequence keys' || CHR(13)||CHR(10) ||
|
||||||
|
'1.5.0 (2025-10-22): Added package versioning system using centralized ENV_MANAGER functions' || CHR(13)||CHR(10) ||
|
||||||
|
'1.2.0 (2025-10-05): Added workflow property management (SET/GET_WORKFLOW_PROPERTY)' || CHR(13)||CHR(10) ||
|
||||||
|
'1.0.0 (2025-09-10): Initial release with workflow and task lifecycle management';
|
||||||
|
|
||||||
|
-- Line break constant for consistent CRLF formatting across Windows environments
|
||||||
|
cgBL CONSTANT VARCHAR2(2) := CHR(13)||CHR(10);
|
||||||
|
|
||||||
|
FUNCTION INIT_WORKFLOW(pServiceName IN VARCHAR2, pWorkflowRunId IN VARCHAR2, pWorkflowName in VARCHAR2)
|
||||||
|
RETURN NUMBER;
|
||||||
|
|
||||||
|
PROCEDURE FINALISE_WORKFLOW(pWorkflowHistoryKey IN NUMBER, pWorkflowStatus IN VARCHAR2);
|
||||||
|
PROCEDURE FINALISE_WORKFLOW(pWorkflowHistoryKey IN NUMBER, pServiceName IN VARCHAR2, pWorkflowStatus IN VARCHAR2);
|
||||||
|
|
||||||
|
FUNCTION INIT_TASK(pTaskRunId IN VARCHAR2, pTaskName in VARCHAR2, pWorkflowHistoryKey IN NUMBER)
|
||||||
|
RETURN NUMBER;
|
||||||
|
|
||||||
|
PROCEDURE FINALISE_TASK(pTaskHistoryKey IN NUMBER, pTaskStatus IN VARCHAR2);
|
||||||
|
|
||||||
|
FUNCTION STORE_TASK_SOURCE_INFO(pTaskHistoryKey IN NUMBER, pSourceName IN VARCHAR2, pNumRows IN NUMBER)
|
||||||
|
RETURN NUMBER;
|
||||||
|
|
||||||
|
FUNCTION STORE_TASK_TARGET_INFO(pTaskHistoryKey IN NUMBER, pSourceName IN VARCHAR2,
|
||||||
|
pNumRowsApplied IN NUMBER, pNumRowsRejected IN NUMBER,
|
||||||
|
pLoadSuccessful IN VARCHAR2, pServiceName IN VARCHAR2)
|
||||||
|
RETURN NUMBER;
|
||||||
|
|
||||||
|
FUNCTION FINALISE_TASK_WITH_TARGET_INFO(pTaskHistoryKey IN NUMBER, pTaskStatus IN VARCHAR2,
|
||||||
|
pTargetName IN VARCHAR2, pNumRowsApplied IN NUMBER,
|
||||||
|
pNumRowsRejected IN NUMBER, pLoadSuccessful IN VARCHAR2,
|
||||||
|
pServiceName IN VARCHAR2)
|
||||||
|
RETURN NUMBER;
|
||||||
|
|
||||||
|
vpRunningStatus CT_MRDS.A_WORKFLOW_HISTORY.WORKFLOW_SUCCESSFUL%TYPE := 'R';
|
||||||
|
|
||||||
|
PROCEDURE SET_WORKFLOW_PROPERTY(
|
||||||
|
pWorkflowHistoryKey IN NUMBER
|
||||||
|
,pServiceName IN VARCHAR2
|
||||||
|
,pProperty IN VARCHAR2
|
||||||
|
,pValue IN VARCHAR2
|
||||||
|
);
|
||||||
|
|
||||||
|
FUNCTION GET_WORKFLOW_PROPERTY(
|
||||||
|
pWorkflowHistoryKey IN NUMBER
|
||||||
|
,pServiceName IN VARCHAR2
|
||||||
|
,pProperty IN VARCHAR2)
|
||||||
|
RETURN VARCHAR2;
|
||||||
|
|
||||||
|
FUNCTION GET_VERSION RETURN VARCHAR2;
|
||||||
|
FUNCTION GET_BUILD_INFO RETURN VARCHAR2;
|
||||||
|
FUNCTION GET_VERSION_HISTORY RETURN VARCHAR2;
|
||||||
|
|
||||||
|
END WORKFLOW_MANAGER;
|
||||||
|
/
|
||||||
|
|
||||||
|
prompt ========================================
|
||||||
|
prompt WORKFLOW_MANAGER Package Specification rolled back to v1.7.0
|
||||||
|
prompt ========================================
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
SET SERVEROUTPUT ON;
|
||||||
|
SET ECHO ON;
|
||||||
|
|
||||||
|
prompt ========================================
|
||||||
|
prompt Tracking WORKFLOW_MANAGER Package Version After Rollback
|
||||||
|
prompt MARS-1011: Record rollback to version 1.7.0
|
||||||
|
prompt ========================================
|
||||||
|
|
||||||
|
BEGIN
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Tracking WORKFLOW_MANAGER rollback version...');
|
||||||
|
|
||||||
|
-- Track the rolled-back version (1.7.0)
|
||||||
|
CT_MRDS.ENV_MANAGER.TRACK_PACKAGE_VERSION(
|
||||||
|
pPackageOwner => 'CT_MRDS',
|
||||||
|
pPackageName => 'WORKFLOW_MANAGER',
|
||||||
|
pPackageVersion => CT_MRDS.WORKFLOW_MANAGER.PACKAGE_VERSION,
|
||||||
|
pPackageBuildDate => CT_MRDS.WORKFLOW_MANAGER.PACKAGE_BUILD_DATE,
|
||||||
|
pPackageAuthor => CT_MRDS.WORKFLOW_MANAGER.PACKAGE_AUTHOR
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('WORKFLOW_MANAGER rollback version tracked successfully.');
|
||||||
|
|
||||||
|
-- Display rollback version info
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Rollback Version: ' || CT_MRDS.WORKFLOW_MANAGER.GET_VERSION());
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Build Info: ' || CT_MRDS.WORKFLOW_MANAGER.GET_BUILD_INFO());
|
||||||
|
|
||||||
|
EXCEPTION
|
||||||
|
WHEN OTHERS THEN
|
||||||
|
DBMS_OUTPUT.PUT_LINE('ERROR tracking WORKFLOW_MANAGER rollback: ' || SQLERRM);
|
||||||
|
RAISE;
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
prompt ========================================
|
||||||
|
prompt WORKFLOW_MANAGER Package rollback version tracking completed
|
||||||
|
prompt Rolled back to version 1.7.0 (2025-10-31)
|
||||||
|
prompt ========================================
|
||||||
54
MARS_Packages/REL01/MARS-1011/README.md
Normal file
54
MARS_Packages/REL01/MARS-1011/README.md
Normal file
@@ -0,0 +1,54 @@
|
|||||||
|
# MARS-1011: WORKFLOW_MANAGER Package Update v1.7.1
|
||||||
|
|
||||||
|
## Description
|
||||||
|
Updates WORKFLOW_MANAGER package to version 1.7.1, adding automatic SERVICE_NAME='ODS' assignment in INIT_TASK function.
|
||||||
|
|
||||||
|
## Changes
|
||||||
|
- Modified `INIT_TASK` function to automatically set `SERVICE_NAME='ODS'` when inserting into `CT_MRDS.A_TASK_HISTORY`
|
||||||
|
- Updated package version from 1.7.0 to 1.7.1
|
||||||
|
- Updated build date to 2025-11-18
|
||||||
|
|
||||||
|
## Installation Files
|
||||||
|
1. `01_MARS_1011_install_CT_MRDS_WORKFLOW_MANAGER_SPEC.sql` - Package specification
|
||||||
|
2. `02_MARS_1011_install_CT_MRDS_WORKFLOW_MANAGER_BODY.sql` - Package body with SERVICE_NAME modification
|
||||||
|
3. `03_MARS_1011_verify_CT_MRDS_WORKFLOW_MANAGER.sql` - Verification and version check
|
||||||
|
4. `04_MARS_1011_track_CT_MRDS_WORKFLOW_MANAGER_version.sql` - Package version tracking in A_PACKAGE_VERSION_TRACKING
|
||||||
|
|
||||||
|
## Rollback Files
|
||||||
|
1. `91_MARS_1011_rollback_CT_MRDS_WORKFLOW_MANAGER_SPEC.sql` - Rollback package specification to v1.7.0
|
||||||
|
2. `91_MARS_1011_rollback_CT_MRDS_WORKFLOW_MANAGER_BODY.sql` - Rollback package body (removes SERVICE_NAME from INIT_TASK)
|
||||||
|
3. `92_MARS_1011_track_rollback_version.sql` - Track rollback version in A_PACKAGE_VERSION_TRACKING
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
```bash
|
||||||
|
sqlplus CT_MRDS/password@database @install_mars1011.sql
|
||||||
|
```
|
||||||
|
|
||||||
|
## Rollback
|
||||||
|
```bash
|
||||||
|
sqlplus CT_MRDS/password@database @rollback_mars1011.sql
|
||||||
|
```
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
- Requires CT_MRDS schema
|
||||||
|
- Requires A_TASK_HISTORY table with SERVICE_NAME column
|
||||||
|
- Requires ENV_MANAGER package for version management functions
|
||||||
|
- Requires A_PACKAGE_VERSION_TRACKING table for version tracking
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
After installation, verify:
|
||||||
|
1. Package compiles successfully (check `03_MARS_1011_verify_CT_MRDS_WORKFLOW_MANAGER.sql` output)
|
||||||
|
2. Version shows as 1.7.1
|
||||||
|
3. INIT_TASK function properly inserts SERVICE_NAME='ODS'
|
||||||
|
4. Package version is tracked in A_PACKAGE_VERSION_TRACKING table
|
||||||
|
5. Check tracking: `SELECT * FROM CT_MRDS.A_PACKAGE_VERSION_TRACKING WHERE PACKAGE_NAME='WORKFLOW_MANAGER' ORDER BY TRACKING_DATE DESC;`
|
||||||
|
|
||||||
|
After rollback, verify:
|
||||||
|
1. Version shows as 1.7.0 (rolled back)
|
||||||
|
2. INIT_TASK function no longer includes SERVICE_NAME column
|
||||||
|
3. Rollback is tracked in A_PACKAGE_VERSION_TRACKING with CHANGE_STATUS='ROLLBACK'
|
||||||
|
|
||||||
|
## Impact
|
||||||
|
- Low risk change - only affects new task records
|
||||||
|
- Backward compatible - existing functionality unchanged
|
||||||
|
- No breaking changes to function signatures
|
||||||
216
MARS_Packages/REL01/MARS-1011/WORKFLOW_MANAGER_BODY.sql
Normal file
216
MARS_Packages/REL01/MARS-1011/WORKFLOW_MANAGER_BODY.sql
Normal file
@@ -0,0 +1,216 @@
|
|||||||
|
CREATE OR REPLACE PACKAGE BODY CT_MRDS.WORKFLOW_MANAGER
|
||||||
|
IS
|
||||||
|
|
||||||
|
FUNCTION INIT_WORKFLOW(pServiceName IN VARCHAR2, pWorkflowRunId IN VARCHAR2, pWorkflowName in VARCHAR2)
|
||||||
|
RETURN NUMBER
|
||||||
|
IS
|
||||||
|
vWorkflowHistoryKey NUMBER;
|
||||||
|
BEGIN
|
||||||
|
vWorkflowHistoryKey := A_WORKFLOW_HISTORY_KEY_SEQ.NEXTVAL;
|
||||||
|
INSERT INTO CT_MRDS.A_WORKFLOW_HISTORY (SERVICE_NAME, A_WORKFLOW_HISTORY_KEY, ORCHESTRATION_RUN_ID,
|
||||||
|
WORKFLOW_NAME, WORKFLOW_START, WORKFLOW_SUCCESSFUL)
|
||||||
|
VALUES (pServiceName, vWorkflowHistoryKey, pWorkflowRunId,
|
||||||
|
pWorkflowName, SYSTIMESTAMP, vpRunningStatus);
|
||||||
|
|
||||||
|
return vWorkflowHistoryKey;
|
||||||
|
|
||||||
|
END INIT_WORKFLOW;
|
||||||
|
|
||||||
|
|
||||||
|
--
|
||||||
|
-- Overload without service name for backward compatability, to be cleaned up later
|
||||||
|
--
|
||||||
|
PROCEDURE FINALISE_WORKFLOW(pWorkflowHistoryKey IN NUMBER, pWorkflowStatus IN VARCHAR2)
|
||||||
|
IS
|
||||||
|
BEGIN
|
||||||
|
FINALISE_WORKFLOW(pWorkflowHistoryKey, NULL, pWorkflowStatus);
|
||||||
|
END;
|
||||||
|
|
||||||
|
PROCEDURE FINALISE_WORKFLOW(pWorkflowHistoryKey IN NUMBER, pServiceName IN VARCHAR2, pWorkflowStatus IN VARCHAR2)
|
||||||
|
IS
|
||||||
|
BEGIN
|
||||||
|
UPDATE CT_MRDS.A_WORKFLOW_HISTORY SET WORKFLOW_SUCCESSFUL = pWorkflowStatus,
|
||||||
|
WORKFLOW_END = SYSTIMESTAMP
|
||||||
|
WHERE A_WORKFLOW_HISTORY_KEY = pWorkflowHistoryKey
|
||||||
|
AND SERVICE_NAME = NVL(pServiceName,SERVICE_NAME);
|
||||||
|
END FINALISE_WORKFLOW;
|
||||||
|
|
||||||
|
|
||||||
|
FUNCTION INIT_TASK(pTaskRunId IN VARCHAR2, pTaskName in VARCHAR2, pWorkflowHistoryKey IN NUMBER)
|
||||||
|
RETURN NUMBER
|
||||||
|
IS
|
||||||
|
vTaskHistoryKey NUMBER;
|
||||||
|
BEGIN
|
||||||
|
vTaskHistoryKey := A_TASK_HISTORY_KEY_SEQ.NEXTVAL;
|
||||||
|
INSERT INTO CT_MRDS.A_TASK_HISTORY (A_TASK_HISTORY_KEY, TASK_RUN_ID, A_WORKFLOW_HISTORY_KEY,
|
||||||
|
TASK_NAME, TASK_START, TASK_SUCCESSFUL, SERVICE_NAME)
|
||||||
|
VALUES (vTaskHistoryKey, pTaskRunId, pWorkflowHistoryKey,
|
||||||
|
pTaskName, SYSTIMESTAMP, vpRunningStatus, 'ODS');
|
||||||
|
|
||||||
|
return vTaskHistoryKey;
|
||||||
|
|
||||||
|
END INIT_TASK;
|
||||||
|
|
||||||
|
|
||||||
|
PROCEDURE FINALISE_TASK(pTaskHistoryKey IN NUMBER, pTaskStatus IN VARCHAR2)
|
||||||
|
IS
|
||||||
|
BEGIN
|
||||||
|
UPDATE CT_MRDS.A_TASK_HISTORY SET TASK_SUCCESSFUL = pTaskStatus,
|
||||||
|
TASK_END = SYSTIMESTAMP
|
||||||
|
WHERE A_TASK_HISTORY_KEY = pTaskHistoryKey;
|
||||||
|
END FINALISE_TASK;
|
||||||
|
|
||||||
|
-- Internal helper function without COMMIT for use within other functions
|
||||||
|
FUNCTION STORE_TASK_SOURCE_INFO_INTERNAL(pTaskHistoryKey IN NUMBER, pSourceName IN VARCHAR2, pNumRows IN NUMBER)
|
||||||
|
RETURN NUMBER
|
||||||
|
IS
|
||||||
|
vTaskHistorySourceKey NUMBER;
|
||||||
|
BEGIN
|
||||||
|
vTaskHistorySourceKey := A_TASK_HISTORY_SOURCE_KEY_SEQ.NEXTVAL;
|
||||||
|
INSERT INTO CT_MRDS.A_TASK_HISTORY_SOURCE (A_TASK_HISTORY_SOURCE_KEY, A_TASK_HISTORY_KEY,
|
||||||
|
SOURCE_NAME, ROW_COUNT)
|
||||||
|
VALUES (vTaskHistorySourceKey, pTaskHistoryKey,
|
||||||
|
pSourceName, pNumRows);
|
||||||
|
RETURN vTaskHistorySourceKey;
|
||||||
|
END;
|
||||||
|
|
||||||
|
FUNCTION STORE_TASK_SOURCE_INFO(pTaskHistoryKey IN NUMBER, pSourceName IN VARCHAR2, pNumRows IN NUMBER)
|
||||||
|
RETURN NUMBER
|
||||||
|
IS
|
||||||
|
vTaskHistorySourceKey NUMBER;
|
||||||
|
BEGIN
|
||||||
|
vTaskHistorySourceKey := STORE_TASK_SOURCE_INFO_INTERNAL(pTaskHistoryKey, pSourceName, pNumRows);
|
||||||
|
COMMIT;
|
||||||
|
RETURN vTaskHistorySourceKey;
|
||||||
|
END;
|
||||||
|
|
||||||
|
-- Internal helper function without COMMIT for use within other functions
|
||||||
|
FUNCTION STORE_TASK_TARGET_INFO_INTERNAL(pTaskHistoryKey IN NUMBER, pSourceName IN VARCHAR2,
|
||||||
|
pNumRowsApplied IN NUMBER, pNumRowsRejected IN NUMBER,
|
||||||
|
pLoadSuccessful IN VARCHAR2, pServiceName IN VARCHAR2)
|
||||||
|
RETURN NUMBER
|
||||||
|
IS
|
||||||
|
vTaskHistoryTargetKey NUMBER;
|
||||||
|
BEGIN
|
||||||
|
vTaskHistoryTargetKey := A_TASK_HISTORY_TARGET_KEY_SEQ.NEXTVAL;
|
||||||
|
INSERT INTO CT_MRDS.A_TASK_HISTORY_TARGET (A_TASK_HISTORY_TARGET_KEY, A_TASK_HISTORY_KEY,
|
||||||
|
TARGET_NAME, ROW_COUNT_APPLIED, ROW_COUNT_REJECTED,
|
||||||
|
LOAD_SUCCESSFUL, SERVICE_NAME)
|
||||||
|
VALUES (vTaskHistoryTargetKey, pTaskHistoryKey,
|
||||||
|
pSourceName, pNumRowsApplied, pNumRowsRejected,
|
||||||
|
pLoadSuccessful, pServiceName);
|
||||||
|
RETURN vTaskHistoryTargetKey;
|
||||||
|
END;
|
||||||
|
|
||||||
|
FUNCTION STORE_TASK_TARGET_INFO(pTaskHistoryKey IN NUMBER, pSourceName IN VARCHAR2,
|
||||||
|
pNumRowsApplied IN NUMBER, pNumRowsRejected IN NUMBER,
|
||||||
|
pLoadSuccessful IN VARCHAR2, pServiceName IN VARCHAR2)
|
||||||
|
RETURN NUMBER
|
||||||
|
IS
|
||||||
|
vTaskHistoryTargetKey NUMBER;
|
||||||
|
BEGIN
|
||||||
|
vTaskHistoryTargetKey := STORE_TASK_TARGET_INFO_INTERNAL(
|
||||||
|
pTaskHistoryKey, pSourceName, pNumRowsApplied,
|
||||||
|
pNumRowsRejected, pLoadSuccessful, pServiceName
|
||||||
|
);
|
||||||
|
COMMIT;
|
||||||
|
RETURN vTaskHistoryTargetKey;
|
||||||
|
END;
|
||||||
|
|
||||||
|
FUNCTION FINALISE_TASK_WITH_TARGET_INFO(pTaskHistoryKey IN NUMBER, pTaskStatus IN VARCHAR2,
|
||||||
|
pTargetName IN VARCHAR2, pNumRowsApplied IN NUMBER,
|
||||||
|
pNumRowsRejected IN NUMBER, pLoadSuccessful IN VARCHAR2,
|
||||||
|
pServiceName IN VARCHAR2)
|
||||||
|
RETURN NUMBER
|
||||||
|
IS
|
||||||
|
vTaskHistoryTargetKey NUMBER;
|
||||||
|
BEGIN
|
||||||
|
-- Store target information using internal function (without COMMIT)
|
||||||
|
vTaskHistoryTargetKey := STORE_TASK_TARGET_INFO_INTERNAL(
|
||||||
|
pTaskHistoryKey => pTaskHistoryKey,
|
||||||
|
pSourceName => pTargetName,
|
||||||
|
pNumRowsApplied => pNumRowsApplied,
|
||||||
|
pNumRowsRejected => pNumRowsRejected,
|
||||||
|
pLoadSuccessful => pLoadSuccessful,
|
||||||
|
pServiceName => pServiceName
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Finalize the task using existing procedure (no COMMIT in procedure)
|
||||||
|
FINALISE_TASK(pTaskHistoryKey, pTaskStatus);
|
||||||
|
|
||||||
|
-- Single COMMIT for the entire atomic operation
|
||||||
|
COMMIT;
|
||||||
|
|
||||||
|
RETURN vTaskHistoryTargetKey;
|
||||||
|
END;
|
||||||
|
|
||||||
|
PROCEDURE SET_WORKFLOW_PROPERTY(
|
||||||
|
pWorkflowHistoryKey IN NUMBER
|
||||||
|
,pServiceName IN VARCHAR2
|
||||||
|
,pProperty IN VARCHAR2
|
||||||
|
,pValue IN VARCHAR2
|
||||||
|
) IS
|
||||||
|
BEGIN
|
||||||
|
INSERT INTO CT_MRDS.A_WORKFLOW_HISTORY_PROPERTY (A_WORKFLOW_HISTORY_KEY, SERVICE_NAME, PROPERTY, VALUE)
|
||||||
|
VALUES (pWorkflowHistoryKey, pServiceName, pProperty, pValue);
|
||||||
|
END;
|
||||||
|
|
||||||
|
FUNCTION GET_WORKFLOW_PROPERTY(
|
||||||
|
pWorkflowHistoryKey IN NUMBER
|
||||||
|
,pServiceName IN VARCHAR2
|
||||||
|
,pProperty IN VARCHAR2
|
||||||
|
) RETURN VARCHAR2
|
||||||
|
IS
|
||||||
|
vValue CT_MRDS.A_WORKFLOW_HISTORY_PROPERTY.VALUE%TYPE;
|
||||||
|
BEGIN
|
||||||
|
SELECT VALUE
|
||||||
|
INTO vValue
|
||||||
|
FROM CT_MRDS.A_WORKFLOW_HISTORY_PROPERTY
|
||||||
|
WHERE A_WORKFLOW_HISTORY_KEY = pWorkflowHistoryKey
|
||||||
|
AND SERVICE_NAME = pServiceName
|
||||||
|
AND PROPERTY = pProperty;
|
||||||
|
|
||||||
|
RETURN vValue;
|
||||||
|
END;
|
||||||
|
|
||||||
|
----------------------------------------------------------------------------------------------------
|
||||||
|
-- PACKAGE VERSION MANAGEMENT FUNCTIONS IMPLEMENTATION
|
||||||
|
----------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
FUNCTION GET_VERSION
|
||||||
|
RETURN VARCHAR2
|
||||||
|
IS
|
||||||
|
BEGIN
|
||||||
|
RETURN PACKAGE_VERSION;
|
||||||
|
END GET_VERSION;
|
||||||
|
|
||||||
|
----------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
FUNCTION GET_BUILD_INFO
|
||||||
|
RETURN VARCHAR2
|
||||||
|
IS
|
||||||
|
BEGIN
|
||||||
|
RETURN ENV_MANAGER.GET_PACKAGE_VERSION_INFO(
|
||||||
|
pPackageName => 'WORKFLOW_MANAGER',
|
||||||
|
pVersion => PACKAGE_VERSION,
|
||||||
|
pBuildDate => PACKAGE_BUILD_DATE,
|
||||||
|
pAuthor => PACKAGE_AUTHOR
|
||||||
|
);
|
||||||
|
END GET_BUILD_INFO;
|
||||||
|
|
||||||
|
----------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
FUNCTION GET_VERSION_HISTORY
|
||||||
|
RETURN VARCHAR2
|
||||||
|
IS
|
||||||
|
BEGIN
|
||||||
|
RETURN ENV_MANAGER.FORMAT_VERSION_HISTORY(
|
||||||
|
pPackageName => 'WORKFLOW_MANAGER',
|
||||||
|
pVersionHistory => VERSION_HISTORY
|
||||||
|
);
|
||||||
|
END GET_VERSION_HISTORY;
|
||||||
|
|
||||||
|
----------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
END WORKFLOW_MANAGER;
|
||||||
|
/
|
||||||
100
MARS_Packages/REL01/MARS-1011/WORKFLOW_MANAGER_SPEC.sql
Normal file
100
MARS_Packages/REL01/MARS-1011/WORKFLOW_MANAGER_SPEC.sql
Normal file
@@ -0,0 +1,100 @@
|
|||||||
|
CREATE OR REPLACE PACKAGE CT_MRDS.WORKFLOW_MANAGER
|
||||||
|
IS
|
||||||
|
|
||||||
|
-- Package Version Information (Semantic Versioning: MAJOR.MINOR.PATCH)
|
||||||
|
PACKAGE_VERSION CONSTANT VARCHAR2(10) := '1.7.1';
|
||||||
|
PACKAGE_BUILD_DATE CONSTANT VARCHAR2(20) := '2025-11-18 12:00:00';
|
||||||
|
PACKAGE_AUTHOR CONSTANT VARCHAR2(100) := 'Grzegorz Michalski';
|
||||||
|
|
||||||
|
-- Version History (Latest changes first)
|
||||||
|
VERSION_HISTORY CONSTANT VARCHAR2(4000) :=
|
||||||
|
'1.7.1 (2025-11-18): Added SERVICE_NAME column with hardcoded ''ODS'' value to INIT_TASK function' || CHR(13)||CHR(10) ||
|
||||||
|
'1.7.0 (2025-10-31): Added FINALISE_TASK_WITH_TARGET_INFO function for atomic task completion with target info' || CHR(13)||CHR(10) ||
|
||||||
|
'1.6.0 (2025-10-31): Converted STORE_TASK_SOURCE_INFO and STORE_TASK_TARGET_INFO from procedures to functions returning sequence keys' || CHR(13)||CHR(10) ||
|
||||||
|
'1.5.0 (2025-10-22): Added package versioning system using centralized ENV_MANAGER functions' || CHR(13)||CHR(10) ||
|
||||||
|
'1.2.0 (2025-10-05): Added workflow property management (SET/GET_WORKFLOW_PROPERTY)' || CHR(13)||CHR(10) ||
|
||||||
|
'1.0.0 (2025-09-10): Initial release with workflow and task lifecycle management';
|
||||||
|
|
||||||
|
-- Line break constant for consistent CRLF formatting across Windows environments
|
||||||
|
cgBL CONSTANT VARCHAR2(2) := CHR(13)||CHR(10);
|
||||||
|
|
||||||
|
FUNCTION INIT_WORKFLOW(pServiceName IN VARCHAR2, pWorkflowRunId IN VARCHAR2, pWorkflowName in VARCHAR2)
|
||||||
|
RETURN NUMBER;
|
||||||
|
|
||||||
|
PROCEDURE FINALISE_WORKFLOW(pWorkflowHistoryKey IN NUMBER, pWorkflowStatus IN VARCHAR2);
|
||||||
|
PROCEDURE FINALISE_WORKFLOW(pWorkflowHistoryKey IN NUMBER, pServiceName IN VARCHAR2, pWorkflowStatus IN VARCHAR2);
|
||||||
|
|
||||||
|
FUNCTION INIT_TASK(pTaskRunId IN VARCHAR2, pTaskName in VARCHAR2, pWorkflowHistoryKey IN NUMBER)
|
||||||
|
RETURN NUMBER;
|
||||||
|
|
||||||
|
PROCEDURE FINALISE_TASK(pTaskHistoryKey IN NUMBER, pTaskStatus IN VARCHAR2);
|
||||||
|
|
||||||
|
FUNCTION STORE_TASK_SOURCE_INFO(pTaskHistoryKey IN NUMBER, pSourceName IN VARCHAR2, pNumRows IN NUMBER)
|
||||||
|
RETURN NUMBER;
|
||||||
|
|
||||||
|
FUNCTION STORE_TASK_TARGET_INFO(pTaskHistoryKey IN NUMBER, pSourceName IN VARCHAR2,
|
||||||
|
pNumRowsApplied IN NUMBER, pNumRowsRejected IN NUMBER,
|
||||||
|
pLoadSuccessful IN VARCHAR2, pServiceName IN VARCHAR2)
|
||||||
|
RETURN NUMBER;
|
||||||
|
|
||||||
|
FUNCTION FINALISE_TASK_WITH_TARGET_INFO(pTaskHistoryKey IN NUMBER, pTaskStatus IN VARCHAR2,
|
||||||
|
pTargetName IN VARCHAR2, pNumRowsApplied IN NUMBER,
|
||||||
|
pNumRowsRejected IN NUMBER, pLoadSuccessful IN VARCHAR2,
|
||||||
|
pServiceName IN VARCHAR2)
|
||||||
|
RETURN NUMBER;
|
||||||
|
|
||||||
|
vpRunningStatus CT_MRDS.A_WORKFLOW_HISTORY.WORKFLOW_SUCCESSFUL%TYPE := 'R';
|
||||||
|
|
||||||
|
--
|
||||||
|
-- Set and get information on workflow level
|
||||||
|
--
|
||||||
|
PROCEDURE SET_WORKFLOW_PROPERTY(
|
||||||
|
pWorkflowHistoryKey IN NUMBER
|
||||||
|
,pServiceName IN VARCHAR2
|
||||||
|
,pProperty IN VARCHAR2
|
||||||
|
,pValue IN VARCHAR2
|
||||||
|
);
|
||||||
|
|
||||||
|
FUNCTION GET_WORKFLOW_PROPERTY(
|
||||||
|
pWorkflowHistoryKey IN NUMBER
|
||||||
|
,pServiceName IN VARCHAR2
|
||||||
|
,pProperty IN VARCHAR2)
|
||||||
|
RETURN VARCHAR2;
|
||||||
|
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
-- PACKAGE VERSION MANAGEMENT FUNCTIONS
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_VERSION
|
||||||
|
* @desc Returns the current version number of the WORKFLOW_MANAGER package.
|
||||||
|
* Uses semantic versioning format (MAJOR.MINOR.PATCH).
|
||||||
|
* @example SELECT WORKFLOW_MANAGER.GET_VERSION() FROM DUAL;
|
||||||
|
* @ex_rslt 1.5.0
|
||||||
|
**/
|
||||||
|
FUNCTION GET_VERSION RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_BUILD_INFO
|
||||||
|
* @desc Returns comprehensive build information including version, build date, and author.
|
||||||
|
* Uses centralized ENV_MANAGER.GET_PACKAGE_VERSION_INFO function.
|
||||||
|
* @example SELECT WORKFLOW_MANAGER.GET_BUILD_INFO() FROM DUAL;
|
||||||
|
* @ex_rslt Package: WORKFLOW_MANAGER
|
||||||
|
* Version: 1.5.0
|
||||||
|
* Build Date: 2025-10-22 17:00:00
|
||||||
|
* Author: Grzegorz Michalski
|
||||||
|
**/
|
||||||
|
FUNCTION GET_BUILD_INFO RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_VERSION_HISTORY
|
||||||
|
* @desc Returns complete version history with all releases and changes.
|
||||||
|
* Uses centralized ENV_MANAGER.FORMAT_VERSION_HISTORY function.
|
||||||
|
* @example SELECT WORKFLOW_MANAGER.GET_VERSION_HISTORY() FROM DUAL;
|
||||||
|
* @ex_rslt WORKFLOW_MANAGER Version History:
|
||||||
|
* 1.5.0 (2025-10-22): Added package versioning system...
|
||||||
|
**/
|
||||||
|
FUNCTION GET_VERSION_HISTORY RETURN VARCHAR2;
|
||||||
|
|
||||||
|
END WORKFLOW_MANAGER;
|
||||||
|
/
|
||||||
40
MARS_Packages/REL01/MARS-1011/install_mars1011.sql
Normal file
40
MARS_Packages/REL01/MARS-1011/install_mars1011.sql
Normal file
@@ -0,0 +1,40 @@
|
|||||||
|
WHENEVER SQLERROR EXIT FAILURE
|
||||||
|
SET SERVEROUTPUT ON
|
||||||
|
SET TIMING ON
|
||||||
|
SET ECHO ON
|
||||||
|
SET HEADING OFF
|
||||||
|
SET FEEDBACK ON
|
||||||
|
SET VERIFY OFF
|
||||||
|
|
||||||
|
-- Simple filename without PDB reference
|
||||||
|
var filename VARCHAR2(100)
|
||||||
|
BEGIN
|
||||||
|
SELECT 'INSTALL_MARS_1011_' || TO_CHAR(SYSDATE,'YYYYMMDD_HH24MISS')||'.log' INTO :filename from DUAL;
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
column filename new_value _filename
|
||||||
|
select :filename filename from dual;
|
||||||
|
spool &_filename
|
||||||
|
|
||||||
|
prompt ##### started at time #####
|
||||||
|
select systimestamp from dual;
|
||||||
|
prompt ##### database name #####
|
||||||
|
SELECT SYS_CONTEXT('USERENV', 'DB_NAME') AS DATABASE_NAME FROM DUAL;
|
||||||
|
|
||||||
|
prompt ========================================
|
||||||
|
prompt MARS-1011: Installing WORKFLOW_MANAGER v1.7.1
|
||||||
|
prompt Description: Add SERVICE_NAME='ODS' to INIT_TASK function
|
||||||
|
prompt ========================================
|
||||||
|
|
||||||
|
@@01_MARS_1011_install_CT_MRDS_WORKFLOW_MANAGER_SPEC.sql
|
||||||
|
@@02_MARS_1011_install_CT_MRDS_WORKFLOW_MANAGER_BODY.sql
|
||||||
|
@@03_MARS_1011_verify_CT_MRDS_WORKFLOW_MANAGER.sql
|
||||||
|
@@04_MARS_1011_track_CT_MRDS_WORKFLOW_MANAGER_version.sql
|
||||||
|
|
||||||
|
SET ECHO OFF
|
||||||
|
|
||||||
|
prompt ##### completed at time #####
|
||||||
|
select systimestamp from dual;
|
||||||
|
|
||||||
|
SPOOL OFF
|
||||||
|
EXIT
|
||||||
39
MARS_Packages/REL01/MARS-1011/rollback_mars1011.sql
Normal file
39
MARS_Packages/REL01/MARS-1011/rollback_mars1011.sql
Normal file
@@ -0,0 +1,39 @@
|
|||||||
|
WHENEVER SQLERROR EXIT FAILURE
|
||||||
|
SET SERVEROUTPUT ON
|
||||||
|
SET TIMING ON
|
||||||
|
SET ECHO ON
|
||||||
|
SET HEADING OFF
|
||||||
|
SET FEEDBACK ON
|
||||||
|
SET VERIFY OFF
|
||||||
|
|
||||||
|
-- Simple filename without PDB reference
|
||||||
|
var filename VARCHAR2(100)
|
||||||
|
BEGIN
|
||||||
|
SELECT 'ROLLBACK_MARS_1011_' || TO_CHAR(SYSDATE,'YYYYMMDD_HH24MISS')||'.log' INTO :filename from DUAL;
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
column filename new_value _filename
|
||||||
|
select :filename filename from dual;
|
||||||
|
spool &_filename
|
||||||
|
|
||||||
|
prompt ##### started at time #####
|
||||||
|
select systimestamp from dual;
|
||||||
|
prompt ##### database name #####
|
||||||
|
SELECT SYS_CONTEXT('USERENV', 'DB_NAME') AS DATABASE_NAME FROM DUAL;
|
||||||
|
|
||||||
|
prompt ========================================
|
||||||
|
prompt MARS-1011: Rolling back WORKFLOW_MANAGER changes
|
||||||
|
prompt Description: Rollback to previous version 1.7.0
|
||||||
|
prompt ========================================
|
||||||
|
|
||||||
|
@@91_MARS_1011_rollback_CT_MRDS_WORKFLOW_MANAGER_SPEC.sql
|
||||||
|
@@91_MARS_1011_rollback_CT_MRDS_WORKFLOW_MANAGER_BODY.sql
|
||||||
|
@@92_MARS_1011_track_rollback_version.sql
|
||||||
|
|
||||||
|
SET ECHO OFF
|
||||||
|
|
||||||
|
prompt ##### completed at time #####
|
||||||
|
select systimestamp from dual;
|
||||||
|
|
||||||
|
SPOOL OFF
|
||||||
|
EXIT
|
||||||
@@ -0,0 +1,85 @@
|
|||||||
|
--=============================================================================================================================
|
||||||
|
-- MARS-1049-ADHOC: Update CSDB Encoding Configuration
|
||||||
|
--=============================================================================================================================
|
||||||
|
-- Purpose: Change encoding for CSDB configurations from UTF8 to WE8MSWIN1252
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-26
|
||||||
|
-- Related: MARS-1049 (CSV Encoding Support), MARS-821 (Original CSDB Setup)
|
||||||
|
--
|
||||||
|
-- Why WE8MSWIN1252 for CSDB:
|
||||||
|
-- - CSDB files contain international characters (Turkish: Türkiye, etc.)
|
||||||
|
-- - Source system uses Windows-1252 encoding
|
||||||
|
-- - Proper character handling requires matching encoding
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
SET SERVEROUTPUT ON
|
||||||
|
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT MARS-1049-ADHOC: Update CSDB Encoding Configuration
|
||||||
|
PROMPT =========================================================================
|
||||||
|
|
||||||
|
-- Step 1: Show current state
|
||||||
|
PROMPT
|
||||||
|
PROMPT Step 1: Current CSDB encoding configuration
|
||||||
|
PROMPT ========================================
|
||||||
|
|
||||||
|
SELECT
|
||||||
|
TABLE_ID,
|
||||||
|
SOURCE_FILE_ID,
|
||||||
|
ENCODING AS CURRENT_ENCODING,
|
||||||
|
TEMPLATE_TABLE_NAME
|
||||||
|
FROM CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
WHERE TEMPLATE_TABLE_NAME LIKE '%CSDB%'
|
||||||
|
ORDER BY TABLE_ID, SOURCE_FILE_ID;
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT Step 2: Update CSDB configurations to WE8MSWIN1252
|
||||||
|
PROMPT ========================================
|
||||||
|
|
||||||
|
-- Update all CSDB configurations to Windows-1252 encoding
|
||||||
|
UPDATE /*+ NOPARALLEL */ CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
SET ENCODING = 'WE8MSWIN1252'
|
||||||
|
WHERE TEMPLATE_TABLE_NAME LIKE '%CSDB%';
|
||||||
|
|
||||||
|
PROMPT Updated CSDB configurations with WE8MSWIN1252 encoding
|
||||||
|
|
||||||
|
-- Commit changes
|
||||||
|
COMMIT;
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT Step 3: Verify updated encoding configuration
|
||||||
|
PROMPT ========================================
|
||||||
|
|
||||||
|
SELECT
|
||||||
|
TABLE_ID,
|
||||||
|
SOURCE_FILE_ID,
|
||||||
|
ENCODING AS NEW_ENCODING,
|
||||||
|
TEMPLATE_TABLE_NAME
|
||||||
|
FROM CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
WHERE TEMPLATE_TABLE_NAME LIKE '%CSDB%'
|
||||||
|
ORDER BY TABLE_ID, SOURCE_FILE_ID;
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT Step 4: Overall encoding distribution
|
||||||
|
PROMPT ========================================
|
||||||
|
|
||||||
|
SELECT
|
||||||
|
ENCODING,
|
||||||
|
COUNT(*) AS CONFIG_COUNT,
|
||||||
|
LISTAGG(DISTINCT TABLE_ID, ', ') WITHIN GROUP (ORDER BY TABLE_ID) AS TABLES
|
||||||
|
FROM CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
GROUP BY ENCODING
|
||||||
|
ORDER BY ENCODING;
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT CSDB Encoding Update - COMPLETED
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT Summary:
|
||||||
|
PROMPT - All CSDB configurations updated to WE8MSWIN1252
|
||||||
|
PROMPT - Encoding matches source system character set (Windows-1252)
|
||||||
|
PROMPT - Proper handling of international characters enabled
|
||||||
|
PROMPT
|
||||||
|
PROMPT Next Step: Run 01_MARS_1049_ADHOC_drop_external_tables.sql
|
||||||
|
PROMPT =========================================================================
|
||||||
@@ -0,0 +1,94 @@
|
|||||||
|
--=============================================================================================================================
|
||||||
|
-- MARS-1049-ADHOC: Drop External Tables for CSDB Encoding Change
|
||||||
|
--=============================================================================================================================
|
||||||
|
-- Purpose: Drop existing CSDB external tables before recreation with proper encoding parameter
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-26
|
||||||
|
-- Related: MARS-1049 (CSV Encoding Support)
|
||||||
|
--
|
||||||
|
-- Tables to be dropped (16 total - existing and non-existing):
|
||||||
|
-- - CSDB_DEBT: ARCHIVE, INBOX, ODS
|
||||||
|
-- - CSDB_DEBT_DAILY: ARCHIVE, INBOX, ODS
|
||||||
|
-- - CSDB_INSTR_DESC_FULL: ARCHIVE, INBOX, ODS
|
||||||
|
-- - CSDB_INSTR_RAT_FULL: ARCHIVE, ODS
|
||||||
|
-- - CSDB_ISSUER_DESC_FULL: ARCHIVE, ODS
|
||||||
|
-- - CSDB_ISSUER_RAT_FULL: ARCHIVE, INBOX, ODS
|
||||||
|
--
|
||||||
|
-- Note: Script handles both existing and non-existing tables gracefully
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
SET SERVEROUTPUT ON
|
||||||
|
|
||||||
|
DECLARE
|
||||||
|
TYPE t_string_array IS TABLE OF VARCHAR2(100);
|
||||||
|
|
||||||
|
-- Define list of tables to drop (16 total)
|
||||||
|
vTableList t_string_array := t_string_array(
|
||||||
|
'CSDB_DEBT_ARCHIVE',
|
||||||
|
'CSDB_DEBT_INBOX',
|
||||||
|
'CSDB_DEBT_ODS',
|
||||||
|
|
||||||
|
'CSDB_DEBT_DAILY_ARCHIVE',
|
||||||
|
'CSDB_DEBT_DAILY_INBOX',
|
||||||
|
'CSDB_DEBT_DAILY_ODS',
|
||||||
|
|
||||||
|
'CSDB_INSTR_DESC_FULL_ARCHIVE',
|
||||||
|
'CSDB_INSTR_DESC_FULL_INBOX',
|
||||||
|
'CSDB_INSTR_DESC_FULL_ODS',
|
||||||
|
|
||||||
|
'CSDB_INSTR_RAT_FULL_ARCHIVE',
|
||||||
|
'CSDB_INSTR_RAT_FULL_INBOX',
|
||||||
|
'CSDB_INSTR_RAT_FULL_ODS',
|
||||||
|
|
||||||
|
'CSDB_ISSUER_DESC_FULL_ARCHIVE',
|
||||||
|
'CSDB_ISSUER_DESC_FULL_INBOX',
|
||||||
|
'CSDB_ISSUER_DESC_FULL_ODS',
|
||||||
|
|
||||||
|
'CSDB_ISSUER_RAT_FULL_ARCHIVE',
|
||||||
|
'CSDB_ISSUER_RAT_FULL_INBOX',
|
||||||
|
'CSDB_ISSUER_RAT_FULL_ODS'
|
||||||
|
);
|
||||||
|
|
||||||
|
vDroppedCount NUMBER := 0;
|
||||||
|
vSkippedCount NUMBER := 0;
|
||||||
|
vErrorCount NUMBER := 0;
|
||||||
|
BEGIN
|
||||||
|
DBMS_OUTPUT.PUT_LINE('=========================================================================');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('MARS-1049-ADHOC: Drop CSDB External Tables');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('=========================================================================');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Tables to process: ' || vTableList.COUNT);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('');
|
||||||
|
|
||||||
|
-- Loop through table list
|
||||||
|
FOR i IN 1..vTableList.COUNT LOOP
|
||||||
|
BEGIN
|
||||||
|
EXECUTE IMMEDIATE 'DROP TABLE ODS.' || vTableList(i);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Dropped ODS.' || vTableList(i));
|
||||||
|
vDroppedCount := vDroppedCount + 1;
|
||||||
|
EXCEPTION
|
||||||
|
WHEN OTHERS THEN
|
||||||
|
IF SQLCODE = -942 THEN
|
||||||
|
DBMS_OUTPUT.PUT_LINE('INFO: Table ODS.' || vTableList(i) || ' does not exist - skipping');
|
||||||
|
vSkippedCount := vSkippedCount + 1;
|
||||||
|
ELSE
|
||||||
|
DBMS_OUTPUT.PUT_LINE('ERROR: Failed to drop ODS.' || vTableList(i) || ' - ' || SQLERRM);
|
||||||
|
vErrorCount := vErrorCount + 1;
|
||||||
|
RAISE;
|
||||||
|
END IF;
|
||||||
|
END;
|
||||||
|
END LOOP;
|
||||||
|
|
||||||
|
DBMS_OUTPUT.PUT_LINE('');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('=========================================================================');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Drop External Tables - COMPLETED');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('=========================================================================');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Summary:');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('- Tables processed: ' || vTableList.COUNT);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('- Tables dropped: ' || vDroppedCount);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('- Tables skipped: ' || vSkippedCount);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('- Errors: ' || vErrorCount);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Next Step: Run 02_MARS_1049_ADHOC_recreate_external_tables.sql');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('=========================================================================');
|
||||||
|
END;
|
||||||
|
/
|
||||||
@@ -0,0 +1,297 @@
|
|||||||
|
--=============================================================================================================================
|
||||||
|
-- MARS-1049-ADHOC: Recreate External Tables for CSDB with Encoding Support
|
||||||
|
--=============================================================================================================================
|
||||||
|
-- Purpose: Recreate CSDB external tables with proper pEncoding parameter (WE8MSWIN1252)
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-26
|
||||||
|
-- Related: MARS-1049 (CSV Encoding Support), MARS-821 (Original CSDB Setup)
|
||||||
|
--
|
||||||
|
-- Tables to be created (5 existing + 11 missing = 16 total):
|
||||||
|
-- CSDB_DEBT: INBOX, ODS, ARCHIVE
|
||||||
|
-- CSDB_DEBT_DAILY: INBOX, ODS, ARCHIVE
|
||||||
|
-- CSDB_INSTR_DESC_FULL: INBOX, ODS, ARCHIVE
|
||||||
|
-- CSDB_INSTR_RAT_FULL: INBOX, ODS, ARCHIVE
|
||||||
|
-- CSDB_ISSUER_DESC_FULL: INBOX, ODS, ARCHIVE
|
||||||
|
-- CSDB_ISSUER_RAT_FULL: INBOX, ODS, ARCHIVE
|
||||||
|
--
|
||||||
|
-- Prerequisites:
|
||||||
|
-- - Template tables must exist in CT_ET_TEMPLATES schema
|
||||||
|
-- - FILE_MANAGER_ODS v2.1.0+ with pEncoding parameter support
|
||||||
|
-- - Proper bucket URIs configured in ENV_MANAGER
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
SET SERVEROUTPUT ON
|
||||||
|
|
||||||
|
PROMPT ========================================================
|
||||||
|
PROMPT Step 1: Create CSDB_DEBT External Tables with Encoding
|
||||||
|
PROMPT ========================================================
|
||||||
|
|
||||||
|
-- INBOX - For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_DEBT_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_DEBT',
|
||||||
|
pPrefix => 'INBOX/CSDB/CentralizedSecuritiesDissemination/CSDB_DEBT',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pEncoding => 'WE8MSWIN1252'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_DEBT_INBOX with encoding WE8MSWIN1252');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ODS - For processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_DEBT_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_DEBT',
|
||||||
|
pPrefix => 'ODS/CSDB/CSDB_DEBT',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri,
|
||||||
|
pEncoding => 'WE8MSWIN1252'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_DEBT_ODS with encoding WE8MSWIN1252');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_DEBT_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_DEBT',
|
||||||
|
pPrefix => 'ARCHIVE/CSDB/CSDB_DEBT',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri,
|
||||||
|
pEncoding => 'WE8MSWIN1252'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_DEBT_ARCHIVE with encoding WE8MSWIN1252');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
PROMPT ================================================================
|
||||||
|
PROMPT Step 2: Create CSDB_DEBT_DAILY External Tables with Encoding
|
||||||
|
PROMPT ================================================================
|
||||||
|
|
||||||
|
-- INBOX - For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_DEBT_DAILY_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_DEBT_DAILY',
|
||||||
|
pPrefix => 'INBOX/CSDB/CentralizedSecuritiesDailyReferenceDataDissemination/CSDB_DEBT_DAILY',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pEncoding => 'WE8MSWIN1252'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_DEBT_DAILY_INBOX with encoding WE8MSWIN1252');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ODS - For processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_DEBT_DAILY_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_DEBT_DAILY',
|
||||||
|
pPrefix => 'ODS/CSDB/CSDB_DEBT_DAILY',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri,
|
||||||
|
pEncoding => 'WE8MSWIN1252'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_DEBT_DAILY_ODS with encoding WE8MSWIN1252');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_DEBT_DAILY_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_DEBT_DAILY',
|
||||||
|
pPrefix => 'ARCHIVE/CSDB/CSDB_DEBT_DAILY',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri,
|
||||||
|
pEncoding => 'WE8MSWIN1252'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_DEBT_DAILY_ARCHIVE with encoding WE8MSWIN1252');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
PROMPT ========================================================================
|
||||||
|
PROMPT Step 3: Create CSDB_INSTR_DESC_FULL External Tables with Encoding
|
||||||
|
PROMPT ========================================================================
|
||||||
|
|
||||||
|
-- INBOX - For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_INSTR_DESC_FULL_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_INSTR_DESC_FULL',
|
||||||
|
pPrefix => 'INBOX/CSDB/FullRatingsDissemination/CSDB_INSTR_DESC_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pEncoding => 'WE8MSWIN1252'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_INSTR_DESC_FULL_INBOX with encoding WE8MSWIN1252');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ODS - For processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_INSTR_DESC_FULL_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_INSTR_DESC_FULL',
|
||||||
|
pPrefix => 'ODS/CSDB/CSDB_INSTR_DESC_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri,
|
||||||
|
pEncoding => 'WE8MSWIN1252'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_INSTR_DESC_FULL_ODS with encoding WE8MSWIN1252');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_INSTR_DESC_FULL_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_INSTR_DESC_FULL',
|
||||||
|
pPrefix => 'ARCHIVE/CSDB/CSDB_INSTR_DESC_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri,
|
||||||
|
pEncoding => 'WE8MSWIN1252'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_INSTR_DESC_FULL_ARCHIVE with encoding WE8MSWIN1252');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
PROMPT ========================================================================
|
||||||
|
PROMPT Step 4: Create CSDB_INSTR_RAT_FULL External Tables with Encoding
|
||||||
|
PROMPT ========================================================================
|
||||||
|
|
||||||
|
-- INBOX - For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_INSTR_RAT_FULL_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_INSTR_RAT_FULL',
|
||||||
|
pPrefix => 'INBOX/CSDB/FullRatingsDissemination/CSDB_INSTR_RAT_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pEncoding => 'WE8MSWIN1252'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_INSTR_RAT_FULL_INBOX with encoding WE8MSWIN1252');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ODS - For processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_INSTR_RAT_FULL_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_INSTR_RAT_FULL',
|
||||||
|
pPrefix => 'ODS/CSDB/CSDB_INSTR_RAT_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri,
|
||||||
|
pEncoding => 'WE8MSWIN1252'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_INSTR_RAT_FULL_ODS with encoding WE8MSWIN1252');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_INSTR_RAT_FULL_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_INSTR_RAT_FULL',
|
||||||
|
pPrefix => 'ARCHIVE/CSDB/CSDB_INSTR_RAT_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri,
|
||||||
|
pEncoding => 'WE8MSWIN1252'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_INSTR_RAT_FULL_ARCHIVE with encoding WE8MSWIN1252');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Step 5: Create CSDB_ISSUER_DESC_FULL External Tables with Encoding
|
||||||
|
PROMPT =========================================================================
|
||||||
|
|
||||||
|
-- INBOX - For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_ISSUER_DESC_FULL_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_ISSUER_DESC_FULL',
|
||||||
|
pPrefix => 'INBOX/CSDB/FullRatingsDissemination/CSDB_ISSUER_DESC_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pEncoding => 'WE8MSWIN1252'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_ISSUER_DESC_FULL_INBOX with encoding WE8MSWIN1252');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ODS - For processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_ISSUER_DESC_FULL_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_ISSUER_DESC_FULL',
|
||||||
|
pPrefix => 'ODS/CSDB/CSDB_ISSUER_DESC_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri,
|
||||||
|
pEncoding => 'WE8MSWIN1252'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_ISSUER_DESC_FULL_ODS with encoding WE8MSWIN1252');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_ISSUER_DESC_FULL_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_ISSUER_DESC_FULL',
|
||||||
|
pPrefix => 'ARCHIVE/CSDB/CSDB_ISSUER_DESC_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri,
|
||||||
|
pEncoding => 'WE8MSWIN1252'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_ISSUER_DESC_FULL_ARCHIVE with encoding WE8MSWIN1252');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Step 6: Create CSDB_ISSUER_RAT_FULL External Tables with Encoding
|
||||||
|
PROMPT =========================================================================
|
||||||
|
|
||||||
|
-- INBOX - For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_ISSUER_RAT_FULL_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_ISSUER_RAT_FULL',
|
||||||
|
pPrefix => 'INBOX/CSDB/FullRatingsDissemination/CSDB_ISSUER_RAT_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pEncoding => 'WE8MSWIN1252'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_ISSUER_RAT_FULL_INBOX with encoding WE8MSWIN1252');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ODS - For processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_ISSUER_RAT_FULL_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_ISSUER_RAT_FULL',
|
||||||
|
pPrefix => 'ODS/CSDB/CSDB_ISSUER_RAT_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri,
|
||||||
|
pEncoding => 'WE8MSWIN1252'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_ISSUER_RAT_FULL_ODS with encoding WE8MSWIN1252');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_ISSUER_RAT_FULL_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_ISSUER_RAT_FULL',
|
||||||
|
pPrefix => 'ARCHIVE/CSDB/CSDB_ISSUER_RAT_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri,
|
||||||
|
pEncoding => 'WE8MSWIN1252'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_ISSUER_RAT_FULL_ARCHIVE with encoding WE8MSWIN1252');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Recreate External Tables with Encoding - COMPLETED
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT Summary:
|
||||||
|
PROMPT - CSDB_DEBT tables: 3 created (INBOX, ODS, ARCHIVE)
|
||||||
|
PROMPT - CSDB_DEBT_DAILY tables: 3 created (INBOX, ODS, ARCHIVE)
|
||||||
|
PROMPT - CSDB_INSTR_DESC_FULL tables: 3 created (INBOX, ODS, ARCHIVE)
|
||||||
|
PROMPT - CSDB_INSTR_RAT_FULL tables: 3 created (INBOX, ODS, ARCHIVE)
|
||||||
|
PROMPT - CSDB_ISSUER_DESC_FULL tables: 3 created (INBOX, ODS, ARCHIVE)
|
||||||
|
PROMPT - CSDB_ISSUER_RAT_FULL tables: 3 created (INBOX, ODS, ARCHIVE)
|
||||||
|
PROMPT - Total: 18 external tables created with WE8MSWIN1252 encoding
|
||||||
|
PROMPT
|
||||||
|
PROMPT Next Step: Run 03_MARS_1049_ADHOC_verify_external_tables.sql
|
||||||
|
PROMPT =========================================================================
|
||||||
@@ -0,0 +1,184 @@
|
|||||||
|
--=============================================================================================================================
|
||||||
|
-- MARS-1049-ADHOC: Verify CSDB External Tables Creation
|
||||||
|
--=============================================================================================================================
|
||||||
|
-- Purpose: Verify all 18 CSDB external tables exist with correct configuration
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-26
|
||||||
|
-- Related: MARS-1049 (CSV Encoding Support), MARS-821 (Original CSDB Setup)
|
||||||
|
--
|
||||||
|
-- Expected Tables (18 total):
|
||||||
|
-- CSDB_DEBT: INBOX, ODS, ARCHIVE
|
||||||
|
-- CSDB_DEBT_DAILY: INBOX, ODS, ARCHIVE
|
||||||
|
-- CSDB_INSTR_DESC_FULL: INBOX, ODS, ARCHIVE
|
||||||
|
-- CSDB_INSTR_RAT_FULL: INBOX, ODS, ARCHIVE
|
||||||
|
-- CSDB_ISSUER_DESC_FULL: INBOX, ODS, ARCHIVE
|
||||||
|
-- CSDB_ISSUER_RAT_FULL: INBOX, ODS, ARCHIVE
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
SET LINESIZE 200
|
||||||
|
SET PAGESIZE 100
|
||||||
|
SET SERVEROUTPUT ON
|
||||||
|
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT MARS-1049-ADHOC: External Tables Verification Report
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT
|
||||||
|
|
||||||
|
PROMPT ========================================
|
||||||
|
PROMPT Step 1: Check External Tables Existence
|
||||||
|
PROMPT ========================================
|
||||||
|
|
||||||
|
SELECT
|
||||||
|
owner,
|
||||||
|
table_name,
|
||||||
|
CASE
|
||||||
|
WHEN table_name LIKE '%_INBOX' THEN 'INBOX'
|
||||||
|
WHEN table_name LIKE '%_ODS' THEN 'ODS'
|
||||||
|
WHEN table_name LIKE '%_ARCHIVE' THEN 'ARCHIVE'
|
||||||
|
END AS bucket_area,
|
||||||
|
default_directory_name AS directory,
|
||||||
|
type_name
|
||||||
|
FROM all_external_tables
|
||||||
|
WHERE owner = 'ODS'
|
||||||
|
AND table_name IN (
|
||||||
|
'CSDB_DEBT_ARCHIVE',
|
||||||
|
'CSDB_DEBT_INBOX',
|
||||||
|
'CSDB_DEBT_ODS',
|
||||||
|
'CSDB_DEBT_DAILY_ARCHIVE',
|
||||||
|
'CSDB_DEBT_DAILY_INBOX',
|
||||||
|
'CSDB_DEBT_DAILY_ODS',
|
||||||
|
'CSDB_INSTR_DESC_FULL_ARCHIVE',
|
||||||
|
'CSDB_INSTR_DESC_FULL_INBOX',
|
||||||
|
'CSDB_INSTR_DESC_FULL_ODS',
|
||||||
|
'CSDB_INSTR_RAT_FULL_ARCHIVE',
|
||||||
|
'CSDB_INSTR_RAT_FULL_INBOX',
|
||||||
|
'CSDB_INSTR_RAT_FULL_ODS',
|
||||||
|
'CSDB_ISSUER_DESC_FULL_ARCHIVE',
|
||||||
|
'CSDB_ISSUER_DESC_FULL_INBOX',
|
||||||
|
'CSDB_ISSUER_DESC_FULL_ODS',
|
||||||
|
'CSDB_ISSUER_RAT_FULL_ARCHIVE',
|
||||||
|
'CSDB_ISSUER_RAT_FULL_INBOX',
|
||||||
|
'CSDB_ISSUER_RAT_FULL_ODS'
|
||||||
|
)
|
||||||
|
ORDER BY table_name;
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT ========================================
|
||||||
|
PROMPT Step 2: Verify Table Counts by Category
|
||||||
|
PROMPT ========================================
|
||||||
|
|
||||||
|
SELECT
|
||||||
|
CASE
|
||||||
|
WHEN table_name LIKE 'CSDB_DEBT_DAILY%' THEN 'CSDB_DEBT_DAILY'
|
||||||
|
WHEN table_name LIKE 'CSDB_DEBT%' THEN 'CSDB_DEBT'
|
||||||
|
WHEN table_name LIKE 'CSDB_INSTR_DESC_FULL%' THEN 'CSDB_INSTR_DESC_FULL'
|
||||||
|
WHEN table_name LIKE 'CSDB_INSTR_RAT_FULL%' THEN 'CSDB_INSTR_RAT_FULL'
|
||||||
|
WHEN table_name LIKE 'CSDB_ISSUER_DESC_FULL%' THEN 'CSDB_ISSUER_DESC_FULL'
|
||||||
|
WHEN table_name LIKE 'CSDB_ISSUER_RAT_FULL%' THEN 'CSDB_ISSUER_RAT_FULL'
|
||||||
|
END AS table_category,
|
||||||
|
COUNT(*) AS table_count,
|
||||||
|
CASE WHEN COUNT(*) = 3 THEN 'COMPLETE' ELSE 'INCOMPLETE' END AS status
|
||||||
|
FROM all_external_tables
|
||||||
|
WHERE owner = 'ODS'
|
||||||
|
AND table_name IN (
|
||||||
|
'CSDB_DEBT_ARCHIVE', 'CSDB_DEBT_INBOX', 'CSDB_DEBT_ODS',
|
||||||
|
'CSDB_DEBT_DAILY_ARCHIVE', 'CSDB_DEBT_DAILY_INBOX', 'CSDB_DEBT_DAILY_ODS',
|
||||||
|
'CSDB_INSTR_DESC_FULL_ARCHIVE', 'CSDB_INSTR_DESC_FULL_INBOX', 'CSDB_INSTR_DESC_FULL_ODS',
|
||||||
|
'CSDB_INSTR_RAT_FULL_ARCHIVE', 'CSDB_INSTR_RAT_FULL_INBOX', 'CSDB_INSTR_RAT_FULL_ODS',
|
||||||
|
'CSDB_ISSUER_DESC_FULL_ARCHIVE', 'CSDB_ISSUER_DESC_FULL_INBOX', 'CSDB_ISSUER_DESC_FULL_ODS',
|
||||||
|
'CSDB_ISSUER_RAT_FULL_ARCHIVE', 'CSDB_ISSUER_RAT_FULL_INBOX', 'CSDB_ISSUER_RAT_FULL_ODS'
|
||||||
|
)
|
||||||
|
GROUP BY
|
||||||
|
CASE
|
||||||
|
WHEN table_name LIKE 'CSDB_DEBT_DAILY%' THEN 'CSDB_DEBT_DAILY'
|
||||||
|
WHEN table_name LIKE 'CSDB_DEBT%' THEN 'CSDB_DEBT'
|
||||||
|
WHEN table_name LIKE 'CSDB_INSTR_DESC_FULL%' THEN 'CSDB_INSTR_DESC_FULL'
|
||||||
|
WHEN table_name LIKE 'CSDB_INSTR_RAT_FULL%' THEN 'CSDB_INSTR_RAT_FULL'
|
||||||
|
WHEN table_name LIKE 'CSDB_ISSUER_DESC_FULL%' THEN 'CSDB_ISSUER_DESC_FULL'
|
||||||
|
WHEN table_name LIKE 'CSDB_ISSUER_RAT_FULL%' THEN 'CSDB_ISSUER_RAT_FULL'
|
||||||
|
END
|
||||||
|
ORDER BY table_category;
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT ========================================
|
||||||
|
PROMPT Step 3: Overall Summary
|
||||||
|
PROMPT ========================================
|
||||||
|
|
||||||
|
DECLARE
|
||||||
|
v_total_count NUMBER;
|
||||||
|
v_expected_count NUMBER := 18;
|
||||||
|
v_status VARCHAR2(20);
|
||||||
|
BEGIN
|
||||||
|
SELECT COUNT(*) INTO v_total_count
|
||||||
|
FROM all_external_tables
|
||||||
|
WHERE owner = 'ODS'
|
||||||
|
AND table_name IN (
|
||||||
|
'CSDB_DEBT_ARCHIVE', 'CSDB_DEBT_INBOX', 'CSDB_DEBT_ODS',
|
||||||
|
'CSDB_DEBT_DAILY_ARCHIVE', 'CSDB_DEBT_DAILY_INBOX', 'CSDB_DEBT_DAILY_ODS',
|
||||||
|
'CSDB_INSTR_DESC_FULL_ARCHIVE', 'CSDB_INSTR_DESC_FULL_INBOX', 'CSDB_INSTR_DESC_FULL_ODS',
|
||||||
|
'CSDB_INSTR_RAT_FULL_ARCHIVE', 'CSDB_INSTR_RAT_FULL_INBOX', 'CSDB_INSTR_RAT_FULL_ODS',
|
||||||
|
'CSDB_ISSUER_DESC_FULL_ARCHIVE', 'CSDB_ISSUER_DESC_FULL_INBOX', 'CSDB_ISSUER_DESC_FULL_ODS',
|
||||||
|
'CSDB_ISSUER_RAT_FULL_ARCHIVE', 'CSDB_ISSUER_RAT_FULL_INBOX', 'CSDB_ISSUER_RAT_FULL_ODS'
|
||||||
|
);
|
||||||
|
|
||||||
|
IF v_total_count = v_expected_count THEN
|
||||||
|
v_status := 'SUCCESS';
|
||||||
|
ELSE
|
||||||
|
v_status := 'FAILED';
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
DBMS_OUTPUT.PUT_LINE('=========================================================================');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('VERIFICATION STATUS: ' || v_status);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('=========================================================================');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Expected Tables: ' || v_expected_count);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Found Tables: ' || v_total_count);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Missing Tables: ' || (v_expected_count - v_total_count));
|
||||||
|
DBMS_OUTPUT.PUT_LINE('=========================================================================');
|
||||||
|
|
||||||
|
IF v_total_count = v_expected_count THEN
|
||||||
|
DBMS_OUTPUT.PUT_LINE('');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('All 18 CSDB external tables created successfully with WE8MSWIN1252 encoding');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('MARS-1049-ADHOC deployment completed successfully');
|
||||||
|
ELSE
|
||||||
|
DBMS_OUTPUT.PUT_LINE('');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('WARNING: Not all tables were created. Review errors from step 02.');
|
||||||
|
RAISE_APPLICATION_ERROR(-20001, 'External tables verification failed');
|
||||||
|
END IF;
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT ========================================
|
||||||
|
PROMPT Step 4: Check Missing Tables (if any)
|
||||||
|
PROMPT ========================================
|
||||||
|
|
||||||
|
WITH expected_tables AS (
|
||||||
|
SELECT 'CSDB_DEBT_ARCHIVE' AS table_name FROM DUAL UNION ALL
|
||||||
|
SELECT 'CSDB_DEBT_INBOX' FROM DUAL UNION ALL
|
||||||
|
SELECT 'CSDB_DEBT_ODS' FROM DUAL UNION ALL
|
||||||
|
SELECT 'CSDB_DEBT_DAILY_ARCHIVE' FROM DUAL UNION ALL
|
||||||
|
SELECT 'CSDB_DEBT_DAILY_INBOX' FROM DUAL UNION ALL
|
||||||
|
SELECT 'CSDB_DEBT_DAILY_ODS' FROM DUAL UNION ALL
|
||||||
|
SELECT 'CSDB_INSTR_DESC_FULL_ARCHIVE' FROM DUAL UNION ALL
|
||||||
|
SELECT 'CSDB_INSTR_DESC_FULL_INBOX' FROM DUAL UNION ALL
|
||||||
|
SELECT 'CSDB_INSTR_DESC_FULL_ODS' FROM DUAL UNION ALL
|
||||||
|
SELECT 'CSDB_INSTR_RAT_FULL_ARCHIVE' FROM DUAL UNION ALL
|
||||||
|
SELECT 'CSDB_INSTR_RAT_FULL_INBOX' FROM DUAL UNION ALL
|
||||||
|
SELECT 'CSDB_INSTR_RAT_FULL_ODS' FROM DUAL UNION ALL
|
||||||
|
SELECT 'CSDB_ISSUER_DESC_FULL_ARCHIVE' FROM DUAL UNION ALL
|
||||||
|
SELECT 'CSDB_ISSUER_DESC_FULL_INBOX' FROM DUAL UNION ALL
|
||||||
|
SELECT 'CSDB_ISSUER_DESC_FULL_ODS' FROM DUAL UNION ALL
|
||||||
|
SELECT 'CSDB_ISSUER_RAT_FULL_ARCHIVE' FROM DUAL UNION ALL
|
||||||
|
SELECT 'CSDB_ISSUER_RAT_FULL_INBOX' FROM DUAL UNION ALL
|
||||||
|
SELECT 'CSDB_ISSUER_RAT_FULL_ODS' FROM DUAL
|
||||||
|
)
|
||||||
|
SELECT e.table_name AS missing_table
|
||||||
|
FROM expected_tables e
|
||||||
|
LEFT JOIN all_external_tables t ON e.table_name = t.table_name AND t.owner = 'ODS'
|
||||||
|
WHERE t.table_name IS NULL
|
||||||
|
ORDER BY e.table_name;
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Verification Report Complete
|
||||||
|
PROMPT =========================================================================
|
||||||
@@ -0,0 +1,84 @@
|
|||||||
|
--=============================================================================================================================
|
||||||
|
-- MARS-1049-ADHOC: Rollback CSDB Encoding Configuration
|
||||||
|
--=============================================================================================================================
|
||||||
|
-- Purpose: Rollback CSDB encoding from WE8MSWIN1252 to UTF8
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-26
|
||||||
|
-- Related: MARS-1049 (CSV Encoding Support), MARS-821 (Original CSDB Setup)
|
||||||
|
--
|
||||||
|
-- Use Case: Restore original UTF8 encoding if WE8MSWIN1252 causes issues
|
||||||
|
--
|
||||||
|
-- WARNING: External tables must be recreated after encoding change
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
SET SERVEROUTPUT ON
|
||||||
|
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT MARS-1049-ADHOC: Rollback CSDB Encoding to UTF8
|
||||||
|
PROMPT =========================================================================
|
||||||
|
|
||||||
|
-- Step 1: Show current state
|
||||||
|
PROMPT
|
||||||
|
PROMPT Step 1: Current CSDB encoding configuration
|
||||||
|
PROMPT ========================================
|
||||||
|
|
||||||
|
SELECT
|
||||||
|
TABLE_ID,
|
||||||
|
SOURCE_FILE_ID,
|
||||||
|
ENCODING AS CURRENT_ENCODING,
|
||||||
|
TEMPLATE_TABLE_NAME
|
||||||
|
FROM CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
WHERE TEMPLATE_TABLE_NAME LIKE '%CSDB%'
|
||||||
|
ORDER BY TABLE_ID, SOURCE_FILE_ID;
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT Step 2: Rollback CSDB configurations to UTF8
|
||||||
|
PROMPT ========================================
|
||||||
|
|
||||||
|
-- Rollback all CSDB configurations to UTF8 encoding
|
||||||
|
UPDATE /*+ NOPARALLEL */ CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
SET ENCODING = 'UTF8'
|
||||||
|
WHERE TEMPLATE_TABLE_NAME LIKE '%CSDB%';
|
||||||
|
|
||||||
|
PROMPT Rolled back CSDB configurations to UTF8 encoding
|
||||||
|
|
||||||
|
-- Commit changes
|
||||||
|
COMMIT;
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT Step 3: Verify rolled back encoding configuration
|
||||||
|
PROMPT ========================================
|
||||||
|
|
||||||
|
SELECT
|
||||||
|
TABLE_ID,
|
||||||
|
SOURCE_FILE_ID,
|
||||||
|
ENCODING AS RESTORED_ENCODING,
|
||||||
|
TEMPLATE_TABLE_NAME
|
||||||
|
FROM CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
WHERE TEMPLATE_TABLE_NAME LIKE '%CSDB%'
|
||||||
|
ORDER BY TABLE_ID, SOURCE_FILE_ID;
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT Step 4: Overall encoding distribution
|
||||||
|
PROMPT ========================================
|
||||||
|
|
||||||
|
SELECT
|
||||||
|
ENCODING,
|
||||||
|
COUNT(*) AS CONFIG_COUNT,
|
||||||
|
LISTAGG(DISTINCT TABLE_ID, ', ') WITHIN GROUP (ORDER BY TABLE_ID) AS TABLES
|
||||||
|
FROM CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
GROUP BY ENCODING
|
||||||
|
ORDER BY ENCODING;
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT CSDB Encoding Rollback - COMPLETED
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT Summary:
|
||||||
|
PROMPT - All CSDB configurations rolled back to UTF8
|
||||||
|
PROMPT - Original encoding restored
|
||||||
|
PROMPT
|
||||||
|
PROMPT IMPORTANT: External tables must be recreated with new encoding
|
||||||
|
PROMPT - Use MARS-821 scripts to recreate external tables with UTF8
|
||||||
|
PROMPT =========================================================================
|
||||||
@@ -0,0 +1,297 @@
|
|||||||
|
--=============================================================================================================================
|
||||||
|
-- MARS-1049-ADHOC: Recreate External Tables with UTF8 Encoding (Rollback Step)
|
||||||
|
--=============================================================================================================================
|
||||||
|
-- Purpose: Recreate CSDB external tables with UTF8 encoding (original encoding)
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-26
|
||||||
|
-- Related: MARS-1049 (CSV Encoding Support), MARS-821 (Original CSDB Setup)
|
||||||
|
--
|
||||||
|
-- Tables to be created (18 total with UTF8 encoding):
|
||||||
|
-- CSDB_DEBT: INBOX, ODS, ARCHIVE
|
||||||
|
-- CSDB_DEBT_DAILY: INBOX, ODS, ARCHIVE
|
||||||
|
-- CSDB_INSTR_DESC_FULL: INBOX, ODS, ARCHIVE
|
||||||
|
-- CSDB_INSTR_RAT_FULL: INBOX, ODS, ARCHIVE
|
||||||
|
-- CSDB_ISSUER_DESC_FULL: INBOX, ODS, ARCHIVE
|
||||||
|
-- CSDB_ISSUER_RAT_FULL: INBOX, ODS, ARCHIVE
|
||||||
|
--
|
||||||
|
-- Prerequisites:
|
||||||
|
-- - Template tables must exist in CT_ET_TEMPLATES schema
|
||||||
|
-- - FILE_MANAGER_ODS v2.1.0+ with pEncoding parameter support
|
||||||
|
-- - Proper bucket URIs configured in ENV_MANAGER
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
SET SERVEROUTPUT ON
|
||||||
|
|
||||||
|
PROMPT ========================================================
|
||||||
|
PROMPT Step 1: Create CSDB_DEBT External Tables with UTF8
|
||||||
|
PROMPT ========================================================
|
||||||
|
|
||||||
|
-- INBOX - For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_DEBT_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_DEBT',
|
||||||
|
pPrefix => 'INBOX/CSDB/CentralizedSecuritiesDissemination/CSDB_DEBT',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pEncoding => 'UTF8'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_DEBT_INBOX with encoding UTF8');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ODS - For processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_DEBT_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_DEBT',
|
||||||
|
pPrefix => 'ODS/CSDB/CSDB_DEBT',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri,
|
||||||
|
pEncoding => 'UTF8'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_DEBT_ODS with encoding UTF8');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_DEBT_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_DEBT',
|
||||||
|
pPrefix => 'ARCHIVE/CSDB/CSDB_DEBT',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri,
|
||||||
|
pEncoding => 'UTF8'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_DEBT_ARCHIVE with encoding UTF8');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
PROMPT ================================================================
|
||||||
|
PROMPT Step 2: Create CSDB_DEBT_DAILY External Tables with UTF8
|
||||||
|
PROMPT ================================================================
|
||||||
|
|
||||||
|
-- INBOX - For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_DEBT_DAILY_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_DEBT_DAILY',
|
||||||
|
pPrefix => 'INBOX/CSDB/CentralizedSecuritiesDailyReferenceDataDissemination/CSDB_DEBT_DAILY',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pEncoding => 'UTF8'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_DEBT_DAILY_INBOX with encoding UTF8');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ODS - For processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_DEBT_DAILY_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_DEBT_DAILY',
|
||||||
|
pPrefix => 'ODS/CSDB/CSDB_DEBT_DAILY',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri,
|
||||||
|
pEncoding => 'UTF8'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_DEBT_DAILY_ODS with encoding UTF8');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_DEBT_DAILY_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_DEBT_DAILY',
|
||||||
|
pPrefix => 'ARCHIVE/CSDB/CSDB_DEBT_DAILY',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri,
|
||||||
|
pEncoding => 'UTF8'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_DEBT_DAILY_ARCHIVE with encoding UTF8');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
PROMPT ========================================================================
|
||||||
|
PROMPT Step 3: Create CSDB_INSTR_DESC_FULL External Tables with UTF8
|
||||||
|
PROMPT ========================================================================
|
||||||
|
|
||||||
|
-- INBOX - For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_INSTR_DESC_FULL_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_INSTR_DESC_FULL',
|
||||||
|
pPrefix => 'INBOX/CSDB/FullRatingsDissemination/CSDB_INSTR_DESC_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pEncoding => 'UTF8'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_INSTR_DESC_FULL_INBOX with encoding UTF8');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ODS - For processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_INSTR_DESC_FULL_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_INSTR_DESC_FULL',
|
||||||
|
pPrefix => 'ODS/CSDB/CSDB_INSTR_DESC_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri,
|
||||||
|
pEncoding => 'UTF8'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_INSTR_DESC_FULL_ODS with encoding UTF8');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_INSTR_DESC_FULL_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_INSTR_DESC_FULL',
|
||||||
|
pPrefix => 'ARCHIVE/CSDB/CSDB_INSTR_DESC_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri,
|
||||||
|
pEncoding => 'UTF8'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_INSTR_DESC_FULL_ARCHIVE with encoding UTF8');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
PROMPT ========================================================================
|
||||||
|
PROMPT Step 4: Create CSDB_INSTR_RAT_FULL External Tables with UTF8
|
||||||
|
PROMPT ========================================================================
|
||||||
|
|
||||||
|
-- INBOX - For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_INSTR_RAT_FULL_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_INSTR_RAT_FULL',
|
||||||
|
pPrefix => 'INBOX/CSDB/FullRatingsDissemination/CSDB_INSTR_RAT_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pEncoding => 'UTF8'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_INSTR_RAT_FULL_INBOX with encoding UTF8');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ODS - For processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_INSTR_RAT_FULL_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_INSTR_RAT_FULL',
|
||||||
|
pPrefix => 'ODS/CSDB/CSDB_INSTR_RAT_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri,
|
||||||
|
pEncoding => 'UTF8'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_INSTR_RAT_FULL_ODS with encoding UTF8');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_INSTR_RAT_FULL_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_INSTR_RAT_FULL',
|
||||||
|
pPrefix => 'ARCHIVE/CSDB/CSDB_INSTR_RAT_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri,
|
||||||
|
pEncoding => 'UTF8'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_INSTR_RAT_FULL_ARCHIVE with encoding UTF8');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Step 5: Create CSDB_ISSUER_DESC_FULL External Tables with UTF8
|
||||||
|
PROMPT =========================================================================
|
||||||
|
|
||||||
|
-- INBOX - For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_ISSUER_DESC_FULL_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_ISSUER_DESC_FULL',
|
||||||
|
pPrefix => 'INBOX/CSDB/FullRatingsDissemination/CSDB_ISSUER_DESC_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pEncoding => 'UTF8'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_ISSUER_DESC_FULL_INBOX with encoding UTF8');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ODS - For processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_ISSUER_DESC_FULL_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_ISSUER_DESC_FULL',
|
||||||
|
pPrefix => 'ODS/CSDB/CSDB_ISSUER_DESC_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri,
|
||||||
|
pEncoding => 'UTF8'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_ISSUER_DESC_FULL_ODS with encoding UTF8');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_ISSUER_DESC_FULL_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_ISSUER_DESC_FULL',
|
||||||
|
pPrefix => 'ARCHIVE/CSDB/CSDB_ISSUER_DESC_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri,
|
||||||
|
pEncoding => 'UTF8'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_ISSUER_DESC_FULL_ARCHIVE with encoding UTF8');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Step 6: Create CSDB_ISSUER_RAT_FULL External Tables with UTF8
|
||||||
|
PROMPT =========================================================================
|
||||||
|
|
||||||
|
-- INBOX - For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_ISSUER_RAT_FULL_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_ISSUER_RAT_FULL',
|
||||||
|
pPrefix => 'INBOX/CSDB/FullRatingsDissemination/CSDB_ISSUER_RAT_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pEncoding => 'UTF8'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_ISSUER_RAT_FULL_INBOX with encoding UTF8');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ODS - For processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_ISSUER_RAT_FULL_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_ISSUER_RAT_FULL',
|
||||||
|
pPrefix => 'ODS/CSDB/CSDB_ISSUER_RAT_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri,
|
||||||
|
pEncoding => 'UTF8'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_ISSUER_RAT_FULL_ODS with encoding UTF8');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_ISSUER_RAT_FULL_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_ISSUER_RAT_FULL',
|
||||||
|
pPrefix => 'ARCHIVE/CSDB/CSDB_ISSUER_RAT_FULL',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri,
|
||||||
|
pEncoding => 'UTF8'
|
||||||
|
);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: Created ODS.CSDB_ISSUER_RAT_FULL_ARCHIVE with encoding UTF8');
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Recreate External Tables with UTF8 - COMPLETED
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT Summary:
|
||||||
|
PROMPT - CSDB_DEBT tables: 3 created (INBOX, ODS, ARCHIVE)
|
||||||
|
PROMPT - CSDB_DEBT_DAILY tables: 3 created (INBOX, ODS, ARCHIVE)
|
||||||
|
PROMPT - CSDB_INSTR_DESC_FULL tables: 3 created (INBOX, ODS, ARCHIVE)
|
||||||
|
PROMPT - CSDB_INSTR_RAT_FULL tables: 3 created (INBOX, ODS, ARCHIVE)
|
||||||
|
PROMPT - CSDB_ISSUER_DESC_FULL tables: 3 created (INBOX, ODS, ARCHIVE)
|
||||||
|
PROMPT - CSDB_ISSUER_RAT_FULL tables: 3 created (INBOX, ODS, ARCHIVE)
|
||||||
|
PROMPT - Total: 18 external tables created with UTF8 encoding
|
||||||
|
PROMPT
|
||||||
|
PROMPT Next Step: Run 03_MARS_1049_ADHOC_verify_external_tables.sql
|
||||||
|
PROMPT =========================================================================
|
||||||
102
MARS_Packages/REL01/MARS-1049-ADHOC/install_mars1049_adhoc.sql
Normal file
102
MARS_Packages/REL01/MARS-1049-ADHOC/install_mars1049_adhoc.sql
Normal file
@@ -0,0 +1,102 @@
|
|||||||
|
--=============================================================================================================================
|
||||||
|
-- MARS-1049-ADHOC: Master Installation Script
|
||||||
|
--=============================================================================================================================
|
||||||
|
-- Purpose: Execute all MARS-1049-ADHOC scripts to recreate CSDB external tables with encoding support
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-26
|
||||||
|
-- Related: MARS-1049 (CSV Encoding Support), MARS-821 (Original CSDB Setup)
|
||||||
|
--
|
||||||
|
-- Prerequisites:
|
||||||
|
-- - FILE_MANAGER v3.2.1+ with pEncoding parameter support
|
||||||
|
-- - FILE_MANAGER_ODS v2.1.0+ with pEncoding parameter support
|
||||||
|
-- - Template tables exist in CT_ET_TEMPLATES schema
|
||||||
|
-- - Proper bucket URIs configured in ENV_MANAGER
|
||||||
|
-- - Execute as ODS user or with sufficient privileges
|
||||||
|
--
|
||||||
|
-- Deployment Sequence:
|
||||||
|
-- Step 1: Drop existing CSDB external tables
|
||||||
|
-- Step 2: Recreate all 18 CSDB external tables with WE8MSWIN1252 encoding
|
||||||
|
-- Step 3: Verify all tables created successfully
|
||||||
|
--
|
||||||
|
-- Rollback:
|
||||||
|
-- No rollback script provided - tables can be recreated using MARS-821 scripts if needed
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
SET SERVEROUTPUT ON
|
||||||
|
SET ECHO ON
|
||||||
|
SET VERIFY OFF
|
||||||
|
|
||||||
|
-- ===================================================================
|
||||||
|
-- MARS-1049-ADHOC INSTALL SCRIPT: CSDB External Tables Encoding Update
|
||||||
|
-- ===================================================================
|
||||||
|
-- Purpose: Recreate CSDB external tables with WE8MSWIN1252 encoding
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-26
|
||||||
|
-- Version: 1.0.0
|
||||||
|
|
||||||
|
-- Dynamic spool file generation (using SYS_CONTEXT - no DBA privileges required)
|
||||||
|
var filename VARCHAR2(100)
|
||||||
|
BEGIN
|
||||||
|
:filename := 'INSTALL_MARS_1049_ADHOC_' || SYS_CONTEXT('USERENV', 'CON_NAME') || '_' || TO_CHAR(SYSDATE,'YYYYMMDD_HH24MISS') || '.log';
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
column filename new_value _filename
|
||||||
|
select :filename filename from dual;
|
||||||
|
spool &_filename
|
||||||
|
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT MARS-1049-ADHOC: CSDB External Tables Encoding Update
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT This script will:
|
||||||
|
PROMPT - Update CSDB configurations to WE8MSWIN1252 encoding
|
||||||
|
PROMPT - Drop existing CSDB external tables (if any)
|
||||||
|
PROMPT - Recreate 18 CSDB external tables with WE8MSWIN1252 encoding
|
||||||
|
PROMPT - Verify successful creation
|
||||||
|
PROMPT
|
||||||
|
PROMPT Expected Duration: 2-3 minutes
|
||||||
|
PROMPT =========================================================================
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Step 0: Update CSDB Encoding Configuration
|
||||||
|
PROMPT =========================================================================
|
||||||
|
@@00_MARS_1049_ADHOC_update_encoding.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Step 1: Drop Existing CSDB External Tables
|
||||||
|
PROMPT =========================================================================
|
||||||
|
@@01_MARS_1049_ADHOC_drop_external_tables.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Step 2: Recreate CSDB External Tables with Encoding
|
||||||
|
PROMPT =========================================================================
|
||||||
|
@@02_MARS_1049_ADHOC_recreate_external_tables.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Step 3: Verify External Tables Creation
|
||||||
|
PROMPT =========================================================================
|
||||||
|
@@03_MARS_1049_ADHOC_verify_external_tables.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT MARS-1049-ADHOC Installation - COMPLETED
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Check the log file for complete installation details.
|
||||||
|
PROMPT =========================================================================
|
||||||
|
|
||||||
|
spool off
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT MARS-1049-ADHOC Installation Complete
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT All CSDB external tables have been updated with WE8MSWIN1252 encoding.
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
|
||||||
|
quit;
|
||||||
106
MARS_Packages/REL01/MARS-1049-ADHOC/rollback_mars1049_adhoc.sql
Normal file
106
MARS_Packages/REL01/MARS-1049-ADHOC/rollback_mars1049_adhoc.sql
Normal file
@@ -0,0 +1,106 @@
|
|||||||
|
--=============================================================================================================================
|
||||||
|
-- MARS-1049-ADHOC: Master Rollback Script
|
||||||
|
--=============================================================================================================================
|
||||||
|
-- Purpose: Rollback CSDB external tables encoding changes from WE8MSWIN1252 to UTF8
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-26
|
||||||
|
-- Related: MARS-1049 (CSV Encoding Support), MARS-821 (Original CSDB Setup)
|
||||||
|
--
|
||||||
|
-- Prerequisites:
|
||||||
|
-- - FILE_MANAGER v3.2.1+ with pEncoding parameter support
|
||||||
|
-- - FILE_MANAGER_ODS v2.1.0+ with pEncoding parameter support
|
||||||
|
-- - Template tables exist in CT_ET_TEMPLATES schema
|
||||||
|
-- - Proper bucket URIs configured in ENV_MANAGER
|
||||||
|
-- - Execute as ODS user or with sufficient privileges
|
||||||
|
--
|
||||||
|
-- Rollback Sequence:
|
||||||
|
-- Step 1: Rollback CSDB configurations to UTF8 encoding
|
||||||
|
-- Step 2: Drop CSDB external tables with WE8MSWIN1252 encoding
|
||||||
|
-- Step 3: Recreate CSDB external tables with UTF8 encoding (using MARS-821 patterns)
|
||||||
|
-- Step 4: Verify all tables created successfully
|
||||||
|
--
|
||||||
|
-- Post-Rollback:
|
||||||
|
-- - CSDB configurations restored to UTF8
|
||||||
|
-- - External tables recreated with UTF8 encoding
|
||||||
|
-- - System restored to pre-MARS-1049-ADHOC state
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
SET SERVEROUTPUT ON
|
||||||
|
SET ECHO ON
|
||||||
|
SET VERIFY OFF
|
||||||
|
|
||||||
|
-- ===================================================================
|
||||||
|
-- MARS-1049-ADHOC ROLLBACK SCRIPT: Restore UTF8 Encoding for CSDB
|
||||||
|
-- ===================================================================
|
||||||
|
-- Purpose: Rollback CSDB external tables to UTF8 encoding
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-26
|
||||||
|
-- Version: 1.0.0
|
||||||
|
|
||||||
|
-- Dynamic spool file generation (using SYS_CONTEXT - no DBA privileges required)
|
||||||
|
var filename VARCHAR2(100)
|
||||||
|
BEGIN
|
||||||
|
:filename := 'ROLLBACK_MARS_1049_ADHOC_' || SYS_CONTEXT('USERENV', 'CON_NAME') || '_' || TO_CHAR(SYSDATE,'YYYYMMDD_HH24MISS') || '.log';
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
column filename new_value _filename
|
||||||
|
select :filename filename from dual;
|
||||||
|
spool &_filename
|
||||||
|
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT MARS-1049-ADHOC: Rollback to UTF8 Encoding
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT This script will:
|
||||||
|
PROMPT - Rollback CSDB configurations to UTF8 encoding
|
||||||
|
PROMPT - Drop existing CSDB external tables (WE8MSWIN1252)
|
||||||
|
PROMPT - Recreate 18 CSDB external tables with UTF8 encoding
|
||||||
|
PROMPT - Verify successful rollback
|
||||||
|
PROMPT
|
||||||
|
PROMPT Expected Duration: 2-3 minutes
|
||||||
|
PROMPT =========================================================================
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Step 1: Rollback CSDB Encoding Configuration to UTF8
|
||||||
|
PROMPT =========================================================================
|
||||||
|
@@91_MARS_1049_ADHOC_rollback_encoding.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Step 2: Drop Existing CSDB External Tables
|
||||||
|
PROMPT =========================================================================
|
||||||
|
@@01_MARS_1049_ADHOC_drop_external_tables.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Step 3: Recreate CSDB External Tables with UTF8 Encoding
|
||||||
|
PROMPT =========================================================================
|
||||||
|
@@92_MARS_1049_ADHOC_recreate_tables_utf8.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Step 4: Verify External Tables Creation
|
||||||
|
PROMPT =========================================================================
|
||||||
|
@@03_MARS_1049_ADHOC_verify_external_tables.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT MARS-1049-ADHOC Rollback - COMPLETED
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Check the log file for complete rollback details.
|
||||||
|
PROMPT =========================================================================
|
||||||
|
|
||||||
|
spool off
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT MARS-1049-ADHOC Rollback Complete
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT All CSDB external tables have been restored to UTF8 encoding.
|
||||||
|
PROMPT System returned to pre-MARS-1049-ADHOC state.
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
|
||||||
|
quit;
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
-- ===================================================================
|
||||||
|
-- MARS-1049 INSTALLATION COMPONENT: Grant Permissions for Hash Tracking
|
||||||
|
-- ===================================================================
|
||||||
|
-- Purpose: Grant CT_MRDS necessary permissions to read package source across schemas
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-25
|
||||||
|
-- Version: 1.0.0
|
||||||
|
-- Step: 00 (First step - must run before package deployment)
|
||||||
|
--
|
||||||
|
-- PROBLEM:
|
||||||
|
-- ENV_MANAGER.CALCULATE_PACKAGE_HASH cannot read ALL_SOURCE for ODS schema
|
||||||
|
-- This causes NULL hash values for ODS.FILE_MANAGER_ODS PACKAGE BODY
|
||||||
|
--
|
||||||
|
-- SOLUTION:
|
||||||
|
-- Grant SELECT on specific dictionary views (ALL_SOURCE, ALL_OBJECTS) to CT_MRDS
|
||||||
|
-- This is the MINIMUM required privilege for cross-schema package hash tracking
|
||||||
|
-- More secure than SELECT ANY DICTIONARY (only grants access to specific views needed)
|
||||||
|
--
|
||||||
|
-- USAGE:
|
||||||
|
-- - Execute as ADMIN user (system privilege grant required)
|
||||||
|
-- - Should be first step in MARS-1049 installation
|
||||||
|
-- - One-time setup that enables cross-schema package tracking
|
||||||
|
-- ===================================================================
|
||||||
|
|
||||||
|
PROMPT ===============================================
|
||||||
|
PROMPT Fixing permissions for cross-schema hash tracking
|
||||||
|
PROMPT ===============================================
|
||||||
|
|
||||||
|
-- Grant SELECT on ALL_SOURCE (required to read package source code for hash calculation)
|
||||||
|
GRANT SELECT ON ALL_SOURCE TO CT_MRDS;
|
||||||
|
|
||||||
|
PROMPT Granted SELECT on SYS.ALL_SOURCE to CT_MRDS
|
||||||
|
|
||||||
|
PROMPT ===============================================
|
||||||
|
PROMPT Permission fix completed (MINIMUM PRIVILEGE)
|
||||||
|
PROMPT ===============================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT CT_MRDS can now:
|
||||||
|
PROMPT - Read ALL_SOURCE for any schema (package source code)
|
||||||
|
PROMPT - Calculate SHA256 hashes for packages in ODS and other schemas
|
||||||
|
PROMPT
|
||||||
|
PROMPT CT_MRDS CANNOT:
|
||||||
|
PROMPT - Read user tables in ODS or other schemas
|
||||||
|
PROMPT - Access other dictionary views
|
||||||
|
PROMPT - Access any data outside of ALL_SOURCE view
|
||||||
|
PROMPT
|
||||||
|
PROMPT SECURITY: Absolute minimum privilege - only ALL_SOURCE view granted
|
||||||
|
PROMPT ===============================================
|
||||||
@@ -0,0 +1,69 @@
|
|||||||
|
-- ===================================================================
|
||||||
|
-- MARS-1049: CSV Encoding Support - Database Structure Changes
|
||||||
|
-- ===================================================================
|
||||||
|
-- Purpose: Add ENCODING column to A_SOURCE_FILE_CONFIG table to support
|
||||||
|
-- character set specification for CSV files in external tables
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-24
|
||||||
|
-- Version: 1.1.0
|
||||||
|
--
|
||||||
|
-- IMPLEMENTATION:
|
||||||
|
-- - Adds ENCODING VARCHAR2(50) column to CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
-- - Column stores Oracle character set names (UTF8, WE8MSWIN1252, etc.)
|
||||||
|
-- - Used by FILE_MANAGER.CREATE_EXTERNAL_TABLE for CHARACTERSET clause
|
||||||
|
-- - Enables proper handling of non-ASCII characters in CSV files
|
||||||
|
-- ===================================================================
|
||||||
|
|
||||||
|
PROMPT ================================================
|
||||||
|
PROMPT MARS-1049: Adding ENCODING column to A_SOURCE_FILE_CONFIG
|
||||||
|
PROMPT ================================================
|
||||||
|
|
||||||
|
-- Check if ENCODING column already exists
|
||||||
|
DECLARE
|
||||||
|
vCount NUMBER := 0;
|
||||||
|
BEGIN
|
||||||
|
SELECT COUNT(*) INTO vCount
|
||||||
|
FROM USER_TAB_COLUMNS
|
||||||
|
WHERE TABLE_NAME = 'A_SOURCE_FILE_CONFIG'
|
||||||
|
AND COLUMN_NAME = 'ENCODING';
|
||||||
|
|
||||||
|
IF vCount = 0 THEN
|
||||||
|
EXECUTE IMMEDIATE 'ALTER TABLE CT_MRDS.A_SOURCE_FILE_CONFIG ADD (ENCODING VARCHAR2(50) DEFAULT ''UTF8'')';
|
||||||
|
DBMS_OUTPUT.PUT_LINE('SUCCESS: ENCODING column added to A_SOURCE_FILE_CONFIG with DEFAULT UTF8');
|
||||||
|
ELSE
|
||||||
|
DBMS_OUTPUT.PUT_LINE('INFO: ENCODING column already exists in A_SOURCE_FILE_CONFIG');
|
||||||
|
END IF;
|
||||||
|
EXCEPTION
|
||||||
|
WHEN OTHERS THEN
|
||||||
|
DBMS_OUTPUT.PUT_LINE('ERROR: Failed to add ENCODING column: ' || SQLERRM);
|
||||||
|
RAISE;
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
-- Add column comment for documentation
|
||||||
|
COMMENT ON COLUMN CT_MRDS.A_SOURCE_FILE_CONFIG.ENCODING
|
||||||
|
IS 'Character encoding for files (e.g., UTF8, WE8MSWIN1252, EE8ISO8859P2). Used by CREATE_EXTERNAL_TABLE for CHARACTERSET specification.';
|
||||||
|
|
||||||
|
-- Verify the column was added successfully
|
||||||
|
SELECT
|
||||||
|
COLUMN_NAME,
|
||||||
|
DATA_TYPE,
|
||||||
|
DATA_LENGTH,
|
||||||
|
NULLABLE,
|
||||||
|
(SELECT COMMENTS FROM USER_COL_COMMENTS WHERE TABLE_NAME = 'A_SOURCE_FILE_CONFIG' AND COLUMN_NAME = 'ENCODING') AS COLUMN_COMMENT
|
||||||
|
FROM USER_TAB_COLUMNS
|
||||||
|
WHERE TABLE_NAME = 'A_SOURCE_FILE_CONFIG'
|
||||||
|
AND COLUMN_NAME = 'ENCODING';
|
||||||
|
|
||||||
|
PROMPT ===============================================
|
||||||
|
PROMPT MARS-1049: Database structure changes completed
|
||||||
|
PROMPT ===============================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT ENCODING column specifications:
|
||||||
|
PROMPT - Type: VARCHAR2(50)
|
||||||
|
PROMPT - Purpose: Store Oracle character set names for CSV files
|
||||||
|
PROMPT - Examples: UTF8, WE8MSWIN1252, EE8ISO8859P2, CL8MSWIN1251
|
||||||
|
PROMPT - Usage: Used by CREATE_EXTERNAL_TABLE procedures
|
||||||
|
PROMPT
|
||||||
|
PROMPT Next Step: Deploy updated FILE_MANAGER packages
|
||||||
|
PROMPT ===============================================
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
-- ===================================================================
|
||||||
|
-- MARS-1049: Update Existing Configurations with Encoding Values
|
||||||
|
-- ===================================================================
|
||||||
|
-- Purpose: Set default encoding values for existing A_SOURCE_FILE_CONFIG records
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-25
|
||||||
|
-- Version: 1.0.0
|
||||||
|
--
|
||||||
|
-- ENCODING STRATEGY:
|
||||||
|
-- - UTF8: Default for all configurations (modern Unicode standard)
|
||||||
|
-- - WE8MSWIN1252: Special case for CSDB templates (Windows-1252 compatibility)
|
||||||
|
-- ===================================================================
|
||||||
|
|
||||||
|
PROMPT ===============================================
|
||||||
|
PROMPT MARS-1049: Updating existing configurations with encoding values
|
||||||
|
PROMPT ===============================================
|
||||||
|
|
||||||
|
-- Check current state before updates
|
||||||
|
SELECT COUNT(*) AS TOTAL_CONFIGURATIONS,
|
||||||
|
COUNT(CASE WHEN ENCODING IS NULL THEN 1 END) AS NULL_ENCODING_COUNT,
|
||||||
|
COUNT(CASE WHEN ENCODING IS NOT NULL THEN 1 END) AS WITH_ENCODING_COUNT
|
||||||
|
FROM CT_MRDS.A_SOURCE_FILE_CONFIG;
|
||||||
|
|
||||||
|
-- Update all configurations to UTF8 (default)
|
||||||
|
UPDATE /*+ NOPARALLEL */ CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
SET ENCODING = 'UTF8'
|
||||||
|
WHERE ENCODING IS NULL;
|
||||||
|
|
||||||
|
PROMPT Updated configurations with UTF8 (default)
|
||||||
|
|
||||||
|
-- Commit first batch to avoid deadlock
|
||||||
|
COMMIT;
|
||||||
|
|
||||||
|
-- PROMPT Committed UTF8 updates - now processing CSDB configurations...
|
||||||
|
|
||||||
|
-- -- Update CSDB-specific configurations to Windows-1252
|
||||||
|
-- UPDATE /*+ NOPARALLEL */ CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
-- SET ENCODING = 'WE8MSWIN1252'
|
||||||
|
-- WHERE TEMPLATE_TABLE_NAME LIKE '%CSDB%';
|
||||||
|
|
||||||
|
-- PROMPT Updated CSDB configurations with WE8MSWIN1252
|
||||||
|
|
||||||
|
-- -- Commit changes
|
||||||
|
-- COMMIT;
|
||||||
|
|
||||||
|
-- Verify final state
|
||||||
|
PROMPT
|
||||||
|
PROMPT Final encoding distribution:
|
||||||
|
SELECT ENCODING, COUNT(*) AS COUNT
|
||||||
|
FROM CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
GROUP BY ENCODING
|
||||||
|
ORDER BY ENCODING;
|
||||||
|
|
||||||
|
PROMPT ===============================================
|
||||||
|
PROMPT Encoding configuration update completed
|
||||||
|
PROMPT ===============================================
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
-- ===================================================================
|
||||||
|
-- MARS-1049 ROLLBACK COMPONENT: Revoke Permissions for Hash Tracking
|
||||||
|
-- ===================================================================
|
||||||
|
-- Purpose: Revoke CT_MRDS permissions granted during MARS-1049 installation
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-25
|
||||||
|
-- Version: 1.0.0
|
||||||
|
-- Step: 90 (Last step - execute after package rollback)
|
||||||
|
--
|
||||||
|
-- ROLLBACK OPERATION:
|
||||||
|
-- Revokes SELECT ANY DICTIONARY from CT_MRDS
|
||||||
|
--
|
||||||
|
-- WARNING:
|
||||||
|
-- This may affect other packages that depend on cross-schema hash tracking
|
||||||
|
-- Only revoke if no other packages in ODS or other schemas need tracking
|
||||||
|
--
|
||||||
|
-- USAGE:
|
||||||
|
-- - Execute as ADMIN user (system privilege revoke required)
|
||||||
|
-- - Should be last step in MARS-1049 rollback (after restoring packages)
|
||||||
|
-- ===================================================================
|
||||||
|
|
||||||
|
PROMPT ===============================================
|
||||||
|
PROMPT Revoking cross-schema hash tracking permissions
|
||||||
|
PROMPT ===============================================
|
||||||
|
|
||||||
|
-- Check if privilege exists before revoking
|
||||||
|
SELECT PRIVILEGE, GRANTEE
|
||||||
|
FROM DBA_SYS_PRIVS
|
||||||
|
WHERE GRANTEE = 'CT_MRDS'
|
||||||
|
AND PRIVILEGE = 'SELECT ANY DICTIONARY';
|
||||||
|
|
||||||
|
-- Revoke SELECT ANY DICTIONARY from CT_MRDS
|
||||||
|
REVOKE SELECT ANY DICTIONARY FROM CT_MRDS;
|
||||||
|
|
||||||
|
PROMPT Revoked SELECT ANY DICTIONARY from CT_MRDS
|
||||||
|
|
||||||
|
PROMPT ===============================================
|
||||||
|
PROMPT Permission revocation completed
|
||||||
|
PROMPT ===============================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT CT_MRDS can NO LONGER:
|
||||||
|
PROMPT - Read ALL_SOURCE for schemas other than CT_MRDS
|
||||||
|
PROMPT - Calculate SHA256 hashes for packages in ODS and other schemas
|
||||||
|
PROMPT
|
||||||
|
PROMPT WARNING: This may cause NULL hash values for cross-schema packages
|
||||||
|
PROMPT If other packages depend on this, re-grant the privilege
|
||||||
|
PROMPT ===============================================
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
-- ===================================================================
|
||||||
|
-- MARS-1049 ROLLBACK COMPONENT: Remove ENCODING Column
|
||||||
|
-- ===================================================================
|
||||||
|
-- Purpose: Remove ENCODING column from A_SOURCE_FILE_CONFIG table
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-25
|
||||||
|
-- Version: 1.0.0
|
||||||
|
--
|
||||||
|
-- ROLLBACK OPERATION:
|
||||||
|
-- - Removes ENCODING VARCHAR2(50) column from CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
-- - Part of complete MARS-1049 rollback sequence
|
||||||
|
--
|
||||||
|
-- USAGE:
|
||||||
|
-- - Can be executed standalone or as part of main rollback script
|
||||||
|
-- - Includes verification steps for safe execution
|
||||||
|
-- ===================================================================
|
||||||
|
|
||||||
|
PROMPT ===============================================
|
||||||
|
PROMPT MARS-1049: Remove ENCODING Column from A_SOURCE_FILE_CONFIG
|
||||||
|
PROMPT ===============================================
|
||||||
|
|
||||||
|
-- Step 1: Remove ENCODING column from A_SOURCE_FILE_CONFIG
|
||||||
|
PROMPT Step 1: Remove ENCODING column from A_SOURCE_FILE_CONFIG...
|
||||||
|
PROMPT ===============================================
|
||||||
|
|
||||||
|
-- First check if column exists
|
||||||
|
SELECT COUNT(*) AS ENCODING_COLUMN_EXISTS
|
||||||
|
FROM all_tab_columns
|
||||||
|
WHERE owner = 'CT_MRDS'
|
||||||
|
AND table_name = 'A_SOURCE_FILE_CONFIG'
|
||||||
|
AND column_name = 'ENCODING';
|
||||||
|
|
||||||
|
-- Remove the ENCODING column
|
||||||
|
ALTER TABLE CT_MRDS.A_SOURCE_FILE_CONFIG DROP COLUMN ENCODING;
|
||||||
|
|
||||||
|
PROMPT ENCODING column removed from A_SOURCE_FILE_CONFIG
|
||||||
|
|
||||||
|
-- Final verification
|
||||||
|
SELECT COUNT(*) AS ENCODING_COLUMN_EXISTS_AFTER_REMOVAL
|
||||||
|
FROM all_tab_columns
|
||||||
|
WHERE owner = 'CT_MRDS'
|
||||||
|
AND table_name = 'A_SOURCE_FILE_CONFIG'
|
||||||
|
AND column_name = 'ENCODING';
|
||||||
|
|
||||||
|
PROMPT ===============================================
|
||||||
|
PROMPT ENCODING column removal completed successfully
|
||||||
|
PROMPT ===============================================
|
||||||
493
MARS_Packages/REL01/MARS-1049/README.md
Normal file
493
MARS_Packages/REL01/MARS-1049/README.md
Normal file
@@ -0,0 +1,493 @@
|
|||||||
|
# MARS-1049: CSV Encoding Support - Complete Implementation
|
||||||
|
|
||||||
|
## 🎯 Implementation Status: ✅ COMPLETED & FULLY TESTED
|
||||||
|
|
||||||
|
**Implementation Date:** 2025-11-24
|
||||||
|
**Production Testing Date:** 2025-11-25
|
||||||
|
**Final Validation Date:** 2025-11-25
|
||||||
|
**Database Version:** Oracle 23.26.0.1.0
|
||||||
|
**Package Versions:** CT_MRDS.FILE_MANAGER v3.2.1, ODS.FILE_MANAGER_ODS v2.1.0
|
||||||
|
**Status:** Production Ready & Fully Validated ✅
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📋 Overview
|
||||||
|
|
||||||
|
MARS-1049 implements comprehensive CSV encoding support in the Oracle FILE_MANAGER system, enabling proper handling of character sets when creating external tables for CSV file processing. This enhancement allows for proper processing of international data with various character encodings.
|
||||||
|
|
||||||
|
### Key Benefits
|
||||||
|
- **Enhanced Data Integrity**: Proper character set handling for international data
|
||||||
|
- **Flexibility**: Support for multiple encoding standards (UTF-8, Windows-1252, ISO-8859, etc.)
|
||||||
|
- **Backward Compatibility**: All existing code continues working unchanged
|
||||||
|
- **Simple Configuration**: Easy-to-use encoding parameter in existing procedures
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📁 Project Structure & Version Control
|
||||||
|
|
||||||
|
This implementation uses organized folder structure for version control and rollback capabilities:
|
||||||
|
|
||||||
|
```
|
||||||
|
MARS_Packages/REL01/MARS-1049/
|
||||||
|
├── current_version/ # 📦 Pre-MARS-1049 Versions
|
||||||
|
│ ├── FILE_MANAGER.pkg # v3.2.0 (without pEncoding)
|
||||||
|
│ ├── FILE_MANAGER.pkb # v3.2.0 (without pEncoding)
|
||||||
|
│ ├── FILE_MANAGER_ODS.pkg # v2.0.0 (without pEncoding)
|
||||||
|
│ └── FILE_MANAGER_ODS.pkb # v2.0.0 (without pEncoding)
|
||||||
|
├── new_version/ # 🚀 MARS-1049 Enhanced Versions
|
||||||
|
│ ├── FILE_MANAGER_SPEC.sql # v3.2.1 (with pEncoding)
|
||||||
|
│ ├── FILE_MANAGER_BODY.sql # v3.2.1 (with pEncoding)
|
||||||
|
│ ├── FILE_MANAGER_ODS_SPEC.sql # v2.1.0 (with pEncoding)
|
||||||
|
│ └── FILE_MANAGER_ODS_BODY.sql # v2.1.0 (with pEncoding)
|
||||||
|
├── install_mars1049.sql # 📥 Main Installation Script (with spool & tracking)
|
||||||
|
├── rollback_mars1049.sql # 🔄 Complete Rollback Script (with spool & tracking)
|
||||||
|
├── 04_MARS_1049_track_CT_MRDS_FILE_MANAGER_version.sql # 📈 Version Tracking (Install)
|
||||||
|
├── 92_MARS_1049_track_rollback_version.sql # 📈 Version Tracking (Rollback)
|
||||||
|
├── 91_MARS_1049_rollback_DROP_ENCODING_COLUMN.sql # 🔄 Column Removal Component
|
||||||
|
└── README.md # 📝 This Documentation
|
||||||
|
```
|
||||||
|
|
||||||
|
### Version Control Strategy
|
||||||
|
- **current_version/**: Original packages used by `rollback_mars1049.sql`
|
||||||
|
- **new_version/**: Enhanced packages used by `install_mars1049.sql`
|
||||||
|
- **Dynamic Spool Logging**: Automatic log file generation with timestamps
|
||||||
|
- **Version Tracking**: Complete audit trail through ENV_MANAGER.TRACK_PACKAGE_VERSION
|
||||||
|
- **Complete Change Tracking**: Full history of all modifications maintained
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔧 Database Changes Implemented
|
||||||
|
|
||||||
|
### 1. Table Structure Enhancement
|
||||||
|
```sql
|
||||||
|
-- Added to CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
ALTER TABLE CT_MRDS.A_SOURCE_FILE_CONFIG ADD (
|
||||||
|
ENCODING VARCHAR2(50) DEFAULT NULL -- Character encoding for CSV files
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Package Version Updates
|
||||||
|
| Package | Before | After | Changes |
|
||||||
|
|---------|---------|--------|---------|
|
||||||
|
| **CT_MRDS.FILE_MANAGER** | v3.2.0 | v3.2.1 | Added `pEncoding` parameter |
|
||||||
|
| **ODS.FILE_MANAGER_ODS** | v2.0.0 | v2.1.0 | Added encoding wrapper support |
|
||||||
|
|
||||||
|
### 3. Enhanced Procedures
|
||||||
|
- `FILE_MANAGER.ADD_SOURCE_FILE_CONFIG` - Added `pEncoding` parameter
|
||||||
|
- `FILE_MANAGER.CREATE_EXTERNAL_TABLE` - Added encoding support
|
||||||
|
- `FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE` - Added encoding delegation
|
||||||
|
|
||||||
|
### 4. Dynamic Spool Logging
|
||||||
|
```sql
|
||||||
|
-- Automatic log file generation
|
||||||
|
'INSTALL_MARS_1049_[PDB_NAME]_YYYYMMDD_HH24MISS.log'
|
||||||
|
'ROLLBACK_MARS_1049_[PDB_NAME]_YYYYMMDD_HH24MISS.log'
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. Version Tracking System
|
||||||
|
- `04_MARS_1049_track_CT_MRDS_FILE_MANAGER_version.sql` - Installation tracking
|
||||||
|
- `92_MARS_1049_track_rollback_version.sql` - Rollback tracking
|
||||||
|
- Complete audit trail via `CT_MRDS.ENV_MANAGER.TRACK_PACKAGE_VERSION`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🌍 Supported Character Encodings
|
||||||
|
|
||||||
|
| Encoding | Description | Use Case | Example |
|
||||||
|
|----------|-------------|----------|---------|
|
||||||
|
| `UTF8` / `UTF-8` | Unicode UTF-8 | Modern systems, international | Global applications |
|
||||||
|
| `WE8MSWIN1252` | Windows-1252 | Western European, Windows | Legacy Windows systems |
|
||||||
|
| `EE8ISO8859P2` | ISO-8859-2 | Central European | Polish, Czech, Hungarian |
|
||||||
|
| `CL8MSWIN1251` | Windows-1251 | Cyrillic | Russian, Bulgarian |
|
||||||
|
| `AL32UTF8` | Unicode UTF-8 (32-bit) | Full Unicode support | Enterprise systems |
|
||||||
|
| `JA16SJIS` | Shift JIS | Japanese | Japanese systems |
|
||||||
|
| `ZHS16GBK` | GBK | Chinese Simplified | Chinese systems |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚀 Installation & Deployment
|
||||||
|
|
||||||
|
### Quick Installation
|
||||||
|
```sql
|
||||||
|
-- Single command installation with automatic logging and version tracking
|
||||||
|
@@install_mars1049.sql
|
||||||
|
-- Creates: INSTALL_MARS_1049_[PDB]_[TIMESTAMP].log
|
||||||
|
-- Includes: 7 steps with version tracking in ENV_MANAGER
|
||||||
|
```
|
||||||
|
|
||||||
|
### Quick Rollback
|
||||||
|
```sql
|
||||||
|
-- Single command rollback with automatic logging and version tracking
|
||||||
|
@@rollback_mars1049.sql
|
||||||
|
-- Creates: ROLLBACK_MARS_1049_[PDB]_[TIMESTAMP].log
|
||||||
|
-- Includes: 4 steps with complete restoration and tracking
|
||||||
|
```
|
||||||
|
|
||||||
|
### Manual Step-by-Step Installation
|
||||||
|
```sql
|
||||||
|
-- Run in sequence with appropriate user privileges:
|
||||||
|
@@01_MARS_1049_install_CT_MRDS_ADD_ENCODING_COLUMN.sql -- CT_MRDS user
|
||||||
|
@@new_version/FILE_MANAGER_SPEC.sql -- CT_MRDS user
|
||||||
|
@@new_version/FILE_MANAGER_BODY.sql -- CT_MRDS user
|
||||||
|
@@new_version/FILE_MANAGER_ODS_SPEC.sql -- ODS user
|
||||||
|
@@new_version/FILE_MANAGER_ODS_BODY.sql -- ODS user
|
||||||
|
```
|
||||||
|
|
||||||
|
### Verification
|
||||||
|
```sql
|
||||||
|
-- Comprehensive functionality testing
|
||||||
|
@@test/05_MARS_1049_verify_encoding_functionality.sql
|
||||||
|
```
|
||||||
|
|
||||||
|
### Rollback (if needed)
|
||||||
|
```sql
|
||||||
|
-- Complete rollback to pre-MARS-1049 state
|
||||||
|
@@rollback_mars1049.sql
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 💡 Usage Examples
|
||||||
|
|
||||||
|
### 1. Basic Configuration with Encoding
|
||||||
|
```sql
|
||||||
|
-- Add source system with UTF-8 support
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE(
|
||||||
|
pSourceKey => 'INTL_SYS',
|
||||||
|
pSourceName => 'International Data System'
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Configure file processing with encoding
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'INTL_SYS',
|
||||||
|
pSourceFileType => 'INPUT',
|
||||||
|
pSourceFileId => 'CUSTOMER_DATA',
|
||||||
|
pSourceFileDesc => 'Customer data with international characters',
|
||||||
|
pSourceFileNamePattern => 'customers_*.csv',
|
||||||
|
pTableId => 'CUSTOMERS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CUSTOMERS',
|
||||||
|
pEncoding => 'UTF-8' -- 🆕 NEW: Encoding specification
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. External Table Creation with Encoding
|
||||||
|
```sql
|
||||||
|
-- Create external table with UTF-8 encoding
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CUSTOMERS_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CUSTOMERS',
|
||||||
|
pPrefix => 'INBOX/INTL_SYS/CUSTOMER_DATA/CUSTOMERS',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pEncoding => 'UTF-8' -- 🆕 NEW: Character set specification
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Backward Compatibility (No Changes Required)
|
||||||
|
```sql
|
||||||
|
-- Existing code continues working unchanged
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LEGACY_SOURCE',
|
||||||
|
pSourceFileType => 'INPUT',
|
||||||
|
pSourceFileId => 'LEGACY_DATA',
|
||||||
|
pSourceFileDesc => 'Legacy data files',
|
||||||
|
pSourceFileNamePattern => 'data_*.csv',
|
||||||
|
pTableId => 'LEGACY_TABLE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LEGACY'
|
||||||
|
-- No pEncoding parameter - uses default behavior
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. File Processing with Automatic Encoding
|
||||||
|
```sql
|
||||||
|
-- Process file using encoding from configuration
|
||||||
|
EXEC CT_MRDS.FILE_MANAGER.PROCESS_SOURCE_FILE(
|
||||||
|
'INBOX/INTL_SYS/CUSTOMER_DATA/CUSTOMERS/customers_20251124.csv'
|
||||||
|
);
|
||||||
|
-- Encoding automatically applied from A_SOURCE_FILE_CONFIG.ENCODING
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. Real Data Testing (CSDB Example)
|
||||||
|
```sql
|
||||||
|
-- Tested with real CSDB data file containing international characters
|
||||||
|
-- File: temp_upload.csv with Turkish characters ("Türkiye", "Turkiye")
|
||||||
|
-- Encoding: WE8MSWIN1252 for proper character handling
|
||||||
|
CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_DEBT_TEST',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_DEBT',
|
||||||
|
pPrefix => 'DATA/CSDB/DEBT',
|
||||||
|
pBucketUri => '...',
|
||||||
|
pEncoding => 'WE8MSWIN1252' -- For CSDB data with special characters
|
||||||
|
);
|
||||||
|
-- ✅ Successfully handles international character data
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚙️ Technical Implementation Details
|
||||||
|
|
||||||
|
### JSON Format Generation
|
||||||
|
```sql
|
||||||
|
-- IMPLEMENTATION: Conditional JSON_OBJECT construction
|
||||||
|
IF pEncoding IS NOT NULL AND LENGTH(TRIM(pEncoding)) > 0 THEN
|
||||||
|
vFormatJson := JSON_OBJECT(
|
||||||
|
'type' VALUE 'csv',
|
||||||
|
'delimiter' VALUE pDelimiter,
|
||||||
|
'characterset' VALUE pEncoding -- 🆕 Character set added
|
||||||
|
);
|
||||||
|
ELSE
|
||||||
|
vFormatJson := JSON_OBJECT(
|
||||||
|
'type' VALUE 'csv',
|
||||||
|
'delimiter' VALUE pDelimiter
|
||||||
|
-- No characterset for backward compatibility
|
||||||
|
);
|
||||||
|
END IF;
|
||||||
|
```
|
||||||
|
|
||||||
|
### External Table Result
|
||||||
|
**With Encoding:**
|
||||||
|
```
|
||||||
|
FORMAT JSON ('{"type":"csv","delimiter":",","characterset":"UTF-8"}')
|
||||||
|
```
|
||||||
|
|
||||||
|
**Without Encoding (backward compatible):**
|
||||||
|
```
|
||||||
|
FORMAT JSON ('{"type":"csv","delimiter":","}')
|
||||||
|
```
|
||||||
|
|
||||||
|
### Oracle 23c Compatibility
|
||||||
|
- **Issue Solved**: Replaced non-available `JSON_MERGEPATCH` with `JSON_OBJECT`
|
||||||
|
- **Result**: Full compatibility with Oracle 23.26.0.1.0
|
||||||
|
- **Performance**: Optimized JSON generation for better performance
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ✅ Comprehensive Testing Results
|
||||||
|
|
||||||
|
### Database Structure Tests
|
||||||
|
```sql
|
||||||
|
-- ✅ PASSED: ENCODING column added successfully
|
||||||
|
DESC CT_MRDS.A_SOURCE_FILE_CONFIG;
|
||||||
|
-- Shows: ENCODING VARCHAR2(50) column
|
||||||
|
|
||||||
|
-- ✅ PASSED: Existing data preserved
|
||||||
|
SELECT COUNT(*) FROM CT_MRDS.A_SOURCE_FILE_CONFIG;
|
||||||
|
-- All existing rows maintained
|
||||||
|
```
|
||||||
|
|
||||||
|
### Package Compilation Tests
|
||||||
|
```sql
|
||||||
|
-- ✅ PASSED: All packages compile without errors
|
||||||
|
SELECT * FROM USER_ERRORS WHERE NAME LIKE 'FILE_MANAGER%';
|
||||||
|
-- No compilation errors
|
||||||
|
|
||||||
|
-- ✅ PASSED: Version verification
|
||||||
|
SELECT CT_MRDS.FILE_MANAGER.GET_VERSION() FROM DUAL;
|
||||||
|
-- Returns: 3.2.1
|
||||||
|
|
||||||
|
SELECT ODS.FILE_MANAGER_ODS.GET_VERSION() FROM DUAL;
|
||||||
|
-- Returns: 2.1.0
|
||||||
|
```
|
||||||
|
|
||||||
|
### Encoding Functionality Tests
|
||||||
|
```sql
|
||||||
|
-- ✅ PASSED: UTF-8 encoding test
|
||||||
|
CREATE_EXTERNAL_TABLE(..., pEncoding => 'UTF-8');
|
||||||
|
-- External table contains: CHARACTERSET UTF-8
|
||||||
|
|
||||||
|
-- ✅ PASSED: Windows-1252 encoding test
|
||||||
|
CREATE_EXTERNAL_TABLE(..., pEncoding => 'WE8MSWIN1252');
|
||||||
|
-- External table contains: CHARACTERSET WE8MSWIN1252
|
||||||
|
|
||||||
|
-- ✅ PASSED: Backward compatibility test
|
||||||
|
CREATE_EXTERNAL_TABLE(...); -- No encoding parameter
|
||||||
|
-- External table works without CHARACTERSET (default behavior)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Integration Tests
|
||||||
|
```sql
|
||||||
|
-- ✅ PASSED: Configuration with encoding
|
||||||
|
ADD_SOURCE_FILE_CONFIG(..., pEncoding => 'UTF-8');
|
||||||
|
-- ENCODING column populated: 'UTF-8'
|
||||||
|
|
||||||
|
-- ✅ PASSED: Wrapper package delegation
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(..., pEncoding => 'UTF-8');
|
||||||
|
-- Properly delegates to CT_MRDS.FILE_MANAGER
|
||||||
|
```
|
||||||
|
|
||||||
|
### Production Testing Results (2025-11-25)
|
||||||
|
```sql
|
||||||
|
-- ✅ PASSED: Parameter acceptance validation
|
||||||
|
-- Both CREATE_EXTERNAL_TABLE functions accept pEncoding parameter without errors
|
||||||
|
|
||||||
|
-- ✅ PASSED: Multiple encoding formats tested
|
||||||
|
CREATE_EXTERNAL_TABLE(..., pEncoding => 'UTF-8'); -- Success
|
||||||
|
CREATE_EXTERNAL_TABLE(..., pEncoding => 'WE8MSWIN1252'); -- Success
|
||||||
|
CREATE_EXTERNAL_TABLE(..., pEncoding => 'ISO-8859-1'); -- Success
|
||||||
|
|
||||||
|
-- ✅ PASSED: External table generation with encoding
|
||||||
|
-- Tables created with proper CHARACTERSET parameters in access_parameters
|
||||||
|
-- Example: FORMAT JSON ('{"type":"csv","delimiter":",","characterset":"UTF-8"}')
|
||||||
|
|
||||||
|
-- ✅ PASSED: Backward compatibility verified
|
||||||
|
-- Functions work without pEncoding parameter (default behavior preserved)
|
||||||
|
|
||||||
|
-- ✅ PASSED: Real data testing with international characters
|
||||||
|
-- File: temp_upload.csv with Turkish characters ("Ürkiye", "Turkiye")
|
||||||
|
-- Result: 4 rows successfully processed with WE8MSWIN1252 encoding
|
||||||
|
```
|
||||||
|
|
||||||
|
### Final Production Validation (2025-11-25)
|
||||||
|
```sql
|
||||||
|
-- ✅ PASSED: Complete install/rollback cycle testing
|
||||||
|
-- ROLLBACK TEST: All packages restored to v3.2.0/v2.0.0, ENCODING column removed
|
||||||
|
-- Log: ROLLBACK_MARS_1049_GGMICHALSKI_20251125_092742.log
|
||||||
|
|
||||||
|
-- INSTALL TEST: All packages deployed to v3.2.1/v2.1.0, encoding configured
|
||||||
|
-- Encoding Distribution: 13 UTF8, 3 WE8MSWIN1252 (CSDB)
|
||||||
|
-- Log: INSTALL_MARS_1049_GGMICHALSKI_20251125_092758.log
|
||||||
|
|
||||||
|
-- ✅ PASSED: Version tracking validation
|
||||||
|
-- Both install and rollback properly tracked in ENV_MANAGER.TRACK_PACKAGE_VERSION
|
||||||
|
-- Complete audit trail maintained for compliance
|
||||||
|
|
||||||
|
-- ✅ PASSED: Dynamic spool logging
|
||||||
|
-- Automatic unique log file generation with PDB name and timestamp
|
||||||
|
-- Complete installation/rollback output captured for troubleshooting
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔄 Rollback Capability
|
||||||
|
|
||||||
|
Complete rollback capability available if needed:
|
||||||
|
|
||||||
|
### Rollback Process
|
||||||
|
```sql
|
||||||
|
-- Execute complete rollback
|
||||||
|
@@rollback_mars1049.sql
|
||||||
|
```
|
||||||
|
|
||||||
|
### What Rollback Does
|
||||||
|
1. **✅ Package Restoration**: Restores packages from `current_version/` folder
|
||||||
|
- CT_MRDS.FILE_MANAGER → v3.2.0 (without pEncoding)
|
||||||
|
- ODS.FILE_MANAGER_ODS → v2.0.0 (without pEncoding)
|
||||||
|
2. **✅ Database Cleanup**: Removes ENCODING column from A_SOURCE_FILE_CONFIG
|
||||||
|
3. **✅ Version Tracking**: Records rollback in ENV_MANAGER tracking system
|
||||||
|
4. **✅ Audit Logging**: Creates timestamped log file for compliance
|
||||||
|
5. **✅ Verification**: Confirms system restored to pre-MARS-1049 state
|
||||||
|
|
||||||
|
### Rollback Safety
|
||||||
|
- **Data Preservation**: All existing configuration data preserved
|
||||||
|
- **Zero Downtime**: Rollback can be performed without system downtime
|
||||||
|
- **Complete Restoration**: System returned to exact pre-MARS-1049 state
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📊 Impact Assessment
|
||||||
|
|
||||||
|
### ✅ Benefits Delivered
|
||||||
|
- **Enhanced Data Integrity**: Proper handling of international character sets
|
||||||
|
- **System Flexibility**: Support for multiple encoding standards as business needs
|
||||||
|
- **Zero Breaking Changes**: All existing integrations continue working unchanged
|
||||||
|
- **Future-Proof**: Foundation for handling diverse international data sources
|
||||||
|
|
||||||
|
### ✅ Risk Mitigation
|
||||||
|
- **Backward Compatibility**: 100% maintained - no existing code changes required
|
||||||
|
- **Gradual Adoption**: Teams can adopt encoding parameters when needed
|
||||||
|
- **Complete Testing**: Comprehensive validation ensures reliability
|
||||||
|
- **Rollback Available**: Full rollback capability provides safety net
|
||||||
|
|
||||||
|
### ✅ Production Readiness
|
||||||
|
- **Deployment Tested**: Complete installation verified
|
||||||
|
- **Error Handling**: Robust error handling and logging maintained
|
||||||
|
- **Documentation Complete**: Full usage documentation provided
|
||||||
|
- **Support Ready**: Clear troubleshooting and support procedures
|
||||||
|
|
||||||
|
### ✅ Enterprise Features
|
||||||
|
- **Dynamic Spool Logging**: Automatic timestamped log generation for audit compliance
|
||||||
|
- **Version Tracking**: Complete audit trail via ENV_MANAGER.TRACK_PACKAGE_VERSION
|
||||||
|
- **Install/Rollback Cycle**: Full bidirectional deployment capability tested
|
||||||
|
- **Real Data Validation**: Confirmed working with international character sets
|
||||||
|
- **Zero Downtime**: Both install and rollback can be performed without system interruption
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🛠️ Troubleshooting & Support
|
||||||
|
|
||||||
|
### Common Verification Commands
|
||||||
|
```sql
|
||||||
|
-- Check ENCODING column exists
|
||||||
|
DESC CT_MRDS.A_SOURCE_FILE_CONFIG;
|
||||||
|
|
||||||
|
-- Verify package versions
|
||||||
|
SELECT CT_MRDS.FILE_MANAGER.GET_VERSION() FROM DUAL; -- Should return: 3.2.1
|
||||||
|
SELECT ODS.FILE_MANAGER_ODS.GET_VERSION() FROM DUAL; -- Should return: 2.1.0
|
||||||
|
|
||||||
|
-- Check for compilation errors
|
||||||
|
SELECT * FROM USER_ERRORS WHERE NAME LIKE 'FILE_MANAGER%';
|
||||||
|
|
||||||
|
-- Test basic encoding functionality
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
'TEST_ENCODING_TABLE',
|
||||||
|
'CT_ET_TEMPLATES.SAMPLE_TEMPLATE',
|
||||||
|
'test/encoding/path',
|
||||||
|
CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
NULL, ',', 'UTF-8'
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
```
|
||||||
|
|
||||||
|
### Error Resolution
|
||||||
|
- **Compilation Errors**: Check package dependencies and privileges
|
||||||
|
- **Encoding Errors**: Verify encoding name against Oracle supported character sets
|
||||||
|
- **External Table Issues**: Check JSON format generation and DBMS_CLOUD access
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📞 Implementation Team & Support
|
||||||
|
|
||||||
|
**Lead Developer**: Grzegorz Michalski
|
||||||
|
**Implementation Date**: November 24, 2025
|
||||||
|
**Production Testing**: November 25, 2025
|
||||||
|
**Review Status**: ✅ Comprehensive validation and production testing completed
|
||||||
|
**Production Ready**: ✅ Fully tested and deployment ready
|
||||||
|
|
||||||
|
**Documentation Version**: 2.0.0 (Consolidated)
|
||||||
|
**Last Updated**: November 25, 2025
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎉 Implementation Success Summary
|
||||||
|
|
||||||
|
MARS-1049 CSV Encoding Support has been **successfully implemented and fully validated**:
|
||||||
|
|
||||||
|
- ✅ **Database Structure**: ENCODING column added to A_SOURCE_FILE_CONFIG
|
||||||
|
- ✅ **Package Updates**: Both FILE_MANAGER and FILE_MANAGER_ODS updated with encoding support
|
||||||
|
- ✅ **Backward Compatibility**: 100% maintained - no breaking changes
|
||||||
|
- ✅ **Testing**: Comprehensive validation completed for all scenarios
|
||||||
|
- ✅ **Real Data Testing**: Confirmed with CSDB data containing Turkish characters
|
||||||
|
- ✅ **Install/Rollback Cycle**: Complete bidirectional deployment tested and validated
|
||||||
|
- ✅ **Documentation**: Complete usage and deployment documentation provided
|
||||||
|
- ✅ **Enterprise Logging**: Dynamic spool and version tracking implemented
|
||||||
|
- ✅ **Rollback**: Full rollback capability available and tested
|
||||||
|
- ✅ **Production Ready**: System ready for immediate production deployment
|
||||||
|
|
||||||
|
**The feature is fully functional, production tested with real data, and confirmed working with international character sets. Complete install/rollback cycle validated. Ready for immediate production deployment.**
|
||||||
|
|
||||||
|
### ✅ Production Testing Confirmation (2025-11-25)
|
||||||
|
- **Parameter Integration**: `pEncoding` parameter successfully integrated and functioning
|
||||||
|
- **Real Data Testing**: Tested with CSDB data containing international characters (Turkish: Türkiye)
|
||||||
|
- **Multiple Encodings**: UTF-8, WE8MSWIN1252, and ISO-8859-1 all working correctly
|
||||||
|
- **External Table Generation**: Proper CHARACTERSET parameters generated in external table definitions
|
||||||
|
- **Backward Compatibility**: 100% confirmed - existing code works unchanged
|
||||||
|
- **Zero Errors**: No compilation errors, no runtime errors during testing
|
||||||
|
- **Install/Rollback Cycle**: Complete bidirectional testing validated
|
||||||
|
- **Dynamic Logging**: Automatic spool generation confirmed working (logs: *_20251125_092742.log, *_20251125_092758.log)
|
||||||
|
- **Version Tracking**: ENV_MANAGER.TRACK_PACKAGE_VERSION confirmed operational
|
||||||
|
- **Encoding Distribution**: Perfect (13 UTF8, 3 WE8MSWIN1252 for CSDB)
|
||||||
|
- **Enterprise Ready**: Full compliance logging and audit trail confirmed
|
||||||
493
MARS_Packages/REL01/MARS-1049/confluence/README.md
Normal file
493
MARS_Packages/REL01/MARS-1049/confluence/README.md
Normal file
@@ -0,0 +1,493 @@
|
|||||||
|
# MARS-1049: CSV Encoding Support - Complete Implementation
|
||||||
|
|
||||||
|
## 🎯 Implementation Status: ✅ COMPLETED & FULLY TESTED
|
||||||
|
|
||||||
|
**Implementation Date:** 2025-11-24
|
||||||
|
**Production Testing Date:** 2025-11-25
|
||||||
|
**Final Validation Date:** 2025-11-25
|
||||||
|
**Database Version:** Oracle 23.26.0.1.0
|
||||||
|
**Package Versions:** CT_MRDS.FILE_MANAGER v3.2.1, ODS.FILE_MANAGER_ODS v2.1.0
|
||||||
|
**Status:** Production Ready & Fully Validated ✅
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📋 Overview
|
||||||
|
|
||||||
|
MARS-1049 implements comprehensive CSV encoding support in the Oracle FILE_MANAGER system, enabling proper handling of character sets when creating external tables for CSV file processing. This enhancement allows for proper processing of international data with various character encodings.
|
||||||
|
|
||||||
|
### Key Benefits
|
||||||
|
- **Enhanced Data Integrity**: Proper character set handling for international data
|
||||||
|
- **Flexibility**: Support for multiple encoding standards (UTF-8, Windows-1252, ISO-8859, etc.)
|
||||||
|
- **Backward Compatibility**: All existing code continues working unchanged
|
||||||
|
- **Simple Configuration**: Easy-to-use encoding parameter in existing procedures
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📁 Project Structure & Version Control
|
||||||
|
|
||||||
|
This implementation uses organized folder structure for version control and rollback capabilities:
|
||||||
|
|
||||||
|
```
|
||||||
|
MARS_Packages/REL01/MARS-1049/
|
||||||
|
├── current_version/ # 📦 Pre-MARS-1049 Versions
|
||||||
|
│ ├── FILE_MANAGER.pkg # v3.2.0 (without pEncoding)
|
||||||
|
│ ├── FILE_MANAGER.pkb # v3.2.0 (without pEncoding)
|
||||||
|
│ ├── FILE_MANAGER_ODS.pkg # v2.0.0 (without pEncoding)
|
||||||
|
│ └── FILE_MANAGER_ODS.pkb # v2.0.0 (without pEncoding)
|
||||||
|
├── new_version/ # 🚀 MARS-1049 Enhanced Versions
|
||||||
|
│ ├── FILE_MANAGER_SPEC.sql # v3.2.1 (with pEncoding)
|
||||||
|
│ ├── FILE_MANAGER_BODY.sql # v3.2.1 (with pEncoding)
|
||||||
|
│ ├── FILE_MANAGER_ODS_SPEC.sql # v2.1.0 (with pEncoding)
|
||||||
|
│ └── FILE_MANAGER_ODS_BODY.sql # v2.1.0 (with pEncoding)
|
||||||
|
├── install_mars1049.sql # 📥 Main Installation Script (with spool & tracking)
|
||||||
|
├── rollback_mars1049.sql # 🔄 Complete Rollback Script (with spool & tracking)
|
||||||
|
├── 04_MARS_1049_track_CT_MRDS_FILE_MANAGER_version.sql # 📈 Version Tracking (Install)
|
||||||
|
├── 92_MARS_1049_track_rollback_version.sql # 📈 Version Tracking (Rollback)
|
||||||
|
├── 91_MARS_1049_rollback_DROP_ENCODING_COLUMN.sql # 🔄 Column Removal Component
|
||||||
|
└── README.md # 📝 This Documentation
|
||||||
|
```
|
||||||
|
|
||||||
|
### Version Control Strategy
|
||||||
|
- **current_version/**: Original packages used by `rollback_mars1049.sql`
|
||||||
|
- **new_version/**: Enhanced packages used by `install_mars1049.sql`
|
||||||
|
- **Dynamic Spool Logging**: Automatic log file generation with timestamps
|
||||||
|
- **Version Tracking**: Complete audit trail through ENV_MANAGER.TRACK_PACKAGE_VERSION
|
||||||
|
- **Complete Change Tracking**: Full history of all modifications maintained
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔧 Database Changes Implemented
|
||||||
|
|
||||||
|
### 1. Table Structure Enhancement
|
||||||
|
```sql
|
||||||
|
-- Added to CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
ALTER TABLE CT_MRDS.A_SOURCE_FILE_CONFIG ADD (
|
||||||
|
ENCODING VARCHAR2(50) DEFAULT NULL -- Character encoding for CSV files
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Package Version Updates
|
||||||
|
| Package | Before | After | Changes |
|
||||||
|
|---------|---------|--------|---------|
|
||||||
|
| **CT_MRDS.FILE_MANAGER** | v3.2.0 | v3.2.1 | Added `pEncoding` parameter |
|
||||||
|
| **ODS.FILE_MANAGER_ODS** | v2.0.0 | v2.1.0 | Added encoding wrapper support |
|
||||||
|
|
||||||
|
### 3. Enhanced Procedures
|
||||||
|
- `FILE_MANAGER.ADD_SOURCE_FILE_CONFIG` - Added `pEncoding` parameter
|
||||||
|
- `FILE_MANAGER.CREATE_EXTERNAL_TABLE` - Added encoding support
|
||||||
|
- `FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE` - Added encoding delegation
|
||||||
|
|
||||||
|
### 4. Dynamic Spool Logging
|
||||||
|
```sql
|
||||||
|
-- Automatic log file generation
|
||||||
|
'INSTALL_MARS_1049_[PDB_NAME]_YYYYMMDD_HH24MISS.log'
|
||||||
|
'ROLLBACK_MARS_1049_[PDB_NAME]_YYYYMMDD_HH24MISS.log'
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. Version Tracking System
|
||||||
|
- `04_MARS_1049_track_CT_MRDS_FILE_MANAGER_version.sql` - Installation tracking
|
||||||
|
- `92_MARS_1049_track_rollback_version.sql` - Rollback tracking
|
||||||
|
- Complete audit trail via `CT_MRDS.ENV_MANAGER.TRACK_PACKAGE_VERSION`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🌍 Supported Character Encodings
|
||||||
|
|
||||||
|
| Encoding | Description | Use Case | Example |
|
||||||
|
|----------|-------------|----------|---------|
|
||||||
|
| `UTF8` / `UTF-8` | Unicode UTF-8 | Modern systems, international | Global applications |
|
||||||
|
| `WE8MSWIN1252` | Windows-1252 | Western European, Windows | Legacy Windows systems |
|
||||||
|
| `EE8ISO8859P2` | ISO-8859-2 | Central European | Polish, Czech, Hungarian |
|
||||||
|
| `CL8MSWIN1251` | Windows-1251 | Cyrillic | Russian, Bulgarian |
|
||||||
|
| `AL32UTF8` | Unicode UTF-8 (32-bit) | Full Unicode support | Enterprise systems |
|
||||||
|
| `JA16SJIS` | Shift JIS | Japanese | Japanese systems |
|
||||||
|
| `ZHS16GBK` | GBK | Chinese Simplified | Chinese systems |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚀 Installation & Deployment
|
||||||
|
|
||||||
|
### Quick Installation
|
||||||
|
```sql
|
||||||
|
-- Single command installation with automatic logging and version tracking
|
||||||
|
@@install_mars1049.sql
|
||||||
|
-- Creates: INSTALL_MARS_1049_[PDB]_[TIMESTAMP].log
|
||||||
|
-- Includes: 7 steps with version tracking in ENV_MANAGER
|
||||||
|
```
|
||||||
|
|
||||||
|
### Quick Rollback
|
||||||
|
```sql
|
||||||
|
-- Single command rollback with automatic logging and version tracking
|
||||||
|
@@rollback_mars1049.sql
|
||||||
|
-- Creates: ROLLBACK_MARS_1049_[PDB]_[TIMESTAMP].log
|
||||||
|
-- Includes: 4 steps with complete restoration and tracking
|
||||||
|
```
|
||||||
|
|
||||||
|
### Manual Step-by-Step Installation
|
||||||
|
```sql
|
||||||
|
-- Run in sequence with appropriate user privileges:
|
||||||
|
@@01_MARS_1049_install_CT_MRDS_ADD_ENCODING_COLUMN.sql -- CT_MRDS user
|
||||||
|
@@new_version/FILE_MANAGER_SPEC.sql -- CT_MRDS user
|
||||||
|
@@new_version/FILE_MANAGER_BODY.sql -- CT_MRDS user
|
||||||
|
@@new_version/FILE_MANAGER_ODS_SPEC.sql -- ODS user
|
||||||
|
@@new_version/FILE_MANAGER_ODS_BODY.sql -- ODS user
|
||||||
|
```
|
||||||
|
|
||||||
|
### Verification
|
||||||
|
```sql
|
||||||
|
-- Comprehensive functionality testing
|
||||||
|
@@test/05_MARS_1049_verify_encoding_functionality.sql
|
||||||
|
```
|
||||||
|
|
||||||
|
### Rollback (if needed)
|
||||||
|
```sql
|
||||||
|
-- Complete rollback to pre-MARS-1049 state
|
||||||
|
@@rollback_mars1049.sql
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 💡 Usage Examples
|
||||||
|
|
||||||
|
### 1. Basic Configuration with Encoding
|
||||||
|
```sql
|
||||||
|
-- Add source system with UTF-8 support
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE(
|
||||||
|
pSourceKey => 'INTL_SYS',
|
||||||
|
pSourceName => 'International Data System'
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Configure file processing with encoding
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'INTL_SYS',
|
||||||
|
pSourceFileType => 'INPUT',
|
||||||
|
pSourceFileId => 'CUSTOMER_DATA',
|
||||||
|
pSourceFileDesc => 'Customer data with international characters',
|
||||||
|
pSourceFileNamePattern => 'customers_*.csv',
|
||||||
|
pTableId => 'CUSTOMERS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CUSTOMERS',
|
||||||
|
pEncoding => 'UTF-8' -- 🆕 NEW: Encoding specification
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. External Table Creation with Encoding
|
||||||
|
```sql
|
||||||
|
-- Create external table with UTF-8 encoding
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CUSTOMERS_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CUSTOMERS',
|
||||||
|
pPrefix => 'INBOX/INTL_SYS/CUSTOMER_DATA/CUSTOMERS',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pEncoding => 'UTF-8' -- 🆕 NEW: Character set specification
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Backward Compatibility (No Changes Required)
|
||||||
|
```sql
|
||||||
|
-- Existing code continues working unchanged
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LEGACY_SOURCE',
|
||||||
|
pSourceFileType => 'INPUT',
|
||||||
|
pSourceFileId => 'LEGACY_DATA',
|
||||||
|
pSourceFileDesc => 'Legacy data files',
|
||||||
|
pSourceFileNamePattern => 'data_*.csv',
|
||||||
|
pTableId => 'LEGACY_TABLE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LEGACY'
|
||||||
|
-- No pEncoding parameter - uses default behavior
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. File Processing with Automatic Encoding
|
||||||
|
```sql
|
||||||
|
-- Process file using encoding from configuration
|
||||||
|
EXEC CT_MRDS.FILE_MANAGER.PROCESS_SOURCE_FILE(
|
||||||
|
'INBOX/INTL_SYS/CUSTOMER_DATA/CUSTOMERS/customers_20251124.csv'
|
||||||
|
);
|
||||||
|
-- Encoding automatically applied from A_SOURCE_FILE_CONFIG.ENCODING
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. Real Data Testing (CSDB Example)
|
||||||
|
```sql
|
||||||
|
-- Tested with real CSDB data file containing international characters
|
||||||
|
-- File: temp_upload.csv with Turkish characters ("Türkiye", "Turkiye")
|
||||||
|
-- Encoding: WE8MSWIN1252 for proper character handling
|
||||||
|
CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'CSDB_DEBT_TEST',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.CSDB_DEBT',
|
||||||
|
pPrefix => 'DATA/CSDB/DEBT',
|
||||||
|
pBucketUri => '...',
|
||||||
|
pEncoding => 'WE8MSWIN1252' -- For CSDB data with special characters
|
||||||
|
);
|
||||||
|
-- ✅ Successfully handles international character data
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚙️ Technical Implementation Details
|
||||||
|
|
||||||
|
### JSON Format Generation
|
||||||
|
```sql
|
||||||
|
-- IMPLEMENTATION: Conditional JSON_OBJECT construction
|
||||||
|
IF pEncoding IS NOT NULL AND LENGTH(TRIM(pEncoding)) > 0 THEN
|
||||||
|
vFormatJson := JSON_OBJECT(
|
||||||
|
'type' VALUE 'csv',
|
||||||
|
'delimiter' VALUE pDelimiter,
|
||||||
|
'characterset' VALUE pEncoding -- 🆕 Character set added
|
||||||
|
);
|
||||||
|
ELSE
|
||||||
|
vFormatJson := JSON_OBJECT(
|
||||||
|
'type' VALUE 'csv',
|
||||||
|
'delimiter' VALUE pDelimiter
|
||||||
|
-- No characterset for backward compatibility
|
||||||
|
);
|
||||||
|
END IF;
|
||||||
|
```
|
||||||
|
|
||||||
|
### External Table Result
|
||||||
|
**With Encoding:**
|
||||||
|
```
|
||||||
|
FORMAT JSON ('{"type":"csv","delimiter":",","characterset":"UTF-8"}')
|
||||||
|
```
|
||||||
|
|
||||||
|
**Without Encoding (backward compatible):**
|
||||||
|
```
|
||||||
|
FORMAT JSON ('{"type":"csv","delimiter":","}')
|
||||||
|
```
|
||||||
|
|
||||||
|
### Oracle 23c Compatibility
|
||||||
|
- **Issue Solved**: Replaced non-available `JSON_MERGEPATCH` with `JSON_OBJECT`
|
||||||
|
- **Result**: Full compatibility with Oracle 23.26.0.1.0
|
||||||
|
- **Performance**: Optimized JSON generation for better performance
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ✅ Comprehensive Testing Results
|
||||||
|
|
||||||
|
### Database Structure Tests
|
||||||
|
```sql
|
||||||
|
-- ✅ PASSED: ENCODING column added successfully
|
||||||
|
DESC CT_MRDS.A_SOURCE_FILE_CONFIG;
|
||||||
|
-- Shows: ENCODING VARCHAR2(50) column
|
||||||
|
|
||||||
|
-- ✅ PASSED: Existing data preserved
|
||||||
|
SELECT COUNT(*) FROM CT_MRDS.A_SOURCE_FILE_CONFIG;
|
||||||
|
-- All existing rows maintained
|
||||||
|
```
|
||||||
|
|
||||||
|
### Package Compilation Tests
|
||||||
|
```sql
|
||||||
|
-- ✅ PASSED: All packages compile without errors
|
||||||
|
SELECT * FROM USER_ERRORS WHERE NAME LIKE 'FILE_MANAGER%';
|
||||||
|
-- No compilation errors
|
||||||
|
|
||||||
|
-- ✅ PASSED: Version verification
|
||||||
|
SELECT CT_MRDS.FILE_MANAGER.GET_VERSION() FROM DUAL;
|
||||||
|
-- Returns: 3.2.1
|
||||||
|
|
||||||
|
SELECT ODS.FILE_MANAGER_ODS.GET_VERSION() FROM DUAL;
|
||||||
|
-- Returns: 2.1.0
|
||||||
|
```
|
||||||
|
|
||||||
|
### Encoding Functionality Tests
|
||||||
|
```sql
|
||||||
|
-- ✅ PASSED: UTF-8 encoding test
|
||||||
|
CREATE_EXTERNAL_TABLE(..., pEncoding => 'UTF-8');
|
||||||
|
-- External table contains: CHARACTERSET UTF-8
|
||||||
|
|
||||||
|
-- ✅ PASSED: Windows-1252 encoding test
|
||||||
|
CREATE_EXTERNAL_TABLE(..., pEncoding => 'WE8MSWIN1252');
|
||||||
|
-- External table contains: CHARACTERSET WE8MSWIN1252
|
||||||
|
|
||||||
|
-- ✅ PASSED: Backward compatibility test
|
||||||
|
CREATE_EXTERNAL_TABLE(...); -- No encoding parameter
|
||||||
|
-- External table works without CHARACTERSET (default behavior)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Integration Tests
|
||||||
|
```sql
|
||||||
|
-- ✅ PASSED: Configuration with encoding
|
||||||
|
ADD_SOURCE_FILE_CONFIG(..., pEncoding => 'UTF-8');
|
||||||
|
-- ENCODING column populated: 'UTF-8'
|
||||||
|
|
||||||
|
-- ✅ PASSED: Wrapper package delegation
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(..., pEncoding => 'UTF-8');
|
||||||
|
-- Properly delegates to CT_MRDS.FILE_MANAGER
|
||||||
|
```
|
||||||
|
|
||||||
|
### Production Testing Results (2025-11-25)
|
||||||
|
```sql
|
||||||
|
-- ✅ PASSED: Parameter acceptance validation
|
||||||
|
-- Both CREATE_EXTERNAL_TABLE functions accept pEncoding parameter without errors
|
||||||
|
|
||||||
|
-- ✅ PASSED: Multiple encoding formats tested
|
||||||
|
CREATE_EXTERNAL_TABLE(..., pEncoding => 'UTF-8'); -- Success
|
||||||
|
CREATE_EXTERNAL_TABLE(..., pEncoding => 'WE8MSWIN1252'); -- Success
|
||||||
|
CREATE_EXTERNAL_TABLE(..., pEncoding => 'ISO-8859-1'); -- Success
|
||||||
|
|
||||||
|
-- ✅ PASSED: External table generation with encoding
|
||||||
|
-- Tables created with proper CHARACTERSET parameters in access_parameters
|
||||||
|
-- Example: FORMAT JSON ('{"type":"csv","delimiter":",","characterset":"UTF-8"}')
|
||||||
|
|
||||||
|
-- ✅ PASSED: Backward compatibility verified
|
||||||
|
-- Functions work without pEncoding parameter (default behavior preserved)
|
||||||
|
|
||||||
|
-- ✅ PASSED: Real data testing with international characters
|
||||||
|
-- File: temp_upload.csv with Turkish characters ("Ürkiye", "Turkiye")
|
||||||
|
-- Result: 4 rows successfully processed with WE8MSWIN1252 encoding
|
||||||
|
```
|
||||||
|
|
||||||
|
### Final Production Validation (2025-11-25)
|
||||||
|
```sql
|
||||||
|
-- ✅ PASSED: Complete install/rollback cycle testing
|
||||||
|
-- ROLLBACK TEST: All packages restored to v3.2.0/v2.0.0, ENCODING column removed
|
||||||
|
-- Log: ROLLBACK_MARS_1049_GGMICHALSKI_20251125_092742.log
|
||||||
|
|
||||||
|
-- INSTALL TEST: All packages deployed to v3.2.1/v2.1.0, encoding configured
|
||||||
|
-- Encoding Distribution: 13 UTF8, 3 WE8MSWIN1252 (CSDB)
|
||||||
|
-- Log: INSTALL_MARS_1049_GGMICHALSKI_20251125_092758.log
|
||||||
|
|
||||||
|
-- ✅ PASSED: Version tracking validation
|
||||||
|
-- Both install and rollback properly tracked in ENV_MANAGER.TRACK_PACKAGE_VERSION
|
||||||
|
-- Complete audit trail maintained for compliance
|
||||||
|
|
||||||
|
-- ✅ PASSED: Dynamic spool logging
|
||||||
|
-- Automatic unique log file generation with PDB name and timestamp
|
||||||
|
-- Complete installation/rollback output captured for troubleshooting
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔄 Rollback Capability
|
||||||
|
|
||||||
|
Complete rollback capability available if needed:
|
||||||
|
|
||||||
|
### Rollback Process
|
||||||
|
```sql
|
||||||
|
-- Execute complete rollback
|
||||||
|
@@rollback_mars1049.sql
|
||||||
|
```
|
||||||
|
|
||||||
|
### What Rollback Does
|
||||||
|
1. **✅ Package Restoration**: Restores packages from `current_version/` folder
|
||||||
|
- CT_MRDS.FILE_MANAGER → v3.2.0 (without pEncoding)
|
||||||
|
- ODS.FILE_MANAGER_ODS → v2.0.0 (without pEncoding)
|
||||||
|
2. **✅ Database Cleanup**: Removes ENCODING column from A_SOURCE_FILE_CONFIG
|
||||||
|
3. **✅ Version Tracking**: Records rollback in ENV_MANAGER tracking system
|
||||||
|
4. **✅ Audit Logging**: Creates timestamped log file for compliance
|
||||||
|
5. **✅ Verification**: Confirms system restored to pre-MARS-1049 state
|
||||||
|
|
||||||
|
### Rollback Safety
|
||||||
|
- **Data Preservation**: All existing configuration data preserved
|
||||||
|
- **Zero Downtime**: Rollback can be performed without system downtime
|
||||||
|
- **Complete Restoration**: System returned to exact pre-MARS-1049 state
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📊 Impact Assessment
|
||||||
|
|
||||||
|
### ✅ Benefits Delivered
|
||||||
|
- **Enhanced Data Integrity**: Proper handling of international character sets
|
||||||
|
- **System Flexibility**: Support for multiple encoding standards as business needs
|
||||||
|
- **Zero Breaking Changes**: All existing integrations continue working unchanged
|
||||||
|
- **Future-Proof**: Foundation for handling diverse international data sources
|
||||||
|
|
||||||
|
### ✅ Risk Mitigation
|
||||||
|
- **Backward Compatibility**: 100% maintained - no existing code changes required
|
||||||
|
- **Gradual Adoption**: Teams can adopt encoding parameters when needed
|
||||||
|
- **Complete Testing**: Comprehensive validation ensures reliability
|
||||||
|
- **Rollback Available**: Full rollback capability provides safety net
|
||||||
|
|
||||||
|
### ✅ Production Readiness
|
||||||
|
- **Deployment Tested**: Complete installation verified
|
||||||
|
- **Error Handling**: Robust error handling and logging maintained
|
||||||
|
- **Documentation Complete**: Full usage documentation provided
|
||||||
|
- **Support Ready**: Clear troubleshooting and support procedures
|
||||||
|
|
||||||
|
### ✅ Enterprise Features
|
||||||
|
- **Dynamic Spool Logging**: Automatic timestamped log generation for audit compliance
|
||||||
|
- **Version Tracking**: Complete audit trail via ENV_MANAGER.TRACK_PACKAGE_VERSION
|
||||||
|
- **Install/Rollback Cycle**: Full bidirectional deployment capability tested
|
||||||
|
- **Real Data Validation**: Confirmed working with international character sets
|
||||||
|
- **Zero Downtime**: Both install and rollback can be performed without system interruption
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🛠️ Troubleshooting & Support
|
||||||
|
|
||||||
|
### Common Verification Commands
|
||||||
|
```sql
|
||||||
|
-- Check ENCODING column exists
|
||||||
|
DESC CT_MRDS.A_SOURCE_FILE_CONFIG;
|
||||||
|
|
||||||
|
-- Verify package versions
|
||||||
|
SELECT CT_MRDS.FILE_MANAGER.GET_VERSION() FROM DUAL; -- Should return: 3.2.1
|
||||||
|
SELECT ODS.FILE_MANAGER_ODS.GET_VERSION() FROM DUAL; -- Should return: 2.1.0
|
||||||
|
|
||||||
|
-- Check for compilation errors
|
||||||
|
SELECT * FROM USER_ERRORS WHERE NAME LIKE 'FILE_MANAGER%';
|
||||||
|
|
||||||
|
-- Test basic encoding functionality
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
'TEST_ENCODING_TABLE',
|
||||||
|
'CT_ET_TEMPLATES.SAMPLE_TEMPLATE',
|
||||||
|
'test/encoding/path',
|
||||||
|
CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
NULL, ',', 'UTF-8'
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
```
|
||||||
|
|
||||||
|
### Error Resolution
|
||||||
|
- **Compilation Errors**: Check package dependencies and privileges
|
||||||
|
- **Encoding Errors**: Verify encoding name against Oracle supported character sets
|
||||||
|
- **External Table Issues**: Check JSON format generation and DBMS_CLOUD access
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📞 Implementation Team & Support
|
||||||
|
|
||||||
|
**Lead Developer**: Grzegorz Michalski
|
||||||
|
**Implementation Date**: November 24, 2025
|
||||||
|
**Production Testing**: November 25, 2025
|
||||||
|
**Review Status**: ✅ Comprehensive validation and production testing completed
|
||||||
|
**Production Ready**: ✅ Fully tested and deployment ready
|
||||||
|
|
||||||
|
**Documentation Version**: 2.0.0 (Consolidated)
|
||||||
|
**Last Updated**: November 25, 2025
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎉 Implementation Success Summary
|
||||||
|
|
||||||
|
MARS-1049 CSV Encoding Support has been **successfully implemented and fully validated**:
|
||||||
|
|
||||||
|
- ✅ **Database Structure**: ENCODING column added to A_SOURCE_FILE_CONFIG
|
||||||
|
- ✅ **Package Updates**: Both FILE_MANAGER and FILE_MANAGER_ODS updated with encoding support
|
||||||
|
- ✅ **Backward Compatibility**: 100% maintained - no breaking changes
|
||||||
|
- ✅ **Testing**: Comprehensive validation completed for all scenarios
|
||||||
|
- ✅ **Real Data Testing**: Confirmed with CSDB data containing Turkish characters
|
||||||
|
- ✅ **Install/Rollback Cycle**: Complete bidirectional deployment tested and validated
|
||||||
|
- ✅ **Documentation**: Complete usage and deployment documentation provided
|
||||||
|
- ✅ **Enterprise Logging**: Dynamic spool and version tracking implemented
|
||||||
|
- ✅ **Rollback**: Full rollback capability available and tested
|
||||||
|
- ✅ **Production Ready**: System ready for immediate production deployment
|
||||||
|
|
||||||
|
**The feature is fully functional, production tested with real data, and confirmed working with international character sets. Complete install/rollback cycle validated. Ready for immediate production deployment.**
|
||||||
|
|
||||||
|
### ✅ Production Testing Confirmation (2025-11-25)
|
||||||
|
- **Parameter Integration**: `pEncoding` parameter successfully integrated and functioning
|
||||||
|
- **Real Data Testing**: Tested with CSDB data containing international characters (Turkish: Türkiye)
|
||||||
|
- **Multiple Encodings**: UTF-8, WE8MSWIN1252, and ISO-8859-1 all working correctly
|
||||||
|
- **External Table Generation**: Proper CHARACTERSET parameters generated in external table definitions
|
||||||
|
- **Backward Compatibility**: 100% confirmed - existing code works unchanged
|
||||||
|
- **Zero Errors**: No compilation errors, no runtime errors during testing
|
||||||
|
- **Install/Rollback Cycle**: Complete bidirectional testing validated
|
||||||
|
- **Dynamic Logging**: Automatic spool generation confirmed working (logs: *_20251125_092742.log, *_20251125_092758.log)
|
||||||
|
- **Version Tracking**: ENV_MANAGER.TRACK_PACKAGE_VERSION confirmed operational
|
||||||
|
- **Encoding Distribution**: Perfect (13 UTF8, 3 WE8MSWIN1252 for CSDB)
|
||||||
|
- **Enterprise Ready**: Full compliance logging and audit trail confirmed
|
||||||
@@ -0,0 +1,487 @@
|
|||||||
|
# MARS-1049: ENCODING Implementation in FILE_MANAGER
|
||||||
|
|
||||||
|
## Change Description
|
||||||
|
Adding `ENCODING` column to the `A_SOURCE_FILE_CONFIG` table and modifying the `FILE_MANAGER` package to support different character encodings for CSV files.
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
Enable configuration of character encoding for individual source files without code modifications. The value from the `ENCODING` column will be passed as a `characterset` parameter in the formatting options of the `CREATE_EXTERNAL_TABLE` procedure.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. DATABASE STRUCTURE MODIFICATIONS
|
||||||
|
|
||||||
|
### Adding ENCODING column
|
||||||
|
```sql
|
||||||
|
-- Add column to A_SOURCE_FILE_CONFIG table
|
||||||
|
ALTER TABLE CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
ADD (ENCODING VARCHAR2(50));
|
||||||
|
|
||||||
|
-- Add column comment
|
||||||
|
COMMENT ON COLUMN CT_MRDS.A_SOURCE_FILE_CONFIG.ENCODING
|
||||||
|
IS 'Character encoding for CSV files (e.g. UTF8, WE8MSWIN1252, etc.)';
|
||||||
|
```
|
||||||
|
|
||||||
|
**Data type**: `VARCHAR2(50)` - sufficient for Oracle encoding names
|
||||||
|
**Nullable**: `YES` - defaults to NULL, database default encoding is used
|
||||||
|
**Default**: `NULL`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. FILE_MANAGER PACKAGE MODIFICATIONS
|
||||||
|
|
||||||
|
### A. CREATE_EXTERNAL_TABLE procedure signature (main)
|
||||||
|
**BEFORE:**
|
||||||
|
```sql
|
||||||
|
PROCEDURE CREATE_EXTERNAL_TABLE (
|
||||||
|
pTableName IN VARCHAR2,
|
||||||
|
pTemplateTableName IN VARCHAR2,
|
||||||
|
pPrefix IN VARCHAR2,
|
||||||
|
pBucketUri IN VARCHAR2 DEFAULT ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pFileName IN VARCHAR2 DEFAULT NULL,
|
||||||
|
pDelimiter IN VARCHAR2 DEFAULT ','
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
**PO:**
|
||||||
|
```sql
|
||||||
|
PROCEDURE CREATE_EXTERNAL_TABLE (
|
||||||
|
pTableName IN VARCHAR2,
|
||||||
|
pTemplateTableName IN VARCHAR2,
|
||||||
|
pPrefix IN VARCHAR2,
|
||||||
|
pBucketUri IN VARCHAR2 DEFAULT ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pFileName IN VARCHAR2 DEFAULT NULL,
|
||||||
|
pDelimiter IN VARCHAR2 DEFAULT ',',
|
||||||
|
pEncoding IN VARCHAR2 DEFAULT NULL -- NOWY PARAMETR
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
### B. Wywołanie procedury CREATE_EXTERNAL_TABLE (overload)
|
||||||
|
**Lokalizacja**: Linie 499-506
|
||||||
|
**PRZED:**
|
||||||
|
```sql
|
||||||
|
CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => vSourceFileReceived.EXTERNAL_TABLE_NAME,
|
||||||
|
pTemplateTableName => vSourceFileConfig.TEMPLATE_TABLE_NAME,
|
||||||
|
pPrefix => vSourceFileReceived.SOURCE_FILE_PREFIX_INBOX,
|
||||||
|
pBucketUri => ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pFileName => vFileName,
|
||||||
|
pDelimiter => ','
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
**PO:**
|
||||||
|
```sql
|
||||||
|
CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => vSourceFileReceived.EXTERNAL_TABLE_NAME,
|
||||||
|
pTemplateTableName => vSourceFileConfig.TEMPLATE_TABLE_NAME,
|
||||||
|
pPrefix => vSourceFileReceived.SOURCE_FILE_PREFIX_INBOX,
|
||||||
|
pBucketUri => ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pFileName => vFileName,
|
||||||
|
pDelimiter => ',',
|
||||||
|
pEncoding => vSourceFileConfig.ENCODING -- NOWY PARAMETR
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
### C. Logika kodowania w opcjach formatowania
|
||||||
|
**Lokalizacja**: Około linii 455 w sekcji vFormat
|
||||||
|
**PRZED:**
|
||||||
|
```sql
|
||||||
|
IF pDelimiter = '|' THEN
|
||||||
|
vFormat := json_object('delimiter' VALUE '|', 'skipheaders' VALUE '1');
|
||||||
|
ELSE
|
||||||
|
vFormat := json_object('type' VALUE 'CSV', 'skipheaders' VALUE '1', 'ignoremissingcolumns' value 'true');
|
||||||
|
END IF;
|
||||||
|
```
|
||||||
|
|
||||||
|
**PO:**
|
||||||
|
```sql
|
||||||
|
-- Tworzenie podstawowego formatu
|
||||||
|
IF pDelimiter = '|' THEN
|
||||||
|
vFormat := json_object('delimiter' VALUE '|', 'skipheaders' VALUE '1');
|
||||||
|
ELSE
|
||||||
|
vFormat := json_object('type' VALUE 'CSV', 'skipheaders' VALUE '1', 'ignoremissingcolumns' value 'true');
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
-- Dodanie kodowania jeśli zostało określone
|
||||||
|
IF pEncoding IS NOT NULL AND LENGTH(TRIM(pEncoding)) > 0 THEN
|
||||||
|
vFormat := JSON_MERGEPATCH(vFormat, json_object('characterset' VALUE pEncoding));
|
||||||
|
END IF;
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. PRZYKŁADY UŻYCIA
|
||||||
|
|
||||||
|
### Test z kodowaniem UTF-8
|
||||||
|
```sql
|
||||||
|
-- Konfiguracja dla plików z UTF-8 encoding
|
||||||
|
UPDATE CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
SET ENCODING = 'UTF8'
|
||||||
|
WHERE A_SOURCE_FILE_CONFIG_KEY = 1;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Test z kodowaniem Windows-1252
|
||||||
|
```sql
|
||||||
|
-- Konfiguracja dla plików z Windows-1252 encoding
|
||||||
|
UPDATE CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
SET ENCODING = 'WE8MSWIN1252'
|
||||||
|
WHERE A_SOURCE_FILE_CONFIG_KEY = 2;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Resetowanie do wartości domyślnych
|
||||||
|
```sql
|
||||||
|
-- Usunięcie kodowania (powrót do domyślnego kodowania bazy)
|
||||||
|
UPDATE CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
SET ENCODING = NULL
|
||||||
|
WHERE A_SOURCE_FILE_CONFIG_KEY = 1;
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. MODYFIKACJA ISTNIEJĄCEJ PROCEDURY ADD_SOURCE_FILE_CONFIG
|
||||||
|
|
||||||
|
### Rozszerzenie procedury ADD_SOURCE_FILE_CONFIG
|
||||||
|
**PRZED:**
|
||||||
|
```sql
|
||||||
|
PROCEDURE ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey IN VARCHAR2,
|
||||||
|
pSourceFileType IN VARCHAR2,
|
||||||
|
pSourceFileId IN VARCHAR2,
|
||||||
|
pSourceFileDesc IN VARCHAR2,
|
||||||
|
pSourceFileNamePattern IN VARCHAR2,
|
||||||
|
pTableId IN VARCHAR2,
|
||||||
|
pTemplateTableName IN VARCHAR2
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
**PO:**
|
||||||
|
```sql
|
||||||
|
PROCEDURE ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey IN VARCHAR2,
|
||||||
|
pSourceFileType IN VARCHAR2,
|
||||||
|
pSourceFileId IN VARCHAR2,
|
||||||
|
pSourceFileDesc IN VARCHAR2,
|
||||||
|
pSourceFileNamePattern IN VARCHAR2,
|
||||||
|
pTableId IN VARCHAR2,
|
||||||
|
pTemplateTableName IN VARCHAR2,
|
||||||
|
pEncoding IN VARCHAR2 DEFAULT NULL -- NOWY PARAMETR
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Modyfikacja INSERT statement w ADD_SOURCE_FILE_CONFIG
|
||||||
|
**PRZED:**
|
||||||
|
```sql
|
||||||
|
INSERT INTO CT_MRDS.A_SOURCE_FILE_CONFIG (
|
||||||
|
A_SOURCE_KEY, SOURCE_FILE_TYPE, SOURCE_FILE_ID, SOURCE_FILE_DESC,
|
||||||
|
SOURCE_FILE_NAME_PATTERN, TABLE_ID, TEMPLATE_TABLE_NAME
|
||||||
|
) VALUES (
|
||||||
|
pSourceKey, pSourceFileType, pSourceFileId, pSourceFileDesc,
|
||||||
|
pSourceFileNamePattern, pTableId, pTemplateTableName
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
**PO:**
|
||||||
|
```sql
|
||||||
|
INSERT INTO CT_MRDS.A_SOURCE_FILE_CONFIG (
|
||||||
|
A_SOURCE_KEY, SOURCE_FILE_TYPE, SOURCE_FILE_ID, SOURCE_FILE_DESC,
|
||||||
|
SOURCE_FILE_NAME_PATTERN, TABLE_ID, TEMPLATE_TABLE_NAME, ENCODING
|
||||||
|
) VALUES (
|
||||||
|
pSourceKey, pSourceFileType, pSourceFileId, pSourceFileDesc,
|
||||||
|
pSourceFileNamePattern, pTableId, pTemplateTableName, pEncoding
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
### Przykład użycia zmodyfikowanej procedury
|
||||||
|
```sql
|
||||||
|
-- Dodanie konfiguracji z kodowaniem UTF-8
|
||||||
|
CALL FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'C2D',
|
||||||
|
pSourceFileType => 'INPUT',
|
||||||
|
pSourceFileId => 'UC_DISSEM',
|
||||||
|
pSourceFileDesc => 'UC DISSEM Metadata with UTF-8 encoding',
|
||||||
|
pSourceFileNamePattern => 'UC_NMA_DISSEM-*.csv',
|
||||||
|
pTableId => 'A_UC_DISSEM_METADATA_LOADS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.C2D_A_UC_DISSEM_METADATA_LOADS',
|
||||||
|
pEncoding => 'UTF8'
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Dodanie konfiguracji bez kodowania (jak dotychczas)
|
||||||
|
CALL FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LM',
|
||||||
|
pSourceFileType => 'INPUT',
|
||||||
|
pSourceFileId => 'STANDING_FACILITIES',
|
||||||
|
pSourceFileDesc => 'Standing Facilities Data',
|
||||||
|
pSourceFileNamePattern => 'SF_*.csv',
|
||||||
|
pTableId => 'STANDING_FACILITIES',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES'
|
||||||
|
-- pEncoding pomijamy - będzie NULL (domyślne kodowanie)
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. TESTY WALIDACYJNE
|
||||||
|
|
||||||
|
### Test 1: Podstawowa funkcjonalność
|
||||||
|
```sql
|
||||||
|
-- 1. Ustawienie kodowania UTF-8
|
||||||
|
UPDATE CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
SET ENCODING = 'UTF8'
|
||||||
|
WHERE A_SOURCE_FILE_CONFIG_KEY = 1;
|
||||||
|
|
||||||
|
-- 2. Test przetwarzania pliku
|
||||||
|
CALL FILE_MANAGER.PROCESS_SOURCE_FILE('INBOX/TEST/FILE_ID/TABLE_NAME/test_file.csv');
|
||||||
|
|
||||||
|
-- 3. Sprawdzenie logów
|
||||||
|
SELECT * FROM CT_MRDS.A_PROCESS_LOG
|
||||||
|
WHERE PROCEDURE_NAME = 'CREATE_EXTERNAL_TABLE'
|
||||||
|
ORDER BY LOG_DATE DESC;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Test 2: Fallback do wartości domyślnych
|
||||||
|
```sql
|
||||||
|
-- 1. Ustawienie NULL
|
||||||
|
UPDATE CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
SET ENCODING = NULL
|
||||||
|
WHERE A_SOURCE_FILE_CONFIG_KEY = 1;
|
||||||
|
|
||||||
|
-- 2. Test przetwarzania - powinno być używane domyślne kodowanie
|
||||||
|
CALL FILE_MANAGER.PROCESS_SOURCE_FILE('INBOX/TEST/FILE_ID/TABLE_NAME/test_file.csv');
|
||||||
|
```
|
||||||
|
|
||||||
|
### Test 3: Różne kodowania
|
||||||
|
```sql
|
||||||
|
-- 1. Test z Windows-1252
|
||||||
|
UPDATE CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
SET ENCODING = 'WE8MSWIN1252'
|
||||||
|
WHERE A_SOURCE_FILE_CONFIG_KEY = 1;
|
||||||
|
|
||||||
|
-- 2. Test z UTF-8
|
||||||
|
UPDATE CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
SET ENCODING = 'UTF8'
|
||||||
|
WHERE A_SOURCE_FILE_CONFIG_KEY = 2;
|
||||||
|
|
||||||
|
-- 3. Test przetwarzania obu plików
|
||||||
|
CALL FILE_MANAGER.PROCESS_SOURCE_FILE('INBOX/TEST1/FILE_ID/TABLE_NAME/test_file1.csv');
|
||||||
|
CALL FILE_MANAGER.PROCESS_SOURCE_FILE('INBOX/TEST2/FILE_ID/TABLE_NAME/test_file2.csv');
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. HARMONOGRAM WDROŻENIA
|
||||||
|
|
||||||
|
### Faza 1: Przygotowanie struktury (KRYTYCZNA)
|
||||||
|
1. ✅ Dodanie kolumny `ENCODING` do tabeli
|
||||||
|
2. ✅ Zaktualizowanie definicji tabeli w plikach DDL
|
||||||
|
|
||||||
|
### Faza 2: Modyfikacja pakietu (WYSOKA)
|
||||||
|
1. ⏳ Dodanie parametru do sygnatury `CREATE_EXTERNAL_TABLE`
|
||||||
|
2. ⏳ Implementacja logiki kodowania w formatowaniu JSON
|
||||||
|
3. ⏳ Aktualizacja wywołania w procedurze overload
|
||||||
|
4. ⏳ Aktualizacja parametrów logowania
|
||||||
|
|
||||||
|
### Faza 3: Testy i walidacja (ŚREDNIA)
|
||||||
|
1. ⏳ Testy jednostkowe z różnymi kodowaniami
|
||||||
|
2. ⏳ Testy integracyjne z różnymi typami plików
|
||||||
|
3. ⏳ Walidacja fallback do domyślnego kodowania
|
||||||
|
|
||||||
|
### Faza 4: Modyfikacja procedur konfiguracyjnych (NISKA)
|
||||||
|
1. ⏳ Modyfikacja `ADD_SOURCE_FILE_CONFIG` - dodanie parametru `pEncoding`
|
||||||
|
2. ⏳ Aktualizacja dokumentacji
|
||||||
|
3. ⏳ Przykłady użycia w confluence
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. ROLLBACK
|
||||||
|
|
||||||
|
W przypadku problemów można wykonać rollback:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Usunięcie kolumny (OSTROŻNIE - spowoduje utratę danych)
|
||||||
|
ALTER TABLE CT_MRDS.A_SOURCE_FILE_CONFIG DROP COLUMN ENCODING;
|
||||||
|
|
||||||
|
-- Przywrrócenie oryginalnej wersji pakietu FILE_MANAGER
|
||||||
|
-- (wymagane przywrrócenie z backupu)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. UWAGI TECHNICZNE
|
||||||
|
|
||||||
|
### Ograniczenia
|
||||||
|
- Maksymalna długość nazwy kodowania: 50 znaków
|
||||||
|
- Obsługiwane kodowania: wszystkie wspierane przez Oracle Database
|
||||||
|
- Przykłady poprawnych nazw: `UTF8`, `WE8MSWIN1252`, `AL32UTF8`, `WE8ISO8859P1`
|
||||||
|
|
||||||
|
### Bezpieczeństwo
|
||||||
|
- Brak walidacji nazwy kodowania na poziomie aplikacji
|
||||||
|
- Oracle Database zwaliduje kodowanie podczas tworzenia external table
|
||||||
|
- Fallback do domyślnego kodowania bazy w przypadku NULL
|
||||||
|
|
||||||
|
### Wydajność
|
||||||
|
- Minimalne obciążenie - kodowanie dodawane tylko jeśli nie jest NULL
|
||||||
|
- Brak wpływu na istniejące procesy gdy kolumna jest NULL
|
||||||
|
- Używa natywnej funkcji JSON_MERGEPATCH Oracle Database
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. PRZYGOTOWANIE PACZKI INSTALACYJNEJ
|
||||||
|
|
||||||
|
### Struktura paczki MARS-1049
|
||||||
|
```
|
||||||
|
MARS_Packages/REL02/MARS-1049/
|
||||||
|
├── 01_MARS_1049_install_CT_MRDS_ADD_ENCODING_COLUMN.sql
|
||||||
|
├── 02_MARS_1049_install_CT_MRDS_FILE_MANAGER_SPEC.sql
|
||||||
|
├── 03_MARS_1049_install_CT_MRDS_FILE_MANAGER_BODY.sql
|
||||||
|
├── 04_MARS_1049_verify_ENCODING_FUNCTIONALITY.sql
|
||||||
|
├── 05_MARS_1049_track_version.sql
|
||||||
|
├── 91_MARS_1049_rollback_FILE_MANAGER_BODY.sql
|
||||||
|
├── 92_MARS_1049_rollback_FILE_MANAGER_SPEC.sql
|
||||||
|
├── 93_MARS_1049_rollback_ENCODING_COLUMN.sql
|
||||||
|
├── 94_MARS_1049_track_rollback_version.sql
|
||||||
|
├── install_mars1049.sql
|
||||||
|
├── rollback_mars1049.sql
|
||||||
|
└── README_ENCODING.md
|
||||||
|
```
|
||||||
|
|
||||||
|
### Zawartość skryptów instalacyjnych
|
||||||
|
|
||||||
|
#### 01_MARS_1049_install_CT_MRDS_ADD_ENCODING_COLUMN.sql
|
||||||
|
```sql
|
||||||
|
-- MARS-1049: Dodanie kolumny ENCODING do A_SOURCE_FILE_CONFIG
|
||||||
|
-- Autor: System Enhancement
|
||||||
|
-- Data: 2025-11-24
|
||||||
|
|
||||||
|
PROMPT Adding ENCODING column to CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
|
||||||
|
ALTER TABLE CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
ADD (ENCODING VARCHAR2(50));
|
||||||
|
|
||||||
|
COMMENT ON COLUMN CT_MRDS.A_SOURCE_FILE_CONFIG.ENCODING
|
||||||
|
IS 'Character encoding for CSV files (e.g. UTF8, WE8MSWIN1252, etc.)';
|
||||||
|
|
||||||
|
PROMPT ENCODING column added successfully
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 02_MARS_1049_install_CT_MRDS_FILE_MANAGER_SPEC.sql
|
||||||
|
```sql
|
||||||
|
-- MARS-1049: Aktualizacja specyfikacji FILE_MANAGER - dodanie parametru pEncoding
|
||||||
|
-- Zawiera: Modyfikację sygnatury CREATE_EXTERNAL_TABLE i ADD_SOURCE_FILE_CONFIG
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 04_MARS_1049_verify_ENCODING_FUNCTIONALITY.sql
|
||||||
|
```sql
|
||||||
|
-- MARS-1049: Weryfikacja funkcjonalności ENCODING
|
||||||
|
PROMPT Testing ENCODING functionality...
|
||||||
|
|
||||||
|
-- Test 1: Sprawdzenie struktury tabeli
|
||||||
|
SELECT COLUMN_NAME, DATA_TYPE, DATA_LENGTH, NULLABLE
|
||||||
|
FROM USER_TAB_COLUMNS
|
||||||
|
WHERE TABLE_NAME = 'A_SOURCE_FILE_CONFIG'
|
||||||
|
AND COLUMN_NAME = 'ENCODING';
|
||||||
|
|
||||||
|
-- Test 2: Test podstawowej funkcjonalności
|
||||||
|
INSERT INTO CT_MRDS.A_SOURCE_FILE_CONFIG (
|
||||||
|
A_SOURCE_KEY, SOURCE_FILE_TYPE, SOURCE_FILE_ID,
|
||||||
|
SOURCE_FILE_DESC, SOURCE_FILE_NAME_PATTERN,
|
||||||
|
TABLE_ID, TEMPLATE_TABLE_NAME, ENCODING
|
||||||
|
) VALUES (
|
||||||
|
'TEST', 'INPUT', 'ENCODING_TEST',
|
||||||
|
'Test ENCODING functionality', 'test_*.csv',
|
||||||
|
'TEST_TABLE', 'CT_ET_TEMPLATES.TEST_TABLE', 'UTF8'
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Sprawdzenie czy rekord został dodany
|
||||||
|
SELECT A_SOURCE_FILE_CONFIG_KEY, ENCODING
|
||||||
|
FROM CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
WHERE A_SOURCE_KEY = 'TEST' AND SOURCE_FILE_ID = 'ENCODING_TEST';
|
||||||
|
|
||||||
|
-- Cleanup
|
||||||
|
DELETE FROM CT_MRDS.A_SOURCE_FILE_CONFIG
|
||||||
|
WHERE A_SOURCE_KEY = 'TEST' AND SOURCE_FILE_ID = 'ENCODING_TEST';
|
||||||
|
|
||||||
|
PROMPT ENCODING functionality verification completed successfully
|
||||||
|
```
|
||||||
|
|
||||||
|
#### install_mars1049.sql (Master Install Script)
|
||||||
|
```sql
|
||||||
|
-- MARS-1049: Master Installation Script
|
||||||
|
-- CSV Encoding Support Implementation
|
||||||
|
|
||||||
|
PROMPT ================================================
|
||||||
|
PROMPT MARS-1049: Installing CSV Encoding Support
|
||||||
|
PROMPT ================================================
|
||||||
|
|
||||||
|
SPOOL install_mars1049.log
|
||||||
|
|
||||||
|
-- Faza 1: Struktura bazy danych
|
||||||
|
PROMPT Phase 1: Database Structure Changes
|
||||||
|
@@01_MARS_1049_install_CT_MRDS_ADD_ENCODING_COLUMN.sql
|
||||||
|
|
||||||
|
-- Faza 2: Pakiet FILE_MANAGER (Specyfikacja)
|
||||||
|
PROMPT Phase 2: FILE_MANAGER Package Specification
|
||||||
|
@@02_MARS_1049_install_CT_MRDS_FILE_MANAGER_SPEC.sql
|
||||||
|
|
||||||
|
-- Faza 3: Pakiet FILE_MANAGER (Ciało)
|
||||||
|
PROMPT Phase 3: FILE_MANAGER Package Body
|
||||||
|
@@03_MARS_1049_install_CT_MRDS_FILE_MANAGER_BODY.sql
|
||||||
|
|
||||||
|
-- Faza 4: Weryfikacja
|
||||||
|
PROMPT Phase 4: Functionality Verification
|
||||||
|
@@04_MARS_1049_verify_ENCODING_FUNCTIONALITY.sql
|
||||||
|
|
||||||
|
-- Faza 5: Tracking wersji
|
||||||
|
PROMPT Phase 5: Version Tracking
|
||||||
|
@@05_MARS_1049_track_version.sql
|
||||||
|
|
||||||
|
PROMPT ================================================
|
||||||
|
PROMPT MARS-1049: Installation completed successfully
|
||||||
|
PROMPT ================================================
|
||||||
|
|
||||||
|
SPOOL OFF
|
||||||
|
```
|
||||||
|
|
||||||
|
#### rollback_mars1049.sql (Master Rollback Script)
|
||||||
|
```sql
|
||||||
|
-- MARS-1049: Master Rollback Script
|
||||||
|
-- WARNING: This will remove ENCODING functionality
|
||||||
|
|
||||||
|
PROMPT ================================================
|
||||||
|
PROMPT MARS-1049: Rolling back CSV Encoding Support
|
||||||
|
PROMPT WARNING: This will remove ENCODING column and data!
|
||||||
|
PROMPT ================================================
|
||||||
|
|
||||||
|
SPOOL rollback_mars1049.log
|
||||||
|
|
||||||
|
-- Rollback w odwrotnej kolejności
|
||||||
|
@@94_MARS_1049_track_rollback_version.sql
|
||||||
|
@@93_MARS_1049_rollback_ENCODING_COLUMN.sql
|
||||||
|
@@92_MARS_1049_rollback_FILE_MANAGER_SPEC.sql
|
||||||
|
@@91_MARS_1049_rollback_FILE_MANAGER_BODY.sql
|
||||||
|
|
||||||
|
PROMPT ================================================
|
||||||
|
PROMPT MARS-1049: Rollback completed
|
||||||
|
PROMPT ================================================
|
||||||
|
|
||||||
|
SPOOL OFF
|
||||||
|
```
|
||||||
|
|
||||||
|
### Deployment Commands
|
||||||
|
```powershell
|
||||||
|
# Instalacja paczki MARS-1049
|
||||||
|
Get-Content "MARS_Packages\REL02\MARS-1049\install_mars1049.sql" | sql "CT_MRDS/Cloudpass#34@ggmichalski_high"
|
||||||
|
|
||||||
|
# W przypadku problemów - rollback
|
||||||
|
Get-Content "MARS_Packages\REL02\MARS-1049\rollback_mars1049.sql" | sql "CT_MRDS/Cloudpass#34@ggmichalski_high"
|
||||||
|
|
||||||
|
# Archiwizacja po instalacji
|
||||||
|
7z a -pMojeSuperHaslo -mhe=on MARS_1049_arch.7z MARS_Packages\REL02\MARS-1049\*.sql
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Status: GOTOWE DO IMPLEMENTACJI
|
||||||
|
**Utworzone**: 2025-11-24
|
||||||
|
**Autor**: System FILE_MANAGER Enhancement
|
||||||
|
**MARS Issue**: MARS-1049
|
||||||
|
**Priorytet**: WYSOKI
|
||||||
1898
MARS_Packages/REL01/MARS-1049/current_version/FILE_MANAGER.pkb
Normal file
1898
MARS_Packages/REL01/MARS-1049/current_version/FILE_MANAGER.pkb
Normal file
File diff suppressed because it is too large
Load Diff
618
MARS_Packages/REL01/MARS-1049/current_version/FILE_MANAGER.pkg
Normal file
618
MARS_Packages/REL01/MARS-1049/current_version/FILE_MANAGER.pkg
Normal file
@@ -0,0 +1,618 @@
|
|||||||
|
create or replace PACKAGE CT_MRDS.FILE_MANAGER
|
||||||
|
AUTHID CURRENT_USER
|
||||||
|
AS
|
||||||
|
/**
|
||||||
|
* General comment for package: Please put comments for functions and procedures as shown in below example.
|
||||||
|
* It is a standard.
|
||||||
|
* The structure of comment is used by GET_PACKAGE_DOCUMENTATION function
|
||||||
|
* which returns documentation text for confluence page (to Copy-Paste it).
|
||||||
|
**/
|
||||||
|
|
||||||
|
-- Example comment:
|
||||||
|
/**
|
||||||
|
* @name EX_PROCEDURE_NAME
|
||||||
|
* @desc Procedure description
|
||||||
|
* @example select FILE_MANAGER.EX_PROCEDURE_NAME(pParameter => 129) from dual;
|
||||||
|
* @ex_rslt Example Result
|
||||||
|
**/
|
||||||
|
|
||||||
|
-- Package Version Information (Semantic Versioning: MAJOR.MINOR.PATCH)
|
||||||
|
PACKAGE_VERSION CONSTANT VARCHAR2(10) := '3.2.0';
|
||||||
|
PACKAGE_BUILD_DATE CONSTANT VARCHAR2(20) := '2025-10-22 16:30:00';
|
||||||
|
PACKAGE_AUTHOR CONSTANT VARCHAR2(100) := 'Grzegorz Michalski';
|
||||||
|
|
||||||
|
-- Version History (Latest changes first)
|
||||||
|
VERSION_HISTORY CONSTANT VARCHAR2(4000) :=
|
||||||
|
'3.2.0 (2025-10-22): Added package versioning system using centralized ENV_MANAGER functions' || CHR(13)||CHR(10) ||
|
||||||
|
'3.1.0 (2025-10-20): Enhanced PROCESS_SOURCE_FILE with 6-step validation workflow' || CHR(13)||CHR(10) ||
|
||||||
|
'3.0.0 (2025-10-15): Separated export procedures into dedicated DATA_EXPORTER package' || CHR(13)||CHR(10) ||
|
||||||
|
'2.5.0 (2025-10-10): Added DELETE_SOURCE_CASCADE for safe configuration removal' || CHR(13)||CHR(10) ||
|
||||||
|
'2.0.0 (2025-09-25): Added official path patterns support (INBOX 3-level, ODS 2-level, ARCHIVE 2-level)' || CHR(13)||CHR(10) ||
|
||||||
|
'1.0.0 (2025-09-01): Initial release with file processing and validation capabilities';
|
||||||
|
|
||||||
|
TYPE tSourceFileReceived IS RECORD
|
||||||
|
(
|
||||||
|
A_SOURCE_FILE_RECEIVED_KEY CT_MRDS.A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY%TYPE,
|
||||||
|
A_SOURCE_FILE_CONFIG_KEY CT_MRDS.A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_CONFIG_KEY%TYPE,
|
||||||
|
SOURCE_FILE_PREFIX_INBOX VARCHAR2(430),
|
||||||
|
SOURCE_FILE_PREFIX_ODS VARCHAR2(430),
|
||||||
|
SOURCE_FILE_PREFIX_QUARANTINE VARCHAR2(430),
|
||||||
|
SOURCE_FILE_PREFIX_ARCHIVE VARCHAR2(430),
|
||||||
|
SOURCE_FILE_NAME CT_MRDS.A_SOURCE_FILE_RECEIVED.SOURCE_FILE_NAME%TYPE,
|
||||||
|
RECEPTION_DATE CT_MRDS.A_SOURCE_FILE_RECEIVED.RECEPTION_DATE%TYPE,
|
||||||
|
PROCESSING_STATUS CT_MRDS.A_SOURCE_FILE_RECEIVED.PROCESSING_STATUS%TYPE,
|
||||||
|
EXTERNAL_TABLE_NAME CT_MRDS.A_SOURCE_FILE_RECEIVED.EXTERNAL_TABLE_NAME%TYPE
|
||||||
|
);
|
||||||
|
|
||||||
|
cgBL CONSTANT VARCHAR2(2) := CHR(13)||CHR(10);
|
||||||
|
vgSourceFileConfigKey PLS_INTEGER;
|
||||||
|
vgMsgTmp VARCHAR2(32000);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_SOURCE_FILE_CONFIG
|
||||||
|
* @desc Get source file type by matching the source file name against source file type naming patterns
|
||||||
|
* or by specifying the id of a received source file.
|
||||||
|
* @example ...
|
||||||
|
* @ex_rslt "CT_MRDS.A_SOURCE_FILE_CONFIG%ROWTYPE"
|
||||||
|
**/
|
||||||
|
FUNCTION GET_SOURCE_FILE_CONFIG(pFileUri IN VARCHAR2 DEFAULT NULL
|
||||||
|
, pSourceFileReceivedKey IN NUMBER DEFAULT NULL
|
||||||
|
, pSourceFileConfigKey IN NUMBER DEFAULT NULL)
|
||||||
|
RETURN CT_MRDS.A_SOURCE_FILE_CONFIG%ROWTYPE;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name REGISTER_SOURCE_FILE_RECEIVED
|
||||||
|
* @desc Register a newly received source file in A_SOURCE_FILE_RECEIVED table.
|
||||||
|
* This overload automatically determines source file type from the file name.
|
||||||
|
* It returns the value of A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY column for newly added record.
|
||||||
|
* @example vSourceFileReceivedKey := FILE_MANAGER.REGISTER_SOURCE_FILE_RECEIVED(pSourceFileReceivedName => 'INBOX/C2D/UC_DISSEM/UC_NMA_DISSEM/UC_NMA_DISSEM-277740.csv');
|
||||||
|
* @ex_rslt 3245
|
||||||
|
**/
|
||||||
|
FUNCTION REGISTER_SOURCE_FILE_RECEIVED (
|
||||||
|
pSourceFileReceivedName IN VARCHAR2
|
||||||
|
)
|
||||||
|
RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name REGISTER_SOURCE_FILE_RECEIVED
|
||||||
|
* @desc Register a new new source file in A_SOURCE_FILE_RECEIVED table based on pSourceFileReceivedName and pSourceFileConfig.
|
||||||
|
* Then it returns the value of A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY column for newly added record.
|
||||||
|
* @example vSourceFileReceivedKey := FILE_MANAGER.REGISTER_SOURCE_FILE_RECEIVED(
|
||||||
|
* pSourceFileReceivedName => 'INBOX/C2D/UC_DISSEM/UC_NMA_DISSEM/UC_NMA_DISSEM-277740.csv'
|
||||||
|
* ,pSourceFileConfig => ...A_SOURCE_FILE_CONFIG%ROWTYPE... );
|
||||||
|
* @ex_rslt 3245
|
||||||
|
**/
|
||||||
|
FUNCTION REGISTER_SOURCE_FILE_RECEIVED (
|
||||||
|
pSourceFileReceivedName IN VARCHAR2,
|
||||||
|
pSourceFileConfig IN CT_MRDS.A_SOURCE_FILE_CONFIG%ROWTYPE
|
||||||
|
)
|
||||||
|
RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name SET_SOURCE_FILE_RECEIVED_STATUS
|
||||||
|
* @desc Set status of file in A_SOURCE_FILE_RECEIVED table - PROCESSING_STATUS column
|
||||||
|
* based on A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY
|
||||||
|
* and provided value of pStatus parameter
|
||||||
|
* @example exec FILE_MANAGER.SET_SOURCE_FILE_RECEIVED_STATUS(pSourceFileReceivedKey => 377, pStatus => 'READY_FOR_INGESTION');
|
||||||
|
**/
|
||||||
|
PROCEDURE SET_SOURCE_FILE_RECEIVED_STATUS(
|
||||||
|
pSourceFileReceivedKey IN PLS_INTEGER,
|
||||||
|
pStatus IN VARCHAR2
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_EXTERNAL_TABLE_COLUMNS
|
||||||
|
* @desc Function used to get string with all table columns definitions based on pTargetTableTemplate "TEMPLATE TABLE" name.
|
||||||
|
* It used for creating "EXTERNAL TABLE" using CREATE_EXTERNAL_TABLE procedure.
|
||||||
|
* @example select FILE_MANAGER.GET_EXTERNAL_TABLE_COLUMNS(pTargetTableTemplate => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES_HEADER') from dual;
|
||||||
|
* @ex_rslt "A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
* "A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
* "REV_NUMBER" NUMBER(28,0),
|
||||||
|
* "REF_DATE" DATE,
|
||||||
|
* "FREE_TEXT" VARCHAR2(1000 CHAR),
|
||||||
|
* "MLF_BS_TOTAL" NUMBER(28,10),
|
||||||
|
* "DF_BS_TOTAL" NUMBER(28,10),
|
||||||
|
* "MLF_SF_TOTAL" NUMBER(28,10),
|
||||||
|
* "DF_SF_TOTAL" NUMBER(28,10)
|
||||||
|
**/
|
||||||
|
FUNCTION GET_EXTERNAL_TABLE_COLUMNS (
|
||||||
|
pTargetTableTemplate IN VARCHAR2
|
||||||
|
)
|
||||||
|
RETURN CLOB;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name CREATE_EXTERNAL_TABLE
|
||||||
|
* @desc A wrapper procedure for DBMS_CLOUD.CREATE_EXTERNAL_TABLE which creates External Table
|
||||||
|
* @example
|
||||||
|
* begin
|
||||||
|
* FILE_MANAGER.CREATE_EXTERNAL_TABLE(
|
||||||
|
* pTableName => 'STANDING_FACILITIES_HEADER',
|
||||||
|
* pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES_HEADER',
|
||||||
|
* pPrefix => 'ODS/LM/STANDING_FACILITIES_HEADER/',
|
||||||
|
* pBucketUri => 'https://objectstorage.eu-frankfurt-1.oraclecloud.com/n/frcnomajoc7v/b/mrds_data_tst/o/'
|
||||||
|
* pFileName => 'https://objectstorage.eu-frankfurt-1.oraclecloud.com/n/frcnomajoc7v/b/mrds_data_tst/o/ODS/LM/STANDING_FACILITIES_HEADER/*.csv',
|
||||||
|
* pDelimiter => ',',
|
||||||
|
|
||||||
|
* );
|
||||||
|
* end;
|
||||||
|
**/
|
||||||
|
PROCEDURE CREATE_EXTERNAL_TABLE (
|
||||||
|
pTableName IN VARCHAR2,
|
||||||
|
pTemplateTableName IN VARCHAR2,
|
||||||
|
pPrefix IN VARCHAR2,
|
||||||
|
pBucketUri IN VARCHAR2 DEFAULT ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pFileName IN VARCHAR2 DEFAULT NULL,
|
||||||
|
pDelimiter IN VARCHAR2 DEFAULT ','
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name CREATE_EXTERNAL_TABLE
|
||||||
|
* @desc Creates External Table for single file provided by
|
||||||
|
* pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* @example exec FILE_MANAGER.CREATE_EXTERNAL_TABLE(pSourceFileReceivedKey => 377);;
|
||||||
|
**/
|
||||||
|
PROCEDURE CREATE_EXTERNAL_TABLE (
|
||||||
|
pSourceFileReceivedKey IN NUMBER
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name VALIDATE_SOURCE_FILE_RECEIVED
|
||||||
|
* @desc A wrapper procedure for DBMS_CLOUD.VALIDATE_EXTERNAL_TABLE
|
||||||
|
* It validate External table build upon single file
|
||||||
|
* provided by pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* @example exec FILE_MANAGER.VALIDATE_SOURCE_FILE_RECEIVED(pSourceFileReceivedKey => 377);
|
||||||
|
**/
|
||||||
|
PROCEDURE VALIDATE_SOURCE_FILE_RECEIVED
|
||||||
|
(
|
||||||
|
pSourceFileReceivedKey IN NUMBER
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name VALIDATE_EXTERNAL_TABLE
|
||||||
|
* @desc A wrapper function for DBMS_CLOUD.VALIDATE_EXTERNAL_TABLE.
|
||||||
|
* It validates External Table provided by parameter pTableName.
|
||||||
|
* It returns: PASSED or FAILED.
|
||||||
|
* @example
|
||||||
|
* declare
|
||||||
|
* vStatus VARCHAR2(100);
|
||||||
|
* begin
|
||||||
|
* vStatus := FILE_MANAGER.VALIDATE_EXTERNAL_TABLE(pTableName => 'STANDING_FACILITIES_HEADER');
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vStatus = '||vStatus);
|
||||||
|
* end;
|
||||||
|
*
|
||||||
|
* @ex_rslt FAILED
|
||||||
|
**/
|
||||||
|
FUNCTION VALIDATE_EXTERNAL_TABLE(pTableName IN VARCHAR2)
|
||||||
|
RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name S_VALIDATE_EXTERNAL_TABLE
|
||||||
|
* @desc A function which checks if SELECT query reterns any rows.
|
||||||
|
* It trys to selects External Table provided by parameter pTableName.
|
||||||
|
* It returns: PASSED or FAILED.
|
||||||
|
* @example
|
||||||
|
* declare
|
||||||
|
* vStatus VARCHAR2(100);
|
||||||
|
* begin
|
||||||
|
* vStatus := FILE_MANAGER.S_VALIDATE_EXTERNAL_TABLE(pTableName => 'STANDING_FACILITIES_HEADER');
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vStatus = '||vStatus);
|
||||||
|
* end;
|
||||||
|
*
|
||||||
|
* @ex_rslt PASSED
|
||||||
|
**/
|
||||||
|
FUNCTION S_VALIDATE_EXTERNAL_TABLE(pTableName IN VARCHAR2)
|
||||||
|
RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name DROP_EXTERNAL_TABLE
|
||||||
|
* @desc It drops External Table for single file provided by
|
||||||
|
* pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* @example exec FILE_MANAGER.DROP_EXTERNAL_TABLE(pSourceFileReceivedKey => 377);
|
||||||
|
**/
|
||||||
|
PROCEDURE DROP_EXTERNAL_TABLE (
|
||||||
|
pSourceFileReceivedKey IN NUMBER
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name COPY_FILE
|
||||||
|
* @desc It copies file provided by
|
||||||
|
* pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* into destination provided by pDestination parameter.
|
||||||
|
* pDestination parameter allowed values are: 'ODS'
|
||||||
|
* @example exec FILE_MANAGER.COPY_FILE(pSourceFileReceivedKey => 377, pDestination => 'ODS');
|
||||||
|
**/
|
||||||
|
PROCEDURE COPY_FILE(
|
||||||
|
pSourceFileReceivedKey IN NUMBER,
|
||||||
|
pDestination IN VARCHAR2
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name MOVE_FILE
|
||||||
|
* @desc It moves file provided by
|
||||||
|
* pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* into destination provided by pDestination parameter.
|
||||||
|
* pDestination parameter allowed values are: 'ODS', 'QUARANTINE'
|
||||||
|
* @example exec FILE_MANAGER.MOVE_FILE(pSourceFileReceivedKey => 377, pDestination => 'ODS');
|
||||||
|
**/
|
||||||
|
PROCEDURE MOVE_FILE(
|
||||||
|
pSourceFileReceivedKey IN NUMBER,
|
||||||
|
pDestination IN VARCHAR2
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name DELETE_FOLDER_CONTENTS
|
||||||
|
* @desc It deletes all files from specified folder in the cloud storage.
|
||||||
|
* The procedure lists all objects in the specified folder prefix and deletes them one by one.
|
||||||
|
* pBucketArea parameter specifies which bucket to use: 'INBOX', 'DATA', 'ARCHIVE'
|
||||||
|
* pFolderPrefix parameter specifies the folder path within the bucket (e.g., 'C2D/UC_DISSEM/UC_NMA_DISSEM/')
|
||||||
|
* @example exec FILE_MANAGER.DELETE_FOLDER_CONTENTS(pBucketArea => 'INBOX', pFolderPrefix => 'C2D/UC_DISSEM/UC_NMA_DISSEM/');
|
||||||
|
**/
|
||||||
|
PROCEDURE DELETE_FOLDER_CONTENTS(
|
||||||
|
pBucketArea IN VARCHAR2,
|
||||||
|
pFolderPrefix IN VARCHAR2
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name PROCESS_SOURCE_FILE
|
||||||
|
* @desc It process file provided by pSourceFileReceivedName parameter.
|
||||||
|
* Ubmrella procedure that calls:
|
||||||
|
* - REGISTER_SOURCE_FILE_RECEIVED;
|
||||||
|
* - CREATE_EXTERNAL_TABLE;
|
||||||
|
* - VALIDATE_SOURCE_FILE_RECEIVED;
|
||||||
|
* - DROP_EXTERNAL_TABLE;
|
||||||
|
* - MOVE_FILE;
|
||||||
|
* @example exec FILE_MANAGER.PROCESS_SOURCE_FILE(pSourceFileReceivedName => 'INBOX/C2D/UC_DISSEM/UC_NMA_DISSEM/UC_NMA_DISSEM-277740.csv');
|
||||||
|
**/
|
||||||
|
PROCEDURE PROCESS_SOURCE_FILE(pSourceFileReceivedName IN VARCHAR2)
|
||||||
|
;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name PROCESS_SOURCE_FILE
|
||||||
|
* @desc It process file provided by pSourceFileReceivedName parameter and return processing result value.
|
||||||
|
* It returns (success/failure) => 0 / -(value).
|
||||||
|
* Ubmrella function that calls PROCESS_SOURCE_FILE procedure.
|
||||||
|
* @example
|
||||||
|
* declare
|
||||||
|
* vResult PLS_INTEGER;
|
||||||
|
* begin
|
||||||
|
* vResult := CT_MRDS.FILE_MANAGER.PROCESS_SOURCE_FILE(PSOURCEFILERECEIVEDNAME => 'INBOX/C2D/UC_DISSEM/UC_NMA_DISSEM/UC_NMA_DISSEM-277740.csv');
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vResult = ' || vResult);
|
||||||
|
* end;
|
||||||
|
* @ex_rslt 0
|
||||||
|
* -20021
|
||||||
|
**/
|
||||||
|
FUNCTION PROCESS_SOURCE_FILE(pSourceFileReceivedName IN VARCHAR2)
|
||||||
|
RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_DATE_FORMAT
|
||||||
|
* @desc Returns date format for specified template table name and column name.
|
||||||
|
* Date is taken from configuration A_COLUMN_DATE_FORMAT table.
|
||||||
|
* @example select FILE_MANAGER.GET_DATE_FORMAT(
|
||||||
|
* pTemplateTableName => 'STANDING_FACILITIES_HEADER',
|
||||||
|
* pColumnName => 'SNAPSHOT_DATE')
|
||||||
|
* from dual;
|
||||||
|
* @ex_rslt DD/MM/YYYY HH24:MI:SS
|
||||||
|
**/
|
||||||
|
FUNCTION GET_DATE_FORMAT(
|
||||||
|
pTemplateTableName IN VARCHAR2,
|
||||||
|
pColumnName IN VARCHAR2
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GENERATE_EXTERNAL_TABLE_PARAMS
|
||||||
|
* @desc It builds two strings: pColumnList and pFieldList for specified Template Table name, by parameter: pTemplateTableName.
|
||||||
|
* @example
|
||||||
|
* declare
|
||||||
|
* vColumnList CLOB;
|
||||||
|
* vFieldList CLOB;
|
||||||
|
* begin
|
||||||
|
* FILE_MANAGER.GENERATE_EXTERNAL_TABLE_PARAMS (
|
||||||
|
* pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES_HEADER'
|
||||||
|
* ,pColumnList => vColumnList
|
||||||
|
* ,pFieldList => vFieldList
|
||||||
|
* );
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vColumnList = '||vColumnList);
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vFieldList = '||vFieldList);
|
||||||
|
* end;
|
||||||
|
* /
|
||||||
|
**/
|
||||||
|
PROCEDURE GENERATE_EXTERNAL_TABLE_PARAMS (
|
||||||
|
pTemplateTableName IN VARCHAR2,
|
||||||
|
pColumnList OUT CLOB,
|
||||||
|
pFieldList OUT CLOB
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name ADD_SOURCE
|
||||||
|
* @desc Insert a new record to A_SOURCE table.
|
||||||
|
* pSourceKey is a PRIMARY KEY value.
|
||||||
|
**/
|
||||||
|
PROCEDURE ADD_SOURCE (
|
||||||
|
pSourceKey IN CT_MRDS.A_SOURCE.A_SOURCE_KEY%TYPE,
|
||||||
|
pSourceName IN CT_MRDS.A_SOURCE.SOURCE_NAME%TYPE
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name DELETE_SOURCE_CASCADE
|
||||||
|
* @desc Safely deletes a SOURCE specified by pSourceKey parameter from A_SOURCE table and all dependent tables:
|
||||||
|
* - A_SOURCE_FILE_CONFIG
|
||||||
|
* - A_SOURCE_FILE_RECEIVED
|
||||||
|
* - A_COLUMN_DATE_FORMAT (only if template table is not shared with other source systems)
|
||||||
|
* The procedure checks if template tables are shared before deleting date format configurations.
|
||||||
|
* If a template table is used by multiple source systems, date formats are preserved.
|
||||||
|
* @example CALL CT_MRDS.FILE_MANAGER.DELETE_SOURCE_CASCADE(pSourceKey => 'TEST_SYS');
|
||||||
|
**/
|
||||||
|
PROCEDURE DELETE_SOURCE_CASCADE (
|
||||||
|
pSourceKey IN CT_MRDS.A_SOURCE.A_SOURCE_KEY%TYPE
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_CONTAINER_SOURCE_FILE_CONFIG_KEY
|
||||||
|
* @desc For specified parameter pSourceFileId (A_SOURCE_FILE_CONFIG.SOURCE_FILE_ID)
|
||||||
|
* it returns A_SOURCE_FILE_CONFIG.A_SOURCE_FILE_CONFIG_KEY for related CONTAINER record.
|
||||||
|
* @example select FILE_MANAGER.GET_CONTAINER_SOURCE_FILE_CONFIG_KEY(
|
||||||
|
* pSourceFileId => 'UC_DISSEM')
|
||||||
|
* from dual;
|
||||||
|
* @ex_rslt 126
|
||||||
|
**/
|
||||||
|
FUNCTION GET_CONTAINER_SOURCE_FILE_CONFIG_KEY (
|
||||||
|
pSourceFileId IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_ID%TYPE
|
||||||
|
) RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_SOURCE_FILE_CONFIG_KEY
|
||||||
|
* @desc For specified input parameters,
|
||||||
|
* it returns A_SOURCE_FILE_CONFIG.A_SOURCE_FILE_CONFIG_KEY.
|
||||||
|
* @example select FILE_MANAGER.GET_SOURCE_FILE_CONFIG_KEY (
|
||||||
|
* pSourceFileType => 'INPUT'
|
||||||
|
* ,pSourceFileId => 'UC_DISSEM'
|
||||||
|
* ,pTableId => 'UC_NMA_DISSEM')
|
||||||
|
* from dual;
|
||||||
|
* @ex_rslt 126
|
||||||
|
**/
|
||||||
|
FUNCTION GET_SOURCE_FILE_CONFIG_KEY (
|
||||||
|
pSourceFileType IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_TYPE%TYPE DEFAULT 'INPUT'
|
||||||
|
,pSourceFileId IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_ID%TYPE
|
||||||
|
,pTableId IN CT_MRDS.A_SOURCE_FILE_CONFIG.TABLE_ID%TYPE
|
||||||
|
) RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name ADD_SOURCE_FILE_CONFIG
|
||||||
|
* @desc Insert a new record to A_SOURCE_FILE_CONFIG table.
|
||||||
|
**/
|
||||||
|
PROCEDURE ADD_SOURCE_FILE_CONFIG (
|
||||||
|
pSourceKey IN CT_MRDS.A_SOURCE_FILE_CONFIG.A_SOURCE_KEY%TYPE
|
||||||
|
,pSourceFileType IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_TYPE%TYPE
|
||||||
|
,pSourceFileId IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_ID%TYPE
|
||||||
|
,pSourceFileDesc IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_DESC%TYPE
|
||||||
|
,pSourceFileNamePattern IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_NAME_PATTERN%TYPE
|
||||||
|
,pTableId IN CT_MRDS.A_SOURCE_FILE_CONFIG.TABLE_ID%TYPE DEFAULT NULL
|
||||||
|
,pTemplateTableName IN CT_MRDS.A_SOURCE_FILE_CONFIG.TEMPLATE_TABLE_NAME%TYPE DEFAULT NULL
|
||||||
|
,pContainerFileKey IN CT_MRDS.A_SOURCE_FILE_CONFIG.CONTAINER_FILE_KEY%TYPE DEFAULT NULL
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name ADD_COLUMN_DATE_FORMAT
|
||||||
|
* @desc Insert a new record to A_COLUMN_DATE_FORMAT table.
|
||||||
|
**/
|
||||||
|
PROCEDURE ADD_COLUMN_DATE_FORMAT (
|
||||||
|
pTemplateTableName IN CT_MRDS.A_COLUMN_DATE_FORMAT.TEMPLATE_TABLE_NAME%TYPE
|
||||||
|
,pColumnName IN CT_MRDS.A_COLUMN_DATE_FORMAT.COLUMN_NAME%TYPE
|
||||||
|
,pDateFormat IN CT_MRDS.A_COLUMN_DATE_FORMAT.DATE_FORMAT%TYPE
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_BUCKET_URI
|
||||||
|
* @desc Function used to get string with bucket http url.
|
||||||
|
* Possible input values for pBucketArea are: 'INBOX', 'ODS', 'DATA', 'ARCHIVE'
|
||||||
|
* @example select FILE_MANAGER.GET_BUCKET_URI(pBucketArea => 'ODS') from dual;
|
||||||
|
* @ex_rslt https://objectstorage.eu-frankfurt-1.oraclecloud.com/n/frcnomajoc7v/b/mrds_data_tst/o/
|
||||||
|
**/
|
||||||
|
FUNCTION GET_BUCKET_URI(pBucketArea VARCHAR2)
|
||||||
|
RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_DET_SOURCE_FILE_CONFIG_INFO
|
||||||
|
* @desc Function returns details about A_SOURCE_FILE_CONFIG record
|
||||||
|
* for specified pSourceFileConfigKey (A_SOURCE_FILE_CONFIG.A_SOURCE_FILE_CONFIG_KEY).
|
||||||
|
* If pIncludeContainerInfo is <> 0 it returns additional info about related Container config record (A_SOURCE_FILE_CONFIG)
|
||||||
|
* If pIncludeColumnFormatInfo is <> 0 it returns additional info about related ColumnFormat config record (A_COLUMN_DATE_FORMAT)
|
||||||
|
* @example select FILE_MANAGER.GET_DET_SOURCE_FILE_CONFIG_INFO (
|
||||||
|
* pSourceFileConfigKey => 128
|
||||||
|
* ,pIncludeContainerInfo => 1
|
||||||
|
* ,pIncludeColumnFormatInfo => 1
|
||||||
|
* ) from dual;
|
||||||
|
* @ex_rslt
|
||||||
|
* Details about File Configuration:
|
||||||
|
* --------------------------------
|
||||||
|
* A_SOURCE_FILE_CONFIG_KEY = 128
|
||||||
|
* A_SOURCE_KEY = C2D
|
||||||
|
* ...
|
||||||
|
* --------------------------------
|
||||||
|
*
|
||||||
|
* Details about related Container Config:
|
||||||
|
* --------------------------------
|
||||||
|
* A_SOURCE_FILE_CONFIG_KEY = 126
|
||||||
|
* A_SOURCE_KEY = C2D
|
||||||
|
* ...
|
||||||
|
* --------------------------------
|
||||||
|
*
|
||||||
|
* Column Date Format config entries:
|
||||||
|
* --------------------------------
|
||||||
|
* TEMPLATE_TABLE_NAME = CT_ET_TEMPLATES.C2D_UC_MA_DISSEM
|
||||||
|
* ...
|
||||||
|
* --------------------------------
|
||||||
|
**/
|
||||||
|
FUNCTION GET_DET_SOURCE_FILE_CONFIG_INFO (
|
||||||
|
pSourceFileConfigKey IN CT_MRDS.A_SOURCE_FILE_CONFIG.A_SOURCE_FILE_CONFIG_KEY%TYPE
|
||||||
|
,pIncludeContainerInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
,pIncludeColumnFormatInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_DET_SOURCE_FILE_RECEIVED_INFO
|
||||||
|
* @desc Function returns details about A_SOURCE_FILE_RECEIVED record
|
||||||
|
* for specified pSourceFileReceivedKey (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY).
|
||||||
|
* If pIncludeConfigInfo is <> 0 it returns additional info about related Container config record (A_SOURCE_FILE_CONFIG)
|
||||||
|
* If pIncludeContainerInfo is <> 0 it returns additional info about related Container config record (A_SOURCE_FILE_CONFIG)
|
||||||
|
* If pIncludeColumnFormatInfo is <> 0 it returns additional info about related ColumnFormat config record (A_COLUMN_DATE_FORMAT)
|
||||||
|
* @example select FILE_MANAGER.GET_DET_SOURCE_FILE_RECEIVED_INFO (
|
||||||
|
* pSourceFileReceivedKey => 377
|
||||||
|
* ,pIncludeConfigInfo => 1
|
||||||
|
* ,pIncludeContainerInfo => 1
|
||||||
|
* ,pIncludeColumnFormatInfo => 1
|
||||||
|
* ) from dual;
|
||||||
|
*
|
||||||
|
**/
|
||||||
|
FUNCTION GET_DET_SOURCE_FILE_RECEIVED_INFO (
|
||||||
|
pSourceFileReceivedKey IN CT_MRDS.A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY%TYPE
|
||||||
|
,pIncludeConfigInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
,pIncludeContainerInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
,pIncludeColumnFormatInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_DET_USER_LOAD_OPERATIONS
|
||||||
|
* @desc Function returns details from USER_LOAD_OPERATIONS table
|
||||||
|
* for specified pOperationId.
|
||||||
|
* @example select FILE_MANAGER.GET_DET_USER_LOAD_OPERATIONS (pOperationId => 3608) from dual;
|
||||||
|
* @ex_rslt
|
||||||
|
* Details about USER_LOAD_OPERATIONS where ID = 3608
|
||||||
|
* --------------------------------
|
||||||
|
* ID = 3608
|
||||||
|
* TYPE = VALIDATE
|
||||||
|
* SID = 31260
|
||||||
|
* SERIAL# = 52915
|
||||||
|
* START_TIME = 2025-05-20 10.08.24.436983 EUROPE/BELGRADE
|
||||||
|
* UPDATE_TIME = 2025-05-20 10.08.24.458643 EUROPE/BELGRADE
|
||||||
|
* STATUS = FAILED
|
||||||
|
* OWNER_NAME = CT_MRDS
|
||||||
|
* TABLE_NAME = STANDING_FACILITIES_HEADER
|
||||||
|
* PARTITION_NAME =
|
||||||
|
* SUBPARTITION_NAME =
|
||||||
|
* FILE_URI_LIST =
|
||||||
|
* ROWS_LOADED =
|
||||||
|
* LOGFILE_TABLE = VALIDATE$3608_LOG
|
||||||
|
* BADFILE_TABLE = VALIDATE$3608_BAD
|
||||||
|
* STATUS_TABLE =
|
||||||
|
* TEMPEXT_TABLE =
|
||||||
|
* CREDENTIAL_NAME =
|
||||||
|
* EXPIRATION_TIME = 2025-05-22 10.08.24.436983000 EUROPE/BELGRADE
|
||||||
|
* --------------------------------
|
||||||
|
**/
|
||||||
|
FUNCTION GET_DET_USER_LOAD_OPERATIONS (
|
||||||
|
pOperationId PLS_INTEGER
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name ANALYZE_VALIDATION_ERRORS
|
||||||
|
* @desc Wrapper function that analyzes validation errors for a source file using its received key.
|
||||||
|
* Automatically derives template schema, table name, CSV URI and validation log table
|
||||||
|
* from file metadata and calls ENV_MANAGER.ANALYZE_VALIDATION_ERRORS.
|
||||||
|
* @example SELECT FILE_MANAGER.ANALYZE_VALIDATION_ERRORS(63) FROM DUAL;
|
||||||
|
* @ex_rslt Detailed validation analysis report with column mismatches and solutions
|
||||||
|
**/
|
||||||
|
FUNCTION ANALYZE_VALIDATION_ERRORS(
|
||||||
|
pSourceFileReceivedKey IN NUMBER
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
-- PACKAGE VERSION MANAGEMENT FUNCTIONS
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_VERSION
|
||||||
|
* @desc Returns the current version number of the FILE_MANAGER package.
|
||||||
|
* Uses semantic versioning format (MAJOR.MINOR.PATCH).
|
||||||
|
* @example SELECT FILE_MANAGER.GET_VERSION() FROM DUAL;
|
||||||
|
* @ex_rslt 3.2.0
|
||||||
|
**/
|
||||||
|
FUNCTION GET_VERSION RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_BUILD_INFO
|
||||||
|
* @desc Returns comprehensive build information including version, build date, and author.
|
||||||
|
* Uses centralized ENV_MANAGER.GET_PACKAGE_VERSION_INFO function.
|
||||||
|
* @example SELECT FILE_MANAGER.GET_BUILD_INFO() FROM DUAL;
|
||||||
|
* @ex_rslt Package: FILE_MANAGER
|
||||||
|
* Version: 3.2.0
|
||||||
|
* Build Date: 2025-10-22 16:30:00
|
||||||
|
* Author: Grzegorz Michalski
|
||||||
|
**/
|
||||||
|
FUNCTION GET_BUILD_INFO RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_VERSION_HISTORY
|
||||||
|
* @desc Returns complete version history with all releases and changes.
|
||||||
|
* Uses centralized ENV_MANAGER.FORMAT_VERSION_HISTORY function.
|
||||||
|
* @example SELECT FILE_MANAGER.GET_VERSION_HISTORY() FROM DUAL;
|
||||||
|
* @ex_rslt FILE_MANAGER Version History:
|
||||||
|
* 3.2.0 (2025-10-22): Added package versioning system...
|
||||||
|
**/
|
||||||
|
FUNCTION GET_VERSION_HISTORY RETURN VARCHAR2;
|
||||||
|
|
||||||
|
END;
|
||||||
|
|
||||||
|
/
|
||||||
@@ -0,0 +1,97 @@
|
|||||||
|
create or replace PACKAGE BODY ODS.FILE_MANAGER_ODS
|
||||||
|
AS
|
||||||
|
|
||||||
|
/**
|
||||||
|
* CREATE_EXTERNAL_TABLE - Wrapper for CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLE
|
||||||
|
*/
|
||||||
|
PROCEDURE CREATE_EXTERNAL_TABLE (
|
||||||
|
pTableName IN VARCHAR2,
|
||||||
|
pTemplateTableName IN VARCHAR2,
|
||||||
|
pPrefix IN VARCHAR2,
|
||||||
|
pBucketUri IN VARCHAR2 DEFAULT CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pFileName IN VARCHAR2 DEFAULT NULL,
|
||||||
|
pDelimiter IN VARCHAR2 DEFAULT ','
|
||||||
|
)
|
||||||
|
IS
|
||||||
|
vParameters CT_MRDS.A_PROCESS_LOG.PROCEDURE_PARAMETERS%TYPE;
|
||||||
|
BEGIN
|
||||||
|
-- Log the start of the procedure
|
||||||
|
vParameters := CT_MRDS.ENV_MANAGER.FORMAT_PARAMETERS(
|
||||||
|
SYS.ODCIVARCHAR2LIST(
|
||||||
|
'pTableName => ''' || pTableName || '''',
|
||||||
|
'pTemplateTableName => ''' || pTemplateTableName || '''',
|
||||||
|
'pPrefix => ''' || pPrefix || '''',
|
||||||
|
'pBucketUri => ''' || pBucketUri || '''',
|
||||||
|
'pFileName => ''' || NVL(pFileName, 'NULL') || '''',
|
||||||
|
'pDelimiter => ''' || pDelimiter || ''''
|
||||||
|
)
|
||||||
|
);
|
||||||
|
|
||||||
|
CT_MRDS.ENV_MANAGER.LOG_PROCESS_EVENT('Start FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE', 'INFO', vParameters);
|
||||||
|
|
||||||
|
-- Call the original CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLE procedure
|
||||||
|
-- This ensures all logic remains centralized in the original package
|
||||||
|
-- and ODS wrapper simply delegates execution with DEFINER rights
|
||||||
|
CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => pTableName,
|
||||||
|
pTemplateTableName => pTemplateTableName,
|
||||||
|
pPrefix => pPrefix,
|
||||||
|
pBucketUri => pBucketUri,
|
||||||
|
pFileName => pFileName,
|
||||||
|
pDelimiter => pDelimiter
|
||||||
|
);
|
||||||
|
|
||||||
|
CT_MRDS.ENV_MANAGER.LOG_PROCESS_EVENT('End FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE', 'INFO', vParameters);
|
||||||
|
|
||||||
|
EXCEPTION
|
||||||
|
WHEN OTHERS THEN
|
||||||
|
CT_MRDS.ENV_MANAGER.LOG_PROCESS_EVENT(
|
||||||
|
'Error in ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE: ' || SQLERRM,
|
||||||
|
'ERROR',
|
||||||
|
vParameters
|
||||||
|
);
|
||||||
|
RAISE;
|
||||||
|
END CREATE_EXTERNAL_TABLE;
|
||||||
|
|
||||||
|
----------------------------------------------------------------------------------------------------
|
||||||
|
-- PACKAGE VERSION MANAGEMENT FUNCTIONS IMPLEMENTATION
|
||||||
|
----------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
FUNCTION GET_VERSION
|
||||||
|
RETURN VARCHAR2
|
||||||
|
IS
|
||||||
|
BEGIN
|
||||||
|
RETURN PACKAGE_VERSION;
|
||||||
|
END GET_VERSION;
|
||||||
|
|
||||||
|
----------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
FUNCTION GET_BUILD_INFO
|
||||||
|
RETURN VARCHAR2
|
||||||
|
IS
|
||||||
|
BEGIN
|
||||||
|
RETURN CT_MRDS.ENV_MANAGER.GET_PACKAGE_VERSION_INFO(
|
||||||
|
pPackageName => 'FILE_MANAGER_ODS',
|
||||||
|
pVersion => PACKAGE_VERSION,
|
||||||
|
pBuildDate => PACKAGE_BUILD_DATE,
|
||||||
|
pAuthor => PACKAGE_AUTHOR
|
||||||
|
);
|
||||||
|
END GET_BUILD_INFO;
|
||||||
|
|
||||||
|
----------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
FUNCTION GET_VERSION_HISTORY
|
||||||
|
RETURN VARCHAR2
|
||||||
|
IS
|
||||||
|
BEGIN
|
||||||
|
RETURN CT_MRDS.ENV_MANAGER.FORMAT_VERSION_HISTORY(
|
||||||
|
pPackageName => 'FILE_MANAGER_ODS',
|
||||||
|
pVersionHistory => VERSION_HISTORY
|
||||||
|
);
|
||||||
|
END GET_VERSION_HISTORY;
|
||||||
|
|
||||||
|
----------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
END FILE_MANAGER_ODS;
|
||||||
|
|
||||||
|
/
|
||||||
@@ -0,0 +1,91 @@
|
|||||||
|
create or replace PACKAGE ODS.FILE_MANAGER_ODS
|
||||||
|
AUTHID DEFINER
|
||||||
|
AS
|
||||||
|
/**
|
||||||
|
* FILE_MANAGER_ODS - Wrapper package for CT_MRDS.FILE_MANAGER
|
||||||
|
*
|
||||||
|
* This package serves as a wrapper for CT_MRDS.FILE_MANAGER procedures.
|
||||||
|
* The key difference is that this package uses AUTHID DEFINER instead of
|
||||||
|
* AUTHID CURRENT_USER, which means all objects will be created in the ODS
|
||||||
|
* schema regardless of which user executes the procedures.
|
||||||
|
*
|
||||||
|
* This solves the execution context issue where users need to connect as
|
||||||
|
* ODS user to use CT_MRDS.FILE_MANAGER procedures.
|
||||||
|
*/
|
||||||
|
|
||||||
|
-- Package Version Information (Semantic Versioning: MAJOR.MINOR.PATCH)
|
||||||
|
PACKAGE_VERSION CONSTANT VARCHAR2(10) := '2.0.0';
|
||||||
|
PACKAGE_BUILD_DATE CONSTANT VARCHAR2(20) := '2025-10-22 17:15:00';
|
||||||
|
PACKAGE_AUTHOR CONSTANT VARCHAR2(100) := 'Grzegorz Michalski';
|
||||||
|
|
||||||
|
-- Version History (Latest changes first)
|
||||||
|
VERSION_HISTORY CONSTANT VARCHAR2(4000) :=
|
||||||
|
'2.0.0 (2025-10-22): Added package versioning system using centralized ENV_MANAGER functions' || CHR(13)||CHR(10) ||
|
||||||
|
'1.5.0 (2025-10-12): Enhanced external table creation with official path patterns support' || CHR(13)||CHR(10) ||
|
||||||
|
'1.0.0 (2025-09-20): Initial release with AUTHID DEFINER wrapper for external table creation';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name CREATE_EXTERNAL_TABLE
|
||||||
|
* @desc Creates external tables that can read data from Oracle Cloud Storage.
|
||||||
|
* This is a wrapper for CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLE
|
||||||
|
* but uses DEFINER rights to ensure objects are created in ODS schema.
|
||||||
|
* @param pTableName - Name of the external table to create
|
||||||
|
* @param pTemplateTableName - Template table defining the structure
|
||||||
|
* @param pPrefix - Storage path prefix in Oracle Cloud Storage
|
||||||
|
* @param pBucketUri - URI of the target bucket (default: ENV_MANAGER.gvInboxBucketUri)
|
||||||
|
* @param pFileName - Specific file name (optional)
|
||||||
|
* @param pDelimiter - Field delimiter (default: ',')
|
||||||
|
* @example EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
* 'C2D_A_UC_DISSEM_METADATA_LOADS_INBOX',
|
||||||
|
* 'CT_ET_TEMPLATES.C2D_A_UC_DISSEM_METADATA_LOADS',
|
||||||
|
* 'INBOX/C2D/UC_DISSEM/A_UC_DISSEM_METADATA_LOADS',
|
||||||
|
* CT_MRDS.ENV_MANAGER.gvInboxBucketUri
|
||||||
|
* );
|
||||||
|
*/
|
||||||
|
PROCEDURE CREATE_EXTERNAL_TABLE (
|
||||||
|
pTableName IN VARCHAR2,
|
||||||
|
pTemplateTableName IN VARCHAR2,
|
||||||
|
pPrefix IN VARCHAR2,
|
||||||
|
pBucketUri IN VARCHAR2 DEFAULT CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pFileName IN VARCHAR2 DEFAULT NULL,
|
||||||
|
pDelimiter IN VARCHAR2 DEFAULT ','
|
||||||
|
);
|
||||||
|
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
-- PACKAGE VERSION MANAGEMENT FUNCTIONS
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_VERSION
|
||||||
|
* @desc Returns the current version number of the FILE_MANAGER_ODS package.
|
||||||
|
* Uses semantic versioning format (MAJOR.MINOR.PATCH).
|
||||||
|
* @example SELECT FILE_MANAGER_ODS.GET_VERSION() FROM DUAL;
|
||||||
|
* @ex_rslt 2.0.0
|
||||||
|
**/
|
||||||
|
FUNCTION GET_VERSION RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_BUILD_INFO
|
||||||
|
* @desc Returns comprehensive build information including version, build date, and author.
|
||||||
|
* Uses centralized ENV_MANAGER.GET_PACKAGE_VERSION_INFO function.
|
||||||
|
* @example SELECT FILE_MANAGER_ODS.GET_BUILD_INFO() FROM DUAL;
|
||||||
|
* @ex_rslt Package: FILE_MANAGER_ODS
|
||||||
|
* Version: 2.0.0
|
||||||
|
* Build Date: 2025-10-22 17:15:00
|
||||||
|
* Author: Grzegorz Michalski
|
||||||
|
**/
|
||||||
|
FUNCTION GET_BUILD_INFO RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_VERSION_HISTORY
|
||||||
|
* @desc Returns complete version history with all releases and changes.
|
||||||
|
* Uses centralized ENV_MANAGER.FORMAT_VERSION_HISTORY function.
|
||||||
|
* @example SELECT FILE_MANAGER_ODS.GET_VERSION_HISTORY() FROM DUAL;
|
||||||
|
* @ex_rslt FILE_MANAGER_ODS Version History:
|
||||||
|
* 2.0.0 (2025-10-22): Added package versioning system...
|
||||||
|
**/
|
||||||
|
FUNCTION GET_VERSION_HISTORY RETURN VARCHAR2;
|
||||||
|
|
||||||
|
END FILE_MANAGER_ODS;
|
||||||
|
|
||||||
|
/
|
||||||
114
MARS_Packages/REL01/MARS-1049/install_mars1049.sql
Normal file
114
MARS_Packages/REL01/MARS-1049/install_mars1049.sql
Normal file
@@ -0,0 +1,114 @@
|
|||||||
|
-- ===================================================================
|
||||||
|
-- MARS-1049 MAIN INSTALL SCRIPT: CSV Encoding Support
|
||||||
|
-- ===================================================================
|
||||||
|
-- Purpose: Complete installation of CSV encoding support in FILE_MANAGER system
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-24
|
||||||
|
-- Version: 1.1.0
|
||||||
|
|
||||||
|
-- Dynamic spool file generation
|
||||||
|
var filename VARCHAR2(100)
|
||||||
|
BEGIN
|
||||||
|
SELECT 'INSTALL_MARS_1049_' || SUBSTR(PDB_NAME, (INSTR(PDB_NAME,'_',1)+1), (LENGTH(PDB_NAME)-INSTR(PDB_NAME,'_',1))) || '_' ||TO_CHAR(SYSDATE,'YYYYMMDD_HH24MISS')||'.log' INTO :filename from DBA_PDBS;
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
column filename new_value _filename
|
||||||
|
select :filename filename from dual;
|
||||||
|
spool &_filename
|
||||||
|
--
|
||||||
|
-- IMPLEMENTATION STATUS: COMPLETED and DEPLOYED
|
||||||
|
-- - A_SOURCE_FILE_CONFIG: ENCODING VARCHAR2(50) column added
|
||||||
|
-- - CT_MRDS.FILE_MANAGER: Updated to version 3.2.1 with pEncoding parameter
|
||||||
|
-- - ODS.FILE_MANAGER_ODS: Updated to version 2.1.0 with pEncoding wrapper support
|
||||||
|
-- - JSON Implementation: Uses JSON_OBJECT due to Oracle 23c compatibility
|
||||||
|
--
|
||||||
|
-- IMPORTANT: Tests are NOT part of this installation.
|
||||||
|
-- Testing is separate and optional (see test/ directory).
|
||||||
|
--
|
||||||
|
-- DEPLOYMENT ORDER (CRITICAL):
|
||||||
|
-- 0. Grant SELECT ANY DICTIONARY to CT_MRDS (for cross-schema hash tracking)
|
||||||
|
-- 1. Database structure changes (A_SOURCE_FILE_CONFIG.ENCODING column)
|
||||||
|
-- 2. Update existing configurations with default encoding values
|
||||||
|
-- 3. CT_MRDS.FILE_MANAGER package specification
|
||||||
|
-- 4. CT_MRDS.FILE_MANAGER package body
|
||||||
|
-- 5. ODS.FILE_MANAGER_ODS package specification
|
||||||
|
-- 6. ODS.FILE_MANAGER_ODS package body
|
||||||
|
-- 7. Track package versions
|
||||||
|
-- 8. Verify all tracked packages
|
||||||
|
--
|
||||||
|
-- NOTE: Testing is separate from installation.
|
||||||
|
-- For verification and testing, run separately:
|
||||||
|
-- test/05_MARS_1049_verify_encoding_functionality.sql
|
||||||
|
-- ===================================================================
|
||||||
|
|
||||||
|
PROMPT ===============================================
|
||||||
|
PROMPT MARS-1049: CSV Encoding Support - Full Installation
|
||||||
|
PROMPT ===============================================
|
||||||
|
|
||||||
|
-- Check prerequisites
|
||||||
|
PROMPT Checking prerequisites...
|
||||||
|
SELECT USER AS CURRENT_USER FROM DUAL;
|
||||||
|
SELECT COUNT(*) AS TABLE_COUNT FROM ALL_TABLES WHERE OWNER = 'CT_MRDS' AND TABLE_NAME = 'A_SOURCE_FILE_CONFIG';
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT Step 0: Granting permissions for cross-schema hash tracking...
|
||||||
|
PROMPT ===============================================
|
||||||
|
@@00_MARS_1049_grant_permissions.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT Step 1: Installing database structure changes...
|
||||||
|
PROMPT ===============================================
|
||||||
|
@@01_MARS_1049_install_CT_MRDS_ADD_ENCODING_COLUMN.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT Step 2: Updating existing configurations with encoding values...
|
||||||
|
PROMPT ===============================================
|
||||||
|
@@02_MARS_1049_update_CT_MRDS_ENCODING_COLUMN.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT Step 3: Installing CT_MRDS.FILE_MANAGER specification...
|
||||||
|
PROMPT ===============================================
|
||||||
|
@@new_version/FILE_MANAGER.pkg
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT Step 4: Installing CT_MRDS.FILE_MANAGER package body...
|
||||||
|
PROMPT ===============================================
|
||||||
|
@@new_version/FILE_MANAGER.pkb
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT Step 5: Installing ODS.FILE_MANAGER_ODS specification...
|
||||||
|
PROMPT ===============================================
|
||||||
|
@@new_version/FILE_MANAGER_ODS.pkg
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT Step 6: Installing ODS.FILE_MANAGER_ODS package body...
|
||||||
|
PROMPT ===============================================
|
||||||
|
@@new_version/FILE_MANAGER_ODS.pkb
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT Step 7: Tracking package versions...
|
||||||
|
PROMPT ===============================================
|
||||||
|
@@track_package_versions.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT Step 8: Verifying all tracked packages...
|
||||||
|
PROMPT ===============================================
|
||||||
|
@@verify_packages_version.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT ===============================================
|
||||||
|
PROMPT MARS-1049: Installation Complete!
|
||||||
|
PROMPT ===============================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT Summary of Changes:
|
||||||
|
PROMPT - A_SOURCE_FILE_CONFIG: Added ENCODING VARCHAR2(50) column for CSV character set specification
|
||||||
|
PROMPT - A_SOURCE_FILE_CONFIG: Updated existing configurations with default encoding values (UTF8 default, WE8MSWIN1252 for CSDB)
|
||||||
|
PROMPT - CT_MRDS.FILE_MANAGER v3.2.1: Added pEncoding parameter to ADD_SOURCE_FILE_CONFIG and CREATE_EXTERNAL_TABLE
|
||||||
|
PROMPT - ODS.FILE_MANAGER_ODS v2.1.0: Updated wrapper package with pEncoding parameter delegation
|
||||||
|
PROMPT
|
||||||
|
PROMPT Installation completed successfully!
|
||||||
|
PROMPT
|
||||||
|
|
||||||
|
spool off
|
||||||
|
|
||||||
|
exit
|
||||||
1922
MARS_Packages/REL01/MARS-1049/new_version/FILE_MANAGER.pkb
Normal file
1922
MARS_Packages/REL01/MARS-1049/new_version/FILE_MANAGER.pkb
Normal file
File diff suppressed because it is too large
Load Diff
633
MARS_Packages/REL01/MARS-1049/new_version/FILE_MANAGER.pkg
Normal file
633
MARS_Packages/REL01/MARS-1049/new_version/FILE_MANAGER.pkg
Normal file
@@ -0,0 +1,633 @@
|
|||||||
|
create or replace PACKAGE CT_MRDS.FILE_MANAGER
|
||||||
|
AUTHID CURRENT_USER
|
||||||
|
AS
|
||||||
|
/**
|
||||||
|
* General comment for package: Please put comments for functions and procedures as shown in below example.
|
||||||
|
* It is a standard.
|
||||||
|
* The structure of comment is used by GET_PACKAGE_DOCUMENTATION function
|
||||||
|
* which returns documentation text for confluence page (to Copy-Paste it).
|
||||||
|
**/
|
||||||
|
|
||||||
|
-- Example comment:
|
||||||
|
/**
|
||||||
|
* @name EX_PROCEDURE_NAME
|
||||||
|
* @desc Procedure description
|
||||||
|
* @example select FILE_MANAGER.EX_PROCEDURE_NAME(pParameter => 129) from dual;
|
||||||
|
* @ex_rslt Example Result
|
||||||
|
**/
|
||||||
|
|
||||||
|
-- Package Version Information (Semantic Versioning: MAJOR.MINOR.PATCH)
|
||||||
|
PACKAGE_VERSION CONSTANT VARCHAR2(10) := '3.2.1';
|
||||||
|
PACKAGE_BUILD_DATE CONSTANT VARCHAR2(20) := '2025-11-24 12:00:00';
|
||||||
|
PACKAGE_AUTHOR CONSTANT VARCHAR2(100) := 'Grzegorz Michalski';
|
||||||
|
|
||||||
|
-- Version History (Latest changes first)
|
||||||
|
VERSION_HISTORY CONSTANT VARCHAR2(4000) :=
|
||||||
|
'3.2.1 (2025-11-24): MARS-1049 - Added pEncoding parameter support for CSV character set specification' || CHR(13)||CHR(10) ||
|
||||||
|
'3.2.0 (2025-10-22): Added package versioning system using centralized ENV_MANAGER functions' || CHR(13)||CHR(10) ||
|
||||||
|
'3.1.0 (2025-10-20): Enhanced PROCESS_SOURCE_FILE with 6-step validation workflow' || CHR(13)||CHR(10) ||
|
||||||
|
'3.0.0 (2025-10-15): Separated export procedures into dedicated DATA_EXPORTER package' || CHR(13)||CHR(10) ||
|
||||||
|
'2.5.0 (2025-10-10): Added DELETE_SOURCE_CASCADE for safe configuration removal' || CHR(13)||CHR(10) ||
|
||||||
|
'2.0.0 (2025-09-25): Added official path patterns support (INBOX 3-level, ODS 2-level, ARCHIVE 2-level)' || CHR(13)||CHR(10) ||
|
||||||
|
'1.0.0 (2025-09-01): Initial release with file processing and validation capabilities';
|
||||||
|
|
||||||
|
TYPE tSourceFileReceived IS RECORD
|
||||||
|
(
|
||||||
|
A_SOURCE_FILE_RECEIVED_KEY CT_MRDS.A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY%TYPE,
|
||||||
|
A_SOURCE_FILE_CONFIG_KEY CT_MRDS.A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_CONFIG_KEY%TYPE,
|
||||||
|
SOURCE_FILE_PREFIX_INBOX VARCHAR2(430),
|
||||||
|
SOURCE_FILE_PREFIX_ODS VARCHAR2(430),
|
||||||
|
SOURCE_FILE_PREFIX_QUARANTINE VARCHAR2(430),
|
||||||
|
SOURCE_FILE_PREFIX_ARCHIVE VARCHAR2(430),
|
||||||
|
SOURCE_FILE_NAME CT_MRDS.A_SOURCE_FILE_RECEIVED.SOURCE_FILE_NAME%TYPE,
|
||||||
|
RECEPTION_DATE CT_MRDS.A_SOURCE_FILE_RECEIVED.RECEPTION_DATE%TYPE,
|
||||||
|
PROCESSING_STATUS CT_MRDS.A_SOURCE_FILE_RECEIVED.PROCESSING_STATUS%TYPE,
|
||||||
|
EXTERNAL_TABLE_NAME CT_MRDS.A_SOURCE_FILE_RECEIVED.EXTERNAL_TABLE_NAME%TYPE
|
||||||
|
);
|
||||||
|
|
||||||
|
cgBL CONSTANT VARCHAR2(2) := CHR(13)||CHR(10);
|
||||||
|
vgSourceFileConfigKey PLS_INTEGER;
|
||||||
|
vgMsgTmp VARCHAR2(32000);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_SOURCE_FILE_CONFIG
|
||||||
|
* @desc Get source file type by matching the source file name against source file type naming patterns
|
||||||
|
* or by specifying the id of a received source file.
|
||||||
|
* @example ...
|
||||||
|
* @ex_rslt "CT_MRDS.A_SOURCE_FILE_CONFIG%ROWTYPE"
|
||||||
|
**/
|
||||||
|
FUNCTION GET_SOURCE_FILE_CONFIG(pFileUri IN VARCHAR2 DEFAULT NULL
|
||||||
|
, pSourceFileReceivedKey IN NUMBER DEFAULT NULL
|
||||||
|
, pSourceFileConfigKey IN NUMBER DEFAULT NULL)
|
||||||
|
RETURN CT_MRDS.A_SOURCE_FILE_CONFIG%ROWTYPE;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name REGISTER_SOURCE_FILE_RECEIVED
|
||||||
|
* @desc Register a newly received source file in A_SOURCE_FILE_RECEIVED table.
|
||||||
|
* This overload automatically determines source file type from the file name.
|
||||||
|
* It returns the value of A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY column for newly added record.
|
||||||
|
* @example vSourceFileReceivedKey := FILE_MANAGER.REGISTER_SOURCE_FILE_RECEIVED(pSourceFileReceivedName => 'INBOX/C2D/UC_DISSEM/UC_NMA_DISSEM/UC_NMA_DISSEM-277740.csv');
|
||||||
|
* @ex_rslt 3245
|
||||||
|
**/
|
||||||
|
FUNCTION REGISTER_SOURCE_FILE_RECEIVED (
|
||||||
|
pSourceFileReceivedName IN VARCHAR2
|
||||||
|
)
|
||||||
|
RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name REGISTER_SOURCE_FILE_RECEIVED
|
||||||
|
* @desc Register a new new source file in A_SOURCE_FILE_RECEIVED table based on pSourceFileReceivedName and pSourceFileConfig.
|
||||||
|
* Then it returns the value of A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY column for newly added record.
|
||||||
|
* @example vSourceFileReceivedKey := FILE_MANAGER.REGISTER_SOURCE_FILE_RECEIVED(
|
||||||
|
* pSourceFileReceivedName => 'INBOX/C2D/UC_DISSEM/UC_NMA_DISSEM/UC_NMA_DISSEM-277740.csv'
|
||||||
|
* ,pSourceFileConfig => ...A_SOURCE_FILE_CONFIG%ROWTYPE... );
|
||||||
|
* @ex_rslt 3245
|
||||||
|
**/
|
||||||
|
FUNCTION REGISTER_SOURCE_FILE_RECEIVED (
|
||||||
|
pSourceFileReceivedName IN VARCHAR2,
|
||||||
|
pSourceFileConfig IN CT_MRDS.A_SOURCE_FILE_CONFIG%ROWTYPE
|
||||||
|
)
|
||||||
|
RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name SET_SOURCE_FILE_RECEIVED_STATUS
|
||||||
|
* @desc Set status of file in A_SOURCE_FILE_RECEIVED table - PROCESSING_STATUS column
|
||||||
|
* based on A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY
|
||||||
|
* and provided value of pStatus parameter
|
||||||
|
* @example exec FILE_MANAGER.SET_SOURCE_FILE_RECEIVED_STATUS(pSourceFileReceivedKey => 377, pStatus => 'READY_FOR_INGESTION');
|
||||||
|
**/
|
||||||
|
PROCEDURE SET_SOURCE_FILE_RECEIVED_STATUS(
|
||||||
|
pSourceFileReceivedKey IN PLS_INTEGER,
|
||||||
|
pStatus IN VARCHAR2
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_EXTERNAL_TABLE_COLUMNS
|
||||||
|
* @desc Function used to get string with all table columns definitions based on pTargetTableTemplate "TEMPLATE TABLE" name.
|
||||||
|
* It used for creating "EXTERNAL TABLE" using CREATE_EXTERNAL_TABLE procedure.
|
||||||
|
* @example select FILE_MANAGER.GET_EXTERNAL_TABLE_COLUMNS(pTargetTableTemplate => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES_HEADER') from dual;
|
||||||
|
* @ex_rslt "A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
* "A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
* "REV_NUMBER" NUMBER(28,0),
|
||||||
|
* "REF_DATE" DATE,
|
||||||
|
* "FREE_TEXT" VARCHAR2(1000 CHAR),
|
||||||
|
* "MLF_BS_TOTAL" NUMBER(28,10),
|
||||||
|
* "DF_BS_TOTAL" NUMBER(28,10),
|
||||||
|
* "MLF_SF_TOTAL" NUMBER(28,10),
|
||||||
|
* "DF_SF_TOTAL" NUMBER(28,10)
|
||||||
|
**/
|
||||||
|
FUNCTION GET_EXTERNAL_TABLE_COLUMNS (
|
||||||
|
pTargetTableTemplate IN VARCHAR2
|
||||||
|
)
|
||||||
|
RETURN CLOB;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name CREATE_EXTERNAL_TABLE
|
||||||
|
* @desc A wrapper procedure for DBMS_CLOUD.CREATE_EXTERNAL_TABLE which creates External Table
|
||||||
|
* MARS-1049: Added pEncoding parameter for CSV character set specification
|
||||||
|
* @param pEncoding - Character set encoding for CSV files (e.g., 'UTF8', 'WE8MSWIN1252')
|
||||||
|
* If provided, adds CHARACTERSET clause to external table definition
|
||||||
|
* @example
|
||||||
|
* begin
|
||||||
|
* FILE_MANAGER.CREATE_EXTERNAL_TABLE(
|
||||||
|
* pTableName => 'STANDING_FACILITIES_HEADER',
|
||||||
|
* pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES_HEADER',
|
||||||
|
* pPrefix => 'ODS/LM/STANDING_FACILITIES_HEADER/',
|
||||||
|
* pBucketUri => 'https://objectstorage.eu-frankfurt-1.oraclecloud.com/n/frcnomajoc7v/b/mrds_data_tst/o/',
|
||||||
|
* pFileName => NULL,
|
||||||
|
* pDelimiter => ',',
|
||||||
|
* pEncoding => 'UTF8'
|
||||||
|
* );
|
||||||
|
* end;
|
||||||
|
**/
|
||||||
|
PROCEDURE CREATE_EXTERNAL_TABLE (
|
||||||
|
pTableName IN VARCHAR2,
|
||||||
|
pTemplateTableName IN VARCHAR2,
|
||||||
|
pPrefix IN VARCHAR2,
|
||||||
|
pBucketUri IN VARCHAR2 DEFAULT ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pFileName IN VARCHAR2 DEFAULT NULL,
|
||||||
|
pDelimiter IN VARCHAR2 DEFAULT ',',
|
||||||
|
pEncoding IN VARCHAR2 DEFAULT NULL -- MARS-1049: NOWY PARAMETR
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name CREATE_EXTERNAL_TABLE
|
||||||
|
* @desc Creates External Table for single file provided by
|
||||||
|
* pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* @example exec FILE_MANAGER.CREATE_EXTERNAL_TABLE(pSourceFileReceivedKey => 377);;
|
||||||
|
**/
|
||||||
|
PROCEDURE CREATE_EXTERNAL_TABLE (
|
||||||
|
pSourceFileReceivedKey IN NUMBER
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name VALIDATE_SOURCE_FILE_RECEIVED
|
||||||
|
* @desc A wrapper procedure for DBMS_CLOUD.VALIDATE_EXTERNAL_TABLE
|
||||||
|
* It validate External table build upon single file
|
||||||
|
* provided by pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* @example exec FILE_MANAGER.VALIDATE_SOURCE_FILE_RECEIVED(pSourceFileReceivedKey => 377);
|
||||||
|
**/
|
||||||
|
PROCEDURE VALIDATE_SOURCE_FILE_RECEIVED
|
||||||
|
(
|
||||||
|
pSourceFileReceivedKey IN NUMBER
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name VALIDATE_EXTERNAL_TABLE
|
||||||
|
* @desc A wrapper function for DBMS_CLOUD.VALIDATE_EXTERNAL_TABLE.
|
||||||
|
* It validates External Table provided by parameter pTableName.
|
||||||
|
* It returns: PASSED or FAILED.
|
||||||
|
* @example
|
||||||
|
* declare
|
||||||
|
* vStatus VARCHAR2(100);
|
||||||
|
* begin
|
||||||
|
* vStatus := FILE_MANAGER.VALIDATE_EXTERNAL_TABLE(pTableName => 'STANDING_FACILITIES_HEADER');
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vStatus = '||vStatus);
|
||||||
|
* end;
|
||||||
|
*
|
||||||
|
* @ex_rslt FAILED
|
||||||
|
**/
|
||||||
|
FUNCTION VALIDATE_EXTERNAL_TABLE(pTableName IN VARCHAR2)
|
||||||
|
RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name S_VALIDATE_EXTERNAL_TABLE
|
||||||
|
* @desc A function which checks if SELECT query reterns any rows.
|
||||||
|
* It trys to selects External Table provided by parameter pTableName.
|
||||||
|
* It returns: PASSED or FAILED.
|
||||||
|
* @example
|
||||||
|
* declare
|
||||||
|
* vStatus VARCHAR2(100);
|
||||||
|
* begin
|
||||||
|
* vStatus := FILE_MANAGER.S_VALIDATE_EXTERNAL_TABLE(pTableName => 'STANDING_FACILITIES_HEADER');
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vStatus = '||vStatus);
|
||||||
|
* end;
|
||||||
|
*
|
||||||
|
* @ex_rslt PASSED
|
||||||
|
**/
|
||||||
|
FUNCTION S_VALIDATE_EXTERNAL_TABLE(pTableName IN VARCHAR2)
|
||||||
|
RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name DROP_EXTERNAL_TABLE
|
||||||
|
* @desc It drops External Table for single file provided by
|
||||||
|
* pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* @example exec FILE_MANAGER.DROP_EXTERNAL_TABLE(pSourceFileReceivedKey => 377);
|
||||||
|
**/
|
||||||
|
PROCEDURE DROP_EXTERNAL_TABLE (
|
||||||
|
pSourceFileReceivedKey IN NUMBER
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name COPY_FILE
|
||||||
|
* @desc It copies file provided by
|
||||||
|
* pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* into destination provided by pDestination parameter.
|
||||||
|
* pDestination parameter allowed values are: 'ODS'
|
||||||
|
* @example exec FILE_MANAGER.COPY_FILE(pSourceFileReceivedKey => 377, pDestination => 'ODS');
|
||||||
|
**/
|
||||||
|
PROCEDURE COPY_FILE(
|
||||||
|
pSourceFileReceivedKey IN NUMBER,
|
||||||
|
pDestination IN VARCHAR2
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name MOVE_FILE
|
||||||
|
* @desc It moves file provided by
|
||||||
|
* pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* into destination provided by pDestination parameter.
|
||||||
|
* pDestination parameter allowed values are: 'ODS', 'QUARANTINE'
|
||||||
|
* @example exec FILE_MANAGER.MOVE_FILE(pSourceFileReceivedKey => 377, pDestination => 'ODS');
|
||||||
|
**/
|
||||||
|
PROCEDURE MOVE_FILE(
|
||||||
|
pSourceFileReceivedKey IN NUMBER,
|
||||||
|
pDestination IN VARCHAR2
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name DELETE_FOLDER_CONTENTS
|
||||||
|
* @desc It deletes all files from specified folder in the cloud storage.
|
||||||
|
* The procedure lists all objects in the specified folder prefix and deletes them one by one.
|
||||||
|
* pBucketArea parameter specifies which bucket to use: 'INBOX', 'DATA', 'ARCHIVE'
|
||||||
|
* pFolderPrefix parameter specifies the folder path within the bucket (e.g., 'C2D/UC_DISSEM/UC_NMA_DISSEM/')
|
||||||
|
* @example exec FILE_MANAGER.DELETE_FOLDER_CONTENTS(pBucketArea => 'INBOX', pFolderPrefix => 'C2D/UC_DISSEM/UC_NMA_DISSEM/');
|
||||||
|
**/
|
||||||
|
PROCEDURE DELETE_FOLDER_CONTENTS(
|
||||||
|
pBucketArea IN VARCHAR2,
|
||||||
|
pFolderPrefix IN VARCHAR2
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name PROCESS_SOURCE_FILE
|
||||||
|
* @desc It process file provided by pSourceFileReceivedName parameter.
|
||||||
|
* Ubmrella procedure that calls:
|
||||||
|
* - REGISTER_SOURCE_FILE_RECEIVED;
|
||||||
|
* - CREATE_EXTERNAL_TABLE;
|
||||||
|
* - VALIDATE_SOURCE_FILE_RECEIVED;
|
||||||
|
* - DROP_EXTERNAL_TABLE;
|
||||||
|
* - MOVE_FILE;
|
||||||
|
* @example exec FILE_MANAGER.PROCESS_SOURCE_FILE(pSourceFileReceivedName => 'INBOX/C2D/UC_DISSEM/UC_NMA_DISSEM/UC_NMA_DISSEM-277740.csv');
|
||||||
|
**/
|
||||||
|
PROCEDURE PROCESS_SOURCE_FILE(pSourceFileReceivedName IN VARCHAR2)
|
||||||
|
;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name PROCESS_SOURCE_FILE
|
||||||
|
* @desc It process file provided by pSourceFileReceivedName parameter and return processing result value.
|
||||||
|
* It returns (success/failure) => 0 / -(value).
|
||||||
|
* Ubmrella function that calls PROCESS_SOURCE_FILE procedure.
|
||||||
|
* @example
|
||||||
|
* declare
|
||||||
|
* vResult PLS_INTEGER;
|
||||||
|
* begin
|
||||||
|
* vResult := CT_MRDS.FILE_MANAGER.PROCESS_SOURCE_FILE(PSOURCEFILERECEIVEDNAME => 'INBOX/C2D/UC_DISSEM/UC_NMA_DISSEM/UC_NMA_DISSEM-277740.csv');
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vResult = ' || vResult);
|
||||||
|
* end;
|
||||||
|
* @ex_rslt 0
|
||||||
|
* -20021
|
||||||
|
**/
|
||||||
|
FUNCTION PROCESS_SOURCE_FILE(pSourceFileReceivedName IN VARCHAR2)
|
||||||
|
RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_DATE_FORMAT
|
||||||
|
* @desc Returns date format for specified template table name and column name.
|
||||||
|
* Date is taken from configuration A_COLUMN_DATE_FORMAT table.
|
||||||
|
* @example select FILE_MANAGER.GET_DATE_FORMAT(
|
||||||
|
* pTemplateTableName => 'STANDING_FACILITIES_HEADER',
|
||||||
|
* pColumnName => 'SNAPSHOT_DATE')
|
||||||
|
* from dual;
|
||||||
|
* @ex_rslt DD/MM/YYYY HH24:MI:SS
|
||||||
|
**/
|
||||||
|
FUNCTION GET_DATE_FORMAT(
|
||||||
|
pTemplateTableName IN VARCHAR2,
|
||||||
|
pColumnName IN VARCHAR2
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GENERATE_EXTERNAL_TABLE_PARAMS
|
||||||
|
* @desc It builds two strings: pColumnList and pFieldList for specified Template Table name, by parameter: pTemplateTableName.
|
||||||
|
* @example
|
||||||
|
* declare
|
||||||
|
* vColumnList CLOB;
|
||||||
|
* vFieldList CLOB;
|
||||||
|
* begin
|
||||||
|
* FILE_MANAGER.GENERATE_EXTERNAL_TABLE_PARAMS (
|
||||||
|
* pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES_HEADER'
|
||||||
|
* ,pColumnList => vColumnList
|
||||||
|
* ,pFieldList => vFieldList
|
||||||
|
* );
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vColumnList = '||vColumnList);
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vFieldList = '||vFieldList);
|
||||||
|
* end;
|
||||||
|
* /
|
||||||
|
**/
|
||||||
|
PROCEDURE GENERATE_EXTERNAL_TABLE_PARAMS (
|
||||||
|
pTemplateTableName IN VARCHAR2,
|
||||||
|
pColumnList OUT CLOB,
|
||||||
|
pFieldList OUT CLOB
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name ADD_SOURCE
|
||||||
|
* @desc Insert a new record to A_SOURCE table.
|
||||||
|
* pSourceKey is a PRIMARY KEY value.
|
||||||
|
**/
|
||||||
|
PROCEDURE ADD_SOURCE (
|
||||||
|
pSourceKey IN CT_MRDS.A_SOURCE.A_SOURCE_KEY%TYPE,
|
||||||
|
pSourceName IN CT_MRDS.A_SOURCE.SOURCE_NAME%TYPE
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name DELETE_SOURCE_CASCADE
|
||||||
|
* @desc Safely deletes a SOURCE specified by pSourceKey parameter from A_SOURCE table and all dependent tables:
|
||||||
|
* - A_SOURCE_FILE_CONFIG
|
||||||
|
* - A_SOURCE_FILE_RECEIVED
|
||||||
|
* - A_COLUMN_DATE_FORMAT (only if template table is not shared with other source systems)
|
||||||
|
* The procedure checks if template tables are shared before deleting date format configurations.
|
||||||
|
* If a template table is used by multiple source systems, date formats are preserved.
|
||||||
|
* @example CALL CT_MRDS.FILE_MANAGER.DELETE_SOURCE_CASCADE(pSourceKey => 'TEST_SYS');
|
||||||
|
**/
|
||||||
|
PROCEDURE DELETE_SOURCE_CASCADE (
|
||||||
|
pSourceKey IN CT_MRDS.A_SOURCE.A_SOURCE_KEY%TYPE
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_CONTAINER_SOURCE_FILE_CONFIG_KEY
|
||||||
|
* @desc For specified parameter pSourceFileId (A_SOURCE_FILE_CONFIG.SOURCE_FILE_ID)
|
||||||
|
* it returns A_SOURCE_FILE_CONFIG.A_SOURCE_FILE_CONFIG_KEY for related CONTAINER record.
|
||||||
|
* @example select FILE_MANAGER.GET_CONTAINER_SOURCE_FILE_CONFIG_KEY(
|
||||||
|
* pSourceFileId => 'UC_DISSEM')
|
||||||
|
* from dual;
|
||||||
|
* @ex_rslt 126
|
||||||
|
**/
|
||||||
|
FUNCTION GET_CONTAINER_SOURCE_FILE_CONFIG_KEY (
|
||||||
|
pSourceFileId IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_ID%TYPE
|
||||||
|
) RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_SOURCE_FILE_CONFIG_KEY
|
||||||
|
* @desc For specified input parameters,
|
||||||
|
* it returns A_SOURCE_FILE_CONFIG.A_SOURCE_FILE_CONFIG_KEY.
|
||||||
|
* @example select FILE_MANAGER.GET_SOURCE_FILE_CONFIG_KEY (
|
||||||
|
* pSourceFileType => 'INPUT'
|
||||||
|
* ,pSourceFileId => 'UC_DISSEM'
|
||||||
|
* ,pTableId => 'UC_NMA_DISSEM')
|
||||||
|
* from dual;
|
||||||
|
* @ex_rslt 126
|
||||||
|
**/
|
||||||
|
FUNCTION GET_SOURCE_FILE_CONFIG_KEY (
|
||||||
|
pSourceFileType IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_TYPE%TYPE DEFAULT 'INPUT'
|
||||||
|
,pSourceFileId IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_ID%TYPE
|
||||||
|
,pTableId IN CT_MRDS.A_SOURCE_FILE_CONFIG.TABLE_ID%TYPE
|
||||||
|
) RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name ADD_SOURCE_FILE_CONFIG
|
||||||
|
* @desc Insert a new record to A_SOURCE_FILE_CONFIG table.
|
||||||
|
* MARS-1049: Added pEncoding parameter for CSV character set specification.
|
||||||
|
* @param pEncoding - Character set encoding for CSV files (e.g., 'UTF8', 'WE8MSWIN1252', 'EE8ISO8859P2')
|
||||||
|
* If NULL, no CHARACTERSET clause is added to external table definitions
|
||||||
|
* @example CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
* pSourceKey => 'C2D', pSourceFileType => 'INPUT',
|
||||||
|
* pSourceFileId => 'UC_DISSEM', pTableId => 'METADATA_LOADS',
|
||||||
|
* pTemplateTableName => 'CT_ET_TEMPLATES.C2D_A_UC_DISSEM_METADATA_LOADS',
|
||||||
|
* pEncoding => 'UTF8'
|
||||||
|
* );
|
||||||
|
**/
|
||||||
|
PROCEDURE ADD_SOURCE_FILE_CONFIG (
|
||||||
|
pSourceKey IN CT_MRDS.A_SOURCE_FILE_CONFIG.A_SOURCE_KEY%TYPE
|
||||||
|
,pSourceFileType IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_TYPE%TYPE
|
||||||
|
,pSourceFileId IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_ID%TYPE
|
||||||
|
,pSourceFileDesc IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_DESC%TYPE
|
||||||
|
,pSourceFileNamePattern IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_NAME_PATTERN%TYPE
|
||||||
|
,pTableId IN CT_MRDS.A_SOURCE_FILE_CONFIG.TABLE_ID%TYPE DEFAULT NULL
|
||||||
|
,pTemplateTableName IN CT_MRDS.A_SOURCE_FILE_CONFIG.TEMPLATE_TABLE_NAME%TYPE DEFAULT NULL
|
||||||
|
,pContainerFileKey IN CT_MRDS.A_SOURCE_FILE_CONFIG.CONTAINER_FILE_KEY%TYPE DEFAULT NULL
|
||||||
|
,pEncoding IN CT_MRDS.A_SOURCE_FILE_CONFIG.ENCODING%TYPE DEFAULT NULL -- MARS-1049: NOWY PARAMETR
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name ADD_COLUMN_DATE_FORMAT
|
||||||
|
* @desc Insert a new record to A_COLUMN_DATE_FORMAT table.
|
||||||
|
**/
|
||||||
|
PROCEDURE ADD_COLUMN_DATE_FORMAT (
|
||||||
|
pTemplateTableName IN CT_MRDS.A_COLUMN_DATE_FORMAT.TEMPLATE_TABLE_NAME%TYPE
|
||||||
|
,pColumnName IN CT_MRDS.A_COLUMN_DATE_FORMAT.COLUMN_NAME%TYPE
|
||||||
|
,pDateFormat IN CT_MRDS.A_COLUMN_DATE_FORMAT.DATE_FORMAT%TYPE
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_BUCKET_URI
|
||||||
|
* @desc Function used to get string with bucket http url.
|
||||||
|
* Possible input values for pBucketArea are: 'INBOX', 'ODS', 'DATA', 'ARCHIVE'
|
||||||
|
* @example select FILE_MANAGER.GET_BUCKET_URI(pBucketArea => 'ODS') from dual;
|
||||||
|
* @ex_rslt https://objectstorage.eu-frankfurt-1.oraclecloud.com/n/frcnomajoc7v/b/mrds_data_tst/o/
|
||||||
|
**/
|
||||||
|
FUNCTION GET_BUCKET_URI(pBucketArea VARCHAR2)
|
||||||
|
RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_DET_SOURCE_FILE_CONFIG_INFO
|
||||||
|
* @desc Function returns details about A_SOURCE_FILE_CONFIG record
|
||||||
|
* for specified pSourceFileConfigKey (A_SOURCE_FILE_CONFIG.A_SOURCE_FILE_CONFIG_KEY).
|
||||||
|
* If pIncludeContainerInfo is <> 0 it returns additional info about related Container config record (A_SOURCE_FILE_CONFIG)
|
||||||
|
* If pIncludeColumnFormatInfo is <> 0 it returns additional info about related ColumnFormat config record (A_COLUMN_DATE_FORMAT)
|
||||||
|
* @example select FILE_MANAGER.GET_DET_SOURCE_FILE_CONFIG_INFO (
|
||||||
|
* pSourceFileConfigKey => 128
|
||||||
|
* ,pIncludeContainerInfo => 1
|
||||||
|
* ,pIncludeColumnFormatInfo => 1
|
||||||
|
* ) from dual;
|
||||||
|
* @ex_rslt
|
||||||
|
* Details about File Configuration:
|
||||||
|
* --------------------------------
|
||||||
|
* A_SOURCE_FILE_CONFIG_KEY = 128
|
||||||
|
* A_SOURCE_KEY = C2D
|
||||||
|
* ...
|
||||||
|
* --------------------------------
|
||||||
|
*
|
||||||
|
* Details about related Container Config:
|
||||||
|
* --------------------------------
|
||||||
|
* A_SOURCE_FILE_CONFIG_KEY = 126
|
||||||
|
* A_SOURCE_KEY = C2D
|
||||||
|
* ...
|
||||||
|
* --------------------------------
|
||||||
|
*
|
||||||
|
* Column Date Format config entries:
|
||||||
|
* --------------------------------
|
||||||
|
* TEMPLATE_TABLE_NAME = CT_ET_TEMPLATES.C2D_UC_MA_DISSEM
|
||||||
|
* ...
|
||||||
|
* --------------------------------
|
||||||
|
**/
|
||||||
|
FUNCTION GET_DET_SOURCE_FILE_CONFIG_INFO (
|
||||||
|
pSourceFileConfigKey IN CT_MRDS.A_SOURCE_FILE_CONFIG.A_SOURCE_FILE_CONFIG_KEY%TYPE
|
||||||
|
,pIncludeContainerInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
,pIncludeColumnFormatInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_DET_SOURCE_FILE_RECEIVED_INFO
|
||||||
|
* @desc Function returns details about A_SOURCE_FILE_RECEIVED record
|
||||||
|
* for specified pSourceFileReceivedKey (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY).
|
||||||
|
* If pIncludeConfigInfo is <> 0 it returns additional info about related Container config record (A_SOURCE_FILE_CONFIG)
|
||||||
|
* If pIncludeContainerInfo is <> 0 it returns additional info about related Container config record (A_SOURCE_FILE_CONFIG)
|
||||||
|
* If pIncludeColumnFormatInfo is <> 0 it returns additional info about related ColumnFormat config record (A_COLUMN_DATE_FORMAT)
|
||||||
|
* @example select FILE_MANAGER.GET_DET_SOURCE_FILE_RECEIVED_INFO (
|
||||||
|
* pSourceFileReceivedKey => 377
|
||||||
|
* ,pIncludeConfigInfo => 1
|
||||||
|
* ,pIncludeContainerInfo => 1
|
||||||
|
* ,pIncludeColumnFormatInfo => 1
|
||||||
|
* ) from dual;
|
||||||
|
*
|
||||||
|
**/
|
||||||
|
FUNCTION GET_DET_SOURCE_FILE_RECEIVED_INFO (
|
||||||
|
pSourceFileReceivedKey IN CT_MRDS.A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY%TYPE
|
||||||
|
,pIncludeConfigInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
,pIncludeContainerInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
,pIncludeColumnFormatInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_DET_USER_LOAD_OPERATIONS
|
||||||
|
* @desc Function returns details from USER_LOAD_OPERATIONS table
|
||||||
|
* for specified pOperationId.
|
||||||
|
* @example select FILE_MANAGER.GET_DET_USER_LOAD_OPERATIONS (pOperationId => 3608) from dual;
|
||||||
|
* @ex_rslt
|
||||||
|
* Details about USER_LOAD_OPERATIONS where ID = 3608
|
||||||
|
* --------------------------------
|
||||||
|
* ID = 3608
|
||||||
|
* TYPE = VALIDATE
|
||||||
|
* SID = 31260
|
||||||
|
* SERIAL# = 52915
|
||||||
|
* START_TIME = 2025-05-20 10.08.24.436983 EUROPE/BELGRADE
|
||||||
|
* UPDATE_TIME = 2025-05-20 10.08.24.458643 EUROPE/BELGRADE
|
||||||
|
* STATUS = FAILED
|
||||||
|
* OWNER_NAME = CT_MRDS
|
||||||
|
* TABLE_NAME = STANDING_FACILITIES_HEADER
|
||||||
|
* PARTITION_NAME =
|
||||||
|
* SUBPARTITION_NAME =
|
||||||
|
* FILE_URI_LIST =
|
||||||
|
* ROWS_LOADED =
|
||||||
|
* LOGFILE_TABLE = VALIDATE$3608_LOG
|
||||||
|
* BADFILE_TABLE = VALIDATE$3608_BAD
|
||||||
|
* STATUS_TABLE =
|
||||||
|
* TEMPEXT_TABLE =
|
||||||
|
* CREDENTIAL_NAME =
|
||||||
|
* EXPIRATION_TIME = 2025-05-22 10.08.24.436983000 EUROPE/BELGRADE
|
||||||
|
* --------------------------------
|
||||||
|
**/
|
||||||
|
FUNCTION GET_DET_USER_LOAD_OPERATIONS (
|
||||||
|
pOperationId PLS_INTEGER
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name ANALYZE_VALIDATION_ERRORS
|
||||||
|
* @desc Wrapper function that analyzes validation errors for a source file using its received key.
|
||||||
|
* Automatically derives template schema, table name, CSV URI and validation log table
|
||||||
|
* from file metadata and calls ENV_MANAGER.ANALYZE_VALIDATION_ERRORS.
|
||||||
|
* @example SELECT FILE_MANAGER.ANALYZE_VALIDATION_ERRORS(63) FROM DUAL;
|
||||||
|
* @ex_rslt Detailed validation analysis report with column mismatches and solutions
|
||||||
|
**/
|
||||||
|
FUNCTION ANALYZE_VALIDATION_ERRORS(
|
||||||
|
pSourceFileReceivedKey IN NUMBER
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
-- PACKAGE VERSION MANAGEMENT FUNCTIONS
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_VERSION
|
||||||
|
* @desc Returns the current version number of the FILE_MANAGER package.
|
||||||
|
* Uses semantic versioning format (MAJOR.MINOR.PATCH).
|
||||||
|
* @example SELECT FILE_MANAGER.GET_VERSION() FROM DUAL;
|
||||||
|
* @ex_rslt 3.2.0
|
||||||
|
**/
|
||||||
|
FUNCTION GET_VERSION RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_BUILD_INFO
|
||||||
|
* @desc Returns comprehensive build information including version, build date, and author.
|
||||||
|
* Uses centralized ENV_MANAGER.GET_PACKAGE_VERSION_INFO function.
|
||||||
|
* @example SELECT FILE_MANAGER.GET_BUILD_INFO() FROM DUAL;
|
||||||
|
* @ex_rslt Package: FILE_MANAGER
|
||||||
|
* Version: 3.2.0
|
||||||
|
* Build Date: 2025-10-22 16:30:00
|
||||||
|
* Author: Grzegorz Michalski
|
||||||
|
**/
|
||||||
|
FUNCTION GET_BUILD_INFO RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_VERSION_HISTORY
|
||||||
|
* @desc Returns complete version history with all releases and changes.
|
||||||
|
* Uses centralized ENV_MANAGER.FORMAT_VERSION_HISTORY function.
|
||||||
|
* @example SELECT FILE_MANAGER.GET_VERSION_HISTORY() FROM DUAL;
|
||||||
|
* @ex_rslt FILE_MANAGER Version History:
|
||||||
|
* 3.2.0 (2025-10-22): Added package versioning system...
|
||||||
|
**/
|
||||||
|
FUNCTION GET_VERSION_HISTORY RETURN VARCHAR2;
|
||||||
|
|
||||||
|
END;
|
||||||
|
|
||||||
|
/
|
||||||
118
MARS_Packages/REL01/MARS-1049/new_version/FILE_MANAGER_ODS.pkb
Normal file
118
MARS_Packages/REL01/MARS-1049/new_version/FILE_MANAGER_ODS.pkb
Normal file
@@ -0,0 +1,118 @@
|
|||||||
|
create or replace PACKAGE BODY ODS.FILE_MANAGER_ODS
|
||||||
|
AS
|
||||||
|
|
||||||
|
/**
|
||||||
|
* CREATE_EXTERNAL_TABLE - Wrapper for CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLE
|
||||||
|
*/
|
||||||
|
PROCEDURE CREATE_EXTERNAL_TABLE (
|
||||||
|
pTableName IN VARCHAR2,
|
||||||
|
pTemplateTableName IN VARCHAR2,
|
||||||
|
pPrefix IN VARCHAR2,
|
||||||
|
pBucketUri IN VARCHAR2 DEFAULT CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pFileName IN VARCHAR2 DEFAULT NULL,
|
||||||
|
pDelimiter IN VARCHAR2 DEFAULT ',',
|
||||||
|
pEncoding IN VARCHAR2 DEFAULT NULL
|
||||||
|
)
|
||||||
|
IS
|
||||||
|
vParameters CT_MRDS.A_PROCESS_LOG.PROCEDURE_PARAMETERS%TYPE;
|
||||||
|
BEGIN
|
||||||
|
-- Log the start of the procedure
|
||||||
|
vParameters := CT_MRDS.ENV_MANAGER.FORMAT_PARAMETERS(
|
||||||
|
SYS.ODCIVARCHAR2LIST(
|
||||||
|
'pTableName => ''' || pTableName || '''',
|
||||||
|
'pTemplateTableName => ''' || pTemplateTableName || '''',
|
||||||
|
'pPrefix => ''' || pPrefix || '''',
|
||||||
|
'pBucketUri => ''' || pBucketUri || '''',
|
||||||
|
'pFileName => ''' || NVL(pFileName, 'NULL') || '''',
|
||||||
|
'pDelimiter => ''' || pDelimiter || '''',
|
||||||
|
'pEncoding => ''' || NVL(pEncoding, 'NULL') || ''''
|
||||||
|
)
|
||||||
|
);
|
||||||
|
|
||||||
|
CT_MRDS.ENV_MANAGER.LOG_PROCESS_EVENT('Start FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE', 'INFO', vParameters);
|
||||||
|
|
||||||
|
-- Call the original CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLE procedure
|
||||||
|
-- This ensures all logic remains centralized in the original package
|
||||||
|
-- and ODS wrapper simply delegates execution with DEFINER rights
|
||||||
|
CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => pTableName,
|
||||||
|
pTemplateTableName => pTemplateTableName,
|
||||||
|
pPrefix => pPrefix,
|
||||||
|
pBucketUri => pBucketUri,
|
||||||
|
pFileName => pFileName,
|
||||||
|
pDelimiter => pDelimiter,
|
||||||
|
pEncoding => pEncoding
|
||||||
|
);
|
||||||
|
|
||||||
|
CT_MRDS.ENV_MANAGER.LOG_PROCESS_EVENT('End FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE', 'INFO', vParameters);
|
||||||
|
|
||||||
|
EXCEPTION
|
||||||
|
WHEN OTHERS THEN
|
||||||
|
CT_MRDS.ENV_MANAGER.LOG_PROCESS_EVENT(
|
||||||
|
'Error in ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE: ' || SQLERRM,
|
||||||
|
'ERROR',
|
||||||
|
vParameters
|
||||||
|
);
|
||||||
|
RAISE;
|
||||||
|
END CREATE_EXTERNAL_TABLE;
|
||||||
|
|
||||||
|
----------------------------------------------------------------------------------------------------
|
||||||
|
-- PACKAGE VERSION MANAGEMENT FUNCTIONS IMPLEMENTATION
|
||||||
|
----------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
FUNCTION GET_VERSION
|
||||||
|
RETURN VARCHAR2
|
||||||
|
IS
|
||||||
|
BEGIN
|
||||||
|
RETURN PACKAGE_VERSION;
|
||||||
|
END GET_VERSION;
|
||||||
|
|
||||||
|
----------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
FUNCTION GET_BUILD_DATE
|
||||||
|
RETURN VARCHAR2
|
||||||
|
IS
|
||||||
|
BEGIN
|
||||||
|
RETURN PACKAGE_BUILD_DATE;
|
||||||
|
END GET_BUILD_DATE;
|
||||||
|
|
||||||
|
----------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
FUNCTION GET_AUTHOR
|
||||||
|
RETURN VARCHAR2
|
||||||
|
IS
|
||||||
|
BEGIN
|
||||||
|
RETURN PACKAGE_AUTHOR;
|
||||||
|
END GET_AUTHOR;
|
||||||
|
|
||||||
|
----------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
FUNCTION GET_BUILD_INFO
|
||||||
|
RETURN VARCHAR2
|
||||||
|
IS
|
||||||
|
BEGIN
|
||||||
|
RETURN CT_MRDS.ENV_MANAGER.GET_PACKAGE_VERSION_INFO(
|
||||||
|
pPackageName => 'FILE_MANAGER_ODS',
|
||||||
|
pVersion => PACKAGE_VERSION,
|
||||||
|
pBuildDate => PACKAGE_BUILD_DATE,
|
||||||
|
pAuthor => PACKAGE_AUTHOR
|
||||||
|
);
|
||||||
|
END GET_BUILD_INFO;
|
||||||
|
|
||||||
|
----------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
FUNCTION GET_VERSION_HISTORY
|
||||||
|
RETURN VARCHAR2
|
||||||
|
IS
|
||||||
|
BEGIN
|
||||||
|
RETURN CT_MRDS.ENV_MANAGER.FORMAT_VERSION_HISTORY(
|
||||||
|
pPackageName => 'FILE_MANAGER_ODS',
|
||||||
|
pVersionHistory => VERSION_HISTORY
|
||||||
|
);
|
||||||
|
END GET_VERSION_HISTORY;
|
||||||
|
|
||||||
|
----------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
END FILE_MANAGER_ODS;
|
||||||
|
|
||||||
|
/
|
||||||
116
MARS_Packages/REL01/MARS-1049/new_version/FILE_MANAGER_ODS.pkg
Normal file
116
MARS_Packages/REL01/MARS-1049/new_version/FILE_MANAGER_ODS.pkg
Normal file
@@ -0,0 +1,116 @@
|
|||||||
|
create or replace PACKAGE ODS.FILE_MANAGER_ODS
|
||||||
|
AUTHID DEFINER
|
||||||
|
AS
|
||||||
|
/**
|
||||||
|
* FILE_MANAGER_ODS - Wrapper package for CT_MRDS.FILE_MANAGER
|
||||||
|
*
|
||||||
|
* This package serves as a wrapper for CT_MRDS.FILE_MANAGER procedures.
|
||||||
|
* The key difference is that this package uses AUTHID DEFINER instead of
|
||||||
|
* AUTHID CURRENT_USER, which means all objects will be created in the ODS
|
||||||
|
* schema regardless of which user executes the procedures.
|
||||||
|
*
|
||||||
|
* This solves the execution context issue where users need to connect as
|
||||||
|
* ODS user to use CT_MRDS.FILE_MANAGER procedures.
|
||||||
|
*
|
||||||
|
* MARS-1049: Added pEncoding parameter support for CSV character set specification
|
||||||
|
*/
|
||||||
|
|
||||||
|
-- Package Version Information (Semantic Versioning: MAJOR.MINOR.PATCH)
|
||||||
|
PACKAGE_VERSION CONSTANT VARCHAR2(10) := '2.1.0';
|
||||||
|
PACKAGE_BUILD_DATE CONSTANT VARCHAR2(20) := '2025-11-24 11:50:00';
|
||||||
|
PACKAGE_AUTHOR CONSTANT VARCHAR2(100) := 'Grzegorz Michalski';
|
||||||
|
|
||||||
|
-- Version History (Latest changes first)
|
||||||
|
VERSION_HISTORY CONSTANT VARCHAR2(4000) :=
|
||||||
|
'2.1.0 (2025-11-24): MARS-1049 - Added pEncoding parameter support for CSV character set specification' || CHR(13)||CHR(10) ||
|
||||||
|
'2.0.0 (2025-10-22): Added package versioning system using centralized ENV_MANAGER functions' || CHR(13)||CHR(10) ||
|
||||||
|
'1.5.0 (2025-10-12): Enhanced external table creation with official path patterns support' || CHR(13)||CHR(10) ||
|
||||||
|
'1.0.0 (2025-09-20): Initial release with AUTHID DEFINER wrapper for external table creation';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name CREATE_EXTERNAL_TABLE
|
||||||
|
* @desc Creates external tables that can read data from Oracle Cloud Storage.
|
||||||
|
* This is a wrapper for CT_MRDS.FILE_MANAGER.CREATE_EXTERNAL_TABLE
|
||||||
|
* but uses DEFINER rights to ensure objects are created in ODS schema.
|
||||||
|
*
|
||||||
|
* MARS-1049: Added support for CSV character set encoding specification
|
||||||
|
* @param pTableName - Name of the external table to create
|
||||||
|
* @param pTemplateTableName - Template table defining the structure
|
||||||
|
* @param pPrefix - Storage path prefix in Oracle Cloud Storage
|
||||||
|
* @param pBucketUri - URI of the target bucket (default: ENV_MANAGER.gvInboxBucketUri)
|
||||||
|
* @param pFileName - Specific file name (optional)
|
||||||
|
* @param pDelimiter - Field delimiter (default: ',')
|
||||||
|
* @param pEncoding - Character set encoding for CSV files (e.g., 'UTF-8', 'WE8MSWIN1252', 'EE8ISO8859P2')
|
||||||
|
* If NULL, no CHARACTERSET clause is added to external table definition
|
||||||
|
* @example EXEC ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
* 'C2D_A_UC_DISSEM_METADATA_LOADS_INBOX',
|
||||||
|
* 'CT_ET_TEMPLATES.C2D_A_UC_DISSEM_METADATA_LOADS',
|
||||||
|
* 'INBOX/C2D/UC_DISSEM/A_UC_DISSEM_METADATA_LOADS',
|
||||||
|
* CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
* NULL, ',', 'UTF-8'
|
||||||
|
* );
|
||||||
|
*/
|
||||||
|
PROCEDURE CREATE_EXTERNAL_TABLE (
|
||||||
|
pTableName IN VARCHAR2,
|
||||||
|
pTemplateTableName IN VARCHAR2,
|
||||||
|
pPrefix IN VARCHAR2,
|
||||||
|
pBucketUri IN VARCHAR2 DEFAULT CT_MRDS.ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pFileName IN VARCHAR2 DEFAULT NULL,
|
||||||
|
pDelimiter IN VARCHAR2 DEFAULT ',',
|
||||||
|
pEncoding IN VARCHAR2 DEFAULT NULL
|
||||||
|
);
|
||||||
|
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
-- PACKAGE VERSION MANAGEMENT FUNCTIONS
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_VERSION
|
||||||
|
* @desc Returns the current version number of the FILE_MANAGER_ODS package.
|
||||||
|
* Uses semantic versioning format (MAJOR.MINOR.PATCH).
|
||||||
|
* @example SELECT FILE_MANAGER_ODS.GET_VERSION() FROM DUAL;
|
||||||
|
* @ex_rslt 2.1.0
|
||||||
|
**/
|
||||||
|
FUNCTION GET_VERSION RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_BUILD_DATE
|
||||||
|
* @desc Returns the build date of the FILE_MANAGER_ODS package.
|
||||||
|
* @return VARCHAR2 - Build date in format 'YYYY-MM-DD HH24:MI:SS'
|
||||||
|
* @example SELECT ODS.FILE_MANAGER_ODS.GET_BUILD_DATE() FROM DUAL;
|
||||||
|
*/
|
||||||
|
FUNCTION GET_BUILD_DATE RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_AUTHOR
|
||||||
|
* @desc Returns the author of the FILE_MANAGER_ODS package.
|
||||||
|
* @return VARCHAR2 - Package author name
|
||||||
|
* @example SELECT ODS.FILE_MANAGER_ODS.GET_AUTHOR() FROM DUAL;
|
||||||
|
*/
|
||||||
|
FUNCTION GET_AUTHOR RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_BUILD_INFO
|
||||||
|
* @desc Returns comprehensive build information including version, build date, and author.
|
||||||
|
* Uses centralized ENV_MANAGER.GET_PACKAGE_VERSION_INFO function.
|
||||||
|
* @example SELECT FILE_MANAGER_ODS.GET_BUILD_INFO() FROM DUAL;
|
||||||
|
* @ex_rslt Package: FILE_MANAGER_ODS
|
||||||
|
* Version: 2.1.0
|
||||||
|
* Build Date: 2025-11-24 11:50:00
|
||||||
|
* Author: Grzegorz Michalski
|
||||||
|
**/
|
||||||
|
FUNCTION GET_BUILD_INFO RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_VERSION_HISTORY
|
||||||
|
* @desc Returns complete version history with all releases and changes.
|
||||||
|
* Uses centralized ENV_MANAGER.FORMAT_VERSION_HISTORY function.
|
||||||
|
* @example SELECT FILE_MANAGER_ODS.GET_VERSION_HISTORY() FROM DUAL;
|
||||||
|
* @ex_rslt FILE_MANAGER_ODS Version History:
|
||||||
|
* 2.0.0 (2025-10-22): Added package versioning system...
|
||||||
|
**/
|
||||||
|
FUNCTION GET_VERSION_HISTORY RETURN VARCHAR2;
|
||||||
|
|
||||||
|
END FILE_MANAGER_ODS;
|
||||||
|
|
||||||
|
/
|
||||||
112
MARS_Packages/REL01/MARS-1049/rollback_mars1049.sql
Normal file
112
MARS_Packages/REL01/MARS-1049/rollback_mars1049.sql
Normal file
@@ -0,0 +1,112 @@
|
|||||||
|
-- ===================================================================
|
||||||
|
-- MARS-1049 ROLLBACK SCRIPT: CSV Encoding Support
|
||||||
|
-- ===================================================================
|
||||||
|
-- Purpose: Complete rollback of CSV encoding support changes
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-25
|
||||||
|
-- Version: 1.5.0
|
||||||
|
|
||||||
|
-- Dynamic spool file generation
|
||||||
|
var filename VARCHAR2(100)
|
||||||
|
BEGIN
|
||||||
|
SELECT 'ROLLBACK_MARS_1049_' || SUBSTR(PDB_NAME, (INSTR(PDB_NAME,'_',1)+1), (LENGTH(PDB_NAME)-INSTR(PDB_NAME,'_',1))) || '_' ||TO_CHAR(SYSDATE,'YYYYMMDD_HH24MISS')||'.log' INTO :filename from DBA_PDBS;
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
column filename new_value _filename
|
||||||
|
select :filename filename from dual;
|
||||||
|
spool &_filename
|
||||||
|
--
|
||||||
|
-- CURRENT IMPLEMENTATION STATUS:
|
||||||
|
-- - CT_MRDS.FILE_MANAGER: Version 3.2.1 with pEncoding parameter (deployed)
|
||||||
|
-- - ODS.FILE_MANAGER_ODS: Version 2.1.0 with pEncoding wrapper (deployed)
|
||||||
|
-- - A_SOURCE_FILE_CONFIG: ENCODING VARCHAR2(50) column (deployed)
|
||||||
|
--
|
||||||
|
-- ROLLBACK STRATEGY:
|
||||||
|
-- - Uses current_version/ folder for pre-MARS-1049 package versions
|
||||||
|
-- - Uses new_version/ folder for MARS-1049 enhanced versions (for reference)
|
||||||
|
--
|
||||||
|
-- ROLLBACK ORDER:
|
||||||
|
-- 1. Restore ODS.FILE_MANAGER_ODS package from current_version/ (version 2.0.0)
|
||||||
|
-- 2. Restore CT_MRDS.FILE_MANAGER package from current_version/ (version 3.2.0)
|
||||||
|
-- 3. Remove ENCODING column from A_SOURCE_FILE_CONFIG
|
||||||
|
-- 4. Track package versions
|
||||||
|
-- 5. Verify all tracked packages
|
||||||
|
-- 6. Revoke SELECT ANY DICTIONARY from CT_MRDS
|
||||||
|
-- ===================================================================
|
||||||
|
|
||||||
|
PROMPT ===============================================
|
||||||
|
PROMPT MARS-1049: CSV Encoding Support - Full Rollback
|
||||||
|
PROMPT ===============================================
|
||||||
|
|
||||||
|
-- Set error handling
|
||||||
|
WHENEVER SQLERROR CONTINUE
|
||||||
|
WHENEVER OSERROR CONTINUE
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT Warning: This will remove the ENCODING column and revert all package changes.
|
||||||
|
PROMPT Make sure you have backed up your data if needed.
|
||||||
|
PROMPT
|
||||||
|
|
||||||
|
-- Step 1: Restore ODS.FILE_MANAGER_ODS package to version 2.0.0
|
||||||
|
PROMPT Step 1: Restoring ODS.FILE_MANAGER_ODS package to version 2.0.0...
|
||||||
|
PROMPT ===============================================
|
||||||
|
|
||||||
|
-- Restore ODS.FILE_MANAGER_ODS specification from current_version (before MARS-1049)
|
||||||
|
@@current_version/FILE_MANAGER_ODS.pkg
|
||||||
|
|
||||||
|
-- Restore ODS.FILE_MANAGER_ODS body from current_version (before MARS-1049)
|
||||||
|
@@current_version/FILE_MANAGER_ODS.pkb
|
||||||
|
|
||||||
|
PROMPT ODS.FILE_MANAGER_ODS package restored to version 2.0.0
|
||||||
|
|
||||||
|
-- Step 2: Restore CT_MRDS.FILE_MANAGER package to version 3.2.0
|
||||||
|
PROMPT Step 2: Restoring CT_MRDS.FILE_MANAGER package to version 3.2.0...
|
||||||
|
PROMPT ===============================================
|
||||||
|
|
||||||
|
-- Restore CT_MRDS.FILE_MANAGER specification from current_version (before MARS-1049)
|
||||||
|
@@current_version/FILE_MANAGER.pkg
|
||||||
|
|
||||||
|
-- Restore CT_MRDS.FILE_MANAGER body from current_version (before MARS-1049)
|
||||||
|
@@current_version/FILE_MANAGER.pkb
|
||||||
|
|
||||||
|
PROMPT CT_MRDS.FILE_MANAGER package restored to version 3.2.0
|
||||||
|
|
||||||
|
-- Step 3: Remove ENCODING column from A_SOURCE_FILE_CONFIG
|
||||||
|
PROMPT Step 3: Remove ENCODING column from A_SOURCE_FILE_CONFIG...
|
||||||
|
PROMPT ===============================================
|
||||||
|
|
||||||
|
@@91_MARS_1049_rollback_DROP_ENCODING_COLUMN.sql
|
||||||
|
|
||||||
|
-- Step 4: Track rollback version
|
||||||
|
PROMPT Step 4: Tracking package versions...
|
||||||
|
PROMPT ===============================================
|
||||||
|
|
||||||
|
@@track_package_versions.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT Step 5: Verifying all tracked packages...
|
||||||
|
PROMPT ===============================================
|
||||||
|
@@verify_packages_version.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT Step 6: Revoking cross-schema hash tracking permissions...
|
||||||
|
PROMPT ===============================================
|
||||||
|
@@90_MARS_1049_rollback_revoke_permissions.sql
|
||||||
|
|
||||||
|
PROMPT ===============================================
|
||||||
|
PROMPT MARS-1049: Rollback Complete!
|
||||||
|
PROMPT ===============================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT Summary of Rollback Changes:
|
||||||
|
PROMPT - A_SOURCE_FILE_CONFIG: Removed ENCODING VARCHAR2(50) column
|
||||||
|
PROMPT - CT_MRDS.FILE_MANAGER: Restored to version 3.2.0 (before MARS-1049)
|
||||||
|
PROMPT - ODS.FILE_MANAGER_ODS: Restored to version 2.0.0 (before MARS-1049)
|
||||||
|
PROMPT - SELECT ANY DICTIONARY: Revoked from CT_MRDS (cross-schema tracking disabled)
|
||||||
|
PROMPT
|
||||||
|
PROMPT All MARS-1049 changes have been successfully rolled back.
|
||||||
|
PROMPT System is now in pre-MARS-1049 state.
|
||||||
|
PROMPT
|
||||||
|
|
||||||
|
spool off
|
||||||
|
|
||||||
|
exit
|
||||||
96
MARS_Packages/REL01/MARS-1049/track_package_versions.sql
Normal file
96
MARS_Packages/REL01/MARS-1049/track_package_versions.sql
Normal file
@@ -0,0 +1,96 @@
|
|||||||
|
-- ===================================================================
|
||||||
|
-- Simple Package Version Tracking Script
|
||||||
|
-- ===================================================================
|
||||||
|
-- Purpose: Track specified Oracle package versions
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-25
|
||||||
|
-- Version: 3.1.0 - List-Based Edition
|
||||||
|
--
|
||||||
|
-- USAGE:
|
||||||
|
-- 1. Edit package list below (add/remove packages as needed)
|
||||||
|
-- 2. Include in your install/rollback script: @@track_package_versions.sql
|
||||||
|
-- ===================================================================
|
||||||
|
|
||||||
|
SET SERVEROUTPUT ON;
|
||||||
|
|
||||||
|
DECLARE
|
||||||
|
TYPE t_package_rec IS RECORD (
|
||||||
|
owner VARCHAR2(50),
|
||||||
|
package_name VARCHAR2(50),
|
||||||
|
version VARCHAR2(50)
|
||||||
|
);
|
||||||
|
TYPE t_packages IS TABLE OF t_package_rec;
|
||||||
|
TYPE t_string_array IS TABLE OF VARCHAR2(100);
|
||||||
|
|
||||||
|
-- ===================================================================
|
||||||
|
-- PACKAGE LIST - Edit this array to specify packages to track
|
||||||
|
-- ===================================================================
|
||||||
|
-- Add or remove entries as needed for your MARS issue
|
||||||
|
-- Format: 'SCHEMA.PACKAGE_NAME'
|
||||||
|
-- ===================================================================
|
||||||
|
vPackageList t_string_array := t_string_array(
|
||||||
|
'CT_MRDS.FILE_MANAGER',
|
||||||
|
'ODS.FILE_MANAGER_ODS'
|
||||||
|
);
|
||||||
|
-- ===================================================================
|
||||||
|
|
||||||
|
vPackages t_packages := t_packages();
|
||||||
|
vVersion VARCHAR2(50);
|
||||||
|
vCount NUMBER := 0;
|
||||||
|
vOwner VARCHAR2(50);
|
||||||
|
vPackageName VARCHAR2(50);
|
||||||
|
vDotPos NUMBER;
|
||||||
|
BEGIN
|
||||||
|
DBMS_OUTPUT.PUT_LINE('========================================');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Package Version Tracking');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('========================================');
|
||||||
|
|
||||||
|
-- Process each package in the list
|
||||||
|
FOR i IN 1..vPackageList.COUNT LOOP
|
||||||
|
vDotPos := INSTR(vPackageList(i), '.');
|
||||||
|
IF vDotPos > 0 THEN
|
||||||
|
vOwner := SUBSTR(vPackageList(i), 1, vDotPos - 1);
|
||||||
|
vPackageName := SUBSTR(vPackageList(i), vDotPos + 1);
|
||||||
|
|
||||||
|
-- Get package version
|
||||||
|
BEGIN
|
||||||
|
EXECUTE IMMEDIATE 'SELECT ' || vOwner || '.' || vPackageName || '.GET_VERSION() FROM DUAL' INTO vVersion;
|
||||||
|
vPackages.EXTEND;
|
||||||
|
vPackages(vPackages.COUNT).owner := vOwner;
|
||||||
|
vPackages(vPackages.COUNT).package_name := vPackageName;
|
||||||
|
vPackages(vPackages.COUNT).version := vVersion;
|
||||||
|
|
||||||
|
-- Track in ENV_MANAGER
|
||||||
|
BEGIN
|
||||||
|
CT_MRDS.ENV_MANAGER.TRACK_PACKAGE_VERSION(
|
||||||
|
pPackageOwner => vOwner,
|
||||||
|
pPackageName => vPackageName,
|
||||||
|
pPackageVersion => vVersion,
|
||||||
|
pPackageBuildDate => TO_CHAR(SYSDATE, 'YYYY-MM-DD HH24:MI:SS'),
|
||||||
|
pPackageAuthor => 'Grzegorz Michalski'
|
||||||
|
);
|
||||||
|
vCount := vCount + 1;
|
||||||
|
EXCEPTION
|
||||||
|
WHEN OTHERS THEN NULL; -- Continue even if tracking fails
|
||||||
|
END;
|
||||||
|
EXCEPTION
|
||||||
|
WHEN OTHERS THEN NULL; -- Skip packages that fail
|
||||||
|
END;
|
||||||
|
END IF;
|
||||||
|
END LOOP;
|
||||||
|
|
||||||
|
-- Display results
|
||||||
|
IF vPackages.COUNT > 0 THEN
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Packages tracked: ' || vCount || ' of ' || vPackages.COUNT);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('');
|
||||||
|
|
||||||
|
FOR i IN 1..vPackages.COUNT LOOP
|
||||||
|
DBMS_OUTPUT.PUT_LINE(vPackages(i).owner || '.' || vPackages(i).package_name || ' = ' || vPackages(i).version);
|
||||||
|
END LOOP;
|
||||||
|
ELSE
|
||||||
|
DBMS_OUTPUT.PUT_LINE('No packages found in list');
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
DBMS_OUTPUT.PUT_LINE('========================================');
|
||||||
|
END;
|
||||||
|
/
|
||||||
62
MARS_Packages/REL01/MARS-1049/verify_packages_version.sql
Normal file
62
MARS_Packages/REL01/MARS-1049/verify_packages_version.sql
Normal file
@@ -0,0 +1,62 @@
|
|||||||
|
-- ===================================================================
|
||||||
|
-- Universal Package Version Verification Script
|
||||||
|
-- ===================================================================
|
||||||
|
-- Purpose: Verify all tracked Oracle packages for code changes
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-25
|
||||||
|
-- Version: 1.0.0
|
||||||
|
--
|
||||||
|
-- USAGE:
|
||||||
|
-- Include at the end of install/rollback scripts: @@verify_packages_version.sql
|
||||||
|
--
|
||||||
|
-- OUTPUT:
|
||||||
|
-- - List of all tracked packages with their current status
|
||||||
|
-- - OK: Package has not changed since last tracking
|
||||||
|
-- - WARNING: Package code changed without version update
|
||||||
|
-- ===================================================================
|
||||||
|
|
||||||
|
SET LINESIZE 200
|
||||||
|
SET PAGESIZE 1000
|
||||||
|
SET FEEDBACK OFF
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT ========================================
|
||||||
|
PROMPT Package Version Verification
|
||||||
|
PROMPT ========================================
|
||||||
|
PROMPT
|
||||||
|
|
||||||
|
COLUMN PACKAGE_OWNER FORMAT A15
|
||||||
|
COLUMN PACKAGE_NAME FORMAT A20
|
||||||
|
COLUMN VERSION FORMAT A10
|
||||||
|
COLUMN STATUS FORMAT A80
|
||||||
|
|
||||||
|
SELECT
|
||||||
|
PACKAGE_OWNER,
|
||||||
|
PACKAGE_NAME,
|
||||||
|
PACKAGE_VERSION AS VERSION,
|
||||||
|
CT_MRDS.ENV_MANAGER.CHECK_PACKAGE_CHANGES(PACKAGE_OWNER, PACKAGE_NAME) AS STATUS
|
||||||
|
FROM (
|
||||||
|
SELECT
|
||||||
|
PACKAGE_OWNER,
|
||||||
|
PACKAGE_NAME,
|
||||||
|
PACKAGE_VERSION,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY PACKAGE_OWNER, PACKAGE_NAME ORDER BY TRACKING_DATE DESC) AS RN
|
||||||
|
FROM CT_MRDS.A_PACKAGE_VERSION_TRACKING
|
||||||
|
)
|
||||||
|
WHERE RN = 1
|
||||||
|
ORDER BY PACKAGE_OWNER, PACKAGE_NAME;
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT ========================================
|
||||||
|
PROMPT Verification Complete
|
||||||
|
PROMPT ========================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT Legend:
|
||||||
|
PROMPT OK - Package has not changed since last tracking
|
||||||
|
PROMPT WARNING - Package code changed without version update
|
||||||
|
PROMPT
|
||||||
|
PROMPT For detailed hash information, use:
|
||||||
|
PROMPT SELECT ENV_MANAGER.GET_PACKAGE_HASH_INFO('OWNER', 'PACKAGE') FROM DUAL;
|
||||||
|
PROMPT ========================================
|
||||||
|
|
||||||
|
SET FEEDBACK ON
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
/*
|
||||||
|
================================================================================
|
||||||
|
MARS-1056: Update FILE_MANAGER package with corrected VARCHAR2 definitions
|
||||||
|
Script: 01_MARS_1056_update_FILE_MANAGER_package.sql
|
||||||
|
Author: Grzegorz Michalski
|
||||||
|
Date: November 25, 2025
|
||||||
|
Purpose: Implementation of VARCHAR2 definition fixes in GENERATE_EXTERNAL_TABLE_PARAMS
|
||||||
|
================================================================================
|
||||||
|
*/
|
||||||
|
|
||||||
|
SET SERVEROUTPUT ON SIZE UNLIMITED
|
||||||
|
|
||||||
|
PROMPT ===================================================================
|
||||||
|
PROMPT Installing corrected FILE_MANAGER package (MARS-1056)
|
||||||
|
PROMPT Fix: VARCHAR2 definitions will preserve CHAR/BYTE semantics
|
||||||
|
PROMPT ===================================================================
|
||||||
|
|
||||||
|
-- Install updated FILE_MANAGER specification
|
||||||
|
@@new_version/FILE_MANAGER.pkg
|
||||||
|
|
||||||
|
-- Install updated FILE_MANAGER body with VARCHAR2 fixes
|
||||||
|
@@new_version/FILE_MANAGER.pkb
|
||||||
|
|
||||||
|
-- Verify compilation (ADMIN user checks CT_MRDS schema)
|
||||||
|
SELECT object_name, object_type, status
|
||||||
|
FROM all_objects
|
||||||
|
WHERE owner = 'CT_MRDS'
|
||||||
|
AND object_name = 'FILE_MANAGER'
|
||||||
|
AND object_type IN ('PACKAGE', 'PACKAGE BODY');
|
||||||
|
|
||||||
|
-- Check for compilation errors (ADMIN user checks CT_MRDS schema)
|
||||||
|
SELECT * FROM all_errors
|
||||||
|
WHERE owner = 'CT_MRDS'
|
||||||
|
AND name = 'FILE_MANAGER';
|
||||||
|
|
||||||
|
PROMPT ===================================================================
|
||||||
|
PROMPT MARS-1056: FILE_MANAGER package updated with VARCHAR2 definition fixes
|
||||||
|
PROMPT ===================================================================
|
||||||
|
|
||||||
|
/
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
/*
|
||||||
|
================================================================================
|
||||||
|
MARS-1056: Rollback FILE_MANAGER packages to pre-MARS-1056 versions
|
||||||
|
Author: Grzegorz Michalski
|
||||||
|
Date: November 25, 2025
|
||||||
|
Purpose: Rollback packages to versions before VARCHAR2 changes
|
||||||
|
================================================================================
|
||||||
|
*/
|
||||||
|
|
||||||
|
SET SERVEROUTPUT ON SIZE UNLIMITED
|
||||||
|
|
||||||
|
PROMPT ===================================================================
|
||||||
|
PROMPT MARS-1056: Rolling back to pre-MARS-1056 package versions
|
||||||
|
PROMPT WARNING: This will restore original VARCHAR2 definition behavior
|
||||||
|
PROMPT ===================================================================
|
||||||
|
|
||||||
|
-- Rollback FILE_MANAGER to current_version
|
||||||
|
PROMPT Rolling back FILE_MANAGER package...
|
||||||
|
@@current_version/FILE_MANAGER.pkg
|
||||||
|
@@current_version/FILE_MANAGER.pkb
|
||||||
|
|
||||||
|
-- Verify compilation (ADMIN user checks CT_MRDS schema)
|
||||||
|
SELECT object_name, object_type, status
|
||||||
|
FROM all_objects
|
||||||
|
WHERE owner = 'CT_MRDS'
|
||||||
|
AND object_name = 'FILE_MANAGER'
|
||||||
|
AND object_type IN ('PACKAGE', 'PACKAGE BODY')
|
||||||
|
ORDER BY object_name, object_type;
|
||||||
|
|
||||||
|
-- Check for compilation errors (ADMIN user checks CT_MRDS schema)
|
||||||
|
SELECT name, type, line, text FROM all_errors
|
||||||
|
WHERE owner = 'CT_MRDS'
|
||||||
|
AND name = 'FILE_MANAGER';
|
||||||
|
|
||||||
|
PROMPT ===================================================================
|
||||||
|
PROMPT MARS-1056: Rollback completed - packages restored to pre-MARS-1056 state
|
||||||
|
PROMPT VARCHAR2 definitions will use original (data_length) behavior
|
||||||
|
PROMPT ===================================================================
|
||||||
|
|
||||||
|
/
|
||||||
1923
MARS_Packages/REL01/MARS-1056/current_version/FILE_MANAGER.pkb
Normal file
1923
MARS_Packages/REL01/MARS-1056/current_version/FILE_MANAGER.pkb
Normal file
File diff suppressed because it is too large
Load Diff
635
MARS_Packages/REL01/MARS-1056/current_version/FILE_MANAGER.pkg
Normal file
635
MARS_Packages/REL01/MARS-1056/current_version/FILE_MANAGER.pkg
Normal file
@@ -0,0 +1,635 @@
|
|||||||
|
create or replace PACKAGE CT_MRDS.FILE_MANAGER
|
||||||
|
AUTHID CURRENT_USER
|
||||||
|
AS
|
||||||
|
/**
|
||||||
|
* General comment for package: Please put comments for functions and procedures as shown in below example.
|
||||||
|
* It is a standard.
|
||||||
|
* The structure of comment is used by GET_PACKAGE_DOCUMENTATION function
|
||||||
|
* which returns documentation text for confluence page (to Copy-Paste it).
|
||||||
|
**/
|
||||||
|
|
||||||
|
-- Example comment:
|
||||||
|
/**
|
||||||
|
* @name EX_PROCEDURE_NAME
|
||||||
|
* @desc Procedure description
|
||||||
|
* @example select FILE_MANAGER.EX_PROCEDURE_NAME(pParameter => 129) from dual;
|
||||||
|
* @ex_rslt Example Result
|
||||||
|
**/
|
||||||
|
|
||||||
|
-- Package Version Information (Semantic Versioning: MAJOR.MINOR.PATCH)
|
||||||
|
PACKAGE_VERSION CONSTANT VARCHAR2(10) := '3.2.1';
|
||||||
|
PACKAGE_BUILD_DATE CONSTANT VARCHAR2(20) := '2025-11-24 12:00:00';
|
||||||
|
PACKAGE_AUTHOR CONSTANT VARCHAR2(100) := 'Grzegorz Michalski';
|
||||||
|
|
||||||
|
-- Version History (Latest changes first)
|
||||||
|
VERSION_HISTORY CONSTANT VARCHAR2(4000) :=
|
||||||
|
'3.2.1 (2025-11-24): MARS-1049 - Added pEncoding parameter support for CSV character set specification' || CHR(13)||CHR(10) ||
|
||||||
|
'3.2.0 (2025-10-22): Added package versioning system using centralized ENV_MANAGER functions' || CHR(13)||CHR(10) ||
|
||||||
|
'3.1.0 (2025-10-20): Enhanced PROCESS_SOURCE_FILE with 6-step validation workflow' || CHR(13)||CHR(10) ||
|
||||||
|
'3.0.0 (2025-10-15): Separated export procedures into dedicated DATA_EXPORTER package' || CHR(13)||CHR(10) ||
|
||||||
|
'2.5.0 (2025-10-10): Added DELETE_SOURCE_CASCADE for safe configuration removal' || CHR(13)||CHR(10) ||
|
||||||
|
'2.0.0 (2025-09-25): Added official path patterns support (INBOX 3-level, ODS 2-level, ARCHIVE 2-level)' || CHR(13)||CHR(10) ||
|
||||||
|
'1.0.0 (2025-09-01): Initial release with file processing and validation capabilities';
|
||||||
|
|
||||||
|
TYPE tSourceFileReceived IS RECORD
|
||||||
|
(
|
||||||
|
A_SOURCE_FILE_RECEIVED_KEY CT_MRDS.A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY%TYPE,
|
||||||
|
A_SOURCE_FILE_CONFIG_KEY CT_MRDS.A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_CONFIG_KEY%TYPE,
|
||||||
|
SOURCE_FILE_PREFIX_INBOX VARCHAR2(430),
|
||||||
|
SOURCE_FILE_PREFIX_ODS VARCHAR2(430),
|
||||||
|
SOURCE_FILE_PREFIX_QUARANTINE VARCHAR2(430),
|
||||||
|
SOURCE_FILE_PREFIX_ARCHIVE VARCHAR2(430),
|
||||||
|
SOURCE_FILE_NAME CT_MRDS.A_SOURCE_FILE_RECEIVED.SOURCE_FILE_NAME%TYPE,
|
||||||
|
RECEPTION_DATE CT_MRDS.A_SOURCE_FILE_RECEIVED.RECEPTION_DATE%TYPE,
|
||||||
|
PROCESSING_STATUS CT_MRDS.A_SOURCE_FILE_RECEIVED.PROCESSING_STATUS%TYPE,
|
||||||
|
EXTERNAL_TABLE_NAME CT_MRDS.A_SOURCE_FILE_RECEIVED.EXTERNAL_TABLE_NAME%TYPE
|
||||||
|
);
|
||||||
|
|
||||||
|
cgBL CONSTANT VARCHAR2(2) := CHR(13)||CHR(10);
|
||||||
|
vgSourceFileConfigKey PLS_INTEGER;
|
||||||
|
vgMsgTmp VARCHAR2(32000);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_SOURCE_FILE_CONFIG
|
||||||
|
* @desc Get source file type by matching the source file name against source file type naming patterns
|
||||||
|
* or by specifying the id of a received source file.
|
||||||
|
* @example ...
|
||||||
|
* @ex_rslt "CT_MRDS.A_SOURCE_FILE_CONFIG%ROWTYPE"
|
||||||
|
**/
|
||||||
|
FUNCTION GET_SOURCE_FILE_CONFIG(pFileUri IN VARCHAR2 DEFAULT NULL
|
||||||
|
, pSourceFileReceivedKey IN NUMBER DEFAULT NULL
|
||||||
|
, pSourceFileConfigKey IN NUMBER DEFAULT NULL)
|
||||||
|
RETURN CT_MRDS.A_SOURCE_FILE_CONFIG%ROWTYPE;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name REGISTER_SOURCE_FILE_RECEIVED
|
||||||
|
* @desc Register a newly received source file in A_SOURCE_FILE_RECEIVED table.
|
||||||
|
* This overload automatically determines source file type from the file name.
|
||||||
|
* It returns the value of A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY column for newly added record.
|
||||||
|
* @example vSourceFileReceivedKey := FILE_MANAGER.REGISTER_SOURCE_FILE_RECEIVED(pSourceFileReceivedName => 'INBOX/C2D/UC_DISSEM/UC_NMA_DISSEM/UC_NMA_DISSEM-277740.csv');
|
||||||
|
* @ex_rslt 3245
|
||||||
|
**/
|
||||||
|
FUNCTION REGISTER_SOURCE_FILE_RECEIVED (
|
||||||
|
pSourceFileReceivedName IN VARCHAR2
|
||||||
|
)
|
||||||
|
RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name REGISTER_SOURCE_FILE_RECEIVED
|
||||||
|
* @desc Register a new new source file in A_SOURCE_FILE_RECEIVED table based on pSourceFileReceivedName and pSourceFileConfig.
|
||||||
|
* Then it returns the value of A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY column for newly added record.
|
||||||
|
* @example vSourceFileReceivedKey := FILE_MANAGER.REGISTER_SOURCE_FILE_RECEIVED(
|
||||||
|
* pSourceFileReceivedName => 'INBOX/C2D/UC_DISSEM/UC_NMA_DISSEM/UC_NMA_DISSEM-277740.csv'
|
||||||
|
* ,pSourceFileConfig => ...A_SOURCE_FILE_CONFIG%ROWTYPE... );
|
||||||
|
* @ex_rslt 3245
|
||||||
|
**/
|
||||||
|
FUNCTION REGISTER_SOURCE_FILE_RECEIVED (
|
||||||
|
pSourceFileReceivedName IN VARCHAR2,
|
||||||
|
pSourceFileConfig IN CT_MRDS.A_SOURCE_FILE_CONFIG%ROWTYPE
|
||||||
|
)
|
||||||
|
RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name SET_SOURCE_FILE_RECEIVED_STATUS
|
||||||
|
* @desc Set status of file in A_SOURCE_FILE_RECEIVED table - PROCESSING_STATUS column
|
||||||
|
* based on A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY
|
||||||
|
* and provided value of pStatus parameter
|
||||||
|
* @example exec FILE_MANAGER.SET_SOURCE_FILE_RECEIVED_STATUS(pSourceFileReceivedKey => 377, pStatus => 'READY_FOR_INGESTION');
|
||||||
|
**/
|
||||||
|
PROCEDURE SET_SOURCE_FILE_RECEIVED_STATUS(
|
||||||
|
pSourceFileReceivedKey IN PLS_INTEGER,
|
||||||
|
pStatus IN VARCHAR2
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_EXTERNAL_TABLE_COLUMNS
|
||||||
|
* @desc Function used to get string with all table columns definitions based on pTargetTableTemplate "TEMPLATE TABLE" name.
|
||||||
|
* It used for creating "EXTERNAL TABLE" using CREATE_EXTERNAL_TABLE procedure.
|
||||||
|
* @example select FILE_MANAGER.GET_EXTERNAL_TABLE_COLUMNS(pTargetTableTemplate => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES_HEADER') from dual;
|
||||||
|
* @ex_rslt "A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
* "A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
* "REV_NUMBER" NUMBER(28,0),
|
||||||
|
* "REF_DATE" DATE,
|
||||||
|
* "FREE_TEXT" VARCHAR2(1000 CHAR),
|
||||||
|
* "MLF_BS_TOTAL" NUMBER(28,10),
|
||||||
|
* "DF_BS_TOTAL" NUMBER(28,10),
|
||||||
|
* "MLF_SF_TOTAL" NUMBER(28,10),
|
||||||
|
* "DF_SF_TOTAL" NUMBER(28,10)
|
||||||
|
**/
|
||||||
|
FUNCTION GET_EXTERNAL_TABLE_COLUMNS (
|
||||||
|
pTargetTableTemplate IN VARCHAR2
|
||||||
|
)
|
||||||
|
RETURN CLOB;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name CREATE_EXTERNAL_TABLE
|
||||||
|
* @desc A wrapper procedure for DBMS_CLOUD.CREATE_EXTERNAL_TABLE which creates External Table
|
||||||
|
* MARS-1049: Added pEncoding parameter for CSV character set specification
|
||||||
|
* @param pEncoding - Character set encoding for CSV files (e.g., 'UTF8', 'WE8MSWIN1252')
|
||||||
|
* If provided, adds CHARACTERSET clause to external table definition
|
||||||
|
* @example
|
||||||
|
* begin
|
||||||
|
* FILE_MANAGER.CREATE_EXTERNAL_TABLE(
|
||||||
|
* pTableName => 'STANDING_FACILITIES_HEADER',
|
||||||
|
* pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES_HEADER',
|
||||||
|
* pPrefix => 'ODS/LM/STANDING_FACILITIES_HEADER/',
|
||||||
|
* pBucketUri => 'https://objectstorage.eu-frankfurt-1.oraclecloud.com/n/frcnomajoc7v/b/mrds_data_tst/o/',
|
||||||
|
* pFileName => NULL,
|
||||||
|
* pDelimiter => ',',
|
||||||
|
* pEncoding => 'UTF8'
|
||||||
|
* );
|
||||||
|
* end;
|
||||||
|
**/
|
||||||
|
PROCEDURE CREATE_EXTERNAL_TABLE (
|
||||||
|
pTableName IN VARCHAR2,
|
||||||
|
pTemplateTableName IN VARCHAR2,
|
||||||
|
pPrefix IN VARCHAR2,
|
||||||
|
pBucketUri IN VARCHAR2 DEFAULT ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pFileName IN VARCHAR2 DEFAULT NULL,
|
||||||
|
pDelimiter IN VARCHAR2 DEFAULT ',',
|
||||||
|
pEncoding IN VARCHAR2 DEFAULT NULL -- MARS-1049: NOWY PARAMETR
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name CREATE_EXTERNAL_TABLE
|
||||||
|
* @desc Creates External Table for single file provided by
|
||||||
|
* pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* @example exec FILE_MANAGER.CREATE_EXTERNAL_TABLE(pSourceFileReceivedKey => 377);;
|
||||||
|
**/
|
||||||
|
PROCEDURE CREATE_EXTERNAL_TABLE (
|
||||||
|
pSourceFileReceivedKey IN NUMBER
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name VALIDATE_SOURCE_FILE_RECEIVED
|
||||||
|
* @desc A wrapper procedure for DBMS_CLOUD.VALIDATE_EXTERNAL_TABLE
|
||||||
|
* It validate External table build upon single file
|
||||||
|
* provided by pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* @example exec FILE_MANAGER.VALIDATE_SOURCE_FILE_RECEIVED(pSourceFileReceivedKey => 377);
|
||||||
|
**/
|
||||||
|
PROCEDURE VALIDATE_SOURCE_FILE_RECEIVED
|
||||||
|
(
|
||||||
|
pSourceFileReceivedKey IN NUMBER
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name VALIDATE_EXTERNAL_TABLE
|
||||||
|
* @desc A wrapper function for DBMS_CLOUD.VALIDATE_EXTERNAL_TABLE.
|
||||||
|
* It validates External Table provided by parameter pTableName.
|
||||||
|
* It returns: PASSED or FAILED.
|
||||||
|
* @example
|
||||||
|
* declare
|
||||||
|
* vStatus VARCHAR2(100);
|
||||||
|
* begin
|
||||||
|
* vStatus := FILE_MANAGER.VALIDATE_EXTERNAL_TABLE(pTableName => 'STANDING_FACILITIES_HEADER');
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vStatus = '||vStatus);
|
||||||
|
* end;
|
||||||
|
*
|
||||||
|
* @ex_rslt FAILED
|
||||||
|
**/
|
||||||
|
FUNCTION VALIDATE_EXTERNAL_TABLE(pTableName IN VARCHAR2)
|
||||||
|
RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name S_VALIDATE_EXTERNAL_TABLE
|
||||||
|
* @desc A function which checks if SELECT query reterns any rows.
|
||||||
|
* It trys to selects External Table provided by parameter pTableName.
|
||||||
|
* It returns: PASSED or FAILED.
|
||||||
|
* @example
|
||||||
|
* declare
|
||||||
|
* vStatus VARCHAR2(100);
|
||||||
|
* begin
|
||||||
|
* vStatus := FILE_MANAGER.S_VALIDATE_EXTERNAL_TABLE(pTableName => 'STANDING_FACILITIES_HEADER');
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vStatus = '||vStatus);
|
||||||
|
* end;
|
||||||
|
*
|
||||||
|
* @ex_rslt PASSED
|
||||||
|
**/
|
||||||
|
FUNCTION S_VALIDATE_EXTERNAL_TABLE(pTableName IN VARCHAR2)
|
||||||
|
RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name DROP_EXTERNAL_TABLE
|
||||||
|
* @desc It drops External Table for single file provided by
|
||||||
|
* pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* @example exec FILE_MANAGER.DROP_EXTERNAL_TABLE(pSourceFileReceivedKey => 377);
|
||||||
|
**/
|
||||||
|
PROCEDURE DROP_EXTERNAL_TABLE (
|
||||||
|
pSourceFileReceivedKey IN NUMBER
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name COPY_FILE
|
||||||
|
* @desc It copies file provided by
|
||||||
|
* pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* into destination provided by pDestination parameter.
|
||||||
|
* pDestination parameter allowed values are: 'ODS'
|
||||||
|
* @example exec FILE_MANAGER.COPY_FILE(pSourceFileReceivedKey => 377, pDestination => 'ODS');
|
||||||
|
**/
|
||||||
|
PROCEDURE COPY_FILE(
|
||||||
|
pSourceFileReceivedKey IN NUMBER,
|
||||||
|
pDestination IN VARCHAR2
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name MOVE_FILE
|
||||||
|
* @desc It moves file provided by
|
||||||
|
* pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* into destination provided by pDestination parameter.
|
||||||
|
* pDestination parameter allowed values are: 'ODS', 'QUARANTINE'
|
||||||
|
* @example exec FILE_MANAGER.MOVE_FILE(pSourceFileReceivedKey => 377, pDestination => 'ODS');
|
||||||
|
**/
|
||||||
|
PROCEDURE MOVE_FILE(
|
||||||
|
pSourceFileReceivedKey IN NUMBER,
|
||||||
|
pDestination IN VARCHAR2
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name DELETE_FOLDER_CONTENTS
|
||||||
|
* @desc It deletes all files from specified folder in the cloud storage.
|
||||||
|
* The procedure lists all objects in the specified folder prefix and deletes them one by one.
|
||||||
|
* pBucketArea parameter specifies which bucket to use: 'INBOX', 'DATA', 'ARCHIVE'
|
||||||
|
* pFolderPrefix parameter specifies the folder path within the bucket (e.g., 'C2D/UC_DISSEM/UC_NMA_DISSEM/')
|
||||||
|
* @example exec FILE_MANAGER.DELETE_FOLDER_CONTENTS(pBucketArea => 'INBOX', pFolderPrefix => 'C2D/UC_DISSEM/UC_NMA_DISSEM/');
|
||||||
|
**/
|
||||||
|
PROCEDURE DELETE_FOLDER_CONTENTS(
|
||||||
|
pBucketArea IN VARCHAR2,
|
||||||
|
pFolderPrefix IN VARCHAR2
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name PROCESS_SOURCE_FILE
|
||||||
|
* @desc It process file provided by pSourceFileReceivedName parameter.
|
||||||
|
* Ubmrella procedure that calls:
|
||||||
|
* - REGISTER_SOURCE_FILE_RECEIVED;
|
||||||
|
* - CREATE_EXTERNAL_TABLE;
|
||||||
|
* - VALIDATE_SOURCE_FILE_RECEIVED;
|
||||||
|
* - DROP_EXTERNAL_TABLE;
|
||||||
|
* - MOVE_FILE;
|
||||||
|
* @example exec FILE_MANAGER.PROCESS_SOURCE_FILE(pSourceFileReceivedName => 'INBOX/C2D/UC_DISSEM/UC_NMA_DISSEM/UC_NMA_DISSEM-277740.csv');
|
||||||
|
**/
|
||||||
|
PROCEDURE PROCESS_SOURCE_FILE(pSourceFileReceivedName IN VARCHAR2)
|
||||||
|
;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name PROCESS_SOURCE_FILE
|
||||||
|
* @desc It process file provided by pSourceFileReceivedName parameter and return processing result value.
|
||||||
|
* It returns (success/failure) => 0 / -(value).
|
||||||
|
* Ubmrella function that calls PROCESS_SOURCE_FILE procedure.
|
||||||
|
* @example
|
||||||
|
* declare
|
||||||
|
* vResult PLS_INTEGER;
|
||||||
|
* begin
|
||||||
|
* vResult := CT_MRDS.FILE_MANAGER.PROCESS_SOURCE_FILE(PSOURCEFILERECEIVEDNAME => 'INBOX/C2D/UC_DISSEM/UC_NMA_DISSEM/UC_NMA_DISSEM-277740.csv');
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vResult = ' || vResult);
|
||||||
|
* end;
|
||||||
|
* @ex_rslt 0
|
||||||
|
* -20021
|
||||||
|
**/
|
||||||
|
FUNCTION PROCESS_SOURCE_FILE(pSourceFileReceivedName IN VARCHAR2)
|
||||||
|
RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_DATE_FORMAT
|
||||||
|
* @desc Returns date format for specified template table name and column name.
|
||||||
|
* Date is taken from configuration A_COLUMN_DATE_FORMAT table.
|
||||||
|
* @example select FILE_MANAGER.GET_DATE_FORMAT(
|
||||||
|
* pTemplateTableName => 'STANDING_FACILITIES_HEADER',
|
||||||
|
* pColumnName => 'SNAPSHOT_DATE')
|
||||||
|
* from dual;
|
||||||
|
* @ex_rslt DD/MM/YYYY HH24:MI:SS
|
||||||
|
**/
|
||||||
|
FUNCTION GET_DATE_FORMAT(
|
||||||
|
pTemplateTableName IN VARCHAR2,
|
||||||
|
pColumnName IN VARCHAR2
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GENERATE_EXTERNAL_TABLE_PARAMS
|
||||||
|
* @desc It builds two strings: pColumnList and pFieldList for specified Template Table name, by parameter: pTemplateTableName.
|
||||||
|
* @example
|
||||||
|
* declare
|
||||||
|
* vColumnList CLOB;
|
||||||
|
* vFieldList CLOB;
|
||||||
|
* begin
|
||||||
|
* FILE_MANAGER.GENERATE_EXTERNAL_TABLE_PARAMS (
|
||||||
|
* pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES_HEADER'
|
||||||
|
* ,pColumnList => vColumnList
|
||||||
|
* ,pFieldList => vFieldList
|
||||||
|
* );
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vColumnList = '||vColumnList);
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vFieldList = '||vFieldList);
|
||||||
|
* end;
|
||||||
|
* /
|
||||||
|
**/
|
||||||
|
PROCEDURE GENERATE_EXTERNAL_TABLE_PARAMS (
|
||||||
|
pTemplateTableName IN VARCHAR2,
|
||||||
|
pColumnList OUT CLOB,
|
||||||
|
pFieldList OUT CLOB
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name ADD_SOURCE
|
||||||
|
* @desc Insert a new record to A_SOURCE table.
|
||||||
|
* pSourceKey is a PRIMARY KEY value.
|
||||||
|
**/
|
||||||
|
PROCEDURE ADD_SOURCE (
|
||||||
|
pSourceKey IN CT_MRDS.A_SOURCE.A_SOURCE_KEY%TYPE,
|
||||||
|
pSourceName IN CT_MRDS.A_SOURCE.SOURCE_NAME%TYPE
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name DELETE_SOURCE_CASCADE
|
||||||
|
* @desc Safely deletes a SOURCE specified by pSourceKey parameter from A_SOURCE table and all dependent tables:
|
||||||
|
* - A_SOURCE_FILE_CONFIG
|
||||||
|
* - A_SOURCE_FILE_RECEIVED
|
||||||
|
* - A_COLUMN_DATE_FORMAT (only if template table is not shared with other source systems)
|
||||||
|
* The procedure checks if template tables are shared before deleting date format configurations.
|
||||||
|
* If a template table is used by multiple source systems, date formats are preserved.
|
||||||
|
* @example CALL CT_MRDS.FILE_MANAGER.DELETE_SOURCE_CASCADE(pSourceKey => 'TEST_SYS');
|
||||||
|
**/
|
||||||
|
PROCEDURE DELETE_SOURCE_CASCADE (
|
||||||
|
pSourceKey IN CT_MRDS.A_SOURCE.A_SOURCE_KEY%TYPE
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_CONTAINER_SOURCE_FILE_CONFIG_KEY
|
||||||
|
* @desc For specified parameter pSourceFileId (A_SOURCE_FILE_CONFIG.SOURCE_FILE_ID)
|
||||||
|
* it returns A_SOURCE_FILE_CONFIG.A_SOURCE_FILE_CONFIG_KEY for related CONTAINER record.
|
||||||
|
* @example select FILE_MANAGER.GET_CONTAINER_SOURCE_FILE_CONFIG_KEY(
|
||||||
|
* pSourceFileId => 'UC_DISSEM')
|
||||||
|
* from dual;
|
||||||
|
* @ex_rslt 126
|
||||||
|
**/
|
||||||
|
FUNCTION GET_CONTAINER_SOURCE_FILE_CONFIG_KEY (
|
||||||
|
pSourceFileId IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_ID%TYPE
|
||||||
|
) RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_SOURCE_FILE_CONFIG_KEY
|
||||||
|
* @desc For specified input parameters,
|
||||||
|
* it returns A_SOURCE_FILE_CONFIG.A_SOURCE_FILE_CONFIG_KEY.
|
||||||
|
* @example select FILE_MANAGER.GET_SOURCE_FILE_CONFIG_KEY (
|
||||||
|
* pSourceFileType => 'INPUT'
|
||||||
|
* ,pSourceFileId => 'UC_DISSEM'
|
||||||
|
* ,pTableId => 'UC_NMA_DISSEM')
|
||||||
|
* from dual;
|
||||||
|
* @ex_rslt 126
|
||||||
|
**/
|
||||||
|
FUNCTION GET_SOURCE_FILE_CONFIG_KEY (
|
||||||
|
pSourceFileType IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_TYPE%TYPE DEFAULT 'INPUT'
|
||||||
|
,pSourceFileId IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_ID%TYPE
|
||||||
|
,pTableId IN CT_MRDS.A_SOURCE_FILE_CONFIG.TABLE_ID%TYPE
|
||||||
|
) RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name ADD_SOURCE_FILE_CONFIG
|
||||||
|
* @desc Insert a new record to A_SOURCE_FILE_CONFIG table.
|
||||||
|
* MARS-1049: Added pEncoding parameter for CSV character set specification.
|
||||||
|
* @param pEncoding - Character set encoding for CSV files (e.g., 'UTF8', 'WE8MSWIN1252', 'EE8ISO8859P2')
|
||||||
|
* If NULL, no CHARACTERSET clause is added to external table definitions
|
||||||
|
* @example CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
* pSourceKey => 'C2D', pSourceFileType => 'INPUT',
|
||||||
|
* pSourceFileId => 'UC_DISSEM', pTableId => 'METADATA_LOADS',
|
||||||
|
* pTemplateTableName => 'CT_ET_TEMPLATES.C2D_A_UC_DISSEM_METADATA_LOADS',
|
||||||
|
* pEncoding => 'UTF8'
|
||||||
|
* );
|
||||||
|
**/
|
||||||
|
PROCEDURE ADD_SOURCE_FILE_CONFIG (
|
||||||
|
pSourceKey IN CT_MRDS.A_SOURCE_FILE_CONFIG.A_SOURCE_KEY%TYPE
|
||||||
|
,pSourceFileType IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_TYPE%TYPE
|
||||||
|
,pSourceFileId IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_ID%TYPE
|
||||||
|
,pSourceFileDesc IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_DESC%TYPE
|
||||||
|
,pSourceFileNamePattern IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_NAME_PATTERN%TYPE
|
||||||
|
,pTableId IN CT_MRDS.A_SOURCE_FILE_CONFIG.TABLE_ID%TYPE DEFAULT NULL
|
||||||
|
,pTemplateTableName IN CT_MRDS.A_SOURCE_FILE_CONFIG.TEMPLATE_TABLE_NAME%TYPE DEFAULT NULL
|
||||||
|
,pContainerFileKey IN CT_MRDS.A_SOURCE_FILE_CONFIG.CONTAINER_FILE_KEY%TYPE DEFAULT NULL
|
||||||
|
,pEncoding IN CT_MRDS.A_SOURCE_FILE_CONFIG.ENCODING%TYPE DEFAULT NULL -- MARS-1049: NOWY PARAMETR
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name ADD_COLUMN_DATE_FORMAT
|
||||||
|
* @desc Insert a new record to A_COLUMN_DATE_FORMAT table.
|
||||||
|
**/
|
||||||
|
PROCEDURE ADD_COLUMN_DATE_FORMAT (
|
||||||
|
pTemplateTableName IN CT_MRDS.A_COLUMN_DATE_FORMAT.TEMPLATE_TABLE_NAME%TYPE
|
||||||
|
,pColumnName IN CT_MRDS.A_COLUMN_DATE_FORMAT.COLUMN_NAME%TYPE
|
||||||
|
,pDateFormat IN CT_MRDS.A_COLUMN_DATE_FORMAT.DATE_FORMAT%TYPE
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_BUCKET_URI
|
||||||
|
* @desc Function used to get string with bucket http url.
|
||||||
|
* Possible input values for pBucketArea are: 'INBOX', 'ODS', 'DATA', 'ARCHIVE'
|
||||||
|
* @example select FILE_MANAGER.GET_BUCKET_URI(pBucketArea => 'ODS') from dual;
|
||||||
|
* @ex_rslt https://objectstorage.eu-frankfurt-1.oraclecloud.com/n/frcnomajoc7v/b/mrds_data_tst/o/
|
||||||
|
**/
|
||||||
|
FUNCTION GET_BUCKET_URI(pBucketArea VARCHAR2)
|
||||||
|
RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_DET_SOURCE_FILE_CONFIG_INFO
|
||||||
|
* @desc Function returns details about A_SOURCE_FILE_CONFIG record
|
||||||
|
* for specified pSourceFileConfigKey (A_SOURCE_FILE_CONFIG.A_SOURCE_FILE_CONFIG_KEY).
|
||||||
|
* If pIncludeContainerInfo is <> 0 it returns additional info about related Container config record (A_SOURCE_FILE_CONFIG)
|
||||||
|
* If pIncludeColumnFormatInfo is <> 0 it returns additional info about related ColumnFormat config record (A_COLUMN_DATE_FORMAT)
|
||||||
|
* @example select FILE_MANAGER.GET_DET_SOURCE_FILE_CONFIG_INFO (
|
||||||
|
* pSourceFileConfigKey => 128
|
||||||
|
* ,pIncludeContainerInfo => 1
|
||||||
|
* ,pIncludeColumnFormatInfo => 1
|
||||||
|
* ) from dual;
|
||||||
|
* @ex_rslt
|
||||||
|
* Details about File Configuration:
|
||||||
|
* --------------------------------
|
||||||
|
* A_SOURCE_FILE_CONFIG_KEY = 128
|
||||||
|
* A_SOURCE_KEY = C2D
|
||||||
|
* ...
|
||||||
|
* --------------------------------
|
||||||
|
*
|
||||||
|
* Details about related Container Config:
|
||||||
|
* --------------------------------
|
||||||
|
* A_SOURCE_FILE_CONFIG_KEY = 126
|
||||||
|
* A_SOURCE_KEY = C2D
|
||||||
|
* ...
|
||||||
|
* --------------------------------
|
||||||
|
*
|
||||||
|
* Column Date Format config entries:
|
||||||
|
* --------------------------------
|
||||||
|
* TEMPLATE_TABLE_NAME = CT_ET_TEMPLATES.C2D_UC_MA_DISSEM
|
||||||
|
* ...
|
||||||
|
* --------------------------------
|
||||||
|
**/
|
||||||
|
FUNCTION GET_DET_SOURCE_FILE_CONFIG_INFO (
|
||||||
|
pSourceFileConfigKey IN CT_MRDS.A_SOURCE_FILE_CONFIG.A_SOURCE_FILE_CONFIG_KEY%TYPE
|
||||||
|
,pIncludeContainerInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
,pIncludeColumnFormatInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_DET_SOURCE_FILE_RECEIVED_INFO
|
||||||
|
* @desc Function returns details about A_SOURCE_FILE_RECEIVED record
|
||||||
|
* for specified pSourceFileReceivedKey (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY).
|
||||||
|
* If pIncludeConfigInfo is <> 0 it returns additional info about related Container config record (A_SOURCE_FILE_CONFIG)
|
||||||
|
* If pIncludeContainerInfo is <> 0 it returns additional info about related Container config record (A_SOURCE_FILE_CONFIG)
|
||||||
|
* If pIncludeColumnFormatInfo is <> 0 it returns additional info about related ColumnFormat config record (A_COLUMN_DATE_FORMAT)
|
||||||
|
* @example select FILE_MANAGER.GET_DET_SOURCE_FILE_RECEIVED_INFO (
|
||||||
|
* pSourceFileReceivedKey => 377
|
||||||
|
* ,pIncludeConfigInfo => 1
|
||||||
|
* ,pIncludeContainerInfo => 1
|
||||||
|
* ,pIncludeColumnFormatInfo => 1
|
||||||
|
* ) from dual;
|
||||||
|
*
|
||||||
|
**/
|
||||||
|
FUNCTION GET_DET_SOURCE_FILE_RECEIVED_INFO (
|
||||||
|
pSourceFileReceivedKey IN CT_MRDS.A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY%TYPE
|
||||||
|
,pIncludeConfigInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
,pIncludeContainerInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
,pIncludeColumnFormatInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_DET_USER_LOAD_OPERATIONS
|
||||||
|
* @desc Function returns details from USER_LOAD_OPERATIONS table
|
||||||
|
* for specified pOperationId.
|
||||||
|
* @example select FILE_MANAGER.GET_DET_USER_LOAD_OPERATIONS (pOperationId => 3608) from dual;
|
||||||
|
* @ex_rslt
|
||||||
|
* Details about USER_LOAD_OPERATIONS where ID = 3608
|
||||||
|
* --------------------------------
|
||||||
|
* ID = 3608
|
||||||
|
* TYPE = VALIDATE
|
||||||
|
* SID = 31260
|
||||||
|
* SERIAL# = 52915
|
||||||
|
* START_TIME = 2025-05-20 10.08.24.436983 EUROPE/BELGRADE
|
||||||
|
* UPDATE_TIME = 2025-05-20 10.08.24.458643 EUROPE/BELGRADE
|
||||||
|
* STATUS = FAILED
|
||||||
|
* OWNER_NAME = CT_MRDS
|
||||||
|
* TABLE_NAME = STANDING_FACILITIES_HEADER
|
||||||
|
* PARTITION_NAME =
|
||||||
|
* SUBPARTITION_NAME =
|
||||||
|
* FILE_URI_LIST =
|
||||||
|
* ROWS_LOADED =
|
||||||
|
* LOGFILE_TABLE = VALIDATE$3608_LOG
|
||||||
|
* BADFILE_TABLE = VALIDATE$3608_BAD
|
||||||
|
* STATUS_TABLE =
|
||||||
|
* TEMPEXT_TABLE =
|
||||||
|
* CREDENTIAL_NAME =
|
||||||
|
* EXPIRATION_TIME = 2025-05-22 10.08.24.436983000 EUROPE/BELGRADE
|
||||||
|
* --------------------------------
|
||||||
|
**/
|
||||||
|
FUNCTION GET_DET_USER_LOAD_OPERATIONS (
|
||||||
|
pOperationId PLS_INTEGER
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name ANALYZE_VALIDATION_ERRORS
|
||||||
|
* @desc Wrapper function that analyzes validation errors for a source file using its received key.
|
||||||
|
* Automatically derives template schema, table name, CSV URI and validation log table
|
||||||
|
* from file metadata and calls ENV_MANAGER.ANALYZE_VALIDATION_ERRORS.
|
||||||
|
* @example SELECT FILE_MANAGER.ANALYZE_VALIDATION_ERRORS(63) FROM DUAL;
|
||||||
|
* @ex_rslt Detailed validation analysis report with column mismatches and solutions
|
||||||
|
**/
|
||||||
|
FUNCTION ANALYZE_VALIDATION_ERRORS(
|
||||||
|
pSourceFileReceivedKey IN NUMBER
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
-- PACKAGE VERSION MANAGEMENT FUNCTIONS
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_VERSION
|
||||||
|
* @desc Returns the current version number of the FILE_MANAGER package.
|
||||||
|
* Uses semantic versioning format (MAJOR.MINOR.PATCH).
|
||||||
|
* @example SELECT FILE_MANAGER.GET_VERSION() FROM DUAL;
|
||||||
|
* @ex_rslt 3.2.0
|
||||||
|
**/
|
||||||
|
FUNCTION GET_VERSION RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_BUILD_INFO
|
||||||
|
* @desc Returns comprehensive build information including version, build date, and author.
|
||||||
|
* Uses centralized ENV_MANAGER.GET_PACKAGE_VERSION_INFO function.
|
||||||
|
* @example SELECT FILE_MANAGER.GET_BUILD_INFO() FROM DUAL;
|
||||||
|
* @ex_rslt Package: FILE_MANAGER
|
||||||
|
* Version: 3.2.0
|
||||||
|
* Build Date: 2025-10-22 16:30:00
|
||||||
|
* Author: Grzegorz Michalski
|
||||||
|
**/
|
||||||
|
FUNCTION GET_BUILD_INFO RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_VERSION_HISTORY
|
||||||
|
* @desc Returns complete version history with all releases and changes.
|
||||||
|
* Uses centralized ENV_MANAGER.FORMAT_VERSION_HISTORY function.
|
||||||
|
* @example SELECT FILE_MANAGER.GET_VERSION_HISTORY() FROM DUAL;
|
||||||
|
* @ex_rslt FILE_MANAGER Version History:
|
||||||
|
* 3.2.0 (2025-10-22): Added package versioning system...
|
||||||
|
**/
|
||||||
|
FUNCTION GET_VERSION_HISTORY RETURN VARCHAR2;
|
||||||
|
|
||||||
|
END;
|
||||||
|
|
||||||
|
/
|
||||||
|
|
||||||
|
/
|
||||||
99
MARS_Packages/REL01/MARS-1056/install_mars1056.sql
Normal file
99
MARS_Packages/REL01/MARS-1056/install_mars1056.sql
Normal file
@@ -0,0 +1,99 @@
|
|||||||
|
/*
|
||||||
|
================================================================================
|
||||||
|
MARS-1056: Master Installation Script for VARCHAR2 Definition Fixes
|
||||||
|
Author: Grzegorz Michalski
|
||||||
|
Date: November 25, 2025
|
||||||
|
Purpose: Main installation script for VARCHAR2 definition fixes in external tables
|
||||||
|
|
||||||
|
PROBLEM:
|
||||||
|
FILE_MANAGER.GENERATE_EXTERNAL_TABLE_PARAMS uses data_length instead of char_length
|
||||||
|
for VARCHAR2 columns, causing incorrect definitions in external tables.
|
||||||
|
|
||||||
|
SOLUTION:
|
||||||
|
Modify column definition generation logic to preserve CHAR/BYTE semantics
|
||||||
|
from template tables.
|
||||||
|
|
||||||
|
USAGE:
|
||||||
|
sql 'CT_MRDS/Cloudpass#34@ggmichalski_high' '@install_mars1056.sql'
|
||||||
|
================================================================================
|
||||||
|
*/
|
||||||
|
|
||||||
|
-- Dynamic spool file generation (using SYS_CONTEXT - no DBA privileges required)
|
||||||
|
var filename VARCHAR2(100)
|
||||||
|
BEGIN
|
||||||
|
:filename := 'INSTALL_MARS_1056_' || SYS_CONTEXT('USERENV', 'CON_NAME') || '_' || TO_CHAR(SYSDATE,'YYYYMMDD_HH24MISS') || '.log';
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
column filename new_value _filename
|
||||||
|
select :filename filename from dual;
|
||||||
|
spool &_filename
|
||||||
|
|
||||||
|
SET ECHO OFF
|
||||||
|
SET TIMING ON
|
||||||
|
SET SERVEROUTPUT ON SIZE UNLIMITED
|
||||||
|
SET PAUSE OFF
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT ========================================================================
|
||||||
|
PROMPT MARS-1056: VARCHAR2 Definition Fixes Installation
|
||||||
|
PROMPT ========================================================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT Problem: FILE_MANAGER creates external tables with incorrect VARCHAR2 definitions
|
||||||
|
PROMPT Solution: Preserve CHAR/BYTE semantics from template tables
|
||||||
|
PROMPT
|
||||||
|
PROMPT Installation will:
|
||||||
|
PROMPT 1. Install updated FILE_MANAGER with VARCHAR2 fixes (v3.3.0)
|
||||||
|
PROMPT 2. Track package version
|
||||||
|
PROMPT 3. Verify all tracked packages
|
||||||
|
PROMPT
|
||||||
|
PROMPT ========================================================================
|
||||||
|
|
||||||
|
-- Confirm installation with user
|
||||||
|
ACCEPT continue CHAR PROMPT 'Type YES to continue with installation, or Ctrl+C to abort: '
|
||||||
|
WHENEVER SQLERROR EXIT SQL.SQLCODE
|
||||||
|
BEGIN
|
||||||
|
IF '&continue' IS NULL OR TRIM('&continue') IS NULL OR UPPER(TRIM('&continue')) != 'YES' THEN
|
||||||
|
RAISE_APPLICATION_ERROR(-20001, 'Installation aborted by user');
|
||||||
|
END IF;
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
WHENEVER SQLERROR CONTINUE
|
||||||
|
|
||||||
|
-- Set current schema context
|
||||||
|
ALTER SESSION SET CURRENT_SCHEMA = CT_MRDS;
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT === Step 1: Install updated FILE_MANAGER package ===
|
||||||
|
@@01_MARS_1056_update_FILE_MANAGER_package.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT === Step 2: Track package version ===
|
||||||
|
@@track_package_versions.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT === Step 3: Verify all tracked packages ===
|
||||||
|
@@verify_packages_version.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT ========================================================================
|
||||||
|
PROMPT MARS-1056: Installation completed successfully
|
||||||
|
PROMPT ========================================================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT VARCHAR2 definition fixes have been implemented:
|
||||||
|
PROMPT - GENERATE_EXTERNAL_TABLE_PARAMS now preserves CHAR/BYTE semantics
|
||||||
|
PROMPT - External tables will match template table definitions
|
||||||
|
PROMPT - Test results shown above should indicate SUCCESS
|
||||||
|
PROMPT
|
||||||
|
PROMPT Next steps:
|
||||||
|
PROMPT 1. Verify test results above show all SUCCESS indicators
|
||||||
|
PROMPT 2. Test external table creation with various templates
|
||||||
|
PROMPT 3. Monitor for any issues with existing processes
|
||||||
|
PROMPT 4. If problems occur, use rollback_mars1056.sql
|
||||||
|
PROMPT
|
||||||
|
PROMPT ========================================================================
|
||||||
|
PROMPT Installation log saved to: &_filename
|
||||||
|
PROMPT ========================================================================
|
||||||
|
|
||||||
|
spool off
|
||||||
|
|
||||||
|
quit;
|
||||||
1939
MARS_Packages/REL01/MARS-1056/new_version/FILE_MANAGER.pkb
Normal file
1939
MARS_Packages/REL01/MARS-1056/new_version/FILE_MANAGER.pkb
Normal file
File diff suppressed because it is too large
Load Diff
636
MARS_Packages/REL01/MARS-1056/new_version/FILE_MANAGER.pkg
Normal file
636
MARS_Packages/REL01/MARS-1056/new_version/FILE_MANAGER.pkg
Normal file
@@ -0,0 +1,636 @@
|
|||||||
|
create or replace PACKAGE CT_MRDS.FILE_MANAGER
|
||||||
|
AUTHID CURRENT_USER
|
||||||
|
AS
|
||||||
|
/**
|
||||||
|
* General comment for package: Please put comments for functions and procedures as shown in below example.
|
||||||
|
* It is a standard.
|
||||||
|
* The structure of comment is used by GET_PACKAGE_DOCUMENTATION function
|
||||||
|
* which returns documentation text for confluence page (to Copy-Paste it).
|
||||||
|
**/
|
||||||
|
|
||||||
|
-- Example comment:
|
||||||
|
/**
|
||||||
|
* @name EX_PROCEDURE_NAME
|
||||||
|
* @desc Procedure description
|
||||||
|
* @example select FILE_MANAGER.EX_PROCEDURE_NAME(pParameter => 129) from dual;
|
||||||
|
* @ex_rslt Example Result
|
||||||
|
**/
|
||||||
|
|
||||||
|
-- Package Version Information (Semantic Versioning: MAJOR.MINOR.PATCH)
|
||||||
|
PACKAGE_VERSION CONSTANT VARCHAR2(10) := '3.3.0';
|
||||||
|
PACKAGE_BUILD_DATE CONSTANT VARCHAR2(20) := '2025-11-26 10:00:00';
|
||||||
|
PACKAGE_AUTHOR CONSTANT VARCHAR2(100) := 'Grzegorz Michalski';
|
||||||
|
|
||||||
|
-- Version History (Latest changes first)
|
||||||
|
VERSION_HISTORY CONSTANT VARCHAR2(4000) :=
|
||||||
|
'3.3.0 (2025-11-26): MARS-1056 - Fixed VARCHAR2 definitions in GENERATE_EXTERNAL_TABLE_PARAMS to preserve CHAR/BYTE semantics from template tables' || CHR(13)||CHR(10) ||
|
||||||
|
'3.2.1 (2025-11-24): MARS-1049 - Added pEncoding parameter support for CSV character set specification' || CHR(13)||CHR(10) ||
|
||||||
|
'3.2.0 (2025-10-22): Added package versioning system using centralized ENV_MANAGER functions' || CHR(13)||CHR(10) ||
|
||||||
|
'3.1.0 (2025-10-20): Enhanced PROCESS_SOURCE_FILE with 6-step validation workflow' || CHR(13)||CHR(10) ||
|
||||||
|
'3.0.0 (2025-10-15): Separated export procedures into dedicated DATA_EXPORTER package' || CHR(13)||CHR(10) ||
|
||||||
|
'2.5.0 (2025-10-10): Added DELETE_SOURCE_CASCADE for safe configuration removal' || CHR(13)||CHR(10) ||
|
||||||
|
'2.0.0 (2025-09-25): Added official path patterns support (INBOX 3-level, ODS 2-level, ARCHIVE 2-level)' || CHR(13)||CHR(10) ||
|
||||||
|
'1.0.0 (2025-09-01): Initial release with file processing and validation capabilities';
|
||||||
|
|
||||||
|
TYPE tSourceFileReceived IS RECORD
|
||||||
|
(
|
||||||
|
A_SOURCE_FILE_RECEIVED_KEY CT_MRDS.A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY%TYPE,
|
||||||
|
A_SOURCE_FILE_CONFIG_KEY CT_MRDS.A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_CONFIG_KEY%TYPE,
|
||||||
|
SOURCE_FILE_PREFIX_INBOX VARCHAR2(430),
|
||||||
|
SOURCE_FILE_PREFIX_ODS VARCHAR2(430),
|
||||||
|
SOURCE_FILE_PREFIX_QUARANTINE VARCHAR2(430),
|
||||||
|
SOURCE_FILE_PREFIX_ARCHIVE VARCHAR2(430),
|
||||||
|
SOURCE_FILE_NAME CT_MRDS.A_SOURCE_FILE_RECEIVED.SOURCE_FILE_NAME%TYPE,
|
||||||
|
RECEPTION_DATE CT_MRDS.A_SOURCE_FILE_RECEIVED.RECEPTION_DATE%TYPE,
|
||||||
|
PROCESSING_STATUS CT_MRDS.A_SOURCE_FILE_RECEIVED.PROCESSING_STATUS%TYPE,
|
||||||
|
EXTERNAL_TABLE_NAME CT_MRDS.A_SOURCE_FILE_RECEIVED.EXTERNAL_TABLE_NAME%TYPE
|
||||||
|
);
|
||||||
|
|
||||||
|
cgBL CONSTANT VARCHAR2(2) := CHR(13)||CHR(10);
|
||||||
|
vgSourceFileConfigKey PLS_INTEGER;
|
||||||
|
vgMsgTmp VARCHAR2(32000);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_SOURCE_FILE_CONFIG
|
||||||
|
* @desc Get source file type by matching the source file name against source file type naming patterns
|
||||||
|
* or by specifying the id of a received source file.
|
||||||
|
* @example ...
|
||||||
|
* @ex_rslt "CT_MRDS.A_SOURCE_FILE_CONFIG%ROWTYPE"
|
||||||
|
**/
|
||||||
|
FUNCTION GET_SOURCE_FILE_CONFIG(pFileUri IN VARCHAR2 DEFAULT NULL
|
||||||
|
, pSourceFileReceivedKey IN NUMBER DEFAULT NULL
|
||||||
|
, pSourceFileConfigKey IN NUMBER DEFAULT NULL)
|
||||||
|
RETURN CT_MRDS.A_SOURCE_FILE_CONFIG%ROWTYPE;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name REGISTER_SOURCE_FILE_RECEIVED
|
||||||
|
* @desc Register a newly received source file in A_SOURCE_FILE_RECEIVED table.
|
||||||
|
* This overload automatically determines source file type from the file name.
|
||||||
|
* It returns the value of A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY column for newly added record.
|
||||||
|
* @example vSourceFileReceivedKey := FILE_MANAGER.REGISTER_SOURCE_FILE_RECEIVED(pSourceFileReceivedName => 'INBOX/C2D/UC_DISSEM/UC_NMA_DISSEM/UC_NMA_DISSEM-277740.csv');
|
||||||
|
* @ex_rslt 3245
|
||||||
|
**/
|
||||||
|
FUNCTION REGISTER_SOURCE_FILE_RECEIVED (
|
||||||
|
pSourceFileReceivedName IN VARCHAR2
|
||||||
|
)
|
||||||
|
RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name REGISTER_SOURCE_FILE_RECEIVED
|
||||||
|
* @desc Register a new new source file in A_SOURCE_FILE_RECEIVED table based on pSourceFileReceivedName and pSourceFileConfig.
|
||||||
|
* Then it returns the value of A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY column for newly added record.
|
||||||
|
* @example vSourceFileReceivedKey := FILE_MANAGER.REGISTER_SOURCE_FILE_RECEIVED(
|
||||||
|
* pSourceFileReceivedName => 'INBOX/C2D/UC_DISSEM/UC_NMA_DISSEM/UC_NMA_DISSEM-277740.csv'
|
||||||
|
* ,pSourceFileConfig => ...A_SOURCE_FILE_CONFIG%ROWTYPE... );
|
||||||
|
* @ex_rslt 3245
|
||||||
|
**/
|
||||||
|
FUNCTION REGISTER_SOURCE_FILE_RECEIVED (
|
||||||
|
pSourceFileReceivedName IN VARCHAR2,
|
||||||
|
pSourceFileConfig IN CT_MRDS.A_SOURCE_FILE_CONFIG%ROWTYPE
|
||||||
|
)
|
||||||
|
RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name SET_SOURCE_FILE_RECEIVED_STATUS
|
||||||
|
* @desc Set status of file in A_SOURCE_FILE_RECEIVED table - PROCESSING_STATUS column
|
||||||
|
* based on A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY
|
||||||
|
* and provided value of pStatus parameter
|
||||||
|
* @example exec FILE_MANAGER.SET_SOURCE_FILE_RECEIVED_STATUS(pSourceFileReceivedKey => 377, pStatus => 'READY_FOR_INGESTION');
|
||||||
|
**/
|
||||||
|
PROCEDURE SET_SOURCE_FILE_RECEIVED_STATUS(
|
||||||
|
pSourceFileReceivedKey IN PLS_INTEGER,
|
||||||
|
pStatus IN VARCHAR2
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_EXTERNAL_TABLE_COLUMNS
|
||||||
|
* @desc Function used to get string with all table columns definitions based on pTargetTableTemplate "TEMPLATE TABLE" name.
|
||||||
|
* It used for creating "EXTERNAL TABLE" using CREATE_EXTERNAL_TABLE procedure.
|
||||||
|
* @example select FILE_MANAGER.GET_EXTERNAL_TABLE_COLUMNS(pTargetTableTemplate => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES_HEADER') from dual;
|
||||||
|
* @ex_rslt "A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
* "A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
* "REV_NUMBER" NUMBER(28,0),
|
||||||
|
* "REF_DATE" DATE,
|
||||||
|
* "FREE_TEXT" VARCHAR2(1000 CHAR),
|
||||||
|
* "MLF_BS_TOTAL" NUMBER(28,10),
|
||||||
|
* "DF_BS_TOTAL" NUMBER(28,10),
|
||||||
|
* "MLF_SF_TOTAL" NUMBER(28,10),
|
||||||
|
* "DF_SF_TOTAL" NUMBER(28,10)
|
||||||
|
**/
|
||||||
|
FUNCTION GET_EXTERNAL_TABLE_COLUMNS (
|
||||||
|
pTargetTableTemplate IN VARCHAR2
|
||||||
|
)
|
||||||
|
RETURN CLOB;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name CREATE_EXTERNAL_TABLE
|
||||||
|
* @desc A wrapper procedure for DBMS_CLOUD.CREATE_EXTERNAL_TABLE which creates External Table
|
||||||
|
* MARS-1049: Added pEncoding parameter for CSV character set specification
|
||||||
|
* @param pEncoding - Character set encoding for CSV files (e.g., 'UTF8', 'WE8MSWIN1252')
|
||||||
|
* If provided, adds CHARACTERSET clause to external table definition
|
||||||
|
* @example
|
||||||
|
* begin
|
||||||
|
* FILE_MANAGER.CREATE_EXTERNAL_TABLE(
|
||||||
|
* pTableName => 'STANDING_FACILITIES_HEADER',
|
||||||
|
* pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES_HEADER',
|
||||||
|
* pPrefix => 'ODS/LM/STANDING_FACILITIES_HEADER/',
|
||||||
|
* pBucketUri => 'https://objectstorage.eu-frankfurt-1.oraclecloud.com/n/frcnomajoc7v/b/mrds_data_tst/o/',
|
||||||
|
* pFileName => NULL,
|
||||||
|
* pDelimiter => ',',
|
||||||
|
* pEncoding => 'UTF8'
|
||||||
|
* );
|
||||||
|
* end;
|
||||||
|
**/
|
||||||
|
PROCEDURE CREATE_EXTERNAL_TABLE (
|
||||||
|
pTableName IN VARCHAR2,
|
||||||
|
pTemplateTableName IN VARCHAR2,
|
||||||
|
pPrefix IN VARCHAR2,
|
||||||
|
pBucketUri IN VARCHAR2 DEFAULT ENV_MANAGER.gvInboxBucketUri,
|
||||||
|
pFileName IN VARCHAR2 DEFAULT NULL,
|
||||||
|
pDelimiter IN VARCHAR2 DEFAULT ',',
|
||||||
|
pEncoding IN VARCHAR2 DEFAULT NULL -- MARS-1049: NOWY PARAMETR
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name CREATE_EXTERNAL_TABLE
|
||||||
|
* @desc Creates External Table for single file provided by
|
||||||
|
* pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* @example exec FILE_MANAGER.CREATE_EXTERNAL_TABLE(pSourceFileReceivedKey => 377);;
|
||||||
|
**/
|
||||||
|
PROCEDURE CREATE_EXTERNAL_TABLE (
|
||||||
|
pSourceFileReceivedKey IN NUMBER
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name VALIDATE_SOURCE_FILE_RECEIVED
|
||||||
|
* @desc A wrapper procedure for DBMS_CLOUD.VALIDATE_EXTERNAL_TABLE
|
||||||
|
* It validate External table build upon single file
|
||||||
|
* provided by pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* @example exec FILE_MANAGER.VALIDATE_SOURCE_FILE_RECEIVED(pSourceFileReceivedKey => 377);
|
||||||
|
**/
|
||||||
|
PROCEDURE VALIDATE_SOURCE_FILE_RECEIVED
|
||||||
|
(
|
||||||
|
pSourceFileReceivedKey IN NUMBER
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name VALIDATE_EXTERNAL_TABLE
|
||||||
|
* @desc A wrapper function for DBMS_CLOUD.VALIDATE_EXTERNAL_TABLE.
|
||||||
|
* It validates External Table provided by parameter pTableName.
|
||||||
|
* It returns: PASSED or FAILED.
|
||||||
|
* @example
|
||||||
|
* declare
|
||||||
|
* vStatus VARCHAR2(100);
|
||||||
|
* begin
|
||||||
|
* vStatus := FILE_MANAGER.VALIDATE_EXTERNAL_TABLE(pTableName => 'STANDING_FACILITIES_HEADER');
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vStatus = '||vStatus);
|
||||||
|
* end;
|
||||||
|
*
|
||||||
|
* @ex_rslt FAILED
|
||||||
|
**/
|
||||||
|
FUNCTION VALIDATE_EXTERNAL_TABLE(pTableName IN VARCHAR2)
|
||||||
|
RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name S_VALIDATE_EXTERNAL_TABLE
|
||||||
|
* @desc A function which checks if SELECT query reterns any rows.
|
||||||
|
* It trys to selects External Table provided by parameter pTableName.
|
||||||
|
* It returns: PASSED or FAILED.
|
||||||
|
* @example
|
||||||
|
* declare
|
||||||
|
* vStatus VARCHAR2(100);
|
||||||
|
* begin
|
||||||
|
* vStatus := FILE_MANAGER.S_VALIDATE_EXTERNAL_TABLE(pTableName => 'STANDING_FACILITIES_HEADER');
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vStatus = '||vStatus);
|
||||||
|
* end;
|
||||||
|
*
|
||||||
|
* @ex_rslt PASSED
|
||||||
|
**/
|
||||||
|
FUNCTION S_VALIDATE_EXTERNAL_TABLE(pTableName IN VARCHAR2)
|
||||||
|
RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name DROP_EXTERNAL_TABLE
|
||||||
|
* @desc It drops External Table for single file provided by
|
||||||
|
* pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* @example exec FILE_MANAGER.DROP_EXTERNAL_TABLE(pSourceFileReceivedKey => 377);
|
||||||
|
**/
|
||||||
|
PROCEDURE DROP_EXTERNAL_TABLE (
|
||||||
|
pSourceFileReceivedKey IN NUMBER
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name COPY_FILE
|
||||||
|
* @desc It copies file provided by
|
||||||
|
* pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* into destination provided by pDestination parameter.
|
||||||
|
* pDestination parameter allowed values are: 'ODS'
|
||||||
|
* @example exec FILE_MANAGER.COPY_FILE(pSourceFileReceivedKey => 377, pDestination => 'ODS');
|
||||||
|
**/
|
||||||
|
PROCEDURE COPY_FILE(
|
||||||
|
pSourceFileReceivedKey IN NUMBER,
|
||||||
|
pDestination IN VARCHAR2
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name MOVE_FILE
|
||||||
|
* @desc It moves file provided by
|
||||||
|
* pSourceFileReceivedKey parameter (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY)
|
||||||
|
* into destination provided by pDestination parameter.
|
||||||
|
* pDestination parameter allowed values are: 'ODS', 'QUARANTINE'
|
||||||
|
* @example exec FILE_MANAGER.MOVE_FILE(pSourceFileReceivedKey => 377, pDestination => 'ODS');
|
||||||
|
**/
|
||||||
|
PROCEDURE MOVE_FILE(
|
||||||
|
pSourceFileReceivedKey IN NUMBER,
|
||||||
|
pDestination IN VARCHAR2
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name DELETE_FOLDER_CONTENTS
|
||||||
|
* @desc It deletes all files from specified folder in the cloud storage.
|
||||||
|
* The procedure lists all objects in the specified folder prefix and deletes them one by one.
|
||||||
|
* pBucketArea parameter specifies which bucket to use: 'INBOX', 'DATA', 'ARCHIVE'
|
||||||
|
* pFolderPrefix parameter specifies the folder path within the bucket (e.g., 'C2D/UC_DISSEM/UC_NMA_DISSEM/')
|
||||||
|
* @example exec FILE_MANAGER.DELETE_FOLDER_CONTENTS(pBucketArea => 'INBOX', pFolderPrefix => 'C2D/UC_DISSEM/UC_NMA_DISSEM/');
|
||||||
|
**/
|
||||||
|
PROCEDURE DELETE_FOLDER_CONTENTS(
|
||||||
|
pBucketArea IN VARCHAR2,
|
||||||
|
pFolderPrefix IN VARCHAR2
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name PROCESS_SOURCE_FILE
|
||||||
|
* @desc It process file provided by pSourceFileReceivedName parameter.
|
||||||
|
* Ubmrella procedure that calls:
|
||||||
|
* - REGISTER_SOURCE_FILE_RECEIVED;
|
||||||
|
* - CREATE_EXTERNAL_TABLE;
|
||||||
|
* - VALIDATE_SOURCE_FILE_RECEIVED;
|
||||||
|
* - DROP_EXTERNAL_TABLE;
|
||||||
|
* - MOVE_FILE;
|
||||||
|
* @example exec FILE_MANAGER.PROCESS_SOURCE_FILE(pSourceFileReceivedName => 'INBOX/C2D/UC_DISSEM/UC_NMA_DISSEM/UC_NMA_DISSEM-277740.csv');
|
||||||
|
**/
|
||||||
|
PROCEDURE PROCESS_SOURCE_FILE(pSourceFileReceivedName IN VARCHAR2)
|
||||||
|
;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name PROCESS_SOURCE_FILE
|
||||||
|
* @desc It process file provided by pSourceFileReceivedName parameter and return processing result value.
|
||||||
|
* It returns (success/failure) => 0 / -(value).
|
||||||
|
* Ubmrella function that calls PROCESS_SOURCE_FILE procedure.
|
||||||
|
* @example
|
||||||
|
* declare
|
||||||
|
* vResult PLS_INTEGER;
|
||||||
|
* begin
|
||||||
|
* vResult := CT_MRDS.FILE_MANAGER.PROCESS_SOURCE_FILE(PSOURCEFILERECEIVEDNAME => 'INBOX/C2D/UC_DISSEM/UC_NMA_DISSEM/UC_NMA_DISSEM-277740.csv');
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vResult = ' || vResult);
|
||||||
|
* end;
|
||||||
|
* @ex_rslt 0
|
||||||
|
* -20021
|
||||||
|
**/
|
||||||
|
FUNCTION PROCESS_SOURCE_FILE(pSourceFileReceivedName IN VARCHAR2)
|
||||||
|
RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_DATE_FORMAT
|
||||||
|
* @desc Returns date format for specified template table name and column name.
|
||||||
|
* Date is taken from configuration A_COLUMN_DATE_FORMAT table.
|
||||||
|
* @example select FILE_MANAGER.GET_DATE_FORMAT(
|
||||||
|
* pTemplateTableName => 'STANDING_FACILITIES_HEADER',
|
||||||
|
* pColumnName => 'SNAPSHOT_DATE')
|
||||||
|
* from dual;
|
||||||
|
* @ex_rslt DD/MM/YYYY HH24:MI:SS
|
||||||
|
**/
|
||||||
|
FUNCTION GET_DATE_FORMAT(
|
||||||
|
pTemplateTableName IN VARCHAR2,
|
||||||
|
pColumnName IN VARCHAR2
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GENERATE_EXTERNAL_TABLE_PARAMS
|
||||||
|
* @desc It builds two strings: pColumnList and pFieldList for specified Template Table name, by parameter: pTemplateTableName.
|
||||||
|
* @example
|
||||||
|
* declare
|
||||||
|
* vColumnList CLOB;
|
||||||
|
* vFieldList CLOB;
|
||||||
|
* begin
|
||||||
|
* FILE_MANAGER.GENERATE_EXTERNAL_TABLE_PARAMS (
|
||||||
|
* pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES_HEADER'
|
||||||
|
* ,pColumnList => vColumnList
|
||||||
|
* ,pFieldList => vFieldList
|
||||||
|
* );
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vColumnList = '||vColumnList);
|
||||||
|
* DBMS_OUTPUT.PUT_LINE('vFieldList = '||vFieldList);
|
||||||
|
* end;
|
||||||
|
* /
|
||||||
|
**/
|
||||||
|
PROCEDURE GENERATE_EXTERNAL_TABLE_PARAMS (
|
||||||
|
pTemplateTableName IN VARCHAR2,
|
||||||
|
pColumnList OUT CLOB,
|
||||||
|
pFieldList OUT CLOB
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name ADD_SOURCE
|
||||||
|
* @desc Insert a new record to A_SOURCE table.
|
||||||
|
* pSourceKey is a PRIMARY KEY value.
|
||||||
|
**/
|
||||||
|
PROCEDURE ADD_SOURCE (
|
||||||
|
pSourceKey IN CT_MRDS.A_SOURCE.A_SOURCE_KEY%TYPE,
|
||||||
|
pSourceName IN CT_MRDS.A_SOURCE.SOURCE_NAME%TYPE
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name DELETE_SOURCE_CASCADE
|
||||||
|
* @desc Safely deletes a SOURCE specified by pSourceKey parameter from A_SOURCE table and all dependent tables:
|
||||||
|
* - A_SOURCE_FILE_CONFIG
|
||||||
|
* - A_SOURCE_FILE_RECEIVED
|
||||||
|
* - A_COLUMN_DATE_FORMAT (only if template table is not shared with other source systems)
|
||||||
|
* The procedure checks if template tables are shared before deleting date format configurations.
|
||||||
|
* If a template table is used by multiple source systems, date formats are preserved.
|
||||||
|
* @example CALL CT_MRDS.FILE_MANAGER.DELETE_SOURCE_CASCADE(pSourceKey => 'TEST_SYS');
|
||||||
|
**/
|
||||||
|
PROCEDURE DELETE_SOURCE_CASCADE (
|
||||||
|
pSourceKey IN CT_MRDS.A_SOURCE.A_SOURCE_KEY%TYPE
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_CONTAINER_SOURCE_FILE_CONFIG_KEY
|
||||||
|
* @desc For specified parameter pSourceFileId (A_SOURCE_FILE_CONFIG.SOURCE_FILE_ID)
|
||||||
|
* it returns A_SOURCE_FILE_CONFIG.A_SOURCE_FILE_CONFIG_KEY for related CONTAINER record.
|
||||||
|
* @example select FILE_MANAGER.GET_CONTAINER_SOURCE_FILE_CONFIG_KEY(
|
||||||
|
* pSourceFileId => 'UC_DISSEM')
|
||||||
|
* from dual;
|
||||||
|
* @ex_rslt 126
|
||||||
|
**/
|
||||||
|
FUNCTION GET_CONTAINER_SOURCE_FILE_CONFIG_KEY (
|
||||||
|
pSourceFileId IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_ID%TYPE
|
||||||
|
) RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_SOURCE_FILE_CONFIG_KEY
|
||||||
|
* @desc For specified input parameters,
|
||||||
|
* it returns A_SOURCE_FILE_CONFIG.A_SOURCE_FILE_CONFIG_KEY.
|
||||||
|
* @example select FILE_MANAGER.GET_SOURCE_FILE_CONFIG_KEY (
|
||||||
|
* pSourceFileType => 'INPUT'
|
||||||
|
* ,pSourceFileId => 'UC_DISSEM'
|
||||||
|
* ,pTableId => 'UC_NMA_DISSEM')
|
||||||
|
* from dual;
|
||||||
|
* @ex_rslt 126
|
||||||
|
**/
|
||||||
|
FUNCTION GET_SOURCE_FILE_CONFIG_KEY (
|
||||||
|
pSourceFileType IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_TYPE%TYPE DEFAULT 'INPUT'
|
||||||
|
,pSourceFileId IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_ID%TYPE
|
||||||
|
,pTableId IN CT_MRDS.A_SOURCE_FILE_CONFIG.TABLE_ID%TYPE
|
||||||
|
) RETURN PLS_INTEGER;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name ADD_SOURCE_FILE_CONFIG
|
||||||
|
* @desc Insert a new record to A_SOURCE_FILE_CONFIG table.
|
||||||
|
* MARS-1049: Added pEncoding parameter for CSV character set specification.
|
||||||
|
* @param pEncoding - Character set encoding for CSV files (e.g., 'UTF8', 'WE8MSWIN1252', 'EE8ISO8859P2')
|
||||||
|
* If NULL, no CHARACTERSET clause is added to external table definitions
|
||||||
|
* @example CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
* pSourceKey => 'C2D', pSourceFileType => 'INPUT',
|
||||||
|
* pSourceFileId => 'UC_DISSEM', pTableId => 'METADATA_LOADS',
|
||||||
|
* pTemplateTableName => 'CT_ET_TEMPLATES.C2D_A_UC_DISSEM_METADATA_LOADS',
|
||||||
|
* pEncoding => 'UTF8'
|
||||||
|
* );
|
||||||
|
**/
|
||||||
|
PROCEDURE ADD_SOURCE_FILE_CONFIG (
|
||||||
|
pSourceKey IN CT_MRDS.A_SOURCE_FILE_CONFIG.A_SOURCE_KEY%TYPE
|
||||||
|
,pSourceFileType IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_TYPE%TYPE
|
||||||
|
,pSourceFileId IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_ID%TYPE
|
||||||
|
,pSourceFileDesc IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_DESC%TYPE
|
||||||
|
,pSourceFileNamePattern IN CT_MRDS.A_SOURCE_FILE_CONFIG.SOURCE_FILE_NAME_PATTERN%TYPE
|
||||||
|
,pTableId IN CT_MRDS.A_SOURCE_FILE_CONFIG.TABLE_ID%TYPE DEFAULT NULL
|
||||||
|
,pTemplateTableName IN CT_MRDS.A_SOURCE_FILE_CONFIG.TEMPLATE_TABLE_NAME%TYPE DEFAULT NULL
|
||||||
|
,pContainerFileKey IN CT_MRDS.A_SOURCE_FILE_CONFIG.CONTAINER_FILE_KEY%TYPE DEFAULT NULL
|
||||||
|
,pEncoding IN CT_MRDS.A_SOURCE_FILE_CONFIG.ENCODING%TYPE DEFAULT NULL -- MARS-1049: NOWY PARAMETR
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name ADD_COLUMN_DATE_FORMAT
|
||||||
|
* @desc Insert a new record to A_COLUMN_DATE_FORMAT table.
|
||||||
|
**/
|
||||||
|
PROCEDURE ADD_COLUMN_DATE_FORMAT (
|
||||||
|
pTemplateTableName IN CT_MRDS.A_COLUMN_DATE_FORMAT.TEMPLATE_TABLE_NAME%TYPE
|
||||||
|
,pColumnName IN CT_MRDS.A_COLUMN_DATE_FORMAT.COLUMN_NAME%TYPE
|
||||||
|
,pDateFormat IN CT_MRDS.A_COLUMN_DATE_FORMAT.DATE_FORMAT%TYPE
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_BUCKET_URI
|
||||||
|
* @desc Function used to get string with bucket http url.
|
||||||
|
* Possible input values for pBucketArea are: 'INBOX', 'ODS', 'DATA', 'ARCHIVE'
|
||||||
|
* @example select FILE_MANAGER.GET_BUCKET_URI(pBucketArea => 'ODS') from dual;
|
||||||
|
* @ex_rslt https://objectstorage.eu-frankfurt-1.oraclecloud.com/n/frcnomajoc7v/b/mrds_data_tst/o/
|
||||||
|
**/
|
||||||
|
FUNCTION GET_BUCKET_URI(pBucketArea VARCHAR2)
|
||||||
|
RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_DET_SOURCE_FILE_CONFIG_INFO
|
||||||
|
* @desc Function returns details about A_SOURCE_FILE_CONFIG record
|
||||||
|
* for specified pSourceFileConfigKey (A_SOURCE_FILE_CONFIG.A_SOURCE_FILE_CONFIG_KEY).
|
||||||
|
* If pIncludeContainerInfo is <> 0 it returns additional info about related Container config record (A_SOURCE_FILE_CONFIG)
|
||||||
|
* If pIncludeColumnFormatInfo is <> 0 it returns additional info about related ColumnFormat config record (A_COLUMN_DATE_FORMAT)
|
||||||
|
* @example select FILE_MANAGER.GET_DET_SOURCE_FILE_CONFIG_INFO (
|
||||||
|
* pSourceFileConfigKey => 128
|
||||||
|
* ,pIncludeContainerInfo => 1
|
||||||
|
* ,pIncludeColumnFormatInfo => 1
|
||||||
|
* ) from dual;
|
||||||
|
* @ex_rslt
|
||||||
|
* Details about File Configuration:
|
||||||
|
* --------------------------------
|
||||||
|
* A_SOURCE_FILE_CONFIG_KEY = 128
|
||||||
|
* A_SOURCE_KEY = C2D
|
||||||
|
* ...
|
||||||
|
* --------------------------------
|
||||||
|
*
|
||||||
|
* Details about related Container Config:
|
||||||
|
* --------------------------------
|
||||||
|
* A_SOURCE_FILE_CONFIG_KEY = 126
|
||||||
|
* A_SOURCE_KEY = C2D
|
||||||
|
* ...
|
||||||
|
* --------------------------------
|
||||||
|
*
|
||||||
|
* Column Date Format config entries:
|
||||||
|
* --------------------------------
|
||||||
|
* TEMPLATE_TABLE_NAME = CT_ET_TEMPLATES.C2D_UC_MA_DISSEM
|
||||||
|
* ...
|
||||||
|
* --------------------------------
|
||||||
|
**/
|
||||||
|
FUNCTION GET_DET_SOURCE_FILE_CONFIG_INFO (
|
||||||
|
pSourceFileConfigKey IN CT_MRDS.A_SOURCE_FILE_CONFIG.A_SOURCE_FILE_CONFIG_KEY%TYPE
|
||||||
|
,pIncludeContainerInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
,pIncludeColumnFormatInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_DET_SOURCE_FILE_RECEIVED_INFO
|
||||||
|
* @desc Function returns details about A_SOURCE_FILE_RECEIVED record
|
||||||
|
* for specified pSourceFileReceivedKey (A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY).
|
||||||
|
* If pIncludeConfigInfo is <> 0 it returns additional info about related Container config record (A_SOURCE_FILE_CONFIG)
|
||||||
|
* If pIncludeContainerInfo is <> 0 it returns additional info about related Container config record (A_SOURCE_FILE_CONFIG)
|
||||||
|
* If pIncludeColumnFormatInfo is <> 0 it returns additional info about related ColumnFormat config record (A_COLUMN_DATE_FORMAT)
|
||||||
|
* @example select FILE_MANAGER.GET_DET_SOURCE_FILE_RECEIVED_INFO (
|
||||||
|
* pSourceFileReceivedKey => 377
|
||||||
|
* ,pIncludeConfigInfo => 1
|
||||||
|
* ,pIncludeContainerInfo => 1
|
||||||
|
* ,pIncludeColumnFormatInfo => 1
|
||||||
|
* ) from dual;
|
||||||
|
*
|
||||||
|
**/
|
||||||
|
FUNCTION GET_DET_SOURCE_FILE_RECEIVED_INFO (
|
||||||
|
pSourceFileReceivedKey IN CT_MRDS.A_SOURCE_FILE_RECEIVED.A_SOURCE_FILE_RECEIVED_KEY%TYPE
|
||||||
|
,pIncludeConfigInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
,pIncludeContainerInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
,pIncludeColumnFormatInfo IN PLS_INTEGER DEFAULT 1
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_DET_USER_LOAD_OPERATIONS
|
||||||
|
* @desc Function returns details from USER_LOAD_OPERATIONS table
|
||||||
|
* for specified pOperationId.
|
||||||
|
* @example select FILE_MANAGER.GET_DET_USER_LOAD_OPERATIONS (pOperationId => 3608) from dual;
|
||||||
|
* @ex_rslt
|
||||||
|
* Details about USER_LOAD_OPERATIONS where ID = 3608
|
||||||
|
* --------------------------------
|
||||||
|
* ID = 3608
|
||||||
|
* TYPE = VALIDATE
|
||||||
|
* SID = 31260
|
||||||
|
* SERIAL# = 52915
|
||||||
|
* START_TIME = 2025-05-20 10.08.24.436983 EUROPE/BELGRADE
|
||||||
|
* UPDATE_TIME = 2025-05-20 10.08.24.458643 EUROPE/BELGRADE
|
||||||
|
* STATUS = FAILED
|
||||||
|
* OWNER_NAME = CT_MRDS
|
||||||
|
* TABLE_NAME = STANDING_FACILITIES_HEADER
|
||||||
|
* PARTITION_NAME =
|
||||||
|
* SUBPARTITION_NAME =
|
||||||
|
* FILE_URI_LIST =
|
||||||
|
* ROWS_LOADED =
|
||||||
|
* LOGFILE_TABLE = VALIDATE$3608_LOG
|
||||||
|
* BADFILE_TABLE = VALIDATE$3608_BAD
|
||||||
|
* STATUS_TABLE =
|
||||||
|
* TEMPEXT_TABLE =
|
||||||
|
* CREDENTIAL_NAME =
|
||||||
|
* EXPIRATION_TIME = 2025-05-22 10.08.24.436983000 EUROPE/BELGRADE
|
||||||
|
* --------------------------------
|
||||||
|
**/
|
||||||
|
FUNCTION GET_DET_USER_LOAD_OPERATIONS (
|
||||||
|
pOperationId PLS_INTEGER
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name ANALYZE_VALIDATION_ERRORS
|
||||||
|
* @desc Wrapper function that analyzes validation errors for a source file using its received key.
|
||||||
|
* Automatically derives template schema, table name, CSV URI and validation log table
|
||||||
|
* from file metadata and calls ENV_MANAGER.ANALYZE_VALIDATION_ERRORS.
|
||||||
|
* @example SELECT FILE_MANAGER.ANALYZE_VALIDATION_ERRORS(63) FROM DUAL;
|
||||||
|
* @ex_rslt Detailed validation analysis report with column mismatches and solutions
|
||||||
|
**/
|
||||||
|
FUNCTION ANALYZE_VALIDATION_ERRORS(
|
||||||
|
pSourceFileReceivedKey IN NUMBER
|
||||||
|
) RETURN VARCHAR2;
|
||||||
|
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
-- PACKAGE VERSION MANAGEMENT FUNCTIONS
|
||||||
|
---------------------------------------------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_VERSION
|
||||||
|
* @desc Returns the current version number of the FILE_MANAGER package.
|
||||||
|
* Uses semantic versioning format (MAJOR.MINOR.PATCH).
|
||||||
|
* @example SELECT FILE_MANAGER.GET_VERSION() FROM DUAL;
|
||||||
|
* @ex_rslt 3.2.0
|
||||||
|
**/
|
||||||
|
FUNCTION GET_VERSION RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_BUILD_INFO
|
||||||
|
* @desc Returns comprehensive build information including version, build date, and author.
|
||||||
|
* Uses centralized ENV_MANAGER.GET_PACKAGE_VERSION_INFO function.
|
||||||
|
* @example SELECT FILE_MANAGER.GET_BUILD_INFO() FROM DUAL;
|
||||||
|
* @ex_rslt Package: FILE_MANAGER
|
||||||
|
* Version: 3.2.0
|
||||||
|
* Build Date: 2025-10-22 16:30:00
|
||||||
|
* Author: Grzegorz Michalski
|
||||||
|
**/
|
||||||
|
FUNCTION GET_BUILD_INFO RETURN VARCHAR2;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @name GET_VERSION_HISTORY
|
||||||
|
* @desc Returns complete version history with all releases and changes.
|
||||||
|
* Uses centralized ENV_MANAGER.FORMAT_VERSION_HISTORY function.
|
||||||
|
* @example SELECT FILE_MANAGER.GET_VERSION_HISTORY() FROM DUAL;
|
||||||
|
* @ex_rslt FILE_MANAGER Version History:
|
||||||
|
* 3.2.0 (2025-10-22): Added package versioning system...
|
||||||
|
**/
|
||||||
|
FUNCTION GET_VERSION_HISTORY RETURN VARCHAR2;
|
||||||
|
|
||||||
|
END;
|
||||||
|
|
||||||
|
/
|
||||||
|
|
||||||
|
/
|
||||||
93
MARS_Packages/REL01/MARS-1056/rollback_mars1056.sql
Normal file
93
MARS_Packages/REL01/MARS-1056/rollback_mars1056.sql
Normal file
@@ -0,0 +1,93 @@
|
|||||||
|
/*
|
||||||
|
================================================================================
|
||||||
|
MARS-1056: Master Rollback Script for VARCHAR2 Definition Fixes
|
||||||
|
Author: Grzegorz Michalski
|
||||||
|
Date: November 25, 2025
|
||||||
|
Purpose: Main rollback script for VARCHAR2 fixes in case of issues
|
||||||
|
|
||||||
|
USAGE:
|
||||||
|
sql 'CT_MRDS/Cloudpass#34@ggmichalski_high' '@rollback_mars1056.sql'
|
||||||
|
================================================================================
|
||||||
|
*/
|
||||||
|
|
||||||
|
-- Dynamic spool file generation (using SYS_CONTEXT - no DBA privileges required)
|
||||||
|
var filename VARCHAR2(100)
|
||||||
|
BEGIN
|
||||||
|
:filename := 'ROLLBACK_MARS_1056_' || SYS_CONTEXT('USERENV', 'CON_NAME') || '_' || TO_CHAR(SYSDATE,'YYYYMMDD_HH24MISS') || '.log';
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
column filename new_value _filename
|
||||||
|
select :filename filename from dual;
|
||||||
|
spool &_filename
|
||||||
|
|
||||||
|
SET ECHO OFF
|
||||||
|
SET TIMING ON
|
||||||
|
SET SERVEROUTPUT ON SIZE UNLIMITED
|
||||||
|
SET PAUSE OFF
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT ========================================================================
|
||||||
|
PROMPT MARS-1056: VARCHAR2 Definition Fixes ROLLBACK
|
||||||
|
PROMPT ========================================================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT WARNING: This will rollback all MARS-1056 changes!
|
||||||
|
PROMPT
|
||||||
|
PROMPT After rollback:
|
||||||
|
PROMPT - FILE_MANAGER will use original VARCHAR2 definition logic
|
||||||
|
PROMPT - External tables will again have data_length based definitions
|
||||||
|
PROMPT - CHAR/BYTE semantics will NOT be preserved from templates
|
||||||
|
PROMPT
|
||||||
|
PROMPT Continue only if you are experiencing issues with MARS-1056 changes
|
||||||
|
PROMPT
|
||||||
|
PROMPT ========================================================================
|
||||||
|
|
||||||
|
-- Confirm rollback with user
|
||||||
|
ACCEPT continue CHAR PROMPT 'Type YES to continue with rollback, or Ctrl+C to abort: '
|
||||||
|
WHENEVER SQLERROR EXIT SQL.SQLCODE
|
||||||
|
BEGIN
|
||||||
|
IF '&continue' IS NULL OR TRIM('&continue') IS NULL OR UPPER(TRIM('&continue')) != 'YES' THEN
|
||||||
|
RAISE_APPLICATION_ERROR(-20001, 'Rollback aborted by user');
|
||||||
|
END IF;
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
WHENEVER SQLERROR CONTINUE
|
||||||
|
|
||||||
|
-- Set current schema context
|
||||||
|
ALTER SESSION SET CURRENT_SCHEMA = CT_MRDS;
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT === Step 1: Rollback packages to pre-MARS-1056 versions ===
|
||||||
|
@@91_MARS_1056_rollback_packages.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT === Step 2: Track rollback version ===
|
||||||
|
@@track_package_versions.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT === Step 3: Verify all tracked packages ===
|
||||||
|
@@verify_packages_version.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT ========================================================================
|
||||||
|
PROMPT MARS-1056: Rollback completed successfully
|
||||||
|
PROMPT ========================================================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT All packages have been restored to pre-MARS-1056 state:
|
||||||
|
PROMPT - FILE_MANAGER: Original VARCHAR2 definition logic restored (v3.2.1)
|
||||||
|
PROMPT - FILE_MANAGER_ODS: Unchanged (v2.1.0 - wrapper only)
|
||||||
|
PROMPT - External tables will use data_length based definitions again
|
||||||
|
PROMPT
|
||||||
|
PROMPT Impact:
|
||||||
|
PROMPT - VARCHAR2 columns in external tables will NOT match template definitions
|
||||||
|
PROMPT - CHAR semantics will be lost (converted to BYTE with larger sizes)
|
||||||
|
PROMPT - This is the original problematic behavior that MARS-1056 was meant to fix
|
||||||
|
PROMPT
|
||||||
|
PROMPT Rollback tracking has been logged for audit purposes.
|
||||||
|
PROMPT
|
||||||
|
PROMPT ========================================================================
|
||||||
|
PROMPT Rollback log saved to: &_filename
|
||||||
|
PROMPT ========================================================================
|
||||||
|
|
||||||
|
spool off
|
||||||
|
|
||||||
|
quit;
|
||||||
95
MARS_Packages/REL01/MARS-1056/track_package_versions.sql
Normal file
95
MARS_Packages/REL01/MARS-1056/track_package_versions.sql
Normal file
@@ -0,0 +1,95 @@
|
|||||||
|
-- ===================================================================
|
||||||
|
-- Simple Package Version Tracking Script
|
||||||
|
-- ===================================================================
|
||||||
|
-- Purpose: Track specified Oracle package versions
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-26
|
||||||
|
-- Version: 3.1.0 - List-Based Edition
|
||||||
|
--
|
||||||
|
-- USAGE:
|
||||||
|
-- 1. Edit package list below (add/remove packages as needed)
|
||||||
|
-- 2. Include in your install/rollback script: @@track_package_versions.sql
|
||||||
|
-- ===================================================================
|
||||||
|
|
||||||
|
SET SERVEROUTPUT ON;
|
||||||
|
|
||||||
|
DECLARE
|
||||||
|
TYPE t_package_rec IS RECORD (
|
||||||
|
owner VARCHAR2(50),
|
||||||
|
package_name VARCHAR2(50),
|
||||||
|
version VARCHAR2(50)
|
||||||
|
);
|
||||||
|
TYPE t_packages IS TABLE OF t_package_rec;
|
||||||
|
TYPE t_string_array IS TABLE OF VARCHAR2(100);
|
||||||
|
|
||||||
|
-- ===================================================================
|
||||||
|
-- PACKAGE LIST - Edit this array to specify packages to track
|
||||||
|
-- ===================================================================
|
||||||
|
-- Add or remove entries as needed for your MARS issue
|
||||||
|
-- Format: 'SCHEMA.PACKAGE_NAME'
|
||||||
|
-- ===================================================================
|
||||||
|
vPackageList t_string_array := t_string_array(
|
||||||
|
'CT_MRDS.FILE_MANAGER'
|
||||||
|
);
|
||||||
|
-- ===================================================================
|
||||||
|
|
||||||
|
vPackages t_packages := t_packages();
|
||||||
|
vVersion VARCHAR2(50);
|
||||||
|
vCount NUMBER := 0;
|
||||||
|
vOwner VARCHAR2(50);
|
||||||
|
vPackageName VARCHAR2(50);
|
||||||
|
vDotPos NUMBER;
|
||||||
|
BEGIN
|
||||||
|
DBMS_OUTPUT.PUT_LINE('========================================');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Package Version Tracking');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('========================================');
|
||||||
|
|
||||||
|
-- Process each package in the list
|
||||||
|
FOR i IN 1..vPackageList.COUNT LOOP
|
||||||
|
vDotPos := INSTR(vPackageList(i), '.');
|
||||||
|
IF vDotPos > 0 THEN
|
||||||
|
vOwner := SUBSTR(vPackageList(i), 1, vDotPos - 1);
|
||||||
|
vPackageName := SUBSTR(vPackageList(i), vDotPos + 1);
|
||||||
|
|
||||||
|
-- Get package version
|
||||||
|
BEGIN
|
||||||
|
EXECUTE IMMEDIATE 'SELECT ' || vOwner || '.' || vPackageName || '.GET_VERSION() FROM DUAL' INTO vVersion;
|
||||||
|
vPackages.EXTEND;
|
||||||
|
vPackages(vPackages.COUNT).owner := vOwner;
|
||||||
|
vPackages(vPackages.COUNT).package_name := vPackageName;
|
||||||
|
vPackages(vPackages.COUNT).version := vVersion;
|
||||||
|
|
||||||
|
-- Track in ENV_MANAGER
|
||||||
|
BEGIN
|
||||||
|
CT_MRDS.ENV_MANAGER.TRACK_PACKAGE_VERSION(
|
||||||
|
pPackageOwner => vOwner,
|
||||||
|
pPackageName => vPackageName,
|
||||||
|
pPackageVersion => vVersion,
|
||||||
|
pPackageBuildDate => TO_CHAR(SYSDATE, 'YYYY-MM-DD HH24:MI:SS'),
|
||||||
|
pPackageAuthor => 'Grzegorz Michalski'
|
||||||
|
);
|
||||||
|
vCount := vCount + 1;
|
||||||
|
EXCEPTION
|
||||||
|
WHEN OTHERS THEN NULL; -- Continue even if tracking fails
|
||||||
|
END;
|
||||||
|
EXCEPTION
|
||||||
|
WHEN OTHERS THEN NULL; -- Skip packages that fail
|
||||||
|
END;
|
||||||
|
END IF;
|
||||||
|
END LOOP;
|
||||||
|
|
||||||
|
-- Display results
|
||||||
|
IF vPackages.COUNT > 0 THEN
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Packages tracked: ' || vCount || ' of ' || vPackages.COUNT);
|
||||||
|
DBMS_OUTPUT.PUT_LINE('');
|
||||||
|
|
||||||
|
FOR i IN 1..vPackages.COUNT LOOP
|
||||||
|
DBMS_OUTPUT.PUT_LINE(vPackages(i).owner || '.' || vPackages(i).package_name || ' = ' || vPackages(i).version);
|
||||||
|
END LOOP;
|
||||||
|
ELSE
|
||||||
|
DBMS_OUTPUT.PUT_LINE('No packages found in list');
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
DBMS_OUTPUT.PUT_LINE('========================================');
|
||||||
|
END;
|
||||||
|
/
|
||||||
62
MARS_Packages/REL01/MARS-1056/verify_packages_version.sql
Normal file
62
MARS_Packages/REL01/MARS-1056/verify_packages_version.sql
Normal file
@@ -0,0 +1,62 @@
|
|||||||
|
-- ===================================================================
|
||||||
|
-- Universal Package Version Verification Script
|
||||||
|
-- ===================================================================
|
||||||
|
-- Purpose: Verify all tracked Oracle packages for code changes
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-11-25
|
||||||
|
-- Version: 1.0.0
|
||||||
|
--
|
||||||
|
-- USAGE:
|
||||||
|
-- Include at the end of install/rollback scripts: @@verify_packages_version.sql
|
||||||
|
--
|
||||||
|
-- OUTPUT:
|
||||||
|
-- - List of all tracked packages with their current status
|
||||||
|
-- - OK: Package has not changed since last tracking
|
||||||
|
-- - WARNING: Package code changed without version update
|
||||||
|
-- ===================================================================
|
||||||
|
|
||||||
|
SET LINESIZE 200
|
||||||
|
SET PAGESIZE 1000
|
||||||
|
SET FEEDBACK OFF
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT ========================================
|
||||||
|
PROMPT Package Version Verification
|
||||||
|
PROMPT ========================================
|
||||||
|
PROMPT
|
||||||
|
|
||||||
|
COLUMN PACKAGE_OWNER FORMAT A15
|
||||||
|
COLUMN PACKAGE_NAME FORMAT A20
|
||||||
|
COLUMN VERSION FORMAT A10
|
||||||
|
COLUMN STATUS FORMAT A80
|
||||||
|
|
||||||
|
SELECT
|
||||||
|
PACKAGE_OWNER,
|
||||||
|
PACKAGE_NAME,
|
||||||
|
PACKAGE_VERSION AS VERSION,
|
||||||
|
CT_MRDS.ENV_MANAGER.CHECK_PACKAGE_CHANGES(PACKAGE_OWNER, PACKAGE_NAME) AS STATUS
|
||||||
|
FROM (
|
||||||
|
SELECT
|
||||||
|
PACKAGE_OWNER,
|
||||||
|
PACKAGE_NAME,
|
||||||
|
PACKAGE_VERSION,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY PACKAGE_OWNER, PACKAGE_NAME ORDER BY TRACKING_DATE DESC) AS RN
|
||||||
|
FROM CT_MRDS.A_PACKAGE_VERSION_TRACKING
|
||||||
|
)
|
||||||
|
WHERE RN = 1
|
||||||
|
ORDER BY PACKAGE_OWNER, PACKAGE_NAME;
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT ========================================
|
||||||
|
PROMPT Verification Complete
|
||||||
|
PROMPT ========================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT Legend:
|
||||||
|
PROMPT OK - Package has not changed since last tracking
|
||||||
|
PROMPT WARNING - Package code changed without version update
|
||||||
|
PROMPT
|
||||||
|
PROMPT For detailed hash information, use:
|
||||||
|
PROMPT SELECT ENV_MANAGER.GET_PACKAGE_HASH_INFO('OWNER', 'PACKAGE') FROM DUAL;
|
||||||
|
PROMPT ========================================
|
||||||
|
|
||||||
|
SET FEEDBACK ON
|
||||||
@@ -0,0 +1,91 @@
|
|||||||
|
--=============================================================================================================================
|
||||||
|
-- MARS-1096: Extend A_PROCESS_LOG.log_message and Enable Daily Interval Partitioning
|
||||||
|
--=============================================================================================================================
|
||||||
|
-- Purpose:
|
||||||
|
-- - Extend log_message column from VARCHAR2(4000) to VARCHAR2(20000)
|
||||||
|
-- - Convert table to daily interval partitioning on log_timestamp
|
||||||
|
-- - Preserve all existing data during conversion
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-12-03
|
||||||
|
-- Version: 1.0.0
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
SET SERVEROUTPUT ON
|
||||||
|
|
||||||
|
PROMPT ========================================================================
|
||||||
|
PROMPT MARS-1096: Extend column and enable partitioning
|
||||||
|
PROMPT ========================================================================
|
||||||
|
|
||||||
|
-- 1) Extend column size (safe operation, preserves data)
|
||||||
|
ALTER TABLE CT_MRDS.A_PROCESS_LOG MODIFY (log_message VARCHAR2(20000));
|
||||||
|
|
||||||
|
-- 2) Convert table to interval partitioning by day on log_timestamp
|
||||||
|
-- Strategy: Create new partitioned table, copy ALL data, rename tables
|
||||||
|
-- This preserves all existing records during conversion
|
||||||
|
|
||||||
|
DECLARE
|
||||||
|
v_has_partitions NUMBER := 0;
|
||||||
|
v_row_count NUMBER := 0;
|
||||||
|
BEGIN
|
||||||
|
-- Check if already partitioned
|
||||||
|
SELECT COUNT(*) INTO v_has_partitions
|
||||||
|
FROM user_tab_partitions
|
||||||
|
WHERE table_name = 'A_PROCESS_LOG';
|
||||||
|
|
||||||
|
IF v_has_partitions = 0 THEN
|
||||||
|
-- Count existing data
|
||||||
|
EXECUTE IMMEDIATE 'SELECT COUNT(*) FROM CT_MRDS.A_PROCESS_LOG' INTO v_row_count;
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Existing records to migrate: ' || v_row_count);
|
||||||
|
|
||||||
|
-- Create new partitioned table with different name
|
||||||
|
EXECUTE IMMEDIATE q'[CREATE TABLE CT_MRDS.A_PROCESS_LOG_NEW (
|
||||||
|
a_process_log_key NUMBER GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
|
||||||
|
guid VARCHAR2(32),
|
||||||
|
Username VARCHAR2(128),
|
||||||
|
Osuser VARCHAR2(128),
|
||||||
|
Machine VARCHAR2(64),
|
||||||
|
Module VARCHAR2(64),
|
||||||
|
process_name VARCHAR2(200),
|
||||||
|
procedure_name VARCHAR2(200),
|
||||||
|
procedure_parameters VARCHAR2(20000),
|
||||||
|
log_level VARCHAR2(10),
|
||||||
|
log_message VARCHAR2(20000),
|
||||||
|
log_timestamp TIMESTAMP DEFAULT SYSTIMESTAMP
|
||||||
|
)
|
||||||
|
PARTITION BY RANGE (log_timestamp)
|
||||||
|
INTERVAL (NUMTODSINTERVAL(1,'DAY'))
|
||||||
|
(
|
||||||
|
PARTITION p_init VALUES LESS THAN (TIMESTAMP '2025-01-01 00:00:00')
|
||||||
|
)]';
|
||||||
|
|
||||||
|
-- Copy ALL existing data (Oracle auto-creates partitions per timestamp)
|
||||||
|
EXECUTE IMMEDIATE 'INSERT /*+ APPEND */ INTO CT_MRDS.A_PROCESS_LOG_NEW (
|
||||||
|
guid, Username, Osuser, Machine, Module, process_name, procedure_name,
|
||||||
|
procedure_parameters, log_level, log_message, log_timestamp
|
||||||
|
) SELECT guid, Username, Osuser, Machine, Module, process_name, procedure_name,
|
||||||
|
procedure_parameters, log_level, log_message, log_timestamp
|
||||||
|
FROM CT_MRDS.A_PROCESS_LOG';
|
||||||
|
|
||||||
|
COMMIT;
|
||||||
|
|
||||||
|
-- Verify data copied successfully
|
||||||
|
EXECUTE IMMEDIATE 'SELECT COUNT(*) FROM CT_MRDS.A_PROCESS_LOG_NEW' INTO v_row_count;
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Records migrated to partitioned table: ' || v_row_count);
|
||||||
|
|
||||||
|
-- Swap tables: rename old to backup, new to original name
|
||||||
|
EXECUTE IMMEDIATE 'ALTER TABLE CT_MRDS.A_PROCESS_LOG RENAME TO A_PROCESS_LOG_OLD';
|
||||||
|
EXECUTE IMMEDIATE 'ALTER TABLE CT_MRDS.A_PROCESS_LOG_NEW RENAME TO A_PROCESS_LOG';
|
||||||
|
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Table conversion complete. Old table preserved as A_PROCESS_LOG_OLD');
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Review and manually DROP TABLE CT_MRDS.A_PROCESS_LOG_OLD PURGE when verified');
|
||||||
|
ELSE
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Table already partitioned. Skipping conversion.');
|
||||||
|
END IF;
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
PROMPT SUCCESS: A_PROCESS_LOG extended and partitioned
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
-- End of Script
|
||||||
|
--=============================================================================================================================
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
-- Verify CT_MRDS.A_PROCESS_LOG structure after MARS-1096
|
||||||
|
-- CRITICAL: Use ALL_* views when installing as ADMIN user
|
||||||
|
SET PAGESIZE 100
|
||||||
|
SET LINESIZE 200
|
||||||
|
COLUMN column_name FORMAT A20
|
||||||
|
COLUMN data_type FORMAT A20
|
||||||
|
|
||||||
|
PROMPT Column length check:
|
||||||
|
SELECT column_name, data_type, data_length
|
||||||
|
FROM all_tab_columns
|
||||||
|
WHERE owner = 'CT_MRDS'
|
||||||
|
AND table_name = 'A_PROCESS_LOG'
|
||||||
|
AND column_name = 'LOG_MESSAGE';
|
||||||
|
|
||||||
|
PROMPT Partitioning check:
|
||||||
|
SELECT partitioning_type, interval
|
||||||
|
FROM all_part_tables
|
||||||
|
WHERE owner = 'CT_MRDS'
|
||||||
|
AND table_name = 'A_PROCESS_LOG';
|
||||||
|
|
||||||
|
PROMPT Sample partitions:
|
||||||
|
SELECT partition_name, high_value, num_rows
|
||||||
|
FROM all_tab_partitions
|
||||||
|
WHERE table_owner = 'CT_MRDS'
|
||||||
|
AND table_name = 'A_PROCESS_LOG'
|
||||||
|
ORDER BY partition_position
|
||||||
|
FETCH FIRST 10 ROWS ONLY;
|
||||||
|
|
||||||
|
PROMPT Total record count:
|
||||||
|
SELECT COUNT(*) AS total_records FROM CT_MRDS.A_PROCESS_LOG;
|
||||||
|
|
||||||
|
PROMPT Check for backup table (should be dropped after verification):
|
||||||
|
SELECT table_name FROM all_tables
|
||||||
|
WHERE owner = 'CT_MRDS'
|
||||||
|
AND table_name LIKE 'A_PROCESS_LOG%';
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
-- Cleanup backup table after successful verification
|
||||||
|
-- This script is automatically called by install_mars1096.sql
|
||||||
|
-- Can also be run manually if needed
|
||||||
|
-- CRITICAL: Use ALL_TABLES when installing as ADMIN user
|
||||||
|
|
||||||
|
SET SERVEROUTPUT ON
|
||||||
|
|
||||||
|
DECLARE
|
||||||
|
v_table_exists NUMBER := 0;
|
||||||
|
BEGIN
|
||||||
|
SELECT COUNT(*) INTO v_table_exists
|
||||||
|
FROM all_tables
|
||||||
|
WHERE owner = 'CT_MRDS'
|
||||||
|
AND table_name = 'A_PROCESS_LOG_OLD';
|
||||||
|
|
||||||
|
IF v_table_exists > 0 THEN
|
||||||
|
EXECUTE IMMEDIATE 'DROP TABLE CT_MRDS.A_PROCESS_LOG_OLD PURGE';
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Backup table A_PROCESS_LOG_OLD dropped successfully.');
|
||||||
|
ELSE
|
||||||
|
DBMS_OUTPUT.PUT_LINE('No backup table found. Nothing to clean up.');
|
||||||
|
END IF;
|
||||||
|
END;
|
||||||
|
/
|
||||||
47
MARS_Packages/REL01/MARS-1096/README.md
Normal file
47
MARS_Packages/REL01/MARS-1096/README.md
Normal file
@@ -0,0 +1,47 @@
|
|||||||
|
# MARS-1096: Extend A_PROCESS_LOG and Enable Daily Interval Partitioning
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
- Extend `CT_MRDS.A_PROCESS_LOG.LOG_MESSAGE` to `VARCHAR2(20000)`
|
||||||
|
- Convert table to interval partitioning by day on `LOG_TIMESTAMP`
|
||||||
|
- **CRITICAL**: Preserves all existing data during conversion
|
||||||
|
|
||||||
|
## Strategy
|
||||||
|
- Creates new partitioned table `A_PROCESS_LOG_NEW`
|
||||||
|
- Copies ALL existing records (preserving data integrity)
|
||||||
|
- Renames old table to `A_PROCESS_LOG_OLD` (backup)
|
||||||
|
- Renames new table to `A_PROCESS_LOG`
|
||||||
|
- Manual cleanup of backup table after verification
|
||||||
|
|
||||||
|
## Files
|
||||||
|
- Baseline update: `mrds_elt-dev-database/database/CT_MRDS/SCHEMA/tables/A_PROCESS_LOG.sql`
|
||||||
|
- Migration: `MARS_Packages/migrations/004_extend_log_message_and_partition.sql`
|
||||||
|
- Rollback: `MARS_Packages/migrations/rollback/004_rollback_log_message_and_partition.sql`
|
||||||
|
- Install: `MARS_Packages/REL02/MARS-1096/install_mars1096.sql` (includes cleanup)
|
||||||
|
- Verify: `MARS_Packages/REL02/MARS-1096/04_MARS_1096_verify_partitioning.sql`
|
||||||
|
- Cleanup: `MARS_Packages/REL02/MARS-1096/05_MARS_1096_cleanup_backup.sql` (auto-executed)
|
||||||
|
|
||||||
|
## Install
|
||||||
|
```powershell
|
||||||
|
# Run with SQLcl - includes verification and automatic backup cleanup
|
||||||
|
sql "CT_MRDS/Cloudpass#34@ggmichalski_high" "@MARS_Packages/REL01/MARS-1096/install_mars1096.sql"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Rollback
|
||||||
|
```powershell
|
||||||
|
# Execute rollback with full logging and verification
|
||||||
|
sql "CT_MRDS/Cloudpass#34@ggmichalski_high" "@MARS_Packages/REL01/MARS-1096/rollback_mars1096.sql"
|
||||||
|
|
||||||
|
# Log file created: ROLLBACK_MARS_1096_<PDB>_<timestamp>.log
|
||||||
|
```
|
||||||
|
|
||||||
|
**Rollback Process:**
|
||||||
|
1. User confirmation required (type "YES")
|
||||||
|
2. Restore table to non-partitioned structure
|
||||||
|
3. Revert log_message to VARCHAR2(4000)
|
||||||
|
4. Preserve all existing data
|
||||||
|
5. Verify changes with automatic checks
|
||||||
|
|
||||||
|
## Performance Notes
|
||||||
|
- Daily partitions optimal for ~200k records/day
|
||||||
|
- Partition pruning improves query performance with date filters
|
||||||
|
- Local indexes can be added per partition if needed
|
||||||
79
MARS_Packages/REL01/MARS-1096/install_mars1096.sql
Normal file
79
MARS_Packages/REL01/MARS-1096/install_mars1096.sql
Normal file
@@ -0,0 +1,79 @@
|
|||||||
|
-- ===================================================================
|
||||||
|
-- MARS-1096 INSTALL SCRIPT: Extend A_PROCESS_LOG and Enable Partitioning
|
||||||
|
-- ===================================================================
|
||||||
|
-- Purpose:
|
||||||
|
-- - Extend log_message column from VARCHAR2(4000) to VARCHAR2(20000)
|
||||||
|
-- - Convert table to daily interval partitioning on log_timestamp
|
||||||
|
-- - Preserve all existing data during conversion
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-12-03
|
||||||
|
-- Version: 1.0.0
|
||||||
|
-- ===================================================================
|
||||||
|
|
||||||
|
-- Dynamic spool file generation (using SYS_CONTEXT - no DBA privileges required)
|
||||||
|
var filename VARCHAR2(100)
|
||||||
|
BEGIN
|
||||||
|
:filename := 'INSTALL_MARS_1096_' || SYS_CONTEXT('USERENV', 'CON_NAME') || '_' || TO_CHAR(SYSDATE,'YYYYMMDD_HH24MISS') || '.log';
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
column filename new_value _filename
|
||||||
|
select :filename filename from dual;
|
||||||
|
spool &_filename
|
||||||
|
|
||||||
|
SET ECHO OFF
|
||||||
|
SET TIMING ON
|
||||||
|
SET SERVEROUTPUT ON SIZE UNLIMITED
|
||||||
|
SET PAUSE OFF
|
||||||
|
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT MARS-1096: Extend A_PROCESS_LOG and Enable Daily Partitioning
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT
|
||||||
|
PROMPT This script will:
|
||||||
|
PROMPT - Extend log_message column from VARCHAR2(4000) to VARCHAR2(20000)
|
||||||
|
PROMPT - Convert A_PROCESS_LOG table to daily interval partitioning
|
||||||
|
PROMPT - Create backup table A_PROCESS_LOG_OLD during conversion
|
||||||
|
PROMPT - Migrate all existing records to partitioned structure
|
||||||
|
PROMPT - Automatically cleanup backup table after verification
|
||||||
|
PROMPT
|
||||||
|
PROMPT Expected Duration: 1-2 minutes (depends on data volume)
|
||||||
|
PROMPT =========================================================================
|
||||||
|
|
||||||
|
-- Confirm installation with user
|
||||||
|
ACCEPT continue CHAR PROMPT 'Type YES to continue with installation, or Ctrl+C to abort: '
|
||||||
|
WHENEVER SQLERROR EXIT SQL.SQLCODE
|
||||||
|
BEGIN
|
||||||
|
IF '&continue' IS NULL OR TRIM('&continue') IS NULL OR UPPER(TRIM('&continue')) != 'YES' THEN
|
||||||
|
RAISE_APPLICATION_ERROR(-20001, 'Installation aborted by user');
|
||||||
|
END IF;
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
WHENEVER SQLERROR CONTINUE
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Step 1: Extend Column and Enable Partitioning
|
||||||
|
PROMPT =========================================================================
|
||||||
|
@@02_MARS_1096_extend_and_partition.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Step 2: Verify Partitioning
|
||||||
|
PROMPT =========================================================================
|
||||||
|
@@04_MARS_1096_verify_partitioning.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Step 3: Cleanup Backup Table
|
||||||
|
PROMPT =========================================================================
|
||||||
|
@@05_MARS_1096_cleanup_backup.sql
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT MARS-1096 Installation - COMPLETED
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Check the log file for complete installation details.
|
||||||
|
PROMPT =========================================================================
|
||||||
|
|
||||||
|
spool off
|
||||||
|
quit;
|
||||||
151
MARS_Packages/REL01/MARS-1096/rollback_mars1096.sql
Normal file
151
MARS_Packages/REL01/MARS-1096/rollback_mars1096.sql
Normal file
@@ -0,0 +1,151 @@
|
|||||||
|
-- ===================================================================
|
||||||
|
-- MARS-1096 ROLLBACK SCRIPT: Revert A_PROCESS_LOG Changes
|
||||||
|
-- ===================================================================
|
||||||
|
-- Purpose: Rollback all changes from MARS-1096 installation
|
||||||
|
-- - Revert log_message column from VARCHAR2(20000) to VARCHAR2(4000)
|
||||||
|
-- - Remove daily interval partitioning (restore to non-partitioned table)
|
||||||
|
-- - Preserve all existing data during rollback
|
||||||
|
-- Author: Grzegorz Michalski
|
||||||
|
-- Date: 2025-12-03
|
||||||
|
-- Version: 1.0.0
|
||||||
|
-- ===================================================================
|
||||||
|
|
||||||
|
-- Dynamic spool file generation (using SYS_CONTEXT - no DBA privileges required)
|
||||||
|
var filename VARCHAR2(100)
|
||||||
|
BEGIN
|
||||||
|
:filename := 'ROLLBACK_MARS_1096_' || SYS_CONTEXT('USERENV', 'CON_NAME') || '_' || TO_CHAR(SYSDATE,'YYYYMMDD_HH24MISS') || '.log';
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
column filename new_value _filename
|
||||||
|
select :filename filename from dual;
|
||||||
|
spool &_filename
|
||||||
|
|
||||||
|
SET ECHO OFF
|
||||||
|
SET TIMING ON
|
||||||
|
SET SERVEROUTPUT ON SIZE UNLIMITED
|
||||||
|
SET PAUSE OFF
|
||||||
|
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT MARS-1096: Rollback A_PROCESS_LOG Changes
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT WARNING: This will reverse all changes from MARS-1096 installation!
|
||||||
|
PROMPT
|
||||||
|
PROMPT Changes to be reverted:
|
||||||
|
PROMPT - Revert log_message column: VARCHAR2(20000) -> VARCHAR2(4000)
|
||||||
|
PROMPT - Remove daily interval partitioning
|
||||||
|
PROMPT - Restore to non-partitioned table structure
|
||||||
|
PROMPT
|
||||||
|
PROMPT CRITICAL: All existing data will be preserved during rollback
|
||||||
|
PROMPT =========================================================================
|
||||||
|
|
||||||
|
-- Confirm rollback with user
|
||||||
|
ACCEPT continue CHAR PROMPT 'Type YES to continue with rollback, or Ctrl+C to abort: '
|
||||||
|
WHENEVER SQLERROR EXIT SQL.SQLCODE
|
||||||
|
BEGIN
|
||||||
|
IF '&continue' IS NULL OR TRIM('&continue') IS NULL OR UPPER(TRIM('&continue')) != 'YES' THEN
|
||||||
|
RAISE_APPLICATION_ERROR(-20001, 'Rollback aborted by user');
|
||||||
|
END IF;
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
WHENEVER SQLERROR CONTINUE
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Step 1: Rollback Table Structure
|
||||||
|
PROMPT =========================================================================
|
||||||
|
|
||||||
|
DECLARE
|
||||||
|
v_row_count NUMBER := 0;
|
||||||
|
v_table_exists NUMBER := 0;
|
||||||
|
BEGIN
|
||||||
|
-- Check if old backup table still exists
|
||||||
|
SELECT COUNT(*) INTO v_table_exists
|
||||||
|
FROM user_tables
|
||||||
|
WHERE table_name = 'A_PROCESS_LOG_OLD';
|
||||||
|
|
||||||
|
IF v_table_exists > 0 THEN
|
||||||
|
-- If backup exists, simply restore it
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Restoring from backup table A_PROCESS_LOG_OLD');
|
||||||
|
|
||||||
|
EXECUTE IMMEDIATE 'ALTER TABLE CT_MRDS.A_PROCESS_LOG RENAME TO A_PROCESS_LOG_PARTITIONED';
|
||||||
|
EXECUTE IMMEDIATE 'ALTER TABLE CT_MRDS.A_PROCESS_LOG_OLD RENAME TO A_PROCESS_LOG';
|
||||||
|
|
||||||
|
-- Revert column size
|
||||||
|
EXECUTE IMMEDIATE 'ALTER TABLE CT_MRDS.A_PROCESS_LOG MODIFY (log_message VARCHAR2(4000))';
|
||||||
|
|
||||||
|
EXECUTE IMMEDIATE 'DROP TABLE CT_MRDS.A_PROCESS_LOG_PARTITIONED PURGE';
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Rollback complete using backup table.');
|
||||||
|
ELSE
|
||||||
|
-- No backup, need to convert back manually
|
||||||
|
DBMS_OUTPUT.PUT_LINE('No backup found. Converting partitioned table to non-partitioned.');
|
||||||
|
|
||||||
|
EXECUTE IMMEDIATE 'SELECT COUNT(*) FROM CT_MRDS.A_PROCESS_LOG' INTO v_row_count;
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Records to migrate: ' || v_row_count);
|
||||||
|
|
||||||
|
-- Create non-partitioned table
|
||||||
|
EXECUTE IMMEDIATE q'[CREATE TABLE CT_MRDS.A_PROCESS_LOG_NP (
|
||||||
|
a_process_log_key NUMBER GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
|
||||||
|
guid VARCHAR2(32),
|
||||||
|
Username VARCHAR2(128),
|
||||||
|
Osuser VARCHAR2(128),
|
||||||
|
Machine VARCHAR2(64),
|
||||||
|
Module VARCHAR2(64),
|
||||||
|
process_name VARCHAR2(200),
|
||||||
|
procedure_name VARCHAR2(200),
|
||||||
|
procedure_parameters VARCHAR2(20000),
|
||||||
|
log_level VARCHAR2(10),
|
||||||
|
log_message VARCHAR2(4000),
|
||||||
|
log_timestamp TIMESTAMP DEFAULT SYSTIMESTAMP
|
||||||
|
)]';
|
||||||
|
|
||||||
|
-- Copy all data
|
||||||
|
EXECUTE IMMEDIATE 'INSERT /*+ APPEND */ INTO CT_MRDS.A_PROCESS_LOG_NP (
|
||||||
|
guid, Username, Osuser, Machine, Module, process_name, procedure_name,
|
||||||
|
procedure_parameters, log_level, log_message, log_timestamp
|
||||||
|
) SELECT guid, Username, Osuser, Machine, Module, process_name, procedure_name,
|
||||||
|
procedure_parameters, log_level, log_message, log_timestamp
|
||||||
|
FROM CT_MRDS.A_PROCESS_LOG';
|
||||||
|
|
||||||
|
COMMIT;
|
||||||
|
|
||||||
|
-- Swap tables
|
||||||
|
EXECUTE IMMEDIATE 'ALTER TABLE CT_MRDS.A_PROCESS_LOG RENAME TO A_PROCESS_LOG_PARTITIONED';
|
||||||
|
EXECUTE IMMEDIATE 'ALTER TABLE CT_MRDS.A_PROCESS_LOG_NP RENAME TO A_PROCESS_LOG';
|
||||||
|
EXECUTE IMMEDIATE 'DROP TABLE CT_MRDS.A_PROCESS_LOG_PARTITIONED PURGE';
|
||||||
|
|
||||||
|
DBMS_OUTPUT.PUT_LINE('Rollback complete.');
|
||||||
|
END IF;
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Step 2: Verify Rollback
|
||||||
|
PROMPT =========================================================================
|
||||||
|
|
||||||
|
-- Verify column size reverted
|
||||||
|
SELECT column_name, data_type, data_length
|
||||||
|
FROM user_tab_columns
|
||||||
|
WHERE table_name = 'A_PROCESS_LOG' AND column_name = 'LOG_MESSAGE';
|
||||||
|
|
||||||
|
-- Verify partitioning removed
|
||||||
|
SELECT COUNT(*) AS partition_count
|
||||||
|
FROM user_tab_partitions
|
||||||
|
WHERE table_name = 'A_PROCESS_LOG';
|
||||||
|
|
||||||
|
-- Verify record count preserved
|
||||||
|
SELECT COUNT(*) AS total_records FROM CT_MRDS.A_PROCESS_LOG;
|
||||||
|
|
||||||
|
PROMPT
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT MARS-1096 Rollback - COMPLETED
|
||||||
|
PROMPT =========================================================================
|
||||||
|
PROMPT Check the log file for complete rollback details.
|
||||||
|
PROMPT Expected results:
|
||||||
|
PROMPT - LOG_MESSAGE column: VARCHAR2(4000)
|
||||||
|
PROMPT - Partition count: 0 (non-partitioned)
|
||||||
|
PROMPT - Total records: Should match pre-rollback count
|
||||||
|
PROMPT =========================================================================
|
||||||
|
|
||||||
|
spool off
|
||||||
|
quit;
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
|
||||||
|
SET SERVEROUTPUT ON;
|
||||||
|
DECLARE
|
||||||
|
v_ins_str VARCHAR2(4096);
|
||||||
|
v_upd_str VARCHAR2(4096);
|
||||||
|
v_trigger_sql VARCHAR2(32767);
|
||||||
|
v_trigger_name VARCHAR2(128);
|
||||||
|
v_ref_table_name VARCHAR2(128);
|
||||||
|
v_table_name VARCHAR2(128); -- GIVE VALUE FOR ONE TABLE TRIGGER GENERATION
|
||||||
|
BEGIN
|
||||||
|
FOR i IN (
|
||||||
|
SELECT
|
||||||
|
'create or replace TRIGGER DW_REF_MAIN.AUTOKEY_'||TABLE_NAME||'
|
||||||
|
BEFORE INSERT ON DW_REF_MAIN.'||TABLE_NAME||'
|
||||||
|
FOR EACH ROW
|
||||||
|
WHEN (NEW.A_KEY IS NULL)
|
||||||
|
BEGIN
|
||||||
|
SELECT
|
||||||
|
NVL(MAX(A_KEY), 0) + 1
|
||||||
|
INTO :NEW.A_KEY
|
||||||
|
FROM DW_REF_MAIN.'||TABLE_NAME||';
|
||||||
|
END;
|
||||||
|
|
||||||
|
' AS trigger_sql,
|
||||||
|
'DW_REF_MAIN.AUTOKEY_'||TABLE_NAME AS trigger_name
|
||||||
|
FROM DBA_TABLES t
|
||||||
|
WHERE
|
||||||
|
OWNER = 'DW_REF_MAIN'
|
||||||
|
AND TABLE_NAME IN (
|
||||||
|
'ASSET_CODE_TYPE', 'ABS_TYPE', 'ALLOTMENT_METHOD', 'AREA_CODE_RIAD', 'ASSET_GROUP_DETAIL', 'ASSET_GROUP_MAPPING', 'ASSET_GROUP_TMS', 'ASSET_GROUP_TMS_OLD_NEW_MAP', 'ASSET_TYPE', 'ASSET_TYPE_TMS',
|
||||||
|
'CAPITAL_KEY', 'COUNTRY', 'CSPP_BENCHMARK', 'CSPP_GROUP_MEMBERS', 'CSPP_GROUPS', 'CURRENCY', 'ELA_COUNTRY_CEILING', 'ELA_FUNDING_GAP', 'ELA_MFI_CEILING', 'ESA_95_INSTR_CLASS_CSDB',
|
||||||
|
'ESA_95_SECTOR', 'ESA_2010_INSTR_CLASS_CSDB', 'ESA_2010_ISSUER_CLASS_CSDB', 'ESAST_SUB_SECTOR_RIAD', 'HAIRCUT_CATEGORIES', 'HAIRCUT_SCHEDULE', 'HOLIDAY', 'INSTITUTION_C2D', 'INSTITUTION_CODE_TYPE', 'INSTITUTION_ECONOMIC_ACTIVITY',
|
||||||
|
'INSTITUTION_IDENTIFIER_TYPE', 'INSTITUTION_RAT_OUTLOOK_VALUE', 'INSTITUTION_RAT_WATCH_VALUE', 'INSTITUTION_ROLE', 'INSTITUTION_SUBROLE', 'ISSUER_DEBTOR_GUARANTOR_GROUP', 'LIMIT_BREACH_CAUSE', 'LIMIT_TYPE', 'LIMIT_VIOLATION_TYPE', 'MAP_COUPON_FREQ',
|
||||||
|
'MAP_COUPON_FREQ_RAR', 'MAP_COUPON_TYPE', 'MAP_COUPON_TYPE_RAR', 'NACE2_INDUSTRY_CLASS_CSDB', 'OPERATION_NAME', 'OPERATION_PROCEDURE', 'OPERATION_PUBLIC', 'OPERATION_TYPE', 'OWN_USE', 'PRICE_TYPE_MAP',
|
||||||
|
'PRICE_TYPE_RAR', 'PSE_TYPE', 'QUOT_BASIS_MAP', 'RATING', 'RATING_ACTION', 'RATING_PD', 'RATING_PRIORITY', 'RATING_SOURCE', 'RATING_TYPE', 'REDEMPTION_FREQUENCY_CSDB',
|
||||||
|
'REDEMPTION_TYPE_CSDB', 'RIC_CODE_CREATION_RULE', 'TMS_ACTIVITY_GROUPING', 'TMS_POSITION_CONFIG', 'MARKET_COUNTRIES', 'MEASUREMENT_UNITS', 'NCB', 'PORTFOLIO', 'PP_BUCKET', 'PP_CASH_FLOW_TYPE',
|
||||||
|
'PP_METHOD', 'PP_PORTF_BNCHMK_MAP', 'SCENARIOS', 'INSTITUTION_ROLE_C2D'
|
||||||
|
)
|
||||||
|
ORDER BY TABLE_NAME
|
||||||
|
) LOOP
|
||||||
|
v_trigger_sql := i.trigger_sql;
|
||||||
|
|
||||||
|
DBMS_OUTPUT.PUT_LINE(i.trigger_name);
|
||||||
|
EXECUTE IMMEDIATE v_trigger_sql;
|
||||||
|
|
||||||
|
-- DBMS_OUTPUT.PUT_LINE(v_trigger_sql||'
|
||||||
|
-- /
|
||||||
|
-- ');
|
||||||
|
|
||||||
|
END LOOP;
|
||||||
|
|
||||||
|
END;
|
||||||
|
/
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
|
||||||
|
SET SERVEROUTPUT ON;
|
||||||
|
DECLARE
|
||||||
|
v_ins_str VARCHAR2(4096);
|
||||||
|
v_upd_str VARCHAR2(4096);
|
||||||
|
v_trigger_sql VARCHAR2(32767);
|
||||||
|
v_trigger_name VARCHAR2(128);
|
||||||
|
v_ref_table_name VARCHAR2(128);
|
||||||
|
v_table_name VARCHAR2(128); -- GIVE VALUE FOR ONE TABLE TRIGGER GENERATION
|
||||||
|
BEGIN
|
||||||
|
FOR i IN (
|
||||||
|
SELECT
|
||||||
|
'create or replace TRIGGER REFERENCE.AUTOKEY_'||TABLE_NAME||'
|
||||||
|
BEFORE INSERT ON REFERENCE.'||TABLE_NAME||'
|
||||||
|
FOR EACH ROW
|
||||||
|
WHEN (NEW.A_KEY IS NULL)
|
||||||
|
BEGIN
|
||||||
|
SELECT
|
||||||
|
NVL(MAX(A_KEY), 0) + 1
|
||||||
|
INTO :NEW.A_KEY
|
||||||
|
FROM REFERENCE.'||TABLE_NAME||';
|
||||||
|
END;
|
||||||
|
|
||||||
|
' AS trigger_sql,
|
||||||
|
'REFERENCE.AUTOKEY_'||TABLE_NAME AS trigger_name
|
||||||
|
FROM DBA_TABLES t
|
||||||
|
WHERE
|
||||||
|
OWNER = 'REFERENCE'
|
||||||
|
AND TABLE_NAME IN (
|
||||||
|
'T_BALANCE_SHEET_STRUCTURE', 'T_CAPITAL_KEY', 'T_COUNTRY', 'T_CSDB_RATING_ACTION', 'T_CURRENCY', 'T_TMS_ACTIVITY_GROUPING'
|
||||||
|
)
|
||||||
|
ORDER BY TABLE_NAME
|
||||||
|
) LOOP
|
||||||
|
v_trigger_sql := i.trigger_sql;
|
||||||
|
|
||||||
|
DBMS_OUTPUT.PUT_LINE(i.trigger_name);
|
||||||
|
EXECUTE IMMEDIATE v_trigger_sql;
|
||||||
|
|
||||||
|
-- DBMS_OUTPUT.PUT_LINE(v_trigger_sql||'
|
||||||
|
-- /
|
||||||
|
-- ');
|
||||||
|
|
||||||
|
END LOOP;
|
||||||
|
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
@@ -0,0 +1,97 @@
|
|||||||
|
ALTER TABLE OU_REF.QUOT_BASIS_MAP ADD CONSTRAINT CHECK_ANALYZED CHECK (UPPER(ANNUALIZED) IN ( 'U', 'N', 'Y'));
|
||||||
|
|
||||||
|
GRANT SELECT ANY TABLE ON SCHEMA DW_REF_MAIN TO OU_REF;
|
||||||
|
GRANT INSERT ANY TABLE ON SCHEMA DW_REF_MAIN TO OU_REF;
|
||||||
|
GRANT UPDATE ANY TABLE ON SCHEMA DW_REF_MAIN TO OU_REF;
|
||||||
|
GRANT DELETE ANY TABLE ON SCHEMA DW_REF_MAIN TO OU_REF;
|
||||||
|
|
||||||
|
GRANT SELECT ANY TABLE ON SCHEMA REFERENCE TO OU_REF;
|
||||||
|
GRANT INSERT ANY TABLE ON SCHEMA REFERENCE TO OU_REF;
|
||||||
|
GRANT UPDATE ANY TABLE ON SCHEMA REFERENCE TO OU_REF;
|
||||||
|
GRANT DELETE ANY TABLE ON SCHEMA REFERENCE TO OU_REF;
|
||||||
|
|
||||||
|
SET SERVEROUTPUT ON;
|
||||||
|
DECLARE
|
||||||
|
v_ins_str VARCHAR2(4096);
|
||||||
|
v_upd_str VARCHAR2(4096);
|
||||||
|
v_trigger_sql VARCHAR2(32767);
|
||||||
|
v_trigger_name VARCHAR2(128);
|
||||||
|
v_ref_table_name VARCHAR2(128);
|
||||||
|
v_table_name VARCHAR2(128); -- GIVE VALUE FOR ONE TABLE TRIGGER GENERATION
|
||||||
|
BEGIN
|
||||||
|
FOR i IN (
|
||||||
|
SELECT
|
||||||
|
OWNER, TABLE_NAME
|
||||||
|
FROM DBA_TABLES t
|
||||||
|
WHERE
|
||||||
|
OWNER = 'OU_REF'
|
||||||
|
AND (v_table_name IS NULL OR TABLE_NAME = v_table_name)
|
||||||
|
AND TABLE_NAME IN (
|
||||||
|
'ASSET_CODE_TYPE', 'ABS_TYPE', 'ALLOTMENT_METHOD', 'AREA_CODE_RIAD', 'ASSET_GROUP_DETAIL', 'ASSET_GROUP_MAPPING', 'ASSET_GROUP_TMS', 'ASSET_GROUP_TMS_OLD_NEW_MAP', 'ASSET_TYPE', 'ASSET_TYPE_TMS',
|
||||||
|
'CSPP_BENCHMARK', 'CSPP_GROUP_MEMBERS', 'CSPP_GROUPS', 'ELA_COUNTRY_CEILING', 'ELA_FUNDING_GAP', 'ELA_MFI_CEILING', 'ESA_95_INSTR_CLASS_CSDB', 'ESA_95_SECTOR', 'ESA_2010_INSTR_CLASS_CSDB', 'ESA_2010_ISSUER_CLASS_CSDB',
|
||||||
|
'ESAST_SUB_SECTOR_RIAD', 'HAIRCUT_CATEGORIES', 'HAIRCUT_SCHEDULE', 'HOLIDAY', 'INSTITUTION_C2D', 'INSTITUTION_ECONOMIC_ACTIVITY', 'INSTITUTION_IDENTIFIER_TYPE', 'INSTITUTION_RAT_OUTLOOK_VALUE', 'INSTITUTION_RAT_WATCH_VALUE', 'INSTITUTION_SUBROLE',
|
||||||
|
'ISSUER_DEBTOR_GUARANTOR_GROUP', 'LIMIT_BREACH_CAUSE', 'LIMIT_TYPE', 'LIMIT_VIOLATION_TYPE', 'MAP_COUPON_FREQ', 'MAP_COUPON_FREQ_RAR', 'MAP_COUPON_TYPE', 'MAP_COUPON_TYPE_RAR', 'NACE2_INDUSTRY_CLASS_CSDB', 'OPERATION_NAME',
|
||||||
|
'OPERATION_PROCEDURE', 'OPERATION_PUBLIC', 'OPERATION_TYPE', 'OWN_USE', 'PRICE_TYPE_MAP', 'PRICE_TYPE_RAR', 'PSE_TYPE', 'RATING_ACTION', 'RATING_PD', 'RATING_PRIORITY',
|
||||||
|
'RATING_SOURCE', 'RATING_TYPE', 'REDEMPTION_FREQUENCY_CSDB', 'REDEMPTION_TYPE_CSDB', 'TMS_POSITION_CONFIG', 'MARKET_COUNTRIES', 'MEASUREMENT_UNITS', 'NCB', 'PORTFOLIO', 'PP_BUCKET',
|
||||||
|
'PP_CASH_FLOW_TYPE', 'PP_METHOD', 'PP_PORTF_BNCHMK_MAP', 'SCENARIOS', 'INSTITUTION_CODE_TYPE', 'QUOT_BASIS_MAP', 'RATING', 'RIC_CODE_CREATION_RULE'
|
||||||
|
)
|
||||||
|
ORDER BY TABLE_NAME
|
||||||
|
) LOOP
|
||||||
|
SELECT
|
||||||
|
'INSERT INTO '||t.owner||'.'||t.table_name||'('||
|
||||||
|
LISTAGG(t.column_name, ', ') WITHIN GROUP (ORDER BY t.column_id) ||')
|
||||||
|
VALUES (' ||LISTAGG(DECODE(t.column_name, 'A_VALID_FROM', 'v_current_date', 'A_VALID_TO', 'v_future_date', DECODE(r.column_name, NULL, 'NULL', ':new.'||t.column_name)), ', ') WITHIN GROUP (ORDER BY t.column_id) ||')' AS v_ins_str,
|
||||||
|
'UPDATE '||t.owner||'.'||t.table_name||' SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
'||LISTAGG(DECODE(r.column_name, NULL, '', ' AND DECODE('||t.column_name|| ', :old.'||t.column_name||', 1, 0) = 1'||chr(13))) WITHIN GROUP (ORDER BY t.column_id) || ' AND A_VALID_TO = v_future_date'||chr(13) AS v_upd_str
|
||||||
|
INTO v_ins_str, v_upd_str
|
||||||
|
FROM dba_tab_columns t, dba_tab_columns r
|
||||||
|
WHERE
|
||||||
|
t.owner = 'DW_REF_MAIN'
|
||||||
|
AND t.table_name = i.TABLE_NAME
|
||||||
|
AND 'OU_REF' = r.owner (+)
|
||||||
|
AND t.table_name = r.table_name (+)
|
||||||
|
AND t.column_name = r.column_name (+)
|
||||||
|
GROUP BY t.owner, t.table_name
|
||||||
|
;
|
||||||
|
|
||||||
|
v_ref_table_name := i.OWNER||'.'||i.TABLE_NAME;
|
||||||
|
v_trigger_name := i.OWNER||'.'||i.TABLE_NAME||'_H';
|
||||||
|
|
||||||
|
v_trigger_sql := '
|
||||||
|
create or replace TRIGGER '||v_trigger_name||'
|
||||||
|
BEFORE INSERT OR UPDATE OR DELETE ON '||v_ref_table_name||'
|
||||||
|
REFERENCING NEW AS new OLD AS old
|
||||||
|
FOR EACH ROW
|
||||||
|
DECLARE
|
||||||
|
v_current_date DATE := SYSDATE;
|
||||||
|
v_future_date DATE := TO_DATE(''31/12/9999'', ''DD/MM/YYYY'');
|
||||||
|
BEGIN
|
||||||
|
IF INSERTING THEN
|
||||||
|
'||v_ins_str||';
|
||||||
|
ELSIF DELETING THEN
|
||||||
|
'||v_upd_str||' ;
|
||||||
|
ELSIF UPDATING THEN
|
||||||
|
'||v_upd_str||' ;
|
||||||
|
'||v_ins_str||';
|
||||||
|
END IF;
|
||||||
|
END;';
|
||||||
|
|
||||||
|
DBMS_OUTPUT.PUT_LINE(v_trigger_name);
|
||||||
|
EXECUTE IMMEDIATE v_trigger_sql;
|
||||||
|
-- DBMS_OUTPUT.PUT_LINE(v_trigger_sql||'
|
||||||
|
--/
|
||||||
|
-- ');
|
||||||
|
|
||||||
|
END LOOP;
|
||||||
|
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,44 @@
|
|||||||
|
create or replace TRIGGER OU_REF.BALANCE_SHEET_STRUCTURE_H
|
||||||
|
BEFORE INSERT OR UPDATE OR DELETE ON OU_REF.BALANCE_SHEET_STRUCTURE
|
||||||
|
REFERENCING NEW AS new OLD AS old
|
||||||
|
FOR EACH ROW
|
||||||
|
DECLARE
|
||||||
|
v_current_date DATE := SYSDATE;
|
||||||
|
v_future_date DATE := TO_DATE('31/12/9999', 'DD/MM/YYYY');
|
||||||
|
BEGIN
|
||||||
|
IF INSERTING THEN
|
||||||
|
INSERT INTO REFERENCE.T_BALANCE_SHEET_STRUCTURE(ISIS_CODE, BS_ITEM_CODE, DESCRIPTION, BS_SUB_ITEM, BS_ITEM, BS_ITEM_GROUP, BS_ITEM_TYPE, A_VALID_FROM, A_VALID_TO, A_KEY, A_MOPDB_LOAD_SET_FK)
|
||||||
|
VALUES (:new.ISIS_CODE, :new.BS_ITEM_CODE, :new.DESCRIPTION, :new.BS_SUB_ITEM, :new.BS_ITEM, :new.BS_ITEM_GROUP, :new.BS_ITEM_TYPE, v_current_date, v_future_date, NULL, -1);
|
||||||
|
ELSIF DELETING THEN
|
||||||
|
UPDATE REFERENCE.T_BALANCE_SHEET_STRUCTURE SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(ISIS_CODE, :old.ISIS_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(BS_ITEM_CODE, :old.BS_ITEM_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(DESCRIPTION, :old.DESCRIPTION, 1, 0) = 1
|
||||||
|
AND DECODE(BS_SUB_ITEM, :old.BS_SUB_ITEM, 1, 0) = 1
|
||||||
|
AND DECODE(BS_ITEM, :old.BS_ITEM, 1, 0) = 1
|
||||||
|
AND DECODE(BS_ITEM_GROUP, :old.BS_ITEM_GROUP, 1, 0) = 1
|
||||||
|
AND DECODE(BS_ITEM_TYPE, :old.BS_ITEM_TYPE, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
ELSIF UPDATING THEN
|
||||||
|
UPDATE REFERENCE.T_BALANCE_SHEET_STRUCTURE SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(ISIS_CODE, :old.ISIS_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(BS_ITEM_CODE, :old.BS_ITEM_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(DESCRIPTION, :old.DESCRIPTION, 1, 0) = 1
|
||||||
|
AND DECODE(BS_SUB_ITEM, :old.BS_SUB_ITEM, 1, 0) = 1
|
||||||
|
AND DECODE(BS_ITEM, :old.BS_ITEM, 1, 0) = 1
|
||||||
|
AND DECODE(BS_ITEM_GROUP, :old.BS_ITEM_GROUP, 1, 0) = 1
|
||||||
|
AND DECODE(BS_ITEM_TYPE, :old.BS_ITEM_TYPE, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
INSERT INTO REFERENCE.T_BALANCE_SHEET_STRUCTURE(ISIS_CODE, BS_ITEM_CODE, DESCRIPTION, BS_SUB_ITEM, BS_ITEM, BS_ITEM_GROUP, BS_ITEM_TYPE, A_VALID_FROM, A_VALID_TO, A_KEY, A_MOPDB_LOAD_SET_FK)
|
||||||
|
VALUES (:new.ISIS_CODE, :new.BS_ITEM_CODE, :new.DESCRIPTION, :new.BS_SUB_ITEM, :new.BS_ITEM, :new.BS_ITEM_GROUP, :new.BS_ITEM_TYPE, v_current_date, v_future_date, NULL, -1);
|
||||||
|
END IF;
|
||||||
|
END;
|
||||||
|
/
|
||||||
@@ -0,0 +1,72 @@
|
|||||||
|
create or replace TRIGGER OU_REF.CAPITAL_KEY_H
|
||||||
|
BEFORE INSERT OR UPDATE OR DELETE ON OU_REF.CAPITAL_KEY
|
||||||
|
REFERENCING NEW AS new OLD AS old
|
||||||
|
FOR EACH ROW
|
||||||
|
DECLARE
|
||||||
|
v_current_date DATE := SYSDATE;
|
||||||
|
v_future_date DATE := TO_DATE('31/12/9999', 'DD/MM/YYYY');
|
||||||
|
BEGIN
|
||||||
|
IF INSERTING THEN
|
||||||
|
INSERT INTO DW_REF_MAIN.CAPITAL_KEY(COUNTRY_CODE, NCB_INDEX, CAPITAL_KEY_ADJ, RAR_NCB_NAME, CAPITAL_KEY_VALUE, PAID_UP_CAPITAL, A_VALID_FROM, A_VALID_TO, A_KEY)
|
||||||
|
VALUES (:new.COUNTRY_CODE, :new.NCB_INDEX, NULL, :new.RAR_NCB_NAME, :new.CAPITAL_KEY_VALUE, :new.PAID_UP_CAPITAL, v_current_date, v_future_date, NULL);
|
||||||
|
INSERT INTO REFERENCE.T_CAPITAL_KEY(COUNTRY_CODE, NCB_NAME, EUROSYSTEM_KEY, EUROSYSTEM_KEY_ECB, ESCB_KEY, ESCB_KEY_ECB, CAPITAL_KEY_VALUE, PAID_UP_CAPITAL, A_VALID_FROM, A_VALID_TO, A_KEY, A_MOPDB_LOAD_SET_FK)
|
||||||
|
VALUES (:new.COUNTRY_CODE, :new.RAR_NCB_NAME, :new.EUROSYSTEM_KEY, :new.EUROSYSTEM_KEY_ECB, :new.ESCB_KEY, :new.ESCB_KEY_ECB, :new.CAPITAL_KEY_VALUE, :new.PAID_UP_CAPITAL, v_current_date, v_future_date, NULL,-1);
|
||||||
|
ELSIF DELETING THEN
|
||||||
|
UPDATE DW_REF_MAIN.CAPITAL_KEY SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(COUNTRY_CODE, :old.COUNTRY_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(NCB_INDEX, :old.NCB_INDEX, 1, 0) = 1
|
||||||
|
AND DECODE(RAR_NCB_NAME, :old.RAR_NCB_NAME, 1, 0) = 1
|
||||||
|
AND DECODE(CAPITAL_KEY_VALUE, :old.CAPITAL_KEY_VALUE, 1, 0) = 1
|
||||||
|
AND DECODE(PAID_UP_CAPITAL, :old.PAID_UP_CAPITAL, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
UPDATE REFERENCE.T_CAPITAL_KEY SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(COUNTRY_CODE, :old.COUNTRY_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(NCB_NAME, :old.RAR_NCB_NAME, 1, 0) = 1
|
||||||
|
AND DECODE(EUROSYSTEM_KEY, :old.EUROSYSTEM_KEY, 1, 0) = 1
|
||||||
|
AND DECODE(EUROSYSTEM_KEY_ECB, :old.EUROSYSTEM_KEY_ECB, 1, 0) = 1
|
||||||
|
AND DECODE(ESCB_KEY, :old.ESCB_KEY, 1, 0) = 1
|
||||||
|
AND DECODE(ESCB_KEY_ECB, :old.ESCB_KEY_ECB, 1, 0) = 1
|
||||||
|
AND DECODE(CAPITAL_KEY_VALUE, :old.CAPITAL_KEY_VALUE, 1, 0) = 1
|
||||||
|
AND DECODE(PAID_UP_CAPITAL, :old.PAID_UP_CAPITAL, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
ELSIF UPDATING THEN
|
||||||
|
UPDATE DW_REF_MAIN.CAPITAL_KEY SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(COUNTRY_CODE, :old.COUNTRY_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(NCB_INDEX, :old.NCB_INDEX, 1, 0) = 1
|
||||||
|
AND DECODE(RAR_NCB_NAME, :old.RAR_NCB_NAME, 1, 0) = 1
|
||||||
|
AND DECODE(CAPITAL_KEY_VALUE, :old.CAPITAL_KEY_VALUE, 1, 0) = 1
|
||||||
|
AND DECODE(PAID_UP_CAPITAL, :old.PAID_UP_CAPITAL, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
UPDATE REFERENCE.T_CAPITAL_KEY SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(COUNTRY_CODE, :old.COUNTRY_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(NCB_NAME, :old.RAR_NCB_NAME, 1, 0) = 1
|
||||||
|
AND DECODE(EUROSYSTEM_KEY, :old.EUROSYSTEM_KEY, 1, 0) = 1
|
||||||
|
AND DECODE(EUROSYSTEM_KEY_ECB, :old.EUROSYSTEM_KEY_ECB, 1, 0) = 1
|
||||||
|
AND DECODE(ESCB_KEY, :old.ESCB_KEY, 1, 0) = 1
|
||||||
|
AND DECODE(ESCB_KEY_ECB, :old.ESCB_KEY_ECB, 1, 0) = 1
|
||||||
|
AND DECODE(CAPITAL_KEY_VALUE, :old.CAPITAL_KEY_VALUE, 1, 0) = 1
|
||||||
|
AND DECODE(PAID_UP_CAPITAL, :old.PAID_UP_CAPITAL, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
INSERT INTO DW_REF_MAIN.CAPITAL_KEY(COUNTRY_CODE, NCB_INDEX, CAPITAL_KEY_ADJ, RAR_NCB_NAME, CAPITAL_KEY_VALUE, PAID_UP_CAPITAL, A_VALID_FROM, A_VALID_TO, A_KEY)
|
||||||
|
VALUES (:new.COUNTRY_CODE, :new.NCB_INDEX, NULL, :new.RAR_NCB_NAME, :new.CAPITAL_KEY_VALUE, :new.PAID_UP_CAPITAL, v_current_date, v_future_date, NULL);
|
||||||
|
INSERT INTO REFERENCE.T_CAPITAL_KEY(COUNTRY_CODE, NCB_NAME, EUROSYSTEM_KEY, EUROSYSTEM_KEY_ECB, ESCB_KEY, ESCB_KEY_ECB, CAPITAL_KEY_VALUE, PAID_UP_CAPITAL, A_VALID_FROM, A_VALID_TO, A_KEY, A_MOPDB_LOAD_SET_FK)
|
||||||
|
VALUES (:new.COUNTRY_CODE, :new.RAR_NCB_NAME, :new.EUROSYSTEM_KEY, :new.EUROSYSTEM_KEY_ECB, :new.ESCB_KEY, :new.ESCB_KEY_ECB, :new.CAPITAL_KEY_VALUE, :new.PAID_UP_CAPITAL, v_current_date, v_future_date, NULL,-1);
|
||||||
|
END IF;
|
||||||
|
END;
|
||||||
|
/
|
||||||
@@ -0,0 +1,108 @@
|
|||||||
|
create or replace TRIGGER OU_REF.COUNTRY_H
|
||||||
|
BEFORE INSERT OR UPDATE OR DELETE ON OU_REF.COUNTRY
|
||||||
|
REFERENCING NEW AS new OLD AS old
|
||||||
|
FOR EACH ROW
|
||||||
|
DECLARE
|
||||||
|
v_current_date DATE := SYSDATE;
|
||||||
|
v_future_date DATE := TO_DATE('31/12/9999', 'DD/MM/YYYY');
|
||||||
|
BEGIN
|
||||||
|
IF INSERTING THEN
|
||||||
|
INSERT INTO DW_REF_MAIN.COUNTRY(COUNTRY_CODE, COUNTRY_NAME, IS_AREA, IS_EUROSYSTEM, IS_EU, IS_EEA, IS_G10, IS_EURO_PERIPHERY, IS_OECD_MEMBER, EXEMPT_FROM_CQR, LOSS_SHARING, NCB_NAME, REGION, A_VALID_FROM, A_VALID_TO, A_KEY)
|
||||||
|
VALUES (:new.COUNTRY_CODE, :new.COUNTRY_NAME, :new.IS_AREA, :new.IS_EUROSYSTEM, :new.IS_EU, :new.IS_EEA, :new.IS_G10, :new.IS_EURO_PERIPHERY, :new.IS_OECD_MEMBER, :new.EXEMPT_FROM_CQR, :new.LOSS_SHARING, :new.NCB_NAME, :new.REGION, v_current_date, v_future_date, NULL);
|
||||||
|
INSERT INTO REFERENCE.T_COUNTRY(COUNTRY_CODE, REF_AREA, COUNTRY_NAME, IS_AREA, IS_EUROSYSTEM, IS_EU, IS_EEA, IS_G10, IS_EURO_PERIPHERY, IS_OECD_MEMBER, EXEMPT_FROM_CQR, LOSS_SHARING, NCB_NAME, REGION, SORT_ORDER, BIZ_VALID_FROM, BIZ_VALID_TO, A_VALID_FROM, A_VALID_TO, A_KEY, A_MOPDB_LOAD_SET_FK)
|
||||||
|
VALUES (:new.COUNTRY_CODE, :new.REPORTING_COUNTRY_CODE, :new.COUNTRY_NAME, :new.IS_AREA, :new.IS_EUROSYSTEM, :new.IS_EU, :new.IS_EEA, :new.IS_G10, :new.IS_EURO_PERIPHERY, :new.IS_OECD_MEMBER, :new.EXEMPT_FROM_CQR, :new.LOSS_SHARING, :new.NCB_NAME, :new.REGION, :new.SORT_ORDER, :new.BIZ_VALID_FROM, :new.BIZ_VALID_TO, v_current_date, v_future_date, NULL, -1);
|
||||||
|
ELSIF DELETING THEN
|
||||||
|
UPDATE DW_REF_MAIN.COUNTRY SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(COUNTRY_CODE, :old.COUNTRY_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(COUNTRY_NAME, :old.COUNTRY_NAME, 1, 0) = 1
|
||||||
|
AND DECODE(IS_AREA, :old.IS_AREA, 1, 0) = 1
|
||||||
|
AND DECODE(IS_EUROSYSTEM, :old.IS_EUROSYSTEM, 1, 0) = 1
|
||||||
|
AND DECODE(IS_EU, :old.IS_EU, 1, 0) = 1
|
||||||
|
AND DECODE(IS_EEA, :old.IS_EEA, 1, 0) = 1
|
||||||
|
AND DECODE(IS_G10, :old.IS_G10, 1, 0) = 1
|
||||||
|
AND DECODE(IS_EURO_PERIPHERY, :old.IS_EURO_PERIPHERY, 1, 0) = 1
|
||||||
|
AND DECODE(IS_OECD_MEMBER, :old.IS_OECD_MEMBER, 1, 0) = 1
|
||||||
|
AND DECODE(EXEMPT_FROM_CQR, :old.EXEMPT_FROM_CQR, 1, 0) = 1
|
||||||
|
AND DECODE(LOSS_SHARING, :old.LOSS_SHARING, 1, 0) = 1
|
||||||
|
AND DECODE(NCB_NAME, :old.NCB_NAME, 1, 0) = 1
|
||||||
|
AND DECODE(REGION, :old.REGION, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
UPDATE REFERENCE.T_COUNTRY SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(COUNTRY_CODE, :old.COUNTRY_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(REF_AREA, :old.REPORTING_COUNTRY_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(COUNTRY_NAME, :old.COUNTRY_NAME, 1, 0) = 1
|
||||||
|
AND DECODE(IS_AREA, :old.IS_AREA, 1, 0) = 1
|
||||||
|
AND DECODE(IS_EUROSYSTEM, :old.IS_EUROSYSTEM, 1, 0) = 1
|
||||||
|
AND DECODE(IS_EU, :old.IS_EU, 1, 0) = 1
|
||||||
|
AND DECODE(IS_EEA, :old.IS_EEA, 1, 0) = 1
|
||||||
|
AND DECODE(IS_G10, :old.IS_G10, 1, 0) = 1
|
||||||
|
AND DECODE(IS_EURO_PERIPHERY, :old.IS_EURO_PERIPHERY, 1, 0) = 1
|
||||||
|
AND DECODE(IS_OECD_MEMBER, :old.IS_OECD_MEMBER, 1, 0) = 1
|
||||||
|
AND DECODE(EXEMPT_FROM_CQR, :old.EXEMPT_FROM_CQR, 1, 0) = 1
|
||||||
|
AND DECODE(LOSS_SHARING, :old.LOSS_SHARING, 1, 0) = 1
|
||||||
|
AND DECODE(NCB_NAME, :old.NCB_NAME, 1, 0) = 1
|
||||||
|
AND DECODE(REGION, :old.REGION, 1, 0) = 1
|
||||||
|
AND DECODE(SORT_ORDER, :old.SORT_ORDER, 1, 0) = 1
|
||||||
|
AND DECODE(BIZ_VALID_FROM, :old.BIZ_VALID_FROM, 1, 0) = 1
|
||||||
|
AND DECODE(BIZ_VALID_TO, :old.BIZ_VALID_TO, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
|
||||||
|
ELSIF UPDATING THEN
|
||||||
|
UPDATE DW_REF_MAIN.COUNTRY SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(COUNTRY_CODE, :old.COUNTRY_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(COUNTRY_NAME, :old.COUNTRY_NAME, 1, 0) = 1
|
||||||
|
AND DECODE(IS_AREA, :old.IS_AREA, 1, 0) = 1
|
||||||
|
AND DECODE(IS_EUROSYSTEM, :old.IS_EUROSYSTEM, 1, 0) = 1
|
||||||
|
AND DECODE(IS_EU, :old.IS_EU, 1, 0) = 1
|
||||||
|
AND DECODE(IS_EEA, :old.IS_EEA, 1, 0) = 1
|
||||||
|
AND DECODE(IS_G10, :old.IS_G10, 1, 0) = 1
|
||||||
|
AND DECODE(IS_EURO_PERIPHERY, :old.IS_EURO_PERIPHERY, 1, 0) = 1
|
||||||
|
AND DECODE(IS_OECD_MEMBER, :old.IS_OECD_MEMBER, 1, 0) = 1
|
||||||
|
AND DECODE(EXEMPT_FROM_CQR, :old.EXEMPT_FROM_CQR, 1, 0) = 1
|
||||||
|
AND DECODE(LOSS_SHARING, :old.LOSS_SHARING, 1, 0) = 1
|
||||||
|
AND DECODE(NCB_NAME, :old.NCB_NAME, 1, 0) = 1
|
||||||
|
AND DECODE(REGION, :old.REGION, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
UPDATE REFERENCE.T_COUNTRY SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(COUNTRY_CODE, :old.COUNTRY_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(REF_AREA, :old.REPORTING_COUNTRY_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(COUNTRY_NAME, :old.COUNTRY_NAME, 1, 0) = 1
|
||||||
|
AND DECODE(IS_AREA, :old.IS_AREA, 1, 0) = 1
|
||||||
|
AND DECODE(IS_EUROSYSTEM, :old.IS_EUROSYSTEM, 1, 0) = 1
|
||||||
|
AND DECODE(IS_EU, :old.IS_EU, 1, 0) = 1
|
||||||
|
AND DECODE(IS_EEA, :old.IS_EEA, 1, 0) = 1
|
||||||
|
AND DECODE(IS_G10, :old.IS_G10, 1, 0) = 1
|
||||||
|
AND DECODE(IS_EURO_PERIPHERY, :old.IS_EURO_PERIPHERY, 1, 0) = 1
|
||||||
|
AND DECODE(IS_OECD_MEMBER, :old.IS_OECD_MEMBER, 1, 0) = 1
|
||||||
|
AND DECODE(EXEMPT_FROM_CQR, :old.EXEMPT_FROM_CQR, 1, 0) = 1
|
||||||
|
AND DECODE(LOSS_SHARING, :old.LOSS_SHARING, 1, 0) = 1
|
||||||
|
AND DECODE(NCB_NAME, :old.NCB_NAME, 1, 0) = 1
|
||||||
|
AND DECODE(REGION, :old.REGION, 1, 0) = 1
|
||||||
|
AND DECODE(SORT_ORDER, :old.SORT_ORDER, 1, 0) = 1
|
||||||
|
AND DECODE(BIZ_VALID_FROM, :old.BIZ_VALID_FROM, 1, 0) = 1
|
||||||
|
AND DECODE(BIZ_VALID_TO, :old.BIZ_VALID_TO, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
INSERT INTO DW_REF_MAIN.COUNTRY(COUNTRY_CODE, COUNTRY_NAME, IS_AREA, IS_EUROSYSTEM, IS_EU, IS_EEA, IS_G10, IS_EURO_PERIPHERY, IS_OECD_MEMBER, EXEMPT_FROM_CQR, LOSS_SHARING, NCB_NAME, REGION, A_VALID_FROM, A_VALID_TO, A_KEY)
|
||||||
|
VALUES (:new.COUNTRY_CODE, :new.COUNTRY_NAME, :new.IS_AREA, :new.IS_EUROSYSTEM, :new.IS_EU, :new.IS_EEA, :new.IS_G10, :new.IS_EURO_PERIPHERY, :new.IS_OECD_MEMBER, :new.EXEMPT_FROM_CQR, :new.LOSS_SHARING, :new.NCB_NAME, :new.REGION, v_current_date, v_future_date, NULL);
|
||||||
|
INSERT INTO REFERENCE.T_COUNTRY(COUNTRY_CODE, REF_AREA, COUNTRY_NAME, IS_AREA, IS_EUROSYSTEM, IS_EU, IS_EEA, IS_G10, IS_EURO_PERIPHERY, IS_OECD_MEMBER, EXEMPT_FROM_CQR, LOSS_SHARING, NCB_NAME, REGION, SORT_ORDER, BIZ_VALID_FROM, BIZ_VALID_TO, A_VALID_FROM, A_VALID_TO, A_KEY, A_MOPDB_LOAD_SET_FK)
|
||||||
|
VALUES (:new.COUNTRY_CODE, :new.REPORTING_COUNTRY_CODE, :new.COUNTRY_NAME, :new.IS_AREA, :new.IS_EUROSYSTEM, :new.IS_EU, :new.IS_EEA, :new.IS_G10, :new.IS_EURO_PERIPHERY, :new.IS_OECD_MEMBER, :new.EXEMPT_FROM_CQR, :new.LOSS_SHARING, :new.NCB_NAME, :new.REGION, :new.SORT_ORDER, :new.BIZ_VALID_FROM, :new.BIZ_VALID_TO, v_current_date, v_future_date, NULL, -1);
|
||||||
|
|
||||||
|
END IF;
|
||||||
|
END;
|
||||||
|
/
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
create or replace TRIGGER OU_REF.CSDB_RATING_ACTION_H
|
||||||
|
BEFORE INSERT OR UPDATE OR DELETE ON OU_REF.CSDB_RATING_ACTION
|
||||||
|
REFERENCING NEW AS new OLD AS old
|
||||||
|
FOR EACH ROW
|
||||||
|
DECLARE
|
||||||
|
v_current_date DATE := SYSDATE;
|
||||||
|
v_future_date DATE := TO_DATE('31/12/9999', 'DD/MM/YYYY');
|
||||||
|
BEGIN
|
||||||
|
IF INSERTING THEN
|
||||||
|
INSERT INTO REFERENCE.T_CSDB_RATING_ACTION(CSDB_RATING_ACTION_CODE, CSDB_RATING_ACTION_DESC, SOURCE, A_VALID_FROM, A_VALID_TO, A_KEY, A_MOPDB_LOAD_SET_FK)
|
||||||
|
VALUES (:new.CSDB_RATING_ACTION_CODE, :new.CSDB_RATING_ACTION_DESC, :new.SOURCE, v_current_date, v_future_date, NULL, -1);
|
||||||
|
ELSIF DELETING THEN
|
||||||
|
UPDATE REFERENCE.T_CSDB_RATING_ACTION SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(CSDB_RATING_ACTION_CODE, :old.CSDB_RATING_ACTION_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(CSDB_RATING_ACTION_DESC, :old.CSDB_RATING_ACTION_DESC, 1, 0) = 1
|
||||||
|
AND DECODE(SOURCE, :old.SOURCE, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
ELSIF UPDATING THEN
|
||||||
|
UPDATE REFERENCE.T_CSDB_RATING_ACTION SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(CSDB_RATING_ACTION_CODE, :old.CSDB_RATING_ACTION_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(CSDB_RATING_ACTION_DESC, :old.CSDB_RATING_ACTION_DESC, 1, 0) = 1
|
||||||
|
AND DECODE(SOURCE, :old.SOURCE, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
INSERT INTO REFERENCE.T_CSDB_RATING_ACTION(CSDB_RATING_ACTION_CODE, CSDB_RATING_ACTION_DESC, SOURCE, A_VALID_FROM, A_VALID_TO, A_KEY, A_MOPDB_LOAD_SET_FK)
|
||||||
|
VALUES (:new.CSDB_RATING_ACTION_CODE, :new.CSDB_RATING_ACTION_DESC, :new.SOURCE, v_current_date, v_future_date, NULL, -1);
|
||||||
|
END IF;
|
||||||
|
END;
|
||||||
|
/
|
||||||
@@ -0,0 +1,66 @@
|
|||||||
|
create or replace TRIGGER OU_REF.CURRENCY_H
|
||||||
|
BEFORE INSERT OR UPDATE OR DELETE ON OU_REF.CURRENCY
|
||||||
|
REFERENCING NEW AS new OLD AS old
|
||||||
|
FOR EACH ROW
|
||||||
|
DECLARE
|
||||||
|
v_current_date DATE := SYSDATE;
|
||||||
|
v_future_date DATE := TO_DATE('31/12/9999', 'DD/MM/YYYY');
|
||||||
|
BEGIN
|
||||||
|
IF INSERTING THEN
|
||||||
|
INSERT INTO DW_REF_MAIN.CURRENCY(CURRENCY_CODE, CURRENCY_NAME, COUNTRY_CODE, EUR_LEGACY, LEGACY, A_VALID_FROM, A_VALID_TO, A_KEY)
|
||||||
|
VALUES (:new.CURRENCY_CODE, :new.CURRENCY_NAME, :new.COUNTRY_CODE, :new.EUR_LEGACY, :new.LEGACY, v_current_date, v_future_date, NULL);
|
||||||
|
INSERT INTO REFERENCE.T_CURRENCY(CURRENCY_CODE, CURRENCY_NAME, COUNTRY_CODE, EUR_LEGACY, LEGACY, A_VALID_FROM, A_VALID_TO, A_KEY, A_MOPDB_LOAD_SET_FK)
|
||||||
|
VALUES (:new.CURRENCY_CODE, :new.CURRENCY_NAME, :new.COUNTRY_CODE, :new.EUR_LEGACY, :new.LEGACY, v_current_date, v_future_date, NULL, -1);
|
||||||
|
ELSIF DELETING THEN
|
||||||
|
UPDATE DW_REF_MAIN.CURRENCY SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(CURRENCY_CODE, :old.CURRENCY_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(CURRENCY_NAME, :old.CURRENCY_NAME, 1, 0) = 1
|
||||||
|
AND DECODE(COUNTRY_CODE, :old.COUNTRY_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(EUR_LEGACY, :old.EUR_LEGACY, 1, 0) = 1
|
||||||
|
AND DECODE(LEGACY, :old.LEGACY, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
UPDATE REFERENCE.T_CURRENCY SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(CURRENCY_CODE, :old.CURRENCY_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(CURRENCY_NAME, :old.CURRENCY_NAME, 1, 0) = 1
|
||||||
|
AND DECODE(COUNTRY_CODE, :old.COUNTRY_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(EUR_LEGACY, :old.EUR_LEGACY, 1, 0) = 1
|
||||||
|
AND DECODE(LEGACY, :old.LEGACY, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
ELSIF UPDATING THEN
|
||||||
|
UPDATE DW_REF_MAIN.CURRENCY SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(CURRENCY_CODE, :old.CURRENCY_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(CURRENCY_NAME, :old.CURRENCY_NAME, 1, 0) = 1
|
||||||
|
AND DECODE(COUNTRY_CODE, :old.COUNTRY_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(EUR_LEGACY, :old.EUR_LEGACY, 1, 0) = 1
|
||||||
|
AND DECODE(LEGACY, :old.LEGACY, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
UPDATE REFERENCE.T_CURRENCY SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(CURRENCY_CODE, :old.CURRENCY_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(CURRENCY_NAME, :old.CURRENCY_NAME, 1, 0) = 1
|
||||||
|
AND DECODE(COUNTRY_CODE, :old.COUNTRY_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(EUR_LEGACY, :old.EUR_LEGACY, 1, 0) = 1
|
||||||
|
AND DECODE(LEGACY, :old.LEGACY, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
INSERT INTO DW_REF_MAIN.CURRENCY(CURRENCY_CODE, CURRENCY_NAME, COUNTRY_CODE, EUR_LEGACY, LEGACY, A_VALID_FROM, A_VALID_TO, A_KEY)
|
||||||
|
VALUES (:new.CURRENCY_CODE, :new.CURRENCY_NAME, :new.COUNTRY_CODE, :new.EUR_LEGACY, :new.LEGACY, v_current_date, v_future_date, NULL);
|
||||||
|
INSERT INTO REFERENCE.T_CURRENCY(CURRENCY_CODE, CURRENCY_NAME, COUNTRY_CODE, EUR_LEGACY, LEGACY, A_VALID_FROM, A_VALID_TO, A_KEY, A_MOPDB_LOAD_SET_FK)
|
||||||
|
VALUES (:new.CURRENCY_CODE, :new.CURRENCY_NAME, :new.COUNTRY_CODE, :new.EUR_LEGACY, :new.LEGACY, v_current_date, v_future_date, NULL, -1);
|
||||||
|
END IF;
|
||||||
|
END;
|
||||||
|
/
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
create or replace TRIGGER OU_REF.INSTITUTION_ROLE_H
|
||||||
|
BEFORE INSERT OR UPDATE OR DELETE ON OU_REF.INSTITUTION_ROLE
|
||||||
|
REFERENCING NEW AS new OLD AS old
|
||||||
|
FOR EACH ROW
|
||||||
|
DECLARE
|
||||||
|
v_current_date DATE := SYSDATE;
|
||||||
|
v_future_date DATE := TO_DATE('31/12/9999', 'DD/MM/YYYY');
|
||||||
|
BEGIN
|
||||||
|
IF INSERTING THEN
|
||||||
|
INSERT INTO DW_REF_MAIN.INSTITUTION_ROLE(ROLE_CODE, ROLE_DESCRIPTION, A_VALID_FROM, A_VALID_TO, A_KEY)
|
||||||
|
VALUES (:new.ROLE_CODE, :new.ROLE_DESCRIPTION, v_current_date, v_future_date, NULL);
|
||||||
|
INSERT INTO DW_REF_MAIN.INSTITUTION_ROLE_C2D(ROLE_CODE, ROLE_DESCRIPTION, A_VALID_FROM, A_VALID_TO, A_KEY)
|
||||||
|
VALUES (:new.ROLE_CODE, :new.ROLE_DESCRIPTION, v_current_date, v_future_date, NULL);
|
||||||
|
ELSIF DELETING THEN
|
||||||
|
UPDATE DW_REF_MAIN.INSTITUTION_ROLE SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(ROLE_CODE, :old.ROLE_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(ROLE_DESCRIPTION, :old.ROLE_DESCRIPTION, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
UPDATE DW_REF_MAIN.INSTITUTION_ROLE_C2D SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(ROLE_CODE, :old.ROLE_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(ROLE_DESCRIPTION, :old.ROLE_DESCRIPTION, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
ELSIF UPDATING THEN
|
||||||
|
UPDATE DW_REF_MAIN.INSTITUTION_ROLE SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(ROLE_CODE, :old.ROLE_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(ROLE_DESCRIPTION, :old.ROLE_DESCRIPTION, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
UPDATE DW_REF_MAIN.INSTITUTION_ROLE_C2D SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(ROLE_CODE, :old.ROLE_CODE, 1, 0) = 1
|
||||||
|
AND DECODE(ROLE_DESCRIPTION, :old.ROLE_DESCRIPTION, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
INSERT INTO DW_REF_MAIN.INSTITUTION_ROLE(ROLE_CODE, ROLE_DESCRIPTION, A_VALID_FROM, A_VALID_TO, A_KEY)
|
||||||
|
VALUES (:new.ROLE_CODE, :new.ROLE_DESCRIPTION, v_current_date, v_future_date, NULL);
|
||||||
|
INSERT INTO DW_REF_MAIN.INSTITUTION_ROLE_C2D(ROLE_CODE, ROLE_DESCRIPTION, A_VALID_FROM, A_VALID_TO, A_KEY)
|
||||||
|
VALUES (:new.ROLE_CODE, :new.ROLE_DESCRIPTION, v_current_date, v_future_date, NULL);
|
||||||
|
END IF;
|
||||||
|
END;
|
||||||
|
/
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
create or replace TRIGGER OU_REF.TMS_ACTIVITY_GROUPING_H
|
||||||
|
BEFORE INSERT OR UPDATE OR DELETE ON OU_REF.TMS_ACTIVITY_GROUPING
|
||||||
|
REFERENCING NEW AS new OLD AS old
|
||||||
|
FOR EACH ROW
|
||||||
|
DECLARE
|
||||||
|
v_current_date DATE := SYSDATE;
|
||||||
|
v_future_date DATE := TO_DATE('31/12/9999', 'DD/MM/YYYY');
|
||||||
|
BEGIN
|
||||||
|
IF INSERTING THEN
|
||||||
|
INSERT INTO DW_REF_MAIN.TMS_ACTIVITY_GROUPING(GROUPING_PARAMETER, GROUPING_VALUE, A_VALID_FROM, A_VALID_TO, A_KEY)
|
||||||
|
VALUES (:new.GROUPING_PARAMETER, :new.GROUPING_VALUE, v_current_date, v_future_date, NULL);
|
||||||
|
INSERT INTO REFERENCE.T_TMS_ACTIVITY_GROUPING(GROUPING_PARAMETER, GROUPING_VALUE, A_VALID_FROM, A_VALID_TO, A_KEY, A_MOPDB_LOAD_SET_FK)
|
||||||
|
VALUES (:new.GROUPING_PARAMETER, :new.GROUPING_VALUE, v_current_date, v_future_date, NULL, -1);
|
||||||
|
ELSIF DELETING THEN
|
||||||
|
UPDATE DW_REF_MAIN.TMS_ACTIVITY_GROUPING SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(GROUPING_PARAMETER, :old.GROUPING_PARAMETER, 1, 0) = 1
|
||||||
|
AND DECODE(GROUPING_VALUE, :old.GROUPING_VALUE, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
UPDATE REFERENCE.T_TMS_ACTIVITY_GROUPING SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(GROUPING_PARAMETER, :old.GROUPING_PARAMETER, 1, 0) = 1
|
||||||
|
AND DECODE(GROUPING_VALUE, :old.GROUPING_VALUE, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
ELSIF UPDATING THEN
|
||||||
|
UPDATE DW_REF_MAIN.TMS_ACTIVITY_GROUPING SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(GROUPING_PARAMETER, :old.GROUPING_PARAMETER, 1, 0) = 1
|
||||||
|
AND DECODE(GROUPING_VALUE, :old.GROUPING_VALUE, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
UPDATE REFERENCE.T_TMS_ACTIVITY_GROUPING SET
|
||||||
|
A_VALID_TO = v_current_date
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(GROUPING_PARAMETER, :old.GROUPING_PARAMETER, 1, 0) = 1
|
||||||
|
AND DECODE(GROUPING_VALUE, :old.GROUPING_VALUE, 1, 0) = 1
|
||||||
|
AND A_VALID_TO = v_future_date
|
||||||
|
;
|
||||||
|
INSERT INTO DW_REF_MAIN.TMS_ACTIVITY_GROUPING(GROUPING_PARAMETER, GROUPING_VALUE, A_VALID_FROM, A_VALID_TO, A_KEY)
|
||||||
|
VALUES (:new.GROUPING_PARAMETER, :new.GROUPING_VALUE, v_current_date, v_future_date, NULL);
|
||||||
|
INSERT INTO REFERENCE.T_TMS_ACTIVITY_GROUPING(GROUPING_PARAMETER, GROUPING_VALUE, A_VALID_FROM, A_VALID_TO, A_KEY, A_MOPDB_LOAD_SET_FK)
|
||||||
|
VALUES (:new.GROUPING_PARAMETER, :new.GROUPING_VALUE, v_current_date, v_future_date, NULL, -1);
|
||||||
|
END IF;
|
||||||
|
END;
|
||||||
|
/
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
create or replace TRIGGER OU_REF.TIME_TRANSFORMATION_H
|
||||||
|
BEFORE INSERT OR UPDATE OR DELETE ON OU_REF.TIME_TRANSFORMATION
|
||||||
|
REFERENCING NEW AS new OLD AS old
|
||||||
|
FOR EACH ROW
|
||||||
|
DECLARE
|
||||||
|
v_current_date DATE := SYSDATE;
|
||||||
|
v_future_date DATE := TO_DATE('31/12/9999', 'DD/MM/YYYY');
|
||||||
|
BEGIN
|
||||||
|
IF INSERTING THEN
|
||||||
|
INSERT INTO DW_REF_MAIN.TIME_TRANSFORMATION(YEAR_START, TIME_ZONE_START, CALENDAR_DATE_START, YEAR_END, TIME_ZONE_END, CALENDAR_DATE_END)
|
||||||
|
VALUES (:new.YEAR_START, :new.TIME_ZONE_START, :new.CALENDAR_DATE_START, :new.YEAR_END, :new.TIME_ZONE_END, :new.CALENDAR_DATE_END);
|
||||||
|
ELSIF DELETING THEN
|
||||||
|
DELETE FROM DW_REF_MAIN.TIME_TRANSFORMATION
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(YEAR_START, :old.YEAR_START, 1, 0) = 1
|
||||||
|
AND DECODE(TIME_ZONE_START, :old.TIME_ZONE_START, 1, 0) = 1
|
||||||
|
AND DECODE(CALENDAR_DATE_START, :old.CALENDAR_DATE_START, 1, 0) = 1
|
||||||
|
AND DECODE(YEAR_END, :old.YEAR_END, 1, 0) = 1
|
||||||
|
AND DECODE(TIME_ZONE_END, :old.TIME_ZONE_END, 1, 0) = 1
|
||||||
|
AND DECODE(CALENDAR_DATE_END, :old.CALENDAR_DATE_END, 1, 0) = 1
|
||||||
|
;
|
||||||
|
ELSIF UPDATING THEN
|
||||||
|
UPDATE DW_REF_MAIN.TIME_TRANSFORMATION SET
|
||||||
|
YEAR_START = :new.YEAR_START,
|
||||||
|
TIME_ZONE_START = :new.TIME_ZONE_START,
|
||||||
|
CALENDAR_DATE_START = :new.CALENDAR_DATE_START,
|
||||||
|
YEAR_END = :new.YEAR_END,
|
||||||
|
TIME_ZONE_END = :new.TIME_ZONE_END,
|
||||||
|
CALENDAR_DATE_END = :new.CALENDAR_DATE_END
|
||||||
|
WHERE
|
||||||
|
1 = 1
|
||||||
|
AND DECODE(YEAR_START, :old.YEAR_START, 1, 0) = 1
|
||||||
|
AND DECODE(TIME_ZONE_START, :old.TIME_ZONE_START, 1, 0) = 1
|
||||||
|
AND DECODE(CALENDAR_DATE_START, :old.CALENDAR_DATE_START, 1, 0) = 1
|
||||||
|
AND DECODE(YEAR_END, :old.YEAR_END, 1, 0) = 1
|
||||||
|
AND DECODE(TIME_ZONE_END, :old.TIME_ZONE_END, 1, 0) = 1
|
||||||
|
AND DECODE(CALENDAR_DATE_END, :old.CALENDAR_DATE_END, 1, 0) = 1
|
||||||
|
;
|
||||||
|
END IF;
|
||||||
|
END;
|
||||||
|
/
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
|
||||||
|
SET SERVEROUTPUT ON;
|
||||||
|
DECLARE
|
||||||
|
v_drop_sql VARCHAR2(32767);
|
||||||
|
BEGIN
|
||||||
|
FOR i IN (
|
||||||
|
SELECT
|
||||||
|
'DROP TRIGGER DW_REF_MAIN.'||TRIGGER_NAME AS sql_str,
|
||||||
|
'DW_REF_MAIN.AUTOKEY_'||TABLE_NAME AS trigger_name
|
||||||
|
FROM DBA_TRIGGERS
|
||||||
|
WHERE
|
||||||
|
OWNER = 'DW_REF_MAIN'
|
||||||
|
AND TRIGGER_NAME LIKE 'AUTOKEY%'
|
||||||
|
AND TABLE_NAME IN (
|
||||||
|
'ASSET_CODE_TYPE', 'ABS_TYPE', 'ALLOTMENT_METHOD', 'AREA_CODE_RIAD', 'ASSET_GROUP_DETAIL', 'ASSET_GROUP_MAPPING', 'ASSET_GROUP_TMS', 'ASSET_GROUP_TMS_OLD_NEW_MAP', 'ASSET_TYPE', 'ASSET_TYPE_TMS',
|
||||||
|
'CAPITAL_KEY', 'COUNTRY', 'CSPP_BENCHMARK', 'CSPP_GROUP_MEMBERS', 'CSPP_GROUPS', 'CURRENCY', 'ELA_COUNTRY_CEILING', 'ELA_FUNDING_GAP', 'ELA_MFI_CEILING', 'ESA_95_INSTR_CLASS_CSDB',
|
||||||
|
'ESA_95_SECTOR', 'ESA_2010_INSTR_CLASS_CSDB', 'ESA_2010_ISSUER_CLASS_CSDB', 'ESAST_SUB_SECTOR_RIAD', 'HAIRCUT_CATEGORIES', 'HAIRCUT_SCHEDULE', 'HOLIDAY', 'INSTITUTION_C2D', 'INSTITUTION_CODE_TYPE', 'INSTITUTION_ECONOMIC_ACTIVITY',
|
||||||
|
'INSTITUTION_IDENTIFIER_TYPE', 'INSTITUTION_RAT_OUTLOOK_VALUE', 'INSTITUTION_RAT_WATCH_VALUE', 'INSTITUTION_ROLE', 'INSTITUTION_SUBROLE', 'ISSUER_DEBTOR_GUARANTOR_GROUP', 'LIMIT_BREACH_CAUSE', 'LIMIT_TYPE', 'LIMIT_VIOLATION_TYPE', 'MAP_COUPON_FREQ',
|
||||||
|
'MAP_COUPON_FREQ_RAR', 'MAP_COUPON_TYPE', 'MAP_COUPON_TYPE_RAR', 'NACE2_INDUSTRY_CLASS_CSDB', 'OPERATION_NAME', 'OPERATION_PROCEDURE', 'OPERATION_PUBLIC', 'OPERATION_TYPE', 'OWN_USE', 'PRICE_TYPE_MAP',
|
||||||
|
'PRICE_TYPE_RAR', 'PSE_TYPE', 'QUOT_BASIS_MAP', 'RATING', 'RATING_ACTION', 'RATING_PD', 'RATING_PRIORITY', 'RATING_SOURCE', 'RATING_TYPE', 'REDEMPTION_FREQUENCY_CSDB',
|
||||||
|
'REDEMPTION_TYPE_CSDB', 'RIC_CODE_CREATION_RULE', 'TMS_ACTIVITY_GROUPING', 'TMS_POSITION_CONFIG', 'MARKET_COUNTRIES', 'MEASUREMENT_UNITS', 'NCB', 'PORTFOLIO', 'PP_BUCKET', 'PP_CASH_FLOW_TYPE',
|
||||||
|
'PP_METHOD', 'PP_PORTF_BNCHMK_MAP', 'SCENARIOS', 'INSTITUTION_ROLE_C2D'
|
||||||
|
)
|
||||||
|
ORDER BY TABLE_NAME
|
||||||
|
) LOOP
|
||||||
|
v_drop_sql := i.sql_str;
|
||||||
|
|
||||||
|
DBMS_OUTPUT.PUT_LINE(i.trigger_name);
|
||||||
|
EXECUTE IMMEDIATE v_drop_sql;
|
||||||
|
|
||||||
|
-- DBMS_OUTPUT.PUT_LINE(v_drop_sql||'
|
||||||
|
-- /
|
||||||
|
-- ');
|
||||||
|
|
||||||
|
END LOOP;
|
||||||
|
|
||||||
|
END;
|
||||||
|
/
|
||||||
@@ -0,0 +1,44 @@
|
|||||||
|
ALTER TABLE OU_REF.QUOT_BASIS_MAP DROP CONSTRAINT CHECK_ANALYZED;
|
||||||
|
|
||||||
|
REVOKE SELECT ANY TABLE ON SCHEMA DW_REF_MAIN FROM OU_REF;
|
||||||
|
REVOKE INSERT ANY TABLE ON SCHEMA DW_REF_MAIN FROM OU_REF;
|
||||||
|
REVOKE UPDATE ANY TABLE ON SCHEMA DW_REF_MAIN FROM OU_REF;
|
||||||
|
REVOKE DELETE ANY TABLE ON SCHEMA DW_REF_MAIN FROM OU_REF;
|
||||||
|
REVOKE SELECT ANY TABLE ON SCHEMA REFERENCE FROM OU_REF;
|
||||||
|
REVOKE INSERT ANY TABLE ON SCHEMA REFERENCE FROM OU_REF;
|
||||||
|
REVOKE UPDATE ANY TABLE ON SCHEMA REFERENCE FROM OU_REF;
|
||||||
|
REVOKE DELETE ANY TABLE ON SCHEMA REFERENCE FROM OU_REF;
|
||||||
|
|
||||||
|
|
||||||
|
SET SERVEROUTPUT ON;
|
||||||
|
DECLARE
|
||||||
|
v_drop_sql VARCHAR2(32767);
|
||||||
|
BEGIN
|
||||||
|
FOR i IN (
|
||||||
|
SELECT
|
||||||
|
'DROP TRIGGER OU_REF.'||TRIGGER_NAME AS sql_str,
|
||||||
|
'DW_REF_MAIN.'||TABLE_NAME||'_H' AS trigger_name
|
||||||
|
FROM DBA_TRIGGERS
|
||||||
|
WHERE
|
||||||
|
OWNER = 'OU_REF'
|
||||||
|
AND TRIGGER_NAME LIKE '%_H'
|
||||||
|
) LOOP
|
||||||
|
v_drop_sql := i.sql_str;
|
||||||
|
|
||||||
|
DBMS_OUTPUT.PUT_LINE(i.trigger_name);
|
||||||
|
EXECUTE IMMEDIATE v_drop_sql;
|
||||||
|
|
||||||
|
-- DBMS_OUTPUT.PUT_LINE(v_drop_sql||'
|
||||||
|
--/
|
||||||
|
-- ');
|
||||||
|
|
||||||
|
END LOOP;
|
||||||
|
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
|
||||||
|
SET SERVEROUTPUT ON;
|
||||||
|
DECLARE
|
||||||
|
v_drop_sql VARCHAR2(32767);
|
||||||
|
BEGIN
|
||||||
|
FOR i IN (
|
||||||
|
SELECT
|
||||||
|
'DROP TRIGGER REFERENCE.'||TRIGGER_NAME AS sql_str,
|
||||||
|
'REFERENCE.AUTOKEY_'||TABLE_NAME AS trigger_name
|
||||||
|
FROM DBA_TRIGGERS
|
||||||
|
WHERE
|
||||||
|
OWNER = 'REFERENCE'
|
||||||
|
AND TRIGGER_NAME LIKE 'AUTOKEY%'
|
||||||
|
AND TABLE_NAME IN (
|
||||||
|
'T_BALANCE_SHEET_STRUCTURE', 'T_CAPITAL_KEY', 'T_COUNTRY', 'T_CSDB_RATING_ACTION', 'T_CURRENCY', 'T_TMS_ACTIVITY_GROUPING'
|
||||||
|
)
|
||||||
|
) LOOP
|
||||||
|
v_drop_sql := i.sql_str;
|
||||||
|
|
||||||
|
DBMS_OUTPUT.PUT_LINE(i.trigger_name);
|
||||||
|
EXECUTE IMMEDIATE v_drop_sql;
|
||||||
|
|
||||||
|
-- DBMS_OUTPUT.PUT_LINE(v_drop_sql||'
|
||||||
|
-- /
|
||||||
|
-- ');
|
||||||
|
|
||||||
|
END LOOP;
|
||||||
|
|
||||||
|
END;
|
||||||
|
/
|
||||||
42
MARS_Packages/REL01/MARS-723/install_mars723.sql
Normal file
42
MARS_Packages/REL01/MARS-723/install_mars723.sql
Normal file
@@ -0,0 +1,42 @@
|
|||||||
|
WHENEVER SQLERROR EXIT FAILURE
|
||||||
|
SET SERVEROUTPUT ON
|
||||||
|
SET TIMING ON
|
||||||
|
SET ECHO ON
|
||||||
|
SET HEADING OFF
|
||||||
|
SET FEEDBACK ON
|
||||||
|
SET VERIFY OFF
|
||||||
|
|
||||||
|
var filename VARCHAR2(100)
|
||||||
|
BEGIN
|
||||||
|
SELECT 'INSTALL_MARS_723_' || SUBSTR(PDB_NAME, (INSTR(PDB_NAME,'_',1)+1), (LENGTH(PDB_NAME)-INSTR(PDB_NAME,'_',1))) || '_' ||TO_CHAR(SYSDATE,'YYYYMMDD_HH24MISS')||'.log' INTO :filename from DBA_PDBS;
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
column filename new_value _filename
|
||||||
|
select :filename filename from dual;
|
||||||
|
spool &_filename
|
||||||
|
|
||||||
|
prompt ##### started at time #####
|
||||||
|
select systimestamp from dual;
|
||||||
|
prompt ##### database name #####
|
||||||
|
SELECT SUBSTR(PDB_NAME, (INSTR(PDB_NAME,'_',1)+1), (LENGTH(PDB_NAME)-INSTR(PDB_NAME,'_',1))) AS PDB_NAME FROM DBA_PDBS;
|
||||||
|
|
||||||
|
|
||||||
|
@@01_MARS_723_install_DW_REF_MAIN_CREATE_AUTOKEY_TRIGGER.sql
|
||||||
|
@@02_MARS_723_install_REFERENCE_CREATE_AUTOKEY_TRIGGER.sql
|
||||||
|
@@03_MARS_723_install_OU_REF_CREATE_HISTORY_TRIGGER_DW_REF_MAIN.sql
|
||||||
|
@@05_MARS_723_install_OU_REF_CREATE_HISTORY_TRIGGER_BALANCE_SHEET_STRUCTURE_H.sql
|
||||||
|
@@06_MARS_723_install_OU_REF_CREATE_HISTORY_TRIGGER_CAPITAL_KEY_H.sql
|
||||||
|
@@07_MARS_723_install_OU_REF_CREATE_HISTORY_TRIGGER_COUNTRY_H.sql
|
||||||
|
@@08_MARS_723_install_OU_REF_CREATE_HISTORY_TRIGGER_CSDB_RATING_ACTION_H.sql
|
||||||
|
@@09_MARS_723_install_OU_REF_CREATE_HISTORY_TRIGGER_CURRENCY_H.sql
|
||||||
|
@@10_MARS_723_install_OU_REF_CREATE_HISTORY_TRIGGER_INSTITUTION_ROLE_H.sql
|
||||||
|
@@11_MARS_723_install_OU_REF_CREATE_HISTORY_TRIGGER_TMS_ACTIVITY_GROUPING_H.sql
|
||||||
|
@@12_MARS_723_install_OU_REF_CREATE_HISTORY_TRIGGER_TIME_TRANSFORMATION_H.sql
|
||||||
|
|
||||||
|
SET ECHO OFF
|
||||||
|
|
||||||
|
prompt ##### completed at time #####
|
||||||
|
select systimestamp from dual;
|
||||||
|
|
||||||
|
SPOOL OFF
|
||||||
|
EXIT
|
||||||
35
MARS_Packages/REL01/MARS-723/rollback_mars723.sql
Normal file
35
MARS_Packages/REL01/MARS-723/rollback_mars723.sql
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
WHENEVER SQLERROR EXIT FAILURE
|
||||||
|
SET SERVEROUTPUT ON
|
||||||
|
SET TIMING ON
|
||||||
|
SET ECHO ON
|
||||||
|
SET HEADING OFF
|
||||||
|
SET FEEDBACK ON
|
||||||
|
SET VERIFY OFF
|
||||||
|
|
||||||
|
var filename VARCHAR2(100)
|
||||||
|
BEGIN
|
||||||
|
SELECT 'ROLLBACK_MARS_723_' || SUBSTR(PDB_NAME, (INSTR(PDB_NAME,'_',1)+1), (LENGTH(PDB_NAME)-INSTR(PDB_NAME,'_',1))) || '_' ||TO_CHAR(SYSDATE,'YYYYMMDD_HH24MISS')||'.log' INTO :filename from DBA_PDBS;
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
column filename new_value _filename
|
||||||
|
select :filename filename from dual;
|
||||||
|
spool &_filename
|
||||||
|
|
||||||
|
prompt ##### started at time #####
|
||||||
|
select systimestamp from dual;
|
||||||
|
prompt ##### database name #####
|
||||||
|
SELECT SUBSTR(PDB_NAME, (INSTR(PDB_NAME,'_',1)+1), (LENGTH(PDB_NAME)-INSTR(PDB_NAME,'_',1))) AS PDB_NAME FROM DBA_PDBS;
|
||||||
|
|
||||||
|
|
||||||
|
@@91_MARS_723_rollback_DW_REF_MAIN_AUTOKEY_TRIGGER.sql
|
||||||
|
@@91_MARS_723_rollback_OU_REF_HISTORY_TRIGGERS.sql
|
||||||
|
@@91_MARS_723_rollback_REFERENCE_AUTOKEY_TRIGGER.sql
|
||||||
|
|
||||||
|
|
||||||
|
SET ECHO OFF
|
||||||
|
|
||||||
|
prompt ##### completed at time #####
|
||||||
|
select systimestamp from dual;
|
||||||
|
|
||||||
|
SPOOL OFF
|
||||||
|
EXIT
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
SET DEFINE OFF
|
||||||
|
|
||||||
|
CREATE TABLE BKP_MRDS.CT_MRDS_A_TASK_HISTORY_MARS772 AS SELECT * FROM CT_MRDS.A_TASK_HISTORY;
|
||||||
|
|
||||||
|
SET DEFINE ON
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
SET DEFINE OFF
|
||||||
|
|
||||||
|
CREATE TABLE BKP_MRDS.CT_MRDS_A_TASK_HISTORY_SOURCE_MARS772 AS SELECT * FROM CT_MRDS.A_TASK_HISTORY_SOURCE;
|
||||||
|
|
||||||
|
SET DEFINE ON
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
SET DEFINE OFF
|
||||||
|
|
||||||
|
CREATE TABLE BKP_MRDS.CT_MRDS_A_TASK_HISTORY_TARGET_MARS772 AS SELECT * FROM CT_MRDS.A_TASK_HISTORY_TARGET;
|
||||||
|
|
||||||
|
SET DEFINE ON
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
SET DEFINE OFF
|
||||||
|
|
||||||
|
CREATE TABLE BKP_MRDS.CT_MRDS_A_WORKFLOW_HISTORY_MARS772 AS SELECT * FROM CT_MRDS.A_WORKFLOW_HISTORY;
|
||||||
|
|
||||||
|
SET DEFINE ON
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
-- MOPDB
|
||||||
|
INSERT INTO CT_MRDS.A_TASK_HISTORY (
|
||||||
|
A_TASK_HISTORY_KEY,
|
||||||
|
A_WORKFLOW_HISTORY_KEY,
|
||||||
|
TASK_RUN_ID,
|
||||||
|
TASK_NAME,
|
||||||
|
TASK_START,
|
||||||
|
TASK_END,
|
||||||
|
TASK_SUCCESSFUL,
|
||||||
|
SERVICE_NAME
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
A_LOAD_SUBPROCESS_KEY AS A_TASK_HISTORY_KEY,
|
||||||
|
A_MOPDB_LOAD_SET_FK AS A_WORKFLOW_HISTORY_KEY,
|
||||||
|
CAST(NULL AS VARCHAR2(200)) AS TASK_RUN_ID,
|
||||||
|
CASE
|
||||||
|
WHEN SUBPROCESS_NAME = 'm_MOPDB_LM_T_STANDING_FACILITIES_OU_LM_STANDING_FACILITY_LOAD' THEN 'm_MOPDB_LM_T_STANDING_FACILITIES_OU_LM_STANDING_FACILITIES_LOAD'
|
||||||
|
ELSE SUBPROCESS_NAME
|
||||||
|
END AS TASK_NAME,
|
||||||
|
PROCESS_START AS TASK_START,
|
||||||
|
PROCESS_END AS TASK_END,
|
||||||
|
PROCESS_SUCCESSFUL AS TASK_SUCCESSFUL,
|
||||||
|
'MOPDB' AS SERVICE_NAME
|
||||||
|
FROM CT_MOPDB.A_LOAD_SUBPROCESS
|
||||||
|
WHERE
|
||||||
|
SUBPROCESS_NAME IN (
|
||||||
|
SELECT DISTINCT SUBPROCESS_NAME
|
||||||
|
FROM CT_MOPDB.A_LOAD_HISTORY A
|
||||||
|
LEFT JOIN CT_MOPDB.A_LOAD_SUBPROCESS B
|
||||||
|
ON A.A_MOPDB_LOAD_SET_KEY = b.A_MOPDB_LOAD_SET_FK
|
||||||
|
WHERE
|
||||||
|
WORKFLOW_NAME IN (
|
||||||
|
'w_MOPDB_LM_ADHOC_ADJUSTMENT',
|
||||||
|
'w_MOPDB_LM_BALANCESHEET',
|
||||||
|
'w_MOPDB_LM_CSM_ADJUSTMENT',
|
||||||
|
'w_MOPDB_LM_STANDING_FACILITY',
|
||||||
|
'w_MOPDB_LM_CURRENT_ACCOUNTS',
|
||||||
|
'w_MOPDB_LM_FORECAST',
|
||||||
|
'w_MOPDB_LM_QRE_ADJUSTMENT',
|
||||||
|
'w_MOPDB_LM_TTS'
|
||||||
|
)
|
||||||
|
);
|
||||||
|
|
||||||
|
COMMIT;
|
||||||
|
|
||||||
@@ -0,0 +1,60 @@
|
|||||||
|
-- MOPDB
|
||||||
|
INSERT INTO CT_MRDS.A_TASK_HISTORY_SOURCE (
|
||||||
|
A_TASK_HISTORY_SOURCE_KEY,
|
||||||
|
A_TASK_HISTORY_KEY,
|
||||||
|
A_WORKFLOW_HISTORY_SOURCE_KEY,
|
||||||
|
SOURCE_NAME,
|
||||||
|
ROW_COUNT,
|
||||||
|
SERVICE_NAME
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
A_LOAD_HISTORY_SOURCE_KEY AS A_TASK_HISTORY_SOURCE_KEY,
|
||||||
|
A_LOAD_SUBPROCESS_FK AS A_TASK_HISTORY_KEY,
|
||||||
|
A_ODS_LOAD_SET_FK AS A_WORKFLOW_HISTORY_SOURCE_KEY,
|
||||||
|
CASE
|
||||||
|
WHEN SOURCE_TABLE_NAME = 'OU_LM_ADHOC_ADJ_HEADER' THEN 'OU_LM_ADHOC_ADJUSTMENTS_HEADER'
|
||||||
|
WHEN SOURCE_TABLE_NAME = 'OU_LM_ADHOC_ADJ_ITEM' THEN 'OU_LM_ADHOC_ADJUSTMENTS_ITEM'
|
||||||
|
WHEN SOURCE_TABLE_NAME = 'OU_LM_ADHOC_ADJ_ITEM_HEADER' THEN 'OU_LM_ADHOC_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
WHEN SOURCE_TABLE_NAME = 'OU_LM_ADHOC_ADJ_ITEM_HEADER' THEN 'OU_LM_ADHOC_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
WHEN SOURCE_TABLE_NAME = 'OU_LM_CSM_ADJ_HEADER' THEN 'OU_LM_CSM_ADJUSTMENTS_HEADER'
|
||||||
|
WHEN SOURCE_TABLE_NAME = 'OU_LM_CSM_ADJ_ITEM' THEN 'OU_LM_CSM_ADJUSTMENTS_ITEM'
|
||||||
|
WHEN SOURCE_TABLE_NAME = 'OU_LM_CSM_ADJ_ITEM_HEADER' THEN 'OU_LM_CSM_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
WHEN SOURCE_TABLE_NAME = 'OU_LM_STANDING_FACILITY' THEN 'OU_LM_STANDING_FACILITIES'
|
||||||
|
WHEN SOURCE_TABLE_NAME = 'OU_LM_STANDING_FACILITY_HEADER' THEN 'OU_LM_STANDING_FACILITIES_HEADER'
|
||||||
|
WHEN SOURCE_TABLE_NAME = 'OU_MRR_MRR_IND_CURR_ACC_HEADER' THEN 'OU_LM_CURRENT_ACCOUNTS_HEADER'
|
||||||
|
WHEN SOURCE_TABLE_NAME = 'OU_MRR_MRR_IND_CURR_ACC_ITEM' THEN 'OU_LM_CURRENT_ACCOUNTS_ITEM'
|
||||||
|
WHEN SOURCE_TABLE_NAME = 'OU_LM_QR_ADJ_HEADER' THEN 'OU_LM_QRE_ADJUSTMENTS_HEADER'
|
||||||
|
WHEN SOURCE_TABLE_NAME = 'OU_LM_QR_ADJ_ITEM' THEN 'OU_LM_QRE_ADJUSTMENTS_ITEM'
|
||||||
|
WHEN SOURCE_TABLE_NAME = 'OU_LM_QR_ADJ_ITEM_HEADER' THEN 'OU_LM_QRE_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
ELSE SOURCE_TABLE_NAME
|
||||||
|
END AS SOURCE_NAME,
|
||||||
|
ROW_COUNT AS ROW_COUNT,
|
||||||
|
'MOPDB' AS SERVICE_NAME
|
||||||
|
FROM
|
||||||
|
CT_MOPDB.A_LOAD_HISTORY_SOURCE
|
||||||
|
WHERE
|
||||||
|
SOURCE_TABLE_NAME IN (
|
||||||
|
'OU_LM_ADHOC_ADJ_HEADER',
|
||||||
|
'OU_LM_ADHOC_ADJ_ITEM',
|
||||||
|
'OU_LM_ADHOC_ADJ_ITEM_HEADER',
|
||||||
|
'OU_LM_BALANCESHEET_HEADER',
|
||||||
|
'OU_LM_BALANCESHEET_ITEM',
|
||||||
|
'OU_LM_CSM_ADJ_HEADER',
|
||||||
|
'OU_LM_CSM_ADJ_ITEM',
|
||||||
|
'OU_LM_CSM_ADJ_ITEM_HEADER',
|
||||||
|
'OU_LM_STANDING_FACILITY',
|
||||||
|
'OU_LM_STANDING_FACILITY_HEADER',
|
||||||
|
'OU_MRR_MRR_IND_CURR_ACC_HEADER',
|
||||||
|
'OU_MRR_MRR_IND_CURR_ACC_ITEM',
|
||||||
|
'OU_LM_FORECAST_HEADER',
|
||||||
|
'OU_LM_FORECAST_ITEM',
|
||||||
|
'OU_LM_QR_ADJ_HEADER',
|
||||||
|
'OU_LM_QR_ADJ_ITEM',
|
||||||
|
'OU_LM_QR_ADJ_ITEM_HEADER',
|
||||||
|
'OU_LM_TTS_HEADER',
|
||||||
|
'OU_LM_TTS_ITEM'
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
COMMIT;
|
||||||
|
|
||||||
@@ -0,0 +1,111 @@
|
|||||||
|
-- MOPDB
|
||||||
|
INSERT INTO CT_MRDS.A_TASK_HISTORY_TARGET (
|
||||||
|
A_TASK_HISTORY_TARGET_KEY,
|
||||||
|
A_TASK_HISTORY_KEY,
|
||||||
|
TARGET_NAME,
|
||||||
|
ROW_COUNT_APPLIED,
|
||||||
|
ROW_COUNT_REJECTED,
|
||||||
|
LOAD_SUCCESSFUL,
|
||||||
|
SERVICE_NAME
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
A_LOAD_HISTORY_TARGET_KEY AS A_TASK_HISTORY_TARGET_KEY,
|
||||||
|
A_LOAD_SUBPROCESS_FK AS A_TASK_HISTORY_KEY,
|
||||||
|
TARGET_TABLE_NAME AS TARGET_NAME,
|
||||||
|
ROW_COUNT_APPLIED AS ROW_COUNT_APPLIED,
|
||||||
|
ROW_COUNT_REJECTED AS ROW_COUNT_REJECTED,
|
||||||
|
LOAD_SUCCESSFUL AS LOAD_SUCCESSFUL,
|
||||||
|
'MOPDB' AS SERVICE_NAME
|
||||||
|
FROM
|
||||||
|
CT_MOPDB.A_LOAD_HISTORY_TARGET
|
||||||
|
WHERE
|
||||||
|
TARGET_TABLE_NAME IN (
|
||||||
|
'LM_T_FC_ADH_ADJUSTMENTS',
|
||||||
|
'LM_T_BALANCE_SHEET',
|
||||||
|
'LM_T_FC_CSM_ADJUSTMENTS',
|
||||||
|
'LM_T_STANDING_FACILITIES',
|
||||||
|
'LM_T_CURRENT_ACCOUNTS',
|
||||||
|
'LM_T_FORECAST',
|
||||||
|
'LM_T_FC_QRE_ADJUSTMENTS',
|
||||||
|
'LM_T_TTS'
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
-- ODS
|
||||||
|
INSERT INTO CT_MRDS.A_TASK_HISTORY_TARGET (
|
||||||
|
A_TASK_HISTORY_TARGET_KEY,
|
||||||
|
A_TASK_HISTORY_KEY,
|
||||||
|
TARGET_NAME,
|
||||||
|
ROW_COUNT_APPLIED,
|
||||||
|
ROW_COUNT_REJECTED,
|
||||||
|
LOAD_SUCCESSFUL,
|
||||||
|
SERVICE_NAME
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
A_LOAD_HISTORY_TARGET_KEY AS A_TASK_HISTORY_TARGET_KEY,
|
||||||
|
A_ETL_LOAD_SET_FK AS A_TASK_HISTORY_KEY,
|
||||||
|
CASE
|
||||||
|
WHEN TARGET_TABLE_NAME = 'OU_LM_ADHOC_ADJ_HEADER' THEN 'OU_LM_ADHOC_ADJUSTMENTS_HEADER'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'OU_LM_ADHOC_ADJ_ITEM' THEN 'OU_LM_ADHOC_ADJUSTMENTS_ITEM'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'OU_LM_ADHOC_ADJ_ITEM_HEADER' THEN 'OU_LM_ADHOC_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'OU_LM_ADHOC_ADJ_ITEM_HEADER' THEN 'OU_LM_ADHOC_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'OU_LM_CSM_ADJ_HEADER' THEN 'OU_LM_CSM_ADJUSTMENTS_HEADER'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'CSM_ADJ_HEADER' THEN 'OU_LM_CSM_ADJUSTMENTS_HEADER'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'OU_LM_CSM_ADJ_ITEM' THEN 'OU_LM_CSM_ADJUSTMENTS_ITEM'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'CSM_ADJ_ITEM' THEN 'OU_LM_CSM_ADJUSTMENTS_ITEM'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'OU_LM_CSM_ADJ_ITEM_HEADER' THEN 'OU_LM_CSM_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'CSM_ADJ_ITEM_HEADER' THEN 'OU_LM_CSM_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'OU_LM_STANDING_FACILITY' THEN 'OU_LM_STANDING_FACILITIES'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'STANDING_FACILITY' THEN 'OU_LM_STANDING_FACILITIES'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'OU_LM_STANDING_FACILITY_HEADER' THEN 'OU_LM_STANDING_FACILITIES_HEADER'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'STANDING_FACILITY_HEADER' THEN 'OU_LM_STANDING_FACILITIES_HEADER'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'OU_MRR_MRR_IND_CURR_ACC_HEADER' THEN 'OU_LM_CURRENT_ACCOUNTS_HEADER'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'OU_MRR_MRR_IND_CURR_ACC_ITEM' THEN 'OU_LM_CURRENT_ACCOUNTS_ITEM'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'OU_LM_QR_ADJ_HEADER' THEN 'OU_LM_QRE_ADJUSTMENTS_HEADER'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'QR_ADJ_HEADER' THEN 'OU_LM_QRE_ADJUSTMENTS_HEADER'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'OU_LM_QR_ADJ_ITEM' THEN 'OU_LM_QRE_ADJUSTMENTS_ITEM'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'QR_ADJ_ITEM' THEN 'OU_LM_QRE_ADJUSTMENTS_ITEM'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'OU_LM_QR_ADJ_ITEM_HEADER' THEN 'OU_LM_QRE_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
WHEN TARGET_TABLE_NAME = 'QR_ADJ_ITEM_HEADER' THEN 'OU_LM_QRE_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
ELSE TARGET_TABLE_NAME
|
||||||
|
END AS TARGET_NAME,
|
||||||
|
ROW_COUNT_APPLIED AS ROW_COUNT_APPLIED,
|
||||||
|
ROW_COUNT_REJECTED AS ROW_COUNT_REJECTED,
|
||||||
|
'Y' AS LOAD_SUCCESSFUL,
|
||||||
|
'ODS' AS SERVICE_NAME
|
||||||
|
FROM
|
||||||
|
CT_ODS.A_LOAD_HISTORY_TARGET
|
||||||
|
WHERE
|
||||||
|
TARGET_TABLE_NAME IN (
|
||||||
|
'OU_LM_ADHOC_ADJ_HEADER',
|
||||||
|
'OU_LM_ADHOC_ADJ_ITEM',
|
||||||
|
'OU_LM_ADHOC_ADJ_ITEM_HEADER',
|
||||||
|
'OU_LM_BALANCESHEET_HEADER',
|
||||||
|
'OU_LM_BALANCESHEET_ITEM',
|
||||||
|
'OU_LM_CSM_ADJ_HEADER',
|
||||||
|
'OU_LM_CSM_ADJ_ITEM',
|
||||||
|
'OU_LM_CSM_ADJ_ITEM_HEADER',
|
||||||
|
'OU_LM_STANDING_FACILITY',
|
||||||
|
'OU_LM_STANDING_FACILITY_HEADER',
|
||||||
|
'OU_MRR_MRR_IND_CURR_ACC_HEADER',
|
||||||
|
'OU_MRR_MRR_IND_CURR_ACC_ITEM',
|
||||||
|
'OU_LM_FORECAST_HEADER',
|
||||||
|
'OU_LM_FORECAST_ITEM',
|
||||||
|
'OU_LM_QR_ADJ_HEADER',
|
||||||
|
'OU_LM_QR_ADJ_ITEM',
|
||||||
|
'OU_LM_QR_ADJ_ITEM_HEADER',
|
||||||
|
'OU_LM_TTS_HEADER',
|
||||||
|
'OU_LM_TTS_ITEM',
|
||||||
|
'CSM_ADJ_HEADER',
|
||||||
|
'CSM_ADJ_ITEM',
|
||||||
|
'CSM_ADJ_ITEM_HEADER',
|
||||||
|
'QR_ADJ_HEADER',
|
||||||
|
'QR_ADJ_ITEM',
|
||||||
|
'QR_ADJ_ITEM_HEADER',
|
||||||
|
'STANDING_FACILITY',
|
||||||
|
'STANDING_FACILITY_HEADER'
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
COMMIT;
|
||||||
|
|
||||||
@@ -0,0 +1,82 @@
|
|||||||
|
-- MOPDB
|
||||||
|
INSERT INTO CT_MRDS.A_WORKFLOW_HISTORY (
|
||||||
|
A_WORKFLOW_HISTORY_KEY,
|
||||||
|
ORCHESTRATION_RUN_ID,
|
||||||
|
WORKFLOW_NAME,
|
||||||
|
WORKFLOW_START,
|
||||||
|
WORKFLOW_END,
|
||||||
|
WORKFLOW_SUCCESSFUL,
|
||||||
|
DBT_INVOCATION_ID,
|
||||||
|
SERVICE_NAME
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
A_MOPDB_LOAD_SET_KEY AS A_WORKFLOW_HISTORY_KEY,
|
||||||
|
CAST(WLA_RUN_ID AS VARCHAR2(200)) AS ORCHESTRATION_RUN_ID,
|
||||||
|
CASE
|
||||||
|
WHEN WORKFLOW_NAME = 'w_MOPDB_LM_STANDING_FACILITY' THEN 'w_MOPDB_LM_STANDING_FACILITIES'
|
||||||
|
ELSE WORKFLOW_NAME
|
||||||
|
END AS WORKFLOW_NAME,
|
||||||
|
LOAD_START AS WORKFLOW_START,
|
||||||
|
LOAD_END AS WORKFLOW_END,
|
||||||
|
'Y' AS WORKFLOW_SUCCESSFUL,
|
||||||
|
CAST(INFA_RUN_ID AS VARCHAR2(200)) AS DBT_INVOCATION_ID,
|
||||||
|
'MOPDB' AS SERVICE_NAME
|
||||||
|
FROM
|
||||||
|
CT_MOPDB.A_LOAD_HISTORY
|
||||||
|
WHERE
|
||||||
|
WORKFLOW_NAME IN (
|
||||||
|
'w_MOPDB_LM_ADHOC_ADJUSTMENT',
|
||||||
|
'w_MOPDB_LM_BALANCESHEET',
|
||||||
|
'w_MOPDB_LM_CSM_ADJUSTMENT',
|
||||||
|
'w_MOPDB_LM_STANDING_FACILITY',
|
||||||
|
'w_MOPDB_LM_CURRENT_ACCOUNTS',
|
||||||
|
'w_MOPDB_LM_FORECAST',
|
||||||
|
'w_MOPDB_LM_QRE_ADJUSTMENT',
|
||||||
|
'w_MOPDB_LM_TTS'
|
||||||
|
)
|
||||||
|
;
|
||||||
|
|
||||||
|
-- ODS
|
||||||
|
INSERT INTO CT_MRDS.A_WORKFLOW_HISTORY (
|
||||||
|
A_WORKFLOW_HISTORY_KEY,
|
||||||
|
ORCHESTRATION_RUN_ID,
|
||||||
|
WORKFLOW_NAME,
|
||||||
|
WORKFLOW_START,
|
||||||
|
WORKFLOW_END,
|
||||||
|
WORKFLOW_SUCCESSFUL,
|
||||||
|
DBT_INVOCATION_ID,
|
||||||
|
SERVICE_NAME
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
A_ETL_LOAD_SET_KEY AS A_WORKFLOW_HISTORY_KEY,
|
||||||
|
CASE
|
||||||
|
WHEN WLA_RUN_ID IS NOT NULL THEN CAST(WLA_RUN_ID AS VARCHAR2(200))
|
||||||
|
ELSE EXDI_CORRELATION_ID
|
||||||
|
END AS ORCHESTRATION_RUN_ID,
|
||||||
|
CASE
|
||||||
|
WHEN WORKFLOW_NAME = 'w_ODS_LM_STANDING_FACILITY' THEN 'w_ODS_LM_STANDING_FACILITIES'
|
||||||
|
WHEN WORKFLOW_NAME = 'w_ODS_MRR_IND_CURRENT_ACCOUNT_ITEM' THEN 'w_ODS_LM_CURRENT_ACCOUNT_ITEM'
|
||||||
|
WHEN WORKFLOW_NAME = 'w_ODS_QUARTERLY_ADJUSTMENT_MSG' THEN 'w_ODS_LM_QUARTERLY_ADJUSTMENT_MSG'
|
||||||
|
WHEN WORKFLOW_NAME = 'w_ODS_ADHOC_ADJUSTMENT_MSG' THEN 'w_ODS_LM_ADHOC_ADJUSTMENTS_MSG'
|
||||||
|
WHEN WORKFLOW_NAME = 'w_ODS_CSM_ADJUSTMENT_MSG' THEN 'w_ODS_LM_CSM_ADJUSTMENT_MSG'
|
||||||
|
ELSE WORKFLOW_NAME
|
||||||
|
END AS WORKFLOW_NAME,
|
||||||
|
LOAD_START AS WORKFLOW_START,
|
||||||
|
LOAD_END AS WORKFLOW_END,
|
||||||
|
LOAD_SUCCESSFUL AS WORKFLOW_SUCCESSFUL,
|
||||||
|
CAST(INFA_RUN_ID AS VARCHAR2(200)) AS DBT_INVOCATION_ID,
|
||||||
|
'ODS' AS SERVICE_NAME
|
||||||
|
FROM CT_ODS.A_LOAD_HISTORY
|
||||||
|
WHERE
|
||||||
|
WORKFLOW_NAME IN (
|
||||||
|
'w_ODS_ADHOC_ADJUSTMENT_MSG',
|
||||||
|
'w_ODS_LM_BALANCESHEET',
|
||||||
|
'w_ODS_CSM_ADJUSTMENT_MSG',
|
||||||
|
'w_ODS_LM_STANDING_FACILITY',
|
||||||
|
'w_ODS_MRR_IND_CURRENT_ACCOUNT_ITEM',
|
||||||
|
'w_ODS_QUARTERLY_ADJUSTMENT_MSG'
|
||||||
|
)
|
||||||
|
;
|
||||||
|
|
||||||
|
COMMIT;
|
||||||
|
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
-- MOPDB
|
||||||
|
DELETE FROM CT_MRDS.A_TASK_HISTORY t WHERE EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM CT_MOPDB.A_LOAD_SUBPROCESS r
|
||||||
|
WHERE
|
||||||
|
r.SUBPROCESS_NAME IN (
|
||||||
|
SELECT DISTINCT SUBPROCESS_NAME
|
||||||
|
FROM CT_MOPDB.A_LOAD_HISTORY A
|
||||||
|
LEFT JOIN CT_MOPDB.A_LOAD_SUBPROCESS B
|
||||||
|
ON A.A_MOPDB_LOAD_SET_KEY = b.A_MOPDB_LOAD_SET_FK
|
||||||
|
WHERE
|
||||||
|
WORKFLOW_NAME IN (
|
||||||
|
'w_MOPDB_LM_ADHOC_ADJUSTMENT',
|
||||||
|
'w_MOPDB_LM_BALANCESHEET',
|
||||||
|
'w_MOPDB_LM_CSM_ADJUSTMENT',
|
||||||
|
'w_MOPDB_LM_STANDING_FACILITY',
|
||||||
|
'w_MOPDB_LM_CURRENT_ACCOUNTS',
|
||||||
|
'w_MOPDB_LM_FORECAST',
|
||||||
|
'w_MOPDB_LM_QRE_ADJUSTMENT',
|
||||||
|
'w_MOPDB_LM_TTS'
|
||||||
|
)
|
||||||
|
)
|
||||||
|
AND t.A_TASK_HISTORY_KEY = r.A_LOAD_SUBPROCESS_KEY
|
||||||
|
AND t.A_WORKFLOW_HISTORY_KEY = r.A_MOPDB_LOAD_SET_FK
|
||||||
|
AND NVL(t.TASK_RUN_ID,' ') = NVL(CAST(NULL AS VARCHAR2(200)),' ')
|
||||||
|
AND NVL(t.TASK_NAME,' ') = NVL(CASE
|
||||||
|
WHEN r.SUBPROCESS_NAME = 'm_MOPDB_LM_T_STANDING_FACILITIES_OU_LM_STANDING_FACILITY_LOAD' THEN 'm_MOPDB_LM_T_STANDING_FACILITIES_OU_LM_STANDING_FACILITIES_LOAD'
|
||||||
|
ELSE r.SUBPROCESS_NAME
|
||||||
|
END,' ')
|
||||||
|
AND NVL(t.TASK_START,SYSDATE) = NVL(r.PROCESS_START,SYSDATE)
|
||||||
|
AND NVL(t.TASK_END,SYSDATE) = NVL(r.PROCESS_END,SYSDATE)
|
||||||
|
AND NVL(t.TASK_SUCCESSFUL,' ')= NVL(r.PROCESS_SUCCESSFUL,' ')
|
||||||
|
AND t.SERVICE_NAME = 'MOPDB'
|
||||||
|
);
|
||||||
|
|
||||||
|
COMMIT;
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
-- MOPDB
|
||||||
|
DELETE FROM CT_MRDS.A_TASK_HISTORY_SOURCE t WHERE EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM
|
||||||
|
CT_MOPDB.A_LOAD_HISTORY_SOURCE r
|
||||||
|
WHERE
|
||||||
|
r.SOURCE_TABLE_NAME IN (
|
||||||
|
'OU_LM_ADHOC_ADJ_HEADER',
|
||||||
|
'OU_LM_ADHOC_ADJ_ITEM',
|
||||||
|
'OU_LM_ADHOC_ADJ_ITEM_HEADER',
|
||||||
|
'OU_LM_BALANCESHEET_HEADER',
|
||||||
|
'OU_LM_BALANCESHEET_ITEM',
|
||||||
|
'OU_LM_CSM_ADJ_HEADER',
|
||||||
|
'OU_LM_CSM_ADJ_ITEM',
|
||||||
|
'OU_LM_CSM_ADJ_ITEM_HEADER',
|
||||||
|
'OU_LM_STANDING_FACILITY',
|
||||||
|
'OU_LM_STANDING_FACILITY_HEADER',
|
||||||
|
'OU_MRR_MRR_IND_CURR_ACC_HEADER',
|
||||||
|
'OU_MRR_MRR_IND_CURR_ACC_ITEM',
|
||||||
|
'OU_LM_FORECAST_HEADER',
|
||||||
|
'OU_LM_FORECAST_ITEM',
|
||||||
|
'OU_LM_QR_ADJ_HEADER',
|
||||||
|
'OU_LM_QR_ADJ_ITEM',
|
||||||
|
'OU_LM_QR_ADJ_ITEM_HEADER',
|
||||||
|
'OU_LM_TTS_HEADER',
|
||||||
|
'OU_LM_TTS_ITEM'
|
||||||
|
)
|
||||||
|
AND t.A_TASK_HISTORY_SOURCE_KEY = r.A_LOAD_HISTORY_SOURCE_KEY
|
||||||
|
AND t.A_TASK_HISTORY_KEY = r.A_LOAD_SUBPROCESS_FK
|
||||||
|
AND NVL(t.A_WORKFLOW_HISTORY_SOURCE_KEY,0) = NVL(r.A_ODS_LOAD_SET_FK,0)
|
||||||
|
AND NVL(t.SOURCE_NAME,' ') = NVL(CASE
|
||||||
|
WHEN r.SOURCE_TABLE_NAME = 'OU_LM_ADHOC_ADJ_HEADER' THEN 'OU_LM_ADHOC_ADJUSTMENTS_HEADER'
|
||||||
|
WHEN r.SOURCE_TABLE_NAME = 'OU_LM_ADHOC_ADJ_ITEM' THEN 'OU_LM_ADHOC_ADJUSTMENTS_ITEM'
|
||||||
|
WHEN r.SOURCE_TABLE_NAME = 'OU_LM_ADHOC_ADJ_ITEM_HEADER' THEN 'OU_LM_ADHOC_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
WHEN r.SOURCE_TABLE_NAME = 'OU_LM_ADHOC_ADJ_ITEM_HEADER' THEN 'OU_LM_ADHOC_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
WHEN r.SOURCE_TABLE_NAME = 'OU_LM_CSM_ADJ_HEADER' THEN 'OU_LM_CSM_ADJUSTMENTS_HEADER'
|
||||||
|
WHEN r.SOURCE_TABLE_NAME = 'OU_LM_CSM_ADJ_ITEM' THEN 'OU_LM_CSM_ADJUSTMENTS_ITEM'
|
||||||
|
WHEN r.SOURCE_TABLE_NAME = 'OU_LM_CSM_ADJ_ITEM_HEADER' THEN 'OU_LM_CSM_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
WHEN r.SOURCE_TABLE_NAME = 'OU_LM_STANDING_FACILITY' THEN 'OU_LM_STANDING_FACILITIES'
|
||||||
|
WHEN r.SOURCE_TABLE_NAME = 'OU_LM_STANDING_FACILITY_HEADER' THEN 'OU_LM_STANDING_FACILITIES_HEADER'
|
||||||
|
WHEN r.SOURCE_TABLE_NAME = 'OU_MRR_MRR_IND_CURR_ACC_HEADER' THEN 'OU_LM_CURRENT_ACCOUNTS_HEADER'
|
||||||
|
WHEN r.SOURCE_TABLE_NAME = 'OU_MRR_MRR_IND_CURR_ACC_ITEM' THEN 'OU_LM_CURRENT_ACCOUNTS_ITEM'
|
||||||
|
WHEN r.SOURCE_TABLE_NAME = 'OU_LM_QR_ADJ_HEADER' THEN 'OU_LM_QRE_ADJUSTMENTS_HEADER'
|
||||||
|
WHEN r.SOURCE_TABLE_NAME = 'OU_LM_QR_ADJ_ITEM' THEN 'OU_LM_QRE_ADJUSTMENTS_ITEM'
|
||||||
|
WHEN r.SOURCE_TABLE_NAME = 'OU_LM_QR_ADJ_ITEM_HEADER' THEN 'OU_LM_QRE_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
ELSE r.SOURCE_TABLE_NAME
|
||||||
|
END,' ')
|
||||||
|
AND NVL(t.ROW_COUNT,0) = NVL(r.ROW_COUNT,0)
|
||||||
|
AND t.SERVICE_NAME = 'MOPDB'
|
||||||
|
)
|
||||||
|
;
|
||||||
|
|
||||||
|
COMMIT;
|
||||||
@@ -0,0 +1,95 @@
|
|||||||
|
-- MOPDB
|
||||||
|
DELETE FROM CT_MRDS.A_TASK_HISTORY_TARGET t WHERE EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM
|
||||||
|
CT_MOPDB.A_LOAD_HISTORY_TARGET r
|
||||||
|
WHERE
|
||||||
|
TARGET_TABLE_NAME IN (
|
||||||
|
'LM_T_FC_ADH_ADJUSTMENTS',
|
||||||
|
'LM_T_BALANCE_SHEET',
|
||||||
|
'LM_T_FC_CSM_ADJUSTMENTS',
|
||||||
|
'LM_T_STANDING_FACILITIES',
|
||||||
|
'LM_T_CURRENT_ACCOUNTS',
|
||||||
|
'LM_T_FORECAST',
|
||||||
|
'LM_T_FC_QRE_ADJUSTMENTS',
|
||||||
|
'LM_T_TTS'
|
||||||
|
)
|
||||||
|
AND t.A_TASK_HISTORY_TARGET_KEY = r.A_LOAD_HISTORY_TARGET_KEY
|
||||||
|
AND t.A_TASK_HISTORY_KEY = r.A_LOAD_SUBPROCESS_FK
|
||||||
|
AND NVL(t.TARGET_NAME,' ') = NVL(r.TARGET_TABLE_NAME,' ')
|
||||||
|
AND NVL(t.ROW_COUNT_APPLIED,0) = NVL(r.ROW_COUNT_APPLIED,0)
|
||||||
|
AND NVL(t.ROW_COUNT_REJECTED,0) = NVL(r.ROW_COUNT_REJECTED,0)
|
||||||
|
AND NVL(t.LOAD_SUCCESSFUL,' ') = NVL(r.LOAD_SUCCESSFUL,' ')
|
||||||
|
AND t.SERVICE_NAME = 'MOPDB'
|
||||||
|
);
|
||||||
|
|
||||||
|
-- ODS
|
||||||
|
DELETE FROM CT_MRDS.A_TASK_HISTORY_TARGET t WHERE EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM
|
||||||
|
CT_ODS.A_LOAD_HISTORY_TARGET r
|
||||||
|
WHERE
|
||||||
|
TARGET_TABLE_NAME IN (
|
||||||
|
'OU_LM_ADHOC_ADJ_HEADER',
|
||||||
|
'OU_LM_ADHOC_ADJ_ITEM',
|
||||||
|
'OU_LM_ADHOC_ADJ_ITEM_HEADER',
|
||||||
|
'OU_LM_BALANCESHEET_HEADER',
|
||||||
|
'OU_LM_BALANCESHEET_ITEM',
|
||||||
|
'OU_LM_CSM_ADJ_HEADER',
|
||||||
|
'OU_LM_CSM_ADJ_ITEM',
|
||||||
|
'OU_LM_CSM_ADJ_ITEM_HEADER',
|
||||||
|
'OU_LM_STANDING_FACILITY',
|
||||||
|
'OU_LM_STANDING_FACILITY_HEADER',
|
||||||
|
'OU_MRR_MRR_IND_CURR_ACC_HEADER',
|
||||||
|
'OU_MRR_MRR_IND_CURR_ACC_ITEM',
|
||||||
|
'OU_LM_FORECAST_HEADER',
|
||||||
|
'OU_LM_FORECAST_ITEM',
|
||||||
|
'OU_LM_QR_ADJ_HEADER',
|
||||||
|
'OU_LM_QR_ADJ_ITEM',
|
||||||
|
'OU_LM_QR_ADJ_ITEM_HEADER',
|
||||||
|
'OU_LM_TTS_HEADER',
|
||||||
|
'OU_LM_TTS_ITEM',
|
||||||
|
'CSM_ADJ_HEADER',
|
||||||
|
'CSM_ADJ_ITEM',
|
||||||
|
'CSM_ADJ_ITEM_HEADER',
|
||||||
|
'QR_ADJ_HEADER',
|
||||||
|
'QR_ADJ_ITEM',
|
||||||
|
'QR_ADJ_ITEM_HEADER',
|
||||||
|
'STANDING_FACILITY',
|
||||||
|
'STANDING_FACILITY_HEADER'
|
||||||
|
)
|
||||||
|
AND t.A_TASK_HISTORY_TARGET_KEY = r.A_LOAD_HISTORY_TARGET_KEY
|
||||||
|
AND t.A_TASK_HISTORY_KEY = r.A_ETL_LOAD_SET_FK
|
||||||
|
AND NVL(t.TARGET_NAME,' ') = NVL(CASE
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'OU_LM_ADHOC_ADJ_HEADER' THEN 'OU_LM_ADHOC_ADJUSTMENTS_HEADER'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'OU_LM_ADHOC_ADJ_ITEM' THEN 'OU_LM_ADHOC_ADJUSTMENTS_ITEM'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'OU_LM_ADHOC_ADJ_ITEM_HEADER' THEN 'OU_LM_ADHOC_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'OU_LM_ADHOC_ADJ_ITEM_HEADER' THEN 'OU_LM_ADHOC_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'OU_LM_CSM_ADJ_HEADER' THEN 'OU_LM_CSM_ADJUSTMENTS_HEADER'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'CSM_ADJ_HEADER' THEN 'OU_LM_CSM_ADJUSTMENTS_HEADER'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'OU_LM_CSM_ADJ_ITEM' THEN 'OU_LM_CSM_ADJUSTMENTS_ITEM'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'CSM_ADJ_ITEM' THEN 'OU_LM_CSM_ADJUSTMENTS_ITEM'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'OU_LM_CSM_ADJ_ITEM_HEADER' THEN 'OU_LM_CSM_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'CSM_ADJ_ITEM_HEADER' THEN 'OU_LM_CSM_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'OU_LM_STANDING_FACILITY' THEN 'OU_LM_STANDING_FACILITIES'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'STANDING_FACILITY' THEN 'OU_LM_STANDING_FACILITIES'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'OU_LM_STANDING_FACILITY_HEADER' THEN 'OU_LM_STANDING_FACILITIES_HEADER'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'STANDING_FACILITY_HEADER' THEN 'OU_LM_STANDING_FACILITIES_HEADER'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'OU_MRR_MRR_IND_CURR_ACC_HEADER' THEN 'OU_LM_CURRENT_ACCOUNTS_HEADER'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'OU_MRR_MRR_IND_CURR_ACC_ITEM' THEN 'OU_LM_CURRENT_ACCOUNTS_ITEM'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'OU_LM_QR_ADJ_HEADER' THEN 'OU_LM_QRE_ADJUSTMENTS_HEADER'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'QR_ADJ_HEADER' THEN 'OU_LM_QRE_ADJUSTMENTS_HEADER'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'OU_LM_QR_ADJ_ITEM' THEN 'OU_LM_QRE_ADJUSTMENTS_ITEM'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'QR_ADJ_ITEM' THEN 'OU_LM_QRE_ADJUSTMENTS_ITEM'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'OU_LM_QR_ADJ_ITEM_HEADER' THEN 'OU_LM_QRE_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
WHEN r.TARGET_TABLE_NAME = 'QR_ADJ_ITEM_HEADER' THEN 'OU_LM_QRE_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
ELSE r.TARGET_TABLE_NAME
|
||||||
|
END,' ')
|
||||||
|
AND NVL(t.ROW_COUNT_APPLIED,0) = NVL(r.ROW_COUNT_APPLIED,0)
|
||||||
|
AND NVL(t.ROW_COUNT_REJECTED,0) = NVL(r.ROW_COUNT_REJECTED,0)
|
||||||
|
AND NVL(t.LOAD_SUCCESSFUL,' ') = 'Y'
|
||||||
|
AND t.SERVICE_NAME = 'ODS'
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
COMMIT;
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
-- MOPDB
|
||||||
|
DELETE FROM CT_MRDS.A_WORKFLOW_HISTORY t WHERE EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM
|
||||||
|
CT_MOPDB.A_LOAD_HISTORY r
|
||||||
|
WHERE
|
||||||
|
r.WORKFLOW_NAME IN (
|
||||||
|
'w_MOPDB_LM_ADHOC_ADJUSTMENT',
|
||||||
|
'w_MOPDB_LM_BALANCESHEET',
|
||||||
|
'w_MOPDB_LM_CSM_ADJUSTMENT',
|
||||||
|
'w_MOPDB_LM_STANDING_FACILITY',
|
||||||
|
'w_MOPDB_LM_CURRENT_ACCOUNTS',
|
||||||
|
'w_MOPDB_LM_FORECAST',
|
||||||
|
'w_MOPDB_LM_QRE_ADJUSTMENT',
|
||||||
|
'w_MOPDB_LM_TTS'
|
||||||
|
)
|
||||||
|
AND t.A_WORKFLOW_HISTORY_KEY = r.A_MOPDB_LOAD_SET_KEY
|
||||||
|
AND NVL(t.ORCHESTRATION_RUN_ID,' ') = NVL(CAST(r.WLA_RUN_ID AS VARCHAR2(200)),' ')
|
||||||
|
AND t.WORKFLOW_NAME = CASE
|
||||||
|
WHEN r.WORKFLOW_NAME = 'w_MOPDB_LM_STANDING_FACILITY' THEN 'w_MOPDB_LM_STANDING_FACILITIES'
|
||||||
|
ELSE r.WORKFLOW_NAME
|
||||||
|
END
|
||||||
|
AND NVL(t.WORKFLOW_START,SYSDATE) = NVL(r.LOAD_START,SYSDATE)
|
||||||
|
AND NVL(t.WORKFLOW_END,SYSDATE) = NVL(r.LOAD_END,SYSDATE)
|
||||||
|
AND NVL(t.WORKFLOW_SUCCESSFUL,' ') = NVL(CAST(NULL AS VARCHAR2(200)),' ')
|
||||||
|
AND NVL(t.DBT_INVOCATION_ID,' ') = NVL(CAST(r.INFA_RUN_ID AS VARCHAR2(200)),' ')
|
||||||
|
AND t.SERVICE_NAME = 'MOPDB'
|
||||||
|
);
|
||||||
|
|
||||||
|
|
||||||
|
-- ODS
|
||||||
|
DELETE FROM CT_MRDS.A_WORKFLOW_HISTORY t WHERE EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM CT_ODS.A_LOAD_HISTORY r
|
||||||
|
WHERE
|
||||||
|
WORKFLOW_NAME IN (
|
||||||
|
'w_ODS_ADHOC_ADJUSTMENT_MSG',
|
||||||
|
'w_ODS_LM_BALANCESHEET',
|
||||||
|
'w_ODS_CSM_ADJUSTMENT_MSG',
|
||||||
|
'w_ODS_LM_STANDING_FACILITY',
|
||||||
|
'w_ODS_MRR_IND_CURRENT_ACCOUNT_ITEM',
|
||||||
|
'w_ODS_QUARTERLY_ADJUSTMENT_MSG'
|
||||||
|
)
|
||||||
|
AND t.A_WORKFLOW_HISTORY_KEY = r.A_ETL_LOAD_SET_KEY
|
||||||
|
AND NVL(t.ORCHESTRATION_RUN_ID,' ') = NVL( CASE WHEN r.WLA_RUN_ID IS NOT NULL THEN CAST(r.WLA_RUN_ID AS VARCHAR2(200)) ELSE r.EXDI_CORRELATION_ID END,' ')
|
||||||
|
AND t.WORKFLOW_NAME = CASE
|
||||||
|
WHEN r.WORKFLOW_NAME = 'w_ODS_LM_STANDING_FACILITY' THEN 'w_ODS_LM_STANDING_FACILITIES'
|
||||||
|
WHEN r.WORKFLOW_NAME = 'w_ODS_MRR_IND_CURRENT_ACCOUNT_ITEM' THEN 'w_ODS_LM_CURRENT_ACCOUNT_ITEM'
|
||||||
|
WHEN r.WORKFLOW_NAME = 'w_ODS_QUARTERLY_ADJUSTMENT_MSG' THEN 'w_ODS_LM_QUARTERLY_ADJUSTMENT_MSG'
|
||||||
|
ELSE r.WORKFLOW_NAME
|
||||||
|
END
|
||||||
|
AND NVL(t.WORKFLOW_START,SYSDATE) = NVL(r.LOAD_START,SYSDATE)
|
||||||
|
AND NVL(t.WORKFLOW_END,SYSDATE) = NVL(r.LOAD_END,SYSDATE)
|
||||||
|
AND NVL(t.WORKFLOW_SUCCESSFUL,' ') = NVL(r.LOAD_SUCCESSFUL,' ')
|
||||||
|
AND NVL(t.DBT_INVOCATION_ID,' ') = NVL(CAST(r.INFA_RUN_ID AS VARCHAR2(200)),' ')
|
||||||
|
AND t.SERVICE_NAME = 'ODS'
|
||||||
|
);
|
||||||
|
|
||||||
|
COMMIT;
|
||||||
40
MARS_Packages/REL01/MARS-772/install_mars772.sql
Normal file
40
MARS_Packages/REL01/MARS-772/install_mars772.sql
Normal file
@@ -0,0 +1,40 @@
|
|||||||
|
WHENEVER SQLERROR EXIT FAILURE
|
||||||
|
SET SERVEROUTPUT ON
|
||||||
|
SET TIMING ON
|
||||||
|
SET ECHO ON
|
||||||
|
SET HEADING OFF
|
||||||
|
SET FEEDBACK ON
|
||||||
|
SET VERIFY OFF
|
||||||
|
|
||||||
|
var filename VARCHAR2(100)
|
||||||
|
BEGIN
|
||||||
|
SELECT 'INSTALL_MARS_772_' || SUBSTR(PDB_NAME, (INSTR(PDB_NAME,'_',1)+1), (LENGTH(PDB_NAME)-INSTR(PDB_NAME,'_',1))) || '_' ||TO_CHAR(SYSDATE,'YYYYMMDD_HH24MISS')||'.log' INTO :filename from DBA_PDBS;
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
column filename new_value _filename
|
||||||
|
select :filename filename from dual;
|
||||||
|
spool &_filename
|
||||||
|
|
||||||
|
prompt ##### started at time #####
|
||||||
|
select systimestamp from dual;
|
||||||
|
prompt ##### database name #####
|
||||||
|
SELECT SUBSTR(PDB_NAME, (INSTR(PDB_NAME,'_',1)+1), (LENGTH(PDB_NAME)-INSTR(PDB_NAME,'_',1))) AS PDB_NAME FROM DBA_PDBS;
|
||||||
|
|
||||||
|
|
||||||
|
@@00_MARS_772_backup_CT_MRDS_A_WORKFLOW_HISTORY.sql
|
||||||
|
@@00_MARS_772_backup_CT_MRDS_A_TASK_HISTORY.sql
|
||||||
|
@@00_MARS_772_backup_CT_MRDS_A_TASK_HISTORY_SOURCE.sql
|
||||||
|
@@00_MARS_772_backup_CT_MRDS_A_TASK_HISTORY_TARGET.sql
|
||||||
|
@@01_MARS_772_install_CT_MRDS_A_WORKFLOW_HISTORY.sql
|
||||||
|
@@01_MARS_772_install_CT_MRDS_A_TASK_HISTORY.sql
|
||||||
|
@@01_MARS_772_install_CT_MRDS_A_TASK_HISTORY_SOURCE.sql
|
||||||
|
@@01_MARS_772_install_CT_MRDS_A_TASK_HISTORY_TARGET.sql
|
||||||
|
|
||||||
|
|
||||||
|
SET ECHO OFF
|
||||||
|
|
||||||
|
prompt ##### completed at time #####
|
||||||
|
select systimestamp from dual;
|
||||||
|
|
||||||
|
SPOOL OFF
|
||||||
|
EXIT
|
||||||
36
MARS_Packages/REL01/MARS-772/rollback_mars772.sql
Normal file
36
MARS_Packages/REL01/MARS-772/rollback_mars772.sql
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
WHENEVER SQLERROR EXIT FAILURE
|
||||||
|
SET SERVEROUTPUT ON
|
||||||
|
SET TIMING ON
|
||||||
|
SET ECHO ON
|
||||||
|
SET HEADING OFF
|
||||||
|
SET FEEDBACK ON
|
||||||
|
SET VERIFY OFF
|
||||||
|
|
||||||
|
var filename VARCHAR2(100)
|
||||||
|
BEGIN
|
||||||
|
SELECT 'ROLLBACK_MARS_772_' || SUBSTR(PDB_NAME, (INSTR(PDB_NAME,'_',1)+1), (LENGTH(PDB_NAME)-INSTR(PDB_NAME,'_',1))) || '_' ||TO_CHAR(SYSDATE,'YYYYMMDD_HH24MISS')||'.log' INTO :filename from DBA_PDBS;
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
column filename new_value _filename
|
||||||
|
select :filename filename from dual;
|
||||||
|
spool &_filename
|
||||||
|
|
||||||
|
prompt ##### started at time #####
|
||||||
|
select systimestamp from dual;
|
||||||
|
prompt ##### database name #####
|
||||||
|
SELECT SUBSTR(PDB_NAME, (INSTR(PDB_NAME,'_',1)+1), (LENGTH(PDB_NAME)-INSTR(PDB_NAME,'_',1))) AS PDB_NAME FROM DBA_PDBS;
|
||||||
|
|
||||||
|
|
||||||
|
@@91_MARS_772_rollback_CT_MRDS_A_WORKFLOW_HISTORY.sql
|
||||||
|
@@91_MARS_772_rollback_CT_MRDS_A_TASK_HISTORY.sql
|
||||||
|
@@91_MARS_772_rollback_CT_MRDS_A_TASK_HISTORY_SOURCE.sql
|
||||||
|
@@91_MARS_772_rollback_CT_MRDS_A_TASK_HISTORY_TARGET.sql
|
||||||
|
|
||||||
|
|
||||||
|
SET ECHO OFF
|
||||||
|
|
||||||
|
prompt ##### completed at time #####
|
||||||
|
select systimestamp from dual;
|
||||||
|
|
||||||
|
SPOOL OFF
|
||||||
|
EXIT
|
||||||
0
MARS_Packages/REL01/MARS-780/.gitkeep
Normal file
0
MARS_Packages/REL01/MARS-780/.gitkeep
Normal file
247
MARS_Packages/REL01/MARS-780/LM_ADH_ADJ_Oracle_FILE_MANAGER.sql
Normal file
247
MARS_Packages/REL01/MARS-780/LM_ADH_ADJ_Oracle_FILE_MANAGER.sql
Normal file
@@ -0,0 +1,247 @@
|
|||||||
|
--=============================================================================================================================
|
||||||
|
--Step 1: Create Template Table
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--table old ADHOC_ADJ_HEADER -> new LM_ADHOC_ADJUSTMENTS_HEADER
|
||||||
|
CREATE TABLE "CT_ET_TEMPLATES"."LM_ADHOC_ADJUSTMENTS_HEADER"
|
||||||
|
( "A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"ADJUSTMENT_DATE" DATE,
|
||||||
|
"VERSION" NUMBER(10,0)
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--table old ADHOC_ADJ_ITEM -> new LM_ADHOC_ADJUSTMENTS_ITEM
|
||||||
|
CREATE TABLE "CT_ET_TEMPLATES"."LM_ADHOC_ADJUSTMENTS_ITEM"
|
||||||
|
( "A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_HEADER_FK" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"FORECAST_NAME" VARCHAR2(50 BYTE) COLLATE "USING_NLS_COMP",
|
||||||
|
"ADJUSTMENT_AMOUNT" NUMBER(25,8)
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--table old ADHOC_ADJ_ITEM_HEADER - > new ADHOC_ADJUSTMENTS_ITEM_HEADER
|
||||||
|
CREATE TABLE "CT_ET_TEMPLATES"."LM_ADHOC_ADJUSTMENTS_ITEM_HEADER"
|
||||||
|
( "A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_HEADER_FK" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"COUNTRY" VARCHAR2(3 BYTE) COLLATE "USING_NLS_COMP",
|
||||||
|
"EFFECTIVE_DATE" DATE,
|
||||||
|
"LAST_DATE_NOT_FORECAST" DATE
|
||||||
|
) ;
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 2: Configure file type for processing
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_ADHOC_ADJUSTMENTS_HEADER
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LM'
|
||||||
|
,pSourceFileType => 'INPUT'
|
||||||
|
,pSourceFileId => 'AdhocAdjustment'
|
||||||
|
,pSourceFileDesc => 'Table provides the ad-hoc revaluations adjustments related to the balance sheet.'
|
||||||
|
,pSourceFileNamePattern => '.*.csv'
|
||||||
|
,pTableId => 'LM_ADHOC_ADJUSTMENTS_HEADER'
|
||||||
|
,pTemplateTableName => 'CT_ET_TEMPLATES.LM_ADHOC_ADJUSTMENTS_HEADER'
|
||||||
|
,pContainerFileKey => NULL
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--Table LM_ADHOC_ADJUSTMENTS_ITEM
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LM'
|
||||||
|
,pSourceFileType => 'INPUT'
|
||||||
|
,pSourceFileId => 'AdhocAdjustment'
|
||||||
|
,pSourceFileDesc => 'Table provides the ad-hoc revaluations adjustments related to the balance sheet.'
|
||||||
|
,pSourceFileNamePattern => '.*.csv'
|
||||||
|
,pTableId => 'LM_ADHOC_ADJUSTMENTS_ITEM'
|
||||||
|
,pTemplateTableName => 'CT_ET_TEMPLATES.LM_ADHOC_ADJUSTMENTS_ITEM'
|
||||||
|
,pContainerFileKey => NULL
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--Table LM_ADHOC_ADJUSTMENTS_ITEM_HEADER
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LM'
|
||||||
|
,pSourceFileType => 'INPUT'
|
||||||
|
,pSourceFileId => 'AdhocAdjustment'
|
||||||
|
,pSourceFileDesc => 'Table provides the ad-hoc revaluations adjustments related to the balance sheet.'
|
||||||
|
,pSourceFileNamePattern => '.*.csv'
|
||||||
|
,pTableId => 'LM_ADHOC_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
,pTemplateTableName => 'CT_ET_TEMPLATES.LM_ADHOC_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
,pContainerFileKey => NULL
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 3: Date format adjustment
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_ADHOC_ADJUSTMENTS_HEADER
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_COLUMN_DATE_FORMAT(
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_ADHOC_ADJUSTMENTS_HEADER',
|
||||||
|
pColumnName => 'ADJUSTMENT_DATE',
|
||||||
|
pDateFormat => 'YYYY-MM-DD'
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--Table LM_ADHOC_ADJUSTMENTS_ITEM_HEADER
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_COLUMN_DATE_FORMAT(
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_ADHOC_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pColumnName => 'EFFECTIVE_DATE', pDateFormat => 'YYYY-MM-DD'
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_COLUMN_DATE_FORMAT(
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_ADHOC_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pColumnName => 'LAST_DATE_NOT_FORECAST',
|
||||||
|
pDateFormat => 'YYYY-MM-DD');
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 4: Create External Tables
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_ADHOC_ADJUSTMENTS_HEADER
|
||||||
|
|
||||||
|
--INBOX -For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_ADHOC_ADJUSTMENTS_HEADER_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_ADHOC_ADJUSTMENTS_HEADER',
|
||||||
|
pPrefix => 'INBOX/LM/AdhocAdjustment/LM_ADHOC_ADJUSTMENTS_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ODS - for processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_ADHOC_ADJUSTMENTS_HEADER_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_ADHOC_ADJUSTMENTS_HEADER',
|
||||||
|
pPrefix => 'ODS/LM/LM_ADHOC_ADJUSTMENTS_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_ADHOC_ADJUSTMENTS_HEADER_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_ADHOC_ADJUSTMENTS_HEADER',
|
||||||
|
pPrefix => 'ARCHIVE/LM/LM_ADHOC_ADJUSTMENTS_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
---------------------------------------------------------------
|
||||||
|
--Table LM_ADHOC_ADJUSTMENTS_ITEM
|
||||||
|
|
||||||
|
--INBOX -For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_ADHOC_ADJUSTMENTS_ITEM_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_ADHOC_ADJUSTMENTS_ITEM',
|
||||||
|
pPrefix => 'INBOX/LM/AdhocAdjustment/LM_ADHOC_ADJUSTMENTS_ITEM',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ODS - for processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_ADHOC_ADJUSTMENTS_ITEM_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_ADHOC_ADJUSTMENTS_ITEM',
|
||||||
|
pPrefix => 'ODS/LM/LM_ADHOC_ADJUSTMENTS_ITEM',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_ADHOC_ADJUSTMENTS_ITEM_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_ADHOC_ADJUSTMENTS_ITEM',
|
||||||
|
pPrefix => 'ARCHIVE/LM/LM_ADHOC_ADJUSTMENTS_ITEM',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
--------------------------------------------------------------------
|
||||||
|
--Table LM_ADHOC_ADJUSTMENTS_ITEM_HEADER
|
||||||
|
--INBOX -For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_ADHOC_ADJUSTMENTS_ITEM_HEADER_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_ADHOC_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pPrefix => 'INBOX/LM/AdhocAdjustment/LM_ADHOC_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ODS - for processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_ADHOC_ADJUSTMENTS_ITEM_HEADER_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_ADHOC_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pPrefix => 'ODS/LM/LM_ADHOC_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_ADHOC_ADJUSTMENTS_ITEM_HEADER_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_ADHOC_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pPrefix => 'ARCHIVE/LM/LM_ADHOC_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 5: Create Compatibility View
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--View ADHOC_ADJUSTMENTS_HEADER_MARS
|
||||||
|
CREATE OR REPLACE VIEW OU_LM.ADHOC_ADJ_HEADER_MARS AS
|
||||||
|
SELECT
|
||||||
|
A_KEY,
|
||||||
|
A_WORKFLOW_HISTORY_KEY AS A_ETL_LOAD_SET_KEY_FK,
|
||||||
|
ADJUSTMENT_DATE,
|
||||||
|
VERSION
|
||||||
|
FROM ODS.LM_ADHOC_ADJUSTMENTS_HEADER_ODS;
|
||||||
|
/
|
||||||
|
|
||||||
|
--View ADHOC_ADJUSTMENTS_ITEM_MARS
|
||||||
|
CREATE OR REPLACE VIEW OU_LM.ADHOC_ADJ_ITEM_MARS AS
|
||||||
|
SELECT
|
||||||
|
A_KEY,
|
||||||
|
A_WORKFLOW_HISTORY_KEY AS A_ETL_LOAD_SET_KEY_FK,
|
||||||
|
A_HEADER_FK,
|
||||||
|
FORECAST_NAME,
|
||||||
|
ADJUSTMENT_AMOUNT
|
||||||
|
FROM ODS.LM_ADHOC_ADJUSTMENTS_ITEM_ODS;
|
||||||
|
/
|
||||||
|
|
||||||
|
--View ADHOC_ADJUSTMENTS_ITEM_HEADER_MARS
|
||||||
|
CREATE OR REPLACE VIEW OU_LM.ADHOC_ADJ_ITEM_HEADER_MARS AS
|
||||||
|
SELECT
|
||||||
|
A_KEY,
|
||||||
|
A_WORKFLOW_HISTORY_KEY AS A_ETL_LOAD_SET_KEY_FK,
|
||||||
|
A_HEADER_FK,
|
||||||
|
COUNTRY,
|
||||||
|
EFFECTIVE_DATE,
|
||||||
|
LAST_DATE_NOT_FORECAST
|
||||||
|
FROM ODS.LM_ADHOC_ADJUSTMENTS_ITEM_HEADER_ODS;
|
||||||
|
/
|
||||||
@@ -0,0 +1,178 @@
|
|||||||
|
--=============================================================================================================================
|
||||||
|
--Step 1: Create Template Table
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_BALANCESHEET_HEADER
|
||||||
|
CREATE TABLE "CT_ET_TEMPLATES"."LM_BALANCESHEET_HEADER"
|
||||||
|
("A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"COUNTRY" VARCHAR2(3 CHAR),
|
||||||
|
"REFERENCE_DATE" DATE,
|
||||||
|
"VERSION" NUMBER(10,0),
|
||||||
|
"STATUS" VARCHAR2(1 CHAR),
|
||||||
|
"FREE_TEXT" VARCHAR2(4000 CHAR)
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
|
||||||
|
--Table LM_BALANCESHEET_ITEM
|
||||||
|
CREATE TABLE "CT_ET_TEMPLATES"."LM_BALANCESHEET_ITEM"
|
||||||
|
("A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_HEADER_FK" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"ITEM_LEVEL" NUMBER(10,0),
|
||||||
|
"POSITION" NUMBER(10,0),
|
||||||
|
"ITEM_TYPE" VARCHAR2(10 CHAR),
|
||||||
|
"FULLY_QUALIFIED_POSITION" VARCHAR2(200 CHAR),
|
||||||
|
"NAME" VARCHAR2(200 CHAR),
|
||||||
|
"AMOUNT" NUMBER(28,8)
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 2: Configure file type for processing
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_BALANCESHEET_HEADER
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LM'
|
||||||
|
,pSourceFileType => 'INPUT'
|
||||||
|
,pSourceFileId => 'AggregatedDailyFinancialStatementOfTheEurosystem'
|
||||||
|
,pSourceFileDesc => 'Table provides data on the assets and liabilities of the ECB and Euro Area NCBs.'
|
||||||
|
,pSourceFileNamePattern => '.*.csv'
|
||||||
|
,pTableId => 'LM_BALANCESHEET_HEADER'
|
||||||
|
,pTemplateTableName => 'CT_ET_TEMPLATES.LM_BALANCESHEET_HEADER'
|
||||||
|
,pContainerFileKey => NULL
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--Table LM_BALANCESHEET_ITEM
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LM'
|
||||||
|
,pSourceFileType => 'INPUT'
|
||||||
|
,pSourceFileId => 'AggregatedDailyFinancialStatementOfTheEurosystem'
|
||||||
|
,pSourceFileDesc => 'Table provides data on the assets and liabilities of the ECB and Euro Area NCBs.'
|
||||||
|
,pSourceFileNamePattern => '.*.csv'
|
||||||
|
,pTableId => 'LM_BALANCESHEET_ITEM'
|
||||||
|
,pTemplateTableName => 'CT_ET_TEMPLATES.LM_BALANCESHEET_ITEM'
|
||||||
|
,pContainerFileKey => NULL
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 3: Date format adjustment
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_BALANCESHEET_HEADER
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_COLUMN_DATE_FORMAT(
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_BALANCESHEET_HEADER',
|
||||||
|
pColumnName => 'REFERENCE_DATE',
|
||||||
|
pDateFormat => 'YYYY-MM-DD'
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 4: Create External Tables
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_BALANCESHEET_HEADER
|
||||||
|
|
||||||
|
--INBOX -For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_BALANCESHEET_HEADER_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_BALANCESHEET_HEADER',
|
||||||
|
pPrefix => 'INBOX/LM/AggregatedDailyFinancialStatementOfTheEurosystem/LM_BALANCESHEET_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ODS - for processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_BALANCESHEET_HEADER_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_BALANCESHEET_HEADER',
|
||||||
|
pPrefix => 'ODS/LM/LM_BALANCESHEET_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_BALANCESHEET_HEADER_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_BALANCESHEET_HEADER',
|
||||||
|
pPrefix => 'ARCHIVE/LM/LM_BALANCESHEET_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
---------------------------------------------------------------
|
||||||
|
--Table LM_BALANCESHEET_ITEM
|
||||||
|
|
||||||
|
--INBOX -For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_BALANCESHEET_ITEM_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_BALANCESHEET_ITEM',
|
||||||
|
pPrefix => 'INBOX/LM/AggregatedDailyFinancialStatementOfTheEurosystem/LM_BALANCESHEET_ITEM',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ODS - for processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_BALANCESHEET_ITEM_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_BALANCESHEET_ITEM',
|
||||||
|
pPrefix => 'ODS/LM/LM_BALANCESHEET_ITEM',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_BALANCESHEET_ITEM_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_BALANCESHEET_ITEM',
|
||||||
|
pPrefix => 'ARCHIVE/LM/LM_BALANCESHEET_ITEM',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 5: Create Compatibility View
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--View BALANCESHEET_HEADER_MARS
|
||||||
|
CREATE OR REPLACE VIEW OU_LM.BALANCESHEET_HEADER_MARS AS
|
||||||
|
SELECT
|
||||||
|
A_KEY,
|
||||||
|
A_WORKFLOW_HISTORY_KEY AS A_ETL_LOAD_SET_KEY,
|
||||||
|
COUNTRY,
|
||||||
|
REFERENCE_DATE,
|
||||||
|
VERSION,
|
||||||
|
STATUS,
|
||||||
|
FREE_TEXT
|
||||||
|
FROM ODS.LM_BALANCESHEET_HEADER_ODS;
|
||||||
|
/
|
||||||
|
|
||||||
|
--View BALANCESHEET_ITEM_MARS
|
||||||
|
CREATE OR REPLACE VIEW OU_LM.BALANCESHEET_ITEM_MARS AS
|
||||||
|
SELECT
|
||||||
|
A_KEY,
|
||||||
|
A_WORKFLOW_HISTORY_KEY AS A_ETL_LOAD_SET_KEY,
|
||||||
|
A_HEADER_FK,
|
||||||
|
ITEM_LEVEL,
|
||||||
|
POSITION,
|
||||||
|
ITEM_TYPE,
|
||||||
|
FULLY_QUALIFIED_POSITION,
|
||||||
|
NAME,
|
||||||
|
AMOUNT
|
||||||
|
FROM ODS.LM_BALANCESHEET_ITEM_ODS;
|
||||||
|
/
|
||||||
250
MARS_Packages/REL01/MARS-780/LM_CSM_ADJ_Oracle_FILE_MANAGER.sql
Normal file
250
MARS_Packages/REL01/MARS-780/LM_CSM_ADJ_Oracle_FILE_MANAGER.sql
Normal file
@@ -0,0 +1,250 @@
|
|||||||
|
--=============================================================================================================================
|
||||||
|
--Step 1: Create Template Table
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--table old CSM_ADJ_HEADER -> new LM_CSM_ADJUSTMENTS_HEADER
|
||||||
|
CREATE TABLE "CT_ET_TEMPLATES"."LM_CSM_ADJUSTMENTS_HEADER"
|
||||||
|
("A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"YEAR" VARCHAR2(4 BYTE) COLLATE "USING_NLS_COMP",
|
||||||
|
"MONTH" VARCHAR2(2 BYTE) COLLATE "USING_NLS_COMP",
|
||||||
|
"VERSION" NUMBER(10,0)
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--table old CSM_ADJ_ITEM -> new LM_CSM_ADJUSTMENTS_ITEM
|
||||||
|
CREATE TABLE "CT_ET_TEMPLATES"."LM_CSM_ADJUSTMENTS_ITEM"
|
||||||
|
("A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_HEADER_FK" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"FORECAST_NAME" VARCHAR2(50 BYTE) COLLATE "USING_NLS_COMP",
|
||||||
|
"ADJUSTMENT_AMOUNT" VARCHAR2(50 BYTE) -- !!!! due to incomming data containing comma
|
||||||
|
--"ADJUSTMENT_AMOUNT" NUMBER(25,8)
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
|
||||||
|
--table old CSM_ADJ_ITEM_HEADER - > new LM_CSM_ADJUSTMENTS_ITEM_HEADER
|
||||||
|
CREATE TABLE "CT_ET_TEMPLATES"."LM_CSM_ADJUSTMENTS_ITEM_HEADER"
|
||||||
|
("A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_HEADER_FK" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"COUNTRY" VARCHAR2(3 BYTE) COLLATE "USING_NLS_COMP",
|
||||||
|
"EFFECTIVE_DATE" DATE,
|
||||||
|
"LAST_DATE_NOT_FORECAST" DATE
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 2: Configure file type for processing
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_CSM_ADJUSTMENTS_HEADER
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LM'
|
||||||
|
,pSourceFileType => 'INPUT'
|
||||||
|
,pSourceFileId => 'MonthlyCSMAdjustment'
|
||||||
|
,pSourceFileDesc => 'Table provides the balance sheet with revised banknotes in line with monthly capital share mechanism.'
|
||||||
|
,pSourceFileNamePattern => '.*.csv'
|
||||||
|
,pTableId => 'LM_CSM_ADJUSTMENTS_HEADER'
|
||||||
|
,pTemplateTableName => 'CT_ET_TEMPLATES.LM_CSM_ADJUSTMENTS_HEADER'
|
||||||
|
,pContainerFileKey => NULL
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--Table LM_CSM_ADJUSTMENTS_ITEM
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LM'
|
||||||
|
,pSourceFileType => 'INPUT'
|
||||||
|
,pSourceFileId => 'MonthlyCSMAdjustment'
|
||||||
|
,pSourceFileDesc => 'Table provides the balance sheet with revised banknotes in line with monthly capital share mechanism.'
|
||||||
|
,pSourceFileNamePattern => '.*.csv'
|
||||||
|
,pTableId => 'LM_CSM_ADJUSTMENTS_ITEM'
|
||||||
|
,pTemplateTableName => 'CT_ET_TEMPLATES.LM_CSM_ADJUSTMENTS_ITEM'
|
||||||
|
,pContainerFileKey => NULL
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--Table LM_CSM_ADJUSTMENTS_ITEM_HEADER
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LM'
|
||||||
|
,pSourceFileType => 'INPUT'
|
||||||
|
,pSourceFileId => 'MonthlyCSMAdjustment'
|
||||||
|
,pSourceFileDesc => 'Table provides the balance sheet with revised banknotes in line with monthly capital share mechanism.'
|
||||||
|
,pSourceFileNamePattern => '.*.csv'
|
||||||
|
,pTableId => 'LM_CSM_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
,pTemplateTableName => 'CT_ET_TEMPLATES.LM_CSM_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
,pContainerFileKey => NULL
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 3: Date format adjustment
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
|
||||||
|
--Table LM_CSM_ADJUSTMENTS_ITEM_HEADER
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_COLUMN_DATE_FORMAT(
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_CSM_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pColumnName => 'EFFECTIVE_DATE',
|
||||||
|
pDateFormat => 'YYYY-MM-DD'
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_COLUMN_DATE_FORMAT(
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_CSM_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pColumnName => 'LAST_DATE_NOT_FORECAST',
|
||||||
|
pDateFormat => 'YYYY-MM-DD'
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 4: Create External Tables
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_CSM_ADJUSTMENTS_HEADER
|
||||||
|
|
||||||
|
--INBOX -For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_CSM_ADJUSTMENTS_HEADER_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_CSM_ADJUSTMENTS_HEADER',
|
||||||
|
pPrefix => 'INBOX/LM/MonthlyCSMAdjustment/LM_CSM_ADJUSTMENTS_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ODS - for processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_CSM_ADJUSTMENTS_HEADER_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_CSM_ADJUSTMENTS_HEADER',
|
||||||
|
pPrefix => 'ODS/LM/LM_CSM_ADJUSTMENTS_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_CSM_ADJUSTMENTS_HEADER_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_CSM_ADJUSTMENTS_HEADER',
|
||||||
|
pPrefix => 'ARCHIVE/LM/LM_CSM_ADJUSTMENTS_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
---------------------------------------------------------------
|
||||||
|
--Table LM_CSM_ADJUSTMENTS_ITEM
|
||||||
|
|
||||||
|
--INBOX -For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_CSM_ADJUSTMENTS_ITEM_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_CSM_ADJUSTMENTS_ITEM',
|
||||||
|
pPrefix => 'INBOX/LM/MonthlyCSMAdjustment/LM_CSM_ADJUSTMENTS_ITEM',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ODS - for processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_CSM_ADJUSTMENTS_ITEM_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_CSM_ADJUSTMENTS_ITEM',
|
||||||
|
pPrefix => 'ODS/LM/LM_CSM_ADJUSTMENTS_ITEM',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_CSM_ADJUSTMENTS_ITEM_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_CSM_ADJUSTMENTS_ITEM',
|
||||||
|
pPrefix => 'ARCHIVE/LM/LM_CSM_ADJUSTMENTS_ITEM',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--------------------------------------------------------------------
|
||||||
|
--Table LM_CSM_ADJUSTMENTS_ITEM_HEADER
|
||||||
|
|
||||||
|
--INBOX -For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_CSM_ADJUSTMENTS_ITEM_HEADER_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_CSM_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pPrefix => 'INBOX/LM/MonthlyCSMAdjustment/LM_CSM_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ODS - for processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_CSM_ADJUSTMENTS_ITEM_HEADER_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_CSM_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pPrefix => 'ODS/LM/LM_CSM_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_CSM_ADJUSTMENTS_ITEM_HEADER_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_CSM_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pPrefix => 'ARCHIVE/LM/LM_CSM_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 5: Create Compatibility View
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
|
||||||
|
--View CSM_ADJUSTMENTS_HEADER_MARS
|
||||||
|
CREATE OR REPLACE VIEW OU_LM.CSM_ADJ_HEADER_MARS AS
|
||||||
|
SELECT
|
||||||
|
A_KEY,
|
||||||
|
A_WORKFLOW_HISTORY_KEY AS A_ETL_LOAD_SET_KEY_FK,
|
||||||
|
YEAR,
|
||||||
|
MONTH,
|
||||||
|
VERSION
|
||||||
|
FROM ODS.LM_CSM_ADJUSTMENTS_HEADER_ODS;
|
||||||
|
/
|
||||||
|
|
||||||
|
--View CSM_ADJUSTMENTS_ITEM_MARS
|
||||||
|
CREATE OR REPLACE VIEW OU_LM.CSM_ADJ_ITEM_MARS AS
|
||||||
|
SELECT
|
||||||
|
A_KEY,
|
||||||
|
A_WORKFLOW_HISTORY_KEY AS A_ETL_LOAD_SET_KEY_FK,
|
||||||
|
A_HEADER_FK,
|
||||||
|
FORECAST_NAME,
|
||||||
|
ADJUSTMENT_AMOUNT
|
||||||
|
FROM ODS.LM_CSM_ADJUSTMENTS_ITEM_ODS;
|
||||||
|
/
|
||||||
|
|
||||||
|
--View CSM_ADJUSTMENTS_ITEM_HEADER_MARS
|
||||||
|
CREATE OR REPLACE VIEW OU_LM.CSM_ADJ_ITEM_HEADER_MARS AS
|
||||||
|
SELECT
|
||||||
|
A_KEY,
|
||||||
|
A_WORKFLOW_HISTORY_KEY AS A_ETL_LOAD_SET_KEY_FK,
|
||||||
|
A_HEADER_FK,
|
||||||
|
COUNTRY,
|
||||||
|
EFFECTIVE_DATE,
|
||||||
|
LAST_DATE_NOT_FORECAST
|
||||||
|
FROM ODS.LM_CSM_ADJUSTMENTS_ITEM_HEADER_ODS;
|
||||||
|
/
|
||||||
@@ -0,0 +1,184 @@
|
|||||||
|
--=============================================================================================================================
|
||||||
|
--Step 1: Create Template Table
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_CURRENT_ACCOUNTS_HEADER
|
||||||
|
CREATE TABLE "CT_ET_TEMPLATES"."LM_CURRENT_ACCOUNTS_HEADER"
|
||||||
|
("A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"REVISION_NUMBER" NUMBER(10,0),
|
||||||
|
"REFERENCE_DATE" DATE,
|
||||||
|
"FREE_TEXT" VARCHAR2(1000 CHAR),
|
||||||
|
"CURRENT_ACCOUNT_BS_TOTAL" NUMBER(28,8),
|
||||||
|
"MRR_FORECAST_TOTAL" NUMBER(28,8),
|
||||||
|
"CURRENT_ACCOUNT_MESSAGE_TOTAL" NUMBER(28,8),
|
||||||
|
"MRR_MESSAGE_TOTAL" NUMBER(28,8),
|
||||||
|
"PERIODICITY" VARCHAR2(50 CHAR)
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--Table LM_CURRENT_ACCOUNTS_ITEM
|
||||||
|
CREATE TABLE "CT_ET_TEMPLATES"."LM_CURRENT_ACCOUNTS_ITEM"
|
||||||
|
("A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_HEADER_FK" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"COUNTRY" VARCHAR2(3 CHAR),
|
||||||
|
"MFI_CODE" VARCHAR2(255 CHAR),
|
||||||
|
"BANK_NAME" VARCHAR2(500 CHAR),
|
||||||
|
"CURRENT_ACCOUNT" NUMBER(28,8),
|
||||||
|
"MINIMUM_RESERVE_REQUIREMENT" NUMBER(28,8),
|
||||||
|
"COMMENT_" VARCHAR2(4000 CHAR)
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 2: Configure file type for processing
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_CURRENT_ACCOUNTS_HEADER
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LM'
|
||||||
|
,pSourceFileType => 'INPUT'
|
||||||
|
,pSourceFileId => 'MinimumReservesRequirements'
|
||||||
|
,pSourceFileDesc => 'Table provides data on the account balance that credit institutions hold with their respective NCB within the Eurosystem. This is primarily used to hold minimum reserves.'
|
||||||
|
,pSourceFileNamePattern => '.*.csv'
|
||||||
|
,pTableId => 'LM_CURRENT_ACCOUNTS_HEADER'
|
||||||
|
,pTemplateTableName => 'CT_ET_TEMPLATES.LM_CURRENT_ACCOUNTS_HEADER'
|
||||||
|
,pContainerFileKey => NULL
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--Table LM_CURRENT_ACCOUNTS_ITEM
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LM'
|
||||||
|
,pSourceFileType => 'INPUT'
|
||||||
|
,pSourceFileId => 'MinimumReservesRequirements'
|
||||||
|
,pSourceFileDesc => 'Table provides data on the account balance that credit institutions hold with their respective NCB within the Eurosystem. This is primarily used to hold minimum reserves.'
|
||||||
|
,pSourceFileNamePattern => '.*.csv'
|
||||||
|
,pTableId => 'LM_CURRENT_ACCOUNTS_ITEM'
|
||||||
|
,pTemplateTableName => 'CT_ET_TEMPLATES.LM_CURRENT_ACCOUNTS_ITEM'
|
||||||
|
,pContainerFileKey => NULL
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 3: Date format adjustment
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_CURRENT_ACCOUNTS_HEADER
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_COLUMN_DATE_FORMAT(
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_CURRENT_ACCOUNTS_HEADER',
|
||||||
|
pColumnName => 'REFERENCE_DATE',
|
||||||
|
pDateFormat => 'YYYY-MM-DD'
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 4: Create External Tables
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_CURRENT_ACCOUNTS_HEADER
|
||||||
|
|
||||||
|
--INBOX -For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_CURRENT_ACCOUNTS_HEADER_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_CURRENT_ACCOUNTS_HEADER',
|
||||||
|
pPrefix => 'INBOX/LM/MinimumReservesRequirements/LM_CURRENT_ACCOUNTS_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ODS - for processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_CURRENT_ACCOUNTS_HEADER_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_CURRENT_ACCOUNTS_HEADER',
|
||||||
|
pPrefix => 'ODS/LM/LM_CURRENT_ACCOUNTS_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_CURRENT_ACCOUNTS_HEADER_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_CURRENT_ACCOUNTS_HEADER',
|
||||||
|
pPrefix => 'ARCHIVE/LM/LM_CURRENT_ACCOUNTS_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
---------------------------------------------------------------
|
||||||
|
--Table LM_CURRENT_ACCOUNTS_ITEM
|
||||||
|
|
||||||
|
--INBOX -For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_CURRENT_ACCOUNTS_ITEM_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_CURRENT_ACCOUNTS_ITEM',
|
||||||
|
pPrefix => 'INBOX/LM/MinimumReservesRequirements/LM_CURRENT_ACCOUNTS_ITEM',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ODS - for processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_CURRENT_ACCOUNTS_ITEM_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_CURRENT_ACCOUNTS_ITEM',
|
||||||
|
pPrefix => 'ODS/LM/LM_CURRENT_ACCOUNTS_ITEM',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_CURRENT_ACCOUNTS_ITEM_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_CURRENT_ACCOUNTS_ITEM',
|
||||||
|
pPrefix => 'ARCHIVE/LM/LM_CURRENT_ACCOUNTS_ITEM',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 5: Create Compatibility View
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--View MRR_IND_CURRENT_ACCOUNT_HEADER_MARS
|
||||||
|
CREATE OR REPLACE VIEW OU_MRR.MRR_IND_CURRENT_ACCOUNT_HEADER_MARS AS
|
||||||
|
SELECT
|
||||||
|
REVISION_NUMBER,
|
||||||
|
REFERENCE_DATE,
|
||||||
|
FREE_TEXT,
|
||||||
|
CURRENT_ACCOUNT_BS_TOTAL,
|
||||||
|
MRR_FORECAST_TOTAL,
|
||||||
|
CURRENT_ACCOUNT_MESSAGE_TOTAL,
|
||||||
|
MRR_MESSAGE_TOTAL,
|
||||||
|
PERIODICITY,
|
||||||
|
A_KEY,
|
||||||
|
A_WORKFLOW_HISTORY_KEY AS A_ETL_LOAD_SET_KEY
|
||||||
|
FROM ODS.LM_CURRENT_ACCOUNTS_HEADER_ODS;
|
||||||
|
/
|
||||||
|
|
||||||
|
--View MRR_IND_CURRENT_ACCOUNT_ITEM_MARS
|
||||||
|
CREATE OR REPLACE VIEW OU_MRR.MRR_IND_CURRENT_ACCOUNT_ITEM_MARS AS
|
||||||
|
SELECT
|
||||||
|
COUNTRY,
|
||||||
|
MFI_CODE,
|
||||||
|
BANK_NAME,
|
||||||
|
CURRENT_ACCOUNT,
|
||||||
|
MINIMUM_RESERVE_REQUIREMENT,
|
||||||
|
COMMENT_,
|
||||||
|
A_KEY,
|
||||||
|
A_WORKFLOW_HISTORY_KEY AS A_ETL_LOAD_SET_KEY,
|
||||||
|
A_HEADER_FK
|
||||||
|
FROM ODS.LM_CURRENT_ACCOUNTS_ITEM_ODS;
|
||||||
|
/
|
||||||
178
MARS_Packages/REL01/MARS-780/LM_FORECAST_Oracle_FILE_MANAGER.sql
Normal file
178
MARS_Packages/REL01/MARS-780/LM_FORECAST_Oracle_FILE_MANAGER.sql
Normal file
@@ -0,0 +1,178 @@
|
|||||||
|
--=============================================================================================================================
|
||||||
|
--Step 1: Create Template Table
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_FORECAST_HEADER
|
||||||
|
CREATE TABLE "CT_ET_TEMPLATES"."LM_FORECAST_HEADER"
|
||||||
|
("A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"COUNTRY" VARCHAR2(3 CHAR),
|
||||||
|
"REFERENCE_DATE" DATE,
|
||||||
|
"REVISION" NUMBER(10,0),
|
||||||
|
"FREE_TEXT" VARCHAR2(4000 CHAR)
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--Table LM_FORECAST_ITEM
|
||||||
|
CREATE TABLE "CT_ET_TEMPLATES"."LM_FORECAST_ITEM"
|
||||||
|
("A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_HEADER_FK" NUMBER(38,0),
|
||||||
|
"FORECAST_DATE" DATE,
|
||||||
|
"FORECAST_NAME" VARCHAR2(50 CHAR),
|
||||||
|
"FORECAST_VALUE" NUMBER(28,10)
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 2: Configure file type for processing
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_FORECAST_HEADER
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LM'
|
||||||
|
,pSourceFileType => 'INPUT'
|
||||||
|
,pSourceFileId => 'EurosystemAutonomousFactorForecast'
|
||||||
|
,pSourceFileDesc => 'Table provides forecasts on the autonomous factors, monetary policy portfolios and reserve requirements. The data is updated weekly and disseminated daily'
|
||||||
|
,pSourceFileNamePattern => '.*.csv'
|
||||||
|
,pTableId => 'LM_FORECAST_HEADER'
|
||||||
|
,pTemplateTableName => 'CT_ET_TEMPLATES.LM_FORECAST_HEADER'
|
||||||
|
,pContainerFileKey => NULL
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--Table LM_FORECAST_ITEM
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LM'
|
||||||
|
,pSourceFileType => 'INPUT'
|
||||||
|
,pSourceFileId => 'EurosystemAutonomousFactorForecast'
|
||||||
|
,pSourceFileDesc => 'Table provides forecasts on the autonomous factors, monetary policy portfolios and reserve requirements. The data is updated weekly and disseminated daily'
|
||||||
|
,pSourceFileNamePattern => '.*.csv'
|
||||||
|
,pTableId => 'LM_FORECAST_ITEM'
|
||||||
|
,pTemplateTableName => 'CT_ET_TEMPLATES.LM_FORECAST_ITEM'
|
||||||
|
,pContainerFileKey => NULL
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 3: Date format adjustment
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_FORECAST_HEADER
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_COLUMN_DATE_FORMAT(
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_FORECAST_HEADER',
|
||||||
|
pColumnName => 'REFERENCE_DATE',
|
||||||
|
pDateFormat => 'YYYY-MM-DD'
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--Table LM_FORECAST_ITEM
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_COLUMN_DATE_FORMAT(
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_FORECAST_ITEM',
|
||||||
|
pColumnName => 'FORECAST_DATE',
|
||||||
|
pDateFormat => 'YYYY-MM-DD'
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 4: Create External Tables
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_FORECAST_HEADER
|
||||||
|
|
||||||
|
--INBOX -For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_FORECAST_HEADER_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_FORECAST_HEADER',
|
||||||
|
pPrefix => 'INBOX/LM/EurosystemAutonomousFactorForecast/LM_FORECAST_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ODS - for processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_FORECAST_HEADER_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_FORECAST_HEADER',
|
||||||
|
pPrefix => 'ODS/LM/LM_FORECAST_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_FORECAST_HEADER_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_FORECAST_HEADER',
|
||||||
|
pPrefix => 'ARCHIVE/LM/LM_FORECAST_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
---------------------------------------------------------------
|
||||||
|
--Table LM_FORECAST_ITEM
|
||||||
|
|
||||||
|
--INBOX -For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_FORECAST_ITEM_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_FORECAST_ITEM',
|
||||||
|
pPrefix => 'INBOX/LM/EurosystemAutonomousFactorForecast/LM_FORECAST_ITEM',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ODS - for processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_FORECAST_ITEM_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_FORECAST_ITEM',
|
||||||
|
pPrefix => 'ODS/LM/LM_FORECAST_ITEM',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_FORECAST_ITEM_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_FORECAST_ITEM',
|
||||||
|
pPrefix => 'ARCHIVE/LM/LM_FORECAST_ITEM',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 5: Create Compatibility View
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--View FORECAST_HEADER_MARS
|
||||||
|
CREATE OR REPLACE VIEW OU_LM.FORECAST_HEADER_MARS AS
|
||||||
|
SELECT
|
||||||
|
A_KEY,
|
||||||
|
A_WORKFLOW_HISTORY_KEY AS A_ETL_LOAD_SET_FK,
|
||||||
|
COUNTRY,
|
||||||
|
REFERENCE_DATE,
|
||||||
|
REVISION,
|
||||||
|
FREE_TEXT
|
||||||
|
FROM ODS.LM_FORECAST_HEADER_ODS;
|
||||||
|
/
|
||||||
|
|
||||||
|
--View FORECAST_ITEM_MARS
|
||||||
|
CREATE OR REPLACE VIEW OU_LM.FORECAST_ITEM_MARS AS
|
||||||
|
SELECT
|
||||||
|
A_KEY,
|
||||||
|
A_WORKFLOW_HISTORY_KEY AS A_ETL_LOAD_SET_FK,
|
||||||
|
A_HEADER_FK,
|
||||||
|
FORECAST_DATE,
|
||||||
|
FORECAST_NAME,
|
||||||
|
FORECAST_VALUE
|
||||||
|
FROM ODS.LM_FORECAST_ITEM_ODS;
|
||||||
|
/
|
||||||
@@ -0,0 +1,247 @@
|
|||||||
|
--=============================================================================================================================
|
||||||
|
--Step 1: Create Template Table
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--table old QR_ADJ_HEADER -> new LM_QRE_ADJUSTMENTS_HEADER
|
||||||
|
CREATE TABLE "CT_ET_TEMPLATES".LM_QRE_ADJUSTMENTS_HEADER
|
||||||
|
("A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"YEAR" VARCHAR2(4 CHAR) COLLATE "USING_NLS_COMP",
|
||||||
|
"QUARTER" NUMBER(1,0),
|
||||||
|
"VERSION" NUMBER(10,0)
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--table old QR_ADJ_ITEM -> new LM_QRE_ADJUSTMENTS_ITEM
|
||||||
|
CREATE TABLE "CT_ET_TEMPLATES".LM_QRE_ADJUSTMENTS_ITEM
|
||||||
|
("A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_HEADER_FK" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"FORECAST_NAME" VARCHAR2(50 BYTE) COLLATE "USING_NLS_COMP",
|
||||||
|
"ADJUSTMENT_AMOUNT" NUMBER(25,8)
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--table old QR_ADJ_ITEM_HEADER - > new LM_QRE_ADJUSTMENTS_ITEM_HEADER
|
||||||
|
CREATE TABLE "CT_ET_TEMPLATES"."LM_QRE_ADJUSTMENTS_ITEM_HEADER"
|
||||||
|
( "A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_HEADER_FK" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"COUNTRY" VARCHAR2(3 BYTE) COLLATE "USING_NLS_COMP",
|
||||||
|
"EFFECTIVE_DATE" DATE,
|
||||||
|
"LAST_DATE_NOT_FORECAST" DATE
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 2: Configure file type for processing
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_QRE_ADJUSTMENTS_HEADER
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LM'
|
||||||
|
,pSourceFileType => 'INPUT'
|
||||||
|
,pSourceFileId => 'QuarterlyRevaluationAdjustment'
|
||||||
|
,pSourceFileDesc => 'Table provides the quarterly revaluations adjustments related to the balance sheet.'
|
||||||
|
,pSourceFileNamePattern => '.*.csv'
|
||||||
|
,pTableId => 'LM_QRE_ADJUSTMENTS_HEADER'
|
||||||
|
,pTemplateTableName => 'CT_ET_TEMPLATES.LM_QRE_ADJUSTMENTS_HEADER'
|
||||||
|
,pContainerFileKey => NULL
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--Table LM_QRE_ADJUSTMENTS_ITEM
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LM'
|
||||||
|
,pSourceFileType => 'INPUT'
|
||||||
|
,pSourceFileId => 'QuarterlyRevaluationAdjustment'
|
||||||
|
,pSourceFileDesc => 'Table provides the quarterly revaluations adjustments related to the balance sheet.'
|
||||||
|
,pSourceFileNamePattern => '.*.csv'
|
||||||
|
,pTableId => 'LM_QRE_ADJUSTMENTS_ITEM'
|
||||||
|
,pTemplateTableName => 'CT_ET_TEMPLATES.LM_QRE_ADJUSTMENTS_ITEM'
|
||||||
|
,pContainerFileKey => NULL
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--Table LM_QRE_ADJUSTMENTS_ITEM_HEADER
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LM'
|
||||||
|
,pSourceFileType => 'INPUT'
|
||||||
|
,pSourceFileId => 'QuarterlyRevaluationAdjustment'
|
||||||
|
,pSourceFileDesc => 'Table provides the quarterly revaluations adjustments related to the balance sheet.'
|
||||||
|
,pSourceFileNamePattern => '.*.csv'
|
||||||
|
,pTableId => 'LM_QRE_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
,pTemplateTableName => 'CT_ET_TEMPLATES.LM_QRE_ADJUSTMENTS_ITEM_HEADER'
|
||||||
|
,pContainerFileKey => NULL
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 3: Date format adjustment
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
|
||||||
|
--Change data format
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_COLUMN_DATE_FORMAT(
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_QRE_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pColumnName => 'EFFECTIVE_DATE',
|
||||||
|
pDateFormat => 'YYYY-MM-DD'
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--Change data format
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_COLUMN_DATE_FORMAT(
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_QRE_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pColumnName => 'LAST_DATE_NOT_FORECAST',
|
||||||
|
pDateFormat => 'YYYY-MM-DD'
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 4: Create External Tables
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_QRE_ADJUSTMENTS_HEADER
|
||||||
|
|
||||||
|
--INBOX -For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_QRE_ADJUSTMENTS_HEADER_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_QRE_ADJUSTMENTS_HEADER',
|
||||||
|
pPrefix => 'INBOX/LM/QuarterlyRevaluationAdjustment/LM_QRE_ADJUSTMENTS_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ODS - for processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_QRE_ADJUSTMENTS_HEADER_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_QRE_ADJUSTMENTS_HEADER',
|
||||||
|
pPrefix => 'ODS/LM/LM_QRE_ADJUSTMENTS_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_QRE_ADJUSTMENTS_HEADER_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_QRE_ADJUSTMENTS_HEADER',
|
||||||
|
pPrefix => 'ARCHIVE/LM/LM_QRE_ADJUSTMENTS_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
---------------------------------------------------------------
|
||||||
|
--Table LM_QRE_ADJUSTMENTS_ITEM
|
||||||
|
|
||||||
|
--INBOX -For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_QRE_ADJUSTMENTS_ITEM_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_QRE_ADJUSTMENTS_ITEM',
|
||||||
|
pPrefix => 'INBOX/LM/QuarterlyRevaluationAdjustment/LM_QRE_ADJUSTMENTS_ITEM',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ODS - for processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_QRE_ADJUSTMENTS_ITEM_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_QRE_ADJUSTMENTS_ITEM',
|
||||||
|
pPrefix => 'ODS/LM/LM_QRE_ADJUSTMENTS_ITEM',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_QRE_ADJUSTMENTS_ITEM_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_QRE_ADJUSTMENTS_ITEM',
|
||||||
|
pPrefix => 'ARCHIVE/LM/LM_QRE_ADJUSTMENTS_ITEM',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--------------------------------------------------------------------
|
||||||
|
--Table LM_QRE_ADJUSTMENTS_ITEM_HEADER
|
||||||
|
--INBOX -For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_QRE_ADJUSTMENTS_ITEM_HEADER_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_QRE_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pPrefix => 'INBOX/LM/QuarterlyRevaluationAdjustment/LM_QRE_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ODS - for processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_QRE_ADJUSTMENTS_ITEM_HEADER_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_QRE_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pPrefix => 'ODS/LM/LM_QRE_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_QRE_ADJUSTMENTS_ITEM_HEADER_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_QRE_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pPrefix => 'ARCHIVE/LM/LM_QRE_ADJUSTMENTS_ITEM_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 5: Create Compatibility View
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
|
||||||
|
--View QRE_ADJUSTMENTS_HEADER_MARS
|
||||||
|
CREATE OR REPLACE VIEW OU_LM.QRE_ADJ_HEADER_MARS AS
|
||||||
|
SELECT
|
||||||
|
A_KEY,
|
||||||
|
A_WORKFLOW_HISTORY_KEY AS A_ETL_LOAD_SET_KEY_FK,
|
||||||
|
YEAR,
|
||||||
|
QUARTER,
|
||||||
|
VERSION
|
||||||
|
FROM ODS.LM_QRE_ADJUSTMENTS_HEADER_ODS;
|
||||||
|
/
|
||||||
|
|
||||||
|
--View QRE_ADJUSTMENTS_ITEM_MARS
|
||||||
|
CREATE OR REPLACE VIEW OU_LM.QRE_ADJ_ITEM_MARS AS
|
||||||
|
SELECT
|
||||||
|
A_KEY,
|
||||||
|
A_WORKFLOW_HISTORY_KEY AS A_ETL_LOAD_SET_KEY_FK,
|
||||||
|
A_HEADER_FK,
|
||||||
|
FORECAST_NAME,
|
||||||
|
ADJUSTMENT_AMOUNT
|
||||||
|
FROM ODS.LM_QRE_ADJUSTMENTS_ITEM_ODS;
|
||||||
|
/
|
||||||
|
|
||||||
|
--View QRE_ADJUSTMENTS_ITEM_HEADER_MARS
|
||||||
|
CREATE OR REPLACE VIEW OU_LM.QRE_ADJ_ITEM_HEADER_MARS AS
|
||||||
|
SELECT
|
||||||
|
A_KEY,
|
||||||
|
A_WORKFLOW_HISTORY_KEY AS A_ETL_LOAD_SET_KEY_FK,
|
||||||
|
A_HEADER_FK,
|
||||||
|
COUNTRY,
|
||||||
|
EFFECTIVE_DATE,
|
||||||
|
LAST_DATE_NOT_FORECAST
|
||||||
|
FROM ODS.LM_QRE_ADJUSTMENTS_ITEM_HEADER_ODS;
|
||||||
|
/
|
||||||
@@ -0,0 +1,180 @@
|
|||||||
|
--=============================================================================================================================
|
||||||
|
--Step 1: Create Template Table
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_STANDING_FACILITIES
|
||||||
|
CREATE TABLE "CT_ET_TEMPLATES"."LM_STANDING_FACILITIES"
|
||||||
|
("A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_SFH_FK" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"COUNTRY" VARCHAR2(12 BYTE),
|
||||||
|
"MFI_ID" VARCHAR2(1020 BYTE),
|
||||||
|
"MFI_NAME" VARCHAR2(2000 BYTE),
|
||||||
|
"MARGINAL_LENDING_FACILITY" NUMBER(28,10),
|
||||||
|
"DEPOSIT_FACILITY" NUMBER(28,10),
|
||||||
|
"COMMENT_" VARCHAR2(16000 BYTE)
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--Table STANDING_FACILITIES_HEADER
|
||||||
|
CREATE TABLE "CT_ET_TEMPLATES"."LM_STANDING_FACILITIES_HEADER"
|
||||||
|
("A_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"A_WORKFLOW_HISTORY_KEY" NUMBER(38,0) NOT NULL ENABLE,
|
||||||
|
"REV_NUMBER" NUMBER(28,0),
|
||||||
|
"REF_DATE" DATE,
|
||||||
|
"FREE_TEXT" VARCHAR2(4000 BYTE),
|
||||||
|
"MLF_BS_TOTAL" NUMBER(28,10),
|
||||||
|
"DF_BS_TOTAL" NUMBER(28,10),
|
||||||
|
"MLF_SF_TOTAL" NUMBER(28,10),
|
||||||
|
"DF_SF_TOTAL" NUMBER(28,10)
|
||||||
|
);
|
||||||
|
/
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 2: Configure file type for processing
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_STANDING_FACILITIES
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LM'
|
||||||
|
,pSourceFileType => 'INPUT'
|
||||||
|
,pSourceFileId => 'DistributeStandingFacilities'
|
||||||
|
,pSourceFileDesc => 'Table provides data on the credit facility available to counterparties at their own initiative. The Eurosystem offers two overnight standing facilities: the marginal lending facility and the deposit facility'
|
||||||
|
,pSourceFileNamePattern => '.*.csv'
|
||||||
|
,pTableId => 'LM_STANDING_FACILITIES'
|
||||||
|
,pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES'
|
||||||
|
,pContainerFileKey => NULL
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--Table LM_STANDING_FACILITIES_HEADER
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_SOURCE_FILE_CONFIG(
|
||||||
|
pSourceKey => 'LM'
|
||||||
|
,pSourceFileType => 'INPUT'
|
||||||
|
,pSourceFileId => 'DistributeStandingFacilities'
|
||||||
|
,pSourceFileDesc => 'Table provides data on the credit facility available to counterparties at their own initiative. The Eurosystem offers two overnight standing facilities: the marginal lending facility and the deposit facility'
|
||||||
|
,pSourceFileNamePattern => '.*.csv'
|
||||||
|
,pTableId => 'LM_STANDING_FACILITIES_HEADER'
|
||||||
|
,pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES_HEADER'
|
||||||
|
,pContainerFileKey => NULL
|
||||||
|
);
|
||||||
|
/
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 3: Date format adjustment
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_STANDING_FACILITIES_HEADER
|
||||||
|
CALL CT_MRDS.FILE_MANAGER.ADD_COLUMN_DATE_FORMAT(
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES_HEADER',
|
||||||
|
pColumnName => 'REF_DATE',
|
||||||
|
pDateFormat => 'YYYY-MM-DD'
|
||||||
|
);
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 4: Create External Tables
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--Table LM_STANDING_FACILITIES
|
||||||
|
|
||||||
|
--INBOX -For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_STANDING_FACILITIES_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES',
|
||||||
|
pPrefix => 'INBOX/LM/DistributeStandingFacilities/LM_STANDING_FACILITIES',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ODS - for processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_STANDING_FACILITIES_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES',
|
||||||
|
pPrefix => 'ODS/LM/LM_STANDING_FACILITIES',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_STANDING_FACILITIES_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES',
|
||||||
|
pPrefix => 'ARCHIVE/LM/LM_STANDING_FACILITIES',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
---------------------------------------------------------------
|
||||||
|
--Table LM_STANDING_FACILITIES_HEADER
|
||||||
|
|
||||||
|
--INBOX -For incoming files awaiting processing
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_STANDING_FACILITIES_HEADER_INBOX',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES_HEADER',
|
||||||
|
pPrefix => 'INBOX/LM/DistributeStandingFacilities/LM_STANDING_FACILITIES_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ODS - for processed files in operational data store
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_STANDING_FACILITIES_HEADER_ODS',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES_HEADER',
|
||||||
|
pPrefix => 'ODS/LM/LM_STANDING_FACILITIES_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvDataBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--ARCHIVE - For historical/archived files
|
||||||
|
BEGIN
|
||||||
|
ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
|
||||||
|
pTableName => 'LM_STANDING_FACILITIES_HEADER_ARCHIVE',
|
||||||
|
pTemplateTableName => 'CT_ET_TEMPLATES.LM_STANDING_FACILITIES_HEADER',
|
||||||
|
pPrefix => 'ARCHIVE/LM/LM_STANDING_FACILITIES_HEADER',
|
||||||
|
pBucketUri => CT_MRDS.ENV_MANAGER.gvArchiveBucketUri
|
||||||
|
);
|
||||||
|
END;
|
||||||
|
/
|
||||||
|
|
||||||
|
--=============================================================================================================================
|
||||||
|
--Step 5: Create Compatibility View
|
||||||
|
--=============================================================================================================================
|
||||||
|
|
||||||
|
--View STANDING_FACILITIES_MARS
|
||||||
|
CREATE OR REPLACE VIEW OU_LM.STANDING_FACILITIES_MARS AS
|
||||||
|
SELECT
|
||||||
|
A_KEY,
|
||||||
|
A_SFH_FK,
|
||||||
|
A_WORKFLOW_HISTORY_KEY AS A_ETL_LOAD_SET_FK,
|
||||||
|
COUNTRY,
|
||||||
|
MFI_ID,
|
||||||
|
MFI_NAME,
|
||||||
|
MARGINAL_LENDING_FACILITY,
|
||||||
|
DEPOSIT_FACILITY,
|
||||||
|
COMMENT_
|
||||||
|
FROM ODS.LM_STANDING_FACILITIES_ODS;
|
||||||
|
/
|
||||||
|
|
||||||
|
--View STANDING_FACILITIES_HEADER_MARS
|
||||||
|
CREATE OR REPLACE VIEW OU_LM.STANDING_FACILITIES_HEADER_MARS AS
|
||||||
|
SELECT
|
||||||
|
A_KEY,
|
||||||
|
A_WORKFLOW_HISTORY_KEY AS A_ETL_LOAD_SET_FK,
|
||||||
|
REV_NUMBER,
|
||||||
|
REF_DATE,
|
||||||
|
FREE_TEXT,
|
||||||
|
MLF_BS_TOTAL,
|
||||||
|
DF_BS_TOTAL,
|
||||||
|
MLF_SF_TOTAL,
|
||||||
|
DF_SF_TOTAL
|
||||||
|
FROM ODS.LM_STANDING_FACILITIES_HEADER_ODS;
|
||||||
|
/
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
SET DEFINE OFF
|
||||||
|
|
||||||
|
CREATE TABLE BKP_MRDS.CT_MRDS_A_TASK_HISTORY_MARS783 AS SELECT * FROM CT_MRDS.A_TASK_HISTORY;
|
||||||
|
|
||||||
|
SET DEFINE ON
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
SET DEFINE OFF
|
||||||
|
|
||||||
|
CREATE TABLE BKP_MRDS.CT_MRDS_A_TASK_HISTORY_SOURCE_MARS783 AS SELECT * FROM CT_MRDS.A_TASK_HISTORY_SOURCE;
|
||||||
|
|
||||||
|
SET DEFINE ON
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
SET DEFINE OFF
|
||||||
|
|
||||||
|
CREATE TABLE BKP_MRDS.CT_MRDS_A_TASK_HISTORY_TARGET_MARS783 AS SELECT * FROM CT_MRDS.A_TASK_HISTORY_TARGET;
|
||||||
|
|
||||||
|
SET DEFINE ON
|
||||||
|
|
||||||
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user