admin/mars

Files

Grzegorz Michalski ecd833f682 Init

2026-02-02 10:59:29 +01:00

56 KiB

Raw Blame History

Package Deployment Guide - Developer Instructions

Overview

This guide provides step-by-step instructions for developers on how to properly deploy new package versions using the integrated version tracking and hash-based change detection system.

Target Audience: Database developers working with CT_MRDS packages
System: Oracle Database 23ai with ENV_MANAGER v3.1.0+
Last Updated: 2025-10-22

Before You Start
Standard Deployment Workflow
Version Update Guidelines
Deployment Scenarios
Creating MARS Installation Packages
Troubleshooting
Best Practices
Quick Reference

Before You Start

Step 0: Prepare Dedicated Working Directory and Git Branch

CRITICAL: Before starting work on any MARS package, create a dedicated working directory with its own feature branch:

# Navigate to your Git repository parent directory
cd c:\_git\_local_rep

# Clone main repository to new working directory for MARS issue
# This creates complete isolation for the feature branch
git clone working_dir_02 working_dir_02_MARS-XXXX

# Navigate to new working directory
cd working_dir_02_MARS-XXXX

# Ensure you're on main branch and it's up to date
git checkout main
git pull origin main

# Create new feature branch for MARS issue
git checkout -b feature/MARS-XXXX

# Verify you're on the new branch
git branch
# You should see: * feature/MARS-XXXX (asterisk indicates current branch)

Alternative: Git Worktree (Recommended for Advanced Users):

# Navigate to your main Git repository
cd c:\_git\_local_rep\working_dir_02

# Create new worktree with dedicated directory for feature branch
git worktree add ..\working_dir_02_MARS-XXXX feature/MARS-XXXX

# Navigate to new working directory
cd ..\working_dir_02_MARS-XXXX

# Verify you're on the new branch
git branch

Branch and Directory Naming Convention:

Branch Format: feature/MARS-XXXX (e.g., feature/MARS-1057)
Directory Format: working_dir_02_MARS-XXXX (e.g., working_dir_02_MARS-1057)
Always branch from main to ensure clean starting point
One working directory per MARS issue for complete isolation

Working Directory Structure (isolated per MARS issue):

c:\_git\_local_rep\
├── working_dir_02\              # Main working directory (main branch)
│   ├── MARS_Packages\
│   ├── database\
│   └── .git\
├── working_dir_02_MARS-1057\    # Dedicated directory for MARS-1057
│   ├── MARS_Packages\
│   │   └── REL01\
│   │       └── MARS-1057\       # New package folder
│   │           ├── rollback_version\
│   │           ├── new_version\
│   │           └── *.sql files
│   ├── database\
│   └── .git\                     # Tracks feature/MARS-1057 branch
└── working_dir_02_MARS-1058\    # Dedicated directory for MARS-1058 (parallel work)
    └── ...

Git Status Check:

# Always verify which branch and directory you're working in
pwd
# Output should show: c:\_git\_local_rep\working_dir_02_MARS-XXXX

git status
# Output should show: On branch feature/MARS-XXXX

# View all changes in current branch
git status --short

Why dedicated working directories are mandatory:

Complete Isolation: Each MARS issue has its own physical directory
Parallel Work: Work on multiple MARS issues simultaneously without conflicts
Clean Separation: No risk of mixing changes from different features
Testing Safety: Test different features independently in separate environments
Rollback Simplicity: Delete entire directory to abandon feature
Code Review: Clear separation for pull request workflow
Deployment Control: Deploy from specific working directory with confidence

Git Worktree Benefits:

Shares .git repository (saves disk space)
Faster than full clone
Automatic branch tracking
Recommended for experienced Git users

Cleanup After Merge:

# After feature branch is merged to main, clean up working directory
cd c:\_git\_local_rep

# If using git worktree
git worktree remove working_dir_02_MARS-XXXX

# If using clone, simply delete directory
Remove-Item -Recurse -Force working_dir_02_MARS-XXXX

Prerequisites

ADMIN user access - MARS package installations require ADMIN privileges for schema operations
Access to CT_MRDS schema (or appropriate schema)
SQLcl or SQL*Plus installed
Understanding of Semantic Versioning (MAJOR.MINOR.PATCH)
Package source files available locally
Git installed and configured - Required for version control workflow

CRITICAL - Data Dictionary Views:

When installing as ADMIN user, you MUST use ALL_* views instead of USER_* views for verification queries:

❌ WRONG (USER_*)	✅ CORRECT (ALL_* with OWNER filter)
`SELECT * FROM USER_ERRORS WHERE NAME = 'PKG'`	`SELECT * FROM ALL_ERRORS WHERE OWNER = 'CT_MRDS' AND NAME = 'PKG'`
`SELECT * FROM USER_OBJECTS WHERE OBJECT_NAME = 'X'`	`SELECT * FROM ALL_OBJECTS WHERE OWNER = 'CT_MRDS' AND OBJECT_NAME = 'X'`
`SELECT * FROM USER_TAB_COLUMNS WHERE TABLE_NAME = 'T'`	`SELECT * FROM ALL_TAB_COLUMNS WHERE OWNER = 'CT_MRDS' AND TABLE_NAME = 'T'`
`SELECT * FROM USER_TABLES WHERE TABLE_NAME = 'T'`	`SELECT * FROM ALL_TABLES WHERE OWNER = 'CT_MRDS' AND TABLE_NAME = 'T'`

Why this matters: USER_* views show objects in the current schema (ADMIN). When creating objects in other schemas (ODS, CT_MRDS), you must query ALL_* views with explicit OWNER filter to verify the changes.

System Components

The deployment system uses these key components:

ENV_MANAGER - Centralized version management and hash tracking
A_PACKAGE_VERSION_TRACKING - History table for all package versions
CALCULATE_PACKAGE_HASH - SHA256 hash calculation for change detection
TRACK_PACKAGE_VERSION - Records package versions with automatic change detection
CHECK_PACKAGE_CHANGES - Validates if package was modified

Standard Deployment Workflow

Step 1: Check Current Package Status

Before making any changes, verify the current state:

-- Connect to database
sql CT_MRDS/password@service_name

-- Check current version
SELECT FILE_MANAGER.GET_VERSION() FROM DUAL;

-- Check for any untracked changes
SELECT ENV_MANAGER.CHECK_PACKAGE_CHANGES('CT_MRDS', 'FILE_MANAGER') FROM DUAL;

Expected Output: OK: Package CT_MRDS.FILE_MANAGER has not changed.

If you see WARNING, it means there are untracked modifications. See Handling Untracked Changes.

Step 2: Make Your Code Changes

Edit the package SPEC and/or BODY files in your local workspace:

database/CT_MRDS/packages/FILE_MANAGER.pkg
database/CT_MRDS/packages/FILE_MANAGER.pkb

Important: Do not modify version constants yet.

Step 3: Update Version Information

Based on the type of changes, update version constants in the SPEC file:

A. Determine Version Increment

Follow Semantic Versioning rules:

MAJOR (X.0.0) - Breaking changes, incompatible API modifications
MINOR (x.Y.0) - New features, backward-compatible additions
PATCH (x.y.Z) - Bug fixes, backward-compatible corrections

B. Update Package Constants

Edit the package SPEC file and update three constants:

-- Package Version Information (Semantic Versioning: MAJOR.MINOR.PATCH)
PACKAGE_VERSION     CONSTANT VARCHAR2(10) := '3.3.0';  -- INCREMENT THIS
PACKAGE_BUILD_DATE  CONSTANT VARCHAR2(20) := '2025-10-22 21:00:00';  -- UPDATE THIS
PACKAGE_AUTHOR      CONSTANT VARCHAR2(100) := 'Your Name';  -- YOUR NAME

-- Version History (Latest changes first)
VERSION_HISTORY     CONSTANT VARCHAR2(4000) := 
   '3.3.0 (2025-10-22): Added new export procedure for historical data' || CHR(13)||CHR(10) ||  -- ADD THIS LINE
   '3.2.0 (2025-10-15): Enhanced validation with column mismatch detection' || CHR(13)||CHR(10) ||
   '3.1.0 (2025-10-01): Initial release with file processing capabilities';

Version History Format:

'VERSION (DATE): Brief description of changes' || CHR(13)||CHR(10) ||

Step 4: Deploy to Database

Deploy SPEC first, then BODY:

# Using PowerShell and SQLcl
Get-Content "database\CT_MRDS\packages\FILE_MANAGER.pkg" | sql "CT_MRDS/password@service_name"
Get-Content "database\CT_MRDS\packages\FILE_MANAGER.pkb" | sql "CT_MRDS/password@service_name"

Or using SQL*Plus:

@database/CT_MRDS/packages/FILE_MANAGER.pkg
@database/CT_MRDS/packages/FILE_MANAGER.pkb

Check for compilation errors:

-- When installing as ADMIN, check specific schema
SELECT * FROM ALL_ERRORS 
WHERE OWNER = 'CT_MRDS'
  AND TYPE IN ('PACKAGE', 'PACKAGE BODY') 
  AND NAME = 'FILE_MANAGER'
ORDER BY SEQUENCE;

Step 5: Track the New Version

After successful deployment, register the new version in the tracking system:

BEGIN
    ENV_MANAGER.TRACK_PACKAGE_VERSION(
        pPackageOwner => 'CT_MRDS',
        pPackageName => 'FILE_MANAGER',
        pPackageVersion => FILE_MANAGER.PACKAGE_VERSION,
        pPackageBuildDate => FILE_MANAGER.PACKAGE_BUILD_DATE,
        pPackageAuthor => FILE_MANAGER.PACKAGE_AUTHOR
    );
END;
/

System Response:

If version updated correctly: First tracking record or End TRACK_PACKAGE_VERSION - Record inserted
If version NOT updated: WARNING: Source code changed without version update!

Step 6: Verify Deployment

Confirm the deployment was successful:

-- 1. Check deployed version
SELECT FILE_MANAGER.GET_BUILD_INFO() FROM DUAL;

-- 2. Verify no untracked changes
SELECT ENV_MANAGER.CHECK_PACKAGE_CHANGES('CT_MRDS', 'FILE_MANAGER') FROM DUAL;

-- 3. Review tracking history
SELECT 
    PACKAGE_VERSION,
    PACKAGE_BUILD_DATE,
    DETECTED_CHANGE_WITHOUT_VERSION,
    TO_CHAR(TRACKING_DATE, 'YYYY-MM-DD HH24:MI:SS') AS TRACKED_AT
FROM A_PACKAGE_VERSION_TRACKING
WHERE PACKAGE_OWNER = 'CT_MRDS' 
  AND PACKAGE_NAME = 'FILE_MANAGER'
ORDER BY TRACKING_DATE DESC
FETCH FIRST 5 ROWS ONLY;

Step 7: Update Source Repository (CODE-FIRST Compliance)

CRITICAL: After successful database deployment, updated packages MUST be copied back to the main source repository to maintain version control integrity.

# Copy updated packages to source repository
# Replace PACKAGE_NAME with actual package name (e.g., FILE_MANAGER)

Copy-Item "database\CT_MRDS\packages\PACKAGE_NAME.pkg" `
          "MARS_Packages\mrds_elt-dev-database\mrds_elt-dev-database\database\CT_MRDS\SCHEMA\packages\PACKAGE_NAME.pkg"

Copy-Item "database\CT_MRDS\packages\PACKAGE_NAME.pkb" `
          "MARS_Packages\mrds_elt-dev-database\mrds_elt-dev-database\database\CT_MRDS\SCHEMA\packages\PACKAGE_NAME.pkb"

Why this is mandatory:

CODE-FIRST PRINCIPLE: Source files are the single source of truth
Version Control: Git tracks changes in source repository
Team Collaboration: Other developers need access to latest code
Deployment Reproducibility: Enables clean deployments to other environments
Rollback Capability: Source repository serves as backup

Directory Structure:

MARS_Packages/
└── mrds_elt-dev-database/
    └── mrds_elt-dev-database/
        └── database/
            ├── CT_MRDS/
            │   └── SCHEMA/
            │       ├── packages/
            │       │   ├── FILE_MANAGER.pkg       # ← Specification
            │       │   ├── FILE_MANAGER.pkb       # ← Body
            │       │   ├── DATA_EXPORTER.pkg
            │       │   ├── DATA_EXPORTER.pkb
            │       │   ├── ENV_MANAGER.pkg
            │       │   └── ENV_MANAGER.pkb
            │       ├── tables/
            │       ├── views/
            │       └── triggers/
            ├── ODS/
            │   └── SCHEMA/
            │       └── packages/
            │           ├── FILE_MANAGER_ODS.pkg   # ← Specification
            │           └── FILE_MANAGER_ODS.pkb   # ← Body
            └── CT_ET_TEMPLATES/
                └── SCHEMA/
                    └── tables/

Verification:

# Verify files were updated
Get-ChildItem "MARS_Packages\mrds_elt-dev-database\mrds_elt-dev-database\database\CT_MRDS\SCHEMA\packages\*.pkg", `
              "MARS_Packages\mrds_elt-dev-database\mrds_elt-dev-database\database\CT_MRDS\SCHEMA\packages\*.pkb" | 
    Select-Object Name, LastWriteTime | 
    Sort-Object LastWriteTime -Descending

Version Update Guidelines

Semantic Versioning Rules

Change Type	Version Impact	Example	Description
Breaking Change	MAJOR (3.0.0 → 4.0.0)	Changed procedure signature, removed parameter	Requires code changes in calling applications
New Feature	MINOR (3.2.0 → 3.3.0)	Added new procedure, new optional parameter	Backward compatible, adds functionality
Bug Fix	PATCH (3.2.1 → 3.2.2)	Fixed calculation error, corrected validation	No API changes, fixes existing functionality
Refactoring	PATCH (3.2.1 → 3.2.2)	Code cleanup, performance optimization	Internal changes only, no behavior change

Build Date Format

Always use ISO 8601 format with time:

PACKAGE_BUILD_DATE CONSTANT VARCHAR2(20) := 'YYYY-MM-DD HH24:MI:SS';

Example: '2025-10-22 21:00:00'

Deployment Scenarios

Scenario 1: New Feature Addition (MINOR Version)

Example: Adding new export procedure to FILE_MANAGER

Current Version: 3.2.0
New Version: 3.3.0
Changes:
- Add new procedure EXPORT_HISTORICAL_DATA to SPEC
- Implement procedure in BODY
- Update PACKAGE_VERSION from '3.2.0' to '3.3.0'
- Update PACKAGE_BUILD_DATE to current timestamp
- Add entry to VERSION_HISTORY
Deploy & Track:

-- Deploy files
@database/CT_MRDS/packages/FILE_MANAGER.pkg
@database/CT_MRDS/packages/FILE_MANAGER.pkb

-- Track new version
BEGIN
    ENV_MANAGER.TRACK_PACKAGE_VERSION(
        pPackageOwner => 'CT_MRDS',
        pPackageName => 'FILE_MANAGER',
        pPackageVersion => FILE_MANAGER.PACKAGE_VERSION,
        pPackageBuildDate => FILE_MANAGER.PACKAGE_BUILD_DATE,
        pPackageAuthor => FILE_MANAGER.PACKAGE_AUTHOR
    );
END;
/

Scenario 2: Bug Fix (PATCH Version)

Example: Fixing date format validation in DATA_EXPORTER

Current Version: 2.1.0
New Version: 2.1.1
Changes:
- Fix validation logic in BODY (no SPEC changes)
- Update PACKAGE_VERSION from '2.1.0' to '2.1.1'
- Update PACKAGE_BUILD_DATE
- Add entry to VERSION_HISTORY
Deploy & Track: Same as Scenario 1

Scenario 3: Breaking Change (MAJOR Version)

Example: Removing deprecated parameter from FILE_ARCHIVER

Current Version: 2.5.3
New Version: 3.0.0
Changes:
- Remove old parameter from procedure signature in SPEC
- Update BODY implementation
- Update PACKAGE_VERSION from '2.5.3' to '3.0.0'
- Update PACKAGE_BUILD_DATE
- Add detailed migration notes to VERSION_HISTORY
Additional Steps:
- Document breaking changes
- Notify dependent application teams
- Update API documentation

Scenario 4: Emergency Hotfix

When urgent fix is needed in production:

Identify Issue: Document the problem clearly
Create Fix: Make minimal changes to address the issue
Increment PATCH: Update to next patch version (e.g., 3.2.5 → 3.2.6)
Fast Track Deployment:

# Quick deployment script
Get-Content "database\CT_MRDS\packages\PACKAGE_NAME.pkg" | sql "CT_MRDS/password@service_name"
Get-Content "database\CT_MRDS\packages\PACKAGE_NAME.pkb" | sql "CT_MRDS/password@service_name"

# Track immediately
echo "BEGIN ENV_MANAGER.TRACK_PACKAGE_VERSION('CT_MRDS', 'PACKAGE_NAME', 'VERSION', 'BUILD_DATE', 'AUTHOR'); END;" | sql "CT_MRDS/password@service_name"

Verify & Document: Record hotfix details in change log

Troubleshooting

Issue 1: Warning - Untracked Changes Detected

Symptom:

WARNING: Source code changed without version update!
Last Version: 3.2.0
Current Version: 3.2.0
SPECIFICATION Changed: Hash mismatch detected

Cause: Code was modified but version constants were not updated.

Solution:

Update PACKAGE_VERSION to next appropriate version
Update PACKAGE_BUILD_DATE to current timestamp
Add entry to VERSION_HISTORY
Redeploy package
Call TRACK_PACKAGE_VERSION again

Issue 2: Compilation Errors After Deployment

Symptom:

Package Body compiled with errors
LINE/COL ERROR
-------- -------------
42/10    PLS-00201: identifier 'PROCEDURE_NAME' must be declared

Solution:

Check ALL_ERRORS for detailed error information:

-- Check specific schema when installing as ADMIN
SELECT LINE, POSITION, TEXT 
FROM ALL_ERRORS 
WHERE OWNER = 'CT_MRDS'  -- Replace with target schema
  AND NAME = 'PACKAGE_NAME' 
  AND TYPE = 'PACKAGE BODY'
ORDER BY SEQUENCE;

Fix errors in source file
Redeploy package
Verify compilation: SELECT * FROM ALL_ERRORS WHERE OWNER = 'CT_MRDS' AND NAME = 'PACKAGE_NAME';

Issue 3: Package State Discarded Error

Symptom:

ORA-04068: existing state of packages has been discarded
ORA-04061: existing state of package body has been invalidated

Cause: Package was recently recompiled, session state needs refresh.

Solution: Simply retry the command. This is a transient error that resolves after first execution.

Issue 4: Cannot Track Package - Not Found

Symptom:

ERROR tracking PACKAGE_NAME: Package not found in database

Solution:

Verify package exists:

-- Check specific schema when installing as ADMIN
SELECT OBJECT_NAME, OBJECT_TYPE, STATUS 
FROM ALL_OBJECTS 
WHERE OWNER = 'CT_MRDS'  -- Replace with target schema
  AND OBJECT_NAME = 'PACKAGE_NAME';

If not found, deploy package first
If status = INVALID, recompile package
Retry tracking

Creating MARS Installation Packages

Overview

MARS packages (e.g., MARS-1049, MARS-1011) are deployment packages that bundle database changes for specific features or fixes. They follow a standardized structure with logging, version tracking, and rollback capabilities.

PREREQUISITE: Before creating any MARS package, ensure you have completed Step 0: Prepare Dedicated Working Directory and Git Branch. Each MARS issue MUST have its own isolated working directory and feature branch.

CRITICAL: All MARS package installations MUST be executed as ADMIN user to ensure proper schema creation, privilege management, and cross-schema operations.

Installation User Requirements:

User: ADMIN (required for all MARS installations)
Privileges: Full DBA privileges, CREATE ANY TABLE, ALTER ANY TABLE, EXECUTE on DBMS_CLOUD
Connection: sql ADMIN/password@service_name

Package Structure

IMPORTANT: Currently, all new MARS packages are being created in the REL01 directory. This is the active release folder for ongoing development.

CRITICAL - .gitignore Configuration:

Before starting MARS package development, create .gitignore file in package root directory to exclude temporary folders from version control:

File Location: MARS_Packages/REL01/MARS-XXXX/.gitignore

Required Content:

# Exclude temporary folders from version control
confluence/
log/
test/
mock_data/

Why exclude these folders:

confluence/: Working documentation files (may contain drafts, not final versions)
log/: SPOOL log files (generated during installation, environment-specific)
test/: Test artifacts, temporary test data, and test execution logs
mock_data/: Test data files for development and validation (environment-specific)

What SHOULD be committed:

Installation scripts (01_, 02_, 91_, 92_)
Master scripts (install_.sql, rollback_.sql)
track_package_versions.sql and verify_packages_version.sql
README.md
.gitignore (with standardized exclusions)
rollback_version/ and new_version/ folders (if package modifications)
Core deployment files only

Standard MARS package directory structure:

MARS_Packages/REL01/MARS-XXXX/
├── .gitignore                      # Git exclusions (REQUIRED)
├── install_marsXXXX.sql           # Master installation script (with SPOOL to log/)
├── rollback_marsXXXX.sql          # Master rollback script (with SPOOL to log/)
├── 01_MARS_XXXX_install_*.sql     # Individual installation scripts
├── 02_MARS_XXXX_install_*.sql
├── ...
├── 91_MARS_XXXX_rollback_*.sql    # Individual rollback scripts (91-99 range)
├── 92_MARS_XXXX_rollback_*.sql
├── track_package_versions.sql      # Version tracking script
├── verify_packages_version.sql     # Package verification script
├── ...
├── README.md                       # Package documentation
├── rollback_version/                # Backup of objects BEFORE changes (for rollback)
│   ├── PACKAGE_NAME.pkg           # Previous package specification
│   └── PACKAGE_NAME.pkb           # Previous package body
├── new_version/                    # Updated objects AFTER changes (for installation)
│   ├── PACKAGE_NAME.pkg           # New package specification
│   └── PACKAGE_NAME.pkb           # New package body
├── test/                           # Test files and verification scripts
│   ├── test_marsXXXX.sql          # Unit tests
│   ├── TEST_RESULTS.md            # Test documentation
│   └── test_data_*.csv            # Test data files
├── log/                            # SPOOL log files (REQUIRED - auto-created)
│   ├── INSTALL_MARS_XXXX_*.log    # Installation logs from SPOOL
│   └── ROLLBACK_MARS_XXXX_*.log   # Rollback logs from SPOOL
└── mock_data/                      # Mock data for testing (optional)
    ├── 00_LOAD_ALL_MOCK_DATA.sql  # Master data loader
    ├── 01_load_*.sql              # Individual table loaders
    └── README.md                  # Mock data documentation

Naming Conventions:

Installation scripts: 01-89 range, executed in numerical order
Rollback scripts: 91-99 range, executed in reverse order
Verification scripts: Any number, typically after main scripts
Master scripts: install_marsXXXX.sql and rollback_marsXXXX.sql

Version Management Folders (for Database Object Modifications):

rollback_version/ - Contains backup of database objects BEFORE changes (for rollback)
new_version/ - Contains updated database objects AFTER changes (for installation)

When to use version folders:

Required when MARS package modifies ANY existing database objects (packages, tables, views, indexes, triggers, etc.)
Copy original object definitions to rollback_version/ before making changes
Place modified object definitions in new_version/ after changes
Examples: FILE_MANAGER.pkg/pkb, A_SOURCE_FILE_CONFIG.sql (table), V_STATUS.sql (view), IDX_CONFIG.sql (index)

Test Files Organization:

test/ - Contains ALL test-related files (unit tests, test data, verification scripts)
log/ - Contains ALL SPOOL log files from installation/rollback executions (REQUIRED directory)
mock_data/ - Contains mock data files for development and testing (optional)
Keep deployment scripts clean - move test artifacts to test/ subfolder after validation
Include track_package_versions.sql and verify_packages_version.sql in test/ subfolder
SPOOL log files automatically created in log/ directory by master scripts
Benefits: Clean deployment structure, easy test replication, organized documentation

Master Install Script Template

All MARS packages MUST include SPOOL logging for audit trail and debugging:

-- ===================================================================
-- MARS-XXXX INSTALL SCRIPT: Brief Description
-- ===================================================================
-- Purpose: Detailed description of what this package does
-- Author: Grzegorz Michalski
-- Date: YYYY-MM-DD
-- Version: X.Y.Z

-- Dynamic spool file generation (using SYS_CONTEXT - no DBA privileges required)
-- Log files are automatically created in log/ subdirectory
-- IMPORTANT: Ensure log/ directory exists before SPOOL (use host mkdir)
host mkdir log 2>nul

var filename VARCHAR2(100)
BEGIN
    :filename := 'log/INSTALL_MARS_XXXX_' || SYS_CONTEXT('USERENV', 'CON_NAME') || '_' || TO_CHAR(SYSDATE,'YYYYMMDD_HH24MISS') || '.log';
END;
/
column filename new_value _filename
select :filename filename from dual;
spool &_filename

SET ECHO OFF
SET TIMING ON
SET SERVEROUTPUT ON SIZE UNLIMITED
SET PAUSE OFF

-- Set current schema context (optional - use when modifying packages in specific schema)
-- ALTER SESSION SET CURRENT_SCHEMA = CT_MRDS;

PROMPT =========================================================================
PROMPT MARS-XXXX: Package Description
PROMPT =========================================================================
PROMPT 
PROMPT This script will:
PROMPT - Change 1
PROMPT - Change 2
PROMPT - Change 3
PROMPT 
PROMPT Expected Duration: X-Y minutes
PROMPT =========================================================================

-- Confirm installation with user
ACCEPT continue CHAR PROMPT 'Type YES to continue with installation, or Ctrl+C to abort: '
WHENEVER SQLERROR EXIT SQL.SQLCODE
BEGIN
    IF '&continue' IS NULL OR TRIM('&continue') IS NULL OR UPPER(TRIM('&continue')) != 'YES' THEN
        RAISE_APPLICATION_ERROR(-20001, 'Installation aborted by user');
    END IF;
END;
/
WHENEVER SQLERROR CONTINUE

PROMPT 
PROMPT =========================================================================
PROMPT Step 1: First Installation Step
PROMPT =========================================================================
@@01_MARS_XXXX_install_step1.sql

PROMPT 
PROMPT =========================================================================
PROMPT Step 2: Second Installation Step
PROMPT =========================================================================
@@02_MARS_XXXX_install_step2.sql

-- ... more steps ...

PROMPT 
PROMPT =========================================================================
PROMPT MARS-XXXX Installation - COMPLETED
PROMPT =========================================================================
PROMPT Check the log file for complete installation details.
PROMPT =========================================================================

spool off

quit;

Key Elements:

Log Directory Creation: host mkdir log 2>nul - Creates log/ directory before SPOOL (suppresses error if exists)
Dynamic Log Name: log/INSTALL_MARS_XXXX_<PDB_NAME>_YYYYMMDD_HH24MISS.log (stored in log/ subdirectory)
SYS_CONTEXT Usage: Uses SYS_CONTEXT('USERENV', 'CON_NAME') instead of DBA_PDBS - no DBA privileges required
SPOOL START: After filename generation, before first PROMPT
SPOOL OFF: At the very end of the script
SET ECHO OFF: Prevents double output of PROMPT commands (shows only results, not the SQL commands themselves)
ACCEPT Validation: User confirmation required before execution (safety feature)
ALTER SESSION SET CURRENT_SCHEMA (optional): Sets working schema for package operations
@@ Includes: All sub-scripts executed via @@ command
quit;: Exits SQLcl/SQL*Plus after completion (important for automated deployments)

SET ECHO OFF Benefits:

Clean Output: PROMPT messages appear only once in console and log files
Readability: Log files are easier to read without SQL command echo
Professional: Produces cleaner, more professional-looking installation logs
Consistency: Works the same in both SQLcl and SQL*Plus
Sub-Scripts: Must NOT include SET ECHO ON - they inherit setting from master script

ACCEPT Validation Benefits:

Safety Check: Prevents accidental execution - requires explicit "YES" confirmation
User Control: User must type YES (case-insensitive) to proceed with changes
Abort Capability: Any other input or Ctrl+C safely aborts before database modifications
SQLcl & SQL*Plus Compatible: Works identically in both tools, unlike PAUSE
Best Practice: Mandatory for both installation and rollback master scripts

Log File Storage:

Directory: All SPOOL logs automatically created in log/ subdirectory
Naming Convention: log/INSTALL_MARS_XXXX_<PDB>_<timestamp>.log or log/ROLLBACK_MARS_XXXX_<PDB>_<timestamp>.log
Auto-Creation: Directory created by host mkdir log 2>nul command before SPOOL (MANDATORY)
Error Suppression: 2>nul suppresses "directory already exists" errors in PowerShell/Windows
Version Control: Excluded via .gitignore (environment-specific files)
Importance: Without host mkdir, SPOOL may fail if log/ directory doesn't exist

Why host mkdir log 2>nul is mandatory:

Reliability: Ensures log/ directory exists before SPOOL attempts to write
Cross-Environment: Works on fresh Git clones where log/ doesn't exist yet
No Errors: Silently succeeds whether directory exists or not
Best Practice: Prevents installation failures due to missing log directory
PowerShell Compatible: Works in both PowerShell (pwsh.exe) and Windows CMD

quit;: Exits SQLcl/SQL*Plus after completion (important for automated deployments)

SET ECHO OFF Benefits:

Clean Output: PROMPT messages appear only once in console and log files
Readability: Log files are easier to read without SQL command echo
Professional: Produces cleaner, more professional-looking installation logs
Consistency: Works the same in both SQLcl and SQL*Plus
Sub-Scripts: Must NOT include SET ECHO ON - they inherit setting from master script

ACCEPT Validation Benefits:

Safety Check: Prevents accidental execution - requires explicit "YES" confirmation
User Control: User must type YES (case-insensitive) to proceed with changes
Abort Capability: Any other input or Ctrl+C safely aborts before database modifications
SQLcl & SQL*Plus Compatible: Works identically in both tools, unlike PAUSE
Best Practice: Mandatory for both installation and rollback master scripts

SPOOL Logging Benefits

Why SPOOL is mandatory:

Audit Trail - Complete record of all installation activities
Debugging - Capture errors, warnings, and execution details
Compliance - Required for production deployments
Troubleshooting - Review what actually happened during installation
Documentation - Proof of successful deployment

Log File Contains:

All PROMPT messages
SQL execution results
Error messages (ORA-XXXXX)
Timing information
Package compilation status
Version tracking confirmations

Example Log File Name:

log/INSTALL_MARS_1049_ggmichalski_20251126_143052.log

Log Location: log/ subdirectory in MARS package root (auto-created by SPOOL)

Creating Individual Installation Scripts

Each numbered script should be self-contained and focused:

Example: 01_MARS_1049_install_CT_MRDS_ADD_ENCODING_COLUMN.sql

--=============================================================================================================================
-- MARS-1049: Add ENCODING Column to A_SOURCE_FILE_CONFIG Table
--=============================================================================================================================
-- Purpose: Add VARCHAR2(50) ENCODING column with UTF8 default
-- Author: Grzegorz Michalski
-- Date: 2025-11-24
-- Related: MARS-1049 CSV Encoding Support
--=============================================================================================================================

SET SERVEROUTPUT ON

PROMPT ========================================================================
PROMPT Adding ENCODING column to CT_MRDS.A_SOURCE_FILE_CONFIG
PROMPT ========================================================================

-- Add column
ALTER TABLE CT_MRDS.A_SOURCE_FILE_CONFIG 
ADD (ENCODING VARCHAR2(50) DEFAULT 'UTF8');

-- Add comment
COMMENT ON COLUMN CT_MRDS.A_SOURCE_FILE_CONFIG.ENCODING IS 
'Character encoding for CSV files (e.g., UTF8, WE8MSWIN1252)';

-- Verify change (check specific schema when installing as ADMIN)
SELECT COUNT(*) AS ENCODING_COLUMN_EXISTS
FROM ALL_TAB_COLUMNS 
WHERE OWNER = 'CT_MRDS'
  AND TABLE_NAME = 'A_SOURCE_FILE_CONFIG' 
  AND COLUMN_NAME = 'ENCODING';

PROMPT SUCCESS: ENCODING column added to A_SOURCE_FILE_CONFIG

--=============================================================================================================================
-- End of Script
--=============================================================================================================================

Script Best Practices:

Clear header with purpose and metadata
SET SERVEROUTPUT ON for debugging
Descriptive PROMPT messages
Verification queries after changes
Success confirmation message

Rollback Script Template

Always provide rollback capability:

-- ===================================================================
-- MARS-XXXX ROLLBACK SCRIPT: Brief Description
-- ===================================================================
-- Purpose: Rollback all changes from MARS-XXXX installation
-- Author: Grzegorz Michalski
-- Date: YYYY-MM-DD

-- Dynamic spool file generation (using SYS_CONTEXT - no DBA privileges required)
-- IMPORTANT: Ensure log/ directory exists before SPOOL (use host mkdir)
host mkdir log 2>nul

var filename VARCHAR2(100)
BEGIN
    :filename := 'log/ROLLBACK_MARS_XXXX_' || SYS_CONTEXT('USERENV', 'CON_NAME') || '_' || TO_CHAR(SYSDATE,'YYYYMMDD_HH24MISS') || '.log';
END;
/
column filename new_value _filename
select :filename filename from dual;
spool &_filename

SET ECHO OFF
SET TIMING ON
SET SERVEROUTPUT ON SIZE UNLIMITED
SET PAUSE OFF

PROMPT =========================================================================
PROMPT MARS-XXXX: Rollback Package
PROMPT =========================================================================
PROMPT WARNING: This will reverse all changes from MARS-XXXX installation!
PROMPT =========================================================================

-- Confirm rollback with user
ACCEPT continue CHAR PROMPT 'Type YES to continue with rollback, or Ctrl+C to abort: '
WHENEVER SQLERROR EXIT SQL.SQLCODE
BEGIN
    IF '&continue' IS NULL OR TRIM('&continue') IS NULL OR UPPER(TRIM('&continue')) != 'YES' THEN
        RAISE_APPLICATION_ERROR(-20001, 'Rollback aborted by user');
    END IF;
END;
/
WHENEVER SQLERROR CONTINUE

-- Execute rollback scripts in REVERSE order
@@92_MARS_XXXX_rollback_step2.sql
@@91_MARS_XXXX_rollback_step1.sql

PROMPT 
PROMPT =========================================================================
PROMPT MARS-XXXX Rollback - COMPLETED
PROMPT =========================================================================

spool off

Rollback Principles:

Execute scripts in REVERSE order (92, 91, not 91, 92)
Undo changes from newest to oldest
Restore previous package versions
Remove added columns/tables
Revert configuration changes
Include verification step after rollback (@@verify_packages_version.sql)

Version Tracking Integration

Every MARS package should track package versions using the universal tracking script:

Recommended Approach: Universal Tracking Script

-- ===================================================================
-- Simple Package Version Tracking Script
-- ===================================================================
-- Purpose: Track specified Oracle package versions
-- Author: Grzegorz Michalski  
-- Date: 2025-11-26
-- Version: 3.1.0 - List-Based Edition
--
-- USAGE:
-- 1. Edit package list below (add/remove packages as needed)
-- 2. Include in your install/rollback script: @@track_package_versions.sql
-- ===================================================================

SET SERVEROUTPUT ON;

DECLARE
   TYPE t_package_rec IS RECORD (
      owner VARCHAR2(50),
      package_name VARCHAR2(50),
      version VARCHAR2(50)
   );
   TYPE t_packages IS TABLE OF t_package_rec;
   TYPE t_string_array IS TABLE OF VARCHAR2(100);
   
   -- ===================================================================
   -- PACKAGE LIST - Edit this array to specify packages to track
   -- ===================================================================
   -- Add or remove entries as needed for your MARS issue
   -- Format: 'SCHEMA.PACKAGE_NAME'
   -- ===================================================================
   vPackageList t_string_array := t_string_array(
      'CT_MRDS.FILE_MANAGER',
      'ODS.FILE_MANAGER_ODS'
   );
   -- ===================================================================
   
   vPackages t_packages := t_packages();
   vVersion VARCHAR2(50);
   vCount NUMBER := 0;
   vOwner VARCHAR2(50);
   vPackageName VARCHAR2(50);
   vDotPos NUMBER;
BEGIN
   DBMS_OUTPUT.PUT_LINE('========================================');
   DBMS_OUTPUT.PUT_LINE('Package Version Tracking');
   DBMS_OUTPUT.PUT_LINE('========================================');
   
   -- Process each package in the list
   FOR i IN 1..vPackageList.COUNT LOOP
      vDotPos := INSTR(vPackageList(i), '.');
      IF vDotPos > 0 THEN
         vOwner := SUBSTR(vPackageList(i), 1, vDotPos - 1);
         vPackageName := SUBSTR(vPackageList(i), vDotPos + 1);
         
         -- Get package version
         BEGIN
            EXECUTE IMMEDIATE 'SELECT ' || vOwner || '.' || vPackageName || '.GET_VERSION() FROM DUAL' INTO vVersion;
            vPackages.EXTEND;
            vPackages(vPackages.COUNT).owner := vOwner;
            vPackages(vPackages.COUNT).package_name := vPackageName;
            vPackages(vPackages.COUNT).version := vVersion;
            
            -- Track in ENV_MANAGER
            BEGIN
               CT_MRDS.ENV_MANAGER.TRACK_PACKAGE_VERSION(
                  pPackageOwner => vOwner,
                  pPackageName => vPackageName,
                  pPackageVersion => vVersion,
                  pPackageBuildDate => TO_CHAR(SYSDATE, 'YYYY-MM-DD HH24:MI:SS'),
                  pPackageAuthor => 'Grzegorz Michalski'
               );
               vCount := vCount + 1;
            EXCEPTION
               WHEN OTHERS THEN NULL; -- Continue even if tracking fails
            END;
         EXCEPTION
            WHEN OTHERS THEN NULL; -- Skip packages that fail
         END;
      END IF;
   END LOOP;
   
   -- Display results
   IF vPackages.COUNT > 0 THEN
      DBMS_OUTPUT.PUT_LINE('Packages tracked: ' || vCount || ' of ' || vPackages.COUNT);
      DBMS_OUTPUT.PUT_LINE('');
      
      FOR i IN 1..vPackages.COUNT LOOP
         DBMS_OUTPUT.PUT_LINE(vPackages(i).owner || '.' || vPackages(i).package_name || ' = ' || vPackages(i).version);
      END LOOP;
   ELSE
      DBMS_OUTPUT.PUT_LINE('No packages found in list');
   END IF;
   
   DBMS_OUTPUT.PUT_LINE('========================================');
END;
/

Key Benefits:

List-Based: Simply edit the vPackageList array to add/remove packages
Automatic Version Detection: Calls GET_VERSION() for each package
Error Tolerant: Continues even if individual packages fail
Summary Output: Shows which packages were successfully tracked
Reusable: Same script can be used across all MARS packages

Package Verification Script

Every MARS package should include a universal verification script to check all tracked packages for untracked changes:

Recommended: verify_packages_version.sql

-- ===================================================================
-- Universal Package Version Verification Script
-- ===================================================================
-- Purpose: Verify all tracked Oracle packages for code changes
-- Author: Grzegorz Michalski  
-- Date: 2025-11-25
-- Version: 1.0.0
--
-- USAGE:
-- Include at the end of install/rollback scripts: @@verify_packages_version.sql
--
-- OUTPUT:
-- - List of all tracked packages with their current status
-- - OK: Package has not changed since last tracking
-- - WARNING: Package code changed without version update
-- ===================================================================

SET LINESIZE 200
SET PAGESIZE 1000
SET FEEDBACK OFF

PROMPT
PROMPT ========================================
PROMPT Package Version Verification
PROMPT ========================================
PROMPT

COLUMN PACKAGE_OWNER FORMAT A15
COLUMN PACKAGE_NAME FORMAT A20
COLUMN VERSION FORMAT A10
COLUMN STATUS FORMAT A80

SELECT 
   PACKAGE_OWNER,
   PACKAGE_NAME,
   PACKAGE_VERSION AS VERSION,
   CT_MRDS.ENV_MANAGER.CHECK_PACKAGE_CHANGES(PACKAGE_OWNER, PACKAGE_NAME) AS STATUS
FROM (
   SELECT 
      PACKAGE_OWNER, 
      PACKAGE_NAME,
      PACKAGE_VERSION,
      ROW_NUMBER() OVER (PARTITION BY PACKAGE_OWNER, PACKAGE_NAME ORDER BY TRACKING_DATE DESC) AS RN
   FROM CT_MRDS.A_PACKAGE_VERSION_TRACKING
)
WHERE RN = 1
ORDER BY PACKAGE_OWNER, PACKAGE_NAME;

PROMPT
PROMPT ========================================
PROMPT Verification Complete
PROMPT ========================================
PROMPT
PROMPT Legend:
PROMPT   OK - Package has not changed since last tracking
PROMPT   WARNING - Package code changed without version update
PROMPT
PROMPT For detailed hash information, use:
PROMPT   SELECT ENV_MANAGER.GET_PACKAGE_HASH_INFO('OWNER', 'PACKAGE') FROM DUAL;
PROMPT ========================================

SET FEEDBACK ON

Key Features:

Universal: Checks ALL packages in A_PACKAGE_VERSION_TRACKING table
Latest Version: Uses ROW_NUMBER() to get most recent tracking record for each package
Status Display: Shows OK or WARNING for each package
Formatted Output: Clean, readable table format with column alignment
Reusable: Same script works for all MARS packages without modification
Legend: Includes helpful legend explaining OK vs WARNING status

Usage in Master Scripts:

PROMPT
PROMPT === Step 3: Verify all tracked packages ===
@@verify_packages_version.sql

Benefits over Package-Specific Verification:

No need to edit script for each MARS package
Automatically detects all tracked packages
Consistent verification across all installations
Easier to maintain (single script for all packages)

Package README Template

Every MARS package MUST include README.md:

# MARS-XXXX: Brief Description

## Overview
Detailed description of what this package does and why it's needed.

## Contents
- `install_marsXXXX.sql` - Master installation script with SPOOL logging
- `rollback_marsXXXX.sql` - Master rollback script
- `01_MARS_XXXX_*.sql` - Individual installation scripts
- `91_MARS_XXXX_*.sql` - Individual rollback scripts

## Prerequisites
- Oracle Database 23ai
- Required package versions (e.g., ENV_MANAGER v3.1.0+)
- Schema privileges needed

## Installation

### Option 1: Master Script (Recommended)
```powershell
# IMPORTANT: Execute as ADMIN user for proper privilege management
Get-Content "MARS_Packages/REL01/MARS-XXXX/install_marsXXXX.sql" | sql "ADMIN/password@service"

# Log file created: INSTALL_MARS_XXXX_<PDB>_<timestamp>.log

Option 2: Individual Scripts

# IMPORTANT: Execute as ADMIN user
Get-Content "01_MARS_XXXX_*.sql" | sql "ADMIN/password@service"
Get-Content "02_MARS_XXXX_*.sql" | sql "ADMIN/password@service"
# ... etc

Verification

-- Verify package versions
SELECT PACKAGE_NAME.GET_VERSION() FROM DUAL;

-- Check for errors (ADMIN user checks specific schema)
SELECT * FROM ALL_ERRORS 
WHERE OWNER = 'CT_MRDS'  -- Replace with target schema
  AND NAME = 'PACKAGE_NAME';

Rollback

# IMPORTANT: Execute as ADMIN user
Get-Content "MARS_Packages/REL01/MARS-XXXX/rollback_marsXXXX.sql" | sql "ADMIN/password@service"

Expected Changes

Package X: v1.0.0 → v1.1.0
Table Y: Added COLUMN_Z
Configuration: Updated DEFAULT_VALUE

Testing

Describe how to test the changes after installation.

Known Issues

List any known limitations or issues.

MARS-YYYY: Related package
Confluence: Link to detailed documentation


---

### MARS Package Checklist

Before creating a MARS package, ensure:

- [ ] **Directory Structure**: Created `log/` and `mock_data/` subdirectories (if needed)
- [ ] **.gitignore file**: Created with standardized exclusions (confluence/, log/, test/, mock_data/)
- [ ] **Author field**: All scripts have `Author: Grzegorz Michalski`
- [ ] **ADMIN user**: All installation instructions specify ADMIN user connection
- [ ] All individual scripts numbered correctly (01-89, 91-99)
- [ ] Master install script includes SPOOL logging to `log/` directory
- [ ] Master rollback script includes SPOOL logging to `log/` directory
- [ ] SPOOL filenames use dynamic PDB name and timestamp with `log/` prefix
- [ ] **ACCEPT validation**: Master scripts require explicit "YES" confirmation
- [ ] **quit; command**: Master scripts end with quit; for clean exit
- [ ] All scripts have proper headers with metadata
- [ ] Version tracking scripts included (in test/ subfolder)
- [ ] Verification scripts confirm changes (in test/ subfolder)
- [ ] README.md documents installation process
- [ ] Rollback scripts reverse all changes
- [ ] Rollback includes verification step (@@test/verify_packages_version.sql)
- [ ] **rollback_version/ folder**: Contains backup of objects before changes (if modifying packages)
- [ ] **new_version/ folder**: Contains updated objects after changes (if modifying packages)
- [ ] **test/ folder**: All test files and verification scripts organized in subfolder
- [ ] **log/ folder**: SPOOL log files automatically created by master scripts
- [ ] **mock_data/ folder**: Mock data files organized (if applicable)
- [ ] Tested in DEV environment
- [ ] Log files reviewed for errors (in log/ subdirectory)
- [ ] **Source repository updated**: Changed packages copied to `MARS_Packages/mrds_elt-dev-database/mrds_elt-dev-database/database/`
- [ ] Git commit with descriptive message (log/, mock_data/ excluded via .gitignore)

---

### Example 1: Complete MARS-1046 Package (Package Modification)

Real-world example from MARS-1046 (ISO 8601 Date Format Fix):

**Structure:**

MARS_Packages/REL01/MARS-1046/ ├── install_mars1046.sql # Master with SPOOL, ACCEPT, quit; ├── rollback_mars1046.sql # Master rollback with ACCEPT, quit; ├── 01_MARS_1046_install_CT_MRDS_FILE_MANAGER_SPEC.sql ├── 02_MARS_1046_install_CT_MRDS_FILE_MANAGER_BODY.sql ├── 91_MARS_1046_rollback_CT_MRDS_FILE_MANAGER_BODY.sql ├── 92_MARS_1046_rollback_CT_MRDS_FILE_MANAGER_SPEC.sql ├── track_package_versions.sql # Version tracking ├── verify_packages_version.sql # Package verification ├── README.md # Comprehensive documentation ├── rollback_version/ # v3.3.0 backup for rollback │ ├── FILE_MANAGER.pkg │ └── FILE_MANAGER.pkb ├── new_version/ # v3.3.1 updated packages │ ├── FILE_MANAGER.pkg │ └── FILE_MANAGER.pkb ├── test/ # All test artifacts │ ├── test_mars1046.sql # Unit tests (7/7 passed) │ ├── TEST_RESULTS.md # Test documentation │ ├── iso8601_test_data.csv # Test data (full format) │ ├── iso8601_test_data_simple.csv # Test data (simple format) │ ├── create_iso8601_test_template.sql # Template table creation │ ├── configure_iso8601_test.sql # Date format configuration │ ├── process_iso8601_test.sql # E2E test │ └── INSTALL_MARS_1046_*.log # Log files (3 iterations) └── log/ # Optional log storage


**Key Features:**
- SPOOL logging: `INSTALL_MARS_1046_<PDB>_<timestamp>.log`
- ACCEPT validation: Requires explicit "YES" to proceed (both install & rollback)
- quit; command: Clean SQLcl exit after completion
- Version folders: rollback_version/ (v3.3.0) and new_version/ (v3.3.1)
- Test subfolder: All test files organized separately
- 4-step workflow: Install packages → Track version → Verify packages → quit
- Full rollback: Restore to previous version with verification
- Comprehensive README: Problem, solution, testing, verification
- Track & verify scripts: Universal scripts work for any package

---

### Example 2: Complete MARS-1049-ADHOC Package (Configuration Change)

Real-world example from MARS-1049-ADHOC (CSDB External Tables):

**Structure:**

MARS_Packages/REL01/MARS-1049-ADHOC/ ├── install_mars1049_adhoc.sql # Master with SPOOL ├── 00_MARS_1049_ADHOC_update_encoding.sql # Update config ├── 01_MARS_1049_ADHOC_drop_external_tables.sql # Drop tables ├── 02_MARS_1049_ADHOC_recreate_external_tables.sql # Recreate with encoding ├── 03_MARS_1049_ADHOC_verify_external_tables.sql # Verify creation └── README.md # Documentation


**Key Features:**
- SPOOL logging: `INSTALL_MARS_1049_ADHOC_<PDB>_<timestamp>.log`
- Numbered sequence: 00 → 01 → 02 → 03
- No rollback (external tables are metadata-only)
- No version folders (no package modifications)
- Verification included
- Clear README with installation options

---

## Post-Installation File Organization

### Organizing Test Files and Logs

After successful installation and verification, organize files for clean Git commit:

**Step 1: Create test/ subfolder**
```powershell
# Create test directory if it doesn't exist
New-Item -ItemType Directory -Force -Path "test" | Out-Null

Step 2: Move test-related files

# Move test scripts and data
Move-Item -Path "test_*.sql" -Destination "test" -Force
Move-Item -Path "*_test_*.csv" -Destination "test" -Force
Move-Item -Path "TEST_RESULTS.md" -Destination "test" -Force

Step 3: Move log files

# Option 1: Move to test/ subfolder
Move-Item -Path "*.log" -Destination "test" -Force

# Option 2: Create dedicated log/ subfolder
New-Item -ItemType Directory -Force -Path "log" | Out-Null
Move-Item -Path "*.log" -Destination "log" -Force

Step 4: Verify clean structure

# Check root directory - should only have deployment scripts and README
Get-ChildItem | Where-Object { $_.Name -notmatch "^(01|02|91|92|install|rollback|README|rollback_version|new_version|test|log)" }
# Expected: No output (all non-deployment files moved)

Final MARS Package Structure:

MARS_Packages/REL01/MARS-XXXX/
├── install_marsXXXX.sql           # Master installation
├── rollback_marsXXXX.sql          # Master rollback
├── 01_MARS_XXXX_*.sql             # Installation scripts
├── 02_MARS_XXXX_*.sql
├── 91_MARS_XXXX_*.sql             # Rollback scripts
├── 92_MARS_XXXX_*.sql
├── track_package_versions.sql
├── verify_packages_version.sql
├── README.md                       # Documentation
├── rollback_version/               # Backup packages
├── new_version/                    # Updated packages
└── test/                           # ALL test artifacts
    ├── test_marsXXXX.sql
    ├── TEST_RESULTS.md
    ├── *.csv (test data)

Benefits:

Clean Git commits: Only deployment-relevant files in root
Easy testing: All test materials in one place
Reproducibility: Test setup can be recreated from test/ folder
Documentation: Log files preserved for troubleshooting
Professional: Organized structure for code review

Best Practices

1. Always Update Version Before Deployment

DO:

-- Update version first, then deploy
PACKAGE_VERSION := '3.3.0';  -- Updated
PACKAGE_BUILD_DATE := '2025-10-22 21:00:00';  -- Updated
VERSION_HISTORY := '3.3.0 (2025-10-22): New feature...' || CHR(13)||CHR(10) || ...

DON'T:

-- Deploy changes without updating version
-- This triggers "untracked changes" warning

2. Use Descriptive Version History

DO:

'3.3.0 (2025-10-22): Added EXPORT_HISTORICAL_DATA procedure for Parquet exports with Hive partitioning'

DON'T:

'3.3.0 (2025-10-22): Updates'  -- Too vague

3. Deploy SPEC Before BODY

Correct Order:

Deploy PACKAGE.pkg (specification)
Deploy PACKAGE.pkb (body)
Track version

Rationale: BODY depends on SPEC, deploying SPEC first ensures clean compilation.

4. Test Before Production Deployment

Deploy to DEV environment first
Run unit tests
Verify functionality
Check for performance issues
Deploy to PROD only after validation

5. Document Breaking Changes

For MAJOR version increments, always document:

What changed
Why it changed
Migration path for existing code
Example of old vs new usage

6. Use Batch Deployment Scripts

For multiple packages, create deployment script:

# deploy_all_packages.ps1
$packages = @('ENV_MANAGER', 'FILE_MANAGER', 'DATA_EXPORTER', 'FILE_ARCHIVER')

foreach ($pkg in $packages) {
    Write-Host "Deploying $pkg..."
    Get-Content "database\CT_MRDS\packages\${pkg}.pkg" | sql "CT_MRDS/password@service"
    Get-Content "database\CT_MRDS\packages\${pkg}.pkb" | sql "CT_MRDS/password@service"
}

# Track all packages
Get-Content "database\scripts\track_all_packages.sql" | sql "CT_MRDS/password@service"

7. Regular Change Detection Audits

Run periodic checks to catch untracked modifications:

-- Check all packages for changes
@database/scripts/check_package_changes_all.sql

Schedule this as part of weekly code review process.

Quick Reference

Essential Commands

Task	Command
Check current version	`SELECT PACKAGE_NAME.GET_VERSION() FROM DUAL;`
Check build info	`SELECT PACKAGE_NAME.GET_BUILD_INFO() FROM DUAL;`
Check for changes	`SELECT ENV_MANAGER.CHECK_PACKAGE_CHANGES('OWNER', 'PACKAGE') FROM DUAL;`
Get hash info	`SELECT ENV_MANAGER.GET_PACKAGE_HASH_INFO('OWNER', 'PACKAGE') FROM DUAL;`
Track version	`CALL ENV_MANAGER.TRACK_PACKAGE_VERSION('OWNER', 'PKG', 'VER', 'DATE', 'AUTHOR');`
Check errors	`SELECT * FROM ALL_ERRORS WHERE OWNER = 'CT_MRDS' AND NAME = 'PACKAGE_NAME';`
View tracking history	`SELECT * FROM A_PACKAGE_VERSION_TRACKING WHERE PACKAGE_NAME = 'X' ORDER BY TRACKING_DATE DESC;`

Version Number Examples

Current	Bug Fix	Feature	Breaking
1.0.0	1.0.1	1.1.0	2.0.0
2.5.3	2.5.4	2.6.0	3.0.0
3.2.0	3.2.1	3.3.0	4.0.0

Deployment Checklist

Check current package status (CHECK_PACKAGE_CHANGES)
Make code changes in local files
Update PACKAGE_VERSION constant (increment appropriately)
Update PACKAGE_BUILD_DATE constant (current timestamp)
Add entry to VERSION_HISTORY constant
Deploy SPEC file
Deploy BODY file
Check for compilation errors (ALL_ERRORS with OWNER filter when using ADMIN)
Track new version (TRACK_PACKAGE_VERSION)
Verify deployment (GET_BUILD_INFO, CHECK_PACKAGE_CHANGES)
Update source repository - Copy updated packages to MARS_Packages/mrds_elt-dev-database/mrds_elt-dev-database/database/
Update documentation (if API changed)
Commit changes to Git
Notify team (if breaking changes)

Package Versioning Guide - Overview of versioning system
Package Hash Tracking Guide - Technical details of hash system
Package Hash Quick Guide - Quick reference for hash commands
Developer Quick Guide - General development guidelines

Support

For questions or issues with the deployment process:

Check Troubleshooting section above
Review package documentation: SELECT GET_PACKAGE_DOCUMENTATION('PACKAGE_NAME', 'SCHEMA') FROM DUAL;
Check process logs: SELECT * FROM A_PROCESS_LOG WHERE LOG_TIMESTAMP > SYSDATE - 1 ORDER BY LOG_TIMESTAMP DESC;
Contact database team lead

Last Updated: 2025-10-22
Document Version: 1.0.0
System Version: ENV_MANAGER 3.1.0+

56 KiB Raw Blame History

Package Deployment Guide - Developer Instructions

Overview

Table of Contents

Before You Start

Step 0: Prepare Dedicated Working Directory and Git Branch

Prerequisites

System Components

Standard Deployment Workflow

Step 1: Check Current Package Status

Step 2: Make Your Code Changes

Step 3: Update Version Information

A. Determine Version Increment

B. Update Package Constants

Step 4: Deploy to Database

Step 5: Track the New Version

Step 6: Verify Deployment

Step 7: Update Source Repository (CODE-FIRST Compliance)

Version Update Guidelines

Semantic Versioning Rules

Build Date Format

Deployment Scenarios

Scenario 1: New Feature Addition (MINOR Version)

Scenario 2: Bug Fix (PATCH Version)

Scenario 3: Breaking Change (MAJOR Version)

Scenario 4: Emergency Hotfix

Troubleshooting

Issue 1: Warning - Untracked Changes Detected

Issue 2: Compilation Errors After Deployment

Issue 3: Package State Discarded Error

Issue 4: Cannot Track Package - Not Found

Creating MARS Installation Packages

Overview

Package Structure

Master Install Script Template

SPOOL Logging Benefits

Creating Individual Installation Scripts

Rollback Script Template

Version Tracking Integration

Package Verification Script

Package README Template

Option 2: Individual Scripts

Verification

Rollback

Expected Changes

Testing

Known Issues

Related

Best Practices

1. Always Update Version Before Deployment

2. Use Descriptive Version History

3. Deploy SPEC Before BODY

4. Test Before Production Deployment

5. Document Breaking Changes

6. Use Batch Deployment Scripts

7. Regular Change Detection Audits

Quick Reference

Essential Commands

Version Number Examples

Deployment Checklist

Related Documentation

Support

56 KiB

Raw Blame History