MARS-1005-PREHOOK: Fix DATA_EXPORTER RFC 4180 Compliance + Parquet Format Support

Overview

Pre-hook for MARS-1005. Deploys an updated CT_MRDS.DATA_EXPORTER package (v2.17.0) that resolves two export format bugs:

RFC 4180 compliance (v2.15.0 / v2.16.0): Previous versions used escape=true in DBMS_CLOUD.EXPORT_DATA, producing backslash-escaped embedded quotes (\"). ODS external tables (FIELDS CSV WITHOUT EMBEDDED) expect RFC 4180 doubling (""). Fix: removed escape=true; implemented REPLACE(col, '"', '""') in SELECT query.
Parquet corruption fix (v2.17.0): The RFC 4180 REPLACE was applied to all export formats, including Parquet — corrupting values that contained double-quotes. Fix: added pFormat parameter to buildQueryWithDateFormats; REPLACE is now applied only when pFormat = 'CSV'. Parquet exports pass column values unchanged.

File	Description
`install_mars1005_prehook.sql`	Master installation script (SPOOL, ACCEPT, quit)
`rollback_mars1005_prehook.sql`	Master rollback script (SPOOL, ACCEPT, quit)
`00_MARS_1005_PREHOOK_install_DATA_EXPORTER.sql`	Deploy DATA_EXPORTER v2.17.0
`90_MARS_1005_PREHOOK_rollback_DATA_EXPORTER.sql`	Restore DATA_EXPORTER v2.14.0
`track_package_versions.sql`	Universal version tracking script
`verify_packages_version.sql`	Universal package verification script
`new_version/DATA_EXPORTER.pkg`	Package specification v2.17.0
`new_version/DATA_EXPORTER.pkb`	Package body v2.17.0
`rollback_version/DATA_EXPORTER.pkg`	Package specification v2.14.0 (backup)
`rollback_version/DATA_EXPORTER.pkb`	Package body v2.14.0 (backup)
`README.md`	This file

Prerequisites

Oracle Database 23ai
CT_MRDS.ENV_MANAGER v3.1.0+
ADMIN user with EXECUTE privileges on CT_MRDS schema
Connection service: ggmichalski_high

Version Change

Package	Before	After
`CT_MRDS.DATA_EXPORTER`	v2.14.0	v2.17.0

Installation

Option 1: Master Script (Recommended)

# Execute as ADMIN user for proper privilege management
Get-Content "MARS_Packages/REL03/MARS-1005-PREHOOK/install_mars1005_prehook.sql" | sql "ADMIN/Cloudpass#34@ggmichalski_high"

Log file created automatically: log/INSTALL_MARS_1005_PREHOOK_<PDB>_<timestamp>.log

Option 2: Individual Scripts

# Execute as ADMIN user
Get-Content "MARS_Packages/REL03/MARS-1005-PREHOOK/00_MARS_1005_PREHOOK_install_DATA_EXPORTER.sql" | sql "ADMIN/Cloudpass#34@ggmichalski_high"

Verification

-- Verify package version
SELECT CT_MRDS.DATA_EXPORTER.GET_VERSION() FROM DUAL;
-- Expected: 2.17.0

-- Verify build info
SELECT CT_MRDS.DATA_EXPORTER.GET_BUILD_INFO() FROM DUAL;

-- Check for compilation errors
SELECT * FROM ALL_ERRORS
WHERE OWNER = 'CT_MRDS'
  AND NAME = 'DATA_EXPORTER';

-- Verify no untracked changes
SELECT CT_MRDS.ENV_MANAGER.CHECK_PACKAGE_CHANGES('CT_MRDS', 'DATA_EXPORTER') FROM DUAL;
-- Expected: OK

Rollback

# Execute as ADMIN user
Get-Content "MARS_Packages/REL03/MARS-1005-PREHOOK/rollback_mars1005_prehook.sql" | sql "ADMIN/Cloudpass#34@ggmichalski_high"

Rollback restores CT_MRDS.DATA_EXPORTER to v2.14.0 (re-enables escape=true).

Testing

After installation, verify the Parquet and CSV export paths:

CSV path (ODS): Export a table, then SELECT from the corresponding ODS external table. Values with embedded double-quotes should appear as single " characters and not trigger ORA-30653.

Parquet path (ARCHIVE): Export a table containing values with embedded double-quotes to the ARCHIVE bucket area. Download the Parquet file and confirm the value is stored verbatim (no extra quotes).

Expected Changes

CT_MRDS.DATA_EXPORTER: v2.14.0 → v2.17.0
No table structure changes
No configuration changes

MARS-1005: Export TOP allotment data (main issue)
CT_MRDS.DATA_EXPORTER source: MARS_Packages/mrds_elt-dev-database/mrds_elt-dev-database/database/CT_MRDS/packages/DATA_EXPORTER.sql

4.0 KiB Raw Blame History