Files

MARS-1005-PREHOOK: Fix DATA_EXPORTER RFC 4180 Compliance + Parquet Format Support

Overview

Pre-hook for MARS-1005. Deploys an updated CT_MRDS.DATA_EXPORTER package (v2.17.0) that resolves two export format bugs:

  1. RFC 4180 compliance (v2.15.0 / v2.16.0): Previous versions used escape=true in DBMS_CLOUD.EXPORT_DATA, producing backslash-escaped embedded quotes (\"). ODS external tables (FIELDS CSV WITHOUT EMBEDDED) expect RFC 4180 doubling (""). Fix: removed escape=true; implemented REPLACE(col, '"', '""') in SELECT query.

  2. Parquet corruption fix (v2.17.0): The RFC 4180 REPLACE was applied to all export formats, including Parquet — corrupting values that contained double-quotes. Fix: added pFormat parameter to buildQueryWithDateFormats; REPLACE is now applied only when pFormat = 'CSV'. Parquet exports pass column values unchanged.

Contents

File Description
install_mars1005_prehook.sql Master installation script (SPOOL, ACCEPT, quit)
rollback_mars1005_prehook.sql Master rollback script (SPOOL, ACCEPT, quit)
00_MARS_1005_PREHOOK_install_DATA_EXPORTER.sql Deploy DATA_EXPORTER v2.17.0
90_MARS_1005_PREHOOK_rollback_DATA_EXPORTER.sql Restore DATA_EXPORTER v2.14.0
track_package_versions.sql Universal version tracking script
verify_packages_version.sql Universal package verification script
new_version/DATA_EXPORTER.pkg Package specification v2.17.0
new_version/DATA_EXPORTER.pkb Package body v2.17.0
rollback_version/DATA_EXPORTER.pkg Package specification v2.14.0 (backup)
rollback_version/DATA_EXPORTER.pkb Package body v2.14.0 (backup)
README.md This file

Prerequisites

  • Oracle Database 23ai
  • CT_MRDS.ENV_MANAGER v3.1.0+
  • ADMIN user with EXECUTE privileges on CT_MRDS schema
  • Connection service: ggmichalski_high

Version Change

Package Before After
CT_MRDS.DATA_EXPORTER v2.14.0 v2.17.0

Installation

# Execute as ADMIN user for proper privilege management
Get-Content "MARS_Packages/REL03/MARS-1005-PREHOOK/install_mars1005_prehook.sql" | sql "ADMIN/Cloudpass#34@ggmichalski_high"

Log file created automatically: log/INSTALL_MARS_1005_PREHOOK_<PDB>_<timestamp>.log

Option 2: Individual Scripts

# Execute as ADMIN user
Get-Content "MARS_Packages/REL03/MARS-1005-PREHOOK/00_MARS_1005_PREHOOK_install_DATA_EXPORTER.sql" | sql "ADMIN/Cloudpass#34@ggmichalski_high"

Verification

-- Verify package version
SELECT CT_MRDS.DATA_EXPORTER.GET_VERSION() FROM DUAL;
-- Expected: 2.17.0

-- Verify build info
SELECT CT_MRDS.DATA_EXPORTER.GET_BUILD_INFO() FROM DUAL;

-- Check for compilation errors
SELECT * FROM ALL_ERRORS
WHERE OWNER = 'CT_MRDS'
  AND NAME = 'DATA_EXPORTER';

-- Verify no untracked changes
SELECT CT_MRDS.ENV_MANAGER.CHECK_PACKAGE_CHANGES('CT_MRDS', 'DATA_EXPORTER') FROM DUAL;
-- Expected: OK

Rollback

# Execute as ADMIN user
Get-Content "MARS_Packages/REL03/MARS-1005-PREHOOK/rollback_mars1005_prehook.sql" | sql "ADMIN/Cloudpass#34@ggmichalski_high"

Rollback restores CT_MRDS.DATA_EXPORTER to v2.14.0 (re-enables escape=true).

Testing

After installation, verify the Parquet and CSV export paths:

CSV path (ODS): Export a table, then SELECT from the corresponding ODS external table. Values with embedded double-quotes should appear as single " characters and not trigger ORA-30653.

Parquet path (ARCHIVE): Export a table containing values with embedded double-quotes to the ARCHIVE bucket area. Download the Parquet file and confirm the value is stored verbatim (no extra quotes).

Expected Changes

  • CT_MRDS.DATA_EXPORTER: v2.14.0 → v2.17.0
  • No table structure changes
  • No configuration changes
  • MARS-1005: Export TOP allotment data (main issue)
  • CT_MRDS.DATA_EXPORTER source: MARS_Packages/mrds_elt-dev-database/mrds_elt-dev-database/database/CT_MRDS/packages/DATA_EXPORTER.sql