admin/mars
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update DATA_EXPORTER package to v2.17.0: Fix RFC 4180 compliance and Parquet format corruption

Browse Source
This commit is contained in:
Grzegorz Michalski
2026-03-12 08:50:08 +01:00
parent 5ba6c30fda
commit 202b535f9f
10 changed files with 346 additions and 46 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
MARS_Packages/REL03/MARS-1005-PREHOOK/install_mars1005_prehook.sql
View File

@@ -1,13 +1,15 @@
-- ===================================================================
-- MARS-1005-PREHOOK INSTALL SCRIPT: Fix DATA_EXPORTER RFC 4180 Compliance
-- ===================================================================
-- Purpose: Pre-hook for MARS-1005 - Deploy updated DATA_EXPORTER (v2.15.0)
-- that exports CSV files in RFC 4180 compliant format.
-- Purpose: Pre-hook for MARS-1005 - Deploy updated DATA_EXPORTER (v2.17.0)
-- that fixes RFC 4180 CSV compliance and Parquet format corruption.
-- Background: DATA_EXPORTER v2.14.0 uses escape=true in DBMS_CLOUD.EXPORT_DATA,
-- which produces backslash-escaped embedded quotes (\")
-- instead of RFC 4180 doubling ("").
-- ODS external tables (FIELDS CSV WITHOUT EMBEDDED) expect RFC 4180.
-- Fix: remove escape=true so Oracle uses RFC 4180 doubling.
-- v2.16.0 fixed CSV by applying REPLACE(col, '"', '""') in SELECT,
-- but this corrupted Parquet exports containing double-quote values.
-- v2.17.0 applies REPLACE only when pFormat = 'CSV', leaving
-- Parquet exports unchanged.
-- Author: Grzegorz Michalski
-- Date: 2026-03-10
@@ -34,14 +36,14 @@ PROMPT =========================================================================
PROMPT MARS-1005-PREHOOK: Fix DATA_EXPORTER RFC 4180 Compliance
PROMPT =========================================================================
PROMPT
PROMPT Problem: DATA_EXPORTER v2.14.0 uses escape=true in DBMS_CLOUD.EXPORT_DATA
PROMPT which causes backslash-escaped embedded quotes (\")
PROMPT instead of RFC 4180 doubling (""). ODS external tables with
PROMPT FIELDS CSV WITHOUT EMBEDDED reject rows with backslash-escape.
PROMPT Problem: DATA_EXPORTER v2.14.0 uses escape=true which produces \"-escaped
PROMPT quotes instead of RFC 4180 doubling "". v2.16.0 fixed CSV but
PROMPT applied REPLACE to Parquet exports too, corrupting quote values.
PROMPT
PROMPT This script will:
PROMPT - Deploy CT_MRDS.DATA_EXPORTER v2.15.0 (removes escape=true parameter)
PROMPT - Exported CSV files will use RFC 4180 doubling ("") for embedded quotes
PROMPT - Deploy CT_MRDS.DATA_EXPORTER v2.17.0 (RFC 4180 CSV + Parquet fix)
PROMPT - CSV exports use RFC 4180 doubling ("") for embedded quotes
PROMPT - Parquet exports pass column values unchanged (no REPLACE applied)
PROMPT - ODS external tables (FIELDS CSV WITHOUT EMBEDDED) are NOT modified
PROMPT
PROMPT Expected Duration: 1-2 minutes
@@ -60,10 +62,22 @@ WHENEVER SQLERROR CONTINUE
PROMPT
PROMPT =========================================================================
PROMPT Step 1: Deploy DATA_EXPORTER v2.15.0 (RFC 4180 Fix)
PROMPT Step 1: Deploy DATA_EXPORTER v2.17.0 (Parquet Fix + RFC 4180)
PROMPT =========================================================================
@@00_MARS_1005_PREHOOK_install_DATA_EXPORTER.sql
PROMPT
PROMPT =========================================================================
PROMPT Step 2: Track Package Versions
PROMPT =========================================================================
@@track_package_versions.sql
PROMPT
PROMPT =========================================================================
PROMPT Step 3: Verify Package Versions
PROMPT =========================================================================
@@verify_packages_version.sql
PROMPT
PROMPT =========================================================================
PROMPT MARS-1005-PREHOOK Installation - COMPLETED