# MARS-1005-PREHOOK: Fix DATA_EXPORTER RFC 4180 Compliance + Parquet Format Support

## Overview

Pre-hook for MARS-1005. Deploys an updated `CT_MRDS.DATA_EXPORTER` package (v2.17.0)
that resolves two export format bugs:

1. **RFC 4180 compliance (v2.15.0 / v2.16.0):** Previous versions used `escape=true`
   in `DBMS_CLOUD.EXPORT_DATA`, producing backslash-escaped embedded quotes (`\"`).
   ODS external tables (`FIELDS CSV WITHOUT EMBEDDED`) expect RFC 4180 doubling (`""`).
   Fix: removed `escape=true`; implemented `REPLACE(col, '"', '""')` in SELECT query.

2. **Parquet corruption fix (v2.17.0):** The RFC 4180 `REPLACE` was applied to all
   export formats, including Parquet — corrupting values that contained double-quotes.
   Fix: added `pFormat` parameter to `buildQueryWithDateFormats`; `REPLACE` is now
   applied only when `pFormat = 'CSV'`. Parquet exports pass column values unchanged.

## Contents

| File | Description |
|------|-------------|
| `install_mars1005_prehook.sql` | Master installation script (SPOOL, ACCEPT, quit) |
| `rollback_mars1005_prehook.sql` | Master rollback script (SPOOL, ACCEPT, quit) |
| `00_MARS_1005_PREHOOK_install_DATA_EXPORTER.sql` | Deploy DATA_EXPORTER v2.17.0 |
| `90_MARS_1005_PREHOOK_rollback_DATA_EXPORTER.sql` | Restore DATA_EXPORTER v2.14.0 |
| `track_package_versions.sql` | Universal version tracking script |
| `verify_packages_version.sql` | Universal package verification script |
| `new_version/DATA_EXPORTER.pkg` | Package specification v2.17.0 |
| `new_version/DATA_EXPORTER.pkb` | Package body v2.17.0 |
| `rollback_version/DATA_EXPORTER.pkg` | Package specification v2.14.0 (backup) |
| `rollback_version/DATA_EXPORTER.pkb` | Package body v2.14.0 (backup) |
| `README.md` | This file |

## Prerequisites

- Oracle Database 23ai
- `CT_MRDS.ENV_MANAGER` v3.1.0+
- ADMIN user with EXECUTE privileges on CT_MRDS schema
- Connection service: `ggmichalski_high`

## Version Change

| Package | Before | After |
|---------|--------|-------|
| `CT_MRDS.DATA_EXPORTER` | v2.14.0 | v2.17.0 |

## Installation

### Option 1: Master Script (Recommended)

```powershell
# Execute as ADMIN user for proper privilege management
Get-Content "MARS_Packages/REL03/MARS-1005-PREHOOK/install_mars1005_prehook.sql" | sql "ADMIN/Cloudpass#34@ggmichalski_high"
```

Log file created automatically: `log/INSTALL_MARS_1005_PREHOOK_<PDB>_<timestamp>.log`

### Option 2: Individual Scripts

```powershell
# Execute as ADMIN user
Get-Content "MARS_Packages/REL03/MARS-1005-PREHOOK/00_MARS_1005_PREHOOK_install_DATA_EXPORTER.sql" | sql "ADMIN/Cloudpass#34@ggmichalski_high"
```

## Verification

```sql
-- Verify package version
SELECT CT_MRDS.DATA_EXPORTER.GET_VERSION() FROM DUAL;
-- Expected: 2.17.0

-- Verify build info
SELECT CT_MRDS.DATA_EXPORTER.GET_BUILD_INFO() FROM DUAL;

-- Check for compilation errors
SELECT * FROM ALL_ERRORS
WHERE OWNER = 'CT_MRDS'
  AND NAME = 'DATA_EXPORTER';

-- Verify no untracked changes
SELECT CT_MRDS.ENV_MANAGER.CHECK_PACKAGE_CHANGES('CT_MRDS', 'DATA_EXPORTER') FROM DUAL;
-- Expected: OK
```

## Rollback

```powershell
# Execute as ADMIN user
Get-Content "MARS_Packages/REL03/MARS-1005-PREHOOK/rollback_mars1005_prehook.sql" | sql "ADMIN/Cloudpass#34@ggmichalski_high"
```

Rollback restores `CT_MRDS.DATA_EXPORTER` to v2.14.0 (re-enables `escape=true`).

## Testing

After installation, verify the Parquet and CSV export paths:

**CSV path (ODS):** Export a table, then SELECT from the corresponding ODS external
table. Values with embedded double-quotes should appear as single `"` characters
and not trigger `ORA-30653`.

**Parquet path (ARCHIVE):** Export a table containing values with embedded
double-quotes to the ARCHIVE bucket area. Download the Parquet file and confirm
the value is stored verbatim (no extra quotes).

## Expected Changes

- `CT_MRDS.DATA_EXPORTER`: v2.14.0 → v2.17.0
- No table structure changes
- No configuration changes

## Related

- MARS-1005: Export TOP allotment data (main issue)
- `CT_MRDS.DATA_EXPORTER` source: `MARS_Packages/mrds_elt-dev-database/mrds_elt-dev-database/database/CT_MRDS/packages/DATA_EXPORTER.sql`