mars/MARS_Packages/REL02/MARS-1046/test/EXTERNAL_TABLE_TEST_RESULTS.md

MARS-1046: External Table Direct Test Results

Test Date: 2026-01-13
Tester: Grzegorz Michalski
Purpose: Verify FILE_MANAGER.CREATE_EXTERNAL_TABLE with ISO 8601 date format

---

Test Execution Summary

Test Scenario

Direct creation of external table using ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE procedure to read CSV file with ISO 8601 timestamp format containing:

• Date separator: T (literal character)
• Milliseconds: .FF3
• Timezone offset: TZH:TZM

Test Steps Performed

Step 1: Template Table Creation

• Template: CT_ET_TEMPLATES.ISO8601_TEST_DATA
• Column: EVENT_TIMESTAMP TIMESTAMP(3) WITH TIME ZONE
• Status: CREATED

Step 2: FILE_MANAGER Configuration

• Source: TEST
• File Config: ISO8601_TEST_DATA
• Date Format: YYYY-MM-DD"T"HH24:MI:SS.FF3TZH:TZM
• Status: CONFIGURED

Step 3: CSV File Upload

• Bucket: inbox
• Path: INBOX/TEST/ISO8601/ISO8601_TEST_DATA/iso8601_test_data.csv
• Records: 5
• Format: Full ISO 8601 with milliseconds and timezone
• Status: UPLOADED

Step 4: External Table Creation

CALL ODS.FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE(
    pTableName => 'ISO8601_TEST_DATA_INBOX',
    pTemplateTableName => 'CT_ET_TEMPLATES.ISO8601_TEST_DATA',
    pPrefix => 'INBOX/TEST/ISO8601/ISO8601_TEST_DATA',
    pBucketUri => CT_MRDS.ENV_MANAGER.gvInboxBucketUri
);

• External Table: ODS.ISO8601_TEST_DATA_INBOX
• Type: ORACLE_LOADER
• Status: CREATED

---

CRITICAL VERIFICATION: ACCESS PARAMETERS

Expected Format

DATE_FORMAT TIMESTAMP WITH TIME ZONE MASK 'YYYY-MM-DD"T"HH24:MI:SS.FF3TZH:TZM'

Actual Format (from all_external_tables)

RECORDS IGNORE_HEADER=1 DELIMITED BY DETECTED NEWLINE NOLOGFILE NOBADFILE NODISCARDFILE READSIZE=10000000 CREDENTIAL '"OCI$RESOURCE_PRINCIPAL"'
    FIELDS CSV WITHOUT EMBEDDED MISSING FIELD VALUES ARE NULL NOTRIM (
        "ID"
        ,"EVENT_NAME"       CHAR(100)
        ,"EVENT_TIMESTAMP"  CHAR(50) DATE_FORMAT TIMESTAMP WITH TIME ZONE MASK 'YYYY-MM-DD"T"HH24:MI:SS.FF3TZH:TZM'
        ,"DESCRIPTION"      CHAR(500)
    )

RESULT: PASS
Separator T is correctly quoted as "T" in the MASK clause.

---

Data Reading Test Results

Test Query

SELECT
    ID,
    EVENT_NAME,
    TO_CHAR(EVENT_TIMESTAMP, 'YYYY-MM-DD HH24:MI:SS.FF3 TZH:TZM') AS PARSED_TIMESTAMP,
    DESCRIPTION
FROM ODS.ISO8601_TEST_DATA_INBOX
ORDER BY ID;

Results

ID	EVENT_NAME	PARSED_TIMESTAMP	DESCRIPTION
1	System Start	2012-03-02 14:16:23.798 +01:00	Initial system startup with timezone offset
2	Data Processing	2024-11-27 09:30:45.123 +01:00	Batch processing job executed
3	User Login	2025-01-15 18:22:10.456 +02:00	User authentication successful
4	Error Event	2023-07-20 03:45:00.789 +00:00	Critical error detected in module X
5	Backup Complete	2024-12-31 23:59:59.999 +01:00	Year-end backup finished successfully

RESULT: PASS
All 5 records successfully parsed with complete timestamp data.

---

Verification Summary

Completeness Check

SELECT
    COUNT(*) AS TOTAL_RECORDS,
    COUNT(EVENT_TIMESTAMP) AS PARSED_TIMESTAMPS,
    CASE
        WHEN COUNT(*) = COUNT(EVENT_TIMESTAMP) THEN 'PASS - All timestamps parsed correctly'
        ELSE 'FAIL - Some timestamps NULL'
    END AS TEST_RESULT
FROM ODS.ISO8601_TEST_DATA_INBOX;

TOTAL_RECORDS	PARSED_TIMESTAMPS	TEST_RESULT
5	5	PASS - All timestamps parsed correctly

---

NORMALIZE_DATE_FORMAT Function Verification

Function Behavior

The NORMALIZE_DATE_FORMAT private function in FILE_MANAGER package body:

1. Detects unquoted T separator in ISO 8601 format strings
2. Automatically wraps T in double quotes for Oracle External Tables compatibility
3. Maintains idempotency - does not re-quote already quoted separators

Input/Output Examples

Input Format	Output Format	Status
YYYY-MM-DDTHH24:MI:SS.FF3TZH:TZM	YYYY-MM-DD"T"HH24:MI:SS.FF3TZH:TZM	NORMALIZED
YYYY-MM-DD"T"HH24:MI:SS	YYYY-MM-DD"T"HH24:MI:SS	UNCHANGED (idempotent)
DD/MM/YYYY HH24:MI:SS	DD/MM/YYYY HH24:MI:SS	UNCHANGED (no T separator)

---

Test Results Summary

Test Category	Test Case	Result	Notes
Package Compilation	FILE_MANAGER SPEC	PASS	VALID
Package Compilation	FILE_MANAGER BODY	PASS	VALID
Version Tracking	ENV_MANAGER.TRACK_PACKAGE	PASS	No untracked changes
External Table Creation	CREATE_EXTERNAL_TABLE call	PASS	Table created successfully
Date Format Normalization	T separator quoting	PASS	"T" correctly inserted
Access Parameters	DDL format verification	PASS	MASK contains YYYY-MM-DD"T"HH24:MI:SS.FF3TZH:TZM
Data Reading	CSV file parsing	PASS	All 5 records read
Timestamp Parsing	Milliseconds preservation	PASS	.FF3 values preserved (.798, .123, .456, .789, .999)
Timestamp Parsing	Timezone preservation	PASS	TZH:TZM values preserved (+01:00, +02:00, +00:00)
Data Completeness	NULL check	PASS	5/5 timestamps successfully parsed

---

FINAL VERDICT

STATUS: ALL TESTS PASSED

Key Achievements

1. External table successfully created using FILE_MANAGER_ODS.CREATE_EXTERNAL_TABLE
2. ISO 8601 format with literal T separator correctly handled
3. Date format automatically normalized with quoted "T"
4. All CSV records successfully parsed from INBOX bucket
5. Milliseconds and timezone information preserved in timestamps
6. No NULL values in timestamp columns

Production Readiness

• Package FILE_MANAGER v3.3.1 is PRODUCTION READY
• NORMALIZE_DATE_FORMAT function operates correctly
• External table creation workflow validated end-to-end
• ISO 8601 date parsing fully functional

---

Test Artifacts

Files Used

• Template Table: create_iso8601_test_template.sql
• Configuration: configure_iso8601_test.sql
• Test Script: test_create_external_table.sql
• Test Data: iso8601_test_data.csv (5 records with full ISO 8601 format)

Database Objects Created

• Template: CT_ET_TEMPLATES.ISO8601_TEST_DATA
• External Table: ODS.ISO8601_TEST_DATA_INBOX
• External Table: ODS.ISO8601_TEST_DATA_ODS

Cleanup (Optional)

-- Drop external tables after testing
DROP TABLE ODS.ISO8601_TEST_DATA_INBOX;
DROP TABLE ODS.ISO8601_TEST_DATA_ODS;

-- Remove template table
DROP TABLE CT_ET_TEMPLATES.ISO8601_TEST_DATA;

-- Remove configuration
CALL CT_MRDS.FILE_MANAGER.DELETE_SOURCE_CASCADE(pSourceKey => 'TEST');

---

Test Completed: 2026-01-13 13:24:48
Approved for Production Deployment: YES