dokumentacja testow

2026-02-09 07:49:12 +01:00
parent 7864c568d7
commit 293f2873b7
3 changed files with 545 additions and 0 deletions
--- a/confluence/additions/pMaxFileSize_Final_Summary.md
+++ b/confluence/additions/pMaxFileSize_Final_Summary.md
@@ -0,0 +1,135 @@
+# pMaxFileSize Testing - Final Summary
+
+## Objective
+Find the maximum practical value for `pMaxFileSize` parameter in DATA_EXPORTER package.
+
+## Test Results
+
+### Successful Tests
+
+#### Test 1: pMaxFileSize=400MB with pParallelDegree=12
+- **Data Volume**: 240x multiplication (~6.24M rows, 4.05GB)
+- **Duration**: 7m 51s
+- **Files Generated**: 12 files (11×400MB + 1×394.46MB = 4.68GB total)
+- **Status**: ✅ SUCCESS
+- **Key Finding**: File splitting works perfectly at 400MB limit
+
+#### Test 2: pMaxFileSize=1GB with pParallelDegree=10
+- **Data Volume**: 300x multiplication (~7.8M rows, 5.07GB)
+- **Duration**: 9m 47s
+- **Files Generated**: 6 files (5×1GB + 1×873.11MB = 5.85GB total)
+- **Status**: ✅ SUCCESS
+- **Key Finding**: 1GB files work with moderate parallelism
+
+### Failed Tests
+
+#### Test 3: pMaxFileSize=400MB with pParallelDegree=16
+- **Error**: ORA-04036 PGA exceeded
+- **Conclusion**: Too many concurrent threads (16×78 partitions)
+
+#### Test 4: pMaxFileSize=1GB with pParallelDegree=1
+- **Data Attempts**: 320x, 300x, 240x, 200x
+- **Error**: ORA-65114 space usage in container too high
+- **Conclusion**: Large data volumes exceed Autonomous DB storage limits for temp tables
+
+## Key Discoveries
+
+### 1. pMaxFileSize ≠ PGA Buffer Size
+**CRITICAL**: File size limit does NOT create proportional memory buffers.
+- ✅ `pParallelDegree=10 × pMaxFileSize=1GB` = SUCCESS
+- ❌ `pParallelDegree=16 × pMaxFileSize=400MB` = FAILURE
+
+**Conclusion**: Number of concurrent threads (parallelism) impacts PGA more than individual file size.
+
+### 2. Parallelism is Primary PGA Driver
+- Higher parallelism = more PGA consumption
+- Lower parallelism allows larger file sizes
+- Sweet spot: pParallelDegree=10-12 for large exports
+
+### 3. Storage Container Limits
+Autonomous Database has strict storage limits for temporary tables:
+- 240x multiplication succeeded with pParallelDegree=12
+- Same 240x multiplication FAILED when tested alone (container cleanup needed)
+- 300x+ multiplication consistently hits ORA-65114 storage limit
+
+### 4. File Splitting Mechanism Works Perfectly
+- Files split at exact pMaxFileSize limit (within ±0.01MB)
+- Example: 11 files @ 400.00MB + 1 file @ 394.46MB
+- Hive-style partitioning maintained correctly
+
+## Theoretical Maximum
+
+### From Oracle Documentation
+- **Maximum file size**: 2^32-1 bytes (~4GB) - limited by VARCHAR2 file naming
+- **Practical limit**: Depends on PGA configuration and data volume
+
+### From Testing
+| pParallelDegree | pMaxFileSize | Status | Notes |
+|-----------------|--------------|---------|-------|
+| 16 | 400MB | ❌ | PGA exceeded |
+| 12 | 400MB | ✅ | Perfect |
+| 10 | 1GB | ✅ | Perfect |
+| 1 | 1GB | ❌ | Data volume too large for temp table creation |
+
+**Tested Maximum**: **1GB with pParallelDegree=10**
+
+**Untested Range**: 1GB-4GB (blocked by storage limits for test data creation)
+
+## Production Recommendations
+
+### Conservative (Safest)
+```sql
+pParallelDegree => 8
+pMaxFileSize => 209715200  -- 200MB
+```
+- Minimal PGA risk
+- Fast enough for most use cases
+- Works with high partition counts
+
+### Balanced (Recommended)
+```sql
+pParallelDegree => 10
+pMaxFileSize => 419430400  -- 400MB
+```
+- Good balance of speed and safety
+- Proven successful in testing
+- Handles large datasets well
+
+### Performance (Maximum Tested)
+```sql
+pParallelDegree => 10
+pMaxFileSize => 1073741824  -- 1GB
+```
+- Maximum tested configuration
+- Best for very large single-partition exports
+- Requires monitoring of PGA usage
+
+### NOT Recommended
+```sql
+pParallelDegree > 12  -- Risks PGA exceeded errors
+pMaxFileSize > 1GB     -- Untested, may hit limits
+```
+
+## Technical Constraints
+
+### PGA Configuration (ggmichalski database)
+- `pga_aggregate_target`: 3GB (soft limit)
+- `pga_aggregate_limit`: 6GB (hard limit - triggers ORA-04036)
+
+### Query Processing PGA Usage
+- Large datasets with DATE transformations consume significant PGA
+- Query processing memory usage is independent of output file size
+- Example: 8.3M rows with TO_CHAR() operations exceeded 6GB PGA
+
+### Storage Limits
+- ORA-65114: Space usage in container too high
+- Affects temp table creation with large UNION ALL chains
+- Cleanup required between large test data creates
+
+## Conclusion
+
+**Maximum Practical pMaxFileSize**: **1GB** (with pParallelDegree=10)
+
+**Optimal Production Configuration**: **400MB** (with pParallelDegree=10-12)
+
+**Key Insight**: **Parallelism matters more than file size** for PGA management. Lower parallelism allows larger files without hitting memory limits.