460,000 VSAM Records. 25 Seconds. Into PostgreSQL. With PropelZ™.
A global outsourcer recently shared a powerful proof point from a large retail customer:
“[A large retailer] just loaded 460K VSAM records into a PostgreSQL database on Google Cloud in 25 seconds.”
That’s not a benchmark test. That’s not a synthetic workload. That’s enterprise production data moving from VSAM into PostgreSQL on Google Cloud — quickly, cleanly, and without custom code.
This is what a modern mainframe-to-Postgres integration should look like. And it’s exactly what PropelZ™ was built to do.
From Two Hours to Under Five Minutes
During the same engagement, the team tested a ~400,000-record load from the same VSAM/flat-file source into PostgreSQL.
- Initial runtime: just under 2 hours.
- After tuning batch and commit parameters: under 5 minutes.
That’s roughly a 24x performance improvement — achieved simply by optimizing batch sizing and commit strategy.
No re-architecture. No middleware changes. No new tooling. Just intelligent configuration of PropelZ™.
Large-Scale Stress Test: 60+ Million Records in 18 Minutes
To push the limits, the team selected one of the largest DB2 backup-style sequential files available on DASD:
- Total file size: ~200 million records (confirmed via IDCAMS)
- Test run stopped at: 60.9 million records written
- Runtime at stop point: ~18 minutes
- Observed throughput:
~3.38 million records per minute
~56,000 records per second
And importantly — this was not a specially crafted performance lab. It was a real enterprise dataset in a production-like environment.
The implication is significant: the same tuned configuration that accelerates 400K-record jobs scales linearly into tens of millions of rows.
Why This Matters
Large enterprises — particularly in retail — run mission-critical systems on IBM Z:
- VSAM KSDS files
- QSAM sequential files
- High-volume batch processing
- Core transactional systems
At the same time, innovation is happening in PostgreSQL:
- Real-time analytics
- AI model training
- Personalization engines
- Cloud-native microservices
The challenge isn’t whether PostgreSQL can scale. The challenge is getting trusted mainframe data into it — fast, reliably, and continuously.
The PropelZ™ Difference
PropelZ™ is a no-code ELT engine purpose-built to connect mainframe data to modern platforms, including PostgreSQL.
In this retailer’s environment:
- VSAM and sequential sources
- PostgreSQL target
- Google Cloud deployment
- 460,000 records in 25 seconds
- 400K records in <5 minutes after tuning
- 60+ million records in 18 minutes
No custom coding. No brittle scripts. No weeks of engineering.
Install. Connect. Run. Optimize.
Incremental Mode: Beyond Traditional CDC
Speed is impressive. But the real architectural shift happens after the first load.
PropelZ’s enhanced incremental mode supports ongoing synchronization of sequential and VSAM files — even when:
- Records are inserted
- Records are reordered
- Records are modified
- Files are rewritten
- No reliable primary keys exist
Historically, enterprises relied on traditional Change Data Capture (CDC) tooling. But CDC implementations can be:
- Complex to configure
- Expensive to license
- Dependent on database logs
- Fragile with non-relational file structures
- Limited in pure file-based environments
PropelZ’s incremental algorithm — conceptually similar to a high-efficiency “diff” engine — compares file states intelligently and identifies true data changes without relying on database logs or heavyweight CDC infrastructure.
In many mainframe-to-Postgres scenarios, this allows organizations to:
- Replace traditional CDC stacks
- Eliminate additional middleware
- Simplify architecture
- Reduce cost and operational overhead
For environments dominated by GDGs and sequential datasets, this represents a meaningful simplification. It’s not just faster. It’s cleaner architecture.
Cloud-Aware Optimization
During testing, teams also explored:
- Batch sizing strategies (optimal ranges often in the 40K–50K row range depending on record size)
- Commit frequency tuning
- Server-side PostgreSQL COPY operations to export data directly to cloud storage buckets
- Single-process output handlers versus multi-process scripts
- Native cloud APIs vs NFS trade-offs (performance vs cost considerations)
These are enterprise-grade optimization discussions — not proof-of-concept shortcuts. The result: PostgreSQL becomes a high-performance extension of trusted enterprise data.
Retail Is Just the Beginning
Retailers face pressure to modernize rapidly:
- Omnichannel intelligence
- Inventory optimization
- Fraud analytics
- AI-driven forecasting
PostgreSQL is often the innovation platform. PropelZ ensures the data that feeds it remains:
- Accurate
- Governed
- Secure
- Fast
- Continuously synchronized
And still anchored to the system of record.
Modernization Without Disruption
The 25-second load is compelling. The 24x performance improvement is meaningful. The 60-million-record stress test proves scale.
But the bigger story is architectural:
- You don’t have to rewrite applications first.
- You don’t need complex CDC stacks.
- You don’t have to delay innovation during migration planning.
You can activate trusted enterprise data now — and keep it synchronized continuously with PostgreSQL in the cloud.
A New Standard for Mainframe-to-Postgres
When a global outsourcer highlights performance like this at a large enterprise retailer, it signals something important:
This is repeatable. This scales. This is enterprise-ready.
And this is modernization without disruption. Because once PostgreSQL has fast, governed, continuously updated access to trusted enterprise data — innovation accelerates everywhere.
Next Steps
Learn More
- Visit our Customer Briefing Center.
- Review all VirtualZ Use Cases, Thought Leadership papers, and Solution Briefs.
- Explore our YouTube channel, podcasts, and blog.
- Still have questions? Contact us.
