Most mainframe-to-Databricks pipelines move data through several staging hops: unload to a flat file, transfer it off the mainframe, land it in object storage, decode and convert it, then load it. PropelZ™, VirtualZ Computing’s patented no-code data access software, removes those hops — it runs where the data lives, reads mainframe data once, transforms it using the COBOL copybooks you already have, and writes it straight into Databricks through a native driver built for bulk loading. Most teams install it in minutes, configure it in hours, and are testing the same day.
What is PropelZ™?
PropelZ is patented, no-code data access software from VirtualZ Computing that moves mainframe and enterprise data directly to modern targets, including Databricks, cloud databases, object storage and APIs. It runs on z/OS, Linux and other platforms, reads data where it lives and writes it straight to the target without a middle tier.
Key Takeaways
- Staging is a legacy habit. Most mainframe data-sharing tools were designed in the ’80s and ’90s, when mainframes couldn’t connect directly to other platforms. That’s why data still passes through middle tiers and staging areas.
- One pass, no middle tier. PropelZ goes from z/OS straight to Databricks with no middle tier, reading the data once and writing it once.
- Native bulk loading. Databricks’ standard JDBC interface isn’t built for bulk inserts. PropelZ’s native Databricks driver loads data in large single transactions, such as 100,000 rows at a time, for roughly a sixfold improvement.
- No code. You point PropelZ at your data and your existing COBOL copybooks. It handles EBCDIC, packed decimal, and floating point.
- Low MIPS, mainframe-grade security. Typically 90 to 95 percent of PropelZ processing runs on zIIP engines, and credentials stay in RACF, ACF2 or Top Secret.
Why Do Mainframe-To-Databricks Pipelines Have So Many Hops?
The mainframe is still where much of the enterprise’s most valuable data lives. According to the IBM Institute for Business Value, the mainframe powers an estimated 70 percent of global transactions on a value basis.¹ Mainframes have been sharing that data with other platforms for decades, and most of the tools that do it were built for a different era. Many were written when mainframes communicated over SNA networks and had no direct path to other systems. The practical answer at the time was a multi-tier architecture: a small piece of software on the mainframe copied the data to a distributed server, and a second layer pushed it on to its destination.
That design was a necessary compromise in the ’80s and ’90s, but it never went away. Data is still read on the mainframe, written to a middle tier, read again and written again before it reaches the target. Each pass adds latency, cost, and another component that can break. And as the number of targets has grown, with Databricks, cloud databases, object storage, and AI platforms all wanting the same data, the chains have multiplied.
Mainframe data itself adds another layer of difficulty. Text is usually EBCDIC rather than ASCII or Unicode. Records can contain hundreds or even thousands of fields in formats that are rare elsewhere, such as packed decimal and mainframe floating point. Turning that into rows and columns for Databricks often means hand-written conversion code, and every time a file layout changes, the work starts over.
What Do the Staging Hops Actually Cost?
| Cost | What it looks like |
|---|---|
| Build and maintenance | Every unload job, file transfer, landing zone, and conversion script has to be built, scheduled, monitored, and fixed when it breaks. |
| Scarce skills | Decoding mainframe records often means reading hexadecimal dumps, a skill that is getting harder to find every year. |
| Security exposure | Middle-tier servers and staged files are extra copies of sensitive data, often with credentials stored outside mainframe security. |
| Mainframe cost | Extracts that run on general-purpose capacity drive up MIPS and, through capacity-based licensing, the cost of the whole environment. |
| Business delay | Every hop adds hours between the system of record and the people who need the data, and new requests join an IT backlog that can stretch to years. |
The result is a pattern that shows up in almost every enterprise with a mainframe — a business unit needs data to launch a new service, and the answer from IT is a long wait or a suggestion to find a solution on its own. When mainframe data is this hard to use, teams often decide to do without it, and the business loses the value of its most important data.
How Does PropelZ Move Mainframe Data Into Databricks?
PropelZ was built today, for today. Instead of passing data through a middle tier, it runs where the data lives and connects directly to the target. Whether you’re sending data to one Databricks workspace, ten of them, or even Databricks and Oracle at the same time, it works the same way. PropelZ has three stages:
- Reads VSAM files, sequential files, tapes, GDGs, and other mainframe sources, as well as networked sources such as cloud databases and SMB or NFS file shares. It’s a plug-in architecture, so it isn’t limited to what ships out of the box.
- Transformation core. Uses metadata, typically COBOL copybooks, to break records into rows and columns, normalize data types, select only the columns you need, and filter the records you want.
- Writes to the target or targets you choose: more than 100 databases through JDBC; cloud object storage such as Amazon S3, Google Cloud Storage and Azure; SMB and NFS file shares; APIs such as Splunk; and Databricks through a dedicated native driver.
Because PropelZ runs on the platform where the data is, it reads the data exactly once and writes it exactly once. There is no intermediate platform to read it again and nothing in the middle to secure or maintain.
Staged Pipeline Vs. PropelZ: How Do They Compare?
| Staged pipeline | PropelZ | |
|---|---|---|
| Path to Databricks | Unload, transfer, land in object storage, decode and convert, then load | Read on the mainframe and write directly into Databricks |
| Passes through the data | Several: read and written again at each hop | One read, one write |
| Middle-tier servers | Usually one or more | None |
| Code to write | Conversion scripts and custom jobs per file | None: configuration and existing copybooks |
| Databricks load method | Often row-by-row inserts over JDBC | Native driver with bulk loading, such as 100,000 rows per transaction |
| Mainframe processing cost | Often general-purpose MIPS | Typically 90-95% zIIP-eligible |
| Credentials | Often stored on middle-tier servers | Kept in RACF, ACF2, or Top Secret |
| Time to first load | Weeks to months of build work | Install in minutes, configure in hours, test the same day |
Why Does a Native Databricks Driver Matter?
Databricks’ standard JDBC interface is designed mainly for queries, not high-volume inserts. Loading data through it means writing one row, committing it, and doing it again, which is slow at any real volume.
PropelZ uses a custom native driver for Databricks that bulk-loads data instead. Rather than committing one row at a time, it can load 100,000 rows in a single transaction from Databricks’ point of view. In VirtualZ’s testing, that delivers roughly a sixfold improvement. More broadly, PropelZ optimizes for each target’s most efficient load method, whether that’s a database’s bulk-insert path or buffer management.
Our Databricks connectors work in both directions: the Databricks Output connector writes data into Databricks through native Databricks APIs, while the Databricks Input connector reads data from Databricks through a certified JDBC configuration and routes it to any PropelZ output target. The two are bundled together and available from the PropelZ connector marketplace.
Do You Need to Write Code to Convert EBCDIC And Packed Decimal Data?
No. PropelZ is no-code. You don’t write Python scripts to carve records into JSON or hand-map fields. You give PropelZ the data and the metadata you almost certainly already have, usually the COBOL copybooks that describe your files. PropelZ works out whether fields are EBCDIC text, packed decimal or floating point, and which precision, from that metadata.
For database sources, you don’t even need a copybook: the database supplies its own metadata. And because PropelZ is designed for IT operations staff rather than developers, it takes far less specialized skill to run day to day.
How Does PropelZ Keep Mainframe Data Secure?
PropelZ supports whatever security your target supports. For Databricks, that includes TLS encryption in flight and Databricks’ authentication methods. On the mainframe side, credentials can be stored and managed in RACF, ACF2, or Top Secret, the same way you protect other sensitive data, and PropelZ can use the mainframe’s built-in cryptographic co-processors.
Removing the middle tier also removes a security risk. There is no separate Windows or Linux server in the path holding copies of the data or credentials for the target.
Will Moving Mainframe Data to Databricks Raise MIPS Costs?
Not with PropelZ. Across customer sites, typically 90-95% of the processing PropelZ uses lands on zIIP specialty engines. Large transfers run on the lowest-cost capacity in the environment, so they don’t drive up software bills or require a capacity upgrade.
How Long Does It Take to Install PropelZ and Start Loading Databricks?
PropelZ ships as an archive that you restore into a new or existing z/OS UNIX directory. There are no prerequisites, no system changes, and no special privileges required. It installs in minutes and configures in hours, and most teams are testing the same day.
The main setup work is connectivity, the same steps you’d take connecting any application to Databricks: Databricks credentials, certificates for an encrypted session, and any firewall ports. If you’ve connected another application to Databricks before, you can reuse what you already have. Most teams prove PropelZ out within a week.
Once it’s running, PropelZ fits the way you already operate:
- Run it from the command line for one-off transfers.
- Trigger it from your job scheduler, such as CA 7, or an automation tool such as OPS/MVS.
- Add it as a step in an existing batch job, so a file is sent to Databricks as soon as it’s created.
- Use the massively parallel option for large inventories — give PropelZ a list of files and a limit, such as 10, 100 or 1,000 at a time, and it orchestrates and monitors the runs so failures are easy to find and rerun. Customers use it for tens of thousands of files and for tape libraries with up to a million data sets.
What Results Do Customers See?
Results depend on bandwidth and environment, but compared with legacy approaches, customers typically see PropelZ run at least twice as fast, and often far faster. Some examples:
- A file-transfer test suite that used to run from Friday afternoon into Monday now finishes in a couple of hours.
- Runs that took three days with legacy tools have dropped to two hours.
- One customer reported its first PropelZ job as broken because it finished in 20 seconds. All of the data was in the target, complete and accurate.
- A software vendor that ingests terabyte-sized customer data feeds replaced its custom ingestion code with PropelZ and cut processing time from eight hours to one to two hours, opening up services it couldn’t offer before.
See more evidence and validation.
When Should You Land Data in Databricks, And When Should You Use Live Access?
PropelZ is the right choice whenever the workload belongs in Databricks: analytics and BI at scale, AI and ML training, recurring data feeds, and data sharing. When an application needs real-time read and write access to the current mainframe record without a copy, Lozen™, VirtualZ’s live data access product, is the better fit. Many enterprises use both, and in either case the mainframe remains the system of record.
Frequently Asked Questions
How do you load VSAM data into Databricks?
With PropelZ, VSAM data is read once on the mainframe, transformed using its existing COBOL copybook, and written directly into Databricks through a native bulk-load driver, with no flat-file unload, staging area, or intermediate platform.
Why is loading Databricks over JDBC slow?
Databricks’ standard JDBC interface is designed mainly for queries and isn’t built for bulk inserts, so row-by-row loading is slow. PropelZ’s native Databricks driver loads data in large single transactions for roughly a sixfold improvement.
Can PropelZ read data from Databricks?
Yes. The PropelZ Databricks Input connector reads Databricks data through a certified JDBC configuration and routes it to any PropelZ output target. It is bundled with the Databricks Output connector.
Does PropelZ need a middle-tier server?
No. PropelZ runs where the data lives and connects directly to Databricks and other targets, so there is no intermediate Windows, Linux, or cloud server to install, secure or maintain.
Which mainframe data sources does PropelZ support?
PropelZ reads VSAM files, sequential files, tapes, GDGs, and other mainframe sources, plus databases and networked file shares. Its plug-in architecture supports additional sources.
Does PropelZ work with targets other than Databricks?
Yes. PropelZ writes to more than 100 databases through JDBC, cloud object storage such as Amazon S3, Google Cloud Storage and Azure, SMB and NFS file shares, and APIs.
How much of PropelZ’s processing is zIIP-eligible?
Typically 90 to 95 percent of the processing PropelZ uses runs on zIIP specialty engines, so it doesn’t add significant general-purpose MIPS.
How long does a PropelZ proof of concept take?
PropelZ installs in minutes and configures in hours, most teams are testing the same day, and most prove it out within a week.
Next Steps
- Watch the webinar.
- Schedule a PropelZ briefing.
- Watch a demo of PropelZ.
- Try PropelZ.
Learn More
- Visit our Customer Briefing Center.
- Review all VirtualZ Use Cases, Thought Leadership papers, and Solution Briefs.
- Explore our YouTube channel, podcasts, and blog.
- Still have questions? Contact us.
Sources
¹ IBM Institute for Business Value (in collaboration with Oxford Economics), “AI drives mainframe innovation,” November 18, 2025. https://www.ibm.com/think/insights/ai-drives-mainframe-innovation.
