How to Read and Write Cloud Object Storage from z/OS with One JCL Change: Mainframe File Sharing and Backup with FlowZ
“Just FTP it.”
That’s usually the answer when someone needs a mainframe file in the cloud, whether it’s for a backup, a partner, an analytics or AI project. Write it to disk or tape. Stage it. Script a transfer with FTP or managed file transfer, push it through a server, and finally land it in the bucket. Every one of those steps is something to build, secure and watch. And when a transfer fails overnight, nobody knows until someone goes looking for the file.
There’s a reason it works this way. The first tools for moving mainframe data were designed in the ’80s and ’90s, when mainframes ran SNA networks and couldn’t talk to the outside world directly, so data was staged on a gateway server in between. Later tools dropped the gateway but kept the staging, just on the mainframe instead. Either way, the data gets read and written three or four times on its way to the cloud. And the “it’s just an API” approach isn’t much easier: authentication works differently for each cloud provider, it changes over time, and keeping it working across Amazon, Azure and Google can turn into a full-time job.
In this recorded session, VirtualZ Computing founder and CEO Jeanne Glass and co-founder and CTO Vince Re show how FlowZ takes all of that out of the picture. Any z/OS application that uses standard QSAM, VSAM or z/OS UNIX file access can write to or read from cloud object storage with a small change to its JCL. FlowZ’s policies decide where each file goes, including to more than one place at once, and there’s no code to write and nothing to install in the cloud. Vince also covers how FlowZ uses the mainframe’s own logging, caching, security and crypto hardware so it behaves the way a mainframe team expects, and where FlowZ isn’t the right fit.
If you’re keeping staging space and a folder full of transfer scripts alive just to get files to the cloud, this 37-minute session is worth your time.
What is FlowZ?
FlowZ is VirtualZ Computing’s patented, no-code software that lets existing z/OS applications read and write files in cloud object storage and network storage. It runs on z/OS as a started task, works with Amazon S3, Azure Blob, Google Cloud Storage and compatible object stores, as well as NFS and SMB file servers, and redirects standard file I/O with a small JCL change. Policies decide where each file goes, credentials are stored in the mainframe’s crypto hardware, and errors show up on the z/OS console and in the job output.
What you’ll see
- Why moving mainframe files to the cloud still means staging, extra copies and scripts: gateway servers from the SNA era, staging on the mainframe, data read and written three or four times, and failures that take days to trace
- Why “object storage is just an API” turns into a programming project once you deal with authentication, credentials, crypto and more than one cloud provider
- How the one JCL change works: any application using standard QSAM, VSAM or z/OS UNIX file access redirects its I/O to FlowZ, and policies send each file to one destination or several at once, with no code
- Running it like any other mainframe software: a started task in the LPARs you choose, command-line configuration, credentials kept in crypto hardware, asynchronous transmission that rides out network glitches, and clear error messages on the console for your automation tools
- Where to start and where not to: write-heavy data such as backups and audit data first, how to avoid cloud egress charges on data you read back often, and why files that need sub-second access over limited bandwidth may belong somewhere else
- Backups in practice: IBM DFDSS output written straight to Amazon S3, with an average volume taking about a minute and a full environment backed up in well under an hour over a consumer-grade gigabit connection in VirtualZ’s own environment, plus tags that let S3 lifecycle policies move data to Glacier
Who should attend
Mainframe storage administrators, systems programmers and operations teams; the people who own backup, archive and disaster recovery; cloud and infrastructure architects; data, analytics and AI teams who need mainframe files in the cloud; and CIOs and infrastructure leaders looking to cut storage, tape and transfer costs. If you’re keeping FTP scripts, staging space or tape runs alive between z/OS and the cloud, this one’s for you.
Hosts
- Jeanne Glass, Founder and CEO, VirtualZ Computing
- Vince Re, Co-Founder and Chief Technology Officer, VirtualZ Computing
Watch the webinar
Reading and writing cloud object storage from z/OS with FlowZ
Q. What does FlowZ do?
FlowZ lets existing z/OS applications read and write files in cloud object storage and network storage directly. Applications keep using standard QSAM, VSAM or z/OS UNIX file access, and FlowZ sends the data to Amazon S3, Azure Blob, Google Cloud Storage, a compatible object store, or an NFS or SMB file server, with no staging and no code.
Q. How do you write mainframe files to Amazon S3, Azure Blob or Google Cloud Storage?
Usually it takes several steps: write the file to disk or tape, stage it, then move it with FTP or managed file transfer, sometimes through a gateway server. With FlowZ, you make a small change to the job’s JCL so the file points at FlowZ, and the job writes straight to the bucket.
Q. What is the one JCL change?
You redirect the job’s file to FlowZ in the JCL instead of to local disk or tape. The application doesn’t change and doesn’t need to be recompiled. FlowZ’s policies then decide where the data goes. VirtualZ has said a coming update will let you name the data sets you want FlowZ to handle, with no JCL change at all.
Q. Can FlowZ send a file to more than one place?
Yes. FlowZ’s policies can send the same file to several destinations at once, such as Amazon S3 and Google Cloud Storage, or a cloud bucket and an on-premises file server. That’s also how you keep a local copy for reads while the cloud holds the protected copy.
Q. Can z/OS applications read files from cloud object storage?
Yes. FlowZ works for reads as well as writes. For data you read back often, it’s worth planning for cloud egress charges, for example by writing to both the cloud and a local network appliance and reading from the local copy.
Q. Do I need to write code or install software in the cloud?
No. FlowZ is zero code, and nothing runs on the cloud side. FlowZ already handles each provider’s authentication and APIs, so you configure destinations and credentials with FlowZ’s command-line utility instead of building and maintaining your own integration.
Q. How does FlowZ handle security and credentials?
FlowZ uses the same credentials you’d use with any other application for that cloud, such as Amazon S3 tokens, and stores them in the mainframe’s crypto hardware rather than in files. It supports system-wide credentials and private credentials limited to specific applications, and it’s multi-user, so different users’ data can go to different places.
Q. What happens when something goes wrong?
FlowZ writes clear messages to the z/OS console and to the job’s output, including the exact error from the cloud provider, such as expired credentials or a storage limit. Automation tools such as OPS/MVS can act on those messages. Data can also be cached and sent asynchronously, so a brief network glitch doesn’t cause a failure.
Q. Can FlowZ back up mainframe volumes to the cloud?
Yes. FlowZ works with IBM DFDSS, so backup output that would go to tape can go straight to cloud object storage. In VirtualZ’s own environment, an average volume takes about a minute and the whole environment backs up in well under an hour over a consumer-grade gigabit connection. FlowZ can tag the data so S3 lifecycle policies move older backups to tiers such as Glacier.
Q. When is FlowZ not the right fit?
Files that need sub-second access over limited bandwidth, especially when many applications share that bandwidth, may be better kept local. For large migrations, such as replacing a virtual tape library, it’s worth benchmarking first: transfer speeds won’t match channel-attached virtual tape, though there’s no mount time. To load structured data into databases or Databricks, use PropelZ; for live access to data that stays on the mainframe, use Lozen.
Q. How long does it take to get FlowZ running?
FlowZ is a simple install that runs as a started task on z/OS, in one LPAR or in every LPAR you choose. Vince says that putting FlowZ into VirtualZ’s own production environment, from idea to finished, took less than an hour of work.