If you were designing a brand-new data access solution for your organization today, Lozen™ could play a prominent role in creating a lower-cost, high-performance solution. Because Lozen’s real-time, read-write approach lets you leave data where it is (securely on the IBM Z platform), you could eliminate the need to replicate or “lift and shift” data — saving time and money, increasing business agility, improving data integrity, and more.
Many customers, though, have already invested many years and millions of dollars in their data access solutions. For these customers, Lozen can help simplify the process, reduce costs, and improve performance.
In a previous blog post titled “Simplify Your ETL With Lozen,” we highlighted some of the reasons you might consider Lozen in addition to — or possibly instead of — existing ETL approaches. Many of the same reasons and factors apply to the tools and processes associated with change data capture (CDC). They all come down to the same things: cost and complexity.
In this blog article, we’ll share how Lozen can simplify or eliminate the need for CDC in the following ways:
- Build new applications: Use Lozen to develop simple, specific applications using tools of your choice to directly access IBM Z data from hybrid clouds, thus lessening or removing the need for expensive vendor software infrastructure and the attendant operational burden.
- Leave your data on the mainframe: For new or migrated applications running in the cloud, use Lozen to keep your data on the mainframe, retaining a single source of truth throughout your environment and avoiding the need for extensive refactoring and the construction of an expensive and needlessly complex bi-directional data pipeline that keeps data perpetually out of sync with systems of record.
- Simplify or eliminate CDC: Use Lozen to simplify your CDC operation — eliminate costly processing of VSAM and other files to detect the changes needed for resynchronizing cloud data stores.
Developing New Applications
Lozen provides easily configured and highly performant access to IBM Z data using NFS, a well-known industry standard file protocol. Most Linux distributions and some Windows versions have NFS drivers already available. We also provide an optional set of NFS libraries to ensure you have the most current, secure, and performant version of NFS available.
On top of our NFS support, we provide a variety of connectors and APIs to allow you to choose the mode of access and the type of applications to deploy to access IBM Z data from cloud and distributed applications. (You can find several posts on the connectors and APIs on our blog.)
To construct simple, cloud-based applications that directly access VSAM or sequential data on z/OS, we provide the following resources:
- Java and C/C++ API libraries to enable the creation of applications with full read/write/delete access to VSAM data.
- OpenAPI support for any new or existing OpenAPI-based applications or tools. This support includes simple, drag-and-drop data workflow-based tools like MuleSoft and Microsoft Logic Apps.
If you need to transfer data off z/OS, for example, you can leverage these various resources to create simple applications to copy the data without the need for any additional software licenses on z/OS. Additionally, the data accessed on z/OS can be done using lower-cost zIIP processing.
If any of the data are changed and need to be updated back to z/OS, it would be a simple matter for the application to insert, update, or delete the affected records on z/OS, without the need for bulk data transfer of entire datasets back to z/OS.
Leave Your Mainframe Data In Place
Use Lozen to maintain a single source of truth for your key business data. Since the data remains on the mainframe, new or migrated applications running in the cloud can access your IBM Z data in a seamless, straightforward, and cost-effective way.
Now you can run applications where they are best suited for your business — on the mainframe, in the cloud, on mobile, or on distributed platforms — without risking other applications that depend on that same data.
Simplify or Eliminate Change Data Capture
When considering all aspects of ETL, change data capture can be one of the hardest things to get right. In any kind of solution that is based on replication — where you have data on your mainframe, with a copy of that data on some other platform — a common approach has been to keep track of any changes that are occurring against that mainframe data and make sure those changes are reflected in the copied data.
In reality, it is hard to do this well. There are many issues that make it difficult to get the change data capture functions to work well in a large production environment. For example, network outages can interfere, or you may have transactional boundaries, where you may need to update multiple files concurrently in the scope of a single transaction.
Customers may also need change data capture in a bidirectional way. In other words, it is not just “capture what is changing on my mainframe and broadcast it to some other platform.” It is also “I’m running applications on that other platform that are changing the same data.”
Changed data needs to be propagated in both directions, without causing collisions. For example, if data gets updated from the mainframe and from some other platform at roughly the same time, which one wins? Or if someone inadvertently restores a replica data set from a now out-of-date backup, everything is out of sync. These are difficult problems, especially if these applications exist outside the scope of a “transaction manager” that understands how to serialize and coordinate access to the data.
Furthermore, when problems occur, they can be difficult to diagnose and repair. If there is contention on updates, it needs to be clear who “wins” and “loses” if two applications try to update the same data at the same time.
Many organizations turn to change data capture and near real-time replication of the data across platforms. But with each of these steps comes more complexity, more overhead, more cost, and more risk for the data to lose synchronization.
There are a lot of complicated problems out there, and sometimes complicated problems require complicated solutions. But if you have a choice between simple and complex, simple is often better. If you have the option to maintain only one copy of the data, you establish a “single source of truth.” You also eliminate the need to coordinate and propagate changes asynchronously across what could potentially be many replicas of the data. If there is only one copy of the data, it is guaranteed to be up to date.
On the other hand, once you commit to the idea of having data replicated in multiple places, now you have to face this issue of how (and how often) to keep them up to date. That comes with the cost of designing, implementing, maintaining, and executing the processes of determining what changed, extracting just the changed data, delivering it to one or more off-platform targets, and updating those replicas.
These are some of the factors that make CDC complex and costly. Lozen’s real-time, read-write, bi-directional capabilities can help simplify — or eliminate — the need for most ETL and CDC use cases.
Summary
Lozen and our family of connectors and API libraries can simplify or even eliminate your change data capture use cases. Because Lozen’s real-time, read-write approach lets you leave data where it is (securely on the IBM Z platform), you eliminate the need to replicate or “lift and shift” data — saving time and money, increasing business agility, improving data integrity, and more.
Use Lozen to keep your data on the mainframe, retaining a single source of truth throughout your environment and providing critical IBM Z data access for new or migrated applications, without risking other applications that depend on that same data
Learn More
To learn more about how to unlock the power of real-time, read-write mainframe data access with Lozen:
