Data Transformation with Lozen

Part of our mission with Lozen™ software is to make mainframe data easy to access from as many types of applications and platforms as possible in a no-code manner.  Our innovative approach to real-time, read-write data access includes data transformation, which is often required to ensure mainframe data can be processed by applications running on other platforms like Windows or Linux.  In this blog, we discuss Lozen’s robust data transformation capability and what it can do for you.

What is Data Transformation?

Data transformation is the process of converting raw data from one format or structure (XML, Excel, SQL, and so on) into another cleansed and ready-to-use format or structure. For most organizations, data transformation can increase the efficiency of analytic and business processes and can improve data-driven decision-making — but it can also be costly, complicated, and resource-intensive.

Transforming Mainframe Data

Because many mainframe data types are not compatible with newer platforms like Windows and Linux, most applications running on these platforms require mainframe data to be transformed before it can be consumed.

For example, mainframe data is typically encoded in EBCDIC format.  To be consumed by applications running on newer platforms like Windows and Linux, it must be converted to native platform encoding (ASCII).

While the simple EBCDIC-to-ASCII data conversion works for strings (text), conversion gets more complex when working with data types like Packed Decimal, Signed Numbers, Comp, and Binary fields.

Lozen’s Robust Data Transformation Capabilities

Lozen offers a robust data transformation capability to map VSAM/QSAM and other mainframe data to more modern data formats.

Most mainframe datasets are record-oriented, and the records in a given dataset are either fixed-length or variable-length.  The use of copybooks is common in COBOL applications to define the structure of the record.  All commercial data integration tools use copybooks to transform z/OS data into the required format for applications to consume.  Like these other tools, Lozen also supports converting copybooks to its record definition format for use cases where copybooks already exist

But we offer another approach for data transformation.  We offer a declarative way of defining record structure in a simple XML format to support data transformation, based on an open-source project called Jrecord.  In the next section, we describe Lozen’s metadata structure and its usage.

Lozen’s Record Definition Format

Lozen’s metadata structure supports 33 field types out-of-the-box, and it provides interfaces to implement custom field types.

In Lozen’s record definition format, each file contains one record definition.  It can contain either sub-records held in a <records> tag or Fields in a <fields> tag.  Only one level of record nesting is allowed; nesting records within records within records is not permitted.

In the code snippet below, we show a simple record definition representing a record in a z/OS KSDS student dataset that is transformed to JSON.

<?xml version="1.0" ?>
<RECORD RECORDNAME="STUDENTS" FONTNAME="CP1047" RECORDTYPE="RecordLayout">
   <FIELDS>
<FIELD NAME="StudentId" POSITION="1" LENGTH="7" TYPE="Positive Binary Integer" />
<FIELD NAME="Surname" POSITION="8" LENGTH="8" TYPE="Char" />
<FIELD NAME="Initials" POSITION="16" LENGTH="2" TYPE="Char" />
<FIELD NAME="DATE" POSITION="18" LENGTH="8" TYPE="DateFormat YYYYMMDD" />
<FIELD NAME="CourseCode" POSITION="26" LENGTH="4" TYPE="Char" />
<FIELD NAME="Gender" POSITION="30" LENGTH="1" TYPE="Char" />
   </FIELDS>
</RECORD>

(Detailed descriptions of record metadata definition structure and supported field types can be found in the Lozen Product Guide.)

Using Record Definition to Transform/Convert Your Data

To enable mainframe data to be accessible from as many types of applications as possible, Lozen offers standard NFS API, Java, and C++ SDK.

Using the NFS API

Applications using Lozen’s NFS API can use an advanced file open directive called TRANSFORM to enable transformation while data is being retrieved. (The list of all Lozen advanced directives is documented in the Lozen Product Guide.)

Here is an example:

 student.ksds{{TRANSFORM=’students.xml’;accept-type:’application/json’}}

Using the Lozen API

For applications using the Lozen API, the open function takes record metadata definitions as parameters and uses it to transform records during both read and write operations.  Here are two examples, one using the Java API and the other using the C API.

JAVA API

 public FCD open(String fileName, int openFlags, boolean reversed,String transform,String mimeType) throws LozenAPIException

 

C API

 FCD* lozen_open(LozenAPIContext* apiContext,const char* fileName,int openFlags, boolean reversed, const char* transform, const char*  mimetype);

Summary

If you are an application developer looking for ways to integrate and transform mainframe data into your applications, Lozen offers a robust data transformation capability to map VSAM/QSAM and other mainframe data to more modern data formats.

Learn More

To learn more about how to unlock the power of real-time, read-write mainframe data access with Lozen:

• Try Lozen for free.
• Explore our demos page and YouTube channel.
• Watch webinars and read the blog.
• Still have questions? Contact us.