Welcome!

Apache Authors: Elizabeth White, Pat Romanski, Liz McMillan, Christopher Harrold, Janakiram MSV

News Feed Item

Syncsort Open Sources Technology to Make Mainframe Big Data Available to Apache Spark

Apache Spark Mainframe Connector Now Available on Databricks' Spark Packages Community Site

WOODCLIFF LAKE, N.J., Sept. 1, 2015 /PRNewswire/ -- Syncsort, a global leader in Big Data software, today announced a milestone open source contribution of an IBM z Systems mainframe connector for Apache Spark. The contribution will enable organizations to easily access and get new insights from their critical mainframe data with Apache Spark's advanced analytics capabilities and Spark SQL. The connector is now available on Spark Packages: https://github.com/Syncsort/spark-mainframe-connector.

Syncsort Data Integration Logo.

Spark Packages, launched by Apache Spark pioneer Databricks – the organization founded by the team that created and continues to drive Apache Spark – makes it easy for users to find, discuss, rate and install packages that enhance Spark's capabilities.

"One of the key elements of Spark's continued success is its integration with important data sources," said Matei Zaharia, creator of Apache Spark and co-founder & CTO of Databricks. "We are excited that Syncsort has made this valuable contribution to the Apache Spark community, making mainframe data easily available for use within Spark."  

Spark has emerged as one of the most active big data open source projects, initially as the lightning fast memory-optimized processing engine for machine learning and now as the single compute platform for all types of workloads including real-time data processing, interactive queries, social graph analysis, and much more. Given its success, there is a growing need to securely access data from a diverse set of sources, including mainframes, and to transform the data into a format that is easily understandable by Spark.

"Syncsort's open source contribution makes it easy to get real-time insights from mainframe data using Apache Spark's advanced analytics capabilities and Spark SQL interactive queries," said Tendü Yoğurtçu, general manager of Syncsort's Big Data business. "We believe that Apache Spark will play a critical role in a wide variety of next-generation use cases, including streaming ETL and the Internet of Things. We will continue to contribute to Spark and related Big Data projects to enable a uniform user experience for batch and real-time workloads across all data sources."

Syncsort's mainframe connector for Spark is similar to the Apache Sqoop mainframe connector that Syncsort released as open source last year. Customers simply specify the location of multiple datasets and the associated COBOL copybook metadata and the Spark mainframe connector automatically transfers the datasets in parallel via a secure connection into Spark's DataFrame objects. Users can then manipulate this DataFrame object and join it with their other data sources for further analysis. Syncsort's mainframe connector conforms to Spark's Data Sources API specification, and because of Spark's ability operate on data in memory, the connector will allow queries to access mainframe data without offloading the data first. Mainframe record formats including fixed, variable, sequential and VSAM files are all supported. The connector also handles compressed data transfer, minimizing network bandwidth and optimizing overall elapsed time.

To download the Spark Mainframe Connector on Spark Packages, click here.

About Syncsort

Syncsort provides enterprise software that allows organizations to collect, integrate, sort and distribute more data in less time, with fewer resources and lower costs. Syncsort software provides specialized solutions spanning "Big Iron to Big Data," including next gen analytical platforms such as Hadoop, cloud, and Splunk. For more than 40 years customers have turned to Syncsort's software and expertise to dramatically improve performance of their data processing environments, while reducing hardware and labor costs.  Experience Syncsort at http://www.syncsort.com.

Media Contacts:

Sarah Borup     
SHIFT Communications
Tel: 617-779-1803
[email protected]

Michael Kornspan
Syncsort Incorporated
Director, Corporate Communications
Tel: 201-930-8216
[email protected]

Logo - http://photos.prnewswire.com/prnh/20130520/NY16823LOGO

To view the original version on PR Newswire, visit:http://www.prnewswire.com/news-releases/syncsort-open-sources-technology-to-make-mainframe-big-data-available-to-apache-spark-300135638.html

SOURCE Syncsort

More Stories By PR Newswire

Copyright © 2007 PR Newswire. All rights reserved. Republication or redistribution of PRNewswire content is expressly prohibited without the prior written consent of PRNewswire. PRNewswire shall not be liable for any errors or delays in the content, or for any actions taken in reliance thereon.

IoT & Smart Cities Stories
Founded in 2000, Chetu Inc. is a global provider of customized software development solutions and IT staff augmentation services for software technology providers. By providing clients with unparalleled niche technology expertise and industry experience, Chetu has become the premiere long-term, back-end software development partner for start-ups, SMBs, and Fortune 500 companies. Chetu is headquartered in Plantation, Florida, with thirteen offices throughout the U.S. and abroad.
DXWorldEXPO | CloudEXPO are the world's most influential, independent events where Cloud Computing was coined and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals. Sponsors of DXWorldEXPO | CloudEXPO benefit from unmatched branding, profile building and lead generation opportunities.
The standardization of container runtimes and images has sparked the creation of an almost overwhelming number of new open source projects that build on and otherwise work with these specifications. Of course, there's Kubernetes, which orchestrates and manages collections of containers. It was one of the first and best-known examples of projects that make containers truly useful for production use. However, more recently, the container ecosystem has truly exploded. A service mesh like Istio addr...
CloudEXPO New York 2018, colocated with DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
@DevOpsSummit at Cloud Expo, taking place November 12-13 in New York City, NY, is co-located with 22nd international CloudEXPO | first international DXWorldEXPO and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time t...
DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
SYS-CON Events announced today that DatacenterDynamics has been named “Media Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY. DatacenterDynamics is a brand of DCD Group, a global B2B media and publishing company that develops products to help senior professionals in the world's most ICT dependent organizations make risk-based infrastructure and capacity decisions.
Cloud-enabled transformation has evolved from cost saving measure to business innovation strategy -- one that combines the cloud with cognitive capabilities to drive market disruption. Learn how you can achieve the insight and agility you need to gain a competitive advantage. Industry-acclaimed CTO and cloud expert, Shankar Kalyana presents. Only the most exceptional IBMers are appointed with the rare distinction of IBM Fellow, the highest technical honor in the company. Shankar has also receive...
Headquartered in Plainsboro, NJ, Synametrics Technologies has provided IT professionals and computer systems developers since 1997. Based on the success of their initial product offerings (WinSQL and DeltaCopy), the company continues to create and hone innovative products that help its customers get more from their computer applications, databases and infrastructure. To date, over one million users around the world have chosen Synametrics solutions to help power their accelerated business or per...
A valuable conference experience generates new contacts, sales leads, potential strategic partners and potential investors; helps gather competitive intelligence and even provides inspiration for new products and services. Conference Guru works with conference organizers to pass great deals to great conferences, helping you discover new conferences and increase your return on investment.