Confluent allows developers to quickly build and deploy streaming data pipelines

Confluent, Inc., announced Stream Designer, a visual interface that allows developers to quickly build and deploy streaming data pipelines. This point-and-click visual builder is a significant step toward democratizing data streams, making them accessible to developers other than specialized Apache Kafka experts. Organizations can quickly connect more data throughout their business for agile development and better, faster, in-the-moment decision making as more teams are able to rapidly build and iterate on streaming pipelines.

“We are in the middle of a major technological shift, where data streaming is making real time the new normal, enabling new business models, better customer experiences, and more efficient operations,” said Jay Kreps, Cofounder and CEO, Confluent. “With Stream Designer we want to democratize this movement towards data streaming and make real time the default for all data flow in an organization.”

Data streaming is the default mode of data operations for successful modern businesses in the streaming era. Streaming technologies that were once on the outskirts have now become critical to critical business functions. This shift is being driven by the increasing demand for data to be delivered instantly and scalably across a wide range of customer experiences and business operations. Traditional batch processing cannot keep up with the growing number of use cases that require sub-millisecond updates from an ever-expanding set of data sources.

As more of their business operates in real time, organizations are looking for ways to accelerate their data streaming initiatives. Kafka is the de facto standard for data streaming, allowing over 80% of Fortune 100 companies to handle large volumes and types of data in real time. Building streaming data pipelines on open-source Kafka, on the other hand, necessitates large teams of highly specialized engineering talent and time-consuming development across multiple tools. Many organizations will be unable to access pervasive data streaming as a result, and existing legacy pipelines will become clogged with stale and outdated data.

Stream Designer: The First Visual Interface for Rapidly Building Streaming Data Pipelines Natively on Kafka

“Data streaming is quickly becoming the central nervous system of our infrastructure as it powers real-time customer experiences across our 12 countries of operations,” said Enes Hoxha, Enterprise Architect, Raiffeisen Bank International. “Stream Designer’s low-code, visual interface will enable more developers, across our entire organization, to leverage data in motion. With a unified, end-to-end view of our streaming data pipelines, it will improve our developer productivity by making real-time applications, pipeline development, and troubleshooting much easier.”

Stream Designer gives developers a flexible point-and-click canvas for quickly building pipelines and easily describing data flows and business logic within the Confluent Cloud UI. It takes a developer-centric approach, allowing users with varying skills and needs to switch seamlessly between the UI, a code editor, and a command-line interface to declaratively build data flow logic at breakneck speed. It introduces developer-oriented practices to pipelines, making it easier for Kafka newcomers to scale data streaming projects more quickly.

With Stream Designer, organizations can: 

• Boost developer productivity: Instead of spending days or months managing individual components on open-source Kafka, developers can build pipelines with the complete Kafka ecosystem accessible in one visual interface. They can build, iterate and test before deploying into production in a modular fashion, keeping with popular agile development methodologies. There’s no longer a need to work across multiple discrete components, like Kafka Streams and Kafka Connect, that each require their own boilerplate code. 

• Unlock a unified end-to-end view: After building a pipeline, the next challenge is maintaining and updating it over its lifecycle as business requirements change and tech stacks evolve. Stream Designer provides a unified, end-to-end view to easily observe, edit, and manage pipelines and keep them up to date. 

• Accelerate development of real-time applications: Pipelines built on Stream Designer can be exported as SQL source code for sharing with other teams, deploying to another environment, or fitting into existing CI/CD workflows. Stream Designer allows multiple users to edit and work on the same pipeline live, enabling seamless collaboration and knowledge transfer.