Software development company InfinyOn simplifies data pipelines to accelerate operational analytics & insights; soon launching a product designed to handle complex data interactions with efficiency.
Santa Clara, California, Aug. 26, 2024 –
InfinyOn, a company pioneering in real-time event streaming, has its roots deeply embedded in decades of experience in holistic back-end ecosystems. Founded by veterans of the networking, security, and distributed systems sectors, InfinyOn is redefining the landscape with its innovative approach to real-time data processing.
Credits: InfinityON
The story of InfinyOn begins with its co-founders, AJ Hunyady and Sehyo Chang, who first collaborated with a renowned cybersecurity platform company. With deep expertise in networking, security, and distributed software, they created Zokets, a Docker container orchestration engine. This venture was then sold to NginX, where they managed Service Mesh – a critical infrastructure layer for fast, reliable, and low-latency network connections in highly distributed applications. This experience highlighted the limitations of Apache Kafka, a tool widely used for handling real-time data streams.
While Kafka has been a market leader in distributed streaming, it was built during the big data era, obviously relying on older hardware and programming languages like Java. As technology advanced, Kafka’s limitations became apparent. The need for performance and scalability in modern applications made Kafka cost-prohibitive. Moreover, to process data through Kafka requires multiple additional tools such as Flink, Spark and Airflow, that have their own infrastructure and maintenance overheads. This prompted AJ and Sehyo to envision a new unified, modular, and composable solution.
Rather than merely evolving Kafka, InfinyOn’s founders decided to rebuild a distributed streaming engine from scratch. This new engine, named Fluvio, leverages modern hardware and advanced programming languages. Central to this approach is the Rust programming language and WebAssembly component model (Wasm), chosen for their memory safety and secure execution. “Kafka is almost a gigabyte if you download it, and with all its dependencies, it can balloon to six gigabytes. It requires multiple server instances, making it impractical for small businesses,” explains AJ. “We wanted to create something leaner, more efficient.”
Rust, although newer and less widespread than Java, offers unparalleled memory safety, essential for preventing leaks and ensuring reliability. Wasm, meanwhile, provides a secure execution environment that isolates functions, enhancing security and performance. “Rust has a steep learning curve, but once mastered, it produces software that works reliably across all systems,” states Sehyo. “Combined with WebAssembly, it ensures our system is robust and secure.”
In 2022, InfinyOn secured a funding round to support the development of their product. Debadyuti RoyChowdhury, Vice President of Products, joined shortly after, bringing extensive experience in large-scale data implementations. “We’ve packaged Fluvio into a 37 MB binary, significantly smaller than Kafka,” Debadyuti explains. “This makes it ideal for microcomputers and edge devices, crucial for IoT applications where resources are limited.”
InfinyOn’s compact, efficient design allows it to run on low-powered devices, maintaining performance even in challenging times. This efficiency also translates to cost savings, as fewer resources are needed to achieve high network throughput. With Fluvio, the company processes tens of thousands of records per second on a single small instance. This level of efficiency and performance is unprecedented in the industry.
After five years of deployment, InfinyOn has emerged with a product that meets the demands of modern data processing. Their journey from recognizing the limitations of existing tools to creating a new standard in real-time streaming is a testament to their vision and expertise.
The company is soon launching its ‘Stateful Data Flow’ product, a significant milestone in Stateful stream processing. Designed to handle complex data interactions with unparalleled efficiency, this product runs on edge devices and integrates seamlessly with the cloud.
Unlike traditional systems, which often struggle with latency and scalability, InfinyOn’s Stateful Data Flow ensures robust, low-latency data processing. It addresses the evolving needs of contemporary applications, particularly in environments with fluctuating network conditions and limited computational resources. This innovative solution is poised to redefine how businesses manage and process data, providing a streamlined, high-performance alternative to legacy systems.
The stateful data flow product stands out not only for its technical prowess but also for its practical applications across various industries. It is compact enough to run on microcomputers and edge devices, making it ideal for Internet of Things (IoT) implementations and other scenarios where space and power are at a premium. By reducing the infrastructure requirements and operational costs, InfinyOn’s product offers a competitive edge, enabling businesses to harness the power of real-time data processing without the hefty investment in traditional, large-scale systems. Debadyuti emphasizes stating, “This product, born from years of experience and innovation, represents a foundational technology poised to drive the next generation of data-centric applications.”
“We realized to keep pace with the rapid advancements in hardware as well as software, we needed to build the streaming system from the ground up. Improving on existing technology was not an option,” the founders proudly state. “InfinyOn is positioned to lead the next generation of real-time event streaming.”
As InfinyOn continues to innovate, the impact of their work will be felt across industries implementing real-time AI, such as IoT, fintech, and e-commerce, paving the way for more efficient, reliable, and scalable data products.
