Hadoop and big data platforms were originally known for scale, not speed. But the arrival of high performance compute engines like Spark and streaming engines have cleared the way for bringing batch ...

It’s become almost a standard career path in Silicon Valley: A talented engineer creates a valuable open source software commodity inside of a larger organization, then leaves that company to create a ...

“Many organizations are interested in using a single software environment for streaming and batch processing, while taking advantage of the power of the Apache Spark compute platform for analytics and ...

The Big Data streaming project Apache Kafka is all over the news lately, highlighted by Confluent Inc.'s new update of its Kafka-based Confluent Platform 2.0. On the same day as the MapR announcement, ...

When the big data movement started it was mostly focused on batch processing. Distributed data storage and querying tools like MapReduce, Hive, and Pig were all designed to process data in batches ...

Analytics is often described as one of the biggest challenges associated with big data, but even before that step can happen, data has to be ingested and made available to enterprise users. That’s ...

After Apache Hadoop got the whole Big Data thing started, Apache Spark emerged as the new darling of the ecosystem, becoming one of the most active open source projects in the world by improving upon ...

Organizations building real-time stream processing systems on Apache Kafka will be able to trust the platform to deliver each messages exactly once when they adopt new Kafka technology planned to be ...

In the first half of this JavaWorld introduction to Apache Kafka, you developed a couple of small-scale producer/consumer applications using Kafka. From these exercises you should be familiar with the ...