Apache Nifi is a recent addition to the family of distributed data processing frameworks. Nifi provides a reliable, UI based mechanism to transport, filter and enrich data across multiple systems. What are some of the unique features of Nifi and how well does Nifi weigh out, in comparison to or in complement to the other data transfer/processing frameworks like Flume,Kakfa, etc?
The recent Hadoop Summit in San Jose brought out a few themes, some familiar and some hyperbole. One standout observation that is indisputable is that conversations in the conference were no longer about "what" but instead were about "how". Hadoop in particular and the big data ecosystem in general is not only game-changing but has been here for a while now.
Kafka has become a center piece in modern enterprise data architectures. Hadoop serving the role of persistent historical store that supports a myriad of workloads. Moving data from Kafka to Hadoop is a common task in such architectures. Read to find out more about the considerations and how to realize critical data pipelines.