NiFi batches SQL. I wanted Teradata to do better.

Why I stopped relying on NiFi's standard JDBC processors

Hi,

Why should a real-time pipeline wait for 10,000 rows before updating a database?

And why should Teradata execute thousands of individual UPDATE statements when it can process them as a set?

I wasn't satisfied with either approach.

So I built my own Teradata MERGE solution for Apache NiFi.

The idea is simple:

If 100,000 rows arrive within a few milliseconds, process them together using one optimized MERGE.

Enter your email to finish this story and get Roland Wenzlofsky's next one straight to your inbox.

Already subscribed? Enter the same address and you'll go straight through. · Free, and you can unsubscribe any time.

More issues

All 87 →