Leverage Apache Flink Dashboard: Real-Time Data Processing

The Apache Flink Managed Service in AWS, offered through Amazon Kinesis data analytics for Apache Flink, allows developers to run Flink-based stream processing applications without the complexities of managing the underlying infrastructure. This fully managed service simplifies the deployment, scaling,...

November 6, 2024

Source Code Analysis of Apache SeaTunnel Zeta Engine (Part 3): Server-Side Task Submission | HackerNoon

News

Source Code Analysis of Apache SeaTunnel Zeta Engine (Part 3): Server-Side Task Submission | HackerNoon

This is the last piece of the series articles to analyze the Apache SeaTunnel Zeta Engine Source Code; review the previous series to get the full picture: Let’s review the components that execute after the server starts: CoordinatorService: Enabled only...

September 21, 2024

Real-Time GenAI With RAG Using Apache Kafka, Flink – DZone

News

Real-Time GenAI With RAG Using Apache Kafka, Flink – DZone

How do you prevent hallucinations from large language models (LLMs) in GenAI applications? LLMs need real-time, contextualized, and trustworthy data to generate the most reliable outputs. This blog post explains how RAG and a data streaming platform with Apache Kafka...

September 11, 2024

Comparison of Apache Astro and Airflow – DZone

News

Comparison of Apache Astro and Airflow – DZone

Effective workflow orchestration is the key to creating automation around complex process-oriented activities in the modern landscape of software development. Considering data engineering and data science, Astro and Apache Airflow rise to the top as important tools used in the...

September 6, 2024

News

DRY Your Apache APISIX Config – DZone

DRY is an important principle in software development. This post will show you how to apply it to Apache APISIX configuration. The DRY Principle “Don’t repeat yourself” (DRY) is a principle of software development aimed at reducing repetition of information...

September 6, 2024

Apache Kafka in KRaft Mode With RisingWave – DZone

News

Apache Kafka in KRaft Mode With RisingWave – DZone

Over the past few years, Apache Kafka has emerged as the top event streaming platform for streaming data/event ingestion. However, in an earlier version of Apache Kafka, 3.5, Zookeeper was the additional and mandatory component for managing and coordinating the...

August 23, 2024

Apache Flink 101: A Guide for Developers – DZone

News

Apache Flink 101: A Guide for Developers – DZone

In recent years, Apache Flink has established itself as the de facto standard for real-time stream processing. Stream processing is a paradigm for system building that treats event streams (sequences of events in time) as its most essential building block....

August 9, 2024

Evolution of Data Sharding Towards Automation and Flexibility in Apache Doris | HackerNoon

News

Evolution of Data Sharding Towards Automation and Flexibility in Apache Doris | HackerNoon

To handle large datasets, distributed databases introduce strategies like partitioning and bucketing. Data is divided into smaller units based on specific rules and distributed across different nodes so databases can perform parallel processing for higher performance and data management flexibility....

July 20, 2024

Apache Hudi: A Deep Dive With Python Code Examples – DZone

News

Apache Hudi: A Deep Dive With Python Code Examples – DZone

In today’s data-driven world, real-time data processing and analytics have become crucial for businesses to stay competitive. Apache Hudi (Hadoop Upserts and Incremental) is an open-source data management framework that provides efficient data ingestion and real-time analytics on large-scale datasets...

July 2, 2024

Partitioning Hot and Cold Data Tier: Apache Kafka – DZone

News

Partitioning Hot and Cold Data Tier: Apache Kafka – DZone

At first, data tiering was a tactic used by storage systems to reduce data storage costs. This involved grouping data that was not accessed as often into more affordable, if less effective, storage array choices. Data that has been idle...

June 28, 2024

1 2 3