Bridge

NiFi cache Cleanup

This article provides instructions for automating the cleanup of Apache NiFi cache directories using a shell script. The cleanup process involves safely stopping the NiFi service, removing temporary and repository data, and restarting the service. It also supports automation through cron, making it suitable for scheduled maintenance with minimal manual intervention.

Purpose

This document outlines a shell script to automate the cleanup of NiFi cache directories. It is intended for use in environments where Apache NiFi runs as a systemd service. The script stops the service, clears cache data, and restarts the service — suitable for automated execution through cron or another job scheduler.

Prerequisites

  • Apache NiFi is installed and managed via a systemd service.
  • The script is executed as a user with root privileges.
  • No manual input or prompts are required, enabling seamless automation.

Script: nifi_cache_cleanup.sh

#!/bin/bash

# ----------------------------------------

# NiFi Cache Cleanup Script

# Maintained by: <Your Team>

# Created on: <Date>

# ----------------------------------------

# Set the NiFi base installation directory ex:/home/ubuntu/nifi-1.16.3

NIFI_DIR="<your_nifi_install_directory>"

# Define the systemd service name for NiFi

NIFI_SERVICE="nifi"

# Validate NiFi installation directory

if [ ! -d "$NIFI_DIR" ]; then

    echo "Error: Directory $NIFI_DIR does not exist."

    exit 1

fi

echo "Stopping NiFi service..."

systemctl stop "$NIFI_SERVICE"

# Wait for complete shutdown

sleep 10

# Define cache directories to clean

CACHE_DIRS=("content_repository" "database_repository" "flowfile_repository" "provenance_repository" "work")

echo "Cleaning NiFi cache directories..."

for DIR in "${CACHE_DIRS[@]}"; do

    TARGET="$NIFI_DIR/$DIR"

    if [ -d "$TARGET" ]; then

        rm -rf "$TARGET"

        echo "Cleaned: $TARGET"

    else

        echo "Warning: Directory $TARGET not found."

    fi

done

echo "Starting NiFi service..."

systemctl start "$NIFI_SERVICE"

echo "NiFi cache cleanup competed successfully."

Note: Ensure this script has execute permissions and is stored securely.

 Example: 

chmod +x nifi_cache_cleanup.sh

Automation via Cron

Edit Root’s Crontab: 

sudo crontab -e

Add Cron Entry

To schedule the script daily at 2:00 AM:

0 2 * * * /path/to/nifi_cache_cleanup.sh >> /path/to/nifi_cleanup.log 2>&1

Logging

  • The script supports standard output redirection for logging.
  • Logs can be stored anywhere as per your logging policy.