Crontab with logging (Automating Tasks on Your Server: A Step-by-Step Guide to Setting Up Crontab with Python and Shell Scripting)

Automating server tasks can save time, reduce human error, and improve efficiency. In this guide, we’ll walk through how to set up a Python script with logging, create a shell script to run it, and schedule it using cron. Step 1: Create a Python Script with Logging

First, create a Python script to handle your automation task while logging its execution.

Python Script: data_updater.py

import datetime
import logging
Configure logging

logging.basicConfig(
filename="/path/to/your/logfile.log", # Specify your log file path
level=logging.INFO,
format="%(asctime)s - %(levelname)s - %(message)s"
)

try:
# Your script logic here
logging.info("Script ran successfully.")
except Exception as e:
logging.error(f"Script failed: {e}")

This script logs every execution attempt and records any errors for debugging. Step 2: Create a Shell Wrapper Script

To ensure proper execution, wrap the Python script in a shell script.

Shell Script: data_updater.sh

#!/bin/bash
Define log file path

LOGFILE="/path/to/your/logfile.log"
Log the start time

echo "$(date '+%Y-%m-%d %H:%M:%S') - Starting data_updater.sh" >> "$LOGFILE"
Activate virtual environment

source /absolute/path/to/venv/bin/activate
Run the Python script and check success

if python3 /absolute/path/to/data_updater.py &>> "$LOGFILE"; then
echo "$(date '+%Y-%m-%d %H:%M:%S') - data_updater.sh completed successfully" >> "$LOGFILE"
else
echo "$(date '+%Y-%m-%d %H:%M:%S') - data_updater.sh failed" >> "$LOGFILE"
fi

This script ensures the virtual environment is activated before running the Python script. Step 3: Schedule the Cron Job

To automate execution, use cron, the Linux task scheduler.

Open the crontab editor:

crontab -e

Add this line to schedule the script to run daily at 11:45 AM:

45 11 * * * /absolute/path/to/data_updater.sh

In the cron expression, the five fields represent:

┌──────── minute (0 - 59)
│ ┌────── hour (0 - 23)
│ │ ┌──── day of the month (1 - 31)
│ │ │ ┌── month (1 - 12)
│ │ │ │ ┌─ day of the week (0 - 7) (Sunday = 0 or 7)
│ │ │ │ │
0 5 * * 1  command

For example : 0 5 * * 1 This cron job runs at 5:00 AM every Monday, regardless of the day of the month or the month itself.

Step 4: Verify the Cron Job

Check the list of scheduled cron jobs:

crontab -l

Manually test the script to ensure it runs correctly:

/absolute/path/to/data_updater.sh

Step 5: Check Execution Logs

View the log file to verify that the script ran successfully:

cat /path/to/your/logfile.log

Bonus: Enhance Logging with Promtail and Grafana

For advanced logging and monitoring, you can scrape the logs using Promtail and visualize them in Grafana. Check out my post about that.

By following these steps, you now have a fully automated system that runs Python scripts on a schedule with logging for easy debugging. 🚀