👋 my name is Logu & I am a Data Plumber

Modern Data Stack

& Cloud Tech

based in Melbourne, AU ❤️

Welcome to my world

Hi, I’m Logu Duraiswamy
a Solution Designer. Data Plumber. Logical Thinker.

Proficiency in data modeling is essential for understanding how an organization generates and uses its data, and how different data points connect, Competence in presenting data insights through visual representations to aid in decision-making

Visit my portfolio

Home Workshop

Innovating Sustainability: From Solar-Powered Smart Displays to the Modern Data Stack and Beyond

A Fusion of Eco-Friendly Home Automation, Data-Driven Insights, and Automated Weather Forecasts

In the dynamic landscape of technology, this blog unveils a narrative that seamlessly intertwines three captivating stories: the transformation of broken laptops into solar-powered smart displays for sustainable home automation, the exploration of the modern data stack’s capabilities in handling real-world data, and the integration of automated weather forecasts using OpenWeather, Apache Airflow, Apache Kafka, and Telegram. Brace yourself for a comprehensive journey that fuses innovation, sustainability, data-driven insights, and automated weather updates.

Repurposing Broken Laptops for Sustainable Home Automation

The quest for sustainable living takes a bold step as we delve into repurposing broken laptops into solar-powered smart displays. This initiative goes beyond technical feats, showcasing the potential of eco-friendly innovation. The blog guides readers through the step-by-step process, from installing Home Assistant and transforming a discarded laptop into a smart display to augmenting its functionality with solar energy and battery storage. Witness the resurrection of outdated technology, creating an environmentally conscious and technologically advanced setup for the home.

More detail

The Modern Data Stack: Empowering Data-Driven Decisions

As we embark on this sustainable journey, the spotlight shifts to the modern data stack—an ensemble of integrated tools and processes designed to redefine how organizations handle data. This section introduces participants to the practical aspects of ELT/ETL processes and data modeling, using weather data sourced from the OpenWeather API as a tangible example.

More detail

Automated Weather Forecasts with OpenWeather, Apache Airflow, Apache Kafka, and Telegram

Taking innovation a step further, we introduce the integration of OpenWeather, Apache Airflow, Apache Kafka, and Telegram for automated weather forecasts. Participants engage in a home workshop, leveraging the modern data stack to automate the delivery of daily weather forecasts. The workflow, orchestrated by Apache Airflow, collects weather data from OpenWeather, processes it, and uses Apache Kafka to facilitate seamless communication. Telegram delivers daily forecasts every four hours, from 6 AM to 9 PM, accompanied by a random proverb Thirukural verse for a touch of inspiration.

More detail

Bridging Two Worlds: Sustainability, Data Integration, and Automated Forecasting

This synthesis of sustainable home automation, the modern data stack, and automated weather forecasting offers participants a holistic understanding of integrated data processing. The blog underscores the potential of the modern data stack in handling diverse data sources and its role in driving data-informed decision-making. The automated weather forecasts serve as a practical application, showcasing the power of technology in delivering real-time, actionable insights seamlessly.

Conclusion: Harmonizing Sustainability, Data Insights, and Automation

In conclusion, this blog serves as a harmonious convergence of sustainable home automation, the modern data stack, and automated weather forecasting. Through the innovative repurposing of broken laptops, exploration of data processing tools, and integration of automated workflows, participants gain not only technical insights but also a profound appreciation for the symbiotic relationship between innovation, sustainability, data-driven decision-making, and automated processes. It’s a journey that invites individuals to embrace a future where eco-friendly practices, cutting-edge technology, and automated insights coalesce to shape a smarter, greener world.

Visit my portfolio

Home Workshop

Simplifying Secure Connections: Setting Up Nginx Reverse Proxy with Certbot for Let’s Encrypt SSL Certificates

LIKE THIS 9

High-level architecture diagram:

Prerequisites:

Nginx Installed: If Nginx isn’t already installed, you can install it using your package manager. For example, on Ubuntu:
```
sudo apt update
sudo apt install nginx
                    
```
Domain Pointing to Your Server: Ensure your domain is correctly pointed to your server’s IP address.

Certbot Installed: Install Certbot on your server:

sudo apt-get update
sudo apt-get install certbot python3-certbot-nginx

Configuring Nginx Reverse Proxy for Each Service:

For each service, create a separate Nginx configuration file in /etc/nginx/sites-available/:

sudo nano /etc/nginx/sites-available/SERVICE_NAME

Replace SERVICE_NAME with the actual service name and BACKEND_ADDRESS with the address of your backend server.

Example for Airbyte (airbyte.logu.au):

server {
    listen 80;
    server_name airbyte.logu.au;

    location / {
        proxy_pass http://BACKEND_ADDRESS;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    location ~ /.well-known/acme-challenge {
        allow all;
        root /var/www/html;
    }
}

Repeat this step for each service, customizing the server_name and proxy_pass accordingly.

Enabling Nginx Configuration:

Create symbolic links to enable the Nginx configurations:

sudo ln -s /etc/nginx/sites-available/SERVICE_NAME /etc/nginx/sites-enabled/

Testing and Restarting Nginx:

Ensure there are no syntax errors:

sudo nginx -t

If no errors are reported, restart Nginx:

sudo service nginx restart

Obtaining Let’s Encrypt SSL Certificates:

Run Certbot for each service:

sudo certbot --nginx -d SERVICE_NAME

Follow the prompts to configure SSL and automatically update Nginx configurations.

Testing Renewal and Updating DNS Records:

Test the renewal process for each service:

sudo certbot renew --dry-run

Ensure DNS records for each service point to your server’s IP address.

Verifying HTTPS Access:

Access each service via HTTPS (e.g., https://airbyte.logu.au). Ensure the SSL padlock icon appears in the browser.

Repeat these steps for each service, replacing SERVICE_NAME with the respective service’s domain.

By following these steps, you’ll have a secure setup with Nginx acting as a reverse proxy and Let’s Encrypt providing SSL certificates for each service. Adjust configurations as needed based on specific service requirements. Stay secure and enjoy your enhanced web services!