Airbnb Data Pipeline & Investment Strategy

Overview

This project simulates a data-driven real-estate investment consultancy. The goal was not just to visualize data, but to build an analytical product that answers: “where should capital be invested for the highest return at the lowest risk in short-term rentals?”

Using public Inside Airbnb data, I built a pipeline that ingests, sanitizes and enriches raw data, culminating in a prescriptive recommendation tool for investors.

Architecture and methodology

Structured around software and data engineering best practices, with responsibilities split into modules:

Ingestion and processing (ETL)

• Statistical cleaning — IQR (interquartile range) rules automatically remove price outliers, keeping extreme-luxury listings and data-entry errors from skewing the market analysis;
• Feature engineering — estimated revenue (occupancy × nightly rate, based on the “San Francisco Model”) and amenity valuation: an algorithm parses unstructured amenity lists to isolate the financial impact of each item.

Optimized storage

Parquet with Snappy compression instead of CSV for the processed layer — strong typing (schema enforcement) and high-performance reads for the frontend.

Visualization and delivery

A Streamlit web app where users filter opportunities by budget and explore “profitability heat zones” on interactive maps, plus an opportunity matrix: a scatter plot isolating neighborhoods in the “low cost / high return” quadrant.

Business results

The automated analysis flagged the Waterfront Communities — The Island micro-market as the ideal investment target: cost efficiency above the city average with minimal revenue drop-off versus the luxury market.

Field notes

Next steps focus on moving from analysis to robust data engineering:

• Containerization with Docker (dev/prod reproducibility);
• Orchestration with Docker Compose;
• CI/CD with GitHub Actions (tests and image builds on every push).