Data pipelines

5 entries

2026

View case study
You Inc

I wanted one view of what I own, owe, earn and spend. I built You Inc around a ledger, then took the accounts and billing back out so people could run it on their own machine.
- Architecture
- Data pipelines
- Finance
- Front-end development
- Self-hosted
- UX design
Source code
- 20 January 2026 Writing
  Putting my Apple Health data in one place
  I brought my nutrition and activity data out of Apple Health and into Postgres, so I could compare the daily and weekly trends.
  - AI
  - Architecture
  - Data pipelines
View case study
HealthAgent

My nutrition and activity lived in different apps. I brought their Apple Health exports together so I could see the trends in one place.
- AI
- Architecture
- Data pipelines
- Front-end development
Source code

2025

- 9 June 2025 Writing Report
  Wildfire Analysis with PySpark
  I used PySpark to train and compare models on 1.88 million US wildfire records. The report covers the features, training runs and what happened when I weighted the rare classes.
  - AI
  - Data pipelines
Recorded random-forest results: class G recall rose from zero to 0.7518 with weighting; class G precision was 0.0099 and overall accuracy was 32 percent.
PySpark experiment

1.88M records

Class G recall

0 → 0.7518 With class weighting

Weighted class G precision

0.0099

Weighted overall accuracy

32%

Recorded random-forest results. Class weighting finds more rare fires, with many more false positives.

View case study
Wildfire PySpark

I worked with 1.88 million US wildfire records for a university project, and got into how a model could look accurate while missing the larger fires.
- AI
- Data pipelines