.jpg)
![[The AI Show Episode 151]: Anthropic CEO: AI Will Destroy 50% of Entry-Level Jobs, Veo 3’s Scary Lifelike Videos, Meta Aims to Fully Automate Ads & Perplexity’s Burning Cash](https://www.marketingaiinstitute.com/hubfs/ep%20151%20cover.png)
![From electrical engineering student to CTO with Hitesh Choudhary [Podcast #175]](https://cdn.hashnode.com/res/hashnode/image/upload/v1749158756824/3996a2ad-53e5-4a8f-ab97-2c77a6f66ba3.png?#)
_WindVector_Alamy.jpg?width=1280&auto=webp&quality=80&disable=upscale#)
![Galaxy Watch 8 Classic goes up for sale with hands-on photos showing new design, more [Gallery]](https://i0.wp.com/9to5google.com/wp-content/uploads/sites/4/2025/06/galaxy-watch-8-classic-ebay-1_a53c3c.jpg?resize=1200%2C628&quality=82&strip=all&ssl=1)
![Watch Apple's WWDC 2025 Keynote Here [Video]](https://www.iclarified.com/images/news/97538/97538/97538-640.jpg)
How I Migrated a Monolith Cron Job to a Serverless Architecture
A brief background about the app It’s an app where users upload their data into categories. The app processes that data, runs some additional calculations and aggregations, and stores everything in a database. After that, users can view the resulting data in charts and grids on the website. There’s also a requirement to capture that data and upload it as CSV files to an S3 bucket — daily, per account and per category. There are around 20 categories and 10,000 accounts. So in total, the app needs to upload about 200,000 CSV files to S3 every day. How it was It all started with a simple cron job that ran a script every day at 5 AM. The script did 3 things per account and category: Loaded data from the MySQL database Prepared the CSV file Uploaded it to S3 Here’s what it looked like: Problems
A brief background about the app
It’s an app where users upload their data into categories. The app processes that data, runs some additional calculations and aggregations, and stores everything in a database. After that, users can view the resulting data in charts and grids on the website.
There’s also a requirement to capture that data and upload it as CSV files to an S3 bucket — daily, per account and per category.
There are around 20 categories and 10,000 accounts. So in total, the app needs to upload about 200,000 CSV files to S3 every day.
How it was
It all started with a simple cron job that ran a script every day at 5 AM.
The script did 3 things per account and category:
- Loaded data from the MySQL database
- Prepared the CSV file
- Uploaded it to S3
Here’s what it looked like:
Problems
