lambda
For my portfolio tracker project (portofli-zen.vercel.app), I wanted to build a feature which takes any arbitrary input like screenshot, PDF, or Excel and extracts the holdings from the input using LLMs, then adds them to the portfolio.
Using a simple REST API endpoint for this wasn't feasible. The response time would be more than 5 seconds in most cases since each request would first need to load the file, make an external API call, wait for the response, and then insert the rows into the database.
This was a classic background job which had to be processed asynchronously.
For the UX, we can keep polling the status of the job at fixed intervals. The job can be pushed onto a message queue like Redis, and a worker running in the background can continuously poll for jobs, pop one from the queue, and process it.
This works well, and in a hypothetical case where we have multiple users trying to upload their portfolios at the same time, we can easily scale up by increasing the number of workers.
This wasn't really a requirement for me since this project has no other users, but I wanted to build it properly.
The problem with this approach is cost.
This architecture requires at least one worker to be provisioned regardless of how many active users we have or how many jobs are currently in the queue. Running a lowest-spec EC2 instance for a month can cost somewhere around $7-8 once you account for storage and other infrastructure costs. The cost increases if the machine needs more RAM or CPU.
For a hobby project, that's a lot of money, especially with the rupee falling.
This is where AWS Lambda and SQS come in.
The system is fairly simple. The user uploads their file, we upload it to a storage provider and get a private URL. I'm using Supabase Storage for this.
We then push a job onto SQS containing the file URL. Lambda can be configured with SQS as an event source, so when messages are available in the queue, Lambda invokes the function and processes them.
The Lambda function does everything that the worker was doing before.
This makes the architecture much more suitable for a low-volume application like mine. Instead of paying for a worker that's running 24/7, I'm only paying for compute when there is actually a job to process.
There are two downsides i noticed. Cold start and Deployment.
If the lambda function is not invoked in a long time, it needs a cold start which adds to the latency of entire process. The user has to wait for a long time before their holdings are loaded. However, according to AWS, only 1% of lambda invocations are cold starts and for a function using Python as runtime, P99 is 600 ms. For the cost we are saving, this is a reasonable trade-off.
Deploying Lambda and setting up CI/CD is more complicated than deploying a normal worker. In my case, the Lambda function has quite a few dependencies, including OpenCV and PaddleOCR, which makes packaging and deployment more annoying.
Once this is set up, though, the development process becomes much easier.
There are probably other approaches I could have taken, but for a hobby project where jobs arrive sporadically, this felt like a reasonable trade-off.