🔗Page views: ...

Page views per hour over time

🔗PRAwN Stack

🦐 🍤 🦐 🍤 🦐 🍤 🦐 🍤 🦐 🍤 🦐 🍤

A modern page view counter to see how unpopular my project is. Powered by a PRAwN stack (Postgres, React, AWs and Node) in the free tier, deployed using CDK.

We're also using Typescript and NextJS for the frontend.

🔗Features

🔗Design

aws infrastructure

sequence diagram Mermaid Diagram

🔗Performant Analytics with Rollups

We have a graph that shows the pageviews per hour and there's a few ways to make that happen.

  1. Calculate the aggregates when they're requested. This is the easiest and will be up to date but it can be slow and resource intensive if it's requested often.
  2. Setup a materialised view and refresh it on an interval. This is a natural extension of the first solution but will be much faster at the cost consistency because it might be stale. This also might be resource intensive if the table is big enough or the aggregates are complicated enough.
  3. Iteratively calculate the aggregates and store them manually. The materialised view solution might recalculate aggregates that aren't going to change because they might be in the past. E.g the pageviews for last month aren't going to change to why recalculate them. This is the approach we've taken here.

I would recommend starting with the first one then escalating to more complicated solutions as required but I wanted to explore the third solution with this project.

🔗Serverless and Lambda

I'm a serverless skeptic. A few timely things swayed me to try Lambda for this project.

  1. serverless-express allows you to run an express app in Lambda, which means you can run the express part locally for a nice development experience.
  2. I wanted to setup some cron jobs which are well suited to Lamdbda.
  3. AWS CDK makes working with AWS ~~easy~~ less hard.

It turns out the simplest way to setup Lambda with RDS is quite expensive. You have to use a NAT Gateway which costs $0.059/h and $44.25/month. That's expensive!.

There's a few ways to get around this:

This all feels very typical of AWS.

I went with the NAT Instance because thankfully CDK makes it easy.

🔗Almost Completely in the Free Tier

bill

The only thing you have to pay for is Secrets Manager which is $0.4/month.

There's also a budget configured with cdk to alert you when you spend more than $1 usd to remind you when the free tier ends.

After the free tier ends it will cost $40 usd/month.

🔗Load Testing

wrk results

With more load, our application can support a throughput of 570 requests per second on average or about 49.2 million per day at 350ms latency. 350ms is still pretty fast and if we had a higher tolerance for latency then this stack could probably do more.

You can install wrk with:

brew install wrk

Then run it with:

wrk -t1 -c1 -d60s https://prawn.cadell.dev/api/home

If you want to go further with load testing then maybe have a look at vegeta, wrk2 or k6.

Load testing goes really deep apparently but I found How percentile approximation works (and why it's more useful than averages) was a good introduction to percentiles, as well as the hacker news comments, which is where I found How NOT to Measure Latency by Gil Tene, which is amazing! There's also an article similar to the talk but I much prefer the talk (and I don't usually watch talks).

Vegeta, wrk2 and k6 all avoid the coordinated omission problem mentioned in Gil's talk, if you use constant throughput modes. It's called something slightly different in k6 though.

🔗Inspiration

The LAMP Stack (Linux, Apache, MySQL, PHP) is the inspiration for this project. I made a terrible chat app with it when I was in year 10 to get around the school's internet filter but it worked and it was really fun.

The best parts:

Some other people also agree.

I wanted to see if I could create a stack with similar qualities with more modern tools.

🔗Roadmap

🔗Ideas

  1. Email me when the scheduled pageview fails, for basic monitoring.
  2. Setup integration tests.
  3. Set a maximum Lambda concurrency.
  4. Use serverless-postgres for connecting to postgres.

🔗Done