🎧 Kaizen 14 - There goes my PgHero - Apr. 5, 2024

[gerhard]

🎧 This is now available as changelog.com/friends/38

See all improvements in this 🐙 GitHub repository: Kaizen 13 ... 14

---

This contains all things related to the next Kaizen episode.

Share your thoughts via comments below:

• Is there something in changelog.com that you, our dear listener, would like us to tackle?
• Is there something that you would like to help out with?
• Anything that we - @adamstac @jerodsanto - want to improve?

A rough timeline of what to expect (red line is today):

gantt
    title 🎧 Kaizen 14 timeline
    dateFormat  YYYY-MM-DD
    axisFormat %b %e, %Y

    section CONTRIBUTE
    💻 ALL CONTRIBUTIONS WELCOME :2024-01-12, 77d

    section RECORD

    🎙 Record :milestone, 2024-03-29
    ✂️ Edit :2024-03-29, 7d
    🚀 SHIP IT! :milestone, 2024-04-05

    section NEXT
    Kick-off Kaizen 15 :2024-04-05, 7d

Loading

---

What changed? Kaizen.13 ... Kaizen.14

• Did we build a CDN?
  • 🤔 Error 503 first byte timeout - only have the db context in the trace #501
  • 😔 CDN cache misses increased 7x after Oct. 8, 2023 #486
  • 🙂 https://bunny.changelog.com cc @lawik
    • Changelog on Fastly
    • Changelog on Fly
    • Changelog on Bunny
    • Changelog on Cloudflare
  • 😔 When logged in, loading the home page can take more than 30s #506
• What about Neon.tech Postgres?
  • Shout-out @brendan-stephens from Neon.tech Support
  • 🙂 Deploy PgHero & connect to Neon.tech Postgres #507
  • 🤩 Enable changelog.com devs to create prod db forks with a single command #508
    • Shout-out to @kislerdm for https://github.com/kislerdm/neon-sdk-go
    • Is it enough?
    • What would make it better?
    • Easily Anonymize Production Data in Postgres
• 55 commits later & 0 PRs
• Make it Work

---

What else is coming up?

• Daggerize action-slack-notify
  • https://github.com/gerhard/daggerverse/tree/main/notify
• The CDN saga™
• Daggerball on K8s
  • Functional CI - https://daggerverse.dev
  • New UI
  • One more thing...

---

This is a follow-up to

• 🎧 Kaizen 13 - Should we build a CDN? - Jan. 12, 2024 #485
• 🎧 Kaizen 12 - Slightly more instant - Oct. 13, 2023 #480
• 🎧 Kaizen 11 - S3 R2 B2 D2 - Aug. 11, 2023 #469

[gerhard]

docker run -it -e DATABASE_URL="$(op read op://changelog/neon/url --account changelog.1password.com --cache)" -p 8081:8080 ankane/pghero

[gerhard]

If anyone wants to compare our 3 setups part of the CDN Saga™️ series, you can do so via:

1. https://changelog.com/ via Fastly
2. https://changelog-2024-01-12.fly.dev/ via Fly.io
3. https://changelog-2024-01-12.b-cdn.net via Bunny CDN

These are the response results for Mar 8 - Apr 7, 2024 according to https://hyperping.io

1. https://changelog.com/ via Fastly - 392ms

Checking the headers by running httpstat locally:

2. https://changelog-2024-01-12.fly.dev/ via Fly.io - 265ms

Checking the headers by running httpstat locally:

3. https://changelog-2024-01-12.b-cdn.net via Bunny CDN - 59ms

Checking the headers by running httpstat locally:

---

FTR: https://www.cdnperf.com/

[lawik]

But since you are running on Fly you have to test their object storage offering with built-in CDN-style caching. Tigris (who I am currently working with so enormously biased) seem like a great fit for this. Push your stuff with eager caching if you want immediate low latency, otherwise you get the usual second-fetch being fast.

[adamstac]

I mentioned them because Lars intro'd me to them. The frequency bias is still one layer, not two.

[lawik]

Yeah, wrote this before I listened.

1. if you want geo-replication with R2 you better enjoy writing edge worker Cache API code because R2 doesn't do it by default. The marketing says yes. Threads on forum says no.
2. Yes, for buckets. Also can be actively told to cache eagerly which is rad. Not for your pages though unless you render them out to disk.
3. No. Not that I am aware.

It essentially rounds out the Fly offering of putting your apps close to your users by also putting your files close to your users.

if you ran more app instances in a geo-distributed fashion and had some light caching in-app on the heavily used pages you'd probably get good results.

But most likely you'll find one or two pathological cases where moving your app and DB too far apart makes something awfully slow.

[lawik]

Also, cool to be a bias with frequency. Signal all on my own :D

[gerhard]

Just to double-check my understanding:

1. Yes, with extras (e.g. object geo-replication that just works)
2. Not for files that do not exist in the bucket. Caching is configured on the object store, not on origins.
3. 👍

Based on the above, this still sounds like an R2 alternative with the added benefit that we could serve those files directly from the object store cache which can be configured to replicate closer to end-users. Tigris doesn't behave as a CDN in the sense that we can't declare origins, request rules, etc.

[lawik]

Yeah, it is more of an R2 alternative than a CDN in that regard. It will CDN your media files and all that but unless you render out your pages to file on change it will not solve your landing page etc.