Work Journal

oEmbeds, Facebook, and WordPress

We want to take some url on somebody else's service and embed it on our page. We could create an iframe ourselves, but then we're fiddling with a sometimes very large chunk of html stuck in our page and we're maintaining that forever. What if the embed needs to change somehow in future (becoming fluid and not fixed-width, maybe)? Also we're asking for that embed every time our page loads, that's a lot of requests our page has to make.

What if the service (a 'provider') could send us prerendered html for that url we want, so instead of an iframe or chunk of html in our content, we have a reference to a cache of that html in our site and the service can send us what that embed html should be? That's the idea behind oEmbeds.

WP registers a bunch of oEmbed providers – as it renders content, it checks for urls (regex patterns) in certain formats and if it finds one, that triggers a request to that provider to see if it can get some html it can cache (you can change those providers, or even add your own).

FB doesn't mind you doing this, but it wants us to get permission for it beforehand so it can connect you with what you're doing with their API, unlike other providers. So we need an FB app, with a domain set to our site's domain, which has 'advanced' permission to 'read' oEmbeds (I'm still unclear on what exactly 'standard' permission allows, if anything). We're going to need credentials to send along when we ask for an oEmbed response: the FB app's ID, and its secret which are found in 'basic' settings for that app.

So who do we now ask for that html? In IG's case we make a GET to: https://graph.facebook.com/v8.0/instagram_oembed?access_token=$_FBAPPID|$_FBAPPSECRET&url=$_SOME_IG_URL

(If you have an http client like Postman, Insomnia, or my favorite Paw you can more easily look at what you're sending and what FB is responding with.)

What we expect FB to pass back to us in the response body is json containing a heap of info about the embed, like its author, dimensions, thumbnail images – and most importantly for us, a big wad of html. (If you do not get a 200 response with that stuff included, you'll instead get a 400 with an error message telling you the embed isn't allowed – either because the post has been marked private, or that your app doesn't have permission to do this at all yet.)

That html is going to get saved to post meta by WP with a key of _oembed_$_HASH_OF_THE_EMBED_URL. There'll be an additional meta with a timestamp as well. This does a couple of things: caches that html for I think 24 hours, and also makes it accessible to other article pages which might end up embedding the same url – no sense fetching and caching it again; if we're going to embed the same thing on a number of pages, let's use the same cache for it. (I think in Gutenberg's case it saves a post, not post meta, with a post_type of oembed_cache. I've been avoiding Gutes so I haven't looked into this in detail.)

If instead you got back a 400 with no html you can cache, you're going to end up with the dreaded `` value for that post meta, and your embed url is just going to sit there inert on your page looking sad. Every so often it's going to try to fetch that embed url again, as the article page is visited, so don't panic if you have a lot of these; they should get refreshed with the actual html once you have access to it. (If you want to get rid of all those at once in your DB because you hate clutter, that's fine too.)

Ok, so how do we get FB to let us do this? You'll need to open an app review in your FB app, asking for advanced access to oEmbed Read. In the first section ('Provide detailed step-by-step instructions on how a reviewer can test your integration') say you're trying to use oEmbeds, and provide a specific url of a page on your site you want to put an embed on. Then in the next section, which isn't very clear about what it really wants ('Please provide a URL where we can test Oembed Read') give them the FB/IG url of what you're trying to embed.

If they reject you, it's most likely that the mods are confused by how you worded things, not that they consider you hostile or anything – you can always ask again. Also, requests are handled by different people on different days, and they look at other data relating to your site which informs their decision, so in some cases you may have to ask a couple of different times in slightly different ways, or correct some other problem they have with you. But for this particular thing – caching oEmbeds – you're not asking for much, there's not a big reason for them to contest it.

Beyond review, all this stuff – checking for FB and IG urls in your content, making requests for oEmbed html with your credentials, caching them – is handled by the oEmbed Plus plugin among others. There are a number of different endpoints to hit and regexes for urls for FB and IG, and that's all kept track of (they might change once again!).

As of this writing the plugin's using v8.0 of the FB graph api; ideally, your FB app should be at the same version or above, so check that.

The NOSQL Store that Everyone Ignored

Incredible talk by Zohaib Sibte Hassan for Postgres Conference – The NOSQL Store that Everyone Ignored. After having spent so many years with MySQL, Postgres and what it's capable of is fascinating ...

Musk’s Five-Step Engineering

Elon Musk is a problematic character, but his accomplishments are extraordinary and listening to him speaking extemporaneously on anything is fascinating. Recently he gave an impromptu tour of SpaceX facilities and while going into detail about rockets he spoke about engineering generally, talking about a five-step program he was trying to inculcate people with. It was worth taking notes on – these are his five steps that if done in order prevent wasting time and effort:

1. Make your requirements less dumb

In other words, question the brief. Requirements must also be attributed to people, not departments (allowing requirements to vanish when no longer relevant rather than be permanent backlog fixtures)

2. Delete steps/parts/processes

The bias is to add things “just in case” – but if you're not adding things back after having removed them 10% of the time you're not deleting enough

3. Simplify or optimize

Most common mistake is to optimize things that shouldn't exist

4. Accelerate the cycle time

5. Automate

Also:

Everyone must be Chief Engineer

Everyone must understand the project as a whole to judge whether the decision on the part is the right one

Svelte SSGs

As much as I like Eleventy as an SSG I’ve been thinking that it‘s a bit trapped by its reliance on CJS caching for its internal workings. I'd like my next SSG to use for this site to be ESM-first as well as Svelte-based – Sapper, the official Svelte-based SSG, has been overtaken by a series of other attempts: JungleJS; Plenti (which is intriguingly written in Go); Routify.

Now we have Elder (the name happily reminds me of one of my favorite metal bands but it's due to it being used to back a catalog of senior-living homes) – I’ll have to spend some more time looking into it but so far it looks very interesting, especially the hook system.

There's also SSGO now, which is a skeletal Reno-based SSG – there's a lot to like here, the concept of 'creators' has a simple appeal.

All of this of course is a horrible distraction from learning Typescript ...

Talks, Alpine, Just JavaScript

Via Chris Ferdinandi's Go Make Things tips, two explainers: a terrific talk on this by Steve Griffith, and a great Jeremy Keith talk on service workers – Keith's talk isn't especially deep, but given how confusing SWs can be it's a solid conceptual grounding.

I've seen a few talks/tutorials on Alpine – a tiny script that allows a lot of interactivity based on attributes in html (think Stimulus or htmx with Vue's syntax). I was having a hard time understanding how I could put it to use until I saw Caleb Porzio's own walkthrough.

Abramov's Just JavaScript course – through which I learned that null has a typeof object – has ended for now, which is a shame since I was really enjoying it. Sounds like the course is popular enough that it's going to be revised and become a website/book, which is terrific news.

TypeScript, WordPress 5.5, the site

On vacation from work, and actively trying to disengage from the day-to-day (there might have to be a post on work-life balance later on), but spending some time learning TypeScript, for which I'm using Maximilian Schwarzmüller's online course. His course on Svelte is also terrific – Svelte itself just announced TS support. I'm starting to see why TS has become so ubiqitous, like Rust it makes it very hard to screw up (at the cost of being much more verbose, but that compiles away); I'm curious to see how long TS will remain dominant before the pattern switches to WASM compilation, which is probably the future. I can't see that happening for at least another five years, but that's just a guess.

A lot of upcoming stuff in WordPress – finally being able to pass arguments to template parts (not sure why this was such a long time coming), sitemaps, environment variables, Blink/Gecko native lazy loading ... some things, like automatic updates for themes and plugins, don't really affect my work as I only have one client and everything is built for that use case. If I were writing themes or plugins for sale I would have an incentive to accommodate that as well as everything relating to Gutenberg (something I'll have to deal with eventually but I wanted to let the whole project settle and have best practices evolve first).

I should take a moment and talk about the site – my experience with Eleventy has been terrific, once you understand how data is passed around and how routes are made it's a pleasure to work with. The only problem – and it's not really a problem, just something I wish was different – is that it's CommonJS, and that's fundamental to its operation. This means that using ES6 module patterns won't fly in a lot of cases where you'd want or expect to use them. The SSG space is on fire right now with a new framework appearing nearly every day so I'd expect someone to recreate Eleventy as ES6-first (Twelvety?), or for some other SSG to capture mindshare among devs.

The Vercel deployment process (née Now), like nearly everything they do, is nearly magical, and is a best-in-class experience everyone should be aiming for. Often times the packaged solutions companies offer to abstract some pain point simply relocate the pain, in sort of a aggravation-neutral approach – but Vercel seems to know how to do it well.

Future Fonts is a brilliant way of supporting people who are making typefaces – you buy them, but also receive updates to them as they're refined and built out. So many new faces have very little personality and are incredibly derivative, whereas the people doing stuff for Future are very inventive. Sometimes the result isn't something you could ever imagine using, even for a display face, but the Overton window of what's acceptable in design is widening after what seemed to me a period of blandness.

A note about the color schemes – they're all based on F1 liveries and helmets. Right now I'm enjoying switching color schemes randomly on every page load, but I know that's not exactly typical behavior for a site, so I might change that later. Or not! It's my site!

Amplify Datastore, programming languages

AWS offerings can be obscured by clouds of their own marketing so it was great to hear ex-AWSer Daniel Vassallo on Software Engineering Daily talking about various AWS bits in plain language. Picked up his book The Good Parts of AWS which so far is really informative and demystifies some of the AWS ecosystem.

One of the more recent AWS solutions is DataStore, which I ran into via reading Shawn Wang’s writeup in reference to using it with Svelte. It sounded brilliant but the presentation at re:Invent by Richard Threlkeld was repeatedly mindblowing; it packages up a comprehensive solution for caching, offline, concurrence, conflicts ... after watching it I can't think of building any future project without it.

Spent some time looking into Nim, Go, and Rust – all are fascinating but Rust seems to be the one that is ... about to explode? Is exploding? Has exploded and I’m only now catching up? Sheshbabu Chinnakonda’s guide to Rust for JavaScript people is terrific, and Sean Grove and Steve Klabnik’s talks about wasm and Rust are fascinating.

As amazing as Rust is, and as excited to learn it as I am, I have to be realistic about where the market is and where my strengths are. I’m much more likely to be working on the front end and in JavaScript so improving my debugging and learning TypeScript is a better use of my time, even if it feels like TS is verbosely ‘correcting’ features of JS which Rust has built-in.

Journal inspired by Darin Doria and Sam Selikoff