Earth Notes: On Website Technicals (2019/01)

Tech updates: FIXME a happy new ear, cssgip, work storage, AMP srcset, LEDs, details, 400kpx image warning...
What might happen in the New Year on this site, including a snapshot of some items from my secret to-do and FIXME list, and lots of AMP-related fiddling...

2019/01/20: CRP Head

With all the different permutations of link rel between page versions, dynamically including support for ads where supported, and optional preloads for speed, keeping the header and CRP (Critical Rendering Path) small enough to get out in the first TCP frame is increasingly complex!

I already had in place an optional -H flag for the main page-generation script to minimise the header if a first attempt created an oversize one. That has had to be made a little more brutal, eg to shut out ads on AMP since ad support for those pages must be inserted in the head. But in general this lets my optimistically cram more into a header than I might otherwise, allowing a valid page to be created by discarding some non-essential stuff if necessary.

Alongside that I have now put in place a mechanism that when large optional chunks are being added to the CRP a LARGEHEAD flag is set. Then some really-optional stuff such as preloads can be silently dropped even without the -H flag, ie a more gentle slimming of the header content on the first attempt.

2019/01/19: Auto Ads Bad?

Having never has a single AMP AdSense 'auto' ad appear that I know of, and being rather concerned that there are just too many non-AMP AutoAds on a page, I'm reverting to having my code insert ads in sensible places. I can nominally switch back to AutoAds at any time with a one-line change.

(Those sensible places are basically immediately above headings. These seem good breaks in the reading flow to interrupt with an ad, if any are. However, the W3 Nu Html Checker does not like anything injected between the start of a section or article and the h2 or whatever that should be its first child. So section and article are also injection points, and should then get the ad rather than the immediately-following heading.)

I have tweaked my code to inject up to a tunable limit of ads on longer pages, less than the ~8 of Auto Ads.

There is still no automatic ad injection for 'lite' pages.

Now I have 'manual' AMP ads showing, in the Chrome and Firefox responsive/mobile developer view and in my Opera Mini on Fairphone, but not on a desktop browser view. In fact, the desktop view (Firefox, Chrome, Safari) is a big ugly empty space that doesn't collapse. Why? What were they thinking?

2019/01/20: curiouser and curiouser. AMP ads will display on (for example) desktop Firefox if the viewport is narrow enough.

2019/01/17: 400k Pixel Minimum Area?

I think the threshold at which GSC issues the "Image size smaller than recommended size" warning may be 400k--500k pixels area. I set my script's warning threshold to 500k for when building a page, and all but a handful are now above that. The smallest area hero was below 300k, and when I went and inspected it in GSC there was indeed a warning lurking that it had not yet volunteered to me. So I beefed up that image and GSC is happy for that page.

It will be interesting to see if GSC complains about the sub-500k images, some of which will be hard to get higher-resolution versions of.

After something of a hiatus during Google's global index update, the new AMP pages seem to be being absorbed again.

2019/01/15: New GSC Image Warning

Today I received an interesting and new warning from the Google Search Console (GSC) for about eight of my EOU pages re "Linked AMP version is valid with warnings" and "Image size smaller than recommended size". The Google guidance says to aim for the page image (or images, in different aspect ratios) to be at least 1200px wide and at least 800k pixels area. While almost all my hero images meet the first of those two, the latter is often not going to be met right now.

The warning arrived by email, and is all described in more detail in the on-line console.

As an experient I fixed one of the sample/example pages to meet the requirements, and the live test indicated that I had indeed fixed the issue.

Quite an efficient and effective workflow, and the change is good for (eg) Twitter postings too.

While complying with Google recommendations is not necessary, if done right doing so benefits pretty-well all uses and users of the page.

Good one Google!

2019/01/11: AMP details, details

Hurrah! As of 5pm GSC had stopped complaining about use of details and summary in my 'canary' page.

So I put back use of this in the home page also, and will, sunshine permitting, roll out the 'Contents' feature to all suitable AMP pages in the next day or three. Excitement! (The Contents implementation depends on details.)

Since I'm having fun and forcing lots of stuff to be rebuilt anyway, I'm taking this opportunity to slightly reduce the threshold on page size before auto-injecting a Contents section.

I've also slightly improved the image background colour (to show while images are loading) to be omitted for images with any transparency, and where that background colour would be the same as the page (ie white). The former is more a correctness issue, the latter a space optimisation.

Also, while the gloves are off, I'm now cross-linking the m-dot and AMP pages, both in the header (link rel=) and in the top navigation.

2019/01/06: AMP LED Reviews

After a considerable amount of rework and general plumbing, the LED lamp review page is available in AMP!

2019/01/04: AMP Progress

According to Google Search Console it seems to be absorbing about eight AMP pages per day (out of ~200 candidates) into the index, and is ~40% done.

GSC is reporting a fairly strong swing away from showing m-dot pages towards AMP pages. It's not yet clear if there will be any switch from desktop pages to AMP.

I'm not yet seeing any sign of the smallest new srcset images (targeted at 360w x 1.5 density) being used and pulled through the AMP cache. Probably far too soon anyway.

... And then it all paused and indeed one page dropped out of the index, with the number of indexed pages rising again slightly on the 8th. During this interval Google has apparently been rolling out a major index update. It would make sense not to add more pages during such an update.

As of 2019/01/12 the rate is still low, but ~47% of the AMP URLs have been indexed.

2019/01/02: AMP srcset

Following Gregable's comments on apparent AMP cache Oddities, I am now inserting an explicit srcset in autogenerated mobile (amp. and m.) body images, with a new lower bound on device size of 360 CCS pixels at 1.5x pixel density, ie 540px real. This smallest image size is eligible for the desktop srcset too. This can provide ~33% image weight savings for the smallest displays.

I'll see if that works once some AMP pages time out in the cache, such as in the cached version of the Ecobuild Exhibition write-up.

By some trickery I can see that my srcset is indeed honoured. So I may extend this feature to more body images (non-autogenerated), and to hero images such as for the home page carousel.

2019/01/01: Happy New Ear

Old to-do lists are often interesting: what seemed important before often seems much less so now! For future amusement then, if nothing else, is a sample (not hearly formatted) from my secret-squirrel list...

In future I might move most of the live list into public view, but a snapshot still has value.

To-Do

Maybe: No Immediate Fix Required

Evergreen

Jumping the Shark

I may just possibly have started tinkering with the first (cssgip) item, reviving some fast and simple ImageMagick code to extract a mean colour: convert XXX.jpg +dither -colors 1 -unique-colors txt:. That hex colour code can then be dropped into a style attribute eg style="background:#efefef" in the image tag.

A quick run in WebPageTest emulating a 56kbps modem connection shows that once the (Chrome in this case) broswer makes space for the image, the background colour is used for undecoded areas. This will be less useful on progressive images, more useful on typical non-progressive PNGs. It is annoying that browsers will not believe supplied width/height attributes and make space ready, before attempting to load metadata from the source image. For a typical mobile device, with (often) plenty of bandwidth, but high latency, there is not much time between receiving metadata to size the image on the page, and starting to paint it. Maybe with decoding=async the gap might be more usefully filled by this background effect.

Work Storage

Parts of the site such as live stats graphs are (re)built more often when there is more energy available in the off-grid system, eg when the sun is shining brightly and the batteries are full. On the grid this would be known as "Demand Control" or "Dynamic Demand". This simply reduces the frequency with which some work is done when energy is scarce. Work avoided in this case may never (need) to be done at all.

Now I've added to that by relaxing the strictness with which some pages are rebuilt unless energy is abundant. Thus I store up work when energy is scarce, which is an alternative to storing the energy to build those pages. This may also mean that some intermediate builds are never done and that energy is permanently 'saved' rather than just having its use postponed. I have done this for the Gallery for many years.