NEWSFLASH: my Radbot project is fundraising. Join us cutting bills and carbon!
Earth Notes: On Website Technicals (2017-11)Updated 2019-06-23 15:00 GMT.
Two of the pages on the site now have working Twitter video 'player'
cards, both with YouTube-hosted videos, but one also with a local
.mp4 as a test. They still have an
og:image in place too, and that image is still used as the
I have now embedded in the footer for 'full' pages, links to the hero image (etc) for each page, including any embedded Twitter player as an inline data minimal HTML page!
(It took about a day for Twitter to approve/whitelist the first player card; the next one just worked. I have never needed approval for image cards.)
2017-11-27: Less Defer
For desktops, which should have more network and CPU bandwidth, and
whose final rendering is delayed until the share buttons appear at
the top of the page (ie ATF), I've decided to switch the Share42
script back to
async, but keep
mobile/lite to keep JS load priority low and to nominally defer
parsing as long as possible even through there is no following HTML.
Also changing the header link to the 'desktop' site to 'full'.
2017-11-26: Moar Inlining
I'm now allowing very small images (no more than 750 bytes) to be inlined in places beyond mobile ATF hero images. This now allows a handful of desktop pages to inline their very-compressable simple PNG ATF hero images, and the home page(s) opportunistically also to inline corresponding the simple column hero image versions.
The 750-byte breakpoint for non-ATF content is chosen for a number
of reasons, mainly the assumed cost of the HTTP/1.0 headers (outbound
and inbound) avoided by inlining, plus some other minor savings such
as not needing an actual URL and not needing the
height attributes. When pages get to be served over
HTTP/2 that breakpoint might bear adjusting a little. Note that for
non-ATF content saving a round trip is generally unimportant.
2017-11-16: Image Inlining
For the mobile/lite site, to try to overcome a potentially-lengthy RTT
(Round-Trip Time, ie latency) to fetch the ATF (Above The Fold) hero image
(~300ms on 2G/3G), if the image (or its lo-fi variant as used for
Save-Data is small enough, then it will be inlined directly into
the HTML as a base-64 data URL. (Some other very small hero images could also
be inlined where that would probably save overall bandwidth.)
The 'lite' site is also intended for bandwidth-limited connections such as dial-up. On modern mobile latency is the key constraint, on dial-up it is bandwidth, but the overal effects can be very similar!
This has the effect of preventing caching of the inlined images, and of getting less text in the initial inbound TCP packet(s), but the tradeoff is hopefully a better overall UX.
With this hero inlining in place the entire mobile home-page ATF content is rendered from the HTML on the first round trip in ~1.7s according to WebPagetest's 'simple' test: Dulles, VA - Moto G4 - Chrome - 3GSlow. The (gzipped) home page HTML is still well within the initcwd at 8.4kB.
Interestingly the sizes of the mobile/lite and main gzipped pages are now fairly similar, the inlined image filling the gap taken by minimising the HTML boilerplate.
Given Prefer DEFER Over ASYNC: "DEFER scripts don’t execute until the HTML document is done being parsed",
I'm testing switching the already-at-the-end Share42 script from
defer to possibly marginally improve
behaviour on mobiles in particular (CPU starved, share icons not shown ATF).
A preliminary WebPagetest suggests at least that the loading of the script stays late (low priority) and that there is no obvious harm from doing this.
defer seems to be slightly less supported than
Opera was reported as not.
2017-11-12: Even Liter
As an experiment I'm dropping most of the ads from the mobile/lite site as the nominal revenue (loss) is tiny but the effect on page weight is huge! Partly inspired by "Banner Ads Considered Harmful (Here)".
2017-11-11: Image Re-optimisation
Always poking the hornets' nest out of sheer devilment, I went and took another run at PageSpeed Insights.
By folding in some of Google's specific ImageMagick
-sampling-factor 4:2:0 for JPEGs,
I was able to hit 100/100 for the mobile home page in a mobile browser,
with all other site/browser combinations in the high 90s.
2017-11-05: Firework Routing
Googlebot owns an interesting warp in the fabric of space-time!
See GSC Crawl Stats up to a couple of days ago. (All the record highs are in that snapshot, plus the record low download time.) Up before the last day the usual rule of thumb of ~200ms + ~1ms/kB was holding true, but for the very last point the download time drops ~50ms below the usual baseline, which was due to nothing at my end that I know of, and usually only happens with a very small number of fetches in a day.
I looked through my logs for being crawled locally (eg from within the EU) which could knock down the RTT signifcantly (vs ~150ms London to California).
Other than a few obvious fakes all the (non-Images) Googlebot IPs appear to be registered in Mountain View, but a couple of traceroutes are interesting:
% traceroute 188.8.131.52 traceroute to 184.108.40.206 (220.127.116.11), 30 hops max, 60 byte packets 1 192.168.0.254 (192.168.0.254) 0.766 ms 0.997 ms 1.160 ms ... 18 crawl-66-249-76-95.googlebot.com (18.104.22.168) 20.784 ms 20.212 ms *
% traceroute 22.214.171.124 traceroute to 126.96.36.199 (188.8.131.52), 30 hops max, 60 byte packets 1 192.168.0.254 (192.168.0.254) 0.825 ms 1.116 ms 1.407 ms ... 19 crawl-66-249-70-31.googlebot.com (184.108.40.206) 103.652 ms 103.580 ms 95.009 ms
Note that it can take ~10--20ms RTT across my local FTTC link to my ISP's infrastructure. StatusCake's London server sees a minimum of ~27ms and a mean of ~70ms to pull down the mobile home page.
One Googlebot is 80ms RTT closer than the other, and both seem closer than Mountain View (for normal traffic)!
(Just after writing the above the data for the 4th became visible, with a more normal download time but still fairly vigorous spidering.)