Browser problem fixed: it was LetsEncrypt’s expired root SSL certificates

The browser problem I described yesterday is fixed.

As a test sample, consistently utterly un-reachable sites in Opera were…

www.majorgeeks.com
www.etools.ch/
www.davidrevoy.com/blog

All loaded perfectly fine and instantly in the Pale Moon Firefox-based browser.

One clear possible cause I found was LetsEncrypt changing its root site certificates, which are used by way too many (20%?) of the world’s smaller website servers…

DST Root CA X3 will expire on September 30, 2021. That means those older devices that don’t trust ISRG Root X1 will start getting certificate warnings when visiting sites that use Let’s Encrypt certificates.

The timing was right. The systems affected were right. The reason for the Chrome vs. Firefox strangeness was right…

Browsers (Chrome, Safari, Edge, Opera) generally trust the same root certificates as the operating system they are running on. Firefox is the exception: it has its own root store.

Thanks to ‘GGG’, who got it right. He had exactly the same problem as me, Chrome (Brave) not working, Firefox working fine. He traced the broken sites to their use of LetsEncrypt root SSL certificates. This led me to the server techie Gunter Born in Germany warning of the same problems a little in advance and describing them in detail. Apparently the certificates are free and thus are widely used by smaller sites. It’s the world’s largest certificate authority. Seriously. The world’s largest certificate authority suddenly revokes its 300-million+ key server certificates and effectively breaks 20% of the Web and… the media don’t tell anyone in advance? So far as I can see only a few gadget sites and some Indian sites gave a few hours warning.

Anyway, assuming rogue SSL certificates rather than iffy DNS servers was the actual problem, as now seemed very likely… how to fix it?

The solution: You need to manually add fresher certificates. Do as the Tech Journal explains in the new page for the DST Root CA X3 Certificate Expiration Problems and Fix. There Stephen Wagner has kindly dug up the links to the new fresh certificates.

The guy who saved the world.

You will need a Firefox or Pale Moon browser to get them, as LetsEncrypt’s problem is blocking LetsEncrypt from itself (durh…). Some Windows users will need to choose the .DER rather than the .PEM version of the certificates. Best to get them all and see which version your Windows recognises and adds an icon to.

Once downloaded you need to double-click them and for each one a Windows Certificate import Wizard will launch. Install it to the correct folder….

Don’t just accept the Windows defaults (could install anywhere…), but guide each certificate to its correct folder. isrgrootx1.der and isrg-root-x2.der go in the “Trusted Root…” and lets-encrypt-r3.der goes in the “Intermediate…”. Intermediate seems just as important as the others, so don’t skip it. There appears to be no need to delete the old defunct certificates, although browser access seemed to speed up a bit when I hard-deleted the Sept 2021 certs from “Trusted Root…” and “Intermediate…”.

Now when you close and re-launch your Chrome-based browser, and after a pause of perhaps 12-20 seconds for each previously blocked site, the problem should be fixed. It was for me. I assume the one-time pause is for the browser to re-cache the page.

I did not need to re-boot Windows for this fix to ‘take’. The Windows-savvy will be able to type MMC at the Windows Start menu and then load a Snap-in to see new certificates and their dates…

This is also the way you delete the old ones, which cannot be done via Settings | Security | Proxy in Chrome/Opera…

---

Update: According the Linux Addicts the problem briefly took out Amazon Web Services, Shopify and The Guardian. The Daily Swig adds Google Cloud, Microsoft Azure, and many others.

Chrome-based browsers – “This site can’t be reached”

A curious problem has developed persistently in recent days, for users of Chrome and Edge browser… but not for Firefox / Pale Moon. Evidently the problem is now shared by others as well as myself.

While browsing a site/page fails to respond to the Chrome browser, but springs instantly into action for Firefox or Pale Moon (based on Firefox). In the Chrome-based Opera you get…

This site can’t be reached. [URL] took too long to respond.

Doesn’t appear to affect the mega-sites like YouTube or WordPress. Sometimes there is a 20-30 second delay in reaching a mid-ranking site, and often nothing at all from smaller sites or certain known recalcitrant mid-ranking sites (e.g. Stack Overflow, GreasyFork). Slack also seems to be badly affected, though that doesn’t affect me…

The problem appears to be cross OS, as I’m on Windows and this other guy (linked above) is on Linux. I have the same symptoms as he has: Chrome often gives this error while Pale Moon (Firefox) is totally fine. The problem occurs even if you are using a DNS server other than that of your ISP. For instance in Opera, it’s possible to select from a number of DNS servers. They all exhibit the same problem. Other fixes tried include:

* Changing the Windows IVP4 DNS to another (9.9.9.9, 8.8.8.8, 1.1.1.1) makes no difference either.

* Running with all browser extensions and scripts off also makes no difference.

* Visiting the page in ‘Incognito mode’ makes no difference.

* Modem reset makes no difference.

* PC reboot makes no difference.

* I don’t have proxies configured.

My first guess was some iffy under-the-hood Chrome update, perhaps some new and imperfect query being made to the some local and rather sluggish and partial DNS cache. Linux-guy’s claimed solution thinks along these lines and he suggests flushing your local DNS, which on Windows is:

1. Start menu.
2. Run.
3. Run dialog box, type…
4. ipconfig /flushdns
5. Confirm. A DOS-box window should flash up for a microsecond, the DNS cache is flushed, and the Run box exits.

Works as described above, but this didn’t cure the problem for me.

Nor did clearing the internal Chrome DNS cache (who knew?) and restarting the browser…

chrome://net-internals/#dns

Then I downgraded the Opera browser, back to Opera 78.0.4093.147 (mid August 2021) with the help of the full offline installer. Still the same problem, and thus it can’t be due to some recently-updated Chrome component.

So… if its not in Windows and not in Chrome, and not due to extensions or other obvious problems… what on earth could it be? It must be some kind of interaction between any DNS server and a Chrome-based browser, even a slightly older one. A problem which Pale Moon/Firefox is not affected by, and which has only recently started in the last few days. It can vary between DNS servers, some loading one page and not the other and visa versa.

One odd thing is that if you click hard and long and quick enough to load such a jammed page, like 50 times, it will often eventually load. This is repeatable. It’s like there’s a ‘black hole’ somewhere along the route, for smaller and mid-ranking sites that need DNS lookup, and eventually the system will ‘get the message’ and use an alternative route. I wonder in DNS servers have been ‘split’ in three and now have different sub-databases for top, middle and lower-ranking sites? And that the low-ranking databases sometimes power down their disks when not being called? That might explain it. The disks could need time to power up. But surely they would be modern always-on SSD’s and not old mechanical hard-drives?

Why Firefox / Pale Moon is unaffected I have no idea. But it is. I’ve been unable to discover if it uses any special DNS routing. Only that Pale Moon has no support for ‘DNS over HTTPS’.

So the temporary solution is then:

1. Open the Pale Moon browser, which has no such problems, and keep it open.
2. Install Andy Portmen’s “Open in Pale Moon” extension in Opera or Chrome.
3. Pin “Open in Pale Moon” button to your bookmarks bar.
4. Launch any recalcitrant page in Pale Moon (Firefox). This browser is already open so it will load instantly, and the supposedly ‘un-findable’ page will also load instantly.

Sadly the above only works once Opera has actually received the “This site can’t be reached. [URL] took too long to respond.” message. If you pass the URL over to Pale Moon while the browser is still waiting (and waiting and waiting…) for a DNS server, you get nothing in Pale Moon. You can however go back and right-click on the original hyperlink and “Open in Pale Moon” that way.

You can also switch your RSS reader to open pages externally in Pale Moon / Firefox.

---

Update: This, at first glance, seems to explain the difference between the browsers…

1. “Chrome uses DNS prefetching to speed up website lookups”

2. DNS pre-fetch is off by default in Pale Moon… “DNS prefetching disabled by default to prevent router hangups”.

Checking the value on about:config / network.dns.disablePrefetch assures that it is indeed off in Pale Moon.

In Chrome/Opera this is now called “Preload pages for faster browsing and searching”, and again it is turned off for Opera. The uBlock Origin addon forces it off.

So, despite sounding plausible, the above can’t be the explanation for the problem.

---

Update: Browser problem fixed: it was LetsEncrypt’s expired root SSL certificates. Install the new ones. Firefox / Pale Moon uses its own SSL certificate store, which was why it was unaffected.

JASPER

The DOAJ has announced JASPER – preserving open access journals forever, a new project. They’re not immediately going to hoover all DOAJ-listed journals into a mountain-covered mega-server, though, and in Phase One an editor has to ask to be included…

The criteria for eligibility for Phase One are that:

– your journal is indexed in DOAJ
– it does not charge any fees of any kind
– it is not archived in a preservation service

Please email preservation[@]doaj[.]org to register your interest in being involved or to be kept up-to-date with developments.

React

React, an interesting new academic development in visual search. It works on the reverse-search principle: pick a picture, and see similar pictures in the results.

The prototype limits results to a couple of the UK’s larger national digitized art collections (National Archives, the V&A) and leavens these with the Edinburgh Botanic Garden for some flowers and curious pods and suchlike. An AI assists the “does it look like this…?” sorting.

Paper to HTML

The Allen Institute for AI has a new prototype Academic Paper to HTML converter, as an online service.

There are of course already polished online services such as a IDR’s PDF to HTML5, but they limit the number and size of uploads.

Such conversions can also be done on a desktop PC via QuarkXPress 2021, which does quick pixel-perfect HTML5 conversion natively and (if you wait for a Black Friday discount) can be had for about £180 on a perpetual licence. Its direct competitor Adobe InDesign is subscription and needs a further expensive plugin (also subscription) to do HTML5 output. Many old-timers will throw up their hands in horror at the name ‘QuarkXPress’, but it’s no longer your grandpa’s creaky old DTP software.

Pinterest, begone

New and useful for picture researchers of various types, a UserScript to “Hide pinterest.com in Google Images” search results. Uses a simple ‘if result contains pinterest.com, do not display’ CSS method. The script is easy to tweak and as such it could be adapted for other image sites that you find are verbose/useless (e.g. Alamy and its ilk), without the need for a full-blown URL blocker add-on.

Why kill Pinterest? Because it poisons search-engines. 70% of the time you can never actually get to the image shown in the results.

Don’t ever add a second email address to your PayPal account

A scary automatic non-appeal-able PayPal account termination for Cryotek, makers of the free open-source Cyotek WebCopy and other free software. It appears that all he did was add a second email address to his PayPal account.

Google Scholar Oneclick Bib

Google Scholar Oneclick Bib, a new UserScript for your Web browser.

“One click to copy the bib information of each Google Scholar entry. The information will be automatically copied into the clipboard.”

Cheap moves

Jonathan Matthis, of the Biology Dept. at Northeastern University, has launched the open source FreeMoCap Project. ‘Mo-cap’ records human body movement in space, in a way that can then be pinned to 3D digital skeletons and ‘played back’. Very useful for animators and researchers alike.

What makes this project different? It’s a “low-cost, research-quality markerless” mo-cap system, that can run from video recorded by… two £1,000 iPhones and an £800 graphics card? Nope, that’s what I was expecting. But in this case you just need two standard USB webcams.

The aim is have… “a 14-year-old, with no technical training and no outside assistance, recreate a research-grade motion capture system for less than $100”. The early results have had animation professionals making very positive noises.

This worthy project then feeds its data to the free open source Blender 3D software. Great. The only problem appears to be that you need an NVIDIA graphics card in your PC, to process the video in a way that can get the skeleton data out of it. Where you’ll find a useful NVIDIA graphics card for less than $100 I don’t know, but I’m guessing the system may eventually be able to run on the sort of ancient graphics cards that get eBay’d for $20.

Anyway, it seems to me that this is an open project that could use some donations and volunteer coders for its early stages.

PrintNightmare – manual fix

There’s now a manual workaround to fix the important all-Windows ‘PrintNightmare’ security hole, at least for domestic and standalone PC users who don’t need to print over a network.

See the official Microsoft Workarounds / Option 2.

Stop the Print Spooler as a Service.
Change the PC’s Group Policy to block “inbound remote printing operations”.
Restart the Print Spooler.

This will “will block the remote attack vector”. Yup, it seems the fix is that easy.

Although… some versions of Windows do not have the required Group Policy Editor. In that case AskVG has instructions for Home users.

---

Update: Cancel that. Ten days later and another hole has been found, which for now means that the Windows Print Spooler service should be stopped totally even on domestic and standalone PCs. From some software you may still be able to “Save as PDF” but not always. There’s now a massive commercial opportunity for someone to develop a way to print on Windows, without the Windows Print Spooler service being active. Someone is going to make millions from that, as Microsoft doesn’t seem to be interested in the possibility.