After Holly Jahangiri’s very useful feedback to my previous post, I thought I’d give the search engines she sampled a go for site:lucire.com.
Bear in mind that Duck Duck Go, Ecosia, Qwant, Onesearch and Swisscows license from Bing, and Startpage picks up Google, so their indices will reflect the mothership.
Here’s how we look today. Bing remains well and truly beaten by Google, Mojeek, Baidu and Yandex.
Virtual Mirage: 100
Duck Duck Go: 49
Onesearch: blocked in New Zealand
I am not alone, it seems. This thread on Microsoft Answers was enlightening. Others in the thread have found themselves gone from Bing (but not Google), and Microsoft appears to know about it, admitting to some fault and escalating the issues internally, but nothing ever gets done.
I had that old theory, blogged about previously, that computer databases get worn after a while. I saw that with Vox, a lot of Facebook’s ills can be put down to it, and maybe Bing has now got there? No tech ever wants to admit it because of how crazy it sounds. But if we can lose data on hard drives and USB sticks, then I don’t care how many back-ups these big firms have, they are still fallible. (What if faults in one database are copied on to another, and the checksums weren’t verified?)
I replied to the Microsoft poster, and it’s a pretty good summary so far:
Hi EbinVThomas, here’s my experience, and I’ve run websites for three decades. The short version is I think Bing is stuffed and it’s not a Microsoft core business, so it doesn’t get much love (indeed, one of their FAQ pages has a heading about ‘seach’). I know the Microsoft fans will attack me for saying this, just as the Apple fans have a go at me when I say something negative about Macs, but I haven’t read anything to change my opinion.
We started vanishing from Bing earlier this year, maybe about three months ago. For some of our sites, I thought it was our belated switch to HTTPS for some of them, but as you’ll read, that wasn’t the case.
These sites date from (at their present domains) 1995, 1997, 2002 and 2008, and they are well linked, well respected, and one has been winning awards from 1997 to today. Google and Mojeek have no problems with any of them. Two of the sites (the 2002 and 1995 ones) did drop from their number-one and two positions on Google (for a name search) when they switched to HTTPS but one has mostly recovered, the other (from 1995, with a lot of inbound links to HTTP) fluctuates.
One of the other sites uses Duck Duck Go for its internal site search (and has done since the 2010s), which is powered by Bing. Earlier this year—say about six weeks ago—I noticed that the internal search was getting more and more useless, even though I knew the articles used to be found by DDG.
I began doing site:domain.com searches for this one. It had c. 50 entries on Bing, down from several thousand earlier in the year.
My first reaction was to blame ourselves—maybe it was the full switch to a secure server (some earlier pages were already on HTTPS), or something else. We also began using Cloudflare again after a 12-year break around this time.
I signed up to Bing Webmaster Tools. The site promptly went down to 10 entries! In other words, signing up to Tools made the site’s presence a lot, lot worse.
I found some weird site maps that I never put in, nor did any of my team. Nevertheless, I put in new, fresh ones last week, all pointing to HTTPS. Most of the pages have not been indexed.
I had to turn off Cloudflare’s IndexNow because it was sending some totally irrelevant and old pages and files to Bing. (So we can blame Cloudflare for some issues, but the majority still rests with Bing.)
Since the new site maps, Bing is now returning 53–5 entries (depending on the hour).
It finally included the home page which had been missing from the site: searches. Yet only yesterday Webmaster Tools said the page was not indexed because of certain issues, but it had been found in 2018. That made no sense as it was present until quite recently. Those issues included a description tag being too long (fine, I edited it), and no H1s (but why should there be? Not everyone wants humungous type on their page). But Bing had been fine historically with the page (since Bing started, so well before 2018) and it even appeared in the index during the last few weeks. A related page for our business doesn’t have H1s, an even longer meta description, and it’s on Bing. (It’s just not been entered into Webmaster Tools, which seems to be a kiss of death!)
Webmaster Tools even said it had accepted the site maps and the thousands of pages listed.
As far as I can make out, Webmaster Tools says one thing but reality says another.
So, was it Cloudflare and HTTPS that had knocked us? Well, no. Of the four sites I mentioned, we didn’t change the set-up of the one started in 2008. It’s a reference site, and has plenty of inbound links from Wikipedia since it’s fairly authoritative.
No Cloudflare, and still on HTTP. All fine on Google and Mojeek.
Also thousands of pages.
On Bing: 51 pages.
Thousands of entries have vanished since earlier this year, and I’m going to hazard a guess to say it began happening around the time you wrote your original post.
It has had a slight impact on our traffic, especially since we had promoted Duck Duck Go so heavily since 2010 and encouraged others to shift from Google to it.
It seems that Bing can now only cope with 50-odd pages from certain sites. The older sites have fewer pages indexed now on Bing than they did on Excite or Hotbot in the 1990s, and certainly far fewer than Altavista! Our sites are so incredibly varied—static, dynamic, HTML, PHP—so it can’t be structural or the way we have set things up. None have had issues at Google other than one that dropped in the index for a certain relevant search, and Mojeek is fine with them all and took the HTTPS shift for three of them in its stride.
These are such old sites with a history in Bing, so my feeling is that a new site won’t stand much of a chance.
This is a long way of confirming your original post: it’s not you, it’s them.