MIA again on Bing: Lucire’s home page. The alt tags are not missing, with perhaps some exceptions for small logos. And not having an H1 tag is not fatal to other pages of ours that have been indexed. It remains bizarre.
After Holly Jahangiri’s very useful feedback to my previous post, I thought I’d give the search engines she sampled a go for site:lucire.com.
Bear in mind that Duck Duck Go, Ecosia, Qwant, Onesearch and Swisscows license from Bing, and Startpage picks up Google, so their indices will reflect the mothership.
Here’s how we look today. Bing remains well and truly beaten by Google, Mojeek, Baidu and Yandex.
Google: 6,100
Mojeek: 3,569
Swisscows: 498
Baidu: 201
Startpage: 198
Virtual Mirage: 100
Yandex: 94
Bing: 50
Qwant: 50
Duck Duck Go: 49
Ecosia: 49
Brave: 14
Searchencrypt: 8
Searx: 0
Onesearch: blocked in New Zealand
I am not alone, it seems. This thread on Microsoft Answers was enlightening. Others in the thread have found themselves gone from Bing (but not Google), and Microsoft appears to know about it, admitting to some fault and escalating the issues internally, but nothing ever gets done.
I had that old theory, blogged about previously, that computer databases get worn after a while. I saw that with Vox, a lot of Facebook’s ills can be put down to it, and maybe Bing has now got there? No tech ever wants to admit it because of how crazy it sounds. But if we can lose data on hard drives and USB sticks, then I don’t care how many back-ups these big firms have, they are still fallible. (What if faults in one database are copied on to another, and the checksums weren’t verified?)
I replied to the Microsoft poster, and it’s a pretty good summary so far:
Hi EbinVThomas, here’s my experience, and I’ve run websites for three decades. The short version is I think Bing is stuffed and it’s not a Microsoft core business, so it doesn’t get much love (indeed, one of their FAQ pages has a heading about ‘seach’). I know the Microsoft fans will attack me for saying this, just as the Apple fans have a go at me when I say something negative about Macs, but I haven’t read anything to change my opinion.
We started vanishing from Bing earlier this year, maybe about three months ago. For some of our sites, I thought it was our belated switch to HTTPS for some of them, but as you’ll read, that wasn’t the case.
These sites date from (at their present domains) 1995, 1997, 2002 and 2008, and they are well linked, well respected, and one has been winning awards from 1997 to today. Google and Mojeek have no problems with any of them. Two of the sites (the 2002 and 1995 ones) did drop from their number-one and two positions on Google (for a name search) when they switched to HTTPS but one has mostly recovered, the other (from 1995, with a lot of inbound links to HTTP) fluctuates.
One of the other sites uses Duck Duck Go for its internal site search (and has done since the 2010s), which is powered by Bing. Earlier this year—say about six weeks ago—I noticed that the internal search was getting more and more useless, even though I knew the articles used to be found by DDG.
I began doing site:domain.com searches for this one. It had c. 50 entries on Bing, down from several thousand earlier in the year.
My first reaction was to blame ourselves—maybe it was the full switch to a secure server (some earlier pages were already on HTTPS), or something else. We also began using Cloudflare again after a 12-year break around this time.
I signed up to Bing Webmaster Tools. The site promptly went down to 10 entries! In other words, signing up to Tools made the site’s presence a lot, lot worse.
I found some weird site maps that I never put in, nor did any of my team. Nevertheless, I put in new, fresh ones last week, all pointing to HTTPS. Most of the pages have not been indexed.
I had to turn off Cloudflare’s IndexNow because it was sending some totally irrelevant and old pages and files to Bing. (So we can blame Cloudflare for some issues, but the majority still rests with Bing.)
Since the new site maps, Bing is now returning 53–5 entries (depending on the hour).
It finally included the home page which had been missing from the site: searches. Yet only yesterday Webmaster Tools said the page was not indexed because of certain issues, but it had been found in 2018. That made no sense as it was present until quite recently. Those issues included a description tag being too long (fine, I edited it), and no H1s (but why should there be? Not everyone wants humungous type on their page). But Bing had been fine historically with the page (since Bing started, so well before 2018) and it even appeared in the index during the last few weeks. A related page for our business doesn’t have H1s, an even longer meta description, and it’s on Bing. (It’s just not been entered into Webmaster Tools, which seems to be a kiss of death!)
Webmaster Tools even said it had accepted the site maps and the thousands of pages listed.
As far as I can make out, Webmaster Tools says one thing but reality says another.
So, was it Cloudflare and HTTPS that had knocked us? Well, no. Of the four sites I mentioned, we didn’t change the set-up of the one started in 2008. It’s a reference site, and has plenty of inbound links from Wikipedia since it’s fairly authoritative.
No Cloudflare, and still on HTTP. All fine on Google and Mojeek.
Also thousands of pages.
On Bing: 51 pages.
Thousands of entries have vanished since earlier this year, and I’m going to hazard a guess to say it began happening around the time you wrote your original post.
It has had a slight impact on our traffic, especially since we had promoted Duck Duck Go so heavily since 2010 and encouraged others to shift from Google to it.
It seems that Bing can now only cope with 50-odd pages from certain sites. The older sites have fewer pages indexed now on Bing than they did on Excite or Hotbot in the 1990s, and certainly far fewer than Altavista! Our sites are so incredibly varied—static, dynamic, HTML, PHP—so it can’t be structural or the way we have set things up. None have had issues at Google other than one that dropped in the index for a certain relevant search, and Mojeek is fine with them all and took the HTTPS shift for three of them in its stride.
These are such old sites with a history in Bing, so my feeling is that a new site won’t stand much of a chance.
This is a long way of confirming your original post: it’s not you, it’s them.
You may also like
Thanks for the mention! Admittedly, I thought it was “just me.” And I went through and fixed everything I could find; I should have taken better notes.
I was down to 0 errors, and now have about 15. I think they’re definitely tweaking some things.
Among the things I have fixed, in the past, that seemed to make a difference:
Bing AND Google seem to expect some new form of sitemap, and maybe only one. I had been using some plug-in and now use the one from wordpress core. (NOT any “advanced sitemap” from a plug-in. I was using Smartcrawl.)
I use Divi, and I do us a SINGLE H1 tag on each page (the TITLE is set to H1, and the first heading is set to H2). I did this years and years ago – all SEO checks will flag this if it’s missing or there’s more than one on a page. This also means customizing themes, sometimes, to insure that titles shown on the home page are NOT using H1, but rather H2.
WTF is IndexNow?? Doesn’t checking the box to say “notify search engines of new posts” within WordPress do the trick, anymore? WHY would I want to install another plug-in for this built-in feature?? https://www.bing.com/webmasters/indexnow (I don’t know – today is the first time I heard of it.)
At https://www.bing.com/webmasters/sitemaps my last crawl was 7/19. I have not had to resubmit anything since the end of May (which, I think, is when I discovered things were “broken” and assumed it was all my fault for not keeping a closer eye on things). When did the last big update to WordPress roll out? May 24, 2022
I think maybe WordPress broke it sitemaps made by other plug-ins? Seems a bit too coincidental, doesn’t it?
I remember noticing because I lost my knowledge block in Google and had to set things up again. It just dropped me. So it may even have been a more minor update prior to WP 6.0 that did it.
Make sure you’re using canonical links (HTTPS over HTTP).
Again, for me, Mojeek was the absolute WORST of the lot. I wish they’d all work harder at being better, and not just better at cloaking your identity while taking results from the big boys. I, too, miss AltaVista. I miss working search operators, too, for the most part. They’re still there, but you really have to search and they don’t always work. :(
You wouldn’t THINK that they’d be so different considering how most of them source results.
You’re welcome! It’s definitely not you, as Bing is broken (it’s tanked). Strange that no one has mentioned this much online, as far as I can see.
I’ve never had a site map till this month, for Bing. Google always managed to spider everything so I left well enough alone. For Bing I uploaded two site maps since Lucire is a hybrid site: not everything is on Wordpress. And Bing was hardly picking up anything on Wordpress.
Actually, more than two, as with thousands of pages, Wordpress needed three or four to cope with Lucire.
I’ve never subscribed to the need for H1 tags since you’ll remember in the early days how humungous that would have made your type. I believe we have a title class, but that’s about it. It seems to be OK—and in fact, it is OK for lucire.net, which Bing seems to want to index over the older, and far more authoritative and linked lucire.com, despite it running foul of its “requirements”.
IndexNow was on either a Cloudflare or Bing page, saying that it would send the newest stuff you required indexing to Bing, and Bing would include it immediately. (A bit like the old Infoseek URL submission in the 1990s.) But it never worked. All it sent were PDFs from the early 2000s, the font file data (which should never be indexed). Don’t bother with it—both companies appear to not know what they are talking about.
Most of our links are relative and not absolute, and where they are absolute, they’ve been changed.
I wrote to Mojeek suggesting they look at returning search results done with proper nouns differently. They do have work to do but their first big cash injection was only three years ago, so I’m going to give them some time—but I agree not everything was relevant in this post. But at least they have their own index and don’t rely on Google, Bing, Yandex, etc.
Altavista was awesome, and we used to license content to them at the AV Entertainment Zone! Google News is not the same! Speaking of the old search engines, presently Bing’s index is smaller than Alltheweb’s and Inktomi’s at their heights.