If Appledystopia is right in its 2020 article, Microsoft loses US$1·5 milliard per annum on Bing. So maybe that explains why it’s worsened so much. Microsoft might well be finding ways to cut its losses, and servers cost money. Pity that none of the Bing clones are saying anything, not even Duck Duck Go’s usually vocal CEO.
I’m glad I discovered Mojeek when I did. We lost some traffic with Duck Duck Go’s near-dead internal search on Lucire, and overall I suspect everyone has lost traffic with Bing dying. With Google now also faltering (they still make plenty from the human farms, but you have to wonder just why it has worsened, even for existing sites), then it’s important that alternative, growing search engines—that’s engines, not services (so you can discount Ecosia, Neeva, Qwant, Duck Duck Go, and many others)—get our support.
There’s really only Mojeek in the occident with a growing index, regularly requiring new servers. If you aren’t anti-Russian, there’s Yandex; and China of course has Baidu. Brave and Yep are making great efforts but their indices are still small, though Yep can do better than Bing on some sites.
I know how web spiders work in theory, and there’s no way that 2002 framesets are coming up in a 2023 crawl. We haven’t linked to those pages for a long, long time. But Google is throwing those into the top 10.
And we can extend this argument: Google, through its advertising, incentivized the creation of the very crap polluting the web.
Mayer said, ‘I think because there’s a lot of economic incentive for misinformation, for clicks, for purchases.
‘There’s a lot more fraud on the web today than there was 20 years ago.’
What’s the bet that these fraudulent pages are carrying Google ads?
As Don Marti, who knows a lot more about this than I do, said to me: ‘It’s all about moving traffic and ads away from sites that people want, and that advertisers want to sponsor, to places where Google gets a bigger % of the ad money (even if they’re on the sketchy side)’.
I think all this was foreseeable, and one could prove negligence on Google’s part. I still remember a time when established publishers like me wouldn’t join Google’s ad programmes because they were seen as an advertising service for second-rate (or worse) sites. They would appear on places like Blogger, which Google wound up buying.
Then the buggers wound up monopolizing the area, and things got worse for digital publishers as the ad rates got lower and lower—and, as Don notes, the money can find its way to the bottom feeders.
And over the years, those posts really helped people and exposed some wrongdoings.
Hence the latest obsession, about Bing, because no one seems to have noticed how Microsoft’s search engine is behaving as though someone at Redmond is unplugging servers left, right and centre.
Someone on Reddit suggested I try Kagi, which is a paid search engine—but from what I can tell, it’s a meta-search (the person who told me about it confirmed this, as did an earlier review).
I’ve seen meta-searches for decades, and admittedly Kagi is the prettiest of them all, but because it’s pulling from Bing and Google, it suffers from the limitations of both, especially the former.
We already have seen how Bing basically favours antiquity over currency, at least where Lucire is concerned, so Kagi’s results contain, in their top 10, pages that have not been updated (or linked) since the mid-2000s. When the Google-sourced results are factored in, it looks a bit better (since there are pages from the 2010s and 2020s), but they still aren’t the most relevant (since it seems Google has been faltering somewhat on site: searches, too).
Here’s a screen shot from Kagi. Results 1, 6 and 7 are current; result 3 is from the early 2010s; results 2, 4, 5 and 8 are framesets from the 2000s; result 9 is from 2014 and hasn’t been linked since then; the remainder are stories which can still be found through spidering but date from between 2011 and 2016.
Since it’s a meta-search, I decided to peer into Google and its top 10 do not look good, either. As I don’t tend to use Google, and the recent tests were about grabbing the number of search results, or analysing their currency, I hadn’t drilled down on a site:lucire.com search for a while.
Let’s see how they look today.
Surprisingly bad. Results 1 and 2 are current; results 3, 4 and 5 are framesets from the early 2000s that have not been linked since then; result 6 is from 2005 and has not been linked since then; result 7 is a 2011 story; result 8 is a 2022 story; result 9 is a 2016 story; and result 10 is a 2011 story.
In other words, the Google top 10 has changed probably due to their algorithm, but I wouldn’t call these relevant to what searchers seek. I could understand the old about.shtml staying in the top 10 despite its antiquity, but some of these top-level pages are really old. Framesets? Seriously?
Result 11 is repeated, which is also odd, while results 14 and 15 are tag pages from the Wordpress part of the site. The 15th is for Whangarei, not exactly the fashion centre of the world.
Google’s fall could explain why these blog posts have suffered traffic-wise as they are seriously irrelevant; there’s no connection to the pages’ popularity, either. It’s really beginning to feel like the Wayback Machine there, too.
Mojeek still makes more sense, since the search there requires a term, i.e. site:lucire.com lucire, so naturally it gives you pages containing the word Lucire more.
Result 1 is our home page (makes infinite sense!); result 2 a current top-level contents’ page; result 5 is the main page from Lucire TV; while the rest are stories that have the word Lucire contained in them more than what is typical for our site.
It looks like the US search engines are faltering while Mojeek is getting better. What an interesting development. I didn’t have worsening Google search on my 2023 bingo card.
Incidentally, for this website, Google still places my mayoral election pages from 2013 in its top 10; while Mojeek links the home page, the blog, a mixture of posts from 2009, 2020, 2021 and 2022, a transcript of a 2008 speech, and a tag page from 2010. Bing has pages from 2003 and 2012, but also some current top-level pages and, amazingly, three blog posts that are likely to be relevant (two of them critical about Bing from 2022 and 2023, and a 2021 post about Vodafone). In other words, Google has done the worst, in my opinion. Bing only has 10 pages so it has the smallest index but what it showed was surprisingly good! That leaves Mojeek, again, as delivering the best balance of relevance and index size.
Strange, a reply I wrote in agreement has vanished.
Basically my earlier point was that Google has also destroyed a lot of legitimate publications’ earnings through depressing ad prices, diverting income to splogs, content mills and spun sites. Not to mention taking a decent cut for itself.
The whole enterprise is a massive con.
From a legal POV I would even say it was all foreseeable and a negligence lawsuit waiting for someone to take it on. It would be great to close it down.
The original reply linked to this post, which is also saying the emperor has no clothes—except this time it’s applied to Google. If Googlers are worried about that, then maybe I’ve cut very close to the chase. The one part which, when attacked, destroys the entire corrupt system.
PS.: Don Marti expresses my point far better than I did.
In January, I can report some have gone worse. And back in July it was already pathetic.
The first figure below is from today, the parenthesized figure from July.
Remember that Mojeek is the only party that appears to report these figures honestly. Bing repeats results from page to page—around 40 per cent from the searches I’ve done with site:lucire.com. Google will show a few hundred so it’s anyone’s guess. I prefer Mojeek’s 1,000 cap and that works particularly well for the Lucire site.
Die Zeit (zeit.de)
Mojeek: 5,279 (4,796)
Google: 2,590,000 (2,600,000)
Bing: 6,010 (3,770)
There we have it: some rises at Bing for Die Zeit and Travel & Leisure, steady at The Gloss, but notable falls at The New York Times (back into the thousands, down from millions) and Microsoft’s own website (340,000, down from over 14 million). If you’re an independent publication, your presence on Bing is not rosy, with Annabelle, Lucire and The Rake netting between 10 and 25 despite thousands of pages on each site; while my friend Stefan Engeseth’s Detective Marketing site is also down to 10 from an already low 51.
I know from Mojeek’s blog that they keep plugging hard drives and servers to cope as their index expands. I can only assume from these numbers that Microsoft is unplugging them though they seem to look after you more if you’re an establishment website from a big company.
PS.: Here’s another way of looking at the data, factoring in the round of tests I did on August 2.
Along came Copytrack again yesterday, identifying an image that they allege we stole and put on Lucire’s website. And once again I had to go back through old emails—only 11 years this time, not 13 like the last—to retrieve the email to prove that I had the correct licence to publish it, and that and the download page where I got it (it’s one of the most famous fashion labels in the world and knowing their budgets, they’ve paid for press). You wonder why they don’t whitelist legitimate publications.
It’s all very well for them to use their automated systems but I have to get the DVD archive manually. I’m just incredibly fortunate that I’ve kept every email since the 1990s.
On that note, I’ve marked most of the gallery entries on this blog as private today. Pretty much every image in the gallery I know to be either licensed for press use or is a publicity pic. But some have come via social media. I simply recognized them to be the press images because I have a photographic memory, and, for fun, I’ve added them to the gallery. Even though legally I have numerous defences, and I’m pretty sure I’d prevail in case of any legal claim, for a personal blog it’s just too much of a hassle when these so-called copyright services come knocking. I’ll do the hunt for work but I’m not being paid to blog. I know a lot of you enjoyed those gallery posts but they’re going to be pretty limited moving forward.
There are plenty of nice pics at Lucire—feel free to pop by there for a gander.
After notifying Mike Masnick, the founder of Techdirt, about my findings about Bing, coincidentally, the search engine began spidering his latest articles. It claimed to have 150 results, and delivered 92, many of which were repeated from page to page as usual. Tonight it’s a claimed 249, delivering 173.
Techdirt is well respected and very popular, and disliked presently only by the Musk bros. What’s the likelihood that Microsoft knew about their shortcomings here and corrected things? I wasn’t exactly quiet, and I told more than Mike and the readers of this blog (I went on Reddit, for example), since it was so ridiculous that Bing could only deliver one result for such a major website. It’s embarrassing for them, so they decided to do the right thing. Like any Big Tech firm: do nothing unless you risk getting bad press. This is right out of the Facebook playbook, for example.
What a pity they could not do the right thing for the rest of us.
Just as a comparison, since I am nothing if not fair. Here are the claimed number of results versus the number delivered for site:techdirt.com:
I’m not kidding: Bing claims it had 2 results and delivered 10. Looks like one of those rare times they underestimated. Well off the mark of the 55 they have been doing since mid-2022 and that was pathetic. There is nothing in the results from after 2007. Maybe fixing Techdirt’s results meant that Bing had so little computing power for every other site!
Well, I guess I can no longer claim that for a site:lucire.com search that Bing is repeating results from page to page, since it only has one page.
Here’s a site I’ve always liked: Techdirt. It’s incredibly influential, and reports on the technology sector. Mike Masnick’s run it for the same length of time as I’ve run Lucire (25 years, and counting).
And when it comes to Bing’s index collapse—or whatever you wish to call it—it’s no more pronounced than here (well, at least among the sites that even get listed). For site:techdirt.com:
One. This is a site that dates back to the 1990s and churns out numerous articles daily, and that’s how bad Bing’s got. Naturally, it’s the same with all the Bing clones, like Yahoo (the one with no logo now), Ecosia, Qwant, Neeva, Duck Duck Go, etc. Unlike Baidu, Bing doesn’t have communist Chinese censorship as an excuse. Or does it?
If you ever needed proof something was really, really off at Redmond, this is it. And still the clones stay silent.
PS.: If you search for Techdirt on Bing, its home page does not even come up in the top 10.
P.PS.: Here’s what WorldWideWebSize.com has to report (thanks to nf3xn for posting it first on their Mastodon). I believe the site is wrong when it calculates that the total index was up as high as it is on the left of the graph: basically it takes what Bing claims is the number of results as the truth, and we know it lies.
When we put our sites on a new server last year, one Wordpress plug-in we retained, despite a known exploit, was Yuzo Related Posts. Basically, nothing else could do related posts as well. It just worked. Everything else, inexplicably, either did not do post relationships terribly well, was too resource-heavy, or was too ugly,
Wordpress’s own directory has a lot of related-post plug-ins, but once again, I had to draw the same conclusion that I did in 2022. In fact, two of them didn’t even function! So much for them having been tested.
Yuzo, of course, was toast, having been removed from the directory.
But a further search revealed that Lenin Zapata, one of the two people behind the original, did indeed rewrite the plug-in completely, taking it from v. 5 to v. 6. The latest, last updated in 2020, was v. 6.2.2.
As far as I can tell, it’s a complete rewrite, but I am no expert on such matters. What I can tell you is the directory structure looks different. The bottom entry in the readme.txt is for the original, where Mr Zapata wrote, ‘Old version (with faults): A bad day’. The new one is ‘renewed and with maximum security’.
I am taking Mr Zapata’s word for it, but I was saddened to note that Wordpress has kicked even the new version off for a ‘Guideline Violation’. Strangely, my web history says I downloaded the latest one from wordpress.org, even though the site says it is ‘not available for download’. It must be in there somewhere and even Wordpress’s own stats said there were a handful of downloads over the last week.
No wonder he stopped developing it after both the disappointment of the exploit and seeing the plug-in get kicked off. Even if it was the best and, it seems, irreplaceable. I don’t know why no one has risen up to meet the quality of the original plug-in (the exploit aside), but maybe Lenin Zapata is just that much cleverer with figuring out how posts relate and with presenting PHP-generated content smartly. Have a look below—I think it looks very good and works very well.
I’m just hoping I’m doing the right thing by using a version that hasn’t reportedly fallen victim to the 2019 exploit. I don’t like someone getting a raw deal if they’ve fixed up something on which they made a mistake. They deserve a second chance.
Do I recommend you do what I did? No, because I don’t understand enough code to be able to report definitively that it was the right decision. But if you understand this stuff, have a peek at v. 6 and see if it does what it’s supposed to—safely. Or write your own to compete with it and do what so many of these plug-ins don’t or can’t.
Here are a few screenshots from a magazine I loved, but sadly, it seems they’ve responded to those SEO emails, and grabbed the US$50 per post.
I don’t blame them, since Google has destroyed the online advertising ecosystem, and they have to make ends meet somehow.
I was in contact with them some years ago, and they’re really good people.
The top articles on their home page are theirs, and they remain excellent in quality, but scroll down and there are articles that are obviously SEO pieces. What’s the bet that Al Woods and Alexa Wang, with the same initials, are the same person? As a result, I made the sad decision to remove them from Lucire’s link directory.
My feeling is that you accept these SEO gigs at your own risk, and those risks include getting demoted by the search engines as I’m sure they have figured out when you’re part of trying to game the system. They also make the site look like a content mill, despite the great original journalism that’s front and centre, and more visible there.
Our sites are our shop windows, so it’s in our interests to remain visible in the search engines. But everyone has different priorities. And I may be wrong: maybe these pieces haven’t affected that site at all. I’d just rather not risk it.
