Posts tagged ‘Mojeek’


Testing the search engines: Bing likes antiquity; most favour HTML over PHP

21.09.2022

Bing is spidering new pages, as long as they’re very, very old.

Last week, we added a handful of Lucire pages from 1998 and 1999. An explanation is given here. And I’ve spotted at least two of those among Bing’s results when I do a site:lucire.com search.

As a couple of newer pages have also shown up, I doubt there’s any issue with the template; and the home page now also appears, too. But, by and large, Bing is Microsoft’s own Wayback Machine, and most of the Lucire results are from the 1990s and early 2000s.

It got me thinking: do the other search engines do this, too? For years, Google grandfathered older pages and they came up earlier. (Meanwhile, searches for my own name still have this site, and the company site, down, having lost first and second when we switched from HTTP to HTTPS in March. Contrary to expert opinion, you don’t recover, at least not quickly.)

As Lucire includes the date of the article in the URL, this should be an easy investigation. We’ll only do the first 50 results as that’s all Bing’s capable of. I’ll try not to include any repeat results out of fairness. ‘Contents’ pages’ include the home page, the Lucire TV and Lucire print shopping pages, and tag and category pages.
 
Bing
Contents’ pages ★★★
1997
1998
1999 ★★★★
2000 ★
2001 ★★★★★★★★
2002 ★★
2003 ★★★
2004 ★★★★
2005 ★★
2006
2007 ★★★
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018 ★
2019 ★
2020
2021
2022
 
Google
Contents’ pages ★★★★★★★★★★★★★
1997
1998
1999
2000
2001
2002 ★★
2003
2004 ★★
2005
2006
2007 ★
2008
2009
2010 ★
2011 ★★★
2012 ★
2013 ★★
2014 ★★★
2015 ★
2016 ★★
2017 ★
2018 ★★★
2019 ★★★
2020 ★★★★★★★
2021 ★
2022 ★★★★
 
Mojeek
Contents’ pages ★★★★★★
1997
1998
1999
2000
2001
2002
2003
2004 ★
2005
2006
2007
2008
2009 ★
2010 ★★
2011 ★★
2012 ★★★
2013 ★★★★
2014 ★★★
2015 ★★★★★
2016 ★★★★★★★
2017 ★★★★★★
2018 ★★★
2019 ★★★★
2020 ★★★
2021
2022
 
Baidu
Contents’ pages ★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018 ★
2019 ★
2020
2021 ★★★
2022 ★
 
Yandex
Contents’ pages ★★★★★
1997
1998
1999 ★★★★★
2000 ★★★★★★
2001 ★★★
2002 ★★★
2003 ★★★
2004 ★
2005
2006
2007 ★★★★
2008 ★★
2009 ★★
2010 ★★★★
2011 ★★★
2012 ★★
2013 ★
2014 ★★
2015
2016
2017
2018
2019
2020 ★★★
2021 ★
2022
 

To me, that was fascinating. My instincts weren’t wrong with Bing: it’s old and it favours the old (two of the restored articles were indexed). From the first 50 results, 18 results were repeats—that’s 36 per cent. I’m of the mind that Bing is so shot that it can only index old pages that don’t take up much space. New ones have a lot more data to them, generally.

Google does a good job with the top-level and second-level contents’ pages, though there were a few strange tag indices. But the distribution is what you’d expect: people would search for more recent stories. I know we had some popular stories from 2002 that still get hit a lot.

Mojeek has a similar distribution, though it should be noted that you can’t do a blanket site: search. There must be a keyword, and in this case it’s Lucire. The 2016 pages form the mode, which I don’t have a huge problem with; it’s better than the 2001 pages, which Bing has over everything else.

Baidu’s one is crazy as individual stories are seldom spat out in the first five pages, the search engine preferring tag indices, though half a dozen later story pages do make it into its top 50.

Finally, Yandex leans toward older pages, too, including our most popular 2002 piece. It’s the 2000 stories it has the most of among the top 50, and there’s a strange empty period between 2015 and 2019. But at least there is a fairer distribution than Bing can muster.

The other query that I had was whether these search engines were biasing their results toward HTML pages, rather than PHP ones. If that’s the case, then it could explain Bing’s preference for the old stuff (Lucire didn’t have PHP pages till 2008; prior to that it was all laboriously hand-coded, albeit within templates.)
 
Bing
★★★★★★★★★★★★★★★★★★★★★★★★★ HTML
★ PHP
 
Google
★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★ HTML
★★★★★★★★★ PHP
 
Mojeek
★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★ HTML
★★★★★★★★★★★★★★★★★ PHP
 
Baidu
★★★★★★★★★★ HTML
★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★ PHP
 
Yandex
★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★ HTML
★★★★★★ PHP
 

I think we can safely say there’s a preference for HTML over PHP. Mojeek brings up a lot of HTML pages after the top 50, even though this sample shows the split isn’t as severe.

Our PHP pages are less significant though: they contain news stories, and these are often ones other media covered, too. But I would have thought some of the more popular stories would have made the cut, and here it’s Mojeek’s distribution that looks superior to the others’. It seems like it’s actually analysing the page content’s text, which is what you want a search engine to do.

Baidu’s PHP-heaviness is down to all the tag indices—rendering it not particularly helpful as a search engine.

On these two tests, Mojeek and Google rank best, and Yandex comes in third. Baidu and Bing are a distant fourth and fifth.

Tags: , , , , , , , , , , ,
Posted in China, culture, internet, media, publishing, technology, UK, USA | No Comments »


Testing the seven search engines in the world

22.08.2022

After reading Mojeek’s blog post from last July, I learned there are only seven search engines in the world now. In other words, I was checking more search engines out in the 1990s. It’s rather depressing, especially as the search market is largely a monopoly with Google dominating it (and all the ills that brings), and Bing and its licensees (like Duck Duck Go) with their 6 per cent.

Knowing there are seven, I fed the site:lucire.com search into all of them to see where each stood.

The first figure is the claimed number of results, the second the actual number shown (without repeats removed, which Bing is guilty of).

I can’t use Brave here as its site search is Bing as well.

Yandex appears to be capped at 250 and Mojeek at 1,000, but at least they aren’t arbitrary like Google and Baidu. Baidu has a lot of category and tag pages from the Wordpress section of our site to bump up the numbers.
 
Gigablast 0/0
Sogou 19/13
Bing 243/50
Baidu 13,700/213
Yandex 2,000/250
Google 6,280/315
Mojeek 3,654/1,000
 

Frankly, more of us should go to Mojeek. It can only get better with a wider user base. Unlike Bing, it hasn’t collapsed. I know most of you will keep going to Google, but I just don’t like the look of those limits (not to mention the massive privacy issues).

Mojeek is now at 5,900 million pages, which must be the largest index in the west outside of Google.

Tags: , , , , , , , , , , , , , , , , ,
Posted in China, internet, publishing, technology, UK, USA | No Comments »


False accusations from Red Points Solution SL

18.08.2022

Yesterday, I returned to find a DMCA claim filed against us by Red Points Solution SL, purporting to act for Harper’s Bazaar España publisher Hearst Magazines SL, falsely accusing us of breaching their copyright with this article. You can read the notice here.

Naturally, I filed a counter-claim because their accusation is baseless.

Our source was PR Newswire, and it’s not uncommon to find stories of interest through that platform. In fact, Armani Beauty was so keen to get this out there on November 3 that we received the release in four languages at 15.28, 15.30, 15.33, 15.36, 15.39, 15.46 and 16.03 UTC.

The quotations and images were supplied by Armani Beauty, which is part of L’Oréal. I’ve worked with people from L’Oréal for over two decades and know their systems well enough, including the money they have for licensing images for press usage.

Lucire has a lot of original articles, but some of our news is release-based, as it is for anyone in our industry.

Our rule is: even when it’s a release, you write it up individually in your own words. You may have something additional to bring to the story. And we aren’t a repository of releases.

The only time we would run a release mostly verbatim is if we issued it, something that might happen once every couple of years.

Naturally, Google has so far done nothing and our story remains absent from their index. Big Tech loves big firms like Hearst.

I’ve tagged Harper’s Bazaar España in social media demanding they front up with their evidence. I’ve also messaged Hearst’s Spanish office with the following.

Ladies and Gentlemen:
 
Yesterday, your firm lobbed a false accusation against us by deceptively claiming your copyright had been breached by one of our articles. I note that you filed this as a DMCA complaint with Google.

We have filed a counter-notice.

We find it appalling that you would claim an original work has breached your copyright.

The imagery and quotations to our articles were sourced from L’Oréal, and we have informed them directly of your deceptive and misleading conduct.

I demand you furnish proof. As you will no doubt fail to, we demand you withdraw the complaint. We reserve the right to pursue our own legal remedies against you.
 
Yours faithfully,
 
Jack Yan
Publisher, Lucire

I basically thought they were being dicks and my friend Oliver Woods chimed in on Twitter about it. Oli’s very insightful and objective, and I respect his opinion.

They are being dicks, but there is a strategy behind it. Petty little minds wanting to look good on Google, not liking someone else ahead of them. (Not that I ever looked to see where our story ranked. I mean, seriously?)

It reminds me of a US designer’s rep who emailed me a while back wanting us to remove an article.

I asked: what’s wrong with it? Did we err in facts? Is it somehow defamatory?

When I probed a bit more deeply, it turned out that they were incensed it came up so highly in a Google image search.

I explained that that wasn’t a good enough reason, especially since the story had been provided to us by a PR firm.

They countered by saying that as they had not heard of us, it was highly unlikely that they would have released us that news.

I thought it was a very strange strategy to accuse someone you wanted a favour from of lying.

I still have the email from their PR firm. Call me Lord of the Files.

I’m not going to reveal the identity of the designer. I asked one of my team to see if he would call me directly instead of having one of his rude staff insult me. He never did call. The image is still there, and I bet they’re seething each time they see it.

It’s not even a bad image. It just doesn’t happen to be hosted by them.

I don’t really know why search engine domination is so important. We all should have a fair crack at it, and let whomever has the most meritorious item on a particular topic come up top.

The American designer, and the Spanish outpost of this American media giant Hearst, are obviously not people who like freedom of the press, freedom of expression, or a meritorious web. American people might like this stuff but a lot of their corporations don’t.

Which is why Google is terrible because it doesn’t allow it. We know through numerous lawsuits it has biases toward its own properties, for a start. I’ve observed them favouring big media brands over independents—even when independents break a news story.

Mojeek is just so, so much better. No agenda. Just search the way it was and should have stayed. That’s the “next Google”, the one that could save the web, that I had asked for in 2010.

Except it shouldn’t be the next Google because we don’t want more surveillance and tribalism.

Fair, unbiased search is where Mojeek excels. I really hope it catches on more. God knows the world needs it.

I think the world needs Lucire, too, the title that Harper’s Bazaar Australia named as part of its ‘A-list of style’. The Aussies are just so much nicer.
 
PS.: Hearst uses a company called Red Points Solution SL to do its supposed copyright infringement detection. Based on this, they must be pretty shit at it. And remember, we don’t even publish in Spanish. Yet.

I see you have falsely accused us of copyright infringement with our article at https://lucire.com/insider/20211103/valentina-sampaio-named-armani-beautys-newest-ambassador/ when we have done nothing of the sort.

We demand that you withdraw your DMCA complaint to Google.
 
https://lumendatabase.org/notices/28469986#
 

Our story’s source is Armani Beauty through PR Newswire, to which we are signed up as a legitimate international media organization. The story is our work, using facts and quotations provided in the release.

PR Newswire provided us with this release on November 3, 2021, at 15.28, 15.30, 15.33, 15.36, 15.39, 15.46 and 16.03.

A counter-notice has been filed.

We require an explanation from you on why you have targeted a legitimate media organization with your deception. Clearly your detection systems are not very good and we would certainly be discouraged from using them.

 
P.PS.: One more email to Red Points Solution SL on August 19, 21.56 UTC after they doubled-down with another notice removing two URLs from Google. Again, no proof of their original work was provided, and none can be seen in Lumen even when requested. It seems Google will lap anything up if it sees a big company behind it.

I have reached out to you through numerous means but yet to hear back.

I publish Lucire, a magazine with a 25-year history and five editions worldwide. You might even say we’re the sort of business that would need Red Points Solution’s services.

However, we’ve found ourselves at the other end, with legitimate media stories from our website removed from Google with DMCA notices you’ve filed.

Your client is Hearst SL.

If your latest efforts are down to Hearst’s orders, then they are claiming ownership over material that is not theirs.

All our content is original, and where it is not, it is properly licensed.

In the first case:
 
https://lucire.com/insider/20211103/valentina-sampaio-named-armani-beautys-newest-ambassador/
 

Your client does not own this material at all. We own the story, and the quotations and images are owned by and licensed to us by L’Oréal. Hearst has no connection to it other than Harper’s Bazaar being mentioned in an editorial fashion.

In the second case:
 
https://lucire.com/insider/20190905/nicky-hilton-hosts-brunch-to-celebrate-her-collaboration-with-french-sole/
 

Your client does not own this material at all. We own the story, and the images are owned by and licensed to us by French Sole and BFA.com. Hearst has no connection to it other than Harper’s Bazaar being mentioned in an editorial fashion.

In the third case:
 
https://lucire.com/insider/page/164/?mobiinsider%2F20120130%2Felizabeth-olsen-models-asos-magazines-cover%2F%3Fwpmp_switcher=mobile
 

Your client does not own this material at all. In fact, we own this material fully. No Hearst properties are even mentioned.

Counter-notifications have been filed on the basis that it is our original content and that your client has no right to make the claim in the first place.

It would be far easier if you would review your systems as presently they are opening your client and yourselves up to a legal claim …

We think you need to go back to your client and have them show you just how they can legitimately claim ownership of material that is not theirs.

In the meantime, we insist you stop these notices as they are unwarranted and unfounded.

We look forward to hearing from you.

Tags: , , , , , , , , , , , , , , , , , ,
Posted in business, internet, media, New Zealand, publishing, USA | No Comments »


What search engines show in their top 10 isn’t always relevant

09.08.2022

The Bing collapse did lead me to look at some of the ancient pages on the Lucire site that the search engines were still very fond of. For instance, the ‘About’ page was still appearing up top, which is bizarre since we haven’t made any links to it for years—it reflected our history in 2004.

Naturally, once I updated it, it promptly disappeared from Bing! Too new for Microsoft’s own Wayback Machine!

I was always told that you shouldn’t delete old pages, and that 301s were the best solution. I’m enough of a computing neophyte to not know how to implement 301s (.htaccess doesn’t work, at least not on our set-up) and page refreshes are often frowned upon, which is why so many old pages are still there.

However, you would naturally expect that a web spider following links would not rank anything that hasn’t been linked to for over a decade very highly. If the spider comes in, picks up the latest stuff from your home page, possibly the latest stuff from individual topic pages, it would figure out what all of these were linking to, and conclude that something from 2000 that was buried deep within the site was no longer current, or of only passing interest to surfers.

I realize I’ve had a go at search engines for burying relevant things in favour of novel things, but we’re talking pages here that aren’t even relevant. ‘About’ I’ll let them have, but a 2000 book reviews’ page? A subject index page from 2005 that hasn’t been linked to since 2005, and the pages that do are well outnumbered by newer ones? Because, the deletion of ‘About’ aside, here is what Bing thinks is the most important for site:lucire.com:
 

 

Google fares a little better. Our home page and current print edition ordering page are top, shopping is third, followed by the fashion contents’ page (makes sense). ‘About’ comes in fifth, for whatever reason, then a 2005 competition page that we should probably delete (it refreshes to another page from 2005—so much for refresh pages being bad for search engines).

Seventh is yet another ancient page from 2005, namely a frameset—which I’ve since updated so at least the main frame loads something current. The remainder are articles from 2011, 2022 and 2016. The next page comprises articles and tags, which seem to make sense.

Mojeek actually makes more sense than Google. Home page in first, the news page (the next most-updated) is second, followed by the travel contents’ page. Then there are two older print edition pages (2020 and 2012), followed by a bunch of articles (2013, 2014, 2013, 2013), and the directory page for Lucire TV. There’s nothing here that I find strange: everything is logically found by a spider going through the site, and maybe those four articles from the 2010s are relevant to the word Lucire (given that you can’t do site: searches on Mojeek without a keyword, so it repeats the word before the TLD)? The reference to the 2012 issue might be down to my having mentioned it recently during our 25th anniversary posts. But there are no refresh pages and no framesets.

Startpage, not Google, has a couple of frameset pages from 2000 and 2002 in their top 10 which again weren’t linked to, at least not purposefully (they were placed there to catch people trying to look at the directory index in the old days). There’s incredibly little “link juice” to these pages. However, ‘About’ (in 10th), and these two framesets aside, its Google-sourced results fare remarkably well. In order: home page, print edition ordering page, the two framesets, the news section, the shopping page (barely updated but I can see why it’s there), the community page, Lucire TV, the fashion contents, ‘About’.

Duck Duck Go is so compromised by Bing that it barely merits a mention here. Four pages from 2000 and 2005 that no current page links, a 404 page that we’ve never even had on our site (!), articles from 2021, 2018, 2007 and 2000 (in that order), and a PDF (!) from 2004. Fancy having a 404 that never even existed in the top 10!

If I had my way, it’d be home page, followed by the different sections’ contents’ pages, then the most popular article—though if a couple of articles go (or went) viral, then I’d expect them sooner.

Both Mojeek and Google do well here, with four of these pages each in their top 10s. But it’s Startpage’s unfiltered Google results that do best, hitting linked, relevant pages in seven results out of the top 10. Bing and its licensees miss the mark completely. If you must have a Google bias, then Startpage is the way to go; for our purposes, Mojeek remains the better option.
 
★★★★★★★☆☆☆ Startpage
★★★★☆☆☆☆☆☆ Mojeek
★★★★☆☆☆☆☆☆ Google
★★☆☆☆☆☆☆☆☆ Virtual Mirage
★☆☆☆☆☆☆☆☆☆ Baidu
★☆☆☆☆☆☆☆☆☆ Yandex
☆☆☆☆☆☆☆☆☆☆ Bing
☆☆☆☆☆☆☆☆☆☆ Qwant
☆☆☆☆☆☆☆☆☆☆ Swisscows
☆☆☆☆☆☆☆☆☆☆ Brave
☆☆☆☆☆☆☆☆☆☆ Duck Duck Go (would give –1 for the 404 if I could)

Tags: , , , , , , , , , , , ,
Posted in France, internet, New Zealand, publishing, technology, UK, USA | No Comments »


Rising popularity on Autocade

07.08.2022

Ever since we had to reset the counter for Autocade in March, because of a new server and a new version of Mediawiki, it’s been interesting to see which pages are most popular.

The old ranking took into account everything from March 2008 to March 2022. With everything set to zero again, I can now see what’s been most popular in the last few months.

Some of the top 20 were among the top pages before March 2022, but what’s surprising is what’s shot up into the top slots.

Over the course of half a day on Friday GMT, the Toyota Corolla (E210) page found itself as the top page, home page excepting. And the Kia Morning (TA) page shot up out of nowhere recently, too.

I know our page on the Corolla is number one on Mojeek for a search of that model but that can’t be the only reason it’s done so well. I haven’t studied the referrer data. A shame that link: no longer works on search engines.
 

 

Corolla fans, thank you for your extra 6,000 page views! It’s helped our overall total, but the viewing rate is still down at 2019 levels thanks to the collapse of the Bing index, and the search engines that it’s taken down with them.

I almost feel I’ve shot myself in the foot for promoting Duck Duck Go so much since 2010! But then I hopefully spared a lot of people from being tracked (as much) by the big G.

Tags: , , , , , , , ,
Posted in cars, interests, internet, media, New Zealand, publishing, technology | No Comments »


Mojeek shows more in its search results than Google

02.08.2022

This was something I had forgotten when doing the numbers on how many pages each search engine had indexed from our sites: what they claim to be their index size and what they let you access are two different things.

And in Lucire’s case, Google, curiously, mostly does not allow access to our dynamic pages in PHP in its main index, reserving them for Google News. Google News, however, has both PHP and HTML. It’s only when you feed in a specific request for one of our stories that we know is on a PHP-generated page that it comes up in the main index’s results.

Let me explain. Remember this from a blog post in July? These are what the search engines said they had indexed for lucire.com (in a site:lucire.com search). I’ve updated it for August 2 and added one more search engine, Yep, another independent, out of interest.
 
Google: 10,600
Mojeek: 3,593
Duck Duck Go: 50
Brave: 19
Bing: 10
Yep: 10
 

But can you see 10,600? Here’s the reality of what is truly visible at the moment when you browse the results’ pages of each search engine as of today:
 
Google: 304
Mojeek: 1,000
Duck Duck Go: 50
Brave: 19
Bing: 10
Yep: 10
 


Above: Google (top) shows fewer pages than Mojeek in a site: search.
 

Mojeek maxes out at 1,000 by design, but like Google, it will find a specific article outside of the 1,000 shown if searched for. Google conks out at 304 (303 when I first did this test).

The bigger Google index is its advantage, but Mojeek does a fine job by sharing more in its results’ pages than Google does—over three times as many. Another win for the plucky independent out of the UK.
 
While we’re on the subject, notice how small the Bing index is getting, returning just 10 pages for lucire.com? It’s really collapsed in a big way. Feeding in the other sites I tested earlier, Bing shows declines all round, apart from Travel & Leisure.

Fancy having only 2,723 results from The New York Times, down from 1,190,000 on the 24th ult. Mojeek has over 1,000 times more than Bing, and Google over 12 times more than that.

Previous numbers in parentheses below.
 
Die Zeit
Google: 2,710,000 (2,600,000)
Mojeek: 4,891 (4,796)
Bing: 3,268 (3,770)
 
Annabelle (Switzerland)
Google: 11,900 (11,700)
Mojeek: 408 (405)
Bing: 26 (105)
 
Holly Jahangiri
Google: 618 (738)
Mojeek: 236 (222)
Bing: 10 (49)
 
The Gloss (Ireland)
Google: 17,600 (19,200)
Mojeek: 2,009 (1,968)
Bing: 20 (71)
 
The New York Times
Google: 36,500,000 (36,200,000)
Mojeek: 2,879,513 (2,823,329)
Bing: 2,723 (1,190,000)
 
Lucire
Google: 10,600 (6,050)
Mojeek: 3,593 (3,572)
Bing: 10 (50)
 
The Rake
Google: 11,100 (11,500)
Mojeek: 1,445 (1,443)
Bing: 16, but claims 4! (49)
 

 
Travel & Leisure
Google: 33,500 (28,100)
Mojeek: 10,081 (9,750)
Bing: 383 (220)
 
Microsoft
Google: 118,000,000 (122,000,000)
Bing: 1,927,118 (14,200,000)
Mojeek: 1,772,165 (1,748,199)
 
Detective Marketing
Google: 961 (998)
Mojeek: 579 (579)
Bing: 16 (51)

Tags: , , , , , , , ,
Posted in internet, technology, UK, USA | No Comments »


More signs of Bing’s tiny index

24.07.2022

Because I have OCD, one more round of stats.

It’s not just us: Bing seems to have a reduced index for everyone. Here are a handful of sites that I fed in at random for site: searches. The only site where it beats Mojeek in indexed pages is, you guessed it, Microsoft’s. I guess since Google favours Google’s own results, Bing does a better job indexing Microsoft’s—and I doubt it’s because their own people conform to Bing’s applied-when-they-choose rules.
 
Die Zeit
Google: 2,600,000
Mojeek: 4,796 (0·18 per cent of Google’s total)
Bing: 3,770 (0·15 per cent of Google’s total)
 
Annabelle (Switzerland)
Google: 11,700
Mojeek: 405 (3·46%)
Bing: 105 (0·90%)
 
Holly Jahangiri
Google: 738
Mojeek: 222 (30·08%)
Bing: 49 (6·64%)
 
The Gloss (Ireland)
Google: 19,200
Mojeek: 1,968 (10·25%)
Bing: 71 (0·37%)
 
The New York Times
Google: 36,200,000
Mojeek: 2,823,329 (7·80%)
Bing: 1,190,000 (3·29%)
 
Lucire
Google: 6,050
Mojeek: 3,572 (59·04%)
Bing: 50 (0·83%)
 
The Rake
Google: 11,500
Mojeek: 1,443 (12·55%)
Bing: 49 (0·43%)
 
Travel & Leisure
Google: 28,100
Mojeek: 9,750 (34·70%)
Bing: 220 (0·78%)
 
Microsoft
Google: 122,000,000
Bing: 14,200,000 (11·64%)
Mojeek: 1,748,199 (1·43%)
 
Detective Marketing
Google: 998
Mojeek: 579 (58·02%)
Bing: 51 (5·11%)
 

In the earlier Microsoft thread I linked, the original poster found that after they joined Bing Webmaster Tools and imported their Google data, that’s when their site vanished from Bing. So, again, we’re not alone.

I’d seriously be rethinking my business model if I was running a search engine that was reliant on Bing.

Tags: , , , , , , , ,
Posted in internet, media, publishing, technology, USA | No Comments »


Putting the search engines through their paces

24.07.2022

One more, and I might give the subject a rest. Here I test the search engines for the term Lucire. This paints quite a different picture.

Lucire is an established site, dating from 1997, indexed by all major search engines from the start. The word did not exist online till the site began. It does exist in old Romanian. There is a (not oft-used) Spanish conjugated verb, I believe, spelt the same.

The original site is very well linked online, as you might expect after 25 years. You would normally expect, given its age and the inbound links, to see lucire.com at the top of any index.

There is a Dr Yolande Lucire in Australia whom I know, who I’m used to seeing in the search engine results.

The scores are simply for getting relevant sites to us into the top 10, and no judgement is made about their quality or relevance.
 
Google
lucire.com
twitter.com
lucire.net
instagram.com
wikipedia.org
linkedin.com
facebook.com
pinterest.nz
neighbourly.co.nz
—I hate to say it, as someone who dislikes Google, but all of the top 10 results are relevant. Fair play. Then again, with the milliards it has, and with this as its original product, it should do well. 10/10
 
Mojeek
scopalto.com
lucirerouge.com
lucire.net
lucire.com
mujerhoy.com
portalfeminino.com
paperblog.com
dailymotion.com
eldiablovistedezara.net
hispanaglobal.com
Mojeek might be flavour of the month for me, but these results are disappointing. Scopalto retails Lucire in France, so that’s fair enough, but disappointing to see the original lucire.com site in fourth. Fifth, sixth, seventh, ninth and tenth are irrelevant and relate to the Spanish word lucir. You’d have to get to no. 25 to see Lucire again, for Yola’s website. Then it’s more lucir results till no. 52, the personal website of one of our editors. 5/10
 
Swisscows
lucire.net
wikipedia.org
lucire.com
spanishdict.com
lucire.net
lucire.com
drlucire.com
facebook.com
spanishdict.com
viyeshierelucre.com
—Considering it sources from Bing, it makes the same mistakes by placing the rarely linked lucire.net up top, and lucire.com in third. Fourth, ninth and tenth are irrelevant, and the last two relate to different words. Yola’s site is seventh, which is fair enough. 6/10
 
Baidu
lucire.net
lucire.com
lucire.cc
lucire.com
kanguowai.com
hhlink.com
vocapp.com
forvo.com
kuwo.cn
lucirehome.com
—Interesting mixture here. Strange, too, that lucire.net comes up top. We own lucire.cc but it’s now a forwarding domain (it was once our link shortener, up to a decade ago). Seventh and ninth relate to the Romanian word strălucire and eighth to the Romanian word lucire. The tenth domain is an old one, succeeded a couple of years ago by lucirerouge.com. Not very current, then. 7/10
 
Startpage
lucire.com
lucire.com
lucire.net
instagram.com
wikipedia.org
linkedin.com
facebook.com
pinterest.nz
fashionmodeldirectory.com
twitter.com
—All relevant, as expected, since it’s all sourced from Google. 10/10
 
Virtual Mirage
lucire.com
instagram.com
wikipedia.org
lucire.net
facebook.com
linkedin.com
pinterest.nz
lucirerouge.com
nih.gov
twitter.com
—I don’t know much about this search engine, since I only heard about it from Holly Jahangiri earlier today. A very good effort, with only the ninth one being irrelevant to us: it’s a paper co-written by Yola. 9/10
 
Yandex
lucire.com
lucire.net
facebook.com
twitter.com
wikipedia.org
instagram.com
wikipedia.eu
pinterest.nz
en-academic.com
wikiru.wiki
—This is the Russian version. All are relevant, and they are fairly expected, other than the ninth result which I’ve not come across this high before, although it still relates to Lucire. 10/10
 
Bing
lucire.net
wikipedia.org
lucire.com
spanishdict.com
lucire.com
facebook.com
drlucire.com
spanishdict.com
twitter.com
lucirahealth.com
—How Bing has slipped. There are sites here relating to the Spanish word lucirse and to Lucira, who makes PCR tests for COVID-19. One is for Yola. 7/10
 
Qwant.com
lucire.net
wikipedia.org
spanishdict.com
drlucire.com
spanishdict.com
tumblr.com
lucirahealth.com
lacire.co
amazon.com
lucirahealth.com
—For a Bing-licensed site, this is even worse. No surprise to see lucire.com gone here, given how inconsistently Bing has treated it of late. But there are results here for Lucira and a company called La Cire. The Amazon link is also for Lucira. 3/10
 
Qwant.fr
lucire.net
wikipedia.org
reverso.net
luciremen.com
lucire.com
twitter.com
lacire.co
lucirahealth.com
viyeshierelucre.com
lucirahealth.com
—The sites change slightly if you use the search box at qwant.fr. The Reverso page is for the Spanish word luciré. Sixth through tenth are irrelevant and do not even relate to the search term. Eleventh and twelfth are for lucire.com and facebook.com, so there were more relevant pages to come. The ranking or relevant results, then, leaves something to be desired. 5/10
 
Duck Duck Go
lucire.com
lucire.net
wikipedia.org
spanishdict.com
drlucire.com
spanishdict.com
lucirahealth.com
amazon.com
lacire.co
luciremen.com
—Well, at least the Duck puts lucire.com up top, and the home page at that (even if Bing can’t). Only four relevant results, with Lucire Men coming in at tenth. 4/10
 
Brave
lucire.com
instagram.com
twitter.com
wikipedia.org
linkedin.com
lucire.net
facebook.com
fashion.net
wiktionary.org
nsw.gov.au
—For the new entrant, not a bad start. Shame about the smaller index size. All of these relate to us except the last two, one a dictionary and the other referring to Yolande Lucire. 8/10
 

The results are surprising from these first results’ pages.
 
★★★★★★★★★★ Google
★★★★★★★★★★ Yandex
★★★★★★★★★★ Startpage
★★★★★★★★★☆ Virtual Mirage
★★★★★★★★☆☆ Brave
★★★★★★★☆☆☆ Baidu
★★★★★★★☆☆☆ Bing
★★★★★★☆☆☆☆ Swisscows
★★★★★☆☆☆☆☆ Mojeek
★★★★★☆☆☆☆☆ Qwant.fr
★★★★☆☆☆☆☆☆ Duck Duck Go
★★★☆☆☆☆☆☆☆ Qwant.com
 

It doesn’t change my mind about the suitability of Mojeek for internal searches though. It’s still the one with the largest index aside from Google, and it doesn’t track you.

Tags: , , , , , , , , , , , , , , , , , , , ,
Posted in China, France, internet, publishing, technology, UK, USA | 2 Comments »


Bing is definitely very broken, and it’s hurting Duck Duck Go

23.07.2022

The last few days have been about ‘How awesome is Mojeek?’ and ‘How shit is Bing?’

I’m finding great search results from Mojeek, and as a site search for Lucire, it’s absolutely brilliant. Blows Duck Duck Go (Bing with privacy) away, even back when DDG had a reasonably comprehensive index of our pages (before the HTTPS switch). I don’t have to subject anyone to Google tracking, and I didn’t have the hassle of installing an internal search ourselves.

Cisene, who I met via Mastodon, very helpfully suggested on that social network that I submit site maps for the Lucire website as that would take a reasonably short time to remedy Bing’s ills. I’ve never had to do them for Google or Mojeek: their spiders work as they have always done since the dawn of search engines. For some reason, Bing needs its hand held if I want it to have thousands of pages again, as it did earlier this year.

One thing I found curious with Bing is its insistence, in a site search, to place a page that we have not linked to since 2005 at the very top. Of course I could delete the page or program in a forwarder, or make a 301, but I was also once told that dead links and forwarders were bad things for search engines. Our ‘About’ page also ranks highly in all search engines, despite not being linked to in anything we’ve done in over 15 years as well.

But where’s the home page? Happily, after submitting site maps, Bing’s index of our pages went from 10 to a whopping 55, and the home page appeared for the first time in a site:lucire.com search:
 

 

‘It’s an improvement,’ I thought, though the search engine is still massively handicapped compared to where it was at the start of 2022.

Checking on Bing Webmaster Tools to see where things were, I was curious to see it claim that it could not crawl or index our home page though it was discovered in 2018:
 

 

But you just crawled and indexed it. Which is it?

The excuses this time (as Big Tech people love to make stuff that blames users) are that there are no <H1> tags (I’ve got news for you, Bing: we don’t use them, and why should we? There was never any rule that stated that headlines must be between them, and no one else seems to care) and that the description is too long (again, it was fine for you before—and actually you’ve just shown that it is fine).

They aren’t in the business of search though, as their explanations reveal. It’s seach:
 

 
Goodness knows how many years that’s been there, ignored.

It’s all so slap-dash and unprofessional, and as Duck Duck Go search results are based on Bing’s, I’m going to have to stop recommending it. Fortunately, I found Mojeek at the perfect time.

I’m also discovering that maybe Bing can no longer handle more than 50-odd pages per site anyway, which, of course, makes it useless as an engine that powers a site search. (Like I keep saying, the defunct Excite in the 1990s could do better. Any search engine from those days could spider and index more effectively.) It would be in line with other Microsoft products, such as Notepad, where the software giant now prevents us from typing £ or , except, presumably, people from the countries where those are the common, keyboard-accessible currency symbols. Want to write Cæsar drinks Nescafé? You can try, but the diphthong and é will be missing.

Today I searched site:autocade.net on Bing. Now, we never switched Autocade to HTTPS. After how all our sites fell, would you risk it? This site is dependent on search-engine traffic.

And here are the number of pages each search engine brings up for a site search.
 
Google: 4,080
Mojeek: 3,348
Bing: 51
Duck Duck Go: 50
Brave: 17 (plus 4 underneath first entry)
 

So I can’t keep blaming the switch to HTTPS, though our troubles with all search engines I knew of then began around this time. Autocade still slipped in Bing despite no down time; we went to a newer Mediawiki version, but that was about it. Everything progressed as it always did.

Google eventually allowed things to recover (for the most part) with the exception of our company website (which rose up to 13th before dropping to 26th today), Mojeek never even had an issue to begin with, but Bing and Duck Duck Go don’t link to Jack Yan & Associates’ website till after the 40th position.

So where are we now with the sites I last looked at?
 
Number of results for site:lucire.com
Google: 6,250
Mojeek: 3,563
Bing: 53
Duck Duck Go: 53
Brave: 15 (plus 4 underneath first entry)
 
Number of results for site:jackyan.com
Google: 1,860
Mojeek: 438
Duck Duck Go: 54
Bing: 43
Brave: 13 (plus 4 underneath first entry)
 
Number of results for site:jyanet.com
Google: 743
Mojeek: 296
Bing: 49
Duck Duck Go: 49
Brave: 20
 

I honestly think Bing is broken.

Just as well no one I know uses it, but quite a number of people do opt for Duck Duck Go, because of the work it’s done in promoting privacy. I still admire them for this stance. But as many of you know, it sources its results from Bing, so if one is broken, both will be. And that’s a darned shame as I almost hit 12 years of having Duck Duck Go as my default (from August 2010 or thereabouts).

All the more reason to retain Mojeek as my default search engine.

Will I bother looking any more into Bing? Probably not, but how do I convince all those I recommended Duck Duck Go to to check out Mojeek?

Tags: , , , , , , , , , ,
Posted in internet, marketing, publishing, technology, USA | 4 Comments »


Forget Duck Duck Go, Bing, and Google—I’m trying Mojeek

17.07.2022

It was disappointing to note that after switching to HTTPS, and signing on to Bing Webmaster Tools, the search engine results for those sites of ours that made the change are still severely compromised.

I’ve written about searches for my own name earlier, where my personal and company sites lost their first and second positions on all search engines that I knew of after we made the switch. Only Google has my personal site back up top, with the company site on the middle of the second page. Bing has my personal site at number two, and I’d love to tell you where the company site is, but their search engine results’ pages won’t let me advance beyond page 2 (clicking ‘next page’ lands you back on the same page; clicking ‘3’ and above still keeps you on p. 2). Duck Duck Go, which uses Bing results, has it well below that—I gave up looking. And this is after I signed up to Bing Webmaster Tools in the hope I could get the sites properly catalogued.

It’s a real shame because Duck Duck Go has been my default for 12 years this August.

However, it was the loss of search results for Lucire that really bothered me. Here’s a site that’s 25 years old, with plenty of inward links, and c. 5,000 pages. Before the switch to HTTPS, the popular search engines had thousands of pages from our site. These days, Bing and Duck Duck Go tell me they have dozens of pages from Lucire’s website. Again, only Google seems to have spidered everything.

When I check Bing Webmaster Tools, the spidering has been shockingly poor.

The received wisdom that you should have HTTPS instead of HTTP to do better in search engines is BS, and the belief that search engines will eventually catch up has also not been realized. We made the switch in March, and I’m to believe that Bing hasn’t completed the indexing of our sites.

Are they using the same computers New Zealand banks do? (Cheques used to clear overnight in the 1970s, and now banks tell us that even electronic payments can take days. When we last used cheques, they were telling us they would take five to seven days. Ergo, bank computers are slower today than in 1976.)

The real downer is that Lucire’s website search box is powered by Duck Duck Go, so our own site visitors can’t find the things they want to look for. If you believe some of the search engine marketing, over 40 per cent of site visitors use your search function.

What to do?

I began looking at having an internal search again. We used to have a WhatUSeek (later SiteLevel) internal site search, but that site’s search functions appear to be dead (the site is still live). A user on Mastodon recommended Sphinx Search, an open-source internal site search, but the instructions were too complex. I even saw real computer geeks having trouble. The only one that I could understand was called Sphider—I could follow the instructions and knew enough about PHP and MySql—but it was last updated many years ago, and successive projects also looked a bit complex.

A Google internal search was absolutely out of the question, as I have no desire to expose our readers to tracking—which is why so many other Big Tech gadgets have been removed from our site(s). Baidu and Yandex also have very limited indices for our sites.

I am very fortunate to have tried Mojeek again, a British search engine recommended to me by Matias on July 2. What I didn’t know then was Mojeek has its own spider and its own index, so it doesn’t have to license anything from Bing. And, happily, it claims to have 3,535 results from lucire.com, which might not be as good as Google’s 5,830, but it beats Bing’s 50 earlier today—in fact, at the time of writing, it showed a grand total of 10. That’s how bad it’s got. Duck Duck Go now has 48, also down from a few thousand before March.

Like Google, it seems to have coped with the switch to HTTPS without falling to pieces! And guess what? For a search of my own name, my personal site is number one, and our work site is number two. Presumably, Mojeek is the only search engine which coped and behaved exactly as the experts said!

You can imagine my next move. Mojeek has a site search, so now all Lucire searches are done through it. And readers can actually find stuff again instead of coming up nearly empty (or having very irrelevant results) as they have done for months.

Duck Duck Go’s lustre had been wearing off as there were recent allegations that its browser allowed Microsoft to track its users, something which Duck Duck Go boss Gabriel Weinberg personally denied on Reddit, saying that users were still anonymous when loading their search results.

I still have good memories of chatting to Gabriel in the early days and figuring out ways of spreading the word on Duck Duck Go. My contribution was going to hotels and changing the search defaults on business centre computers. Back then I had the impression Duck Duck Go did some of its own spidering, but these days, if Bing has a shitty index for your site, the Duck will follow suit. And with HTTPS not living up to its promise, that’s simply not good enough.

Tonight, Mojeek is very much the site of the day here, and I heartily recommend you try it out. I’ve switched the desktop to Mojeek as a default, and I’ll see how it all progresses. Right now I feel it deserves our support more than Duck Duck Go. Finally, we might truly have an alternative to Google, and it’s run from the UK’s greenest data centre. With our servers now being greener, too, running out of Finland, the technology is starting to match up to our beliefs.
 

Google, the biggest index of them all
 

Mojeek, a creditable second place
 

This is it on Bing: a 25-year-old history on the web, and it says it has 10 pages from lucire.com. Altavista, Excite and Hotbot had more in the 1990s
 

Duck Duck Go is slightly better, with 48 results—down from the thousands it once had
 
After switching to HTTPS
Number of results for lucire.com
Google: 5,830
Mojeek: 3,535 (containing the word Lucire, as term-less searches are not allowed)
Duck Duck Go: 48
Bing: 10
 
Number of results for jackyan.com
Google: 878
Mojeek: 437 (containing the term “Jack Yan”)
Duck Duck Go: 54
Bing: 24
 
Number of results for jyanet.com
Google: 635
Mojeek: 297 (containing the word jyanet)
Duck Duck Go: 46
Bing: 10
 

Presumably the only search engine that could handle a server going from HTTP to HTTPS and preserving the domains’ positions

Tags: , , , , , , , , , , , , , , , , ,
Posted in business, internet, publishing, technology, UK | 1 Comment »