I think they have something better - “link rank” which essentially takes into account the quality of backlink.
I believe it is nuanced enough to have different rank per “topic”, or “keyword” etc. but admittedly just kinda guessing from the outside.
The last time I tried to build something like this I realized it’s useless without first having a gigantic amount of data already crawled. When I started crawling I realized I would never catch Google. I think without Wikipedia the LLMs might have taken 10 more years to surpass them.
I think they have something better - “link rank” which essentially takes into account the quality of backlink.
I believe it is nuanced enough to have different rank per “topic”, or “keyword” etc. but admittedly just kinda guessing from the outside.
The last time I tried to build something like this I realized it’s useless without first having a gigantic amount of data already crawled. When I started crawling I realized I would never catch Google. I think without Wikipedia the LLMs might have taken 10 more years to surpass them.