alt Hacker NewsI think they have something better - “link rank” which essentially takes into account the quality of backlink.
I believe it is nuanced enough to have different rank per “topic”, or “keyword” etc. but admittedly just kinda guessing from the outside.
The last time I tried to build something like this I realized it’s useless without first having a gigantic amount of data already crawled. When I started crawling I realized I would never catch Google. I think without Wikipedia the LLMs might have taken 10 more years to surpass them.
Crawlers would need to use backlinks but also rank vector similarity to ensure the linked content matches the linked intent. Some kind of rainbow shades of how relevent the link is to the linkee and reverse.