A new transformers architecture based on relational data instead of text. Been exploring search reranking based on graph data and customer prediction.

https://github.com/relativedb/relational-transformers

https://relationaltransformers.com/

You can read more about the architecture here: https://relativedb.com/blog/relational-transformer