logoalt Hacker News

theptip • today at 5:32 PM • 1 reply • view on HN

This is more about streamlining the Date Engineer / Data Scientist role. Business Intelligence, in other words.

Before: domain expert wants to ask a question about the data, but which of these hundred tables actually has the right “revenue per User” column? Ask a DE to help slice the data and bless the queries.

After: ask your agent. The agent looks at the ontology and can usually find the best blessed data. To enable this you need to write down all the tribal knowledge that lived in the DE/DS head; agents can take the first pass at proposing and updating metadata.

Under the covers the DE and DS roles switch to reviewing the underlying ontology, instead of working directly with business stakeholders on one-off queries.


Replies

ewild • today at 6:08 PM

well im just not gonna write down the tribal knowledge lmao