The Dawn of the Data Intelligence Engineer

It's frustrating how the data discipline is split up into so many different job titles. I regularly have to work with data engineers, data analysts, analytics engineers, data scientists, machine learning scientists, and machine learning engineers. Each view their role really narrowly, so you need to pull in a lot of people to get something done. It's often easier to just do the work yourself.

I get that some specialisation is useful, but I would like to see more T-shaped data folks.

I think there are big risks with just labelling it as "wild west" and let users do whatever they want there.

It's always been possible to massage your data to fit whatwver case you want to make. Previously you had to go through a data analyst and convince them to let go of their principles. Now you can just let an agent figure it out.

So how do you make sure insights are actually shareable (and trustable) with other parts of the company?

I think just exposing the metrics layer is probably the answer here.

It's kind of funny that the industry has tried to democratize data for so long, but now we scramble to restrict access to protect users from making incorrect decisions.

While in general I agree with most of what's said, some of these feel like "Horseless Carriages".

Like for example

>It doesn't know how your company counts revenue, or which of several similar-looking tables is the real one (it is revenue_cpx_new_revised_v2_use_this_one )

This assumption mostly only holds for non-clean data ( which is mostly a result of humans being lazy and backlog being clogged up ). As AI starts building more of the pipelines upstream - this becomes a non-issue.

Even for existing pipelines, I'd expect fixes to move upstream: when an agent encounters ambiguity at query time, the useful output isn't an edit of the context,but a PR that fixes it.

I’m here using Opus 5.5 and Fable 5.2 and I wouldn’t say the code is generally very good.

That's just data engineer enhanced with agents.

everything old is new again. it used to be the venerable ETL guy who would be the pivotal cog in the machine learning gear. someone who knows where the data is and what it means is always important.

The best quote I've seen about AI and work: "If your job can be done from a laptop, you should be seriously considering what you'll be doing in 5 years".

> A model doesn’t know that when the sales team talks about revenue they mean commissionable revenue, and the finance team uses something different.

Umm, the model knows the same way humans know: either by looking into slack threads/confluence pages/emails… If your model has access to those things, then it can know. And I foresee that in the near future the model can simply have an incarnation as a Slack member, and when context is not enough, they will simply ask people around (and update docs accordingly)

People don't don't make decisions by looking at dashboards and visualizations. They do while looking at emails and listening to others in calls.

Nice, i can apply to be one for the 11 months this position shall exist

On a more serious note, I want my next title to be "Super High Intelligence Trainer"

Like a PT for AI's ... Getting them into shape, and fighting strong.

The job you will DIE for.

The correct nomenclature is now “Super Intelligence Engineer”, right? I think it’s required if you want to seek any federal government contracts for the next two years, at least.

Such blatant LLMisms ... "blast radius," concepts living in other concepts, someone "living with the churn number" (!?), etc.

They asked ChatGPT to invent a new job title for marketing, genius. At a previous company our marketing person created some new term for a segment so we could say we are the only ones doing it. I think there was a term for this grift but I don't remember.

This fake job title sounds like what BI analysts or analytics engineers should be doing as part of their job, not a new job. These jobs are the ones that are usually working on the modeling, answering business questions, and making data accessible and would be creating the context, prompts, and evals to make sure it's working as intended. I think really, analytics engineering and BI should be combined.

Can we move this whole business outside of CS please?

so now you want to be a data intelligence engineer (die) ? wait die ? like my career ? oh yeah!

[flagged]