A few years ago, when I'd just started working in grocery retail, I walked into a leadership meeting full of confidence. My team had built a prediction engine that recommended products based on customer shopping preferences and contextual factors like the weather. It was spring, the sun was out, and I had a polished demo showing how we could gently nudge online shoppers to consider adding ice cream to their order, complete with a solid projection of additional sales.
I got interrupted a few minutes into my pitch. The leadership team wasn't interested in selling more ice cream. What I hadn't understood was that adding only one single frozen product to the basket was a drag on ecommerce profitability. Picking, packing, and delivering one box of ice cream alongside regular groceries cost more than it earned. The business wasn't trying to maximize revenue. It was trying to reach profitability. A euro of additional revenue and a euro saved are not the same thing, and I had optimized for the wrong one.
That experience taught me something I've seen confirmed hundreds of times since: the data professionals who consistently deliver outcomes aren't the ones with the strongest technical skills. They're the ones who understand how value moves through the organization.
This isn't about soft skills in the vague, hand-wavy sense. It's a specific capability: understanding the dynamics of the business, how teams depend on each other, how leadership makes trade-offs, and what forces and incentives shape decisions.
Most data professionals optimize locally. They improve a model, streamline a pipeline, build a better dashboard. That work matters. But it operates within a fixed frame: making the current setup perform better. In optimization theory, this is called a local maximum. You're climbing the nearest hill, but you may be on the wrong hill entirely.
The professionals who shape strategy operate differently. They step back and ask whether the organization is even solving the right problem. They connect their team's work to the broader business agenda. They translate between technical possibilities and strategic priorities. They pursue the global maximum.
This is the difference between doing your work and delivering outcomes. Task-oriented professionals complete what's asked. Outcome-oriented professionals understand why it was asked, who it affects, and what success actually looks like beyond their own deliverable.
And this doesn't depend on seniority. Many directors and VPs built successful careers by consistently meeting expectations and hitting KPIs. That got them promoted, and rightly so. But the same approach that drives individual success can keep an organization stuck. When leaders at every level focus on optimizing within the current frame, nobody is questioning whether the frame itself needs to change. The step change to truly impactful outcomes requires someone who looks beyond the scoreboard.
Ask any data leader about their biggest challenges, and most will describe what sounds like technical problems. Fragmented data landscapes, inconsistent definitions, poor data quality. But in nine out of ten cases, these are organizational problems wearing a technical disguise. Who decides what? Who owns which data, and do they have the authority to match that accountability? Are incentive structures aligned, or do they quietly encourage every team to build their own version of the truth?
Working harder doesn't solve this. Neither does better technology.
I'd been reading about organizational change when I encountered Ronald Heifetz's distinction between technical problems and adaptive challenges, and I recognized it straight away. This was the pattern behind most of the data challenges I'd seen. Technical problems have known solutions. Adaptive challenges require people to change how they think and work. Most data problems are adaptive challenges that organizations keep treating with technical fixes.
Understanding how organizations work, and more importantly, how change works in organizations, is the skill that makes lasting impact possible. It's what separates data leaders who deliver tangible results from those who keep running into the same walls with shinier tools.
After the ice cream experience, I started reaching out to people across the organization in completely different functions. I wanted to understand what actually drove profitability. I quickly learned it wasn't just about selling more. It was an intricate balance between customer needs, assortment, and operational efficiency. That understanding reshaped my work from that point forward.
That habit of looking beyond my own domain stuck. And it kept paying off. Decisions that look irrational from the outside, like buying the same tool from different vendors across markets, almost always make perfect sense once you understand the incentive structures behind them. You stop asking "why don't they just standardize?" and start asking "what's making standardization too expensive for them right now?"
AI makes this even more urgent. It puts your data under a magnifying glass. When your data foundations and organizational structures are sound, AI accelerates what you do. When they're not, your innovation projects run straight into a brick wall. And again, most of these obstacles aren't technical. They're the hidden structures, misaligned incentives, competing priorities, and what I'd call organizational data dysfunction, that make data problems so persistent. If these were purely technical challenges, we would have solved them years ago. We've moved from on-prem databases to data warehouses to data lakes to lakehouse architectures. The technology kept evolving. The dysfunction remained.
Nearly twenty years into this field, I'm more convinced than ever that data dysfunction won't be solved by the next platform migration or the next AI tool. It will be solved by people willing to challenge the most powerful sentence in any organization: "This is just how we do things around here." That's uncomfortable. It's also how you stop delivering tasks and start delivering outcomes.