Part 1: The problem of the local maxima in leadership

Reading Time: 2 min 30 seconds

The greatest skill you learn at business school is thinking from the 10,000 foot perspective. You learn systems thinking and figure out how to disrupt a system by making it more efficient. Elon Musk did this for Space X by reducing the cost of a node in the system. Venmo increased the speed of a path in the system by making inter person money transfer easy. The root of all disruptive technologies lies in this line of thought.

Elon Music affirmed this in his interview with Chris Anderson.

"I tend to approach things from a physics framework. And physics teaches you to reason from first principles rather than by analogy. So I said, OK, let’s look at the first principles. What is a rocket made of? Aerospace-grade aluminum alloys, plus some titanium, copper, and carbon fiber. And then I asked, what is the value of those materials on the commodity market? It turned out that the materials cost of a rocket was around 2 percent of the typical price—which is a crazy ratio for a large mechanical product."

Data Science leaders of today face a critical challenge. Data Science leaders need to understand the details. This could the model choice or the deep learning architecture . Their success also depends on the understanding of the broad business strategy. AI is successful only when you use it to solve a systems inefficiency. In essence, you need to be very detailed, operate in the world of specifics and simultaneously step up and think about the big picture. The failure to balance this bi-modal thinking leads to sub-optimal data science implementations.

Let me give you an interesting example. Last week, I attended a very cool research presentation. A group of researchers built an optimization algorithm to help famers decide what combination of seeds to sow. It aimed to solve a fundamental problem facing humanity - food shortage. Obviously, the research has won the praise from both academia and commercial companies. The farmers could use this simple simulation to maximize their farm's yield for the year.

As I was listening to the presentation, I couldn't help but think about the problem of crop rotation. Crop choice has a long term effect on soil fertility. A great yield in the current year may not correlate to ongoing results, year after year. This is foundation of the principle of crop rotation

Let us imagine the case where the data scientist does a fantastic job of optimizing the objective function - the yield for the current year. It would function so well in the short term but fail over the long term. In fact, it might worsen the food shortage problem. This imaginary situation manifests itself in a number of scenarios. You can trade off margin for revenue in pricing or coerce a customer to buy by inundating them with marketing.

A Data Science leader can get easily sucked into the details of the problem, forget about the fundamental issue with the problem statement and ultimately lead the team toward the development of a sub-optimal solution. 

This is the celebrated issue of the local maxima.Seth Godin has written about this issue from the perspective of perseverance. The same logic can be applied toward a data science problem as well. Moving from a local maxima to a global maxima requires traversing through a scary period. You have to let go of a "known" and "working" solution and instead persevere to find a better one. It automatically requires the tradeoff between implementation time and long term success. 

Does your organization successfully apply this perspective?