Adopting a New DataOps Tool

The framework that will convince them

Photo by Christina @ wocintechchat.com on Unsplash

You find yourself gazing at the wall, lost in thought about a significant challenge that has been plaguing the team lately. As you take a moment to reflect on this matter, you begin to wonder if the root of the problem lies in the vast, unstructured tables that exist within the company's data warehouse. Or could it be that the sudden appearance of a new column in the data lake has caused confusion and errors among team members? It's not entirely clear, but one thing is certain: you are determined to get to the bottom of this issue and prevent it from happening in the future.

You have found the perfect DataOps tool for alerting team members early on before pipelines fail. But how do you actually implement it within the team?

Today, I will walk you through a framework that I often use to get buy-in to get implement a new methodology or tool.

Here are the framework components:

1. Define the problem.

2. Convince your teammates

3. Convince your manager

4. Convince the outside team’s leadership

5. Introduce the new tool/methodology session

6. Create internal-team advocates

7. Training conversations to address concerns

Define the Problem (or Pain Point?)

One thing that I often do to figure out the problem is to take a step back and ask myself what the problem is. I will then write out my initial thoughts. Then, try my hardest not to eliminate or diminish any thoughts.

By writing these thoughts down, you can begin to organize your ideas and gain clarity on the situation. Additionally, this exercise can help to identify any underlying issues that may be contributing to the problem, allowing you to address them as well. So, while it may seem odd at first, to write down your thoughts, taking a moment to reflect and write down your thoughts can be a valuable tool in problem-solving.

Plus, now you are able to share this with other folks to get additional feedback (or complaints)!

Convince Your Teammates

Convincing your teammates to support your solution or methodology is generally considered to be one of the easier tasks. However, it is important to remember that this step is still crucial to the success of what you are trying to do.

It may sound silly at first, but it is important to acknowledge that you and your teammates have a shared history and experiences. This shared history and experiences are what allow you to understand each other better and work together more effectively.

For instance, both my teammates and I face similar problems in our project - we are dealing with legacy pipelines, need more insight into our data, and face the same infrastructure breakdowns. However, this shared understanding of our project's challenges can be leveraged to our advantage. We can use this knowledge to better understand our teammates' perspectives, identify what they want, what is most important to them at this moment.

This can help me tailor the presentation of the tool or methodology so that they can view its importance and easily adopt it.

Convince Your Leadership

Convincing leadership will need to be more intentional.

The first thing you need to remember is that they have a lot on their plate. They are typically focused on the big picture and may not be as familiar with the specifics of the data operations within your team. It is therefore vital to communicate the value of the tool or methodology and how it will benefit the organization as a whole.

One effective way to do this is by creating a business case or ROI analysis that demonstrates the tool's potential impact on the company's bottom line. This can help your leadership team understand the tangible benefits of investing in the tool and how it will ultimately save the organization time and resources in the long run. And what better way to do that than with a proof of concept to test the tool's or methodology’s capabilities and gather data on its impact. By presenting this data to your leadership team, you can help them understand the tool's potential and see firsthand how it can improve your team's operations.

Convincing the Outside Team’s Leadership

Convincing someone else's leadership can be challenging, but it is not impossible. We often forget the power of pain points for teams, and being able to solve them with a tool that requires minimal effort will make their day (and their team's).

Combining those pain points with the data acquired to convince the leadership team (and of course the leadership of the team itself!)

Combining those pain points with the data acquired to convince the leadership team (and of course the leadership of the team itself!), we are now able to convince the other team's leadership to adopt the new tool/methodology. By presenting the benefits and potential impact on their team's operations, we can help them see the value in investing in the tool and how it can ultimately save them time and resources in the long run. This will help ensure a smooth integration of the new tool/methodology across both teams and improve overall project operations.

Introduce the new tool/methodology session

Now that your problem, team members, and leadership are on board, it's time to set up a sharing session between the teams to introduce what you will be building together.

This may be the first time any of the teams are realizing they will be building this tool or methodology. Change is hard for anyone, and in this instance, a lot of the code that these data engineers (or data analysts, data scientists, etc.) have built is about to be changed.

This presentation needs to take all of that into consideration.

The sharing session should have two parts.

The first part should cover what has happened so far, including the historical background, what things will look like after the tool is implemented, and how this particular methodology or tool will improve the situation.

The second and most important part showcases some simple examples that they will need to implement in the upcoming months.

Make sure to give them enough time to ask questions and answer them!

Create internal-team advocates

Creating internal-team advocates involves identifying team members who are enthusiastic about the new tool or methodology and are willing to help support its adoption. These advocates can help to promote the value of the tool to their colleagues and encourage its use in day-to-day work. Additionally, they can serve as a resource for team members who have questions or concerns about the tool, providing guidance and support as needed. By fostering a network of internal-team advocates, you can help to ensure the successful adoption of the new tool and improve overall project operations.

Training conversations to address concerns

Everyone knows now. The internal-team advocates and probably the rest of the teams are struggling to implement the tool or methodology. While they have gained some understanding of how it works, they are still struggling to figure out their role in the adoption cycle. This confusion is causing delays and inefficiencies in the process, as team members are hesitant to proceed without a clear understanding of their responsibilities. It may be helpful to provide additional training or resources to help them better understand how to utilize the tool in their day-to-day work. Additionally, open communication channels between the internal-team advocates and the rest of the team can help to clarify any remaining questions or concerns. By taking the time to address these issues and provide ample support, the team can move forward with confidence and efficiency, ultimately leading to the successful adoption of the new tool.

Conclusion

This newsletter discusses a framework for implementing a new methodology or tool in a data operations team. The framework includes defining the problem, convincing teammates and leadership, introducing the new tool, creating internal-team advocates, and providing training to address concerns. The article emphasizes the importance of communicating the value of the tool and its potential impact on the organization's bottom line to leadership and other teams. Additionally, it suggests fostering a network of internal-team advocates to promote the tool's adoption and provide support to team members.