Q: Data Science or Data Engineer?

Picking the Right Position for You

Hi Sarah!

I am currently a Data Scientist at a small company. I am considering a switch to data engineering. How can I determine if this is the right career path for me?

Thank you,

Jane

Hey Jane,

This question is near and dear to my heart as I have struggled with that question on various occasions. I have been a data scientist, data engineer, and now a machine learning engineer as well.

When you first enter the field, everyone tells you to join data science. So that’s what I did. I became a data scientist, but regardless of the projects I was on, I improve the architecture, automate my(or the team’s) code, and develop my own code skills.

That’s how I figured out what my passions were — I wanted to improve my ability to manage architectures and my own coding skills.

And that’s my suggestion to you. Regardless of what route you go, you will want to find out what your passion is. And how to do that is what you naturally want to learn.

The first recommendation was to find your passion.

The second recommendation is to figure out if your interests align with the general definitions of data science or data engineering. Although the lines between a data scientist and a data engineer are often blurred, there are key characteristics that differentiate them.

Data scientists analyze and extract insights from data, often using programming languages like R, Python, or SQL. They use these insights to help businesses make informed decisions, improve processes, and detect patterns that might not be visible to the naked eye. They work with large amounts of data from various sources, including social media, customer interactions, and sales data. They also use statistical models and machine learning algorithms to predict outcomes and identify trends.

Data engineers play a crucial role in the data world. They are responsible for building and maintaining the infrastructure that allows data scientists to do their work efficiently and effectively. This infrastructure includes data storage systems, data processing pipelines, and data integration tools that help to transform raw data into meaningful insights.

To summarize data engineer and data science roles, I have created a table with key characteristics.

	          Data Engineer	  Data Science
Visualization	      No	        Yes
Coding-heavy	      Yes	        No
Analytics 	      No	        Yes
Machine Learning      No	        Yes
Database Tuning	      Yes	        No 
SQL Optimizations     Yes	        No

Are you interested in any of these more than the others? If so, that’s your path forward!

Now, we often have a fear that if we pick the “wrong” job, we cannot go back. Ever. And that’s definitely not the case in the data field. You can become a data engineer and realize this field sucks and move to data science. And vice versa. You do not have to be afraid to pick and to pick wrongly. Data is all about experimentation and iteration, so why not do the same with your career?