– Charanpreet Singh, Founder, Praxis Business School
When we try to understand careers we need to find answers to the following three questions: one, what is the demand-supply equation for resource talent in the field? two, what are the roles and hence the skills required to build a career in the area? and three, is this the right career for me given my aptitude and background? We will examine each of these questions in some detail in the context of Data Science and Data Engineering.
Demand-supply equation
We are in the midst of the Digital Revolution – driven at the core by high-speed internet connectivity and accelerated by the proliferation of mobile devices that provide access. The pandemic has accelerated this digital adoption and we are transacting online at scale – generating data in unimagined volumes as each digital transaction leaves a data footprint.
Our ability to access, store, analyse these huge volumes of data to understand our business better has made data the most valued resource for companies and governments – driving innovation, competitive advantage and customer intimacy.
This has led to exploding demand for the ‘data professional’ – someone who is skilful at working with data and unlocking its value. In India, the current demand-supply gap is estimated at anywhere between 140,000 to 300,000 for data scientists alone – and the demand for data engineers is significantly higher. This gap is likely to grow larger as we keep accelerating data generation and the talent supply struggles to keep pace.
Every large (or even medium-sized) business across domains like banking and finance, retail, health, telecom, CPG, manufacturing, consultancy, IT and ITES etc. requires data scientists and data engineers – and we have seen at campus placements that salary levels for the right resources are continually moving upward.
Skill requirement in Data Engineering and Data Science
Data Engineers create organization and industry-wide access to clean, reliable data at scale with proper data infrastructure and architecture, thereby making it possible for data scientists to perform analytics and generate insights.
Hard skills: Covering but not limited to programing (SQL, Python, Java), DBMS, OS, BI (Data Warehouse), Cloud (AWS, Azure), Big Data (Hadoop, Spark, HIVE, NoSQL), DevOps, Machine Learning, Domain Knowledge.
Data Engineers thus need to be passionate about and proficient in programming and working in the tech environment.
Data Scientists analyze, process, and model data and interpret the results to create actionable plans for organizations.
Hard skills: Covering but not limited to Mathematics, Statistics, Machine Learning, Deep Learning, AI, Visualization; technology skills including Python, R, Tableau, PowerBI, Big Data; business domain expertise.
Data Scientists solve business problems using their expertise in the areas of computer science, mathematics, statistics and business domain.
Both Data Engineers and Data Scientists need to have superior soft skills like critical thinking, collaboration, communication, creativity.
Who should seek a career in Data Engineering or Data Science?
Both data engineering and data science require analytical thinking and a problem-solving mindset. Coding is an integral part of these roles, especially so in the case of Data Engineering. From an academic background perspective, data engineers need to have a tech background – engineering, BCA, MCA etc. or significant experience in a tech role. Data Science requires proficiency with numbers, understanding of statistics and business contexts. While a tech background is helpful, we have also seen people from economics, statistics, BSc/ MSc, business management backgrounds transition to data science as a career.
It is important to understand that both these domains demand resources skilled in solving complex problems using a variety of specialized techniques and technologies. It, therefore, makes a lot of sense to invest substantial time and attention to acquiring these skills, especially if you wish to make either of these your long-term career.
Also read:
Data Science & Engineering: Emerging career of the 21st century