FAANG+ Data Engineer Learning roadmap for 2024
With AI & Data Science taking the center stage of all new tech investments, data engineering is set to grow rapidly with time. Good AI cannot be done without good Data Engineering in place.
Great Data Engineering is fundamental to great Data Science products !
Roadmap to become a FAANG-ready Data Engineer:
🚀 STEP 1: The Basics:
- 📚 Learn #SQL:
use and practise commands like : SELECT, FROM, WHERE, GROUP BY, JOIN, HAVING, etc
2. 🐍 Learn #Python:
Learn data structures and algorithms: array, dictionaries, recursion, string & array manipulation, loops.
3. 🔥 Learn #Pyspark:
Functions like sparkSession, Read, filter, groupBy, aggregations, write.
🚀 STEP 2: Intermediate :
- 🌪️ Learn #Airflow
Airflow is used to for data engineering job scheduling and orchestration
2. 🛢️ Learn #data #lake architecture & concepts
Definition,Structured and Unstructured Data,Hadoop Distributed File System (HDFS),Data Ingestion
3. 🌟 Advanced SQL:
Window functions, CTEs, Nested queries, Solving SQL problems using self-joins.
🚀 STEP 3: Advanced :
- 🚀 Learn #data #modelling techniques:
one big table ,kimball , Inmon , data vault techniques
2. 🧠 Understand #spark architecture:
How Spark does the processing in a scalable manner.
3. #Spark optimization 🌟:
Predicate Pushdown, partitioning, broadcast, cache&persist, shuffle, Parallelism, Repartition&Coalesce
Annnd you’re done..
About me :
I’m working as a Lead Data Engineer. Within 3 years of my career, I was promoted to SDE-3 @ Walmart, and then joined the biggest hiring platform in India as a Lead Data Engnieer, in a team which works on next-gen AI use-cases : like recommendations, generative AI, ML based classification.
I have attended and cleared 20+ interviews including Big Tech cos like Amazon, Walmart, UnitedHealth Group(Fortune top 5), Flipkart(Owned by Walmart), Media.net
If you like learning about data and working with data, consider subscribing to email alerts on medium, or follow me on linkedIn & Medium !!
If you liked this article, do let me know by leaving a comment or “clap” on medium !!
I’m also offering Data Engineering learning links/content, Personalized Data Engineering learning programs, which you can find on Topmate : https://topmate.io/niteshx2
