How to Become an GCP Data Engineer in 2024
- Intellibi Innovations Technologies
- 12 Sept 2024
𝗣𝗿𝗲-𝗿𝗲𝗾𝘂𝗶𝘀𝗶𝘁𝗲𝘀
- SQL
- PL/SQL (Stored Functions, Stored Procedures, Indexes)
- Python
- Spark (PySpark, Spark SQL)
GCP 𝗦𝗲𝗿𝘃𝗶𝗰𝗲𝘀
- GCP Bucket:
- Structured Data: e.g., Table Format (CSV, Excel)
- Unstructured Data: e.g., Videos, Images, Text, PDFs
- Semi-Structured Data: e.g., JSON Files
- Cloud SQL and cloud spanner:
- Google BigQuery:
- Dataflow:
- Apache Airflow
- Cloud Composer
- Dataprep:
- Dataproc:
- Cloud Data Fusion
- Google Pub/Sub:
- Machine Learning (ML) on GCP:
- BigQuery ML:
- Vertex AI:
- Data Lake & Delta Lake on GCP:
- DataOps on GCP:
- Data Visualization:
- Building ETL Pipelines:
𝗘𝗻𝗱-𝘁𝗼-𝗘𝗻𝗱 𝗣𝗿𝗼𝗷𝗲𝗰𝘁𝘀 (At least 2)
- Data Ingestion End-to-End Pipeline: using Cloud Composer, Dataflow, BigQuery, Pub/Sub
- Data Ingestion End-to-End Pipeline: using other GCP services.
𝗔𝗧𝗦-𝗖𝗼𝗺𝗽𝗹𝗶𝗮𝗻𝘁 𝗥𝗲𝘀𝘂𝗺𝗲
- Score: 80+
𝗜𝗻𝘁𝗲𝗿𝘃𝗶𝗲𝘄 𝗣𝗿𝗲𝗽𝗮𝗿𝗮𝘁𝗶𝗼𝗻
- Fundamental Concepts: Master both basic and advanced topics as outlined above.
- Scenario-Based Questions: Solve at least 100+ questions.
- Mock Interviews: Practice regularly.