Implemented Data Engineering Projects

Repo for all the projects ( vertical post)…

Github : https://github.com/Coder-World04/Complete-Data-Engineering

Link to the complete post

Scripting and Automation

Shell Scripting

ETL ( Extract, Tranform and Load) basics

Why ETL is important?

How ETL works

ETL Tools

Relational Databases and SQL

Basic SQL

Advanced SQL

NoSQL Data bases and Map Reduce

Data Warehouses

Data Lakes

Structured Data

Semi Structured Data

Unstructured Data

Data Mart

Map-Reduce

Data Analysis

Pandas

Numpy

Advanced Pandas Techniques

Data Pre-processing

Handling missing values

Data Cleaning

Mean/mode/median Imputation

Hot Deck Imputation

Rescale Data

Binarize Data

Regression Imputation

Stochastic regression imputation

Feature Scaling

Data Augmentation

Read and Process Large Datasets

Data Visualization basics

Data Visualization Projects

Data Visualization using Plotly and Bokeh

Data Profiling

Summary Functions

Indexing

Grouping

Linear Regression

Multi Linear Regression

Polynomial Regression

Regression

Support Vector Regression

Decision Tree Regression

Random Forest Regression

Feature Engineering

GroupBy Features

Categorical and Numerical Features

Missing Value Analysis

Fill the missing Values

Unique Value Analysis

Univariate Analysis

Bivariate Analysis

Multivariate Analysis

Correlation Analysis

Spearman’s ρ

Pearson’s r

Kendall’s τ

Cramér’s V (φc)

Phik (φk)

Data Processing Techniques

Batch Processing

Stream Processing

Apache Spark

Apache Spark Commands

Apache Kafka

How Apache Kafka works

Big Data

Big Data

Types of Big Data

Big data tools

SQL and NoSQL Databases

Hadoop

Hadoop HDFS

Hadoop Yarn

Hive

Zookeeper

Pig

Cassandra

Sqoop

Data Pipelines and WorkFlows

Data Pipelines

Transformation

Processing

Workflow

Monitoring

Airflow

DAG

Infrastructure

Docker

Docker vs Virtual Machines

Most important Docker commands

Kubernetes

Snowflake

Power BI

Power BI

Which chart to use and When?

Power BI — Data Analysis Expressions

Joins

Data Profiling

Cloud Data Engineering

Data Engineering on cloud

AWS

AWS Services

Google Cloud Platform

Google Cloud Platform services

Machine Learning Algorithms

Linear Regression

Logistic Regression

Decision Trees

Random Forest

Support Vector Machines

K Nearest Neighbors

K means Clustering

Hierarchical Clustering

Neural Networks

Continue reading on Coders Mojo »