• Image

Best Big Data Training in Chennai

Learn from our Bigdata experts from the scratch to extreme level to do the analytics about the data. The syllabus designed to give you training more on practical way with set of scenario based question along with mini projects. It helps you to learn in expert level.

What Is Big Data?

It is very difficult to process complex and large set of data With the help of traditional systems like RDBMS, Enterprise System, etc. Now a days any business like Banking, Finance, Telecom or social media like Twitter, Facebook generating more than 90% of data what they generated few years back. Hadoop is the system or frame work used for storing or handling those large volumn of data including different formats like structured, unstructured semi structured. This system is generally designed to scale up to thousands of servers. Also it is open source provided by Apache. Most of large giants like Google, Twitter, Facebook, Amazon, etc are moved their project into hadoop eco system.

Why should join Big Data Training in Infycle?

This course used to analyze more on about data and you can play more on analytics part which will leads to become an data scientist. This course is best base of all data related technologies like Data Science, Machine Learning, IOT, AI, etc. In this digital world, Going forward every business developement cycle purely based on data and data analytics. So it helps everyone to get into analytics part more.

Introduction to Apache Hadoop and the Hadoop Ecosystem

Introduction to Apache Hadoop and the Hadoop Ecosystem

Apache Hadoop Overview

Data Ingestion and Storage

Data locality

Data Analysis and Exploration

Other Ecosystem Tools

Hadoop Ecosystem Installation

Ubuntu 14.04 LTS Installation through VMware Player

Installing Hadoop 2.7.1 on Ubuntu 14.04 LTS (Single-Node Cluster)

Apache Spark Installation

Jdk -8 Installation

Scala Installation

SBT Installation

Apache Hadoop File Storage

Why we need HDFS

Apache Hadoop Cluster Components

HDFS Architecture

Failures of HDFS 1.0

Reading and Writing Data in HDFS

Fault tolerance

Distributed Processing on an Apache Hadoop Cluster

Overview and Architecture of Map Reduce

Components of MapReduce

How MapReduce works

Flow and Difference of MapReduce Version

YARN Architecture

Apache Hive

Hive Installation on Ubuntu 14.04 With MySQL Database Metastore

Hive Overview and Architecture

Hive command execution in shell and HUE

Hive Data Loading methods

Hive Partition and Bucketing

External and Managed tables in Hive

File formats in Hive

Hive Joins

Serde in Hive

Apache Sqoop

Apache Sqoop Overview and Architecture

Apache Sqoop Import Examples

Apache Sqoop Export Examples

Sqoop Incremental load

Python Fundamentals

Introduction and Setting up of Python

Basic Programming Constructs

Functions in Python

Python Collections

Map Reduce operations on Python Collections

Setting up Data Sets for Basic I/O Operations

Basic I/O operations and processing data using Collections

Get revenue for given order id - as application

Pyspark Enviroment setup in different ways

Setup Environment - Locally

Setup Environment - using Cloudera QuickStart VM

Using Windows - Putty and WinSCP

Using Windows - Cygwin

HDFS Quick Preview

YARN Quick Preview

Setup Data Sets

Transform, Stage and Store – Pyspark


Introduction to Spark

Setup Spark on Windows

Quick overview about Spark documentation

Connecting to the environment

Initializing Spark job using pyspark

Create RDD from HDFS files

Create RDD from collection - using parallelize

ead data from different file formats - using sqlContext

Row level transformations - String Manipulation

Row level transformations using map

Row level transformations using flatMap

Filtering the data

Joining Data Sets - Introduction

Joining data sets - inner join

Joining data sets - outer join

Aggregations - Introduction

Aggregations - count and reduce - Get revenue for order id

Aggregations - reduce - Get order item with minimum subtotal for order id

Aggregations - countByKey - Get order count by status

Aggregations - understanding combiner

ggregations - groupByKey - Get revenue for each order id

groupByKey - Get order items sorted by order_item_subtotal for each order id

Aggregations - reduceByKey - Get revenue for each order id

Aggregations - aggregateByKey - Get revenue and count of items for each order id

Sorting - sortByKey - Sort data by product price

Sorting - sortByKey - Sort data by category id and then by price descending

Ranking - Introduction

Ranking - Global Ranking using sortByKey and take

Ranking - Global using takeOrdered or top

Ranking - By Key - Get top N products by price per category - Introduction

Ranking - By Key - Get top N products by price per category - Python collections

Ranking - By Key - Get top N products by price per category - using flatMap

Ranking - By Key - Get top N priced products - Introduction

Ranking - By Key - Get top N priced products - using Python collections API

Ranking - By Key - Get top N priced products - Create Function

Ranking - By Key - Get top N priced products - integrate with flatMap

Set Operations - Introduction

Set Operations - Prepare data

Set Operations - union and distinct

Set Operations - intersect and minus

Saving data into HDFS - text file format

Saving data into HDFS - text file format with compression

Saving data into HDFS using Data Frames - json

Apache Spark - Data Analysis - Spark SQL or HiveQL

Different interfaces to run SQL - Hive, Spark SQL

Create database and tables of text file format - orders and order_items

Create database and tables of ORC file format - orders and order_items

Running SQL/Hive Commands using pyspark

Functions - Getting Started

Functions - String Manipulation

Functions - Date Manipulation

Functions - Aggregate Functions in brief

Functions - case and nvl

Row level transformations

Joining data between multiple tables

Group by and aggregations

Sorting the data

Set operations - union and union all

Analytics functions - aggregations

Analytics functions - ranking

Windowing functions

Creating Data Frames and register as temp tables

Write Spark Application - Processing Data using Spark SQL

Write Spark Application - Saving Data Frame to Hive tables

Data Frame Operations

Data Frames and Pre-Defined Functions


Data Frames - Overview

Create Data Frames from Text Files

Create Data Frames from Hive Tables

Create Data Frames using JDBC

Data Frame Operations - Overview

Spark SQL - Overview

Overview of Functions to manipulate data in Data Frame fields or columns

Processing Data using Data Frames - Basic Transformations

Define Problem Statement - Get Daily Product Revenue

Selection or Projection of Data in Data Frames

Filtering Data from Data Frames

Perform Aggregations using Data Frames

Sorting Data in Data Frames

Development Life Cycle using Data Frames

Run applications using Spark Submit

Processing Data using Data Frames - Window Functions

Data Frame Operations - Window Functions - Overview

Data Frames - Window Functions APIs - Overview

Define Problem Statement - Get Top N Daily Products

Data Frame Operations - Creating Window Spec

Data Frame Operations - Performing Aggregations using sum, avg etc

Data Frame Operations - Time Series Functions such as Lead, Lag etc

Data Frame Operations - Ranking Functions - rank, dense_rank, row_number, etc

Running Pyspark Scripts

Writing a Spark Application

Building and Running an Application

Application Deployment Mode

The Spark Application Web UI

Configuring Application Properties

Apache Flume

Introduction to Flume & features

Flume topology & core concepts

Flume Agents: Sources, Channels and Sinks

Property file parameters logic

Apache Kafka

Apache Kafka Installation

Apache Kafka Overview and Architecture

Consumer and Producer

Deploying Kafka in real world business scenarios

Integration with Spark for Spark Streaming

Apache Zookeeper

Introduction to zookeeper concepts

Overview and Architecture of Zookeeper

Zookeeper principles & usage in Hadoop framework

Use of Zookeeper in Hbase and Kafka

Apache Oozie

Oozie Fundamentals

Oozie workflow creations

Concepts of Coordinates and Bundles