Hadoop

This project includes key pieces used by the big data ecosystem, such as HDFS and the Yarn scheduler. It also contains the famous Hadoop MapReduce which started the Big Data era.

Logo

../_images/apache_hadoop-small.png

Website

http://hadoop.apache.org/

Repository

https://github.com/apache/hadoop

Byline

The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models.

License

Apache 2.0

Project age

12 years

Backers

Apache (Governed by), Yahoo (Creator)

Size score (1 to 10, higher is better)

8.5

Trend score (1 to 10, higher is better)

7.5

Education Resources

URL

Resource Type

Description

https://hadoop.apache.org/docs/current/

Documentation

Official project documentation.

Git Commit Statistics

Statistics computed using Git data through May 31, 2021.

Statistic

Lifetime

Last 12 Months

Commits

37,996

13,021

Lines committed

19,745,498

5,920,015

Unique committers

586

180

Core committers

26

12

../_images/apache_hadoop-monthly-commits.png

Similar Projects

Project

Size Score

Trend Score

Byline

Dask

6.0

6.25

Parallel computing with task scheduling.

HPCC

5.5

7.0

HPCC Systems (High Performance Computing Cluster) is an open source, massive parallel-processing computing platform for big data processing and analytics.

Mars

6.75

6.25

Mars is a tensor-based unified framework for large-scale data computation which scales Numpy, Pandas and Scikit-learn.

Ray

8.0

9.0

An open source framework that provides a simple, universal API for building distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning library.

Spark

9.5

6.75

A unified analytics engine for large-scale data processing.