Luigi

Luigi is for batch jobs, which solves the complicated workflow and dependcy management in Hadoop world.

Logo

../_images/spotify_luigi-small.png

Website

N/A

Repository

https://github.com/spotify/luigi

Byline

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

License

Apache 2.0

Project age

10 years 4 months

Backers

Spotify (Creator and maintainer)

Lastest News (2022-06-20)

3.1.0 Over forty enhancements and bug fixes in the 3.1 release. more

Size score (1 to 10, higher is better)

5.0

Trend score (1 to 10, higher is better)

3.25

Education Resources

URL

Resource Type

Description

https://luigi.readthedocs.io/en/stable/

Documentation

Official project documentation.

Git Commit Statistics

Statistics computed using Git data through March 31, 2022.

Statistic

Lifetime

Last 12 Months

Commits

19,045

789

Lines committed

1,794,158

49,648

Unique committers

616

35

Core committers

19

8

../_images/spotify_luigi-monthly-commits.png

Similar Projects

Project

Size Score

Trend Score

Byline

Airflow

7.75

8.75

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Cromwell

5.25

5.5

Scientific workflow engine designed for simplicity & scalability. Trivially transition between one off use cases to massive scale production environments

Snakemake

6.75

6.25

The Snakemake workflow management system is a tool to create reproducible and scalable data analyses