Luigi

Luigi is for batch jobs, which solves the complicated workflow and dependcy management in Hadoop world.

Logo

../_images/spotify_luigi-small.png

Website

N/A

Repository

https://github.com/spotify/luigi

Byline

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

License

Apache 2.0

Project age

11 years

Backers

Spotify (Creator and maintainer)

Lastest News (2022-06-20)

3.1.0 Over forty enhancements and bug fixes in the 3.1 release. more

Size score (1 to 10, higher is better)

5.0

Trend score (1 to 10, higher is better)

3.75

Education Resources

URL

Resource Type

Description

https://luigi.readthedocs.io/en/stable/

Documentation

Official project documentation.

Git Commit Statistics

Statistics computed using Git data through November 30, 2022.

Statistic

Lifetime

Last 12 Months

Commits

25,105

399

Lines committed

2,343,577

12,347

Unique committers

627

32

Core committers

19

14

../_images/spotify_luigi-monthly-commits.png

Similar Projects

Project

Size Score

Trend Score

Byline

Airflow

8.25

8.5

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Cromwell

5.25

4.25

Scientific workflow engine designed for simplicity & scalability. Trivially transition between one off use cases to massive scale production environments

Snakemake

7.5

6.5

The Snakemake workflow management system is a tool to create reproducible and scalable data analyses