Apache Airflow
Open-source platform for authoring, scheduling and monitoring workflows.
Apache Airflow is an open-source platform for programmatically authoring, scheduling and monitoring data pipelines and workflows. Workflows are defined in Python as directed acyclic graphs (DAGs), where each node describes a task and each edge a dependency.
Airflow runs tasks in the right order, schedules them on a timed basis and provides a web interface for tracking the status of runs and tracing errors. It is often used to orchestrate ETL processes, data preparation and machine learning pipelines.
What Provimedia uses it for
Provimedia orchestrates data and ML pipelines with Apache Airflow, for example to automate recurring data preparation in AI development.
Want to use Apache Airflow for your project?
In a free conversation, let's find out how we can implement this for you.
More from AI & machine learning
Anthropic Claude
Family of large language models from the AI company Anthropic.
Learn moreCUDA
NVIDIA's parallel computing platform for using GPUs for general-purpose computation.
Learn moreElevenLabs
AI platform for synthesizing natural-sounding speech from text.
Learn moreHugging Face
Platform and libraries for pretrained AI models and datasets.
Learn moreKeras
High-level API for building and training neural networks.
Learn moreLangChain
Open-source framework for developing applications based on large language models.
Learn moreMLflow
Open-source platform for managing the lifecycle of machine learning projects.
Learn moreNLTK
Established Python library for natural language processing.
Learn more