Apache_Airflow
This repo implements the idea of programmatically authoring, scheduling, and monitoring data pipelines as code, leveraging Apache Airflow's extensible Python framework to connect with diverse technologies and providing a web interface for workflow management, deployable from local setups to large-scale distributed environments.
















UV Commands:
uv run filename.py: To run a Python script.
pip install uv: To install uv from PyPI.
uv --version: Check the installed UV version.
uv venv: Create a new virtual environment.
.venv\Scripts\activate: Activate the virtual environment (Windows).
uv venv <name> : Create a virtual environment with name (uv venv fastapi)
<name>\Scripts\activate : Activate the virtual environment with name (fastapi\Scripts\activate)
uv add <package_name>[==<version>]: Adds dependency to your project's pyproject.toml and installs it into the active virtual environment(e.g., uv add flask==2.1.0).
uv add -r requirements.txt: Add all dependencies from requirements.txt.
uv remove <package_name>: Removes a dependency from pyproject.toml and uninstalls it
deactivate: Deactivate the virtual environment.
uv sync: to manually update the environment
uv lock: to lock the environment & updates the lock file
Creating a new project
- You can create a new Python project using the uv init command:
uv init hello-world
cd hello-world
- Alternatively, you can initialize a project in the working directory:
mkdir hello-world
cd hello-world
uv init
uv will create the following files:
.
├── .python-version
├── README.md
├── main.py
└── pyproject.toml
The main.py file contains a simple "Hello world" program. Try it out with uv run:
uv run main.py
Project structure
A project consists of a few important parts that work together and allow uv to manage your project. In addition to the files created by uv init, uv will create a virtual environment and uv.lock file in the root of your project the first time you run a project command, i.e., uv run, uv sync, or uv lock.
A complete listing would look like:
.
├── .venv
│ ├── bin
│ ├── lib
│ └── pyvenv.cfg
├── .python-version
├── README.md
├── main.py
├── pyproject.toml
└── uv.lock
The pyproject.toml contains metadata about your project:
[project]
name = "hello-world"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
dependencies = []
You'll use this file to specify dependencies, as well as details about the project such as its description or license. You can edit this file manually, or use commands like uv add and uv remove to manage your project from the terminal.
Docs:
Problem
while running git add. getting this error:
warning: could not open directory 'logs/scheduler/latest/': Function not implemented
Solution
logs/
- Save the file and then try
git add . again.