Skip to content

Building Python Projects with Buck2: A Complete Guide to Virtual Environments and Dependency Management

Building Python Projects with Buck2: A Complete Guide to Virtual Environments and Dependency Management

Section titled “Building Python Projects with Buck2: A Complete Guide to Virtual Environments and Dependency Management”

A comprehensive walkthrough of setting up a modern Python project using Buck2, UV, virtual environments, and wheels.

  1. Introduction
  2. Prerequisites
  3. Project Setup from Scratch
  4. Creating Your First Python Library
  5. Managing Dependencies with UV
  6. Testing Best Practices
  7. Building with Buck2
  8. Dependency Groups and Wheels
  9. Virtual Environments
  10. Troubleshooting

Buck2 is a powerful build system created by Meta that makes building large-scale projects faster and more reliable. When combined with Python, UV (a fast Python package manager), and proper virtual environment management, you get a modern, efficient development workflow.

This guide walks through building a complete Python project with:

  • Buck2 - Fast, reliable build system
  • UV - Lightning-fast Python package manager
  • Virtual environments - Isolated Python environments
  • Wheels - Pre-built Python packages
  • Test organization - Proper separation of concerns
  • Dependency management - Multiple dependency groups

By the end, you’ll have a production-ready Python project structure that scales.

Before starting, ensure you have:

  • Python 3.8+ installed
  • Git for version control
  • 10-15 minutes to follow along
Terminal window
# Install Buck2
# macOS/Linux
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
cargo install buck2
# Or download pre-built binaries
# See: https://buck2.build/docs/getting_started/
# Install UV (Python package manager)
# macOS/Linux
curl -LsSf https://astral.sh/uv/install.sh | sh
# Windows (PowerShell)
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
# Verify installations
buck2 --version
uv --version
python --version

Let’s create a complete Python project with proper structure, testing, and dependency management.

Terminal window
# Create project directory
mkdir buck2-python-project
cd buck2-python-project
# Initialize git
git init
git config user.email "you@example.com"
git config user.name "Your Name"
# Create main directories
mkdir -p lib
mkdir -p tests
mkdir -p toolchains
# Create package markers
touch lib/__init__.py
touch tests/__init__.py
touch toolchains/__init__.py
# Create PACKAGE files (Buck2 package markers)
touch PACKAGE
touch lib/PACKAGE
touch tests/PACKAGE
touch toolchains/PACKAGE
Terminal window
# Create .buckconfig
cat > .buckconfig << 'EOF'
[cells]
root = .
prelude = prelude
toolchains = toolchains
none = none
[cell_aliases]
config = prelude
ovr_config = prelude
fbcode = none
fbsource = none
fbcode_macros = none
buck = none
[external_cells]
prelude = bundled
[parser]
target_platform_detector_spec = target:root//...->prelude//platforms:default \
target:prelude//...->prelude//platforms:default \
target:toolchains//...->prelude//platforms:default
[build]
execution_platforms = prelude//platforms:default
[project]
ignore = .git
EOF
# Create .buckroot marker
touch .buckroot
# Create .gitignore
cat > .gitignore << 'EOF'
/buck-out
.venv/
*.pyc
__pycache__/
*.egg-info/
dist/
build/
.pytest_cache/
.mypy_cache/
*.lock
EOF
Terminal window
cat > pyproject.toml << 'EOF'
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[project]
name = "buck2-example"
version = "0.1.0"
description = "A Buck2 Python project with diverse dependencies"
readme = "README.md"
requires-python = ">=3.8"
license = {text = "MIT"}
authors = [
{name = "Your Name", email = "you@example.com"}
]
# Core runtime dependencies
dependencies = [
"requests>=2.28.0", # HTTP client library
"click>=8.0", # CLI framework
"pydantic>=2.0", # Data validation
]
# Optional dependency groups
[project.optional-dependencies]
dev = [
"pytest>=7.0", # Testing framework
"pytest-cov>=4.0", # Code coverage
"black>=23.0", # Code formatter
"ruff>=0.1.0", # Linter
"mypy>=1.0", # Type checker
]
docs = [
"sphinx>=6.0", # Documentation generator
"sphinx-rtd-theme>=1.0", # ReadTheDocs theme
]
api = [
"fastapi>=0.95.0", # Web framework
"uvicorn>=0.20.0", # ASGI server
"sqlalchemy>=2.0", # ORM
]
data = [
"pandas>=2.0", # Data manipulation
"numpy>=1.24", # Numerical computing
]
# Tool configurations
[tool.uv]
python-version = "3.11"
venv-dir = ".venv"
[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = "test_*.py"
[tool.black]
line-length = 100
target-version = ["py311"]
[tool.ruff]
line-length = 100
select = ["E", "F", "W", "I"]
[tool.mypy]
python_version = "3.11"
warn_return_any = true
disallow_untyped_defs = false
EOF
Terminal window
cat > BUCK << 'EOF'
genrule(
name = "hello_world",
out = "out.txt",
cmd = "echo BUILT BY BUCK2> $OUT",
)
python_library(
name = "wheel_example",
srcs = ["wheel_example.py"],
base_module = "",
)
EOF
Terminal window
cat > toolchains/BUCK << 'EOF'
load("@prelude//toolchains:demo.bzl", "system_demo_toolchains")
# Default toolchains for demo/prototyping
system_demo_toolchains()
EOF

For macOS/Linux:

cat > setup_venv.sh << 'EOF'
#!/bin/bash
set -e
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
echo -e "${GREEN}Buck2 Python Environment Setup${NC}\n"
if ! command -v uv &> /dev/null; then
echo -e "${RED}Error: uv is not installed${NC}"
echo "Install it with: curl -LsSf https://astral.sh/uv/install.sh | sh"
exit 1
fi
echo -e "${GREEN}✓ Found UV$(uv --version)${NC}"
if [ -d ".venv" ]; then
echo -e "${YELLOW}Removing existing .venv directory...${NC}"
rm -rf .venv
fi
echo -e "\n${GREEN}Creating virtual environment...${NC}"
uv venv --python 3.11
echo -e "${GREEN}Activating virtual environment...${NC}"
source .venv/bin/activate
echo -e "\n${GREEN}Installing base dependencies...${NC}"
uv pip install -e .
echo -e "\n${YELLOW}Optional dependency groups:${NC}"
echo "1) dev - Development tools (pytest, black, ruff, mypy)"
echo "2) docs - Documentation (sphinx, sphinx-rtd-theme)"
echo "3) api - Web API (fastapi, uvicorn, sqlalchemy)"
echo "4) data - Data science (pandas, numpy)"
echo "5) all - Install all groups"
echo "6) none - Skip optional dependencies"
read -p "Select groups (1-6): " choice
case $choice in
1) uv pip install -e ".[dev]" ;;
2) uv pip install -e ".[docs]" ;;
3) uv pip install -e ".[api]" ;;
4) uv pip install -e ".[data]" ;;
5) uv pip install -e ".[dev,docs,api,data]" ;;
6) echo -e "${YELLOW}Skipping optional dependencies${NC}" ;;
*) echo -e "${RED}Invalid choice${NC}"; exit 1 ;;
esac
echo -e "\n${GREEN}Creating lock file...${NC}"
uv pip freeze > uv-lock.txt
echo -e "\n${GREEN}✓ Setup complete!${NC}"
echo -e "${YELLOW}To activate environment in future:${NC}"
echo " source .venv/bin/activate"
echo ""
echo -e "${GREEN}Ready to use! 🚀${NC}"
EOF
chmod +x setup_venv.sh

For Windows:

Terminal window
cat > setup_venv.bat << 'EOF'
@echo off
setlocal enabledelayedexpansion
echo Buck2 Python Environment Setup
echo.
where uv >nul 2>nul
if %ERRORLEVEL% NEQ 0 (
echo Error: uv is not installed
echo Install it with: powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
exit /b 1
)
echo Found UV - Creating virtual environment...
if exist ".venv" (
echo Removing existing .venv directory...
rmdir /s /q .venv
)
echo.
echo Creating virtual environment...
call uv venv --python 3.11
echo Activating virtual environment...
call .venv\Scripts\activate.bat
echo.
echo Installing base dependencies...
call uv pip install -e .
echo.
echo Optional dependency groups:
echo 1) dev - Development tools
echo 2) docs - Documentation
echo 3) api - Web API
echo 4) data - Data science
echo 5) all - Install all groups
echo 6) none - Skip optional
set /p choice="Select groups (1-6): "
if "%choice%"=="1" (
call uv pip install -e ".[dev]"
) else if "%choice%"=="5" (
call uv pip install -e ".[dev,docs,api,data]"
)
echo.
echo Creating lock file...
call uv pip freeze > uv-lock.txt
echo.
echo Setup complete!
echo.
echo To activate environment in future:
echo .venv\Scripts\activate.bat
EOF
Terminal window
cat > lib/utils.py << 'EOF'
"""Utility functions for the project."""
def add(a: int, b: int) -> int:
"""Add two numbers.
Args:
a: First number
b: Second number
Returns:
Sum of a and b
"""
return a + b
def multiply(a: int, b: int) -> int:
"""Multiply two numbers.
Args:
a: First number
b: Second number
Returns:
Product of a and b
"""
return a * b
EOF
Terminal window
cat > lib/http_utils.py << 'EOF'
"""HTTP utilities using external library dependencies."""
def fetch_status(url: str, timeout: int = 5):
"""Fetch HTTP status code from a URL.
Args:
url: URL to fetch
timeout: Request timeout in seconds
Returns:
HTTP status code or None if requests not available
"""
try:
import requests
response = requests.get(url, timeout=timeout)
return response.status_code
except ImportError:
print("requests library not installed")
return None
except Exception as e:
print(f"Error fetching {url}: {e}")
return None
def fetch_json(url: str, timeout: int = 5):
"""Fetch and parse JSON from a URL.
Args:
url: URL to fetch JSON from
timeout: Request timeout in seconds
Returns:
Parsed JSON dict or None if error
"""
try:
import requests
response = requests.get(url, timeout=timeout)
response.raise_for_status()
return response.json()
except ImportError:
print("requests library not installed")
return None
except Exception as e:
print(f"Error fetching JSON: {e}")
return None
EOF
Terminal window
cat > lib/BUCK << 'EOF'
python_library(
name = "utils",
srcs = ["utils.py"],
visibility = ["//..."],
)
python_library(
name = "http_utils",
srcs = ["http_utils.py"],
visibility = ["//..."],
)
EOF

Dependencies are organized in pyproject.toml into groups:

Core dependencies - Always installed:

dependencies = [
"requests>=2.28.0", # HTTP client
"click>=8.0", # CLI framework
"pydantic>=2.0", # Data validation
]

Development group - For developers:

[project.optional-dependencies]
dev = [
"pytest>=7.0", # Testing
"black>=23.0", # Code formatting
"ruff>=0.1.0", # Linting
]

Other groups - For specific use cases:

api = ["fastapi>=0.95.0", "uvicorn>=0.20.0"]
data = ["pandas>=2.0", "numpy>=1.24"]
docs = ["sphinx>=6.0"]
Terminal window
# Create virtual environment
uv venv --python 3.11
# Activate it
source .venv/bin/activate # Linux/macOS
# or
.venv\Scripts\activate.bat # Windows
# Install base dependencies
uv pip install -e .
# Install with development tools
uv pip install -e ".[dev]"
# Install multiple groups
uv pip install -e ".[dev,api,data]"
# Install everything
uv pip install -e ".[dev,docs,api,data]"
# Install specific package
uv pip install requests
# Update package
uv pip install --upgrade requests
# Create reproducible lock file
uv pip freeze > uv-lock.txt
# Use lock file (for reproducibility)
uv pip install -r uv-lock.txt

A lock file pins exact versions of all dependencies:

Terminal window
# Generate lock file
uv pip freeze > uv-lock.txt
# View lock file structure
cat uv-lock.txt
# Output:
# requests==2.31.0
# certifi==2024.2.2
# charset-normalizer==3.3.2
# idna==3.6
# Commit to version control
git add uv-lock.txt
git commit -m "Add dependency lock file"
# Team members use it
uv pip install -r uv-lock.txt

Benefits:

  • ✅ Exact versions for all team members
  • ✅ Transitive dependencies pinned
  • ✅ No surprises from newer versions
  • ✅ Perfect for CI/CD pipelines
Terminal window
cat > tests/test_utils.py << 'EOF'
"""Unit tests for lib.utils module."""
import unittest
try:
from lib.utils import add, multiply
except ImportError:
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', 'lib'))
from utils import add, multiply
class TestUtils(unittest.TestCase):
"""Test cases for utility functions."""
def test_add(self):
"""Test the add function."""
self.assertEqual(add(2, 3), 5)
self.assertEqual(add(-1, 1), 0)
self.assertEqual(add(0, 0), 0)
def test_add_floats(self):
"""Test add with floating point numbers."""
self.assertAlmostEqual(add(1.5, 2.5), 4.0)
def test_multiply(self):
"""Test the multiply function."""
self.assertEqual(multiply(2, 3), 6)
self.assertEqual(multiply(-1, 5), -5)
self.assertEqual(multiply(0, 100), 0)
if __name__ == "__main__":
unittest.main()
EOF
Terminal window
cat > tests/test_http_utils.py << 'EOF'
"""Unit tests for lib.http_utils module."""
import unittest
try:
from lib.http_utils import fetch_status, fetch_json
except ImportError:
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', 'lib'))
from http_utils import fetch_status, fetch_json
class TestHttpUtils(unittest.TestCase):
"""Test cases for HTTP utility functions."""
def test_fetch_status_graceful_failure(self):
"""Test that fetch_status handles missing requests gracefully."""
result = fetch_status("https://example.com")
self.assertTrue(result is None or isinstance(result, int))
def test_fetch_json_graceful_failure(self):
"""Test that fetch_json handles missing requests gracefully."""
result = fetch_json("https://api.example.com/data")
self.assertTrue(result is None or isinstance(result, dict))
if __name__ == "__main__":
unittest.main()
EOF
Terminal window
cat > tests/BUCK << 'EOF'
python_test(
name = "test_utils",
srcs = ["test_utils.py"],
deps = ["//lib:utils"],
)
python_test(
name = "test_http_utils",
srcs = ["test_http_utils.py"],
deps = ["//lib:http_utils"],
)
EOF

DO:

  • Separate tests from library code
  • Use clear, descriptive test names
  • Test both success and failure cases
  • Organize tests in classes
  • Use explicit dependencies in BUCK files

DON’T:

  • Mix library and test code
  • Test implementation details
  • Ignore errors
  • Use circular dependencies
Terminal window
# Verify Buck2 is installed
buck2 --version
# List all build targets
buck2 uquery "//..."
# Build all targets
buck2 build //...
# Build specific target
buck2 build //:hello_world
buck2 build //lib:utils
# Build with verbose output
buck2 build //... -v
# View build artifacts
ls -la buck-out/v2/gen/
Terminal window
# Run all tests
buck2 test //...
# Run specific test
buck2 test //tests:test_utils
# Run with verbose output
buck2 test //... -v
# View test results
buck2 test //... --show-output
BUILD ID: abc123def456
[2024-02-18T10:30:45.123Z] Build ID: abc123def456
[2024-02-18T10:30:45.456Z] Cache hits: 75%
[2024-02-18T10:30:45.789Z] Commands: 10 (cached: 8, remote: 0, local: 2)
[2024-02-18T10:30:45.999Z] Network: Up: 50MB Down: 120MB
✓ Pass: root//tests:test_utils (0.8s)
✓ Pass: root//tests:test_http_utils (0.9s)
BUILD SUCCEEDED

Wheels are pre-built Python packages (.whl files):

package_name-1.0-py3-none-any.whl
│ │ │ │ │
│ │ │ │ └─ OS compatibility
│ │ │ └──── ABI compatibility
│ │ └─────── Python version
│ └────────── Version
└──────────────────── Package name

Advantages:

  • ✅ Faster installation (no build needed)
  • ✅ No compiler required
  • ✅ Reproducible builds
  • ✅ Easy to cache
# Using local wheels
http_file(
name = "requests_wheel",
url = "https://files.pythonhosted.org/.../requests-2.31.0-py3-none-any.whl",
sha256 = "d434c6e...",
)
python_library(
name = "http_utils",
srcs = ["http_utils.py"],
resources = [":requests_wheel"],
)
Terminal window
# Create wheels directory
mkdir wheels
cd wheels
# Download specific wheel
pip download requests==2.31.0
# Download all project dependencies
pip download -r ../requirements.txt
# List downloaded wheels
ls -lh
# Output:
# requests-2.31.0-py3-none-any.whl
# certifi-2024.2.2-py3-none-any.whl
# urllib3-2.1.0-py3-none-any.whl
# ...
Terminal window
# Install from wheel file
pip install wheels/requests-2.31.0-py3-none-any.whl
# Install all wheels in directory
pip install wheels/*
# Create requirements.txt from wheels
pip install --no-index --find-links=wheels requests

A virtual environment is an isolated Python installation:

System Python (global)
└── site-packages/
├── Package A (v1.0)
├── Package B (v2.0)
└── Package C (v1.5)
Project 1 Venv
└── site-packages/
├── Package A (v1.0)
├── Package B (v2.0)
└── Package C (v1.5)
Project 2 Venv
└── site-packages/
├── Package A (v1.2)
├── Package B (v3.0)
└── Package D (v1.0)

Benefits:

  • ✅ Isolate project dependencies
  • ✅ Avoid version conflicts
  • ✅ Easy cleanup (delete venv)
  • ✅ Reproducible environments
Terminal window
# Create venv with UV
uv venv --python 3.11
# Activate venv
source .venv/bin/activate # Linux/macOS
.venv\Scripts\activate.bat # Windows
# Check which Python is active
which python # Should show .venv path
python --version # Should show 3.11.x
# Install packages
uv pip install requests
# List installed packages
uv pip list
# Deactivate venv (return to system Python)
deactivate
# Delete venv
rm -rf .venv
Terminal window
# Add to .gitignore
echo ".venv/" >> .gitignore
echo "*.egg-info/" >> .gitignore
echo "dist/" >> .gitignore
echo "build/" >> .gitignore
# Commit changes
git add .gitignore
git commit -m "Add venv to gitignore"

Setup once:

Terminal window
uv venv --python 3.11
source .venv/bin/activate
uv pip install -e ".[dev]"
uv pip freeze > uv-lock.txt

Buck2 automatically:

  • ✓ Detects Python in active venv
  • ✓ Uses installed packages
  • ✓ Resolves from pyproject.toml
  • ✓ Builds and tests successfully
Terminal window
# 1. Create project
mkdir my-buck2-project
cd my-buck2-project
git init
# 2. Copy all files from this guide OR use setup script
./setup_venv.sh # macOS/Linux
# or
.\setup_venv.bat # Windows
# 3. Activate venv
source .venv/bin/activate
# 4. Verify everything
buck2 build //...
buck2 test //...
Terminal window
# 1. Start session
cd my-buck2-project
source .venv/bin/activate
# 2. Write code and tests
# ... edit files ...
# 3. Build and test
buck2 build //...
buck2 test //...
# 4. Format and lint (if dev group installed)
black lib/ tests/
ruff check lib/ tests/
mypy lib/ tests/
# 5. Commit changes
git add lib/ tests/ pyproject.toml
git commit -m "Add new feature"
# 6. End session
deactivate
Terminal window
# Team member clones repo
git clone <repo-url>
cd project
# Recreate exact environment
source .venv/bin/activate
uv pip install -r uv-lock.txt
# Same environment as original developer!
buck2 build //...
buck2 test //...

Solution:

Terminal window
# Ensure buck2 is installed
# macOS/Linux
curl -LsSf https://github.com/facebookincubator/buck2/releases/download/latest/buck2-x86_64-apple-darwin.zst | unzstd | install -m 755 /dev/stdin ~/.local/bin/buck2
# Add to PATH
export PATH="$HOME/.local/bin:$PATH"

Issue: “ModuleNotFoundError: No module named ‘requests’”

Section titled “Issue: “ModuleNotFoundError: No module named ‘requests’””

Solution:

Terminal window
# 1. Verify venv is activated
which python # Should show .venv path
# 2. Install package
uv pip install requests
# 3. Verify installation
uv pip list | grep requests
# 4. Try build again
buck2 build //...

Issue: “error: Error looking up configured node”

Section titled “Issue: “error: Error looking up configured node””

Solution:

Terminal window
# Ensure all PACKAGE files exist
touch PACKAGE
touch lib/PACKAGE
touch tests/PACKAGE
touch toolchains/PACKAGE
# Verify BUCK files exist
ls -la */BUCK
# Try build again
buck2 build //...

Solution:

Terminal window
# Check current Python
python --version
# Recreate venv with specific version
rm -rf .venv
uv venv --python 3.11
uv pip install -e ".[dev]"

Issue: “Test failures with missing dependencies”

Section titled “Issue: “Test failures with missing dependencies””

Solution:

Terminal window
# Check what's installed
uv pip list
# Install missing dependencies
uv pip install -e ".[dev,api,data]"
# Create new lock file
uv pip freeze > uv-lock.txt
# Run tests again
buck2 test //...

Solution:

Terminal window
# 1. Check which Python Buck2 is using
buck2 build //:hello_world -v 2>&1 | grep -i python
# 2. Ensure venv is activated
source .venv/bin/activate
which python
# 3. Verify Python path
echo $VIRTUAL_ENV
# 4. Try again
buck2 build //...
Your Local Machine
├── .venv/ (Virtual environment)
│ └── lib/python3.11/
│ └── site-packages/ (Installed wheels)
├── pyproject.toml (Dependency declaration)
├── uv-lock.txt (Pinned versions)
├── BUCK files (Build targets)
└── buck-out/ (Build artifacts)
Code Changes
uv pip install (manage dependencies)
buck2 build (compile)
buck2 test (verify)
uv pip freeze (update lock file)
git commit (version control)
PurposeCommand
Create venvuv venv --python 3.11
Activatesource .venv/bin/activate
Install alluv pip install -e ".[dev]"
Buildbuck2 build //...
Testbuck2 test //...
Formatblack lib/ tests/
Lintruff check lib/ tests/
Lockuv pip freeze > uv-lock.txt

Always:

  • Use virtual environments
  • Create lock files
  • Separate tests from libraries
  • Use explicit dependencies
  • Commit lock files to git

Never:

  • Use global Python for projects
  • Skip testing
  • Mix library and test code
  • Ignore dependency conflicts
  • Assume others have same setup

By following this guide, you now have a modern Python development setup with:

  1. Buck2 - Fast, scalable builds
  2. UV - Quick dependency resolution
  3. Virtual Environments - Isolated, reproducible Python environments
  4. Wheels - Pre-built, fast packages
  5. Proper Testing - Organized, maintainable tests
  6. Dependency Management - Multiple groups for different use cases

This setup scales from small projects to large, team-based development. The combination of Buck2 and UV provides speed and reliability, while virtual environments ensure reproducibility across team members and CI/CD pipelines.

Start with the basic setup, add your own libraries and tests, and expand dependency groups as your project grows. The foundation is solid, flexible, and production-ready.

Happy building! 🚀

Written: February 2024 Updated: February 2024 Version: 1.0