Build, Automate, and Scale Intelligent Workflows with Kwargify
The Python framework designed for developers to effortlessly create, manage, and version complex data pipelines and AI-driven applications.
Tired of Tangled Scripts and Brittle AI Pipelines?
Building robust data processing pipelines, especially those integrating AI, can quickly become a maze of custom scripts, manual interventions, and versioning nightmares. Debugging is a headache, resuming failed jobs is a chore, and scaling feels like rebuilding from scratch every time.
Kwargify Core: Your Blueprint for Intelligent Automation
Kwargify Core brings clarity and power to your workflow development. Define your processes as modular, interconnected blocks, manage them with a powerful CLI, and let Kwargify handle the complexities of execution, logging, versioning, and even AI integration. Focus on your logic, not the plumbing.
Unlock Superpowers for Your Python Workflows
Intuitive DAG Workflows
Visually design complex processes as Directed Acyclic Graphs. Define dependencies and let Kwargify orchestrate execution flawlessly.
Seamless AI Integration
Drop in pre-built blocks for AI-powered text processing, data extraction, and more. Bring intelligence to your pipelines with ease using models like GPT-4o-mini.
Powerful CLI Control
Run, validate, register, and manage your workflows directly from the command line. Full control at your fingertips for automation and scripting.
Robust Workflow Registry
Version your workflows like code. Register, track changes, and reliably deploy specific versions with our SQLite-backed registry.
Smart Resume & Retry
Never lose work. Automatically retry failed tasks with configurable backoffs and resume interrupted workflows exactly where they left off, saving time and resources.
Comprehensive Logging
Gain deep insights into every run. Detailed SQLite logging captures inputs, outputs, errors, and performance for easy debugging, auditing, and monitoring.
Effortless by Design. Powerful in Practice.
Workflows – Your Master Plan
A Workflow in Kwargify is your entire process, defined as a graph of connected blocks. It manages execution order, handles retries (with default_max_retries), and logs everything.
# Simplified Workflow Example
from kwargify_core.core import Workflow
workflow = Workflow(name="MyDataPipeline")
# ... add blocks and connections ...
workflow.run()Blocks – Your Building Units
Blocks are the individual tasks – reading a file, calling an AI, writing output. They're configurable (via config dict), reusable, and know their inputs and outputs.
# Simplified Block Example
from kwargify_core.blocks import ReadFileBlock, AIProcessorBlock
reader = ReadFileBlock(config={"path": "data.csv"})
processor = AIProcessorBlock(config={"model": "gpt-4o-mini"})
processor.input_map = {"content": (reader, "content")}From Tedious Tasks to Automated Triumphs
Automate Document Processing & Analysis
Extract insights from contracts, generate reports, and classify documents automatically, saving hours of manual work.
Build Robust Data ETL Pipelines
Reliably ingest, transform, and load data from various sources with versioned and resumable pipelines.
Develop AI-Powered Applications
Seamlessly integrate language models for summarization, Q&A, content generation, and custom AI tasks within structured workflows.
Streamline Repetitive Tasks
Automate any multi-step Python process with clarity, control, and built-in resilience.
Key Benefits
Accelerate Development: Build complex pipelines faster.
Enhance Reliability: Minimize failures with smart retries & resume.
Improve Manageability: Version, track, and debug with ease.
Promote Reusability: Create libraries of blocks and workflows.
Scale with Confidence: Build for today, ready for tomorrow.
Command Your Workflows with Precision
The Kwargify CLI is your central hub for interacting with your workflows. No complex GUIs, just powerful, scriptable commands designed for developer efficiency.
# Initialize your project
kwargify init
# Run your logic
kwargify run path/to/your_workflow.py
# Version and save your masterpiece
kwargify register path/to/your_workflow.py
# See all your registered workflows
kwargify list
# Ensure perfection before you run
kwargify validate path/to/your_workflow.py
# Visualize the flow
kwargify show path/to/your_workflow.py --diagramReady-to-Use Power: Our Built-in Block Library
Get a head start with our growing library of pre-built blocks for common tasks. Easily extend Kwargify by creating your own custom blocks.
File Operations
Handle file reading and writing operations with ease.
ReadFileBlockWriteFileBlockAI Processing
Process text and extract structured data using AI models.
AIProcessorBlockAIExtractorBlockTemplating
Generate documents using customizable templates.
DocumentTemplateBlockData Transformation
Transform data between different formats seamlessly.
JsonToStringBlockReady to Kwargify Your Workflows?
# Using uv (Recommended)
uv add kwargify-core
# Or with Poetry
poetry add kwargify-core
# Or with pip
pip install kwargify-coreNext Steps
- Initialize your first project:
kwargify init - Explore our Example Workflows (e.g., Contract Analysis)
- Dive into the Full Documentation