Using AI Agents in Github Actions to Automate your Github Flow
How to effectively leverage the power of large language models (LLMs) in our daily works as software engineers has become an increasingly important question. Traditional chatbots like ChatGPT can miss crucial contextual cues, while agent systems like ChatDev and AutoGen require lots of manual effort and a deep understanding of prompt engineering. Moreover, fully autonomous systems, such as Devin AI, have yet to show their effectiveness in real-world applications.
The Problem
An answer I got on Reddit recently sums it up perfectly:
A lot of devs don’t really believe that the output of these models is worth the time fighting it to do something useful.
While everyone agrees that LLMs have can be a powerful tool, using them effectively in their daily work has yet to become a reality for a lot of developers:
Trust is a big issue. LLM-based tools often produce results that are incorrect or of low quality
A lack of control over the behavior of these tools makes them unpredictable
Knowledge in prompt engineering and LLM best practices is often required to achieve good results, providing a barrier of entry for most developers today
What Developers Seek in AI-Assisted Solutions
As a developer, I seek an AI-assisted solution that embodies the following qualities:
Easy: I shouldn’t have to be an AI expert to use it
Flexible: Easily integrates with existing workflows and tools
Predictable: I need to feel confident that it produces the output I expect based on the input I provide
Intuitive: Let’s replace complex scripts and configuration files with intuitive, conversational interfaces.
Customizable: Adaptable to project-specific needs and workflow nuances.
AI Agents to the Rescue
In very simple terms, an AI agent is an LLM with special instructions and the ability to use tools like opening files, searching the internet or calling APIs. What if there was an easy way for you to outsource small parts of the work you do every day to an agent, with full control over its behavior and the quality and format of its output?
Introducing Smart Workflows
To address this need, I introduce Smart Workflows — a groundbreaking suite of open-source, agent-driven GitHub Action workflows designed to supercharge your development process:
Smart Workflows are built on PR Pilot, a platform that allows you to use AI agents in Github Actions.
Example: Enforcing Contribution Guidelines
As Klint Finley points out in his ReadME article on how open source maintainers keep themselves and their contributors happy:
although you might be tired of answering the same questions over and over, new users and contributors won't know that
It’s very common for open source projects to have contribution guidelines. However, making sure that contributors actually follow them is a significant challenge for maintainers. It can be a tedious and manual task that involves:
Noticing, reading and understanding new PRs / issues
Making sure they contain the necessary information and are formatted correctly
Enforcing the guidelines with a friendly, welcoming tone to encourage further engagement and community growth
This requires careful language, time and effort which most of us would rather spend writing code. Smart Workflows provide a simple and powerful approach to automate this. Let’s look at the following example: 📐 Enforce Contribution Guidelines
name: "📐 Enforce Contribution Guidelines"
# Triggers the workflow when a new pull request is created
on:
pull_request:
types: [opened]
jobs:
enforce-guidelines:
runs-on: ubuntu-latest
steps:
- name: Check PR Against Contribution Guidelines
uses: PR-Pilot-AI/smart-actions/quick-task@v1
with:
api-key: ${{ secrets.PR_PILOT_API_KEY }}
agent-instructions: |
1. Read `CONTRIBUTING.md` to understand the contribution guidelines.
2. Read the PR with ID `${{ github.event.pull_request.number }}` to understand its content and intent.
3. If the PR follows the contribution guidelines, do nothing.
4. Otherwise, comment on the PR explaining which parts of the contribution guidelines were not followed.
Agent-Driven Github Action
This flow chart visualizes how this Github Action uses an AI agent as part of the automation:
The workflow is triggered when a new PR is opened
The agent is instructed to read the project’s contribution guidelines
The quick-task step is used to instruct the AI agent to read the PR and check if the guidelines are followed
If not, it will comment with an appropriate response
And just like that, your life as an open source maintainer has become a whole lot easier:
A tedious, manual part of your daily work now runs on autopilot
You have full control over the agent’s actions and output
Your contributors get immediate feedback on new PRs
But it doesn’t stop here - you can extend and customize this workflow in creative ways:
Add more steps to integrate it with other workflows, systems and platforms
Change the agent instructions to:
Ensure the agent provides answers tailored to your domain
Search the internet based on the PR content
Read/update/process files
… and so much more
The example demonstrates how the use of AI agents can supercharge Github Actions to create AI-driven workflows that are easy to use, highly customizable and incredibly powerful.
Automations
Other automated workflows I’ve built so far include:
Manual Workflows as Tools
There is a special group of workflows that you can only run manually, which I call Tools:
One-Click Installation from Registry
🔄 Import Workflow plays an extra special role in that it allows you to import any Smart Workflow from our registry with the click of a button:
Enter the workflow ID, hit “Run workflow”. Your workflow will come as a pull request, including tips for customization
Key Features of Smart Workflows
Plug-and-Play: These workflows are ready to use right out of the box. Simply copy a YAML file, and you're set to go.
Modular: Smart Workflows can be seamlessly combined with other GitHub actions to create robust and powerful development pipelines.
Fully Customizable: Tailor your workflows using natural language instructions without diving deep into code.
LLM Best Practices: With best practices already integrated, you can focus on what truly matters - writing code.
Getting Started
Begin with the Smart Project Template to instantly set up a ready-to-go project. Choose from our ever-expanding collection of curated Smart Workflows to find the tools that best suit your project's needs or go one level deeper and write your own workflows using our Smart Actions and Python SDK.
Smart Workflows are not just about enhancing productivity; they're about transforming the software developer’s experience, making LLMs more accessible, less tedious, and more fun. Start integrating Smart Workflows today and experience a significant boost in your development efficiency and creativity.
