Skip to main content

Weekly Contributor Meeting Notes

(January 7, 2025 4:00 PM PST)

PR Bottleneck & Agent Strategy

Summary​

This was a weekly contributor meeting for the ai16z Discord, primarily focused on managing pull requests (PRs) and improving the development workflow for the Eliza project.

Project Updates and Transitions:

  • The Eliza project is transitioning to a new Discord server, ElizaOS. Some channels in the old server will be shut down or archived.

Challenges with Pull Requests:

  • The main issue discussed was the high volume of incoming PRs and the difficulty in merging them due to merge conflicts and the inability to directly modify PRs from forks.
  • Contributors are using a tool called PRM to help manage PRs, but its effectiveness and whether it's redundant with existing Git functionality is unclear.
  • The team is exploring the use of a "merge queue" to streamline the merging process and automatically notify contributors of conflicts.
  • Shopify's open-source tool "Ship It" was mentioned as a potential solution for mass merging, as they use it to merge hundreds of PRs daily.

Discussion on Agents and Automation:

  • There are currently two agents working on the repo, but they are manually controlled and not yet fully autonomous.
  • A contributor, AIFlow, described a multi-agent system they built to analyze issues and provide reports. This system is open source but needs cleanup before being shared.
  • The discussion touched on the need for a style guide to improve code consistency and reduce agent hallucinations.
  • Using tools like Code Rabbit or a PR Agent to automate PR review and tagging was proposed to help manage the workload.
  • Concerns were raised about the potential for security issues with the increasing number of plugins and the lack of proper logging and error handling in some code.

Other Topics:

  • The need for a standardized way to test plugins against the runtime was discussed.
  • CJ mentioned a complex PR they were working on related to dynamic imports and secret management. They proposed splitting it into smaller PRs for easier review.
  • A contributor brought up the idea of adding a linter to the release pipeline to enforce code style, but it was clarified that this hadn't been a major focus before.
  • The need for better documentation and onboarding for new contributors was also highlighted.

Action Items:

  • Investigate PRM and merge queue solutions.
  • Explore ways to automate test runs and handle secrets during testing.
  • Surface easy merge conflicts and work on editing outside branches.
  • Consider adding a linter to the release pipeline.
  • Continue working on smaller, more manageable PRs.

Overall Tone:

The meeting was collaborative and focused on problem-solving. Contributors were actively engaged in discussing the challenges and proposing solutions. There was a sense of urgency to improve the workflow and manage the growing complexity of the project.

Quotables​

  • Merging PRs is prioritized over fixing existing bugs

    "What we need, at least by my estimation is yes, we do need bugs fixed and issues resolved, but we need, we don't want to be adding features right now. Cause we have a zillion people trying to do that for us. What we need to be able to do is effectively review the PRs and get them merged and then also have the appropriate issue assigned to the appropriate PR."

  • Agents should be proactive in identifying and fixing issues

    "But with all these agents we have, we should be proactive. So we should run a batch on everything we develop."

  • The influx of new contributors is creating more work than it solves

    "So other people on this problem are going to give us more work. So in a moment like this better not add too much issues for request of new features."

  • The current testing process is inadequate and needs to be improved

    "It would be amazing to have something that tests everything against the runtime without us to go there and install and do."

  • A style guide is necessary to manage the growing complexity of the codebase

    "And in order to build a proper style guide for a repo that will have agents all over it, I think it is prudent to, or at least to me, it seems prudent to take a productized agent that is specifically built to solve the problem we have and then run that and see what breaks so that we are ready for once the other agents or we know what we need to build to help us go faster."