Skip to main content

@elizaos/plugin-3d-generation

A plugin for generating 3D models using the FAL.ai API within the ElizaOS ecosystem.

Description​

The 3D Generation plugin enables AI-powered creation of 3D models through FAL.ai's services. It provides functionality to generate 3D models from text descriptions and save them locally.

Installation​

bun install @elizaos/plugin-3d-generation

Configuration​

The plugin requires the following environment variable or runtime setting to be set:

FAL_API_KEY=<Your FAL.ai API key>

Usage​

Basic Integration​

import { ThreeDGenerationPlugin } from '@elizaos/plugin-3d-generation';

Model Generation Examples​

// The plugin responds to natural language commands like:

'Generate a 3D object of a cat playing piano';
'Create a 3D object of an anime character Goku';
'Make a 3D model of [your description]';

API Reference​

Actions​

GENERATE_3D​

Generates 3D models based on text descriptions.

Aliases:

  • 3D_GENERATION
  • 3D_GEN
  • CREATE_3D
  • MAKE_3D
  • TEXT23D
  • TEXT_TO_3D
  • 3D_CREATE
  • 3D_MAKE

Default Configuration:

{
    geometry_file_format: "glb",  // Available: glb, usdz, fbx, obj, stl
    material: "PBR",             // Available: PBR, Shaded
    quality: "medium",           // Available: extra-low, low, medium, high
    tier: "Regular"              // Available: Regular, Sketch
}

Common Issues & Troubleshooting​

  1. Generation Failures

    • Verify FAL API key is correctly set
    • Ensure prompt is descriptive (minimum 3 characters)
    • Check network connectivity to FAL.ai services

  2. Storage Issues

    • Verify write permissions to content_cache directory
    • Ensure sufficient disk space
    • Check if content_cache directory exists

Security Best Practices​

  1. API Key Management
    • Store FAL API key securely using runtime settings or environment variables
    • Never commit API keys to version control
    • Monitor API usage

Development Guide​

Setting Up Development Environment​

  1. Clone the repository
  2. Install dependencies:
bun install
  1. Build the plugin:
bun run build
  1. Run the plugin:
bun run dev

Future Enhancements​

  1. Advanced Generation Features

    • Multi-object scene generation
    • Texture customization options
    • Animation support
    • Material property controls
    • Advanced lighting systems
    • Physics-based rendering

  2. Model Optimization

    • Automatic mesh simplification
    • LOD (Level of Detail) generation
    • Texture compression
    • File size optimization
    • Performance profiling
    • Mobile-friendly exports

  3. Format Support

    • Additional file format exports
    • Custom format converters
    • Batch format conversion
    • Format-specific optimizations
    • Metadata preservation
    • Version control integration

  4. AI Improvements

    • Enhanced prompt understanding
    • Style transfer capabilities
    • Real-time generation
    • Multi-model support
    • Quality improvements
    • Consistency controls

  5. Scene Management

    • Scene composition tools
    • Environment management
    • Asset library integration
    • Scene presets
    • Batch processing
    • Scene version control

  6. Developer Tools

    • API expansion
    • Testing framework
    • Documentation generator
    • Debug visualization
    • Performance monitoring
    • Integration templates

  7. Rendering Features

    • Real-time preview
    • Custom shader support
    • Post-processing effects
    • Render queue management
    • Batch rendering
    • Cloud rendering options

  8. Collaboration Features

    • Asset sharing
    • Version control
    • Team workspace
    • Review system
    • Access control
    • Change tracking

We welcome community feedback and contributions to help prioritize these enhancements.

Contributing​

Contributions are welcome! Please see the CONTRIBUTING.md file for more information.

Credits​

This plugin integrates with and builds upon several key technologies:

  • FAL.ai: AI model deployment platform
  • Three.js: 3D graphics library
  • glTF: 3D file format standard
  • USD: Universal Scene Description
  • Blender: 3D creation suite

Special thanks to:

  • The FAL.ai team for AI infrastructure
  • The Three.js development community
  • The Khronos Group for glTF standards
  • The Pixar USD team
  • The Blender Foundation
  • The Eliza community for their contributions and feedback

For more information about 3D generation capabilities:

License​

This plugin is part of the Eliza project. See the main project repository for license information.