3D Generative Design for Non-Experts: Multiview Perceptual Similarity with Agent-Based Reinforcement Learning

Advances in additive manufacturing allow architectural elements to be fabricated with increasingly complex geometrical designs, however, corresponding 3D design software requires substantial knowledge and skill to operate, limiting adoption by non-experts or people with disabilities. Established non-expert approaches typically constrain geometry, topology, or character to a pre-established configuration, rather than aligning to figural and aesthetic characteristics defined by a user. A methodology is proposed that enables a user to develop multi-manifold designs from sketches or images in several 3d camera projections. An agent-based design approach responds to computer vision analysis (CVA) and Deep Reinforcement Learning (RL) to design outcomes with perceptual similarity to user input images evaluated by Structural Similarity Indexing (SSIM). Several CVA and RL ratios were explored in training models and tested on untrained images to evaluate their effectiveness. Results demonstrate a combination of CVA and RL motion behavior can produce meshes with perceptual similarity to image content.

Stuart-Smith, R., & Danahy, P. 3D Generative Design for Non-Experts: Multiview Perceptual Similarity with Agent-Based Reinforcement Learning. In P. Herrera, C. Dreifuss-Serrano, P. Gómez, & L. Arris-Calderon (Eds.), Critical Appropriations - Proceedings of the XXVI Conference of the Iberoamerican Society of Digital Graphics (SIGraDi 2022), 2022: 115–126.

The paper is accessible on the cumincad website here.

The Autonomous Manufacturing Lab (AML) explores the integration of robotic manufacturing and construction within architectural design. The AML aims to reduce the environmental and economic costs of design and production while enhancing their cultural and aesthetic impact. The lab's integrative methods for generative design and manufacturing are developed as semi-autonomous systems, exploring possibilities for artisanal-like craftsmanship to be scaled to large production volumes. This approach provides greater specificity and enhanced aesthetic character in design responses compared to traditional automated methods, whilst also reducing material use and waste within bespoke design and production processes. The AML's research enhances design and production capabilities across diverse manufacturing methods and materials by developing industrial and mobile robot fabrication techniques, multi-agent systems, AI/machine learning computational approaches, and applications of real-time sensor and computer vision technologies. This multidisciplinary research includes the development of Aerial Additive Manufacturing, recently published in Nature, which demonstrates the world’s first in-flight additive manufacturing by cooperating drones, representing a rapidly deployable form of collective robotic construction.

University of Pennsylvania Weitzman School of Design
The Weitzman School of Design prepares students to address complex sociocultural and environmental issues through thoughtful inquiry, creative expression, and innovation. As a diverse community of scholars and practitioners, we are committed to advancing the public good–both locally and globally–through art, design, planning, and preservation.

AML-PENN

AML-PENN

list-publications

AML-PENN

3D Generative Design for Non-Experts: Multiview Perceptual Similarity with Agent-Based Reinforcement Learning

AML-PENN