Program

Tutorial 1001: Diffusion Models for Visual Content Generation

Niloy J. Mitra, Daniel Cohen-Or, Minhyuk Sung, Chun-Hao Huang, Duygu Ceylan, Paul Guerrero

Diffusion models are now the state-of-the-art for producing images. These models have been trained on vast datasets and are increasingly repurposed for various image processing and conditional image generation tasks. We expect these models to be widely used in Computer Graphics and related research areas. Image generation has evolved into a rich promise of new possibilities, and in this tutorial, we will guide you through the intricacies of understanding and using diffusion models. This tutorial is targeted towards graphics researchers with an interest in image/video synthesis and manipulation. Attending the tutorial will enable participants to build a working knowledge of the core formulation, understand how to get started in this area, and study practical use cases to explore this new tool. Our goal is to get more researchers with expertise in computer graphics to start exploring the open challenges in this topic and explore innovative use cases in CG contexts in image synthesis and other media formats. From understanding the underlying principles to hands-on implementation, you’ll gain practical skills that bridge theory and application. Throughout the tutorial, we’ll explore techniques for generating lifelike textures, manipulating details, and achieving remarkable visual effects. By the end, you’ll have a solid foundation in utilizing diffusion models for image generation, ready to embark on your creative projects. Join us as we navigate the fascinating intersection of computer graphics and diffusion models, where pixels become canvases and algorithms transform into brushes.

Tutorial 1005: Next Generation 3D Face Models

Prashanth Chandran, Lingchen Yang

Having a compact, expressive and artist friendly way to represent and manipulate human faces has been of prime interest to the visual effects community for the past several decades as face models play a very important role in many face capture workflows. In this short course, we go over the evolution of 3D face models used to model and animate facial identity and expression in the computer graphics community, and discuss how the recent emergence of deep face models is transforming this landscape by enabling new artistic choices. In this first installment, the course will take the audience through the evolution of face models, starting with simple blendshape models introduced in the 1980s; that continue to be extremely popular today, to recent deep shape models that utilize neural networks to represent and manipulate face shapes in an artist friendly fashion. As the course is meant to be beginner friendly, the course will commence with a quick introduction to non-neural parametric shape models starting with linear blendshape and morphable models. We will then switch focus to deep shape models, particularly those that offer intuitive control to artists. We will discuss multiple variants of such deep face models that i) allow semantic control, ii) are agnostic to the underlying topology of the manipulated shape, iii) provide the ability to explicitly model a sequence of 3D shapes or animations, and iv) allow for the simulation of physical effects. Applications that will be discussed include face shape synthesis, identity and expression interpolation, rig generation, performance retargeting, animation synthesis and more.

Tutorial 1000: Predictive Modeling of Material Appearance: From the Drawing Board to Interdisciplinary Applications

Baranoski Gladimir

This tutorial addresses one of the fundamental and timely topics of computer graphics research, namely the predictive modeling of material appearance. Although this topic is deeply rooted in traditional areas like rendering and natural phenomena simulation, this tutorial is not limited to cover contents connected to these areas. It also closely looks into the scientific methodology employed in the development of predictive models of light and matter interactions. Given the widespread use of this methodology to find modeling solutions for problems within and outside computer graphics, its discussion from a “behind the scenes” perspective aims to underscore practical and far-reaching aspects of interdisciplinary research that are often overlooked in related publications. More specifically, this tutorial unveils constraints and pitfalls found in each of the key stages of the model development process, namely data collection, design and evaluation, and brings forward alternatives to tackle them effectively. Furthermore, besides being a central component of realistic image synthesis frameworks, predictive material appearance models have a scope of applications that can be extended far beyond the generation of believable images. For instance, they can be employed to accelerate the hypothesis generation and validation cycles of research across a wide range of fields, from biology and medicine to photonics and remote sensing, among others. These models can also be used to generate comprehensive in silico (computational) datasets to support the translation of knowledge advances in those fields to real-world applications (e.g., the noninvasive screening of medical conditions and the remote detection of environmental hazards). In fact, a number of them are already being used in physical and life sciences, notably to support investigations seeking to strengthen the current understanding about material appearance changes prompted by mechanisms which cannot be fully studied using standard “wet” experimental procedures. Accordingly, such interdisciplinary research initiatives are also discussed in this tutorial through selected case studies involving the use of predictive material appearance models to elucidate challenging scientific questions.

Tutorial 1004: Design and development of VR games for Cultural Heritage using Immersive Storytelling

Selma Rizvic, Bojan Mijatovic

In this tutorial we introduce the whole process of creating, designing and developing a serious VR game for cultural heritage using the concept of immersive storytelling. The use of serious games in education is allowing us to offer a different approach to learning. However, designing an application with gameplay parts as well as educational components in specific area such as cultural heritage can be challenging and different to many other methodologies in the creation of similar applications. The goal of the tutorial is to show different aspects of serious game creation and usage of our immersive storytelling methodology called hyper storytelling to help with the educational elements of the game. We will go through the creation of a story for the game; the creation of scenarios for the educational gameplay part; the filming of actors on green screen and filming of 360 videos; compositing of VR videos with actors and ambisonic sound; the creation of photogrammetry items; combining educational parts with gameplay and creating connection between them; application development and finalization of the product. At the end, we will showcase our example of the game.

STAR 1: A Survey on Cage-based Deformation of 3D Models

Daniel Ströter, Jean-Marc Thiery, Kai Hormann, Jiong Chen, Qingjun Chang, Sebastian Besler, Johannes Sebastian Mueller-Roemer, Tamy Boubekeur, André Stork, and Dieter W. Fellner

Interactive deformation via control handles is essential in computer graphics for the modeling of 3D geometry. Deformation control structures include lattices for free-form deformation and skeletons for character articulation, but this report focuses on cage-based deformation. Cages for deformation control are coarse polygonal meshes that encase the to-be-deformed geometry, enabling high-resolution deformation. Cage-based deformation enables users to quickly manipulate 3D geometry by deforming the cage. Due to their utility, cage-based deformation techniques increasingly appear in many geometry modeling applications. For this reason, the computer graphics community has invested a great deal of effort in the past decade and beyond into improving automatic cage generation and cage-based deformation. Recent advances have significantly extended the practical capabilities of cage-based deformation methods. As a result, there is a large body of research on cage-based deformation. In this report, we provide a comprehensive overview of the current state of the art in cage-based deformation of 3D geometry. We discuss current methods in terms of deformation quality, practicality, and precomputation demands. In addition, we highlight potential future research directions that overcome current issues and extend the set of practical applications. In conjunction with this survey, we publish an application to unify the most relevant deformation methods. Our report is intended for computer graphics researchers, developers of interactive geometry modeling applications, and 3D modeling and character animation artists.

STAR 2: Text-to-3D Shape Generation

Han-Hung Lee, Manolis Savva, and Angel Xuan Chang

Recent years have seen an explosion of work and interest in text-to-3D shape generation. Much of the progress is driven by advances in 3D representations, large-scale pretraining and representation learning for text and image data enabling generative AI models, and differentiable rendering. Computational systems that can perform text-to-3D shape generation have captivated the popular imagination as they enable non-expert users to easily create 3D content directly from text. However, there are still many limitations and challenges remaining in this problem space. In this state-of-the-art report, we provide a survey of the underlying technology and methods enabling text-to-3D shape generation to summarize the background literature. We then derive a systematic categorization of recent work on text-to-3D shape generation based on the type of supervision data required. Finally, we discuss limitations of the existing categories of methods, and delineate promising directions for future work.

STAR 3: Recent Trends in 3D Reconstruction of General Non-Rigid Scenes

Raza Yunus, Jan Eric Lenssen, Michael Niemeyer, Yiyi Liao, Christian Rupprecht, Christian Theobalt, Gerard Pons-Moll, Jia-Bin Huang, Vladislav Golyanik, and Eddy Ilg

Reconstructing models of the real world, including 3D geometry, appearance, and motion of real scenes, is essential for computer graphics and computer vision. It enables the synthesizing of photorealistic novel views, useful for the movie industry and AR/VR applications. It also facilitates the content creation necessary in computer games and AR/VR by avoiding laborious manual design processes. Further, such models are fundamental for intelligent computing systems that need to interpret real-world scenes and actions to act and interact safely with the human world. Notably, the world surrounding us is dynamic, and reconstructing models of dynamic, non-rigidly moving scenes is a severely underconstrained and challenging problem. This state-of-the-art report (STAR) offers the reader a comprehensive summary of state-of-the-art techniques with monocular and multi-view inputs such as data from RGB and RGB-D sensors, among others, conveying an understanding of different approaches, their potential applications, and promising further research directions. The report covers 3D reconstruction of general non-rigid scenes and further addresses the techniques for scene decomposition, editing and controlling, and generalizable and generative modeling. More specifically, we first review the common and fundamental concepts necessary to understand and navigate the field and then discuss the state-of-the-art techniques by reviewing recent approaches that use traditional and machine-learning-based neural representations, including a discussion on the newly enabled applications. The STAR is concluded with a discussion of the remaining limitations and open challenges.

STAR 4: State of the Art on Diffusion Models for Visual Computing

Ryan Po, Yifan Wang, Vladislav Golyanik, Kfir Aberman, Jon T. Barron, Amit Bermano, Eric Chan, Tali Dekel, Aleksander Holynski, Angjoo Kanazawa, C. Karen Liu, Lingjie Liu, Ben Mildenhall, Matthias Niessner, Björn Ommer, Christian Theobalt, Peter Wonka, and Gordon Wetzstein

The field of visual computing is rapidly advancing due to the emergence of generative artificial intelligence (AI), which unlocks unprecedented capabilities for the generation, editing, and reconstruction of images, videos, and 3D scenes. In these domains, diffusion models are the generative AI architecture of choice. Within the last year alone, the literature on diffusion-based tools and applications has seen exponential growth and relevant papers are published across the computer graphics, computer vision, and AI communities with new works appearing daily on arXiv. This rapid growth of the field makes it difficult to keep up with all recent developments. The goal of this state-of-the-art report (STAR) is to introduce the basic mathematical concepts of diffusion models, implementation details and design choices of the popular Stable Diffusion model, as well as overview important aspects of these generative AI tools, including personalization, conditioning, inversion, among others. Moreover, we give a comprehensive overview of the rapidly growing literature on diffusion-based generation and editing, categorized by the type of generated medium, including 2D images, videos, 3D objects, locomotion, and 4D scenes. Finally, we discuss available datasets, metrics, open challenges, and social implications. This STAR provides an intuitive starting point to explore this exciting topic for researchers, artists, and practitioners alike.

STAR 5: A Survey on Realistic Virtual Human Animations: Definitions, Features and Evaluations

Rim Rekik, Stefanie Wuhrer, Ludovic Hoyet, Katja Zibrek, and Anne-Hélène Olivier

Generating realistic animated virtual humans is a problem that has been extensively studied with many applications in different types of virtual environments. However, the creation process of such realistic animations is challenging, especially because of the number and variety of influencing factors, that should then be identified and evaluated. In this paper, we attempt to provide a clearer understanding of how the multiple factors that have been studied in the literature impact the level of realism of animated virtual humans, by providing a survey of studies assessing their realism. This includes a review of features that have been manipulated to increase the realism of virtual humans, as well as evaluation approaches that have been developed. As the challenges of evaluating animated virtual humans in a way that agrees with human perception are still active research problems, this survey further identifies important open problems and directions for future research.

STAR 6: Virtual Instrument Performances (VIP): A Comprehensive Review

Theodoros Kyriakou, Merce Alvarez de la Campa Crespo, Andreas Panayiotou, Yiorgos Chrysanthou, Panayiotis Charalambous, and Andreas Aristidou

Driven by recent advancements in Extended Reality (XR), the hype around the Metaverse, and real-time computer graphics, the transformation of the performing arts, particularly in digitizing and visualizing musical experiences, is an ever-evolving landscape. This transformation offers significant potential in promoting inclusivity, fostering creativity, and enabling live performances in diverse settings. However, despite its immense potential, the field of Virtual Instrument Performances (VIP) has remained relatively unexplored due to numerous challenges. These challenges arise from the complex and multi-modal nature of musical instrument performances, the need for high precision motion capture under occlusions including the intricate interactions between a musician’s body and fingers with instruments, the precise synchronization and seamless integration of various sensory modalities, accommodating variations in musicians’ playing styles, facial expressions, and addressing instrument specific nuances. This comprehensive survey delves into the intersection of technology, innovation, and artistic expression in the domain of virtual instrument performances. It explores musical performance multi-modal databases and investigates a wide range of data acquisition methods, encompassing diverse motion capture techniques, facial expression recording, and various approaches for capturing audio and MIDI data (Musical Instrument Digital Interface). The survey also explores Music Information Retrieval (MIR) tasks, with a particular emphasis on the Musical Performance Analysis (MPA) field, and offers an overview of various works in the realm of Musical Instrument Performance Synthesis (MIPS), encompassing recent advancements in generative models. The ultimate aim of this survey is to unveil the technological limitations, initiate a dialogue about the current challenges, and propose promising avenues for future research at the intersection of technology and the arts.

STAR 7: Cues to fast-forward collaboration: A Survey of Workspace Awareness and Visual Cues in XR Collaborative Systems

Rodrigo Assaf, Daniel Mendes, and Rui Rodrigues

Collaboration in extended reality (XR) environments presents complex challenges that revolve around how users perceive the presence, intentions, and actions of their collaborators. This paper delves into the intricate realm of group awareness, focusing specifically on workspace awareness and the innovative visual cues designed to enhance user comprehension. The research begins by identifying a spectrum of collaborative situations drawn from an analysis of XR prototypes in the existing literature. Then, we describe and introduce a novel classification for workspace awareness, along with an exploration of visual cues recently employed in research endeavors. Lastly, we present the key findings and shine a spotlight on promising yet unexplored topics. This work not only serves as a reference for experienced researchers seeking to inform the design of their own collaborative XR applications but also extends a welcoming hand to newcomers in this dynamic field.

STAR 8: Snow and Ice Animation Methods in Computer Graphics

Prashant Goswami

Snow and ice animation methods are becoming increasingly popular in the field of computer graphics (CG). The applications of snow and ice in CG are varied, ranging from generating realistic background landscapes to avalanches and physical interaction with objects in movies, games, etc. Over the past two decades, several methods have been proposed to capture the time-evolving physical appearance or simulation of snow and ice using different models at different scales. This state-of-the-art report aims to identify existing animation methods in the field, provide an up-to-date summary of the research in CG, and identify gaps for promising future work. Furthermore, we also attempt to identify the primarily related work done on snow and ice in some other disciplines, such as civil or mechanical engineering, and draw a parallel with the similarities and differences in CG.

 Neural Semantic Surface Maps

Luca Morreale, Noam Aigerman, Vladimir Kim, and Niloy J. Mitra

We present an automated technique for computing a map between two genus-zero shapes, which matches semantically corresponding regions to one another. Lack of annotated data prohibits direct inference of 3D semantic priors; instead, current state-of-the-art methods predominantly optimize geometric properties or require varying amounts of manual annotation. To overcome the lack of annotated training data, we distill semantic matches from pre-trained vision models: our method renders the pair of untextured 3D shapes from multiple viewpoints; the resulting renders are then fed into an off-the-shelf image matching strategy that leverages a pre-trained visual model to produce feature points. This yields semantic correspondences, which are projected back to the 3D shapes, producing a raw matching that is inaccurate and inconsistent across different viewpoints. These correspondences are refined and distilled into an inter-surface map by a dedicated optimization scheme, which promotes bijectivity and continuity of the output map. We illustrate that our approach can generate semantic surface-to-surface maps, eliminating manual annotations or any 3D training data requirement. Furthermore, it proves effective in scenarios with high semantic complexity, where objects are non-isometrically related, as well as in situations where they are nearly isometric.

HaLo-NeRF: Learning Geometry-Guided Semantics for Exploring Unconstrained Photo Collections

Chen Dudai, Morris Alper, Hana Bezalel, Rana Hanocka, Itai Lang, and Hadar Averbuch-Elor

Internet image collections containing photos captured by crowds of photographers show promise for enabling digital exploration of large-scale tourist landmarks. However, prior works focus primarily on geometric reconstruction and visualization, neglecting the key role of language in providing a semantic interface for navigation and fine-grained understanding. In more constrained 3D domains, recent methods have leveraged modern vision-and-language models as a strong prior of 2D visual semantics. While these models display an excellent understanding of broad visual semantics, they struggle with unconstrained photo collections depicting such tourist landmarks, as they lack expert knowledge of the architectural domain and fail to exploit the geometric consistency of images capturing multiple views of such scenes. In this work, we present a localization system that connects neural representations of scenes depicting large-scale landmarks with text describing a semantic region within the scene, by harnessing the power of SOTA vision-and-language models with adaptations for understanding landmark scene semantics. To bolster such models with fine-grained knowledge, we leverage large-scale Internet data containing images of similar landmarks along with weakly-related textual information. Our approach is built upon the premise that images physically grounded in space can provide a powerful supervision signal for localizing new concepts, whose semantics may be unlocked from Internet textual metadata with large language models. We use correspondences between views of scenes to bootstrap spatial understanding of these semantics, providing guidance for 3D-compatible segmentation that ultimately lifts to a volumetric scene representation. To evaluate our method, we present a new benchmark dataset containing large-scale scenes with groundtruth segmentations for multiple semantic concepts. Our results show that HaLo-NeRF can accurately localize a variety of semantic concepts related to architectural landmarks, surpassing the results of other 3D models as well as strong 2D segmentation baselines. Our code and data are publicly available at https://tau-vailab.github.io/HaLo-NeRF/.

Raster-to-Graph: Floorplan Recognition via Autoregressive Graph Prediction with an Attention Transformer

Sizhe Hu, Wenming Wu, Ruolin Su, Wanni Hou, Liping Zheng, and Benzhu Xu

Recognizing the detailed information embedded in rasterized floorplans is at the research forefront in the community of computer graphics and vision. With the advent of deep neural networks, automatic floorplan recognition has made tremendous breakthroughs. However, co-recognizing both the structures and semantics of floorplans through one neural network remains a significant challenge. In this paper, we introduce a novel framework Raster-to-Graph, which automatically achieves structura land semantic recognition of floorplans. We represent vectorized floorplans as structural graphs embedded with floorplan semantics, thus transforming the floorplan recognition task into a structural graph prediction problem. We design an autoregressive prediction framework using the neural network architecture of the visual attention Transformer, iteratively predicting the wall junctions and wall segments of floorplans in the order of graph traversal. Additionally, we propose a large-scale floorplan dataset containing over 10,000 real-world residential floorplans. Our autoregressive framework can automatically recognize the structures and semantics of floorplans. Extensive experiments demonstrate the effectiveness of our framework, showing significant improvements on all metrics. Qualitative and quantitative evaluations indicate that our framework outperforms existing state-of-the-art methods. Code and dataset for this paper are available at: https://github.com/HSZVIS/Raster-to-Graph.

Interactive Exploration of Vivid Material Iridescence based on Bragg Mirrors

Gary Fourneau, Romain Pacanowski, and Pascal Barla

Many animals, plants or gems exhibit iridescent material appearance in nature. These are due to specific geometric structures at scales comparable to visible wavelengths, yielding so-called structural colors. The most vivid examples are due to photonic crystals, where a same structure is repeated in one, two or three dimensions, augmenting the magnitude and complexity of interference effects. In this paper, we study the appearance of 1D photonic crystals (repetitive pairs of thin films), also called Bragg mirrors. Previous work has considered the effect of multiple thin films using the classical transfer matrix approach, which increases in complexity when the number of repetitions increases. Our first contribution is to introduce a more efficient closedform reflectance formula [Yeh88] for Bragg mirror reflectance to the Graphics community, as well as an approximation that lends itself to efficient spectral integration for RGB rendering. We then explore the appearance of stacks made of rough Bragg layers. Here our contribution is to show that they may lead to a ballistic transmission, significantly speeding up position-free rendering and leading to an efficient single-reflection BRDF model.

Single-Image SVBRDF Estimation with Learned Gradient Descent

Xuejiao Luo, Leonardo Scandolo, Adrien Bousseau, and Elmar Eisemann

Recovering spatially-varying materials from a single photograph of a surface is inherently ill-posed, making the direct application of a gradient descent on the reflectance parameters prone to poor minima. Recent methods leverage deep learning either by directly regressing reflectance parameters using feed-forward neural networks or by learning a latent space of SVBRDFs using encoder-decoder or generative adversarial networks followed by a gradient-based optimization in latent space. The former is fast but does not account for the likelihood of the prediction, i.e., how well the resulting reflectance explains the input image. The latter provides a strong prior on the space of spatially-varying materials, but this prior can hinder the reconstruction of images that are too different from the training data. Our method combines the strengths of both approaches. We optimize reflectance parameters to best reconstruct the input image using a recurrent neural network, which iteratively predicts how to update the reflectance parameters given the gradient of the reconstruction likelihood. By combining a learned prior with a likelihood measure, our approach provides a maximum a posteriori estimate of the SVBRDF. Our evaluation shows that this learned gradient-descent method achieves state-of-the-art performance for SVBRDF estimation on synthetic and real images.

PossibleImpossibles: Exploratory Procedural Design of Impossible Structures

Yuanbo Li, Tianyi Ma, Zaineb Aljumayaat, and Daniel Ritchie

We present a method for generating structures in three-dimensional space that appear to be impossible when viewed from specific perspectives. Previous approaches focus on helping users to edit specific structures and require users to have knowledge of structural positioning causing the impossibility. On the contrary, our system is designed to aid users without prior knowledge to explore a wide range of potentially impossible structures. The essence of our method lies in features we call visual bridges that confuse viewers regarding the depth of the resulting structure. We use these features as starting points and employ procedural modeling to systematically generate the result. We propose scoring functions for enforcing desirable spatial arrangement of the result and use Sequential Monte Carlo to sample outputs that score well under these functions. We also present a proof-ofconcept user interface and demonstrate various results generated using our system.

Hierarchical Co-generation of Parcels and Streets in Urban Modeling

Zebin Chen, Peng Song, and F. Peter Ortner

We present a computational framework for modeling land parcels and streets. In the real world, parcels and streets are highly coupled with each other since a street network connects all the parcels in a certain area. However, existing works model parcels and streets separately to simplify the problem, resulting in urban layouts with irregular parcels and/or suboptimal streets. In this paper, we propose a hierarchical approach to co-generate parcels and streets from a user-specified polygonal land shape, guided by a set of fundamental urban design requirements. At each hierarchical level, new parcels are generated based on binary splitting of existing parcels, and new streets are subsequently generated by leveraging efficient graph search tools to ensure that each new parcel has a street access. At the end, we optimize the geometry of the generated parcels and streets to further improve their geometric quality. Our computational framework outputs an urban layout with a desired number of regular parcels that are reachable via a connected street network, for which users are allowed to control the modeling process both locally and globally. Quantitative comparisons with state-of-the-art approaches show that our framework is able to generate parcels and streets that are superior in some aspects.

Strokes2Surface: Recovering Curve Networks From 4D Architectural Design Sketches

Shervin Rasoulzadeh, Michael Wimmer, Philipp Stauss, and Iva Kovacic

We present Strokes2Surface, an offline geometry reconstruction pipeline that recovers well-connected curve networks from imprecise 4D sketches to bridge concept design and digital modeling stages in architectural design. The input to our pipeline consists of 3D strokes’ polyline vertices and their timestamps as the 4th dimension, along with additional metadata recorded throughout sketching. Inspired by architectural sketching practices, our pipeline combines a classifier and two clustering models to achieve its goal. First, with a set of extracted hand-engineered features from the sketch, the classifier recognizes the type of individual strokes between those depicting boundaries (Shape strokes) and those depicting enclosed areas (Scribble strokes). Next, the two clustering models parse strokes of each type into distinct groups, each representing an individual edge or face of the intended architectural object. Curve networks are then formed through topology recovery of consolidated Shape clusters and surfaced using Scribble clusters guiding the cycle discovery. Our evaluation is threefold: We confirm the usability of the Strokes2Surface pipeline in architectural design use cases via a user study, we validate our choice of features via statistical analysis and ablation studies on our collected dataset, and we compare our outputs against a range of reconstructions computed using alternative methods.

TRIPS: Trilinear Point Splatting for Real-Time Radiance Field Rendering

Linus Franke, Darius Rückert, Laura Fink, and Marc Stamminger

Point-based radiance field rendering has demonstrated impressive results for novel view synthesis, offering a compelling blend of rendering quality and computational efficiency. However, also latest approaches in this domain are not without their shortcomings.3D Gaussian Splatting [KKLD23] struggles when tasked with rendering highly detailed scenes, due to blurring and cloudy artifacts. On the other hand, ADOP [RFS22] can accommodate crisper images, but the neural reconstruction network decreases performance, it grapples with temporal instability and it is unable to effectively address large gaps in the point cloud. In this paper, we present TRIPS (Trilinear Point Splatting), an approach that combines ideas from both Gaussian Splatting and ADOP. The fundamental concept behind our novel technique involves rasterizing points into a screen-space image pyramid, with the selection of the pyramid layer determined by the projected point size. This approach allows rendering arbitrarily large points using a single trilinear write. A lightweight neural network is then used to reconstruct a hole-free image including detail beyond splat resolution. Importantly, our render pipeline is entirely differentiable, allowing for automatic optimization of both point sizes and positions. Our evaluation demonstrate that TRIPS surpasses existing state-of-the-art methods in terms of rendering quality while maintaining a real-time frame rate of 60 frames per second on readily available hardware. This performance extends to challenging scenarios, such as scenes featuring intricate geometry, expansive landscapes, and auto-exposed footage. The project page is located at: https://lfranke.github.io/trips

Real-time Neural Rendering of Dynamic Light Fields

Arno Coomans, Edoardo Alberto Dominici, Christian Döring, Joerg H. Mueller, Jozef Hladky, and Markus Steinberger

Synthesising high-quality views of dynamic scenes via path tracing is prohibitively expensive. Although caching offline-quality global illumination in neural networks alleviates this issue, existing neural view synthesis methods are limited to mainly static scenes, have low inference performance or do not integrate well with existing rendering paradigms. We propose a novel neural method that is able to capture a dynamic light field, renders at real-time frame rates at 1920×1080 resolution and integrates seamlessly with Monte Carlo ray tracing frameworks. We demonstrate how a combination of spatial, temporal and a novel surface-space encoding are each effective at capturing different kinds of spatio-temporal signals. Together with a compact fully-fused neural network and architectural improvements, we achieve a twenty-fold increase in network inference speed compared to related methods at equal or better quality. Our approach is suitable for providing offline-quality real-time rendering in a variety of scenarios, such as free-viewpoint video, interactive multi-view rendering, or streaming rendering. Finally, our work can be integrated into other rendering paradigms, e.g., providing a dynamic background for interactive scenarios where the foreground is rendered with traditional methods.

Real-Time Neural Materials using Block-Compressed Features

Clément Weinreich, Louis De Oliveira, Antoine Houdard, and Georges Nader

Neural materials typically consist of a collection of neural features along with a decoder network. The main challenge in integrating such models in real-time rendering pipelines lies in the large size required to store their features in GPU memory and the complexity of evaluating the network efficiently. We present a neural material model whose features and decoder are specifically designed to be used in real-time rendering pipelines. Our framework leverages hardware-based block compression(BC) texture formats to store the learned features and trains the model to output the material information continuously in space and scale. To achieve this, we organize the features in a block-based manner and emulate BC6 decompression during training, making it possible to export them as regular BC6 textures. This structure allows us to use high resolution features while maintaining a low memory footprint. Consequently, this enhances our model’s overall capability, enabling the use of a lightweight and simple decoder architecture that can be evaluated directly in a shader. Furthermore, since the learned features can be decoded continuously, it allows for random uv sampling and smooth transition between scales without needing any subsequent filtering. As a result, our neural material has a small memory footprint, can be decoded extremely fast adding a minimal computational overhead to the rendering pipeline.

SENS: Part-Aware Sketch-based Implicit Neural Shape Modeling

Alexandre Binninger, Amir Hertz, Olga Sorkine-Hornung, Daniel Cohen-Or, and Raja Giryes

We present SENS, a novel method for generating and editing 3D models from hand-drawn sketches, including those of abstract nature. Our method allows users to quickly and easily sketch a shape, and then maps the sketch into the latent space of a partaware neural implicit shape architecture. SENS analyzes the sketch and encodes its parts into ViT patch encoding, subsequently feeding them into a transformer decoder that converts them to shape embeddings suitable for editing 3D neural implicit shapes. SENS provides intuitive sketch-based generation and editing, and also succeeds in capturing the intent of the user’s sketch to generate a variety of novel and expressive 3D shapes, even from abstract and imprecise sketches. Additionally, SENS supports refinement via part reconstruction, allowing for nuanced adjustments and artifact removal. It also offers part-based modeling capabilities, enabling the combination of features from multiple sketches to create more complex and customized 3D shapes. We demonstrate the effectiveness of our model compared to the state-of-the-art using objective metric evaluation criteria and a user study, both indicating strong performance on sketches with a medium level of abstraction. Furthermore, we showcase our method’s intuitive sketch-based shape editing capabilities, and validate it through a usability study.

PPSurf: Combining Patches and Point Convolutions for Detailed Surface Reconstruction

Philipp Erler, Lizeth Fuentes-Perez, Pedro Hermosilla, Paul Guerrero, Renato Pajarola, Michael Wimmer

3D surface reconstruction from point clouds is a key step in areas such as content creation, archaeology, digital cultural heritage and engineering. Current approaches either try to optimize a non-data-driven surface representation to fit the points, or learn a data-driven prior over the distribution of commonly occurring surfaces and how they correlate with potentially noisy point clouds.Data-driven methods enable robust handling of noise and typically either focus on a global or a local prior, which trade-off between robustness to noise on the global end and surface detail preservation on the local end. We propose PPSURF as a method that combines a global prior based on point convolutions and a local prior based on processing local point cloud patches. We show that this approach is robust to noise while recovering surface details more accurately than the current state-of-the-art. Our source code, pre-trained model and dataset are available at https://github.com/cg-tuwien/ppsurf .

Physically-Based Lighting for 3D Generative Models of Cars

Nicolas Violante, Alban Gauthier, Stavros Diolatzis, Thomas Leimkühler, and George Drettakis

Recent work has demonstrated that Generative Adversarial Networks (GANs) can be trained to generate 3D content from 2D image collections, by synthesizing features for neural radiance field rendering. However, most such solutions generate radiance, with lighting entangled with materials. This results in unrealistic appearance, since lighting cannot be changed and view-dependent effects such as reflections do not move correctly with the viewpoint. In addition, many methods have difficulty for full, 360◦ rotations, since they are often designed for mainly front-facing scenes such as faces. We introduce a new 3DGAN framework that addresses these shortcomings, allowing multi-view coherent 360◦ viewing and at the same time relighting for objects with shiny reflections, which we exemplify using a car dataset. The success of our solution stems from three main contributions. First, we estimate initial camera poses for a dataset of car images, and then learn to refine the distribution of camera parameters while training the GAN. Second, we propose an efficient Image-Based Lighting model, that we use in a 3DGAN to generate disentangled reflectance, as opposed to the radiance synthesized in most previous work. The material is used for physically-based rendering with a dataset of environment maps. Third, we improve the 3D GAN architecture compared to previous work and design a careful training strategy that allows effective disentanglement. Our model is the first that generate a variety of 3D cars that are multi-view consistent and that can be relit interactively with any environment map.

Real-Time Underwater Spectral Rendering

Nestor Monzon, Diego Gutierrez, Derya Akkaynak, and Adolfo Muñoz

The light field in an underwater environment is characterized by complex multiple scattering interactions and wavelengthdependent attenuation, requiring significant computational resources for the simulation of underwater scenes. We present a novel approach that makes it possible to simulate multi-spectral underwater scenes, in a physically-based manner, in real time. Our key observation is the following: In the vertical direction, the steady decay in irradiance as a function of depth is characterized by the diffuse downwelling attenuation coefficient, which oceanographers routinely measure for different types of waters. We rely on a database of such real-world measurements to obtain an analytical approximation to the Radiative Transfer Equation, allowing for real-time spectral rendering with results comparable to Monte Carlo ground-truth references, in a fraction of the time. We show results simulating underwater appearance for the different optical water types, including volumetric shadows and dynamic, spatially varying lighting near the water surface.

Physically Based Real-Time Rendering of Atmospheres using Mie Theory

Simon Schneegans, Tim Meyran, Ingo Ginkel, Gabriel Zachmann, and Andreas Gerndt

Most real-time rendering models for atmospheric effects have been designed and optimized for Earth’s atmosphere. Some authors have proposed approaches for rendering other atmospheres, but these methods still use approximations that are only valid on Earth. For instance, the iconic blue glow of Martian sunsets can not be represented properly as the complex interference effects of light scattered at dust particles can not be captured by these approximations. In this paper, we present an approach for generalizing an existing model to make it capable of rendering extraterrestrial atmospheres. This is done by replacing the approximations with a physical model based on Mie Theory. We use the particle-size distribution, the particle-density distribution as well as the wavelength-dependent refractive index of atmospheric particles as input. To demonstrate the feasibility of this idea, we extend the model by Bruneton et al. [BN08] and implement it into CosmoScout VR, an open-source visualization of our Solar System. In a first step, we use Mie Theory to precompute the scattering behaviour of a particle mixture. Then, multi-scattering is simulated, and finally the precomputation results are used for real-time rendering. We demonstrate that this not only improves the visualization of the Martian atmosphere, but also creates more realistic results for our own atmosphere.

An Empirically Derived Adjustable Model for Particle Size Distributions in Advection Fog

Monika Kolářová, Loïc Lachiver, and Alexander Wilkie

Realistically modelled atmospheric phenomena are a long-standing research topic in rendering. While significant progress has been made in modelling clear skies and clouds, fog has often been simplified as a medium that is homogeneous throughout, or as a simple density gradient. However, these approximations neglect the characteristic variations real advection fog shows throughout its vertical span, and do not provide the particle distribution data needed for accurate rendering. Based on data from meteorological literature, we developed an analytical model that yields the distribution of particle size as a function of altitude within an advection fog layer. The thickness of the fog layer is an additional input parameter, so that fog layers of varying thickness can be realistically represented. We also demonstrate that based on Mie scattering, one can easily integrate this model into a Monte Carlo renderer. Our model is the first ever non-trivial volumetric model for advection fog that is based on real measurement data, and that contains all the components needed for inclusion in a modern renderer. The model is provided as open source component, and can serve as reference for rendering problems that involve fog layers.

BallMerge: High-quality Fast Surface Reconstruction via Voronoi Balls

Amal Dev Parakkat, Stefan Ohrhallinger, Elmar Eisemann, and Pooran Memari

We introduce a Delaunay-based algorithm for reconstructing the underlying surface of a given set of unstructured points in 3D. The implementation is very simple, and it is designed to work in a parameter-free manner. The solution builds upon the fact that in the continuous case, a closed surface separates the set of maximal empty balls (medial balls) into an interior and exterior. Based on discrete input samples, our reconstructed surface consists of the interface between Voronoi balls, which approximate the interior and exterior medial balls. An initial set of Voronoi balls is iteratively processed, merging Voronoi-ball pairs if they fulfil an overlapping error criterion. Our complete open-source reconstruction pipeline performs up to two quick linear-time passes on the Delaunay complex to output the surface, making it an order of magnitude faster than the state of the art while being competitive in memory usage and often superior in quality. We propose two variants (local and global), which are carefully designed to target two different reconstruction scenarios for watertight surfaces from accurate or noisy samples, as well as real-world scanned data sets, exhibiting noise, outliers, and large areas of missing data. The results of the global variant are, by definition, watertight, suitable for numerical analysis and various applications (e.g., 3D printing). Compared to classical Delaunay-based reconstruction techniques, our method is highly stable and robust to noise and outliers, evidenced via various experiments, including on real-world data with challenges such as scan shadows, outliers, and noise, even without additional preprocessing.

Non-Euclidean Sliced Optimal Transport Sampling

Baptiste Genest, Nicolas Courty, and David Coeurjolly

In machine learning and computer graphics, a fundamental task is the approximation of a probability density function through a well-dispersed collection of samples. Providing a formal metric for measuring the distance between probability measures on general spaces, Optimal Transport (OT) emerges as a pivotal theoretical framework within this context. However, the associated computational burden is prohibitive in most real-world scenarios. Leveraging the simple structure of OT in 1D, Sliced Optimal Transport (SOT) has appeared as an efficient alternative to generate samples in Euclidean spaces. This paper pushes the boundaries of SOT utilization in computational geometry problems by extending its application to sample densities residing on more diverse mathematical domains, including the spherical space Sd, the hyperbolic plane Hd, and the real projective plane Pd. Moreover, it ensures the quality of these samples by achieving a blue noise characteristic, regardless of the dimensionality involved. The robustness of our approach is highlighted through its application to various geometry processing tasks, such as the intrinsic blue noise sampling of meshes, as well as the sampling of directions and rotations. These applications collectively underscore the efficacy of our methodology.

GLS-PIA: n-Dimensional Spherical B-Spline Curve Fitting based on Geodesic Least Square with Adaptive Knot Placement

Yuming Zhao, Zhongke Wu, and Xingce Wang

Due to the widespread applications of curves on n-dimensional spheres, fitting curves on n-dimensional spheres has received increasing attention in recent years. However, due to the non-Euclidean nature of spheres, curve fitting methods on n-dimensional spheres often struggle to balance fitting accuracy and curve fairness. In this paper, we propose a new fitting framework, GLSPIA, for parameterized point sets on n-dimensional spheres to address the challenge. Meanwhile, we provide the proof of the method. Firstly, we propose a progressive iterative approximation method based on geodesic least squares which can directly optimize the geodesic least squares loss on the n-sphere, improving the accuracy of the fitting. Additionally, we use an error allocation method based on contribution coefficients to ensure the fairness of the fitting curve. Secondly, we propose an adaptive knot placement method based on geodesic difference to estimate a more reasonable distribution of control points in the parameter domain, placing more control points in areas with greater detail. This enables B-spline curves to capture more details with a limited number of control points. Experimental results demonstrate that our framework achieves outstanding performance, especially in handling imbalanced data points. (In this paper, “sphere” refers to n-sphere (n ≥ 2) unless otherwise specified.)

Estimating Cloth Simulation Parameters From Tag Information and Cusick Drape Test

 Eunjung Ju, Kwang-yun Kim, Sungjin Yoon, Eungjune Shim, Gyoo-Chul Kang, Phil Sik Chang, and Myung Geol Choi

In recent years, the fashion apparel industry has been increasingly employing virtual simulations for the development of new products. The first step in virtual garment simulation involves identifying the optimal simulation parameters that accurately reproduce the drape properties of the actual fabric. Recent techniques advocate for a data-driven approach, estimating parameters from outcomes of a Cusick drape test. Such methods deviate from standard Cusick drape tests, introducing high-cost tools, which reduces practicality. Our research presents a more practical model, utilizing 2D silhouette images from the ISO-standardized Cusick drape test. Notably, while past models have shown limitations in estimating stretching parameters, our novel approach leverages the fabric’s tag information including fabric type and fiber composition. Our proposed model functions as a cascaded system: first, it estimates stretching parameters using tag information, then, in the subsequent step, it considers the estimated stretching parameters alongside the fabric sample’s Cusick drape test results to determine bending parameters. We validated our model against existing methods and applied it in practical scenarios, showing promising outcomes.

Neural Garment Dynamics via Manifold-Aware Transformers

Peizhuo Li, Tuanfeng Y. Wang, Timur Levent Kesdogan, Duygu Ceylan, and Olga Sorkine-Hornung

Data driven and learning based solutions for modeling dynamic garments have significantly advanced, especially in the context of digital humans. However, existing approaches often focus on modeling garments with respect to a fixed parametric human body model and are limited to garment geometries that were seen during training. In this work, we take a different approach and model the dynamics of a garment by exploiting its local interactions with the underlying human body. Specifically, as the body moves, we detect local garment-body collisions, which drive the deformation of the garment. At the core of our approach is a mesh-agnostic garment representation and a manifold-aware transformer network design, which together enable our method to generalize to unseen garment and body geometries. We evaluate our approach on a wide variety of garment types and motion sequences and provide competitive qualitative and quantitative results with respect to the state of the art.

Practical Method to Estimate Fabric Mechanics from Metadata

Henar Dominguez-Elvira, Alicia Nicás-Miquel, Gabriel Cirio, Alejandro Rodríguez, and Elena Garces

Estimating fabric mechanical properties is crucial to create realistic digital twins. Existing methods typically require testing physical fabric samples with expensive devices or cumbersome capture setups. In this work, we propose a method to estimate fabric mechanics just from known manufacturer metadata such as the fabric family, the density, the composition, and the thickness. Further, to alleviate the need to know the fabric family –which might be ambiguous or unknown for nonspecialists– we propose an end-to-end neural method that works with planar images of the textile as input. We evaluate our methods using extensive tests that include the industry standard Cusick and demonstrate that both of them produce drapes that strongly correlate with the ground truth estimates provided by lab equipment. Our method is the first to propose such a simple capture method for mechanical properties outperforming other methods that require testing the fabric in specific setups.

Polygon Laplacian Made Robust

 Astrid Bunge, Dennis R. Bukenberger, Sven Dominik Wagner, Marc Alexa, and Mario Botsch

Discrete Laplacians are the basis for various tasks in geometry processing. While the most desirable properties of the discretization invariably lead to the so-called cotangent Laplacian for triangle meshes, applying the same principles to polygon Laplacians leaves degrees of freedom in their construction. From linear finite elements it is well-known how the shape of triangles affects both the error and the operator’s condition. We notice that shape quality can be encapsulated as the trace of the Laplacian and suggest that trace minimization is a helpful tool to improve numerical behavior. We apply this observation to the polygon Laplacian constructed from a virtual triangulation [BHKB20] to derive optimal parameters per polygon. Moreover, we devise a smoothing approach for the vertices of a polygon mesh to minimize the trace. We analyze the properties of the optimized discrete operators and show their superiority over generic parameter selection in theory and through various experiments.

Advancing Front Surface Mapping

Marco Livesu

We present Advancing Front Mapping (AFM), a novel algorithm for the computation of injective maps to simple planar domains. AFM is inspired by the advancing front meshing paradigm, which is here revisited to operate on two embeddings at once, becoming a tool for compatible mesh generation. AFM extends the capabilities of existing robust approaches, supporting a broader set of embeddings (star-shaped polygons) with a direct approach, without resorting to intermediate constructions. Our method only relies on two topological operators (split and flip) and on the computation of segment intersections, thus permitting to compute a valid embedding without solving any numerical problem. AFM is therefore easy to implement, debug and deploy. This article is mainly focused on the presentation of the compatible advancing front idea and on the demonstration that the algorithm provably converges to an injective map. We also complement our theoretical analysis with an extensive practical validation, executing more than one billion advancing front moves on 36K mapping tasks.

Quad Mesh QuantizationWithout a T-Mesh

Yoann Coudert-Osmont, David Desobry, Martin Heistermann, David Bommes, Nicolas Ray, Dmitry Sokolov

Grid preserving maps of triangulated surfaces were introduced for quad meshing because the 2D unit grid in such maps corresponds to a sub-division of the surface into quad-shaped charts. These maps can be obtained by solving a mixed integer optimization problem: Real variables define the geometry of the charts and integer variables define the combinatorial structure of the decomposition. To make this optimization problem tractable, a common strategy is to ignore integer constraints at first, then to enforce them in a so-called quantization step. Actual quantization algorithms exploit the geometric interpretation of integer variables to solve an equivalent problem: They consider that the final quad mesh is a sub-division of a T-mesh embedded in the
surface, and optimize the number of sub-divisions for each edge of this T-mesh. We propose to operate on a decimated version of the original surface instead of the T-mesh. It is easier to implement and to adapt to constraints such as free boundaries, complex feature curves network etc.

The Impulse Particle-In-Cell Method

 Sergio Sancho, Jingwei Tang, Christopher Batty, and Vinicius C. Azevedo

An ongoing challenge in fluid animation is the faithful preservation of vortical details, which impacts the visual depiction of flows. We propose the Impulse Particle-In-Cell (IPIC) method, a novel extension of the popular Affine Particle-In-Cell (APIC) method that makes use of the impulse gauge formulation of the fluid equations. Our approach performs a coupled advection-stretching during particle-based advection to better preserve circulation and vortical details. The associated algorithmic changes are simple and straightforward to implement, and our results demonstrate that the proposed method is able to achieve more energetic and visually appealing smoke and liquid flows than APIC.

Wavelet Potentials: An Efficient Potential Recovery Technique for Pointwise Incompressible Fluids

Luan Lyu, Xiaohua Ren, Wei Cao, Jian Zhu, Enhua Wu, and Zhi-Xin Yang

We introduce an efficient technique for recovering the vector potential in wavelet space to simulate pointwise incompressible fluids. This technique ensures that fluid velocities remain divergence-free at any point within the fluid domain and preserves local volume during the simulation. Divergence-free wavelets are utilized to calculate the wavelet coefficients of the vector potential, resulting in a smooth vector potential with enhanced accuracy, even when the input velocities exhibit some degree of divergence. This enhanced accuracy eliminates the need for additional computational time to achieve a specific accuracy threshold, as fewer iterations are required for the pressure Poisson solver. Additionally, in 3D, since the wavelet transform is taken in-place, only the memory for storing the vector potential is required. These two features make the method remarkably efficient for recovering vector potential for fluid simulation. Furthermore, the method can handle various boundary conditions during the wavelet transform, making it adaptable for simulating fluids with Neumann and Dirichlet boundary conditions. Our approach is highly parallelizable and features a time complexity of O(n), allowing for seamless deployment on GPUs and yielding remarkable computational efficiency. Experiments demonstrate that, taking into account the time consumed by the pressure Poisson solver, the method achieves an approximate 2x speedup on GPUs compared to state-of-the-art vector potential recovery techniques while maintaining a precision level of 10−6 when single float precision is employed. The source code of ’Wavelet Potentials’ can be found in https://github.com/yours321dog/WaveletPotentials.

Monte Carlo Vortical Smoothed Particle Hydrodynamics for Simulating Turbulent Flows

Xingyu Ye, Xiaokun Wang, Yanrui Xu, Jiri Kosinka, Alexandru C. Telea, Lihua You, Jian Jun Zhang, and Jian Chang

For vortex particle methods relying on SPH-based simulations, the direct approach of iterating all fluid particles to capture velocity from vorticity can lead to a significant computational overhead during the Biot-Savart summation process. To address this challenge, we present a Monte Carlo vortical smoothed particle hydrodynamics (MCVSPH) method for efficiently simulating turbulent flows within an SPH framework. Our approach harnesses a Monte Carlo estimator and operates exclusively within a pre-sampled particle subset, thus eliminating the need for costly global iterations over all fluid particles. Our algorithm is decoupled from various projection loops which enforce incompressibility, independently handles the recovery of turbulent details, and seamlessly integrates with state-of-the-art SPH-based incompressibility solvers. Our approach rectifies the velocity of all fluid particles based on vorticity loss to respect the evolution of vorticity, effectively enforcing vortex motions. We demonstrate, by several experiments, that our MCVSPH method effectively preserves vorticity and creates visually prominent vortical motions.

Computational Smocking through Fabric-Thread Interaction

Ningfeng Zhou, Jing Ren, and Olga Sorkine-Hornung

We formalize Italian smocking, an intricate embroidery technique that gathers flat fabric into pleats along meandering lines of stitches, resulting in pleats that fold and gather where the stitching veers. In contrast to English smocking, characterized by colorful stitches decorating uniformly shaped pleats, and Canadian smocking, which uses localized knots to form voluminous pleats, Italian smocking permits the fabric to move freely along the stitched threads following curved paths, resulting in complex and unpredictable pleats with highly diverse, irregular structures, achieved simply by pulling on the threads. We introduce a novel method for digital previewing of Italian smocking results, given the thread stitching path as input. Our method uses a coarse-grained mass-spring system to simulate the interaction between the threads and the fabric. This configuration guides the fine-level fabric deformation through an adaptation of the state-of-the-art simulator, C-IPC [LKJ21]. Our method models the general problem of fabric-thread interaction and can be readily adapted to preview Canadian smocking as well.We compare our results to baseline approaches and physical fabrications to demonstrate the accuracy of our method.

Unfolding via Mesh Approximation using Surface Flows

Lars Zawallich and Renato Pajarola

Manufacturing a 3D object by folding from a 2D material is typically done in four steps: 3D surface approximation, unfolding the surface into a plane, printing and cutting the outline of the unfolded shape, and refolding it to a 3D object. Usually, these steps are treated separately from each other. In this work we jointly address the first two pipeline steps by allowing the 3D representation to smoothly change while unfolding. This way, we increase the chances to overcome possible ununfoldability issues. To join the two pipeline steps, our work proposes and combines different surface flows with a Tabu Unfolder. We empirically investigate the effects that different surface flows have on the performance as well as on the quality of the unfoldings. Additionally, we demonstrate the ability to solve cases by approximation which comparable algorithms either have to segment or can not solve at all.

Freeform Shape Fabrication by Kerfing Stiff Materials

Nils Speetzen and Leif Kobbelt

Fast, flexible, and cost efficient production of 3D models from 2D material sheets is a key component in digital fabrication and prototyping. In order to achieve high quality approximations of freeform shapes, a common set of methods aim to produce bendable 2D cutouts that are then assembled. So far bent surfaces are achieved automatically by computing developable patches of the input surface, e.g. in the context of papercraft. For stiff materials such as medium-density fibreboard (MDF) or plywood, the 2D cutouts require the application of additional cutting patterns (“kerfing”) to make them bendable. Such kerf patterns are commonly constructed with considerable user input, e.g. in architectural design. We propose a fully automatic method that produces kerfed cutouts suitable for the assembly of freeform shapes from stiff material sheets. By exploring the degrees of freedom emerging from the choice of bending directions, the creation of box joints at the patch boundaries as well as the application of kerf cuts with adaptive density, our method is able to achieve a high quality approximation of the input.

Physically-based Analytical Erosion for fast Terrain Generation

Petros Tzathas, Boris Gailleton, Philippe Steer, and Guillaume Cordonnier

Terrain generation methods have long been divided between procedural and physically-based. Procedural methods build upon the fast evaluation of a mathematical function but suffer from a lack of geological consistency, while physically-based simulation enforces this consistency at the cost of thousands of iterations unraveling the history of the landscape. In particular, the simulation of the competition between tectonic uplift and fluvial erosion expressed by the stream power law raised recent interest in computer graphics as this allows the generation and control of consistent large-scale mountain ranges, albeit at the cost of a lengthy simulation. In this paper, we explore the analytical solutions of the stream power law and propose a method that is both physically-based and procedural, allowing fast and consistent large-scale terrain generation. In our approach, time is no longer the stopping criterion of an iterative process but acts as the parameter of a mathematical function, a slider that controls the aging of the input terrain from a subtle erosion to the complete replacement by a fully formed mountain range. While analytical solutions have been proposed by the geomorphology community for the 1D case, extending them to a 2D heightmap proves challenging. We propose an efficient implementation of the analytical solutions with a multigrid accelerated iterative process and solutions to incorporate landslides and hillslope processes – two erosion factors that complement the stream power law.

Volcanic Skies: Coupling Explosive Eruptions with Atmospheric Simulation to Create Consistent Skyscapes

Pieter C. Pretorius, James Gain, Maud Lastic, Guillaume Cordonnier, Chen Jiong, Damien Rohmer, and Marie-Paule Cani

Explosive volcanic eruptions rank among the most terrifying natural phenomena, and are thus frequently depicted in films, games, and other media, usually with a bespoke once-off solution. In this paper, we introduce the first general-purpose model for bi-directional interaction between the atmosphere and a volcano plume. In line with recent interactive volcano models, we approximate the plume dynamics with Lagrangian disks and spheres and the atmosphere with sparse layers of 2D Eulerian grids, enabling us to focus on the transfer of physical quantities such as temperature, ash, moisture, and wind velocity between these sub-models. We subsequently generate volumetric animations by noise-based procedural upsampling keyed to aspects of advection, convection, moisture, and ash content to generate a fully-realized volcanic skyscape. Our model captures most of the visually salient features emerging from volcano-sky interaction, such as windswept plumes, enmeshed cap, bell and skirt clouds, shockwave effects, ash rain, and sheathes of lightning visible in the dark.

Real-time Terrain Enhancement with Controlled Procedural Patterns

Charline Grenier, Éric Guérin, Éric Galin, Basile Sauvage

Assisting the authoring of virtual terrains is a perennial challenge in the creation of convincing synthetic landscapes. Particularly, there is a need for augmenting artist-controlled low-resolution models with consistent relief details.We present a structured noise that procedurally enhances terrains in real time by adding spatially varying erosion patterns. The patterns can be cascaded, i.e. narrow ones are nested into large ones. Our model builds upon the Phasor noise, which we adapt to the specific characteristics of terrains (water flow, slope orientation). Relief details correspond to the underlying terrain characteristics and align with the slope to preserve the coherence of generated landforms. Moreover, our model allows for artist control, providing a palette of control maps, and can be efficiently implemented in graphics hardware, thus allowing for real-time synthesis and rendering, therefore permitting effective and intuitive authoring.

Recurrent Motion Refiner for Locomotion Stitching

Haemin Kim, Kyungmin Cho, Seokhyeon Hong, Junyong Noh

Stitching different character motions is one of the most commonly used techniques as it allows the user to make new animations that fit one’s purpose from pieces of motion. However, current motion stitching methods often produce unnatural motion with foot sliding artefacts, depending on the performance of the interpolation. In this paper, we propose a novel motion stitching technique based on a recurrent motion refiner (RMR) that connects discontinuous locomotions into a single natural locomotion. Our model receives different locomotions as input, in which the root of the last pose of the previous motion and that of the first pose of the next motion are aligned. During runtime, the model slides through the sequence, editing frames window by window to output a smoothly connected animation. Our model consists of a two-layer recurrent network that comes between a simple encoder and decoder. To train this network, we created a sufficient number of paired data with a newly designed data generation. This process employs a K-nearest neighbour search that explores a predefined motion database to create the corresponding input to the ground truth. Once trained, the suggested model can connect various lengths of locomotion sequences into a single natural locomotion.

Simplified Physical Model-based Balance-preserving Motion Re-targeting for Physical Simulation

Jaepyung Hwang, Shin Ishii

In this study, we propose a novel motion re-targeting framework that provides natural motions of target robot character models similar to the given source motions of a different skeletal structure. The natural target motion requires satisfying kinematic constraints to show a similar motion to the source motion although the kinematical structure between the source and the target character models differ from each other. Simultaneously, the target motion should maintain physically plausible features such as keeping the balance of the target character model. To handle the issue, we utilize a simple physics model (an invertedpendulum-on-a-cart model) during the motion re-targeting process. By interpreting the source motion’s balancing property via the pendulum model, the target motion inherits the balancing property of the source motion. The inheritance is derived by performing the motion analysis to extract the necessary parameters for re-targeting the pendulum model’s motion pattern and parameter learning to estimate the suitable parameters for the target character model. Based on the simple physics inheritance, the proposed framework provides balance-preserving target motions, even applicable to the full-body physics simulation or a real robot control.We validate the framework by experimenting with motion re-targeting from animal character and human character source models to the quadruped- and humanoid-type target models with Muaythai punching, kicking and walking motions. We also implement comparisons with the existing methods to clarify the enhancement.

Interactive Locomotion Style Control for a Human Character based on Gait Cycle Features

Chaelin Kim, Haekwang Eom, Jung Eun Yoo, Soojin Choi, Junyong Noh

This article introduces a data-driven locomotion style controller for full-body human characters using gait cycle features. Based on gait analysis, we define a set of gait features that can represent various locomotion styles as spatio-temporal patterns within a single gait cycle. We compute the gait features for every single gait cycle in motion capture data and use them to search for the desired motion. Our real-time style controller provides users with visual feedback for the changing inputs, exploiting the Motion Matching algorithm. We also provide a graphical controller interface that visualizes our style representation to enable intuitive control for users. We show that the proposed method is capable of retrieving appropriate locomotions for various gait
cycle features, from simple walking motions to single-foot motions such as hopping and dragging. To validate the effectiveness of our method, we conducted a user study that compares the usability and performance of our system with those of an existing footstep animation tool. The results show that our method is preferred over the baseline method for intuitive control and fast visual feedback.

Navigating the Manifold of Translucent Appearance

Dario Lanza, Belen Masia, and Adrian Jarabo

We present a perceptually-motivated manifold for translucent appearance, designed for intuitive editing of translucent materials by navigating through the manifold. Classic tools for editing translucent appearance, based on the use of sliders to tune a number of parameters, are challenging for non-expert users: These parameters have a highly non-linear effect on appearance, and exhibit complex interplay and similarity relations between them. Instead, we pose editing as a navigation task in a low-dimensional space of appearances, which abstracts the user from the underlying optical parameters. To achieve this, we build a low-dimensional continuous manifold of translucent appearance that correlates with how humans perceive this type of materials. We first analyze the correlation of different distance metrics in image space with human perception. We select the best-performing metric to build a low-dimensional manifold, which can be used to navigate the space of translucent appearance. To evaluate the validity of our proposed manifold within its intended application scenario, we build an editing interface that leverages the manifold, and relies on image navigation plus a fine-tuning step to edit appearance. We compare our intuitive interface to a traditional, slider-based one in a user study, demonstrating its effectiveness and superior performance when editing translucent objects.

Perceptual Quality Assessment of NeRF and Neural View Synthesis Methods for Front-Facing Views

Hanxue Liang, Tianhao Wu, Param Hanji, Francesco Banterle, Hongyun Gao, Rafal Mantiuk, and Cengiz Öztireli

Neural view synthesis (NVS) is one of the most successful techniques for synthesizing free viewpoint videos, capable of achieving high fidelity from only a sparse set of captured images. This success has led to many variants of the techniques, each evaluated on a set of test views typically using image quality metrics such as PSNR, SSIM, or LPIPS. There has been a lack of research on how NVS methods perform with respect to perceived video quality. We present the first study on perceptual evaluation of NVS and NeRF variants. For this study, we collected two datasets of scenes captured in a controlled lab environment as well as in-the-wild. In contrast to existing datasets, these scenes come with reference video sequences, allowing us to test for temporal artifacts and subtle distortions that are easily overlooked when viewing only static images. We measured the quality of videos synthesized by several NVS methods in a well-controlled perceptual quality assessment experiment as well as with many existing state-of-the-art image/video quality metrics. We present a detailed analysis of the results and recommendations for dataset and metric selection for NVS evaluation.

Predicting Perceived Gloss: Do Weak Labels Suffice?

Julia Guerrero-Viu, Jose Daniel Subias, Ana Serrano, Katherine R. Storrs, Roland W. Fleming, Belen Masia, and Diego Gutierrez

Estimating perceptual attributes of materials directly from images is a challenging task due to their complex, not fullyunderstood interactions with external factors, such as geometry and lighting. Supervised deep learning models have recently been shown to outperform traditional approaches, but rely on large datasets of human-annotated images for accurate perception predictions. Obtaining reliable annotations is a costly endeavor, aggravated by the limited ability of these models to generalise to different aspects of appearance. In this work, we show how a much smaller set of human annotations (“strong labels”) can be effectively augmented with automatically derived “weak labels” in the context of learning a low-dimensional image-computable gloss metric. We evaluate three alternative weak labels for predicting human gloss perception from limited annotated data. Incorporating weak labels enhances our gloss prediction beyond the current state of the art. Moreover, it enables a substantial reduction in human annotation costs without sacrificing accuracy, whether working with rendered images or real photographs.

TailorMe: Self-Supervised Learning of an Anatomically Constrained Volumetric Human Shape Model

Stephan Wenninger, Fabian Kemper, Ulrich Schwanecke, and Mario Botsch

Human shape spaces have been extensively studied, as they are a core element of human shape and pose inference tasks. Classic methods for creating a human shape model register a surface template mesh to a database of 3D scans and use dimensionality reduction techniques, such as Principal Component Analysis, to learn a compact representation. While these shape models enable global shape modifications by correlating anthropometric measurements with the learned subspace, they only provide limited localized shape control. We instead register a volumetric anatomical template, consisting of skeleton bones and soft tissue, to the surface scans of the CAESAR database. We further enlarge our training data to the full Cartesian product of all skeletons and all soft tissues using physically plausible volumetric deformation transfer. This data is then used to learn an anatomically constrained volumetric human shape model in a self-supervised fashion. The resulting TAILORME model enables shape sampling, localized shape manipulation, and fast inference from given surface scans.

CharacterMixer: Rig-Aware Interpolation of 3D Characters

Xiao Zhan, Rao Fu, and Daniel Ritchie

We present CharacterMixer, a system for blending two rigged 3D characters with different mesh and skeleton topologies while maintaining a rig throughout interpolation. CharacterMixer also enables interpolation during motion for such characters, a novel feature. Interpolation is an important shape editing operation, but prior methods have limitations when applied to rigged characters: they either ignore the rig (making interpolated characters no longer posable) or use a fixed rig and mesh topology. To handle different mesh topologies, CharacterMixer uses a signed distance field (SDF) representation of character shapes, with one SDF per bone. To handle different skeleton topologies, it computes a hierarchical correspondence between source and target character skeletons and interpolates the SDFs of corresponding bones. This correspondence also allows the creation of a single “unified skeleton” for posing and animating interpolated characters. We show that CharacterMixer produces qualitatively better interpolation results than two state-of-the-art methods while preserving a rig throughout interpolation. Project page: https://seanxzhan.github.io/projects/CharacterMixer.

Stylize My Wrinkles: Bridging the Gap from Simulation to Reality

Sebastian Weiss, Jackson Stanhope, Prashanth Chandran, Gaspard Zoss, and Derek Bradley

Modeling realistic human skin with pores and wrinkles down to the milli- and micrometer resolution is a challenging task. Prior work showed that such micro geometry can be efficiently generated through simulation methods, or in specialized cases via 3D scanning of real skin. Simulation methods allow to highly customize the wrinkles on the face, but can lead to a synthetic look. Scanning methods can lead to a more organic look for the micro details, however these methods are only applicable to small skin patches due to the required image resolution. In this work we aim to overcome the gap between synthetic simulation and real skin scanning, by proposing a method that can be applied to large skin regions (e.g. an entire face) with the controllability of simulation and the organic look of real micro details. Our method is based on style transfer at its core, where we use scanned displacement maps of real skin patches as style images and displacement maps from an artist-friendly simulation method as content images. We build a library of displacement maps as style images by employing a simplified scanning setup that can capture high-resolution patches of real skin. To create the content component for the style transfer and to facilitate parameter-tuning for the simulation, we design a library of preset parameter values depicting different skin types, and present a new method to fit the simulation parameters to scanned skin patches. This allows fully-automatic parameter generation, interpolation and stylization across entire faces. We evaluate our method by generating realistic skin micro details for various subjects of different ages and genders, and demonstrate that our approach achieves a more organic and natural look than simulation alone.

Enhancing Image Quality Prediction with Self-supervised Visual Masking

Ugur Çogalan, Mojtaba Bemana, Hans-Peter Seidel, and Karol Myszkowski

Full-reference image quality metrics (FR-IQMs) aim to measure the visual differences between a pair of reference and distorted images, with the goal of accurately predicting human judgments. However, existing FR-IQMs, including traditional ones like PSNR and SSIM and even perceptual ones such as HDR-VDP, LPIPS, and DISTS, still fall short in capturing the complexities and nuances of human perception. In this work, rather than devising a novel IQM model, we seek to improve upon the perceptual quality of existing FR-IQM methods. We achieve this by considering visual masking, an important characteristic of the human visual system that changes its sensitivity to distortions as a function of local image content. Specifically, for a given FR-IQM metric, we propose to predict a visual masking model that modulates reference and distorted images in a way that penalizes the visual errors based on their visibility. Since the ground truth visual masks are difficult to obtain, we demonstrate how they can be derived in a self-supervised manner solely based on mean opinion scores (MOS) collected from an FR-IQM dataset. Our approach results in enhanced FR-IQM metrics that are more in line with human prediction both visually and quantitatively.

Enhancing Spatiotemporal Resampling with a Novel MIS Weight

Xingyue Pan, Jiaxuan Zhang, Jiancong Huang, and Ligang Liu

In real-time rendering, optimizing the sampling of large-scale candidates is crucial. The spatiotemporal reservoir resampling (ReSTIR) method provides an effective approach for handling large candidate samples, while the Generalized Resampled Importance Sampling (GRIS) theory provides a general framework for resampling algorithms. However, we have observed that when using the generalized multiple importance sampling (MIS) weight in previous work during spatiotemporal reuse, variances gradually amplify in the candidate domain when there are significant differences. To address this issue, we propose a new MIS weight suitable for resampling that blends samples from different sampling domains, ensuring convergence of results as the proportion of non-canonical samples increases. Additionally, we apply this weight to temporal resampling to reduce noise caused by scene changes or jitter. Our method effectively reduces energy loss in the biased version of ReSTIR DI while incurring no additional overhead, and it also suppresses artifacts caused by a high proportion of temporal samples. As a result, our approach leads to lower variance in the sampling results.

Neural Denoising for Deep-Z Monte Carlo Renderings

Xianyao Zhang, Gerhard Röthlin, Shilin Zhu, Tunç Ozan Aydın, Farnood Salehi, Markus Gross, and Marios Papas

We present a kernel-predicting neural denoising method for path-traced deep-Z images that facilitates their usage in animation and visual effects production. Deep-Z images provide enhanced flexibility during compositing as they contain color, opacity, and other rendered data at multiple depth-resolved bins within each pixel. However, they are subject to noise, and rendering until convergence is prohibitively expensive. The current state of the art in deep-Z denoising yields objectionable artifacts, and current neural denoising methods are incapable of handling the variable number of depth bins in deep-Z images. Our method extends kernel-predicting convolutional neural networks to address the challenges stemming from denoising deep-Z images. We propose a hybrid reconstruction architecture that combines the depth-resolved reconstruction at each bin with the flattened reconstruction at the pixel level. Moreover, we propose depth-aware neighbor indexing of the depth-resolved inputs to the convolution and denoising kernel application operators, which reduces artifacts caused by depth misalignment present in deep-Z images. We evaluate our method on a production-quality deep-Z dataset, demonstrating significant improvements in denoising quality and performance compared to the current state-of-the-art deep-Z denoiser. By addressing the significant challenge of the cost associated with rendering path-traced deep-Z images, we believe that our approach will pave the way for broader adoption of deep-Z workflows in future productions.

Deep and Fast Approximate Order Independent Transparency

Grigoris Tsopouridis, Andreas A. Vasilakis, Ioannis Fudos

We present a machine learning approach for efficiently computing order independent transparency (OIT) by deploying a light weight neural network implemented fully on shaders. Our method is fast, requires a small constant amount of memory (depends only on the screen resolution and not on the number of triangles or transparent layers), is more accurate as compared to previous approximate methods, works for every scene without setup and is portable to all platforms running even with commodity GPUs. Our method requires a rendering pass to extract all features that are subsequently used to predict the overall OIT pixel colour with a pre-trained neural network. We provide a comparative experimental evaluation and shader source code of all methods for reproduction of the experiments.

Learning to Stabilize Faces

Jan Bednarik, Erroll Wood, Vassilis Choutas, Timo Bolkart, Daoye Wang, Chenglei Wu, and Thabo Beeler

Nowadays, it is possible to scan faces and automatically register them with high quality. However, the resulting face meshes often need further processing: we need to stabilize them to remove unwanted head movement. Stabilization is important for tasks like game development or movie making which require facial expressions to be cleanly separated from rigid head motion. Since manual stabilization is labor-intensive, there have been attempts to automate it. However, previous methods remain impractical: they either still require some manual input, produce imprecise alignments, rely on dubious heuristics and slow optimization, or assume a temporally ordered input. Instead, we present a new learning-based approach that is simple and fully automatic. We treat stabilization as a regression problem: given two face meshes, our network directly predicts the rigid transform between them that brings their skulls into alignment. We generate synthetic training data using a 3D Morphable Model (3DMM), exploiting the fact that 3DMM parameters separate skull motion from facial skin motion. Through extensive experiments we show that our approach outperforms the state-of-the-art both quantitatively and qualitatively on the tasks of stabilizing discrete sets of facial expressions as well as dynamic facial performances. Furthermore, we provide an ablation study detailing the design choices and best practices to help others adopt our approach for their own uses.

3D Reconstruction and Semantic Modeling of Eyelashes

Glenn Kerbiriou, Quentin Avril, and Maud Marchal

High-fidelity digital human modeling has become crucial in various applications, including gaming, visual effects and virtual reality. Despite the significant impact of eyelashes on facial aesthetics, their reconstruction and modeling have been largely unexplored. In this paper, we introduce the first data-driven generative model of eyelashes based on semantic features. This model is derived from real data by introducing a new 3D eyelash reconstruction method based on multi-view images. The reconstructed data is made available which constitutes the first dataset of 3D eyelashes ever published. Through an innovative extraction process, we determine the features of any set of eyelashes, and present detailed descriptive statistics of human eyelashes shapes. The proposed eyelashes model, which exclusively relies on semantic parameters, effectively captures the appearance of a set of eyelashes. Results show that the proposed model enables interactive, intuitive and realistic eyelashes modeling for non-experts, enriching avatar creation and synthetic data generation pipelines.

ShellNeRF: Learning a Controllable High-resolution Model of the Eye and Periocular Region

Gengyan Li, Kripasindhu Sarkar, Abhimitra Meka, Marcel Buehler, Franziska Mueller, Paulo Gotardo, Otmar Hilliges, and Thabo Beeler

Eye gaze and expressions are crucial non-verbal signals in face-to-face communication. Visual effects and telepresence demand significant improvements in personalized tracking, animation, and synthesis of the eye region to achieve true immersion. Morphable face models, in combination with coordinate-based neural volumetric representations, show promise in solving the difficult problem of reconstructing intricate geometry (eyelashes) and synthesizing photorealistic appearance variations (wrinkles and specularities) of eye performances. We propose a novel hybrid representation – ShellNeRF – that builds a discretized volume around a 3DMM face mesh using concentric surfaces to model the deformable ‘periocular’ region. We define a canonical space using the UV layout of the shells that constrains the space of dense correspondence search. Combined with an explicit eyeball mesh for modeling corneal light-transport, our model allows for animatable photorealistic 3D synthesis of the whole eye region. Using multi-view video input, we demonstrate significant improvements over state-of-the-art in expression re-enactment and transfer for high-resolution close-up views of the eye region.

Region-Aware Simplification and Stylization of 3D Line Drawings

Vivien Nguyen, Matthew Fisher, Aaron Hertzmann, and Szymon Rusinkiewicz

Shape-conveying line drawings generated from 3D models normally create closed regions in image space. These lines and regions can be stylized to mimic various artistic styles, but for complex objects, the extracted topology is unnecessarily dense, leading to unappealing and unnatural results under stylization. Prior works typically simplify line drawings without considering the regions between them, and lines and regions are stylized separately, then composited together, resulting in unintended inconsistencies. We present a method for joint simplification of lines and regions simultaneously that penalizes large changes to region structure, while keeping regions closed. This feature enables region stylization that remains consistent with the outline curves and underlying 3D geometry.

FontCLIP: A Semantic Typography Visual-Language Model for Multilingual Font Applications

Yuki Tatsukawa, I-Chao Shen, Anran Qi, Yuki Koyama, Takeo Igarashi, and Ariel Shamir

Acquiring the desired font for various design tasks can be challenging and requires professional typographic knowledge. While previous font retrieval or generation works have alleviated some of these difficulties, they often lack support for multiple languages and semantic attributes beyond the training data domains. To solve this problem, we present FontCLIP – a model that connects the semantic understanding of a large vision-language model with typographical knowledge. We integrate typographyspecific knowledge into the comprehensive vision-language knowledge of a pretrained CLIP model through a novel finetuning approach. We propose to use a compound descriptive prompt that encapsulates adaptively sampled attributes from a font attribute dataset focusing on Roman alphabet characters. FontCLIP’s semantic typographic latent space demonstrates two unprecedented generalization abilities. First, FontCLIP generalizes to different languages including Chinese, Japanese, and Korean (CJK), capturing the typographical features of fonts across different languages, even though it was only finetuned using fonts of Roman characters. Second, FontCLIP can recognize the semantic attributes that are not presented in the training data. FontCLIP’s dual-modality and generalization abilities enable multilingual and cross-lingual font retrieval and letter shape optimization, reducing the burden of obtaining desired fonts.

Sketch Video Synthesis

Yudian Zheng, Xiaodong Cun, Menghan Xia, and Chi-Man Pun

Understanding semantic intricacies and high-level concepts is essential in image sketch generation, and this challenge becomes even more formidable when applied to the domain of videos. To address this, we propose a novel optimization-based framework for sketching videos represented by the frame-wise Bézier Curves. In detail, we first propose a cross-frame stroke initialization approach to warm up the location and the width of each curve. Then, we optimize the locations of these curves by utilizing a semantic loss based on CLIP features and a newly designed consistency loss using the self-decomposed 2D atlas network. Built upon these design elements, the resulting sketch video showcases notable visual abstraction and temporal coherence. Furthermore, by transforming a video into vector lines through the sketching process, our method unlocks applications in sketch-based video editing and video doodling, enabled through video composition.

Surface-aware Mesh Texture Synthesis with Pre-trained 2D CNNs

Áron Samuel Kovács, Pedro Hermosilla, and Renata Georgia Raidou

Mesh texture synthesis is a key component in the automatic generation of 3D content. Existing learning-based methods have drawbacks—either by disregarding the shape manifold during texture generation or by requiring a large number of different views to mitigate occlusion-related inconsistencies. In this paper, we present a novel surface-aware approach for mesh texture synthesis that overcomes these drawbacks by leveraging the pre-trained weights of 2D Convolutional Neural Networks (CNNs) with the same architecture, but with convolutions designed for 3D meshes. Our proposed network keeps track of the oriented patches surrounding each texel, enabling seamless texture synthesis and retaining local similarity to classical 2D convolutions with square kernels. Our approach allows us to synthesize textures that account for the geometric content of mesh surfaces, eliminating discontinuities and achieving comparable quality to 2D image synthesis algorithms. We compare our approach with state-of-the-art methods where, through qualitative and quantitative evaluations, we demonstrate that our approach is more effective for a variety of meshes and styles, while also producing visually appealing and consistent textures on meshes.

GANtlitz: Ultra High Resolution Generative Model for Multi-Modal Face Textures

Aurel Gruber, Edo Collins, Abhimitra Meka, Franziska Mueller, Kripasindhu Sarkar, Sergio Orts-Escolano, Luca Prasso, Jay Busch, Markus Gross, and Thabo Beeler

High-resolution texture maps are essential to render photoreal digital humans for visual effects or to generate data for machine learning. The acquisition of high resolution assets at scale is cumbersome, it involves enrolling a large number of human subjects, using expensive multi-view camera setups, and significant manual artistic effort to align the textures. To alleviate these problems, we introduce GANtlitz (A play on the german noun Antlitz, meaning face), a generative model that can synthesize multi-modal ultra-high-resolution face appearance maps for novel identities. Our method solves three distinct challenges: 1) unavailability of a very large data corpus generally required for training generative models, 2) memory and computational limitations of training a GAN at ultra-high resolutions, and 3) consistency of appearance features such as skin color, pores and wrinkles in high-resolution textures across different modalities. We introduce dual-style blocks, an extension to the style blocks of the StyleGAN2 architecture, which improve multi-modal synthesis. Our patch-based architecture is trained only on image patches obtained from a small set of face textures (<100) and yet allows us to generate seamless appearance maps of novel identities at 6k×4k resolution. Extensive qualitative and quantitative evaluations and baseline comparisons show the efficacy of our proposed system.

Stylized Face Sketch Extraction via Generative Prior with Limited Data

Kwan Yun, Kwanggyoon Seo, Chang Wook Seo, Soyeon Yoon, Seongcheol Kim, Soohyun Ji, Amirsaman Ashtari, and Junyong Noh

Facial sketches are both a concise way of showing the identity of a person and a means to express artistic intention. While a few techniques have recently emerged that allow sketches to be extracted in different styles, they typically rely on a large amount of data that is difficult to obtain. Here, we propose StyleSketch, a method for extracting high-resolution stylized sketches from a face image. Using the rich semantics of the deep features from a pretrained StyleGAN, we are able to train a sketch generator with 16 pairs of face and the corresponding sketch images. The sketch generator utilizes part-based losses with two-stage learning for fast convergence during training for high-quality sketch extraction. Through a set of comparisons, we show that StyleSketch outperforms existing state-of-the-art sketch extraction methods and few-shot image adaptation methods for the task of extracting high-resolution abstract face sketches.We further demonstrate the versatility of StyleSketch by extending its use to other domains and explore the possibility of semantic editing. The project page can be found in https://kwanyun.github.io/stylesketch_project.

Cinematographic Camera Diffusion Model

Hongda Jiang, Xi Wang, Marc Christie, Libin Liu, and Baoquan Chen

Designing effective camera trajectories in virtual 3D environments is a challenging task even for experienced animators. Despite an elaborate film grammar, forged through years of experience, that enables the specification of camera motions through cinematographic properties (framing, shots sizes, angles, motions), there are endless possibilities in deciding how to place and move cameras with characters. Dealing with these possibilities is part of the complexity of the problem. While numerous techniques have been proposed in the literature (optimization-based solving, encoding of empirical rules, learning from real examples,…), the results either lack variety or ease of control. In this paper, we propose a cinematographic camera diffusion model using a transformer-based architecture to handle temporality and exploit the stochasticity of diffusion models to generate diverse and qualitative trajectories conditioned by high-level textual descriptions. We extend the work by integrating keyframing constraints and the ability to blend naturally between motions using latent interpolation, in a way to augment the degree of control of the designers. We demonstrate the strengths of this text-to-camera motion approach through qualitative and quantitative experiments and gather feedback from professional artists. The code and data are available at https://github.com/jianghd1996/Camera-control.

OptFlowCam: A 3D-Image-Flow-Based Metric in Camera Space for Camera Paths in Scenes with Extreme Scale Variations

Lisa Piotrowski, Michael Motejat, Christian Rössl, and Holger Theisel

Interpolation between camera positions is a standard problem in computer graphics and can be considered the foundation of camera path planning. As the basis for a new interpolation method, we introduce a new Riemannian metric in camera space, which measures the 3D image flow under a small movement of the camera. Building on this, we define a linear interpolation between two cameras as shortest geodesic in camera space, for which we provide a closed-form solution after a mild simplification of the metric. Furthermore, we propose a geodesic Catmull-Rom interpolant for keyframe camera animation. We compare our approach with several standard camera interpolation methods and obtain consistently better camera paths especially for cameras with extremely varying scales.

DivaTrack: Diverse Bodies and Motions from Acceleration-Enhanced 3-Point Trackers

Dongseok Yang, Jiho Kang, Lingni Ma, Joseph Greer, Yuting Ye, and Sung-Hee Lee

Full-body avatar presence is important for immersive social and environmental interactions in digital reality. However, current devices only provide three six degrees of freedom (DOF) poses from the headset and two controllers (i.e. three-point trackers). Because it is a highly under-constrained problem, inferring full-body pose from these inputs is challenging, especially when supporting the full range of body proportions and use cases represented by the general population. In this paper, we propose a deep learning framework, DivaTrack, which outperforms existing methods when applied to diverse body sizes and activities. We augment the sparse three-point inputs with linear accelerations from Inertial Measurement Units (IMU) to improve foot contact prediction. We then condition the otherwise ambiguous lower-body pose with the predictions of foot contact and upper-body pose in a two-stage model. We further stabilize the inferred full-body pose in a wide range of configurations by learning to blend predictions that are computed in two reference frames, each of which is designed for different types of motions. We demonstrate the effectiveness of our design on a large dataset that captures 22 subjects performing challenging locomotion for three-point tracking, including lunges, hula-hooping, and sitting. As shown in a live demo using the Meta VR headset and Xsens IMUs, our method runs in real-time while accurately tracking a user’s motion when they perform a diverse set of movements.

Fast Dynamic Facial Wrinkles

Sebastian Weiss, Prashanth Chandran, Gaspard Zoss, and Derek Bradley  

We present a new method to animate the dynamic motion of skin micro wrinkles under facial expression deformation. Since wrinkles are formed as a reservoir of skin for stretching, our model only deforms wrinkles that are perpendicular to the stress axis. Specifically, those wrinkles become wider and shallower when stretched, and deeper and narrower when compressed. In contrast to previous methods that attempted to modify the neutral wrinkle displacement map, our approach is to modify the way wrinkles are constructed in the displacement map. To this end, we build upon a previous synthetic wrinkle generator that allows us to control the width and depth of individual wrinkles when generated on a per-frame basis. Furthermore, since constructing a displacement map per frame of animation is costly, we present a fast approximation approach using pre-computed displacement maps of wrinkles binned by stretch direction, which can be blended interactively in a shader. We compare both our high quality and fast methods with previous techniques for wrinkle animation and demonstrate that our work retains more realistic details.

FACTS: Facial Animation Creation using the Transfer of Styles

Jack Saunders and Vinay Namboodiri

The ability to accurately capture and express emotions is a critical aspect of creating believable characters in video games and other forms of entertainment. Traditionally, this animation has been achieved with artistic effort or performance capture, both requiring costs in time and labor. More recently, audio-driven models have seen success, however, these often lack expressiveness in areas not correlated to the audio signal. In this paper, we present a novel approach to facial animation by taking existing animations and allowing for the modification of style characteristics. We maintain the lip-sync of the animations with this method thanks to the use of a novel viseme-preserving loss. We perform quantitative and qualitative experiments to demonstrate the effectiveness of our work.

Skeleton-Aware Skin Weight Transfer for Helper Joint Rigs

Cao Ziyuan and Tomohiko Mukai

We propose a method to transfer skin weights and helper joints from a reference model to other targets. Our approach uses two types of spatial proximity to find the correspondence between the target vertex and reference mesh regions. The proposed method first generates a guide weight map to establish a relationship between the skin vertices and skeletal joints using a standard skinning technique. The correspondence between the reference and target skins is established using vertex-to-bone projection and bone-to-skin ray-casting using the guide weights. This method enables fully automated and smooth transfer of skin weight between human-like characters bound to helper joint rigs.

Modern Dance Retargeting using Ribbons as Lines of Action

Manon Vialle, Rémi Ronfard, and Melina Skouras

We present a method for retargetting dancing characters represented as articulated skeletons with possibly different morphologies and topologies. Our approach relies on the use of flexible ribbons that can bend and twist as an intermediate representation, and that can be seen as animated lines of action. These ribbons allow us to abstract away the specific morphology of the bodies and to well transmit the fluidity of modern dance movement from one character to another.

Utilizing Motion Matching with Deep Reinforcement Learning for Target Location Tasks

Jeongmin Lee, Taesoo Kwon, Hyunju Shin, and Yoonsang Lee

We present an approach using deep reinforcement learning (DRL) to directly generate motion matching queries for longterm tasks, particularly targeting the reaching of specific locations. By integrating motion matching and DRL, our method demonstrates the rapid learning of policies for target location tasks within minutes on a standard desktop, employing a simple reward design. Additionally, we propose a unique hit reward and obstacle curriculum scheme to enhance policy learning in environments with moving obstacles.

StarDEM: Efficient Discrete Element Method for Star-shaped Particles

Camille Schreck, Sylvain Lefebvre, David Jourdan, and Jonàs Martínez

Granular materials composed of particles with complex shapes are challenging to simulate due to the high number of collisions between the particles. In this context, star shapes are promising: they cover a wide range of geometries from convex to concave and have interesting geometric properties. We propose an efficient method to simulate a large number of identical star-shaped particles. Our method relies on an effective approximation of the contacts between particles that can handle complex shapes, including highly non-convex ones. We demonstrate our method by implementing it in a 2D simulation using the Discrete Element Method, both on the CPU and GPU.

Accurate Boundary Condition for Moving Least Squares Material Point Method using Augmented Grid Points

Riku Toyota and Nobuyuki Umetani

This paper introduces an accurate boundary-handling method for the moving least squares (MLS) material point method (MPM), which is a popular scheme for robustly simulating deformable objects and fluids using a hybrid of particle and grid representations coupled via MLS interpolation. Despite its versatility with different materials, traditional MPM suffers from undesirable artifacts around wall boundaries, for example, particles pass through the walls and accumulate. To address these issues, we present a technique inspired by a line handler for MLS-based image manipulation. Specifically, we augment the grid by adding points along the wall boundary to numerically compute the integration of the MLS weight. These additional points act as background grid points, improving the accuracy of the MLS interpolation around the boundary, albeit with a marginal increase in computational cost. In particular, our technique makes the velocity perpendicular to the wall nearly zero, preventing particles from passing through the wall. We compare the boundary behavior of 2D simulation against that of naïve approach.

Emotional Responses to Exclusionary Behaviors in Intelligent Embodied Augmented Reality Agents

Kalliopi Apostolou, Vaclav Milata, Filip Škola, and Fotis Liarokapis

This study investigated how interactions with intelligent agents, embodied as augmented reality (AR) avatars displaying exclusionary behaviors, affect users’ emotions. Six participants engaged using voice interaction in a knowledge acquisition scenario in an AR environment with two ChatGPT-driven agents. The gaze-aware avatars, simulating realistic body language, progressively demonstrated social exclusion behaviors. Although not statistically significant, our data suggest a post-interaction emotional shift, manifested by decreased positive and negative affect–aligning with previous studies on social exclusion. Qualitative feedback revealed that some users attributed the exclusionary behavior of avatars to system glitches, leading to their disengagement. Our findings highlight challenges and opportunities for embodied intelligent agents, underscoring their potential to shape user experiences within AR, and the broader extended reality (XR) landscape.

An Inverse Procedural Modeling Pipeline for Stylized Brush Stroke Rendering

Hao Li, Zhongyue Guan, and Zeyu Wang

Stylized brush strokes are crucial for digital artists to create drawings that express a desired artistic style. To obtain the ideal brush, artists need to spend much time manually tuning parameters and creating customized brushes, which hinders the completion, redrawing, or modification of digital drawings. This paper proposes an inverse procedural modeling pipeline for predicting brush parameters and rendering stylized strokes given a single sample drawing. Our pipeline involves patch segmentation as a preprocessing step, parameter prediction based on deep learning, and brush generation using a procedural rendering engine. Our method enhances the overall experience of digital drawing recreation by empowering artists with more intuitive control and consistent brush effects.

Driller: An Intuitive Interface for Designing Tangled and Nested Shapes

Tara Butler, Pascal Guehl, Amal Dev Parakkat, and Marie-Paule Cani

The ability to represent not only isolated shapes but also shapes that interact is essential in various fields, from design to biology or anatomy. In this paper, we propose an intuitive interface to control and edit complex shape arrangements. Using a set of pre-defined shapes that may intersect, our “Driller” interface allows users to trigger their local deformation so that they rest on each other, become tangled, or even nest within each other. Driller provides an intuitive way to specify the relative depth of different shapes beneath user-selected points of interest by setting their local depth ordering perpendicularly to the camera’s viewpoint. Deformations are then automatically generated by locally propagating these ordering constraints. In addition to being part of the final arrangement, some of the shapes can be used as deformers, which can be later deleted to help sculpt the target shapes. We implemented this solution within a sketch-based modeling system designed for novice users.

Real-time Seamless Object Space Shading

Tianyu Li and Xiaoxin Guo

Object space shading remains a challenging problem in real-time rendering due to runtime overhead and object parameterization limitations. While the recently developed algorithm by Baker et al. [BJ22] enables high-performance real-time object space shading, it still suffers from seam artifacts. In this paper, we introduce an innovative object space shading system leveraging a virtualized per-halfedge texturing schema to obviate excessive shading and preclude texture seam artifacts. Moreover, we implement ReSTIR GI on our system (see Figure 1), removing the necessity of temporally reprojecting shading samples and improving the convergence of areas of disocclusion. Our system yields superior results in terms of both efficiency and visual fidelity.

A Highly Adaptable and Flexible Rendering Engine by Minimum API Bindings

Taejoon Kim

This paper presents a method for embedding a rendering engine into different development environments with minimal API bindings. The method separates the engine interfaces into two levels: System APIs and User APIs. System APIs are the lowlevel functions that enable communication between the engine and the user environment, while User APIs are the high-level functions that provide rendering and beyond rendering functionalities to the user. By minimizing the number of System APIs, the method simplifies the adaptation of the engine to various languages and platforms. Its applicability and flexibility are demonstrated by the successful embedding the engine in multiple environments, including C/C++, C#, Python, Javascript, and Matlab. It also demonstrates its versatility in diverse forms such as CLI renderers, Web GUI framework-based renderers, remote renderers, physical simulations, and more, while also enabling the easy adoption of other rendering algorithms to the engine.

A Fresnel Model for Coated Materials

Hannes B. Vernooij

We propose a novel analytical RGB model for rendering coated conductors, which provides improved accuracy of Fresnel reflectance in BRDFs. Our model targets real-time path tracing and approximates the Fresnel reflectance curves with noticeably more accuracy than Schlick’s approximation using Lazanyi’s error compensation term and the external media adjustment. We propose an analytical function with coefficients fitted to measured spectral datasets describing the complex index of refraction for conductors. We utilize second-order polynomials to fit the model, subsequently compressing the fitted coefficients to optimize memory requirements while maintaining quality. Both quantitative and visual results affirm the efficacy of our model in representing the Fresnel reflectance of the tested conductors.

Neural Moment Transparency

Grigoris Tsopouridis, Andreas-Alexandros Vasilakis, and Ioannis Fudos

We have developed a machine learning approach to efficiently compute per-fragment transmittance, using transmittance composed and accumulated with moment statistics, on a fragment shader. Our approach excels in achieving superior visual accuracy for computing order-independent transparency (OIT) in scenes with high depth complexity when compared to prior art.

A Visual Profiling System for Direct Volume Rendering

Max von Buelow, Daniel Ströter, Arne Rak, and Dieter W. Fellner

Direct Volume Rendering (DVR) is a crucial technique that enables interactive exploration of results from scientific computing or computer graphics. Its applications range from virtual prototyping for product design to computer-aided diagnosis in medicine. Although there are many existing DVR optimizations, they do not provide a thorough analysis of memory-specific hardware behavior. This paper introduces a profiling toolkit that enables the extraction of performance metrics, such as cache hit rates and branching, from a compiled GPU-based DVR application. The metrics are visualized in the image domain to facilitate spatial visual analysis. This paper presents a pipeline that automatically extracts memory traces using binary instrumentation, simulates the GPU memory subsystem, and models DVR-specific functionality within it. The profiler is demonstrated using the Octree-Linear Bounding Volume Hierarchy (OLBVH), and the visualized profiling metrics are explained based on the OLBVH implementation. Our discussion demonstrates that optimizing ray traversal for adaptive sampling, cache usage, branching, and global memory access has the potential to improve performance.

A Generative Approach to Light Placement for Street Lighting

Iordanis Evangelou, Nick Vitsas, Georgios Papaioannou, and Anastasios Gkaravelis

The design of plausible and effective street lighting configurations for arbitrary urban sites should attain predetermined illuminance levels and adhere to specific layout intentions and functional requirements. This task can be time consuming, even for automated solutions, since there exists an one-to-many mapping between illumination goals and lighting options. In this work, we propose a generative approach for this task, based on an adversarial optimisation scheme. Our proposed method effectively overcomes these task-specific limitations by providing a range of viable solutions that adhere to the input constraints and can be generated within an interactive design life cycle.

3D Reconstruction from Sketch with Hidden Lines by Two-Branch Diffusion Model

Yuta Fukushima, Anran Qi, I-Chao Shen, Yulia Gryaditskaya, and Takeo Igarashi

We present a method for sketch-based modelling of 3D man-made shapes that exploits not only the commonly considered visible surface lines but also the hidden lines typical for technical drawings. Hidden lines are used by artists and designers to communicate holistic shape structure. Given a single viewpoint sketch, leveraging such lines allows us to resolve the ambiguity of the shape’s surfaces hidden from the observer. We assume that the separation into visible and hidden lines is given, and focus solely on how to leverage this information. Our strategy is to mingle two distinct diffusion networks: one generates denoized occupancy grid estimates from a visible line image, whilst the other generates occupancy grid estimates based on contextualized hidden lines unveiling the occluded shape structure. We iteratively merge noisy estimates from both models in a reverse diffusion process. Importantly, we demonstrate the importance of what we call a contextualized hidden lines image over just a hidden lines image. Our contextualized hidden lines image contains hidden lines and silhouette lines. Such contextualization allows us to achieve superior performance to a range of alternative configurations and reconstruct hidden holes and hidden surfaces.

Efficient and Accurate Multi-Instance Point Cloud Registration with Iterative Main Cluster Detection

Zhiyuan Yu, Zheng Qin, Chenyang Zhu, Kai Xu  

Multi-instance point cloud registration is the problem of recovering the poses of all instances of a model point cloud in a scene point cloud. A traditional solution first extracts correspondences and then clusters the correspondences into different instances. We propose an efficient and robust method which clusters the correspondences in an iterative manner. In each iteration, our method first computes the spatial compatibility matrix between the correspondences, and detects its main cluster. The main cluster indicates a potential occurrence of an instance, and we estimate the pose of this instance with the correspondences in the main cluster. Afterwards, the correspondences are removed to further register new instances in the following iterations. With this simplistic design, our method can adaptively determine the number of instances, achieving significant improvements on both efficiency and accuracy.

DeepIron: Predicting Unwarped Garment Texture from a Single Image

Hyunsong Kwon and Sung-Hee Lee

Realistic reconstruction of 3D clothing from an image has wide applications, such as avatar creation and virtual try-on. This paper presents a novel framework that reconstructs the texture map for 3D garments from a single garment image with pose. Since 3D garments are effectively modeled by stitching 2D garment sewing patterns, our specific goal is to generate a texture image for the sewing patterns. A key component of our framework, the Texture Unwarper, infers the original texture image from the input garment image, which exhibits warping and occlusion of the garment due to the user’s body shape and pose. This is effectively achieved by translating between the input and output images by mapping the latent spaces of the two images. By inferring the unwarped original texture of the input garment, our method helps reconstruct 3D garment models that can show high-quality texture images realistically deformed for new poses. We validate the effectiveness of our approach through a comparison with other methods and ablation studies.

SPnet: Estimating Garment Sewing Patterns from a Single Image of a Posed User

Seungchan Lim, Sumin Kim, and Sung-Hee Lee

This paper presents a novel method for reconstructing 3D garment models from a single image of a posed user. Previous studies that have primarily focused on accurately reconstructing garment geometries to match the input garment image may often result in unnatural-looking garments when deformed for new poses. To overcome this limitation, our work takes a different approach by inferring the fundamental shape of the garment through sewing patterns from a single image, rather than directly reconstructing 3D garments. Our method consists of two stages. Firstly, given a single image of a posed user, it predicts the garment image worn on a T-pose, representing the baseline form of the garment. Then, it estimates the sewing pattern parameters based on the T-pose garment image. By simulating the stitching and draping of the sewing pattern using physics simulation, we can generate 3D garments that can adaptively deform to arbitrary poses. The effectiveness of our method is validated through ablation studies on the major components and a comparison with other methods.

An Overview of Teaching a Virtual and Augmented Reality Course at Postgraduate Level for Ten Years

Bernardo Marques, Beatriz Sousa Santos, and Paulo Dias

In recent years, a multitude of affordable sensors, interaction devices, and displays have entered the market, facilitating the adoption of Virtual and Augmented Reality (VR/AR) in various areas of application. However, the development of such applications demands a solid grasp of the field and specific technical proficiency often missing from existing Computer Science and Engineering education programs. This work describes a post-graduate-level course being taught for the last ten years to several Master’s Degree programs, aiming to introduce students to the fundamental principles, methods, and tools of VR/AR. The course’s main objective is to equip students with the necessary knowledge to comprehend, create, implement, and assess applications using these technologies. This paper provides insights into the course structure, the key topics covered, assessment, as well as the devices, and infrastructure utilized. It also includes a brief overview of various sample practical projects, along the years. Among other reflections, we argue that teaching this course is challenging due to the fast evolution of the field making updating paramount. This maybe alleviated by motivating students to a research oriented approach, encouraging them to bring their own projects and challenges (e.g. related to their Master dissertations). Finally, future perspectives are outlined.

Bridging the Distance in Education: Design and Implementation of a Synchronous, Browser-Based VR Remote Teaching Tool

Abdulmelik Pehlic and Ursula Augsdörfer

The rapid shift to remote education has presented numerous challenges for educators and students alike. Virtual Reality (VR) has emerged as a promising solution, offering immersive and interactive learning experiences. We design and implement a synchronous, browser-based VR teaching tool. The tool is compatible with budget VR equipment and enables meaningful engagement between teachers and students in a virtual setting, as well as active participation and interaction across a range of platforms, thus solving a range of disadvantages of current approaches.

Holistic Approach to Modular Open Educational Resources for Computer Graphics

Florian Diller, Fabian Püschel, Julian Stockemer, Klaus Böhm, and Alexander Wiebel

In this paper, we present a novel holistic approach to open education resources (OER) in computer graphics (CG). Holistic on the one hand refers to the diversity and the interlinked integration of the materials we included in the devised educational modules, on the other hand, it corresponds to the public availability of our work, which manifests in various aspects. Each of our three-part educational modules consists of slides, an experiential education web application (“exploratory”), and a quiz for knowledge assessment. These materials are closely interlinked, as they each teach the same topic from different perspectives and refer to each other. Yet, the resources can be used independently to add additional value to various teaching styles and programs. This modularity and the overall public availability were one of the primary goals of our project. Consequently, every resource is licensed under Creative Commons and developed using open standards or freely usable tools, so that teaching staff from universities and schools can modularly include, adapt, and build upon our resources. The OER aspect is not only reflected by licensing; the complete publishing, design, and development process was formed to serve the general public. Our work is based on an experiential learning approach and does not analyze the pedagogical methodology. Instead, in addition to presenting the devised learning materials, the present paper investigates to what extent the learning materials can support teaching and learning computer graphics. We took several measures to evaluate our approach: 1) teaching staff was interviewed regarding the usability and acceptance of the technologies we used; 2) a usability expert was consulted to assess our system; 3) the developed resources were integrated into existing lectures and the performance of students with and without the assistance of the interactive education applications was compared. The responses to our approach were exclusively positive

Can GPT-4 Trace Rays

Tony Haoran Feng, Burkhard C. Wünsche, Paul Denny, Andrew Luxton-Reilly, and Steffan Hooper

Ray Tracing is a fundamental concept often taught in introductory Computer Graphics courses, and Ray-Object Intersection questions are frequently used as practice for students, as they leverage various skills essential to learning Ray Tracing or Computer Graphics in general, such as geometry and spatial reasoning. Although these questions are useful in teaching practices, they may take some time and effort to produce, as the production procedure can be quite complex and requires careful verification and review. From the recent advancements in Artificial Intelligence, the possibility of automated or assisted exercise generation has emerged. Such applications are unexplored in Ray Tracing education, and if such applications are viable in this area, then it may significantly improve the productivity and efficiency of Computer Graphics instructors. Additionally, Ray Tracing is quite different to the mostly text-based tasks that LLMs have been observed to perform well on, hence it is unclear whether they can cope with these added complexities of Ray Tracing questions, such as visual processing and 3D geometry. Hence we ran some experiments to evaluate the usefulness of leveraging GPT-4 for assistance when creating exercises related to Ray Tracing, more specifically Ray-Object Intersection questions, and we found that an impressive 67% of its generated questions can be used in assessments verbatim, but only 42% of generated model solutions were correct.

The Use of Photogrammetry in Historic Preservation Curriculum: A Comparative Case Study

Anetta Kepczynska-Walczak, Bartosz M. Walczak, and Andrzej Zarzycki

Computer graphic techniques have emerged as a key player in digital heritage preservation and its dissemination. Photogrammetry allows for high-fidelity captures and virtual reconstructions of the built environment that can be further ported into virtual reality (VR) and augmented reality (AR) experiences. This paper provides a comparative analysis of historic details and building documentation methods in heritage preservation in the context of architectural education. Specifically, it compares two educational case studies conducted in 10-year intervals documenting the same set of historic artifacts with corresponding state-of-the-art digital technologies. The methodology for this paper is a qualitative comparative analysis of two surveying projects that utilized distinct emerging digital technology while sharing the same study subjects and similar tool-driven curricular framework. The research also incorporates a student survey, offering perspectives on teaching strategies and outcomes within this dynamic educational context. The outcomes demonstrate that the technological (tool-driven) shift impacts the way students interact with the investigated artifacts and the changing role of the interpretative versus analytical skills needed to delineate the work. It also changes what are considered primary and secondary knowledge sources.

Approaches to Nurturing Undergraduate Research in the Creative Industries – a UK Multi-Institutional Exploration

Eike Falk Anderson, Leigh McLoughlin, Oliver Gingrich, Emmanouil Kanellos, and Valery Adzhiev

Undergraduate students aspiring to pursue careers in the creative industries, such as animation, video games, and computer art, require the ability to adapt and contribute to emerging and disruptive technologies. The cultivation of research skills fosters this adaptability and innovation, which is why research skills are considered important by employers. Promoting undergraduate research in computer graphics and related techniques is therefore necessary to ensure that students graduate not only with the vocational but also with the advanced research skills desired by the creative industries. This paper describes pedagogical approaches to nurturing undergraduate research across teaching, learning and through extracurricular activities – pioneered at three UK Higher Education Institutions. Providing observations, we are sharing educational strategies – reflecting on pedagogic experiences of supporting undergraduate research projects, many of which are practice-based. With this paper, we aim to contribute to a wider discussion around challenges and opportunities of student-led research.

A Research Methodology Course in a Game Development Curriculum

Yan Hu, Veronica Sundstedt, and Prashant Goswami

Research methodology courses can often be considered part of a computer science curriculum. These basic or advanced-level courses are taught in terms of traditional research methods. This paper presents and discusses a research methodology course curriculum for students studying programs focusing on digital game development (more specifically, focusing on game engineering). Our research methodology course prepares students for their upcoming thesis by encouraging a research-oriented approach. This is done by exploring new research areas in game engineering as a basis for research analysis and by applying research methods practically in a smaller project. This paper presents the course structure, assignments, and lessons learned. Together with existing literature, it demonstrates important aspects to consider in teaching and learning game research methodologies. The course evaluation found that the students appreciated the interactive lectures, close staff supervision, and detailed feedback on the scientific writing process.

Tackling Diverse Student Backgrounds and Goals while Teaching an Introductory Visual Computing Course at M.Sc. Level

Samuel Silva

Visual Computing entails a set of competences that are core for those pursuing Digital Game Development and has become a much sought competence for professionals in a wide variety of fields. In the particular case presented here, the course serves a diverse audience from Multimedia and Design students without previous knowledge in the field and low programming competences, to students that have a previous BS.c in Game Development and have already covered the basic concepts in a previous course. Additionally, the course is also offered as an elective for Computer Science M.Sc. students. This diverse set of background competences and goals motivated designing an approach to the course where each student can build on previous knowledge and have a say on its personal learning path. This article shares the overall approach, presents and discusses the outcomes, and reflects on future evolutions.

Gaming to Learn: A Pilot Case Study on Students Acceptance of Playing Video Games as a Learning Method

Louis Nisiotis

This paper presents a case study on playing video games as a method to support the delivery of a game development University module, describing the teaching methodology and presenting details on a ’gaming‘ for learning approach to support the module’s learning objectives. It presents the formulation of a theoretical framework to evaluate students acceptance of playing video games as a learning method, and the results of a pilot study using a modified Technology Acceptance Model. The results revealed that gaming as a learning activity was positively perceived by students, finding this method engaging and relevant to their learning curriculum, playful, enjoyable, useful, easy to use, with positive attitudes and behavioural intentions to use. This pilot case study serves as a practical example of implementing video games to support learning, preparing the methodology for further research to understand students acceptance, and the effect on learning outcomes and knowledge acquisition.

Teaching Game Programming in an Upper-level Computing Course Through the Development of a C++ Framework and Middleware

Steffan Hooper, Burkhard C. Wünsche, Paul Denny, and Andrew Luxton-Reilly

The game development industry has a programming skills shortage, with industry surveys often ranking game programming as the top skill-in-demand across small, mid-sized, and large triple-A (AAA) game studios. C++ programming skills are desired, however, educators can perceive C++ as too difficult to teach due to its size and complexity. We address the challenges of teaching C++ in an upper-level Game Programming course and demonstrate how learners are up-skilled in C++ game programming, providing insights and reflections on the course. We show how through careful educational-design choices, combined with scaffolding a C++ framework and contemporary middleware, it is possible to transition learners to C++ for game programming.

Preserving Cultural Heritage: An Outstanding Students Digital Game Project On Lusíada Art

Roberto Aguiar Ribeiro and Alexandrino Gonçalves

This paper presents an outstanding undergraduate project that resulted from the development of a digital game focusing on cultural heritage and the value of works of art. This topic is addressed through the interaction with reliable representations of Portuguese Lusíada Art, a product of the cultural exchange between Portugal and many countries in Asia, during the Age of Exploration, during the 16th and 17th centuries. The resulting project provides an interactive and entertaining means to learn about the History of Art, by distinguishing different types of Lusíada Furniture.

CS2023: An Update on the 2023 Computer Science Curricular Guidelines (PRESENTATION)

Susan L. Reiser

In early 2024, the 2023 Computer Science Curricular Guidelines (CS2023) were endorsed by their sponsoring organizations: the Association of Computing Machinery (ACM), IEEE Computing Society, and the Association for the Advancement of Artificial Intelligence (AAAI). The CS2023 effort spanned four years and was the collaborative work of over 100 volunteers from six continents. The Eurographics’ education community provided valuable feedback on the guidelines in its draft phase. In this session we would like to present a summary of the guidelines and seek feedback on its adoption and goal of being a living curriculum. The session is geared to anyone interested in computer science education. (see https://csed.acm.org/ cs2023-report-with-feedback/)