Perception-Aware Appearance Fabrication

Wouldnt it be amazing to be able to mirror how an object looks? We could create appealing products, preserve cultural heritage, or create prosthetics undistinguishable from genuine body parts. One of the options to fabricate such objects is 3D printing. 3D printers are good at reproducing complex shapes, and some even print with multiple colors. Unfortunately, accurately reproducing the hue, saturation, and gloss does not work well yet. The main hurdle for printing objects with desired looks is how do we evaluate how an object looks like? We have devices that can measure individual attributes of an object. We can measure either objects shape, color, or gloss. However, when we see an object, we do not observe these attributes separately. Instead, we see the entire object at once, and the shape, color, and gloss combine into a single feeling. Currently, it is unknown how this combination is done and how individual attributes play together. Therefore, the main objective of this work is to investigate how humans perceive the difference in looks. To understand how we see objects looks, we propose a workflow driven by human perception. We start by investigating how humans interpret the looks of an object. To conduct this investigation, we need to create representative objects with different shapes, colors, and glosses. Next, we will invite people to examinate the objects and tells us how big the difference in the objects looks is. We will collect this information and use it to construct a numerical model that quantifies the look of an object. Such a model will look at two objects and tell us the visible difference between them as a number. The larger the number, the larger difference. Thanks to this property, we can directly use our model to improve the fabrication of objects looks. Given the desired look of an object, we can adjust the printing parameters until our model would not see a difference. By minimizing the perceived difference, we can print objects much closer to their real-world counterparts.

Wouldn't it be amazing to be able to mirror how an object looks? We could create appealing products, preserve cultural heritage, or create prosthetics undistinguishable from genuine body parts. One of the options to fabricate such objects is 3D printing. 3D printers are good at reproducing complex shapes, and some even print with multiple colors. Unfortunately, accurately reproducing the hue, saturation, and gloss does not work well yet. The main hurdle for printing objects with desired looks is how do we evaluate how an object looks like? We have devices that can measure individual attributes of an object. We can measure either object's shape, color, or gloss. However, when we see an object, we do not observe these attributes separately. Instead, we see the entire object at once, and the shape, color, and gloss combine into a single feeling. Currently, it is unknown how this combination is done and how individual attributes play together. Therefore, the main objective of this work is to investigate how humans perceive the difference in looks and how we can leverage these insights in manufacturing. To examine human perception of gloss we designed a display that could produce images so realistic that they were indistinguishable from real objects. With the display we could turn on and off individual components of gloss. We found that the key attribute to match virtual and real objects is accurate reproduction of the dynamic range, i.e. matching the strong highlights and the deep blacks of the real world. With these findings we designed a study that investigated the joined perception of color and gloss. The study revealed a simple, yet intuitive effect. The higher the gloss, the more saturated the color. We combined these insights into a single color-gloss management tool. Our tool automatically handles the saturation change between glossy finishes and produces a single consistent appearance. To deploy our algorithm we also require a manufacturing technology. We proposed a novel tattooing-based system that applies inks in silicon media with needle injections. The system is capable of accurately reproducing human skin tones and creating silicon prosthetics that mimic genuine body parts. To achieve such high fidelity with output we need fine control of the fabrication process. To this end, we developed a self-adjusting printing technology. We first create a digital twin of the printing device on a computer. By interacting with the digital printer a computer algorithm learns effective control strategies that can be applied to the real physical device. We showcased our full pipeline from design tool to production by manufacturing several prototypes from colored printouts to tattooed skin-like silicone sheets.