Abe's Art+Code Journey // 48h Neukölln // July 2024
I prepped a new projection-mapping sculpture for the 48h-Neukölln art event, battling tech bugs amidst the chaos.
Every summer since 1999, a big art event takes place in Neukölln, the neighborhood where I live. It starts on a Friday afternoon and ends on the following Sunday, for a total of 48 hours. Hundreds of studios open to the public to present their art and performances.
Prachtsaal Studio, where I have worked since the beginning of 2024, successfully applied for a grant. Only five such grants were awarded.
The weeks leading to the opening were hectic. We rearranged furniture, installed lights, projectors, hung artworks on the walls, developed software, prepared a table with artworks for sale, updated the website, printed information sheets... Transforming our space to make it more public felt nice.
Steps of Impermanence
Visiting a friend's underground stone cave multiple times, and even projecting my real-time visuals in it, made me realize that non-flat surfaces feel very special to our senses.
I wanted to reproduce that feeling in my artwork and wondered what kind of uneven surface I could create. A full 3D sculpture would have been too time-demanding, and the result would have probably been too heavy to transport.
I settled for a construction made out of wood and carton board. The background is made out of egg trays with a wood skeleton to hold them in place. The foreground is a crooked tree trunk I found in the studio's garden. Foreground and background are both gray and not so different.
My friend and studio colleague Claudine helped me design and build the sculpture using glue, metal rods and screws.
I hung my short-throw laser projector from the ceiling, pointing at the sculpture, and built a small platform to place the computer and its power supply near the ceiling, out of reach of anyone.
I placed a Kinect 3D camera to be able to track people's movements and make the whole setup interactive, but I never got time to work on that part.
Once I turned the projector on, I realized its light was creating a brighter rectangle on the wall, even outside the sculpture. I didn't like how it looked, so I bought black paint and applied it to the wall. This way, the leaked light was barely visible.
Next came mapping time. See, I wanted my visuals to react to the uneven surface, to the shape of the tree trunk, to the elevations of the egg trays.
My first approach to mapping involved creating a 3D model of the sculpture using photogrammetry. I shot a video moving slowly around the 3D shape, trying to record every bump and every crack. I loaded the video into a program called Meshroom, but it failed to make use of all the photos I provide. So I paid for a subscription to polycam and uploaded the video shot with my phone. This returned a very good 3D model. But time was running out, and I didn't have a clear idea about how to adjust a virtual camera to perfectly map the 3D model. So what I did instead was to grab my graphics tablet on one hand, the pen on the other, and spend a few hours painting the contour of the tree and every bump on the egg trays. This was a fun process for me, although not that easy. Sometimes I had to move the mouse cursor pixel by pixel to be very precise. See, sometimes a pixel was partially on the tree and on the background.
Once I had 3 layers (the tree, the contour of the background, and the egg-tray bumps) I wrote an OPENRNDR program to calculate a texture in which the colors indicate the distance of every pixel to the nearest edge, and also the angle pointing at the nearest edge. Such a texture would allow me to make visuals that react to the contours of shapes, to produce particles at the bumps, or to make them rotate around the bumps.
So how did the event go? Not as smooth as I wished. I would have really benefited from having more time to test and polish the software.
It is time to open, and a weird bug eats my particles block by block. It’s as if my visuals are made of squares, and the squares fail one by one, starting at the bottom left.
Every few seconds, another square becomes black, and the frame rate decreases. I suspect the particles are falling into tiny buckets, increasing the number of calculations.
What should I do? People are coming in, and my installation is behaving as a failing, unstable system.
The first time it happens, I restart the computer (remember, it is hanging from the ceiling). The second time, I do the same. I need a better approach, so I enable sshd to access the computer remotely from my desk, which is 20 meters away from the installation.
So here we are, with an unstable installation and me walking to my desk every time I see things going wrong. Sometimes that's once per hour, other times it's four times per hour. Not fun.
I want to figure out an actual solution, but with all the people coming in and out, and keeping the installation alive, there's no way to think deeply about what is happening.
So I do one more workaround: I implement a web server, so I can restart the installation directly from my phone, instead of walking to my desk. That's as far as I go towards improving the software.
I suffer from the semi-working artwork and the fact that I did not have time to make the simulation react to the shapes and depths of the sculpture. But visitors still enjoy what they see, and they even sit and stare at the work for a long time. So I comfort myself by thinking that the 48h Neukölln was the motivation I needed to build the first prototype, and soon I will have a more polished work to show everyone at my solo exhibition.
Two months later…
I’ve spent several days last week and this week trying to solve the mystery. It has been one of the most unpleasant bugs I’ve encountered. The reason is that it was not reproducible. Consistently encountering the incorrect behavior makes it easier to fix. But if the issue only sometimes appears, one may believe the problem is solved when it’s not and even turn to superstition :-) The good news is that for several days now my visuals are not disappearing.
The not-so-good news is that, unfortunately, I do not know why. I could try to rewind the project to its old state, see if the problem is there, and then compare the old version to the current one to try understand which change fixed it. I do not have time for that right now, but I’ll try to do this in the future.
Now it’s time to keep moving forward and get my first (computational art) solo exhibition ready. It will take place on November 22–24 at Johan Lorbeer Art Space, on Maxstraße 21, 13347 Berlin. See you there maybe?
I wish you a wonderful Autumn / Spring, depending on your current hemisphere :-)
aBe PaZoS SoLaTie
Art & Code