Art and Everything Else

matteland project

So I thought I’d consolidate 2 posts into 1. I have more things to blog about later so I want to make this as a reflection.

First of all, matteland project, the project involving experiments into digitally painted / projection mapped environments for character animation.

Matteland project has been a big feature, even if I took the effort almost like my side-job (HAVOC was a bigger thing for me.) Either way the goal of Matteland was to bring a surreal 3D experience in 2D paintings, especially as an background for animated films. While the 3D part wasn’t THAT hard, what took longer was the 2D painting. This involved manipulating photographs using Photoshop to painting them in Studio Artist 4. It takes a bit long, since I did go about developing some techniques I never tried before, but ultimately it doesn’t look bad. Overall, I painted 5 backgrounds which 3 of them were used to create simple 3D meshes for projection purposes.

Of course, this project has 1 purpose. I’ve recently began planning my thesis project, where I do want some characters interacting in the environment. So that’s ultimately been tested out here.

Since this project began there has been several reactions one in particular was unexpected. John Dalton, the creator of Studio Artist 4 saw some potential in the senior thesis I’m headed to do. He ended up writing stuff about the whole thing. Take a look at the article here: Studio Artist News: Matteland and The Door to Tomorrow”. That’s pretty cool. I’ve been looking at Studio Artist since version 1.5 when it was first introduced to the Japanese Mac community via Macworld Japan (or something like that.) Either way, since then, I’ve always wanted the program personally. Either way, Studio Artist 4 has been the biggest part of the project and will continue to be one of the primary painting tools I’ll be using for my other projects. So stay tuned if you’re interested.

HAVOC part 1: Havoc part 1: What Your Dreams are Made of received much attention for being what it is. I’ve been wanting to do HAVOC for quite some time, and I’ve finally gotten my chance to do something about it.

HAVOC was too large though. It involved everything from recording voices (my voice), making sounds, timing things, animating, editing, making assets etc etc etc…. Being THAT huge, I think 10 weeks was simply not enough for the whole project. Somethings ended up sadly not done. I had to cheat a bit too, which isn’t always the best but it does give me a result.

Either way once I start blabing about Havoc, it never ends so here’s the final product.

So my overall on both projects. How about we’re done for now but not exactly a full stop. Of course! I’m still doing stuff to them! We’re moving on though, since if we just keep going on and on about the same thing, yea sure the thing gets better but I can’t work on my other things.

That’s it for now. The next project: The Door to Tomorrow. Keep your hopes up for that.

So I’ve been working gently over the last couple days. I’ve been slightly unsatisfied with some results which made me go back and forth between multiple things. Lets talk about some steps to ensure a nice matte to paint on. I first of all realize I’m doing a very rough thing here, but we’ll see how this plays out later as I start painting using Studio Artist 4.03. I decided I should show the steps I’m going through. It’s not much of an example, but I thought it would be nice to be able to explain what I’m doing / and up to right now.

I also wanted to make this blog post probably like a tutorial. I began doing stuff like this not necessarily recently, up until this point I used Photoshop to mainly “paint” or filter photographs for background usage. So far, I ended up learning some of the more interesting aspects of painting through the aid of an computer. That’s what I’m doing here. It’s really not just “hey lets turn a 2D photo into 3D” it’s “hey, lets take that photo, manipulate it a bit, paint on it and make it look fun to look at, take it into 3D, and lets translate it in 3D space!” Maybe I should have said that first. But anyway, that’s my project basically.

Now, let me say that this is unlike the earlier things I’ve done. You can see my earlier experiments here: https://skybase.wordpress.com/2010/10/29/its-been-a-long-time/. On this blog post, which I did a long time ago, I did use Studio Artist 4. But you know what? I changed my workflow completely since then. Now it’s better and improved. So just some differentiation I wanted to point out before we get deeper.

So step 1: Look at the picture, if any thing is in the foreground that would get affected by the parallax, get rid of it somehow. Below, I selected the things that produced an obnoxious problem when projecting the image. I then began refining the edges for a rough delete. It does not have to be perfect in my case. I just need something to fill the space so I can paint on it. Now, if you’re going to use the actual photograph for the 3D-projection, you will need to remove these objects to the best of your ability. The cleaner the better. For me, it’s a different story.

Cool. Step 1 done. I saved the objects on a separate layer so I can call them back into the scene later. Probably using After Effects, or even as alpha-channeled planes in Maya. I don’t know for sure. Too bad the screenshot above isn’t exactly the final version. I selected a couple more areas which you’ll see in… step 2.

Step 2 is this: content-aware delete. If you just went “You’re being lazy,” I’m basically going to say, “That’s right!”

Yup. I love content aware delete. It gives you weird results. Of course that’s not what I’m looking for, though it honestly did a interesting job creating abstract art.

Alright, the problematic things are somewhat gone. It doesn’t have to be perfect like I say, but it has to be clean enough so that I can paint right over it and care less about that stuff.

Next: playing around in the most exciting program! Studio Artist 4.03!! Let me explain, Synthetik Software Studio Artist 4.03 is a powerful painting program that basically goes beyond what your normal painting program does. It’s abnormal, oriented for artists, and totally wacked out. With probably more than enough brush presets to go through, you’re pretty much unlimited to what you can do in it. For more info check out the blog: http://studioartist.blogspot.com/

Another thing to keep in mind about Studio Artist is that it works half-automatically. This means that I can set up certain parameters in the paint patch to synthesize certain traits. So I can pick up the background photograph AS the source for color, for outlines, and otherwise.

Oh and the tools! I’m using a Wacom Bamboo tablet. This actually makes a big difference since these tablets have sensitivity. That makes this whole workflow interesting and fun.

So I placed the image into Studio Artist, and I first of all applied a paint patch based on a preset from Studio Artist 4 (I changed several important settings on it), which does this kinda blocky… chunky abstraction, useful for quick visualization on how (and where) the colors will be, how the final picture will kinda look. A couple quick adjustments, and I’m happy with the result. I’ll be using this as my base to paint on it. This is where the program becomes amazing. Oh, and before I move on, I also resized the image to 3500×2333. Studio Artist works independent of resolution. I can enlarge any image and paint as if it were at that size. Horray for synthetic images!

I then began painting using one of the brush patches I’ve fallen in love with. It’s very basic, but the brush picks up the color from the point of where I place my brush down first, it then smudges that color. So it behaves like paint but not quite. So what I do with this brush is block out some general color details. Basically I’m redoing what I saw with the previous picture (the one that’s abstracted and blocky.) We’re going to again, use this to paint over details.

Add the details back in…. I’m using a brush that now picks up the edges from the photograph along with the color. It has several attributes which allow an easier coloring and detailing. Studio Artist also features an unusual set of “AI” features which allow the user to interactively paint certain things in automatically. This is kinda hard to describe without seeing it, but basically the program can use the source image to manipulate the brush strokes I put down. That way, I get detail, at the same time reserving an abstract look.

I also reserve the right to do some crazy stuff on the painting! I have tons of brush presets I created over the time I’ve been doing this project and it’s kinda looking fun. Some of these presets are based on pre-existing ones, but they’re mostly just stuff I’ve been making and using for this project. In the picture below, I just went ahead and threw in some fun stuff. I drew rough outlines of windows using a brush that resembles a line drawn with a thick, colored pencil. No care for straight lines. Just scribbling away! Ok but with moderation and with style.

So it gets crazier and crazier. But I have to pull this craziness back. What do I do? Add what detail there was back in! So I’m using a special detailing brush which specifically extracts outlines of the photograph and applies them where I place my stylus.

So at this point, I think it looks good enough. I’m happy with the result so allow me to post the final “color corrected” version here.

So those were my basic 3 or 4 days (or maybe 6… haven’t been really keeping track of time here) of just doing some work. So here’s what I’ll have towards the end of this project. 3 pieces of work all made in 3D! Then I’ll put together a movie that you can watch for your enjoyment. So stay tuned for those. BUT for the next blog post, let me try this as my projection in 3D!

This has been a long blog post. It’s 4:16 am in the morning, and I wasn’t intending to be taking screenshots and writing this at this hour.

I haven’t updated in several days! What’s happening with the perspective project you may be wondering? Let me explain what’s happening. So back several days ago, after the critique, I began playing around with the scenery again… which resulted in unusual renders. Here are some below.

Fail.

I must be using the wrong settings!! That’s how I thought, so I just went back and changed some of the parameters around. I made sure everything was based on the tutorial. And here’s basically what happened….

Much like modern art. I'm selling this for $10,000.

… wow. But I see the problem. First of all, the image was not lined up correctly with the image itself. While I made sure that the settings in the image plane were the same in the projection node, I did not realize that if these two don’t mach, you obviously get a messed up result as you see in the two results above.

I decided to take a notch back. I’m not giving up just yet, no, NEVER giving up because this is what I wanted to do to begin with. So I thought I should find another image and give it a shot.

A simple image instead of the painterly one.

Here’s the new image I’ve been using since last week. This image is just easier to work with (I personally think.) So lets start with the basics. I first begin by drawing lines. Not just any line, a perspective line so I can tell where the horizon line is.

Some perspective lines were drawn into the image.

Great! Now, I want to note a couple interesting things I noticed about Japanese architecture. Unlike a grid-based approach to city planning, Japan takes a more organic approach to city planning. Meaning, that the city respects properties owned by people. Which results in a very complex system of roads and “blocks.” This makes home making incredibly weird since you want to use the maximum space possible while abiding all building laws. So, you end up building a house conforming to the roads, the given land space, and laws. This results in the unequal, shifty home design that breaks that clean grid look. Nothing in that image is a straight line. It’s somehow curved slightly.

So what does that have to do with anything? Basically this: determining perspective from squiggly lines from uneven building constructions and roads are hard. I had to really nail it but even with these perfect looking lines, the whole perspective is off. But whatever. It seems relatively ok, and I know I don’t have to go into so much depth.

Either way, I followed the same steps as above, I determined the sensor size, focal length, and finally the perspective on the camera. Then I began throwing some simple geometry down. And this is what I got.

Attempt 5

After five attempts over the course of 3 or 4 days, I got this to work. And it was ridiculously easy. I wondered why it was so easy that I wondered what the trouble was after all. But ok, lets get back on track. Now we want to cast some textures on these simple geometries.

Here we go!!

Next step: Render

IT WORKS!!!!!!!

All that trouble really paid off. The quality is a bit low, but that’s because the image quality to begin with wasn’t that high. We need to obviously fix a couple things, but overall, it works.

Lets try changing the angles a bit.

Awesome happens.

We have some limitations.

In the end we have limitations to this approach. You can see where I drew the arrows, the image is doubling. We also can see some significant blurring. This can probably be solved by using a larger image resolution at the cost of render times.

It’s been a very long post, but this is basically where I am now and what I’ve been hacking at since last week. Overall, it’s pretty easy BUT it gets hard after a while, I mean very tricky. Now I’ll have to edit the photograph so it won’t have any telephone poles and otherwise. I can add those in later as a separate layer.

Where next? Next? I’m planning to obviously get back to that painting somehow. I have no clue if it will work, but I know if I take it step by step it’ll work. Otherwise I’ll force it to work.

The other thing I was thinking of was extracting the textures from each face of the building for UV mapping. And I may go about doing that for this project. So next part: 1. Getting that painting working. 2. More photos to have fun with. 3. Trying a further defined approach (texture extraction from photographs.)

Like this:

Alright, after following the tutorial by Andrew Kramer (Video CoPilot) I was able to create a basic 3D projection map based on the image. It’s a little weird right now, but I’m able to move in the scenery as I expected to do. I originally was following Video CoPilot’s tutorial, though I couldn’t find out one of the settings. I guess I was just impatient.

It is technically easier than using Maya since I’m using only flat surfaces to construct the scene. I can also fill spaces using more, hence creating a complete 3D object within the After Effects environment. This is actually quite an unusual turn around. I was expecting to do this within Maya, though After Effects also works as an alternative method. I will continue to attempt using Maya to making the projection mapping happen. Although After Effects method does work nicely. Here’s a screenshot.

The “floor” is a bit short but… I can always stretch some scenes out.

How it works: I have a light transmitting the map right on the two flat solids I constructed. They meet together at a nice 90 degree angle, and later it was matched to the angle of the image itself. This way we have a floor, and the object in front of us.

To make this a better projection, we must add more solids to construct more details. Of course, it may be a bit more work and it might be easier to just do this all in Maya, though I keep thinking how much easier it is now that I know it works here.

So, in attempt to project this scenery, I began realizing a big issue with this whole scenery. What’s difficult? The angle of the camera. Since there are 3 photographs that were originally stitched together and painted on, we actually have to compensate for 3 different angles which may or may not match. Now that makes this whole projection mapping harder as it already has created several problems when I began putting some geometry in.

Hmm… now this is annoying. 3 images, 3 angles… a couple geometries, and it fails instantly. Or it’s just the fact that it’s difficult to match perspective properly… Ok, so what next? I tried this After Effects and I actually have hopes of it working. It won’t be what I was intended to do, but with some fighting I can probably get it to work right. Worst case scenario, this just turns into multiple image planes that get placed in various areas. That’s actually not what I want to do.

Anyway, I fixed several things as well, roughly….

That should aid some with depth when I put the actual item back into the scene.

Like this:

I’m still kinda stuck on trying out different designs. I actually like this painting overall, in fact I should just use this one! Quite like this a lot. Turns out the geometry doesn’t look too complicated, and can probably look neat overall when its translated in 3D space. So I’m going to start playing with some geometry! :)

Unfortunately this is a tough project. It’s tough simply because of the fact that I draw most of my stuff by hand and I’m not a big fan of using a tiny wacom tablet. Either way, I’ll just have to crank out some simple drawings rough drafts. So I began exploring my massive reference collection for some nice photo to work from. Fortunately I found several that I mashed and played with until I got this….

Some kind of art by Skybase

Basically it’s a rough photo mash. You grab several elements from other photographs, match perspective and smash elements into it like it fit. Basically a way to seriously make a mess quickly since you select, paste, match and repeat process until it looks decent. Though of course, I’m not satisfied usually. My photomashing skills aren’t as high so I won’t be putting that ugly picture online, but the final ones will be just here. So that’s a “start.” I guess.

Much of this painting is based heavily on the photomash earlier. It was created in StudioArtist 4.03 which is my favorite tool for painting cause it has insane and unique features that really resonate with the artist spirit. So my process is this: Manipulate photograph roughly —> use Studio Artist 4 to paint on it —> back to Photoshop and do postwork. The colors are derived from the picture, but modulated so its not JUST from the picture. So this painting is partially done by a computer with my own settings. That’s why some things look kinda algorithmic, but at the same time, I left it so I can intervene and paint things on my own. That way I have significant amount of control over details, highlights, colors, and overall shapes and sizes of things. So how much of it was “computer?” It’s around 20% computer and 80% me with a wacom tablet. Of course I can just do 100% computer but that’d be boring and not fun. I do need control over what I’m doing.

How long did this take? Probably around 5 or 6 hours or so. *shrug* I honestly don’t know when I began.

I’m going to cut-paste-paint-post several more pictures probably tomorrow, eventually I’ll choose which ones to take into 3D. But for now, this is what I have.