Follow by Email

Monday, January 23, 2012

Painter's Image Hose

In Painter 3, a new kind of brush was added: the Image Hose. With this tool, you could draw using random snippets of image data. Each bit of image data could be individually masked and even shadowed. So you could literally paint with reality. To right, I have included an image created with Corel Painter 12. First a background of dirt was drawn with the image hose, and then some leaves we're drawn on top, also with the image hose.


How did the image hose begin? John Derry and I were sitting outside at a Restaurant in Chicago on July 31, 1992 and he suddenly wanted to be able to paint with the tree. I considered this for a while, and then i took down a note. My note said: analyze and grow texture from an example. But how??


Imagine each of the tree's leaves had a picture and they could be placed randomly. I remember we came to the conclusion that with only a few leaf pictures and perhaps some randomness of rotation angle or shading that you could paint a reasonable tree image. This was ideal since memory was, and continues to be, at a premium. In those days, that would be true on a Mac. In these days, that would be true on an iOS device, like an iPad.

The idea was in our heads that we could store a few images and draw using the set of images. So I came up with a way of doing it, which was initially called the structured clone.

As shown on the left, by October 1992 I already had some idea that the images could be stored in an array. Each element was individually w by h pixels, and they were arranged into a potentially three-dimensional structure. Each element is clearly spelled out to have an arbitrary 24-bit color image with an 8-bit alpha mask, treated like a penmask in the usual way.

A penmask is what we called the mask of a dab of paint. In those days, a brush stroke was always made up of a sequence of paint dabs placed along the bush storke's path. In this case, the brush stroke could be made up of a sequence of masked images instead.

I say it can be indexed by 0, 1, 2, or even 3 parameters - this is the dimensionality of the image hose, what I called the rank.
These parameters were such things as random, pressure, tilt, bearing, velocity, etc. In the random case, a new image element was chosen for that rank each time a new dab was placed along the stroke.

Let's imagine an example. I create 36 images in a 6x6 array. The rows contain six different orientations of a leaf, one orientation for each row. The columns contain six different sizes for each leaf, from small to large. I could use pressure to index the column, so the harder I press, the larger the leaf. I could use stroke direction or even stylus bearing to access the rows, which might allow me to control the orientation of the leaves by using my hand also. In this way, you could have more control over creating a tree just be drawing leaves.

Another thing we started doing was giving each image hose element a drop shadow. This meant that as you drew a pile of leaves, the leaves begin to shadow each other and thus shading would develop, making the pile appear to be three-dimensional.

To the right I have included a pile of randomly-oriented cubes created with the Image Hose in Painter 12. The drop shadow on each one is down and slightly to the right. This produces a slight shading below.

When we finished working on the Image Hose, we filed a patent in December 1995. You can find the details in the US Patent Office; it is patent number 5,767,860. It is one of the only things we decided to patent at Fractal Design, and so it is the only patent I share with John Derry.

Thoughts on How I Might Do It Differently


Today, there are quite a few different constraints on how things work, but compute power is much greater than it was in the early 1990s. The main thing I would do differently is to compute most of the changes on th fly. In Painter 4, we decided to do just that, computing scale, rotation, and opacity on the fly. But nowadays, I would probably allow for hue, saturation, and lightness changes, shadow depth, shadow opacity, and shadow feather to change on the fly as well.

This would mean that the only thing stored in the image hose database definition would be the random elements themselves. This would prevent much of the hand work in creating an image hose "nozzle".
It would be good to allow for 3D rotation and scaling as well. So that might mean creating a 3D definition for each element, which could be a bit complicated. The shading could also change on the fly.

I would probably allow for the nozzle to be made of several kinds of items, and separate them into groups. For example, you could make one group the stems, another group the leaves, and a final group the flowers. They would be stored into a single image hose nozzle definition. Then you might use pressure to choose between them, or you can hold down some modifier key, such as the command key or the command and open key to get each level when drawing.

Another thing I would like is to draw into a lens layer using an image hose, so that distortions can be applied s you draw. For example, you could draw shiny stones, then you could draw the water distortions on top to make the stones look like they are in a stream bed. And then perhaps the shines could be drawn on top as the sun sparkles on the water using another kind of image hose.

Something I tried but isn't still available in Painter is a grid-based image hose. This one draws each element inside a grid. You can randomly choose elements that are designed to connect with each other.

Here is an example of a tile design that could go into an image hose element and then placed randomly. Each element is designed to match at top, left, bottom and right, and so it won't matter how you place them, they will always match.

I would like to try this in square and hexagonal arrangements. Even in half-drop arrangements. It would be easy to design elements, since Painter supports drawing into half-drop patterns.

In this way, the image hose could help you to create mazes and other kinds of patterns that repeat, but without really repeating.

4 comments:

  1. I wanted to find a way to automatically generate new randomised shapes for concept art ideation, using nozzle elements for initial shapes pool. After a lot of experimentation, I was eventually successful, using both Auto Clone and Stroke Record/Playback in combination with a Painter macro scripts/custom palette interface.

    The ability to have the individual nozzle elements cluster in and around a selection and remain uncropped by that selection was key to the success in Auto Clone mode. Without that 'clustering without cropping' ability, the technique would not have worked.

    http://www.conceptart.org/forums/showthread.php?t=192645

    David

    ReplyDelete
    Replies
    1. There are so many brush capabilities in Painter, I often forget that they may be used together synergistically. A capability that the image hose needs is a clustering capability where several shapes can be played back into a cluster, and then the cluster can become a single image hose shape. Sort of a recursive way of building upon previous creation to create better and better things.

      That can actually play to the grammar of patterns as well, particularly if only the image hose still had the "snap to grid" capability built into it.

      For me, the ideation is both in the creation of the tool and in what the tool can create.

      Which leads to a self-reference situation.

      Delete
    2. "For me, the ideation is both in the creation of the tool and in what the tool can create."

      I couldn't agree more. Having only limited artistic ability, I believe I have experienced at least as much enjoyment creating tools and experimenting with Painter's features as any artist would have had in the creative painting process.

      Below is a movie demonstrating a scripted Painter interface which enabled me to generate 2x 26 element text nozzles (complete with fills and drop-shadows), and then paint with each of the saved nozzles in less 2 minutes, 30 seconds (the creation of the scripts took significantly longer);

      http://www.jitterbrush.com/videos/Speed_Nozzles.mov

      David

      Delete
    3. That's a fabulous demo! It was in Painter 3 that I started the process of "always scripting" the artist's work in Painter. And to make every single operation scriptable was a goal. But, as you have found, there are some operations that require handwork with scripts. Your automatic nozzle generator scripts are brilliant.

      It wasn't until several years later that Photoshop added their form of scripting. I always figured I'd stay a little ahead of the game.

      Originally, my goal was to use scripts to conquer the high-resolution issue. That is, record at one resolution, and play back at higher resolution. I even used that technique for a couple of poster images I did.

      The basis of Net Painter was scripting. It wouldn't have been possible unless I had already enabled recording and playback of everything.

      I wanted to make sure that everything I did was recorded somehow. And if I produced something really good, I didn't want it to end up in the Journal of Irreproducible Results.

      Delete