AI technology is going to be huge for artists in the near future. The latest demonstration comes from researchers at NVIDIA where they’ve developed a program that turns simple doodles into beautiful landscape images. Not long ago NVIDIA developed AI technology that generated photos of fake people, and this latest app takes things one step further.

Using a type of AI model known as a generative adversarial network (GAN), the software gives users a “smart paint brush.” The AI app named GauGAN — a nod to famed post-Impressionist painter Paul Gauguin — is easy to manipulate. With a few clicks and drags in an app that looks like MS Paint, anyone can make a very basic outline of a landscape before filling in their rough sketch with natural textures like grass, clouds, forests, or rocks. Those textures are assigned different colours, meaning it’s even possible to select weather elements like snow. The app is even smart enough to distinguish between details like gravel and sand.

Turning your simple doodles into beautiful imagery

GauGAN has three tools: a paint bucket, pen and pencil. At the bottom of the screen is a series of objects to choose from. If, for example, you select the cloud object and draw simple doodle with the pencil, the software will produce a wisp of photorealistic clouds. However, these aren’t simple image stamps. The app produces results unique to the input. So if you draw a circle and fill it with the paint bucket, the software will make puffy summer clouds. Users can use the input tools to draw simple doodles of trees and it will produce a forest; draw a straight line and it will produce a bare trunk; draw a bulb at the top and the software will fill it in with leaves producing a full tree.

“It’s like a colouring book picture that describes where a tree is, where the sun is, where the sky is,” explains Bryan Catanzaro, VP of Applied Deep Learning Research at NVIDIA. “And then the neural network is able to fill in all of the detail and texture, and the reflections, shadows and colors, based on what it has learned about real images.”

This means that if you place a body of water next to trees, the program will know to reflect the trees in the pond in the resulting photograph. NVIDIA hopes that the program will be useful for people who design virtual worlds, such as architects and landscape designers. And since the AI was trained using millions of images, it has a rich knowledge that helps it fine-tune its landscapes.

NVIDIA’s deep learning model

“This technology is not just stitching together pieces of other images, or cutting and pasting textures. It’s actually synthesizing new images, very similar to how an artist would draw something.”

While this software isn’t exactly groundbreaking — researchers have shown off similar tools, including one from Google that turns doodles into clipart — but it’s the most polished demonstration of this concept seen to date. Unfortunately, there’s no word on when we might see this technology available to the public.

Source: https://petapixel.com/2019/03/19/nvidias-ai-can-turn-doodles-into-landscape-photos/