By Brian X. Chen
Much of the hype and fears around generative AI has been about text. But there have also been rapid and dramatic developments in systems that can generate images. In many cases, these share a similar structure to text-based generative AI, but they can also be much weirder — and lend themselves to some very fun creative pursuits.
Image generators are trained on billions of images, which enable them to produce new creations that were once the sole dominion of painters and other artists. Sometimes experts can’t tell the difference between AI-created images and actual photographs (a circumstance that has fueled dangerous misinformation campaigns in addition to fun creations).
Compared to products like ChatGPT, image generating AI tools are not as well developed. They require jumping through a few more hoops and may cost a bit of money. But if you’re interested in learning the ropes there’s no better time to start.
AI Photoshop
Last week, Adobe added a generative AI feature into a beta version of Photoshop, its iconic graphics software, and creators on social networks like TikTok and Instagram have been buzzing about it ever since.
When I tested the new feature, called “generative fill,” I was impressed with how quickly and competently the AI carried out tasks that would have taken me at least an hour to do on my own. In less than five minutes and with only a few clicks, I used the feature to remove objects, add objects and swap backgrounds.
(To experiment with these tools yourself, start by signing up for a free trial of Adobe Creative Suite. Then, install the new Adobe Photoshop beta, which includes generative fill.)
Once you have Photoshop beta installed, import a photo and try these tricks:
To change a background, click the “object selection” icon (it has an arrow pointed at a box), then under the Select menu, click “inverse” to select the background. Next, click the “generative fill” box and type in a prompt — or leave it blank to let Photoshop come up with a new background concept for you.
I used these steps to edit a photo of my corgi, Max. I typed “kennel” for the prompt and clicked “generate" to replace the background.
To remove objects, use the lasso tool. In this photo of my motorcycle, I wanted to erase a tractor behind a fence in the background. I traced around the tractor, and then I clicked the “generative fill” box and hit “generate” without entering a prompt. The software correctly removed the tractor and filled in the background while leaving the fence intact.
Photo editors at The New York Times do not enhance or alter photos or generate images using artificial intelligence. But my first thought after testing generative fill was that photo editors working in other contexts, like marketing, could be soon out of work. When I shared this theory with Adobe’s chief technology officer, Ely Greenfield, he said that it might make photo editing more accessible, but he was optimistic that humans would still be needed.
“I can make really pretty images with it, but frankly, I still make boring images,” he said. “When I look at the content that artists create when you put this in their hands versus what I create, their stuff is so much more interesting because they know how to tell a story.”
I confess that what I’ve done with generative fill is far less exciting than what others have been posting on social media. Lorenzo Green, who tweets about AI, posted a collage of famous album covers, including Michael Jackson’s “Thriller” and Adele’s “21” that were expanded with generative fill. The results were quite entertaining.
(One note: If installing Photoshop feels daunting, a quicker way to test Adobe’s AI is to visit the Adobe Firefly website. There, you can open the generative fill tool, upload an image and click the “add” tool to trace around a subject, such as a dog. Then click “background” and type in a prompt like “beach.”)
More image generators
Tools like DALL-E and Midjourney can create entirely new images in seconds. They work similarly to chatbots: You type in a text prompt — the more specific, the better.
To write a quality prompt, start with the medium you’d like to emulate, followed by the subject and any extra details. For example, typing “a photograph of a cat wearing a sweater in a brightly lit room” in the DALL-E prompt box will generate it.
DALL-E, which is owned by Open AI, the maker of ChatGPT, was one of the first widely available AI image generators that was simple for people to use. For $15, you get 115 credits; one credit can be used to generate a set of four images.
Midjourney, another popular image generator, is a work in progress, so the user experience is not as polished. The service costs $10 a month, and entering prompts can be a little more complicated, because it requires joining a separate messaging app, Discord. Nonetheless, the project can create high-quality, realistic images.
To use it, join Discord and then request an invitation to the Midjourney server. After joining the server, inside the chat box, type “/imagine” followed by a prompt. I typed “/imagine a manga cover of a corgi in a ninja turtle costume” and generated a set of convincing images.
Though it’s fine to type in a basic request, some have found obscure prompts that generated exceptional results. At Columbia University, Lance Weiler is teaching students how to leverage AI, including Midjourney, to produce artwork.
Whichever tool you use, bear in mind that the onus is on you to use this tech responsibly.
Technologists warn that image generators can increase the spread of deepfakes and misinformation. But the tools can also be used in positive and constructive ways, like making family photos look better and brainstorming artistic concepts.
The New York Times
