LogInAnd then create a new picture from them.
Zheng Zeng et al.
Researchers presented RGB↔X, a new method based on diffusion models that can break images into albedo, normal, roughness, metallicity, and irradiance channels and also make a picture out of them.
"RGB↔X is a unified diffusion-based framework that enables realistic image analysis (intrinsic channel estimation, denoted as RGB→X) and synthesis. ... RGB↔X explores the connections between diffusion models, realistic rendering, and intrinsic decomposition. We believe it can bring benefits to a wide range of downstream tasks, including material editing, relighting, and realistic rendering from simple/under-specified scene definitions."
Zheng Zeng et al.
The RGB→X model performs intrinsic decomposition, estimating per-pixel intrinsic channels (X) from an image (RGB). The X→RGB model synthesizes an image (RGB) from full or partial intrinsic channels (X), as the authors explain.
From what they demonstrate, the method does seem impressively effective, producing an image very close to the original photo in the X→RGB workflow. It also works pretty well on people.
Zheng Zeng et al.
Zheng Zeng et al.
Find more technical details here and join our 80 Level Talent platform and our new Discord server, follow us on Instagram, Twitter, LinkedIn, Telegram, TikTok, and Threads, where we share breakdowns, the latest news, awesome artworks, and more.