These days, clean and sleek e-Commerce web sites are the way to go. And that almost always includes high quality product photos with a flat white, transparent, or solid background. This is not just a passing fad. There are many practical and well-validated reasons for this choice. The graphic below summarizes a few of these:
But if you are a website or app developer who needs to deliver hundreds or thousands of images, or if your site allows its users to upload product images to your site, getting someone to manually edit photos in a graphic editing tool just won't cut it.
Today, Cloudinary is proud to introduce it's AI Background Removal Add-on, which combines a variety of deep-learning and artificial intelligence algorithms to recognize the primary foreground subject of a photo and accurately remove the photo background in a matter of seconds.
Moments after you upload your photos...
| These: | Become These! |
|
|
|
|
Transparent backgrounds are just a parameter away
Cloudinary is a heavy-duty image and video platform that leverages automation and deep learning algorithms to simplify the entire rich-media workflow for websites and apps -- from the bulk upload of raw photo shots to on-the-fly manipulations for creating multiple variants of media files, automatic optimizations, and responsive delivery across various devices.
The AI Background Removal Add-on can be applied to any image either while uploading it to your Cloudinary account or afterwards, simply by adding the background_removal parameter set to cloudinary_ai in your upload or update method.
To instruct Cloudinary to remove the background from the baseball image shown above while uploading, your upload command (using your chosen Cloudinary SDK) would look something like this:
Cloudinary::Uploader.upload("baseball.jpg", :public_id => "baseball_no_bg", :background_removal => 'cloudinary_ai', :notification_url => "https://mysite.example.com/hooks")
\Cloudinary\Uploader::upload("baseball.jpg", array( "public_id" => "baseball_no_bg", "background_removal" => "cloudinary_ai", "notification_url" => "https://mysite.example.com/hooks"));
cloudinary.uploader.upload("baseball.jpg", public_id = "baseball_no_bg", background_removal = "cloudinary_ai", notification_url = "https://mysite.example.com/hooks")
cloudinary.v2.uploader.upload("baseball.jpg", { public_id: "baseball_no_bg", background_removal: "cloudinary_ai", notification_url: "https://mysite.example.com/hooks" }), function(error, result){console.log(result);});
cloudinary.uploader().upload("baseball.jpg", ObjectUtils.asMap( "public_id", "baseball_no_bg", "background_removal", "cloudinary_ai", "notification_url", "https://mysite.example.com/hooks"));
var uploadParams = new ImageUploadParams(){ File = new FileDescription(@"baseball.jpg"), PublicId = "baseball_no_bg", BackgroundRemoval = "cloudinary_ai", NotificationUrl = "https://mysite.example.com/hooks"}; var uploadResult = cloudinary.Upload(uploadParams);
Initially, your original image is uploaded to Cloudinary, and then the add-on removes the background and replaces the original with a (transparent background) PNG. The original image is automatically backed up and can be easily recovered if needed.
Note the notification_url used in the above upload command. Even though the background removal process only takes a few seconds, if you plan to deliver the transparent image on your site or app right after it uploads, you can use the notification webhook to check the status of the background removal and potentially display a placeholder image or an in-progress animation until the process is complete.
We only make it look simple
Activating the add-on to erase all traces of the background is really as easy as shown above. But what's going on behind the scenes is anything but simple.
Automatically removing a photo background requires the engine to:
- Recognize the salient object(s) in the image
- Accurately segment those object/s
- Separate the foreground to an alpha layer
The ability to segment objects from an image is a fundamental and well-studied challenge in computer vision. Recently, neural networks have been able to achieve the state-of-the-art results that have been used in applications as varied as autonomous driving, medical image analysis, industrial planning and more.
But that's just the tip of the iceberg for image background removal:
First, you need to determine which objects in the image should be classified as foreground versus the background, which depends on the context and composition of the scene.
Additionally, the standard object segmentation used in most neural networks produces coarse segmentation maps. But for production-level background removal, the segmentation maps of foreground vs. background pixels must be near-perfect to make seamless background removal possible. For instance, fur and hair require special care to segment them properly.
Furthermore, there is currently no significantly large public dataset for accurate background removal, which makes it difficult to acquire the quantity of media assets necessary to train a neural network.
And even while addressing all these challenges, we knew we had to find ways to optimize our algorithm so it could deliver the fast results Cloudinary customers expect on images of any size, and at the huge scales that our customer base requires.
Delivering Optimal Results
Once the add-on removes the background, you can then deliver the resulting transparent PNG as-is, or you may want to add a solid or other neutral background and deliver using Cloudinary's auto‑format (f_auto) and auto-quality (q_auto) optimizations to achieve smaller files while retaining visual quality.
Additionally, the returned transparent image retains the original image dimensions. This means your resulting image may have a lot of white space and may not be centered within those original dimensions. To address this, you can also take advantage of our auto-gravity cropping, to automatically detect the visible item(s) in the image no matter where they are, and then resize and crop to your desired delivery size and aspect ratio without cropping off any of the foreground image that Cloudinary's deep learning algorithm just worked so hard to find.
For example, using the background removal add-on, this cute stuffed sheep returns without the table and wall background, but it sure does have a lot of white space on the right side. And as a transparent PNG, it's quite a fat sheep, weighing in at a bit over 1 MB.
But let's deliver our resulting (transparent background) image as a square JPG with a white background, while using f_auto and q_auto to deliver in the most optimized format for the requesting browser. We'll also add g_auto:subject to make sure that no matter what aspect ratio we choose for our crop, the location for the crop is automatically selected based on the main (in this case, the only) subject in the image. For demonstration purposes here, we'll also add a thin gray border.
cl_image_tag("docs/rmv_bgd/stuffed.jpg", :transformation=>[ {:quality=>"auto", :gravity=>"auto:subject", :height=>1280, :aspect_ratio=>"1", :background=>"white", :crop=>"thumb"}, {:border=>"1px_solid_gray"} ])
cl_image_tag("docs/rmv_bgd/stuffed.jpg", array("transformation"=>array( array("quality"=>"auto", "gravity"=>"auto:subject", "height"=>1280, "aspect_ratio"=>"1", "background"=>"white", "crop"=>"thumb"), array("border"=>"1px_solid_gray") )))
CloudinaryImage("docs/rmv_bgd/stuffed.jpg").image(transformation=[ {'quality': "auto", 'gravity': "auto:subject", 'height': 1280, 'aspect_ratio': "1", 'background': "white", 'crop': "thumb"}, {'border': "1px_solid_gray"} ])
cloudinary.image("docs/rmv_bgd/stuffed.jpg", {transformation: [ {quality: "auto", gravity: "auto:subject", height: 1280, aspect_ratio: "1", background: "white", crop: "thumb"}, {border: "1px_solid_gray"} ]})
cloudinary.url().transformation(new Transformation() .quality("auto").gravity("auto:subject").height(1280).aspectRatio("1").background("white").crop("thumb").chain() .border("1px_solid_gray")).imageTag("docs/rmv_bgd/stuffed.jpg");
cloudinary.imageTag('docs/rmv_bgd/stuffed.jpg', {transformation: [ {quality: "auto", gravity: "auto:subject", height: 1280, aspectRatio: "1", background: "white", crop: "thumb"}, {border: "1px_solid_gray"} ]}).toHtml();
$.cloudinary.image("docs/rmv_bgd/stuffed.jpg", {transformation: [ {quality: "auto", gravity: "auto:subject", height: 1280, aspect_ratio: "1", background: "white", crop: "thumb"}, {border: "1px_solid_gray"} ]})
<Image publicId="docs/rmv_bgd/stuffed.jpg" > <Transformation quality="auto" gravity="auto:subject" height="1280" aspectRatio="1" background="white" crop="thumb" /> <Transformation border="1px_solid_gray" /> </Image>
<cl-image public-id="docs/rmv_bgd/stuffed.jpg" > <cl-transformation quality="auto" gravity="auto:subject" height="1280" aspect-ratio="1" background="white" crop="thumb"> </cl-transformation> <cl-transformation border="1px_solid_gray"> </cl-transformation> </cl-image>
cloudinary.Api.UrlImgUp.Transform(new Transformation() .Quality("auto").Gravity("auto:subject").Height(1280).AspectRatio("1").Background("white").Crop("thumb").Chain() .Border("1px_solid_gray")).BuildImageTag("docs/rmv_bgd/stuffed.jpg")
MediaManager.get().url().transformation(new Transformation() .quality("auto").gravity("auto:subject").height(1280).aspectRatio("1").background("white").crop("thumb").chain() .border("1px_solid_gray")).generate("docs/rmv_bgd/stuffed.jpg");
imageView.cldSetImage(cloudinary.createUrl().setTransformation(CLDTransformation() .setQuality("auto").setGravity("auto:subject").setHeight(1280).setAspectRatio("1").setBackground("white").setCrop("thumb").chain() .setBorder("1px_solid_gray")).generate("docs/rmv_bgd/stuffed.jpg")!, cloudinary: cloudinary)
Now we get our sheep with a solid-white background, and even if you were to retain the original 1280 px height, the delivered image would still only weigh 92kb.
Another great way you can deliver your image with no excess whitespace is to apply Cloudinary's trim transformation effect, which trims off all excess background:
cl_image_tag("docs/rmv_bgd/stuffed.jpg", :transformation=>[ {:quality=>"auto", :effect=>"trim", :background=>"white"}, {:border=>"1px_solid_gray"} ])
cl_image_tag("docs/rmv_bgd/stuffed.jpg", array("transformation"=>array( array("quality"=>"auto", "effect"=>"trim", "background"=>"white"), array("border"=>"1px_solid_gray") )))
CloudinaryImage("docs/rmv_bgd/stuffed.jpg").image(transformation=[ {'quality': "auto", 'effect': "trim", 'background': "white"}, {'border': "1px_solid_gray"} ])
cloudinary.image("docs/rmv_bgd/stuffed.jpg", {transformation: [ {quality: "auto", effect: "trim", background: "white"}, {border: "1px_solid_gray"} ]})
cloudinary.url().transformation(new Transformation() .quality("auto").effect("trim").background("white").chain() .border("1px_solid_gray")).imageTag("docs/rmv_bgd/stuffed.jpg");
cloudinary.imageTag('docs/rmv_bgd/stuffed.jpg', {transformation: [ {quality: "auto", effect: "trim", background: "white"}, {border: "1px_solid_gray"} ]}).toHtml();
$.cloudinary.image("docs/rmv_bgd/stuffed.jpg", {transformation: [ {quality: "auto", effect: "trim", background: "white"}, {border: "1px_solid_gray"} ]})
<Image publicId="docs/rmv_bgd/stuffed.jpg" > <Transformation quality="auto" effect="trim" background="white" /> <Transformation border="1px_solid_gray" /> </Image>
<cl-image public-id="docs/rmv_bgd/stuffed.jpg" > <cl-transformation quality="auto" effect="trim" background="white"> </cl-transformation> <cl-transformation border="1px_solid_gray"> </cl-transformation> </cl-image>
cloudinary.Api.UrlImgUp.Transform(new Transformation() .Quality("auto").Effect("trim").Background("white").Chain() .Border("1px_solid_gray")).BuildImageTag("docs/rmv_bgd/stuffed.jpg")
MediaManager.get().url().transformation(new Transformation() .quality("auto").effect("trim").background("white").chain() .border("1px_solid_gray")).generate("docs/rmv_bgd/stuffed.jpg");
imageView.cldSetImage(cloudinary.createUrl().setTransformation(CLDTransformation() .setQuality("auto").setEffect("trim").setBackground("white").chain() .setBorder("1px_solid_gray")).generate("docs/rmv_bgd/stuffed.jpg")!, cloudinary: cloudinary)
Everything isn't always black and white
As you can see, optimized image delivery will often involve replacing the transparent background and delivering in a format other than the PNG that our background removal add-on initially returns. But there are good reasons why we don't just return a solid white JPG to begin with.
Going neutral
For starters, delivering with a light, neutral background color can provide the same advantages as white, but it can break the monotony of an all-white page, and even communicate a certain style or personality without taking the focus away from the product.
This is the approach Walmart.com has taken on their apparel landing page:
To accomplish this with Cloudinary, just change the value of the background (b_) parameter in your URL to the desired color. For example, check out our sheep with a very light blue background.
True colors
Conversely, some retailers are going for brightly colored backgrounds when presenting white or black products, while sticking with neutral backgrounds with their other products, like this example from Zara.com:
Without a shadow of a doubt
Starting with a fully transparent background also makes it easy to add shadows. Of-course you may have already had some sort of shadow in the original photograph, but our Background Removal add-on removes that shadow, enabling you to add a consistent shadow location and size for all your products.
For example:
Original remote control image
No background
No background + added shadow
Here's the transformation we used above to add a shadow to the background-less remote control:
cl_image_tag("docs/rmv_bgd/remote.png", :transformation=>[ {:width=>200, :crop=>"scale"}, {:effect=>"shadow:50", :x=>6, :y=>6} ])
cl_image_tag("docs/rmv_bgd/remote.png", array("transformation"=>array( array("width"=>200, "crop"=>"scale"), array("effect"=>"shadow:50", "x"=>6, "y"=>6) )))
CloudinaryImage("docs/rmv_bgd/remote.png").image(transformation=[ {'width': 200, 'crop': "scale"}, {'effect': "shadow:50", 'x': 6, 'y': 6} ])
cloudinary.image("docs/rmv_bgd/remote.png", {transformation: [ {width: 200, crop: "scale"}, {effect: "shadow:50", x: 6, y: 6} ]})
cloudinary.url().transformation(new Transformation() .width(200).crop("scale").chain() .effect("shadow:50").x(6).y(6)).imageTag("docs/rmv_bgd/remote.png");
cloudinary.imageTag('docs/rmv_bgd/remote.png', {transformation: [ {width: 200, crop: "scale"}, {effect: "shadow:50", x: 6, y: 6} ]}).toHtml();
$.cloudinary.image("docs/rmv_bgd/remote.png", {transformation: [ {width: 200, crop: "scale"}, {effect: "shadow:50", x: 6, y: 6} ]})
<Image publicId="docs/rmv_bgd/remote.png" > <Transformation width="200" crop="scale" /> <Transformation effect="shadow:50" x="6" y="6" /> </Image>
<cl-image public-id="docs/rmv_bgd/remote.png" > <cl-transformation width="200" crop="scale"> </cl-transformation> <cl-transformation effect="shadow:50" x="6" y="6"> </cl-transformation> </cl-image>
cloudinary.Api.UrlImgUp.Transform(new Transformation() .Width(200).Crop("scale").Chain() .Effect("shadow:50").X(6).Y(6)).BuildImageTag("docs/rmv_bgd/remote.png")
MediaManager.get().url().transformation(new Transformation() .width(200).crop("scale").chain() .effect("shadow:50").x(6).y(6)).generate("docs/rmv_bgd/remote.png");
imageView.cldSetImage(cloudinary.createUrl().setTransformation(CLDTransformation() .setWidth(200).setCrop("scale").chain() .setEffect("shadow:50").setX(6).setY(6)).generate("docs/rmv_bgd/remote.png")!, cloudinary: cloudinary)
Green-screening without a green screen
If you want to have some fun (or pass that fun on to your users), you can add a fun background scene behind your transparent image for a green-screen-like effect, regardless of the original background. Just use the underlay parameter set to the public ID of any image in your Cloudinary account as the new background layer.
For example, with an instantaneous on-the-fly transformation, we could send our baseball player to outer space, get the dog off the couch, or place hiking boots product in a more nature-oriented environment:
Below is the delivery code for the hiking boots example.
(Click the other transformed images on the right above to view their delivery URLs.)
cl_image_tag("docs/rmv_bgd/shoes.png", :transformation=>[ {:width=>800, :quality=>"auto", :crop=>"scale"}, {:underlay=>"docs:canyon2", :gravity=>"south", :height=>1000} ])
cl_image_tag("docs/rmv_bgd/shoes.png", array("transformation"=>array( array("width"=>800, "quality"=>"auto", "crop"=>"scale"), array("underlay"=>"docs:canyon2", "gravity"=>"south", "height"=>1000) )))
CloudinaryImage("docs/rmv_bgd/shoes.png").image(transformation=[ {'width': 800, 'quality': "auto", 'crop': "scale"}, {'underlay': "docs:canyon2", 'gravity': "south", 'height': 1000} ])
cloudinary.image("docs/rmv_bgd/shoes.png", {transformation: [ {width: 800, quality: "auto", crop: "scale"}, {underlay: "docs:canyon2", gravity: "south", height: 1000} ]})
cloudinary.url().transformation(new Transformation() .width(800).quality("auto").crop("scale").chain() .underlay(new Layer().publicId("docs:canyon2")).gravity("south").height(1000)).imageTag("docs/rmv_bgd/shoes.png");
cloudinary.imageTag('docs/rmv_bgd/shoes.png', {transformation: [ {width: 800, quality: "auto", crop: "scale"}, {underlay: new cloudinary.Layer().publicId("docs:canyon2"), gravity: "south", height: 1000} ]}).toHtml();
$.cloudinary.image("docs/rmv_bgd/shoes.png", {transformation: [ {width: 800, quality: "auto", crop: "scale"}, {underlay: new cloudinary.Layer().publicId("docs:canyon2"), gravity: "south", height: 1000} ]})
<Image publicId="docs/rmv_bgd/shoes.png" > <Transformation width="800" quality="auto" crop="scale" /> <Transformation underlay="docs:canyon2" gravity="south" height="1000" /> </Image>
<cl-image public-id="docs/rmv_bgd/shoes.png" > <cl-transformation width="800" quality="auto" crop="scale"> </cl-transformation> <cl-transformation underlay="docs:canyon2" gravity="south" height="1000"> </cl-transformation> </cl-image>
cloudinary.Api.UrlImgUp.Transform(new Transformation() .Width(800).Quality("auto").Crop("scale").Chain() .Underlay(new Layer().PublicId("docs:canyon2")).Gravity("south").Height(1000)).BuildImageTag("docs/rmv_bgd/shoes.png")
MediaManager.get().url().transformation(new Transformation() .width(800).quality("auto").crop("scale").chain() .underlay(new Layer().publicId("docs:canyon2")).gravity("south").height(1000)).generate("docs/rmv_bgd/shoes.png");
imageView.cldSetImage(cloudinary.createUrl().setTransformation(CLDTransformation() .setWidth(800).setQuality("auto").setCrop("scale").chain() .setUnderlay("docs:canyon2").setGravity("south").setHeight(1000)).generate("docs/rmv_bgd/shoes.png")!, cloudinary: cloudinary)
Putting it all together
Whether you are delivering product photos on your own site or on multiple marketplaces, or you want to offer background removal services for your end users' uploaded photos, you need an automated solution that will quickly and reliably remove backgrounds, leaving you with a sharp and clear foreground image.
Once the original background is 'out of the picture', you can freely deliver your product images with white backgrounds, neutral or even bright colors, or you can put them in context of a realistic scene.
In fact, you may want to do all of the above. One way to do this is to put all your resulting background-free images together, along with other views such as 3D images and videos, in a single carousel-controlled gallery, using Cloudinary's easy-to-implement Product Gallery. You can learn more in the Product Gallery blog post, video tutorial, and documentation.
Your turn!
To get started removing backgrounds yourself, just sign up for a free Cloudinary account, register for the free trial tier of our Cloudinary AI Background Removal add-on, and check out the Background Removal add-on documentation.
We'd love to see your background removal photo results in the comments!