Tuesday, March 20, 2012

How to run online backup / Datastore GC / Tar Optimization using curl in CQ5.5

Use case: Curl command for online backup / Datastore GC / Tar Optimization for CQ5.4 does not work for CQ5.5 any more

Reason: Many CRX related operation is moved to JMX. Now you can use any JMX console or JMX API to invoke these operations.

Solution: You can use following curl commands

For online backup

curl -u <UID>:<PASSWORD> -X POST http://<HOST>:<PORT>/system/console/jmx/com.adobe.granite:type=Repository/op/startBackup/java.lang.String?target=<PATH OF BACKUP>/test.zip

Following options are available for backup

Delay (Default is 10 mili second)

curl -u <UID>:<PASSWORD> -X POST

Note After running this from front end you will see (null) as response. You have to check CRX log to verify if backup started. In CRX log (/crx-quickstart/logs/error.log) you would see message like this

*INFO* [Backup Worker Thread] com.day.crx.core.backup.Backup Backup started.
*INFO* [Backup Worker Thread] com.day.crx.core.backup.Backup Read size (1.234 gb) in 1.97s

OR Use following CURL for online backup

curl -u admin:admin --data "delay=3&force=false&target=test3.zip" http://HOST:PORT/libs/granite/backup/content/admin/backups/

delay: Time delay between write block.
force: To override last one
Target: Absolute path of backup. If path is not provided and just file name is given, Then it will use current path of installation.

Once backup is started, You can cancel it using following curl command

curl -u admin:admin http://HOST:PORT/libs/granite/backup/content/admin/backups.cancel.html

You can also track online progress using following URL


For Datastore GC

curl -u <UID>:<PASSWORD> -X POST http://<HOST>:<PORT>/system/console/jmx/com.adobe.granite:type=Repository/op/runDataStoreGarbageCollection/java.lang.Boolean

To delete data and delay as 2

curl -u admin:admin -X POST --data "delete=true&delay=2" http://HOST:PORT/system/console/jmx/com.adobe.granite%3Atype%3DRepository/op/runDataStoreGarbageCollection/java.lang.Boolean

CRX Log message to look for

*INFO* [ [1332343886706] POST /system/console/jmx/com.adobe.granite%3Atype%3DRepository/op/runDataStoreGarbageCollection/java.lang.Boolean HTTP/1.1] com.day.crx.sling.server.impl.jmx.GarbageCollection Scanning /libs/wcm/core/content/siteadmin/actions/create/menu/createPage

Automate DataStore GC (AEM 5.6 onward)

Option 1: Using Configuration

create new configuration name 

<some path for config>/com.day.crx.sling.server.impl.jmx.GarbageCollectionConfig-DatastoreGarbageCollector.xml with following values

<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:cq="http://www.day.com/jcr/cq/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0"
    jobName="Datastore Garbage Collector"
    schedule="00 00 23 ? * SUN"
    deleteNodes="{Boolean}true" />

This mean that run Datastore Garbage collection every Sunday at 11 PM.

Option 2: Using CURL and CRON job. Create a Cron job on your machine and run CURL command given above.

For Tar optimization

curl -u <UID>:<PASSWORD> -X POST http://<HOST>:<PORT>/system/console/jmx/com.adobe.granite:type=Repository/op/startTarOptimization/

CRX Log Message to look for

com.day.crx.persistence.tar.file.TarFile /crx-quickstart/repository/workspaces/crx.default/data_00000.tar id:0 length:75436032 append:75435008 943288331 optimize: 2420224

Similarly you can use CURL command to different JMX operation listed in


How to identify master using Curl Command And then run backup only on master

ISMASTER=`curl -s -u admin:admin http://HOST:PORT/system/console/jmx/com.adobe.granite:type=Repository | grep -A 1 crx.cluster.master | sed 's/\<\/*td\>//g' | awk 'NR>1' | awk '{ print $1 }'`
if [ $ISMASTER != 'true' ]; then
echo "Not running backup, that is the slave node";
exit 1;
echo "Running online backup this is master Node"
# Your curl command for online backup goes here

Special Thanks to Andrew Khoury from Adobe for providing this CURL command.

Use firebug to get complete URL and then replace it in above curl command.


  1. Hi Yogesh,

    Sorry for such a stupid question, actually I have not passed the delete parameter in curl command.

    Rohit Varshney

  2. This comment has been removed by a blog administrator.

  3. Yeah, this is really a good writing skill you have and I appreciate your writing skill. You are share awesome post for people. I think everybody will be happy by read your post.

  4. Absolute path of backup. If path is not provided and just file name is given, Then it will use current path of installation.draas.biz

  5. Hi
    I am trying to create a backup with some time delay. I have set absolute path in the taget as shown in the following command:
    curl -u admin:admin --data "delay=3&force=true&target=/backup/2014" http://localhost:8080/libs/granite/backup/content/admin/backups/ but I don't see any backup file created. Could you please help me with that. I just wanted a backup to a directory and I don't want any zip file.

    1. Hello Pavan,

      What response you get when you run this command ? Also does backup works when you do it manually by going to http://localhost:8080/libs/granite/backup/content/admin/backups/ ?


  6. This is first time that I visit here. I found so many exciting matter in this particular blog, One thing I would like to request you that pls keep posting such type of informatics blog.
    werkblad keramiek

  7. I have visit this blog first time and totally enjoyed it really. werkblad keramiek

  8. Hi,

    I want one instance of 155GB to be backed up.
    What is the best way to take online backup in AEM?
    Which method is faster?

    1. I used the following command to take back up.It is taking lot of time.
      "curl -u admin:admin -X POST http://localhost:4502/system/console/jmx/com.adobe.granite:type=Repository/op/startBackup/java.lang.String?target=backup.zip"

  9. This comment has been removed by the author.

  10. hi,
    Thanks for this post. Very useful.
    I have one issue, tho. I need to make a differential backup as the full takes too long. The first bkp doesn't finish when the second one starts. Plus the archive is too big and I end up with huge bkp archives.
    Any idea how it can be done?
    I really appreciate any help.