Wednesday, December 12, 2012

How to remove non referenced node from DAM in CQ 5.5

Use Case: Clean your DAM to remove non referenced node.

Solution: You can use following script to remove all non referenced node from DAM.

# !/bin/bash
# Author:
# The host and port of the source server
# The user credentials on the source server (username:password)
#Filter path
ROOT_PATH="/content/dam/<Your PATH>"
ALL_PATHS=`curl -s -u $SOURCE_CRED "$SOURCE/bin/querybuilder.json?path=$ROOT_PATH&type=dam:Asset&p.limit=-1" | tr ",[" "\n" | sed 's/ /%20/g' | grep path | awk -F \" '{print $4 "\n"}'`
echo "$ALL_PATHS"
REFERENCE_COUNT=`curl -s -u $SOURCE_CRED "$SOURCE/bin/wcm/references.json?path=$SINGLE_PATH" | tr ",[" "\n" | sed 's/ /%20/g' | grep path | awk -F \" '{print $4 "\n"}'`
if [ "$REFERENCE_COUNT" == "" ] ; then
  echo "Removing $SINGLE_PATH"
  #curl -u $SOURCE_CRED -F:operation=delete $SOURCE$SINGLE_PATH

Post Cleaning Step:

1) Run Data store garbage collection see or for that)
2) Remove these nodes from publish as well if they are not deactivated already (usually deleting activated node should delete it from publish as well, But please check)

Caution: As always please test this. Also this might not cover Image referenced through CSS or your code. This only covers Image referenced through Image or Image related component.

Note: You can also use JSAWK for better parsing of response.

Monday, November 19, 2012

How to Fix No Such Item repository corruption in CQ5.5

Use Case: You have repository corruption that can not be fixed using consistency check and fix.


Stack Trace:

05.11.2012 00:30:01.476 *ERROR* [pool-5-thread-2] Unable to perform role import javax.jcr.ItemNotFoundException: failed to build path of 795cff24-7fd7-3f5f-a9b1-4724c3a6ff4d
        at org.apache.jackrabbit.core.HierarchyManagerImpl.getPath(
        at org.apache.jackrabbit.core.CachingHierarchyManager.getPath(
        at org.apache.jackrabbit.core.ItemImpl.getPrimaryPath(
        at org.apache.jackrabbit.core.NodeImpl.getPrimaryPath(
        at org.apache.jackrabbit.core.ItemImpl$2.perform(
        at org.apache.jackrabbit.core.ItemImpl$2.perform(
        at org.apache.jackrabbit.core.session.SessionState.perform(
        at org.apache.jackrabbit.core.ItemImpl.perform(
        at org.apache.jackrabbit.core.ItemImpl.getPath(
        at <Some Custom Code>
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(
        at java.util.concurrent.ThreadPoolExecutor$
Caused by: org.apache.jackrabbit.core.state.NoSuchItemStateException: 34228aff-f648-429a-a097-29de8568fe3e
        at org.apache.jackrabbit.core.state.SharedItemStateManager.getItemState(
        at org.apache.jackrabbit.core.state.LocalItemStateManager.getNodeState(
        at org.apache.jackrabbit.core.state.LocalItemStateManager.getItemState(
        at org.apache.jackrabbit.core.state.XAItemStateManager.getItemState(
        at org.apache.jackrabbit.core.state.SessionItemStateManager.getItemState(
        at org.apache.jackrabbit.core.HierarchyManagerImpl.getItemState(
        at org.apache.jackrabbit.core.HierarchyManagerImpl.buildPath(
        at org.apache.jackrabbit.core.CachingHierarchyManager.buildPath(
        at org.apache.jackrabbit.core.HierarchyManagerImpl.getPath(
        ... 18 more

How to Fix (If it is not cluster then ignore cluster steps):

1) Stop all slave
2) In master create a folder called install under crx-quickstart
3) Copy files for gogo shell in install folder (From Here)
4) Use command telnet localhost 6666 from instance or from felix console go to gogo tab

5) Use command g! "login admin <admin password>"
6) Use command "rm [<UUID of Node you are having Problem in above stack trace it is 795cff24-7fd7-3f5f-a9b1-4724c3a6ff4d>]" then use command "save"
7) Then go to crx-quickstart/repository/workspace/crx.default
8) Create a file called redo_X.log where X is same as latest index_X.tar. for example if latest index tar file name is index_9_24.tar then create a file called redo_9_24.log
9) Add following entry in redo log (Based on above stack trace, This is to recreate search Index)
10) 1 DEL 795cff24-7fd7-3f5f-a9b1-4724c3a6ff4d
      1 DEL 34228aff-f648-429a-a097-29de8568fe3e
      1 COM
11) Restart master instance
12) Start slave instance

Note: Before you delete any node, You can do stat on that node to see what node is causing problem. You can do something like,

g! stat 795cff24-7fd7-3f5f-a9b1-4724c3a6ff4d
Path:         {} {}34228aff-f648-429a-a097-29de8568fe3e
NodeId:       795cff24-7fd7-3f5f-a9b1-4724c3a6ff4d    
ParentId:     34228aff-f648-429a-a097-29de8568fe3e    
NodeTypeName: {internal}Group                        
Memory used:  2400                                    
ChildNode Entries:
+ {internal}policy[1] -> c577627a-bb22-4c82-95af-6acc1881302a

Above output shows node is pointing to orphan node, Which can not be fixed by consistency check and fix.

Use help command to find things that you can do with gogo shell.

Caution: gogo shell (Similar to crx console in earlier version) is very powerful tool and cause system damage as well. Please use this tool with care.

Let me know if you want detail about any command in gogo shell.

Tuesday, October 23, 2012

How to manage Hot backup Or Manage Disaster Recovery in CQ

There are various approach, I am listing them with pros and cons

Approach 1: Use clustering, but direct the requests only to one node. In case of problems with this node just switch over to the other node. Essentially this is an active-passive scenario.
+ proven technology, documented and fully supported feature
+ automatic fail over easily possible
- additional license cost
- Latency issue might effect performance
- Managing cluster is sometime difficult

Approach 2: Use replication (on modification, no version, no status update).
+ proven technology, documented and fully supported
+ easy handling, reconfiguration during runtime
- manual reconfiguration of replication in case of switch
- "on modification" only works for cq:pages and not for other type like DAM
workflows, Events, Replication queues. (But if you activate DAM asset it will be there is stand by system)

So, this approach 2 doesn't achieve a "full-standby" system, but it more looks like a way not to loose content. Everything else is probably gone, so it's only for a really worst-case scenario.

Approach 3: Build periodically content packages and replicate them to the standby-system
+ you can also do a provisioning of other staging systems with this content
+ load on the active system predictable, normal editing actions are not
loaded with this
- self-written, not supported
- you can package workflows and events when grabbing them from repository
- Need testing to see if it will work.

Some Questions:

Q: What is best way to create active passive clustering node and what should be I careful about.
A: Create multi node cluster and make sure that DR nodes are not taking any request. You have to careful about that there is not a lot of latency between DR node active nodes.
Also it is very difficult to keep one node from active node as master in case master goes down. There is no harm of having DR as master node but not recommended (As write always goes through master). You can use either felix console or preferredMaster property to set up master in advance. Please read very carefully to understand CRX better.

To make a node master:

Since this is exposed as a JMX you can monitor it and invoke it during run time.

Q: Do I have to take cold backup of all the nodes
A: No, if you are using clustering then taking backup of any node (usually master) is enough.

Q: How about publish instance 
A: It is not recommended to use clustering in publish instance, Unless there is no other way to support some use case. In that case each publish instane is Hotback of each other. Note that you can not recover publish instance from nightly backup (As things might have been updated from the time back up was created). Usually it is recommended to have a backup publish instance which does not take load but configured as replication agent in author.

Q: I am just left with nightly backup, How should I create new publish instance ?
A: In this case you have to find out things replicated from the time last nightly backup was created and now. You can use nightly backup in conjunction with  to support this use case.

How to work with Configurations in CQ

Use Case: You want to create configuration which can be edited at run time using OSGI console.


CQ already comes with predefined configuration that you can find using following Xpath queries

Or using query builder predicate (http://HOST:PORT/libs/cq/search/content/querydebug.html)


From front end you can find all configurations under config tab in felix console

You can also download all configurations using felix console

You can also go to configuration through Component tab

On file systems you can find configuration under /crx-quickstart/launchpad/config folder.

Configuration can be at any of above location but order of precedence of selecting a configuration is,

1) /apps
2) /libs
3) File system

For more information please see

How to Create a felix Configuration:

1) By creating a sling:OsgiConfig node using CRXDE

Step 1:

Step 2:

2) Using xml

3) By Code

How to Read Property:

1) Using generic class (You can create interface)

2) Within bundle

Cool part about configuration that I like most is, You can create environment specific configurations and based on your environment particular configuration will get selected. You can set configurations as given in

Some Questions:

Q: Where does my configuration get stored when I change it through felix console directly

Ans: In CQ5.5 it get stored in repository under /apps/system/config thus get replicated across all cluster. But in CQ5.4 and before it get stored in file system under /crx-quickstart/launchpad/config/<Name of config>.config thus requires changes in all cluster instance.

Q: What if I just want configuration for particular environment

Ans: In that case you have to set up your config for that environment and then create configuration under that node. See example above.

Q: What is recommended way to create configuration

Ans: It is recommended to change configuration through repository and not through felix console if possible. That way you can track configuration in SVN as well.

Q: I see that there is already a configuration under felix console, How can I create one under repository to override that.
Ans: To do that first go to felix console -> configuration -> open configuration and look for pid

Then create exactly same node under /apps/config<Any enviornment> of type osgiconfig

Q. How to create configuration that has drop down of selected value
A. You can do something like,

How to create Configuration factory to locate services

Some time we want to create configuration factories to create factory of configuration (For example Logger configuration)

In order to create such configuration you could use annotation

To read such configuration though supporting class you can use,

  @Reference (cardinality=ReferenceCardinality.OPTIONAL_MULTIPLE,        policy=ReferencePolicy.DYNAMIC, referenceInterface = MyCustomService.class, name="MyCustomService")

How to locate configurations in configuration factory (Note that you might not need to do this if you are using independent configuration factories, for example configuration factories of scheduler) 

Please ask any other question that you want me to add here.

Tuesday, October 2, 2012

How to implement CQ scheduler to run on a event

Use Case:

1) You want to implement a scheduler to run on a specific event. That event could be any thing ranging from any Node action (Add,Modify,Delete) or any replication action (Activate, Deactivate, Reverse)

Pre requisite:



Step 1

Step 2: Handler to perform action

Note: you can find some more code example at

Tuesday, September 11, 2012

How to put DAM asset back to workflow for processing in CQ

Use Case: Something went wrong during Asset upload and not all steps in Update Asset workflow get executed successfully.

Impact: Some time when Assets are not uploaded properly or something went wrong during upload process, not all steps in update asset workflow completes, as a result of which some time metadata or rendition is missing. You can probably run those script again to complete process but easiest way would be to put those asset back to workflow.


You can use following script to do this

# !/bin/bash
# Author: Yogesh Upadhyay
# The host and port of the source server
# The user credentials on the source server (username:password)
#Filter path
#If you want to run for only those asset for which property is missing
#Query to get all Dam Asset
ALL_PATHS=$(curl -s -u $SOURCE_CRED "$SOURCE/bin/querybuilder.json?path=$ROOT_PATH&type=dam:Asset&property=jcr:content/metadata/@$MISSING_PROPERTY&property.operation=not&p.limit=-1" | tr "[" "\n" | sed -e $'s/},{/\\\n/g'  | grep path | awk -F \" '{print $4 "\n"}')
#echo "$ALL_PATHS"
echo "Fixing $SINGLE_PATH"
curl -s -u $SOURCE_CRED -d "model=/etc/workflow/models/dam/update_asset/jcr:content/model&payloadType=JCR_PATH&payload=$SINGLE_PATH" $SOURCE/etc/workflow/instances
sleep 3

Script to change wrong mimeType and dc:format of pdf document

# !/bin/bash
# Author:
# The host and port of the source server
# The user credentials on the source server (username:password)
#Filter path

#Query to get all Dam Asset with extension pdf but with default mime type
ALL_PATHS=$(curl -s -u $SOURCE_CRED "$SOURCE/bin/querybuilder.json?path=$ROOT_PATH&type=dam:Asset&nodename=*.pdf&property=jcr:content/renditions/original/jcr:content/jcr:mimeType&property.value=application/octet-stream&p.limit=-1" | tr "[" "\n" | sed -e $'s/},{/\\\n/g' | grep path | awk -F \" '{print $4 "\n"}')
echo "$ALL_PATHS"
echo "Fixing $SINGLE_PATH"
curl -s -u $SOURCE_CRED "-Fjcr:mimeType=application/pdf" $SOURCE$SINGLE_PATH/jcr:content/renditions/original/jcr:content
curl -s -u $SOURCE_CRED "-Fdc:format=application/pdf" $SOURCE$SINGLE_PATH/jcr:content/metadata
sleep 2

You can actually check for not equals property as well, But that will require custom predicate evaluator for not.

Some more interesting script:

Change thumbnail process arg to add additional thumbnail

# The user credentials on the source server (username:password)
#Keep previous one as is this is not append operation,
#Off course this could be empty if you dont want any thumbnail
PROCESS_PATH=`curl -s -u $SOURCE_CRED "$SOURCE/bin/querybuilder.json?path=/etc/workflow/models/dam/update_asset/jcr:content/flow&type=nt:unstructured&property=PROCESS&" | tr "[" "\n" | sed 's/},{/\n/g' | grep path | awk -F \" '{print $4 "\n"}'`

Remove all thumbnail and just keep original file

# !/bin/bash
# Author:
# The host and port of the source server
# The user credentials on the source server (username:password)
#Filter path
ROOT_PATH="/content/dam<Your path>"
ALL_PATHS=`curl -s -u $SOURCE_CRED "$SOURCE/bin/querybuilder.json?path=$ROOT_PATH&nodename=*thumbnail*png&type=nt:file&p.limit=-1" | tr "[" "\n" | sed 's/},{/\n/g' | sed 's/ /%20/g' | grep path | awk -F \" '{print $4 "\n"}'`
echo "$ALL_PATHS"
echo "Removing $SINGLE_PATH"
curl -u $SOURCE_CRED -F:operation=delete $SOURCE$SINGLE_PATH

Put dam asset in workflow which is created between certain date

# !/bin/bash
# Author: Yogesh Upadhyay
# The host and port of the source server
# The user credentials on the source server (username:password)
#Filter path

#If you want to run for only those asset for which property is missing
#Query to get all Dam Asset

ALL_PATHS=$(curl -s -u $SOURCE_CRED "$SOURCE/bin/querybuilder.json?path=$ROOT_PATH&type=dam:Asset&$DATE_FROM&daterange.upperBound=$DATE_TO&p.limit=-1" | tr "[" "\n" | sed -e $'s/},{/\\\n/g'  | grep path | awk -F \" '{print $4 "\n"}')
#echo "\n"
echo "$ALL_PATHS"
#echo "\n"
echo "Fixing $SINGLE_PATH"
curl -s -u $SOURCE_CRED -d "model=/etc/workflow/models/dam/update_asset/jcr:content/model&payloadType=JCR_PATH&payload=$SINGLE_PATH" $SOURCE/etc/workflow/instances
sleep 3

Please check and in case you want to achieve this through code.

Note: If you have some custom property in your Asset this might override those property. Please make sure that you have tested this script before using it.

Tuesday, August 21, 2012

How to work with Version in CQ

Use Case: 

1) You have to delete specific version from repository
2) You have to create versions using API
3) You have to remove versions
4) Reduce size of repository by removing versions


CQ uses JCR version management to maintain versions

VersionManger is main service to handle most of task related to versioning.

Within CQ in file systems versions are persisted under /crx-quickstart/repository/version and version index under /crx-quickstart/repository/repository/index

Corresponding configuration is under /crx-quickstart/repository/repository.xml (Which is global configuration)

<Versioning rootPath="${rep.home}/version">

Within repository versions are stored under /jcr:system/jcr:versionStorage node, Which is one per repository.

How Versions are created in CQ:

Following actions are responsible for creating version

1) Activation of page (This is controlled by version manager configuration)

Note that from CQ5.5 this configuration is not available by default. You have to create a osgi:Config within repository for this (

You can control number of version created by activation by setting versionmanager.maxNumberVersions property.

2) By sidekick

3) Using API and example

How to find specific version in CQ

1) Using repository
  • Note that not for all node versions are created. Node has to be of mixin type mix:versionable (node.addMixin("mix:versionable")  
  • Once you are sure that node is of correct mixin type, Find that node using CRX explorer or CRXDE light. 
  • Once node is found find UUID of that node

  • Based on UUID you can find out location of version within repository. Within repository versions are stored like this/jcr:system/jcr:versionStorage/<FIRST TWO LETTER OF UUID>/<NEXT TWO LETTER>/<NEXT TWO LETTER>/<UUID>/<VERSION NUMBER> For example for UUID ce472450-f567-459e-8adc-6fd7a3d8a4f3 you can find it's version under /jcr:system/jcr:versionStorage/ce/47/24/ce472450-f567-459e-8adc-6fd7a3d8a4f3/<Version Number>

2) using API

Something like

VersionManager mgr = session.getWorkspace().getVersionManager();
VersionHistory vh = (VersionHistory) node.getParent().getParent();
VersionIterator vit = vh.getAllVersions();
while (vit.hasNext()) {
Version v = vit.nextVersion();

How to Purge Version in CQ

1) Using configuration

2) Using version Purge Tool

From CQ5.5 onward version purge tool is integrated in CQ that you can use to purge versions

for that go to tools -> versioning -> PurgeVersion

Purging version will help you to reduce size of repository a lot. You can also configure your repository for not to create versions for other environments (Like dev or staging) 

Monday, July 16, 2012

How to Clean Up Nodes in CQ / WEM

Use Case:

1) Clean your repository.
2) remove unwanted nodes.

Solution: You can use following package to clean up nodes based on certain criteria.

1) Install this package using package manager

2) Go to http://HOST:PORT/apps/tools/components/cleanNode/run.html

3) Select your options

4) Select run

Note: Please use this utility with care, As it leads to deletion of node from repository. Please DO NOT use it in production without proper testing.

Thursday, July 12, 2012

How to clear replication queue in CQ / WEM

Use Case

Case 1: Your replication queue is blocking and you have no idea why it is not getting processed.
Case 2: Some one accidentally activated thousands of pages and you want to clean them now.
Case 3: You want safe way to remove replication jobs and then activate them again if required
Case 4: You want to clean a dummy or problematic replication agent

Pre requisite : CQ5.4 or CQ5.3 with replication fix pack.

Solution :

1) Install attached package using package manager

2) Go to http://<host>:<port>/apps/tools/components/cleanReplication/run.html Or go to http://<host>:<port>/etc/replication/ click on your agent and you will will have options.

3) First do dry run and see if entries are getting deleted

4) Then check all checkbox and click on run clean replication.

5) There is also a failed replication Listener configuration, You can configure it to send an email in case replication fails.

Here are some screen shot

Here is package:

For CQ5.4:

Q: What will go to failed queue
A: Replication Job failed even after max retry.

Q: Why not use clear replication from replication page to remove entry ?
A: Some time you won't see entry to remove. And some time you have a lot of items in queue that is not easy to remove using UI. Some time you want to delete just specific entry and they are a lot (In that case you can use filter). See use case for more detail.

Note: Since the way replication data is stored in repository changes a lot over time this might not work with earlier version of CQ till CQ5.3. You can also remove unwanted code and change it according to your Need. If there is some problem let me know and given time I can look in to it. 

Special thanks to Henry Saginor from Adobe for providing code for failed replication listener. 

How to Use Dispatcher with Mapped content

Use Case: You are using /etc/map or resource resolution to map your content and dispatcher flush is not working any more.
More information about Sling mapping can be found here

I have seen a lot of customer using sling mapping or resource resolver setting to map /content/<There site> to / to shorten URL. But as soon as they do that, They claim that dispatcher flush is not working. And here is reason why,

1) Dispatcher flush does not take mapping or resource resolver rule in to account to flush cache or invalidate static file.

To understand it better, Here is use case,

1) From example your site URL is and then you shorten it to by mapping rules or resource resolver rule.

2) Suppose your document root is /docroot/htdocs

3) Now when some one try to access page your page will get cached under /docroot/htdocs/en/us/survey.html

4) But when you will activate this page from author page under will get flush, As dispatcher flush agent has no idea about mapping rules.


Now in order to avoid this problem, You should have mod_rewrite rule along with resource resolver and mapping rules.

Here is basic example of rules you want in your mod_rewrite (These rules will differ from site to site)

RewriteRule ^/$ <Your home page>

RewriteCond %{REQUEST_URI} !^/apps/(.*) [NC]
RewriteCond %{REQUEST_URI} !^/etc(.*) [NC]
RewriteCond %{REQUEST_URI} !^/libs(.*) [NC]
RewriteCond %{REQUEST_URI} !^/content(.*) [NC]
RewriteCond %{REQUEST_URI} !^/system(.*) [NC]
RewriteCond %{REQUEST_URI} !^/dam(.*) [NC]
RewriteRule ^/(.*) /content/mysite/$1 [PT]
RewriteRule ^/(.*)\?(.*) /content/mysite/$1 [PT]

Above rule make sure that content at right path is getting flushed.

Some more trick (But will not work 100% see for detail):

You can also do something like

LoadModule headers_module modules/
RequestHeader edit CQ-Handle /content/mysite(/.*) $1 early

Another solution at replication agent level By David to solve this issue,

code for above is found here

Q: Why I will use etc/map or resource resolution if these rules are enough for mapping ?
A: Mod_rewriter rules will take care of just mapping and not link rewriting. You need /etc/map or resource resolver rules for link rewriting.

Q: Why dispatcher flush do not take mapping rules in to account ?
A: There is already an enhancement request for this.

Q: I don't want to write these rewrite rules, How else I can handle this ?
A: One option is set your stat file level to 0 (Which is default), This might invalidate all resource under invalidation tag of dispatcher.any upon activation. But again if you watch your dispatcher log carefully, wrong file will get evicted on activation. But you still see updated content due to invalidation.

Q: How about vanity URL ?
A: Well thats problematic. You might want to stick with stat file or write mod_rewrite rules for them as well. 

On side note, understanding /statfileslevel is very important to improve performance of your website.

Thursday, June 14, 2012

How to Rotate Logs in CQ / WEM

Use Case:

  • Rotate all logs in CQ
  • Logs are taking a lot of space in file system

Till CQ5.4

There are different kind of logs in CQ. Please refer for that

Here is process to rotate all logs,

CQ Logs:

crx-quickstart/logs/error.log :

You can rotate error log using sling:OsgiConfig under /libs/sling/config/ Please override this under /apps/sling/config/ You can additionally have environment specific configuration as or config.publish

crx-quickstart/logs/request.log and crx-quickstart/logs/access.log:

CRX Log:


You can rotate CRX log through configuration in /crx-quickstart/server/runtime/0/_crx/WEB-INF/log4j.xml by changing configuration of 

<appender name="error" class="org.apache.log4j.RollingFileAppender">
        <param name="File" value="crx-quickstart/logs/crx/error.log"/>
        <param name="maxFileSize" value="10MB"/>
        <param name="maxBackupIndex" value="20"/>
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%d{dd.MM.yyyy HH:mm:ss} *%-5p* %c{1}: %m (%F, line %L)%n"/>


These logs are not controlled by sling or log4j. You have to use apache log rotate utilities to do this. You might have to do following to rotate these logs (By default they are not rotated)

export CQ_LOGDIR

------- ADD ------



And change your rotate option as,

exec $jvmExe 2>&1 | /usr/sbin/rotatelogs "$CQ_LOG.%Y%m%d" 86400
| /usr/sbin/rotatelogs "$CRX_LOG_STDOUT.%Y%m%d" 86400 | /usr/sbin/rotatelogs "$CRX_LOG_STDERR.%Y%m%d" 86400

That mean rotate both log after 86400 second (1 day).

Server Logs


Server logs can not be rotated at sling level. You need to rotate these logs at OS level using logrotate utility similar to

For this in serverctl script you can do following (Adjust log file location accordingly),

export CQ_LOGDIR

------- ADD ------


And change your rotate option as,

exec $jvmExe 2>&1 | /usr/sbin/rotatelogs "$CQ_LOG.%Y%m%d" 86400
| /usr/sbin/rotatelogs "$CQ_LOG_ACCESS.%Y%m%d" 86400

That mean rotate both log after 86400 second (1 day).

If you do not have logrotate utility, OOTB CQ support rotation based on size (Not on date or number), For that go to /crx-quickstart/server/etc/server.xml and add following line



Launchpad Logs


You can configure launchpad error log using felix configuration or creating a custom configuration in crxde may be under /apps/sling/config. Please see for more information. You can also control sling logs rotation though /crx-quickstart/launchad/

Look for, following properties'.'yyyy-MM-dd${sling.home}/logs/error.log

Dispatcher log:

For dispatcher log you have to use (Similar to server log). Something like

DispatcherLog '|/usr/local/apache/bin/rotatelogs <Path-to-your-log>/logs/%Y-%m-%d-dispatcher.log 86400'


Good thing about CQ5.5 is there is not different locations to control log rotation. You can control all log rotations through sling configuration One thing to note here is, unlike CQ5.4 ../logs/<your-log> will point to logs folder under CQ root and not under /crx-quickstart/logs. If you want your logs to go under /crx-quickstart/logs you have to use log location as logs/<your-log> in sling configuration.

It is always recommended to modify your log configuration under /apps/<Path>, That way you can port it to different environment.

Some more reference 

Note: For OS level log rotation, You can use any log rotation you want and give path to CQ log files.

You can try this option as well, If log rotate utility option is not working after config through serverctl.

$NOHUP $jvmExe | /usr/sbin/rotatelogs "$CQ_LOG.%Y%m%d" 86400>> /dev/null 2>&1

Thursday, June 7, 2012

How to add a custom Login Module in CQ / AEM

Use Case:

1) You want to create a custom login module in CQ / AEM
2) You already have a custom login module in CQ5.4 and integrate it with CQ5.5

Prerequisite: Please check


CQ 5.6 and less

Any custom login module should be added in CQ5.6 and less as a JAAS module. That mean CQ should have access to module during load time.

Here is set of steps you might have to do to create custom login module

1) Create a OSGI bundle with class that extend Jackrabbit AbstarctLoginModule (Example

2) Create bundle as frangment bundle attached it to

<Export-Package>....... Package you want to export .... </Export-Package>
lib/<Your custom jar file>.jar,

3) Deploy and start your bundle as start level 15. You can do following things for that

curl -u admin:admin -T <Your custom module bundle>.jar http://<host>:<port>/apps/<Your path>/install/15/<Your login module>.jar



4) Change repository.xml 

Open the file crx-quickstart/repository/repository.xml (on the server's filesystem) and remove the <LoginModule>...</LoginModule> element.
Find the <SecurityManager> element and add the WorkspaceAccessManager as shown below:
<SecurityManager class="">
<WorkspaceAccessManager class=""/>
<UserManager class="">
<param name="usersPath" value="/home/users"/>
<param name="groupsPath" value="/home/groups"/>
<param name="defaultDepth" value="1"/>

5) Create custom jaas config

Now create the file crx-quickstart/conf/jaas.conf. In this file you would have something like this: {
<Your login Module class name> sufficient sufficient;

6) Change start up script

Update your crx-quickstart/bin/start script with this jvm parameter (Replace /path/to/your/jaas.conf with the real path to the jaas.conf file):

7) Restart your CQ

Note: if you already have Non osgi version of your custom login module, You can use to convert to OSGI bundle. Make sure that you follow step 2.

To debug custom login module, you can follow steps mention in

Note that LDAP login module in CQ is good example of custom Login Module.

AEM / CQ 6

From CQ6 onward, Login module can now be configured as pluggable login module in OSGI. To implement Login Module in CQ6 onward you can

Option 1:

implement Here is example of Pluggable Login Module with Authentication handler

More info:

Option 2:

You can also use OAK external Login Module To create custom login module (Which you register and service) as well. Example of that would be LDAP Login Module in AEM6

Here is one more example of how to use this in AEM 6 onward:

You do not have to restart your instance after creating custom login module in AEM 6.

Saturday, June 2, 2012

How to add a new supported language in CQ / WEM

Use case: 

  • You want to add new language to CQ
  • Change display language options in translator grid
  • Change language name and default countries 


You can access translator UI in CQ with following URL http://<HOST>:<PORT>/libs/cq/i18n/translator.html

Create new language location for Dictionary

Go to CRXDE lite (or your favorite JCR browser) and add this structure (assuming /apps/myproject/i18n as a typical location for custom apps):
/apps/myproject/i18n [sling:Folder]
    - de [nt:unstructured] [mix:language]
        + jcr:language = de
    - fr [nt:unstructured] [mix:language]
        + jcr:language = fr
Then reload the translator and the path /apps/myproject/i18n should show up in the drop-down at the top.
Note: the translator will only save translations for languages that are actually present underneath the path (e.g. /apps/myapp/i18n), others will be skipped.

Change List of Languages Display in Grid

To change what languages are shown in the translator's grid, create a multi-value string property /etc/languages/languages (node structure must be created), containing the iso language codes (e.g. ["de", "fr", …]). The default is hard coded in the translator and maps the ootb CQ languages: ['de', 'fr', 'it', 'es', 'ja', 'zh-cn']

Add supported language in WCM

  • You need to overlay/overwrite /libs/cq/security/widgets/source/widgets/security/Preferences.js to change the list of available languages (used in security admin)
  • and overlay /libs/cq/security/content/tools/userProperties (inside change language list under items/common/items/lang/options - similar dialog to Preferences.js) (used for user drop-down in the top right on admin consoles)
  • Then you need to add the new language somewhere in the repo, e.g. /apps/yourapp/i18n, following the layout as under /libs/wcm/core/i18n (see also the above section on the translator UI). Stuff under /libs must not be changed!
Here is a sample package:
  • adds /apps/test/i18n with translation of "New..." in zh-cn ("simplified") and zh-tw ("traditional") languages
  • overlays /apps/cq/security/content/tools/userProperties and /apps/cq/security/widgets/source/widgets/security/Preferences.js to add the zh-tw language
  • to check, set the current user's language to "Traditional Chinese" and look in the siteadmin: the first button in the toolbar should now be "traditional" instead of "New..."
change language name and default country 

Note: since 5.5 load 21
This means:
  • language titles
  • country names
  • default countries (for codes like "en", "de", etc.)
A language list is stored under /libs/wcm/core/resources/languages. It can be overlayed by copying it to /apps/wcm/core/resources/languages and changing or extending the list there. If you do so, also change the configuration of the language manager set langmgr.list.path = /apps/wcm/core/resources/languages at 

The default countries, which come into play when resolving the country for a language code such as "en" ("en_GB" would be such a default country), and in turn for displaying flags in the UI, are configured in this list as well. A property defaultCountry on a language node must contain the full code, such as "ja_jp", which would define "jp" as default country for "ja".

Information is provided by Alexander Klimetschek  from Adobe. Many thanks to him. 

Wednesday, May 2, 2012

How to use dispatcher to serve cache content if render is unavailable in CQ / WEM

Use Case: You want to serve cached content if all the renderer in dispatcher.any is unavailable

Prerequisite: You need dispatcher version 4.1.0. You can download latest dispatcher from package share. You can also connect to Day Care to get latest dispatcher release.

Problem: Currently If all renderer is unavailable, You get "502 Bad response code". It might possible that all your pages are already cached before render went down. It will make sense to serve cached content in that case.

Solution: You can use following parameter in dispatcher.any for this,

/ServeStaleOnError "1"

You have to restart apache after making changes. You can add this tag any where at /website level. So it should be,

  # first farm entry (label is not important, just for your convenience)
/ServeStaleOnError "1"

If you want to serve custom error page if pages are not cached. Feel free to modify httpd.conf and add following line to it,

ErrorDocument 502 <Path for custom 502 page>

This is true for any error page. More information can be found here

If you are using this make sure that DispatcherPassError is set to 0. More information about this can be found here, 

You can also handle 5XX error effectively by adjusting following params in dispatcher.any
      # Page gets contacted when an instance returns a 500
    /retryDelay "1"
    /numberOfRetries "5"
    /unavailablePenalty "1"
Again You can find more information about it on Day Doc. 

Another cool thing you can do with latest dispatcher is, You can use * for client header section, So that you don't have to add all entry. Something like

 /clientheaders {*}

Feel free to comment if something is not clear.

How to Associate Digital Right Management (DRM) with a asset in CQ5.5

Use Case: You want to to associate licensing information to an Asset in DAM before serving it to user.

Solution: With CQ5.5, Now you can associate Licensing information to any Asset before serving this to user.

To do this, You have to add additional metadata to Asset called xmpRights:WebStatement that will internally point to any resource that you want to show before actual Image. CQ OOTB has one example under /content/dam/geometrixx/drm

You can also use Adobe Drive Integration with PS or Bridge to make these changes as well. Information about Adobe drive and Bridge can be found here

You can also modify asset editor page to add external metadata as well.

First, we need to create the necessary folder structure:

curl -u admin:admin -Fjcr:primaryType=sling:Folder http://localhost:4502/apps/dam

curl -u admin:admin -Fjcr:primaryType=sling:Folder http://localhost:4502/apps/dam/content/asseteditors/application/pdf

curl -u admin:admin -Fjcr:primaryType=sling:Folder http://localhost:4502/apps/dam/content/asseteditors/image/jpeg

curl -u admin:admin -Fjcr:primaryType=sling:Folder http://localhost:4502/apps/dam/content/asseteditors/image/tiff

Then we can copy the nodes from libs:

curl -u admin:admin -F:operation=copy -F:dest=/apps/dam/content/asseteditors/ http://localhost:4502/libs/dam/content/asseteditors/formitems

curl -u admin:admin -F:operation=copy -F:dest=/apps/dam/content/asseteditors/application/pdf/ http://localhost:4502/libs/dam/content/asseteditors/application/pdf/formitems

curl -u admin:admin -F:operation=copy -F:dest=/apps/dam/content/asseteditors/image/ http://localhost:4502/libs/dam/content/asseteditors/image/formitems

curl -u admin:admin -F:operation=copy -F:dest=/apps/dam/content/asseteditors/image/jpeg/ http://localhost:4502/libs/dam/content/asseteditors/image/jpeg/formitems

curl -u admin:admin -F:operation=copy -F:dest=/apps/dam/content/asseteditors/image/tiff/ http://localhost:4502/libs/dam/content/asseteditors/image/tiff/formitems

 curl commands to create the fields for the image editor:

curl -u admin:admin "-FfieldLabel=Right Management URL" -Fjcr:primaryType=cq:Widget -Fname=./xmpRights:WebStatement -Fxtype=textfield

curl -u admin:admin "-FfieldLabel=Right Management URL" -Fjcr:primaryType=cq:Widget -Fname=./xmpRights:WebStatement -Fxtype=textfield

curl -u admin:admin "-FfieldLabel=Right Management URL" -Fjcr:primaryType=cq:Widget -Fname=./xmpRights:WebStatement -Fxtype=textfield http://localhost:4502/libs/dam/content/asseteditors/image/formitems

curl -u admin:admin "-FfieldLabel=Right Management URL" -Fjcr:primaryType=cq:Widget -Fname=./xmpRights:WebStatement -Fxtype=textfield http://localhost:4502/libs/dam/content/asseteditors/image/jpeg/formitems

curl -u admin:admin "-FfieldLabel=Right Management URL" -Fjcr:primaryType=cq:Widget -Fname=./xmpRights:WebStatement -Fxtype=textfield http://localhost:4502/libs/dam/content/asseteditors/image/tiff/formitems

You can also create your own page under dam/components/drm/pages/license, But any location is fine as long as you have right URL.

After correct URL you might see your License agreement file before Image is served

You might also have to change configuration to accept license agreement. For that please do following

·         Got to http://<host>:<port>/system/console/configMgr
·         Search for “DAM Event Recorder”, click it to change the configuration
·         Click “Enable this service”.
·         Save and try accepting the license again.

After User Agree with License, They should be able to view page


This is still work in progress and might change in future releases.

How to reduce downtime for consistency check and fix CQ5.4 / WEM

Use case: Consistency check and fix takes days to complete some time, We can not afford this much down time.

Approach:  There are different kind of inconsistencies like,

Orphaned child
Error message : NodeState CHILD_UUID references inexistent parent uuid PARENT_UUID
Current solution : restore entire backup

Parent referencing inexistent child
Error message : NodeState PARENT_UUID references inexistent child {}CHILD_NAME with id CHILD_UUID
Current solution : consistency check/fix should repair such issue Or if you know UUID then slective UUID check
and fix should resolve this issue.

Child referencing invalid parent
Error message : ChildNode has invalid parent uuid: INVALID_PARENT_UUID (instead of VALID_PARENT_UUID)
Current solution : Use CRX Console

Parent not referencing existing child
Error message : javax.jcr.ItemNotFoundException: failed to build path of CHILD_UUID: PARENT_UUID has no child
entry for CHILD_UUID
Current solution : Use CRX Console

Node already exists
Error message : javax.jcr.ItemExistsException: at
org.apache.jackrabbit.core.NodeImpl.internalAddChildNode( solution : consistency check/fix should repair such issueSearch index inconsistencyError message : WARN NodeIteratorImpl: Exception retrieving Node with UUID:
003171fe-e2e8-457b-a3af-f74eed12c1b9: javax.jcr.ItemNotFoundException:
003171fe-e2e8-457b-a3af-f74eed12c1b9Current solution : recreate search index or use redo.log

More information can be found here,
Problem with full consistency check and fix is it even traverse nodes which are not corrupted. And key to do fast consistency check and fix is finding all problematic entry in advance and then doing selective consistency check and fix. Another problem is corruption will not be revealed unless node is accessed.
CRX already provide way to do consistency check and fix using UI, But it does not resolve all inconsistency, though you can use this feature to touch each node and find inconsistencies 

For CQ5.4 or CQ5.5

Or use following CURL command

Phase 1: Finding all entry.
rm login.txt
curl -c login.txt "http://<host>:<port>/crx/login.jsp?UserId=admin&Password=admin"
rm progress.txt
curl -b login.txt -f -o progress.txt -d "type=traversal&traversalLogEach=true&fixInconsistencies=true&rootNode=/" "http://<host>:<port>/crx/config/check_frame.jsp"

Phase 2: Then You can use following program to find out all inconsistencies in log
  • You can use attached file (Unzip files) to monitor your crx log (/logs/crx/error.log in CQ5.4 and /logs/error.log in CQ5.5) file and send an email in case something is wrong. To use this application
  • Compile java file using javac -classpath .:mail.jar:activation.jar
  • Create a cron Job that runs every hour to monitor your log file (For windows you can use service)
  • Use following command to run command java -classpath .:mail.jar:activation.jar FindCorruption
  • Make sure that You put all the files attached in same location Or change PROPERTY_FILE_LOCATION property in
  • Configure your mail server and output folder location information in For configuring email server please check
  • What you expect is an email when program find corruption pattern in log. This also send you location of all files you need to quickly fix inconsistencies "that can be fixed by consistency check and fix"

You can modify this program to just scan logs and output instruction some where else.

Simple Java Program to find pattern can be found here

Note: This is just an example, You can write your own version using powerful parsing scripting languages to do same task. Again this is not supported by Adobe or by Me, Feel free to add your comment though of how it can be improved.