Wednesday, October 7, 2015

How to extend Replication Page Process for workflow in AEM

Use Case: You want to extend existing replication page process and add your own logic during workflow step. Some of common use case could be,
1) Activate Asset as soon as it is uploaded to DAM


Here is sample code you can use for this,

What this code is doing is trying to find DAM asset path from workflow item and then activating DAM asset instead of activating just original node when OOTB activate page workflow is used.

Once you add your code, you can add this as workflow step in your workflow. If you are extending model.xml for workflow then it will look something like this,

            description="A process to activate a page or asset"
            title="Activate Original Asset"

Also you have to make sure that you have right dependency in your pom.xml. If you are using AEM6.1 then you need to use


As usual if you have any question or comment please let me know.

Friday, March 13, 2015

How to set up Shareable Docker Image for Adobe CQ/AEM

Use case

  • Large Set up time for developers
  • Inconsistent environments across all boxes
  • Flexible in container testing
  • No Single repository for whole system artifacts 

What is Docker

Docker is a platform for developers and sysadmins to develop, ship, and run applications. Docker lets you quickly assemble applications from components and eliminates the friction that can come when shipping code. Docker lets you get your code tested and deployed into production as fast as possible.
More information about Docker can be found here
Check Docker Cheat sheet:

Set up

Docker requires two part to work,
  • Docker Hub or Docker registry (Where images are stored). You don't have to set up this part. We already have a private docker registry set up for you.
  • Docker client (To work with docker images). You need to install this in your local. Please check and based on your OS you need to install one. I have instruction to install it on Redhat Linux.

What would you get after following below instruction ?

  1. A Private Docker Hub, where you can manage AEM docker package. 
  2. A docker Image that will have author, publish and dispatcher instance.
  3. Steps to manage your docker hub and local image and make changes to them.

Assumption: I have docker registry set up for Red hat Linux. You might need different steps for different OS. I assume that on your OS mount you have a mount for /export. If this mount is not there then change paths in script.

Docker is only supported FOR RED HAT LINUX VERSION > 6.5 
(Use command cat /etc/*release to find your linux version)

Set Up Private Docker Registry


If you are using redhat then 


Set Up docker Image using docker file

Once docker is install create a dummy directory and put your jar file, license file, Hotfix and other files in to it. More information about Docker file


1) You have AEM jar file
2) You have License File
3) You have your version of JDK
4) You have compiled HTTPD
5) They all are in same folder where Dockerfile is

For this create a file called Dockerfile and copy below. MAKE CHANGES BASED ON YOUR REQUIREMENT

Once Docker file is there build and create your private docker image using following command. This will create a shareable docker image

Once Docker registry is installed and you have shareable image, now you can install docker client on any other machine

Set up Docker Client

Prerequisite: Based on your OS please install docker client using

Change default path if there is not enough space

Docker by default get installed under /var/lib/docker in Red Hat. Make sure that you have enough space there. If not then you have to do following.

Set Up docker for Non SSL version

By default Docker uses SSL to communicate to docker registry. If you do not have valid cert installed then you might need to do following

Set up local docker instance using docker client

Do following to start your local AEM instance using docker registry and docker client

How to update docker image

Please follow below step to update your docker image

Symptom: You are not able to access your instance from external machine
Solution: Try to restart your machine using 'reboot' for linux box and restart in mac
Symptom: Docker is not starting on mac
Solution: Make sure that you have localhost entry in /etc/hosts file. e.g. localhost. then do following

boot2docker delete
boot2docker download
boot2docker init
boot2docker up
# End of this you will be asked to add some env variable in your ~/.bash_profile
# Open bash profile and add them

Symptom: Getting following error in mac
FATA[0032] An error occurred trying to connect: Get dial tcp i/o timeout

boot2docker stop
boot2docker destroy
boot2docker init
boot2docker up
docker version

Symptom: On Mac local box not able to access instance using localhost:<port number> (

# Run following command
echo "$(boot2docker ip) localhost" | sudo tee -a /etc/hosts
# Then access your instance using localhost
# If you want to create other vhost then change above command with your vhost. For example
echo "$(boot2docker ip)" | sudo tee -a /etc/hosts

Some use full docker command

Upload content

You can use Recap tool to install content once image is set up. You can even put this recap tool as part of your image when you are creating docker image. Just put them under install folder. 

Note that every time you update your image, Uploaded content or local changes might get overriden. You have to commit your changes using docker commit and then apply update on top of that and then share your image.

Useful resource:

Note: As usual let me know if you have any question.

Wednesday, December 10, 2014

How to Set Up Clustering In CQ/AEM 6 using MongoDB


With CQ / AEM 6 TarPM is not supported any more. AEM 6 ships with Oak which for now support TarMK and MongoMK Microkernal OOTB. More information about what is New Can be found from . With this change Support from Clustering is moved to actual storage layer it self (Which make more sense, given supporting all issues for clustering in earlier version). TarMK does not have replication or sharding feature so it comes down to MongoDB which support replication and sharding and hence enable High Availability (HA through replication) and Scalability (Through Sharding, Though this is still a question ?? See note below) through clustering in CQ /AEM 6.

Here we will give step by step instruction of how to set up clustering using MongoDB in CQ

Pre requisite:

There are two cases for setting Up Replica Set:

Set up a new MongoDB Instance:

  • Set up additional MongoDB instance based on instruction above
  • Start any one of instance using ./mongod --port <Your Port> --dbpath <Your DB Path> --replSet <Replica Set Name could be any thing> &  
  • You can also use configuration file to do that. More instruction here
  • Once Mongo DB is started you can add additional replica using following instruction 
  • Once Replica set is up, Now set Up AEM
  • Then You can go to each Mongo Instance and check of data is coming using Mongo Log
Convert Existing Mongo Instance:

  • Stop you AEM instance
  • Use Following instruction to convert Mongo to replica
  • Once this is set Change AEM start script to add mongo replica instance as given in approach one 
  • start your AEM instance
  • AEM should be part of replica set now


1) Mongo Replication Only Provide High Availability (HA) it does not provide scalability. For scalability you need to use Sharding feature provided by Mongo. However I am not sure what would be best key to create shard on for Mongo. You can create Shard based on _id attribute. More information about sharding can be obtained here  . If you are using Sharding I would suggest to use sharding with replication (Shard and then replicate shard instance) to provide both HA and scalability.

2)  There are many feature available in Mongo Replication where you can make certain replica instance read only (Data Center replica), you can use this to avoid high latency across Data Center here is all configuration you can do on Mongo

3) MongoDB recently released MMS to monitor and deploy Mongo Cluster easily. This will be useful if you are worried about administrative cost for Mongo 

4) If you don't want to store large documents in Mongo feel free to use custom Data Store using instruction here

5) Mongo Recently launched another feature of pluggable datastore. You can use this for faster read and write based on your requirement (For example Primary with high Write Enabled Storage Like SSD or something and read with cheap storage). More info here (Official Doc yet to come)

Finally .... Some more Mongo Command ...

Special Thanks To Nelson Mei for Setting up POC for Mongo with AEM

Thursday, November 13, 2014

How to use impression Service In CQ/AEM

Use Case:
  1. You often have case where you want to use Impression service provided by CQ to do custom operation for example finding top 10 most viewed page or sorting all page based on there popularity.
  2. It might possible that your impression data (Page Views) is in external system and then you want to import those data as impression in CQ to have more application context.
  3. You want to aggregate all data across all publish instances.

Approach 1:

Creating your Own Impression service

You can create your own impression service by extending here is example
Supporting class

You need to embed following dependency for this
Here is example of how you can use this service

Now you can import data from external system (GA, Site Catalyst, Kafka) and then populate it using this service to your instance.

Once you are ready with all data you can use following service to use data,

Actual Implementation

Approach 2:

You don't want to write your own service as mentioned in Approach 1 and use OOTB service available to you. Only problem with this is, You have multiple publish instance and some how you want to combine all data in to one so that you get accurate picture. It kind of tricky to get all data from all publish instance (through reverse replication) and then combine them on author and then push them over again. However you can use one instance to collect all stat data (king of single source of truth and then replicate it back to all instance every day)

  • Make sure that you enable page view tracking by adding following line

           <cq:include script="/libs/foundation/components/page/stats.jsp" />
  • Then configure all publish instance to point to one DNS using following config (You can always override this under /apps)
  • make sure that pageviewstatistics.trackingurl is pointing to single domain (You need to create a domain, something like that will be stand alone CQ instance to take all impression request)
  • Now you have consolidated page impression on one machine
  • You can easily write a schedular which will run every night and reverse replicate all data to author instance.
  • Once it is on author instance you can use replicator service to replicate to all other publish instance
  • Then you can use code mention in approach 1 to get popular resources.

Note: You can always use GA or something to track data. This is more useful if you want to do something internally and not want top share data with GA.

As usual feel free to ask any question you have.

Friday, November 7, 2014

How to make Instances SSL context aware in CQ/AEM

Use Case:

Most of the times we terminate SSL on load balancer or at dispatcher and communication to publish happen over http. In this case publish server is often not SSL aware and any request specific operations (For example relative path redirect or Link rewriting) happens over http. For example if you do something like request.sendRedirect("/somepath") from server it will get redirected to http://server-name/somepath or when you will do request.isSecure() it will return false. Operations like externalizer.externalLink(resolver, "mydomain", "/my/page") + ".html"; will also return http version of link.


Option 1:

Let all links be http and then do force redirect on dispatcher or Load Balancer. For dispatcher rule can be as simple as this,

RewriteCond %{HTTPS} off
RewriteRule ^(.*)$ https://%{HTTP_HOST}$1 [L,R=301]


  • Not SEO friendly (Unless it is relative link).  
  • Can not embed as portlet or Iframe (Cross Site include error) over https.

Option 2: 

If filter is configured, it looks for header configured in configuration and if header value matches configured value context on publish is considered secure and request.isSecure() will return true.

In above case if X-Forwarded-SSL is present with value on then servlet context is secure.

Note: This might not work in version latest CQ version. You can build your own version from trunk or from

One working version of file can be downloaded from here

As usual let me know if you have any question. Special thanks to Shenghao Huang from LinkedIn for finding this.

Wednesday, November 5, 2014

How to use Sling Models in CQ5.6

Use Case: Use Sling Model in CQ5.6

Background: Sling model brings powerful way of mapping your resources to Java Objects (Beans) . It is supported OOTB in CQ6 however we can use it in CQ5.6 as well.


First add sling models as dependency in your code (In Parent pom using dependency manager is preferred)

Then update your reactor pom (Which actually creates your CQ package using content-package-maven-plugin) you need to add following as dependency and then embed them in to your project (If install path for system is not present you can use any other path you want)

Thats it .. Models are available as service in your OSGI environment.

 To use model go ahead and create your own bundle and them add these as dependency. You should be able to write, deploy and run code against model. Note that you might have to make these model available to osgi using Sling-Model-Packages

Once model is deployed correctly you should be able to see them under sling-model tab in status in felix console or by going to HOST:PORT/system/console/status-slingmodels

Some example code using models

Note: Sling model version number can change as it evolves in future. Make sure that you update model dependencies version accordingly. You can also use Sling testing framework to test sling model which is pretty cool as well. There is also an example of deployable Model package which you can use.

As usual feel free to give your comment and feedback and let me know if you want me to add something else here.

Thursday, October 16, 2014

How to Connect to External Data Source (Teradata / MySQL / Oracle) in CQ / AEM

Use Case: Connect to external datasource using CQ


1) My SQL

Step 1: Create OSGI version of mysql Jar file

  • Click next and then select add external. Select jar file you downloaded above and select next
  •  Give Project Name -> Select Location -> Make sure that Analyze Library Content is checked -> In Target Platform select an OSGI framework -> from drop down select standard -> Check unzip jar file and update reference -> click finish

  • An Plugin Development prospective will open 
  • Click on export tab and make sure that all dependencies are exported (If they are not there, Click on add then select all)

  • Click on Dependencies and make sure that Imported packages are there. If it is not there click on MENIFEST.MF tab and then add following import statement 
  • Once all import and export statement is added your MENIFEST.MF will have following data

  • Then right click on your project and then select export 
  • Select Deployable Deployable plug-in and fragments

  • Click Next and select your plug-in 
  • From Directory section select location where you want to upload.
  • Click finish. This will store OSGI jar in to location you selected.
  • Once you have MySQL Osgi bundle. Install that bundle either using felix console or adding it through deployment process

  • If you are getting error saying javax.Naming not getting resolved then you have to add following line in under crx-quickstart folder
org.osgi.framework.bootdelegationjavax.naming.*, ${}
  • Once bundle is up is running, You need to go to felix osgi config. Search For JDBC connection pool and then click '+'
  • Fill all the information about your datasource

  • Go to your application logic and test your code

If you are having trouble creating osgi version of mysql jar file, You can download it from here

2) Teradata

  • Upload final jar file in CQ, make sure that it is active

  • Open OSGI config to create config for teradata connection pool

  • You can then test it using same code (Just use teradata data source)

You can also download OSGI version of teradata jar file from here

Note: If You want to call instance of your Connection from Java class (Not from service) You can do something like this,

Thursday, August 28, 2014

How to Use Sessions and Resource Resolver through Service Authentication In AEM6

Use Case: As per and using admin session and admin resource resolver through ResourceresolverFactory is now deprecated. Using Service based Authentication for Resourceresolver and Respository session solves problem like (Directly From Sling Doc),
  • Prevent over-use and abuse of administrative ResourceResolvers and/or JCR Sessions
  • Allow services access to ResourceResolvers and/or JCR Sessions without requiring to hard-code or configure passwords
  • Allow services to use service users which have been specially configured for service level access (as is usually done on unixish systems)
  • Allow administrators to configure the assignment of service users to services


Lets see we have two user "alice" and "bob", with following property,
  • "alice" only have READ access to document under /content/somepath path
  • "bob" has both read and write access to document under /content/somepath path
Now we have two service "ReadService" and "WriteService", with following property
  • ReadService should only be allowed to read anything under /content/somepath path
  • WriteService should be allowed for both read and write under /content/somepath path

Assume your package name is

Step 1: Create ReadService and WriteService using resourceResolver Or adminSession using new Authentication Service based API
Step 2: Create ReadService same way
Step 3: Update config by creating osgi:config node under /apps/<your-apps>/config.<Place where you want to run this>/ you can directly go to osgi config through Felix console and change this as well look for  “Apache Sling Service User Mapper Service” for that.

Syntax for service mapping to user is ‘serviceName [ ":" subServiceName ] “=” username’.
and Entry of OSGI config will look like this,

After installing the bundle and configuration and code, You would see something like this in log

*INFO* alice
*INFO* <node type of somepath>

*INFO* bob
*INFO* <node type of somepath>
*INFO* Successfully saved

If you need to use admin session for the configuration you can do something like in osgi config above. Good practice is to have these session based on groups depending upon which group have access to what service.

You might need following dependencies in your POM for api to be available

Please check with some of the changes in AEM6.1 of how to use this.

As always feel free to ask any question you might have.

Wednesday, July 30, 2014

How to Remove White Space From Generated HTML In CQ (Or In general)

Use Case: There are a lot of white spaces in generated Output of CQ increasing size of page and decreasing load time.


Option 1: You can use trimDirectiveWhitespaces directive in jsp response. something like

<%@page  trimDirectiveWhitespaces="true"%>

Problem: Using this directive can cause white space to be removed from taglibs for same property. To avoid this issue make sure that you manually add space there. For example if you have tag lib like <Something class="${test1} ${test2}" class2="test"> replace it with <Something class="${test1} ${space} ${test2}" class2="test"> where ${space} is actual space " "
This approach might not work with Slighly framework.

Option 2: Use %><% tags to start and end scriplets tag and in between html tag

Problem: Code very hard to read and not pretty.

Option 3: Create your own tag library and using html parser remove white spaces during run time. for that check

And code to remove White space would be

Problem: Maintenance of your own tag library. Can Miss Some condition. Have to wrap up your code with tag lib.

Option 4 (Preferred): Use Google Page Speed Module at apache.

Link to Module:
Link to All available Filters for Module:
Instruction of how to install and build:

Steps to integrate it in Dispatcher:

1) Create Apache module using step above. This will give you and

2) Move this file to <Apache location>/modules

3) Change permission (to daemon:daemon (Or Your Apache User)) and permission level (766) using chown and chmod command

4) Open conf/httpd.conf and add following line (If some include is already there ignore that)
Include <Apache Location>/conf/pagespeed.conf

5) Create a folder called <Doc Root>/mod_pagespeed/

6) Add pagespeed.conf under <Apache Location>/conf
* Make sure that All paths mentioned in conf file exist.

7) Restart Apache

1) Only Apache module is available. If you are using IIS or any other web server then there is no module yet.
2) You might have to do build distribution for your own OS if above module build does not work (One attached here is build for Red Hat Linux).
3) When you upgrade your Apache make sure to upgrade your module as well. If there is no distribution for newer version of apache, then also you are out of luck.

Note: Test above methods before using them in production. Feel free to ask any question you have.

Monday, July 28, 2014

How to include all CQ dependencies in CQ6

Use case: Prior to CQ6 you have to add dependencies for each class you are using in your pom.xml, Way to find dependencies was (Maven org or using adobe central through dependency finder HOST:PORT/system/console/depfinder). With CQ6 now all dependencies are provided through one artifactID.

Prerequisite: Maven, CQ Project Set Up 

Solution: Include following line in your dependency management for your pom.xml (Depending upon project this could be at any location usually it is your Project parent pom)

<!-- All Your Third party dependency -->
<!-- AEM 6.0 -->

      <!-- All AEM Dependencies for JSP compilation -->


      <!-- All AEM 6.1 Dependency added to end of file -->




Note that this version could change depending upon new releases of CQ, You can track them from

Some Trick: Note that above will include all AEM-API dependencies, other way to check what minimum dependency is needed is to create a multi module project using AEM plugin for eclipse more example .

You can also upload uber-jar with apis classifiers instead of obfuscate-apis classifier to your Nexus or dependency management system. 

To upload this uber-jar to your nexus you can use something like,

mvn deploy:deploy-file -Durl=YOUR-REPO -Dfile=uber-jar-6.1.0-apis.jar -DartifactId=uber-jar -Dversion=6.1.0 -DgroupId=com.adobe.aem -Dclassifier=apis -Dpackaging=jar -DrepositoryId=YOUR-REPO-ID

Once you have uber-jar with apis classifier ready, You can use something like


I see that minimum these are needed when you use this,

As always let me know if you want me to add more details in this posting.

Wednesday, June 25, 2014

How to add binary property for Node in CQ

Motivation: Recently I was trying to add some binary property to a node through file system (As part of code deployment). Had hard time figuring out how we can do that.

Use Case: You need to add some Binary property to a node through file system. (One use case is adding SAML cert as property idp_cert under /etc/key/saml)


Option 1:

You can use following curl command to do that (Look at \< )

curl -u UID:PWD -F property_name=\<File location -F property_name@TypeHint=Binary http://HOST:PORT/PATH_WHERE_YOU_WANT_TO_ADD_THIS_PROP

If you are trying to upload a file instead of property, You can use something like, (This is even true for if you want to install a bundle using repo path in that case repo path would be something like /apps/your_app/install if this is Adobe bundle then something like /libs/adobe_modules/install and if this is system bundle (Like Authentication bundles) then it would be something like /libs/system/install

curl -u UID:PWD -T File_Location HOST:PORT/PATH

Option 2: 

You can use .content.xml to create property like this

<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:sling="" xmlns:jcr="" xmlns:rep="internal"

And then create a file called  your_property-name.binary at same level and dump your binary content there. When you use maven-content-plugin to build package now, vlt knows how to transform this to a binary property.

----- /somepath
-------- .content.xml
-------- your_property_name.binary

Note that there are various way to create actual binary content in CQ using curl and some of them you can check here 

Wednesday, May 14, 2014

How to find component load time on a page in CQ

Use Case: Your page is loading slow and you want to know which component in a page is slow in render

Solution: You can use CQ timing info to get that data. timing load uses resource dumper to track load time of each jsp within page.

To enable debugging

version <= CQ5.6.1

1) Go to /libs/foundation/components/timing/timing.jsp through CRXDE light and comment following like of code

//uncomment the following to get more timing details in the page
out.println("\nRaw RequestProgressTracker data:");
StringBuilder mb = new StringBuilder();
Iterator<String> it = t.getMessages();
while(it.hasNext()) {
out.println("\nChartData dump:");
for(ChartBar d : chartData) {
    out.print(' ');
    out.print(" (");

2) Make sure that timing.jsp is included in your global template using

<cq:include path="timing" resourceType="foundation/components/timing"/>

3) Load your page and do view source

4) Scroll down to bottom of page where you will see google chart URL

5) Copy that URL and paste it in browser to get timing info

CQ > 6

CQ 6 has this feature OOTB

Notes: Please also read to see how you can analyze request log to find page load time on server side. This is also very useful to find any thread contention issues wither using thread dump or session dump from felix console.

On client side there are different products you can use (I don't have preference) (AppDynamic, New Relic, Google Page Speed, Google Analytics etc) to find client side load time.

Monday, April 21, 2014

How to use sling thread pool in CQ/AEM

Use Case: You are designing an application where you might need to delegate Job to a separate thread and you want to control this using thread pool so that your application it self does not run out of threads.

Prerequisite: Knowledge of Java Threads and basic Knowledge of thread pool. Note that there are Java Level thread pool as well that you can use. Check for that.


Implementation Example:

Step 1: Create your Thread class

public YourThreadClass implements Runnable {

 public void run() {
    //Your custom logic

Step 2: Main class to use sling Thread pool

Use CQ dependency finder to find what dependency you need for above code in your pom.xml

Wednesday, January 22, 2014

How to include CQ package from other projects as dependency in your project

Use Case: We often come across situation where we want to include package or jar files from other CQ project across organization to your project.

Set Up: This assumes that you already have your project set up using Maven, Nexus. You need Nexus or any other repository management system to store artifact of jar or CQ package zip artifact you need to use in your project.

Assumption: You are using "content-package-maven-plugin" to create CQ package. More information about this artifact can be found here


For jar file it is simply adding it to your embed statement and for zip file you could use subPackages configuration. Once you will run maven install with this configuration, Other project package will also get installed with your package. Same works for uninstall of package as well. Geometrixx all package uses similar approach.

your final pom will look like this,

NOTE:  When you are using other CQ projects as dependency in to your project, make sure that other project is not overriding configurations in your project. You might have to adjust other project package filter accordingly.                  

Monday, December 9, 2013

How to fix Could not open Bad file descriptor Issue in CQ

Issue: Server does not start and you see following error in logs

*WARN* [FelixStartLevel] Could not open Bad file descriptor
at Method)
at org.apache.jackrabbit.core.RepositoryImpl.<init>(
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
at java.lang.reflect.Method.invoke(
at org.apache.felix.scr.impl.helper.BaseMethod.invokeMethod(
at org.apache.felix.scr.impl.helper.BaseMethod.access$500(
at org.apache.felix.scr.impl.helper.BaseMethod$Resolved.invoke(
at org.apache.felix.scr.impl.helper.BaseMethod.invoke(

at org.apache.felix.scr.impl.helper.ActivateMethod.invoke(


1) Find a line above this error, it would be something like

09.12.2013 13:42:32.030 *INFO* [FelixStartLevel] scanning index <some path>/crx-quickstart/repository/<either version or workspace>/data_<some number>.tar 

2) Based on path go to that location.
3) STOP YOUR INSTANCE. remove all index files using rm -rf <Path from above>/index*tar
4) change permission of data tar file using chmod 644 <path from above>/data*tar
5) Start instance
6) Some cases data tar files can not be recovered. Please check my other post to fix non recovery data tar files.

Caution: If there are a lot of data tar files, Index creation may take some time. Please create daycare ticket to find root cause of this issue.