Wednesday, December 10, 2014

How to Set Up Clustering In CQ/AEM 6 using MongoDB


With CQ / AEM 6 TarPM is not supported any more. AEM 6 ships with Oak which for now support TarMK and MongoMK Microkernal OOTB. More information about what is New Can be found from . With this change Support from Clustering is moved to actual storage layer it self (Which make more sense, given supporting all issues for clustering in earlier version). TarMK does not have replication or sharding feature so it comes down to MongoDB which support replication and sharding and hence enable High Availability (HA through replication) and Scalability (Through Sharding, Though this is still a question ?? See note below) through clustering in CQ /AEM 6.

Here we will give step by step instruction of how to set up clustering using MongoDB in CQ

Pre requisite:

There are two cases for setting Up Replica Set:

Set up a new MongoDB Instance:

  • Set up additional MongoDB instance based on instruction above
  • Start any one of instance using ./mongod --port <Your Port> --dbpath <Your DB Path> --replSet <Replica Set Name could be any thing> &  
  • You can also use configuration file to do that. More instruction here
  • Once Mongo DB is started you can add additional replica using following instruction 
  • Once Replica set is up, Now set Up AEM
  • Then You can go to each Mongo Instance and check of data is coming using Mongo Log
Convert Existing Mongo Instance:

  • Stop you AEM instance
  • Use Following instruction to convert Mongo to replica
  • Once this is set Change AEM start script to add mongo replica instance as given in approach one 
  • start your AEM instance
  • AEM should be part of replica set now

Backup and Restore

Please check for MongoDB instruction of backup and restore. 

Automated script can be found here: just put this script under /etc/cron.daily and you are set for backup.

Some Common Questions

Should I set up my AEM author instance on MongoDB

Unless you have clustering requirement, I would not suggest to set up your author instance with MongoDB. Mainly because of administrative overhead.

Should I set up my AEM publish instance on MongoDB

Same as above, Unless you have a requirement which requires shared content generation I would suggest not to use MongoDB. With AEM communities, now you have an option to add Mongo Persistence for community feature at any time. Here is more detail and

Should I store Blobs in MongoDB as well in AEM

It is not recommended to store Blob data with MongoDB. There are other options like, Local Storage, NAS, AWS you can use in that case. More detail Configuration and

How can I secure my MongoDB deployment with AEM


1) Mongo Replication Only Provide High Availability (HA) it does not provide scalability. For scalability you need to use Sharding feature provided by Mongo. However I am not sure what would be best key to create shard on for Mongo. You can create Shard based on _id attribute. More information about sharding can be obtained here  . If you are using Sharding I would suggest to use sharding with replication (Shard and then replicate shard instance) to provide both HA and scalability.

2)  There are many feature available in Mongo Replication where you can make certain replica instance read only (Data Center replica), you can use this to avoid high latency across Data Center here is all configuration you can do on Mongo

3) MongoDB recently released MMS to monitor and deploy Mongo Cluster easily. This will be useful if you are worried about administrative cost for Mongo 

4) If you don't want to store large documents in Mongo feel free to use custom Data Store using instruction here

5) Mongo Recently launched another feature of pluggable datastore. You can use this for faster read and write based on your requirement (For example Primary with high Write Enabled Storage Like SSD or something and read with cheap storage). More info here (Official Doc yet to come)

6) Official AEM Documentation:

Finally .... Some more Mongo Command ...

Special Thanks To Nelson Mei for Setting up POC for Mongo with AEM

Thursday, November 13, 2014

How to use impression Service In CQ/AEM

Use Case:
  1. You often have case where you want to use Impression service provided by CQ to do custom operation for example finding top 10 most viewed page or sorting all page based on there popularity.
  2. It might possible that your impression data (Page Views) is in external system and then you want to import those data as impression in CQ to have more application context.
  3. You want to aggregate all data across all publish instances.

Approach 1:

Creating your Own Impression service

You can create your own impression service by extending here is example
Supporting class

You need to embed following dependency for this
Here is example of how you can use this service

Now you can import data from external system (GA, Site Catalyst, Kafka) and then populate it using this service to your instance.

Once you are ready with all data you can use following service to use data,

Actual Implementation

Approach 2:

You don't want to write your own service as mentioned in Approach 1 and use OOTB service available to you. Only problem with this is, You have multiple publish instance and some how you want to combine all data in to one so that you get accurate picture. It kind of tricky to get all data from all publish instance (through reverse replication) and then combine them on author and then push them over again. However you can use one instance to collect all stat data (king of single source of truth and then replicate it back to all instance every day)

  • Make sure that you enable page view tracking by adding following line

           <cq:include script="/libs/foundation/components/page/stats.jsp" />
  • Then configure all publish instance to point to one DNS using following config (You can always override this under /apps)
  • make sure that pageviewstatistics.trackingurl is pointing to single domain (You need to create a domain, something like that will be stand alone CQ instance to take all impression request)
  • Now you have consolidated page impression on one machine
  • You can easily write a schedular which will run every night and reverse replicate all data to author instance.
  • Once it is on author instance you can use replicator service to replicate to all other publish instance
  • Then you can use code mention in approach 1 to get popular resources.

Note: You can always use GA or something to track data. This is more useful if you want to do something internally and not want top share data with GA.

As usual feel free to ask any question you have.

Friday, November 7, 2014

How to make Instances SSL context aware in CQ/AEM

Use Case:

Most of the times we terminate SSL on load balancer or at dispatcher and communication to publish happen over http. In this case publish server is often not SSL aware and any request specific operations (For example relative path redirect or Link rewriting) happens over http. For example if you do something like request.sendRedirect("/somepath") from server it will get redirected to http://server-name/somepath or when you will do request.isSecure() it will return false. Operations like externalizer.externalLink(resolver, "mydomain", "/my/page") + ".html"; will also return http version of link.


Option 1:

Let all links be http and then do force redirect on dispatcher or Load Balancer. For dispatcher rule can be as simple as this,

RewriteCond %{HTTPS} off
RewriteRule ^(.*)$ https://%{HTTP_HOST}$1 [L,R=301]


  • Not SEO friendly (Unless it is relative link).  
  • Can not embed as portlet or Iframe (Cross Site include error) over https.

Option 2: 

If filter is configured, it looks for header configured in configuration and if header value matches configured value context on publish is considered secure and request.isSecure() will return true.

In above case if X-Forwarded-SSL is present with value on then servlet context is secure.

Note: This might not work in version latest CQ version. You can build your own version from trunk or from

One working version of file can be downloaded from here

As usual let me know if you have any question. Special thanks to Shenghao Huang from LinkedIn for finding this.

Wednesday, November 5, 2014

How to use Sling Models in CQ5.6

Use Case: Use Sling Model in CQ5.6

Background: Sling model brings powerful way of mapping your resources to Java Objects (Beans) . It is supported OOTB in CQ6 however we can use it in CQ5.6 as well.


First add sling models as dependency in your code (In Parent pom using dependency manager is preferred)

Then update your reactor pom (Which actually creates your CQ package using content-package-maven-plugin) you need to add following as dependency and then embed them in to your project (If install path for system is not present you can use any other path you want)

Thats it .. Models are available as service in your OSGI environment.

 To use model go ahead and create your own bundle and them add these as dependency. You should be able to write, deploy and run code against model. Note that you might have to make these model available to osgi using Sling-Model-Packages

Once model is deployed correctly you should be able to see them under sling-model tab in status in felix console or by going to HOST:PORT/system/console/status-slingmodels

Some example code using models

Some common How to

How can I inject services in Sling Model

by using annotation @OSGIService Or Using Injector @Inject @Source("osgi-service") 


MyService myService

@Inject @Source("osgi-service")
MyService myService

How Can I Inject Sling Object in Sling Model

by using annotation @SlingObject Or Using Injector @Inject @Source("sling-object") 


ResourceResolver resourceResolver

@Inject @Source("sling-object")
ResourceResolver resourceResolver

How Can I get access to property if adapting through Sling Request

If you are adapting your model to sling request you might not have access to property by just doing @Inject. You have to use @Via in that case. Note that you can use all other annotation here like @Named and all. Here is example

public interface MyModel { 
    @Inject @Via("resource")
    String propertyName;

How Can I use Sling Tag Library with Sling Model

You can use <sling:adaptTo ... /> or ${sling:adaptTo ...} as mentioned in and .In 5.6 you might have to change global.jsp to use
<%@taglib prefix="sling" uri="" %>
instead of
<%@taglib prefix="sling" uri="" %>

How Can I provide default value to property in Sling Model

By using @Default Annotation . Here is example

A default value can be provided (for Strings & primitives) and Array:
public class MyModel {

    @Inject @Default(values="defaultValue")
    private String name; 
    @Inject @Default(intValues={1,2,3,4})
    private int[] integers; 

How Can I inject child resource as Model in Sling Model 

This can be done by Injecting a model in other model class. Here is example

public interface MyModel {

    ImageModel getImage();

public interface ImageModel {

    String getPath();

When a resource is adapted to MyModel, a child resource named image is automatically adapted to an instance of ImageModel

Note: Sling model version number can change as it evolves in future. Make sure that you update model dependencies version accordingly. You can also use Sling testing framework to test sling model which is pretty cool as well. There is also an example of deployable Model package which you can use.

As usual feel free to give your comment and feedback and let me know if you want me to add something else here.

Thursday, October 16, 2014

How to Connect to External Data Source (Teradata / MySQL / Oracle) in CQ / AEM

Use Case: Connect to external datasource using CQ


1) My SQL

Step 1: Create OSGI version of mysql Jar file

  • Click next and then select add external. Select jar file you downloaded above and select next
  •  Give Project Name -> Select Location -> Make sure that Analyze Library Content is checked -> In Target Platform select an OSGI framework -> from drop down select standard -> Check unzip jar file and update reference -> click finish

  • An Plugin Development prospective will open 
  • Click on export tab and make sure that all dependencies are exported (If they are not there, Click on add then select all)

  • Click on Dependencies and make sure that Imported packages are there. If it is not there click on MENIFEST.MF tab and then add following import statement 
  • Once all import and export statement is added your MENIFEST.MF will have following data

  • Then right click on your project and then select export 
  • Select Deployable Deployable plug-in and fragments

  • Click Next and select your plug-in 
  • From Directory section select location where you want to upload.
  • Click finish. This will store OSGI jar in to location you selected.
  • Once you have MySQL Osgi bundle. Install that bundle either using felix console or adding it through deployment process

  • If you are getting error saying javax.Naming not getting resolved then you have to add following line in under crx-quickstart folder
org.osgi.framework.bootdelegationjavax.naming.*, ${}
  • Once bundle is up is running, You need to go to felix osgi config. Search For JDBC connection pool and then click '+'
  • Fill all the information about your datasource

  • Go to your application logic and test your code

If you are having trouble creating osgi version of mysql jar file, You can download it from here

2) Teradata

  • Upload final jar file in CQ, make sure that it is active

  • Open OSGI config to create config for teradata connection pool

  • You can then test it using same code (Just use teradata data source)

You can also download OSGI version of teradata jar file from here

Note: If You want to call instance of your Connection from Java class (Not from service) You can do something like this,

Thursday, August 28, 2014

How to Use Sessions and Resource Resolver through Service Authentication In AEM6

Use Case: As per and using admin session and admin resource resolver through ResourceresolverFactory is now deprecated. Using Service based Authentication for Resourceresolver and Respository session solves problem like (Directly From Sling Doc),
  • Prevent over-use and abuse of administrative ResourceResolvers and/or JCR Sessions
  • Allow services access to ResourceResolvers and/or JCR Sessions without requiring to hard-code or configure passwords
  • Allow services to use service users which have been specially configured for service level access (as is usually done on unixish systems)
  • Allow administrators to configure the assignment of service users to services


NOTE: Use Service Accounts for alice and bob users [jcr:primaryType=rep:SystemUser] instead of regular accounts.

Lets see we have two user "alice" and "bob", with following property,
  • "alice" only have READ access to document under /content/somepath path
  • "bob" has both read and write access to document under /content/somepath path
Now we have two service "ReadService" and "WriteService", with following property
  • ReadService should only be allowed to read anything under /content/somepath path
  • WriteService should be allowed for both read and write under /content/somepath path

Assume your package name is

Step 1: Create ReadService and WriteService using resourceResolver Or adminSession using new Authentication Service based API
Step 2: Create ReadService same way
Step 3: Update config by creating osgi:config node under /apps/<your-apps>/config.<Place where you want to run this>/ you can directly go to osgi config through Felix console and change this as well look for  “Apache Sling Service User Mapper Service” for that.

Syntax for service mapping to user is ‘serviceName [ ":" subServiceName ] “=” username’.
and Entry of OSGI config will look like this,

After installing the bundle and configuration and code, You would see something like this in log

*INFO* alice
*INFO* <node type of somepath>

*INFO* bob
*INFO* <node type of somepath>
*INFO* Successfully saved

If you need to use admin session for the configuration you can do something like in osgi config above. Good practice is to have these session based on groups depending upon which group have access to what service.

You might need following dependencies in your POM for api to be available

Please check with some of the changes in AEM6.1 of how to use this.

As always feel free to ask any question you might have.

Wednesday, July 30, 2014

How to Remove White Space From Generated HTML In CQ (Or In general)

Use Case: There are a lot of white spaces in generated Output of CQ increasing size of page and decreasing load time.


Option 1: You can use trimDirectiveWhitespaces directive in jsp response. something like

<%@page  trimDirectiveWhitespaces="true"%>

Problem: Using this directive can cause white space to be removed from taglibs for same property. To avoid this issue make sure that you manually add space there. For example if you have tag lib like <Something class="${test1} ${test2}" class2="test"> replace it with <Something class="${test1} ${space} ${test2}" class2="test"> where ${space} is actual space " "
This approach might not work with Slighly framework.

Option 2: Use %><% tags to start and end scriplets tag and in between html tag

Problem: Code very hard to read and not pretty.

Option 3: Create your own tag library and using html parser remove white spaces during run time. for that check

And code to remove White space would be

Problem: Maintenance of your own tag library. Can Miss Some condition. Have to wrap up your code with tag lib.

Option 4 (Preferred): Use Google Page Speed Module at apache.

Link to Module:
Link to All available Filters for Module:
Instruction of how to install and build:

Steps to integrate it in Dispatcher:

1) Create Apache module using step above. This will give you and

2) Move this file to <Apache location>/modules

3) Change permission (to daemon:daemon (Or Your Apache User)) and permission level (766) using chown and chmod command

4) Open conf/httpd.conf and add following line (If some include is already there ignore that)
Include <Apache Location>/conf/pagespeed.conf

5) Create a folder called <Doc Root>/mod_pagespeed/

6) Add pagespeed.conf under <Apache Location>/conf
* Make sure that All paths mentioned in conf file exist.

7) Restart Apache

1) Only Apache module is available. If you are using IIS or any other web server then there is no module yet.
2) You might have to do build distribution for your own OS if above module build does not work (One attached here is build for Red Hat Linux).
3) When you upgrade your Apache make sure to upgrade your module as well. If there is no distribution for newer version of apache, then also you are out of luck.

Note: Test above methods before using them in production. Feel free to ask any question you have.

Monday, July 28, 2014

How to include all CQ dependencies in CQ6

Use case: Prior to CQ6 you have to add dependencies for each class you are using in your pom.xml, Way to find dependencies was (Maven org or using adobe central through dependency finder HOST:PORT/system/console/depfinder). With CQ6 now all dependencies are provided through one artifactID.

Prerequisite: Maven, CQ Project Set Up 

Solution: Include following line in your dependency management for your pom.xml (Depending upon project this could be at any location usually it is your Project parent pom)

<!-- All Your Third party dependency -->
<!-- AEM 6.0 -->

      <!-- All AEM Dependencies for JSP compilation -->


      <!-- All AEM 6.1 Dependency added to end of file -->




Note that this version could change depending upon new releases of CQ, You can track them from

Some Trick: Note that above will include all AEM-API dependencies, other way to check what minimum dependency is needed is to create a multi module project using AEM plugin for eclipse more example .

You can also upload uber-jar with apis classifiers instead of obfuscate-apis classifier to your Nexus or dependency management system. 

To upload this uber-jar to your nexus you can use something like,

mvn deploy:deploy-file -Durl=YOUR-REPO -Dfile=uber-jar-6.1.0-apis.jar -DartifactId=uber-jar -Dversion=6.1.0 -DgroupId=com.adobe.aem -Dclassifier=apis -Dpackaging=jar -DrepositoryId=YOUR-REPO-ID

Once you have uber-jar with apis classifier ready, You can use something like


I see that minimum these are needed when you use this,

As always let me know if you want me to add more details in this posting.

Wednesday, June 25, 2014

How to add binary property for Node in CQ

Motivation: Recently I was trying to add some binary property to a node through file system (As part of code deployment). Had hard time figuring out how we can do that.

Use Case: You need to add some Binary property to a node through file system. (One use case is adding SAML cert as property idp_cert under /etc/key/saml)


Option 1:

You can use following curl command to do that (Look at \< )

curl -u UID:PWD -F property_name=\<File location -F property_name@TypeHint=Binary http://HOST:PORT/PATH_WHERE_YOU_WANT_TO_ADD_THIS_PROP

If you are trying to upload a file instead of property, You can use something like, (This is even true for if you want to install a bundle using repo path in that case repo path would be something like /apps/your_app/install if this is Adobe bundle then something like /libs/adobe_modules/install and if this is system bundle (Like Authentication bundles) then it would be something like /libs/system/install

curl -u UID:PWD -T File_Location HOST:PORT/PATH

Option 2: 

You can use .content.xml to create property like this

<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:sling="" xmlns:jcr="" xmlns:rep="internal"

And then create a file called  your_property-name.binary at same level and dump your binary content there. When you use maven-content-plugin to build package now, vlt knows how to transform this to a binary property.

----- /somepath
-------- .content.xml
-------- your_property_name.binary

Note that there are various way to create actual binary content in CQ using curl and some of them you can check here 

Wednesday, May 14, 2014

How to find component load time on a page in CQ

Use Case: Your page is loading slow and you want to know which component in a page is slow in render

Solution: You can use CQ timing info to get that data. timing load uses resource dumper to track load time of each jsp within page.

To enable debugging

version <= CQ5.6.1

1) Go to /libs/foundation/components/timing/timing.jsp through CRXDE light and comment following like of code

//uncomment the following to get more timing details in the page
out.println("\nRaw RequestProgressTracker data:");
StringBuilder mb = new StringBuilder();
Iterator<String> it = t.getMessages();
while(it.hasNext()) {
out.println("\nChartData dump:");
for(ChartBar d : chartData) {
    out.print(' ');
    out.print(" (");

2) Make sure that timing.jsp is included in your global template using

<cq:include path="timing" resourceType="foundation/components/timing"/>

3) Load your page and do view source

4) Scroll down to bottom of page where you will see google chart URL

5) Copy that URL and paste it in browser to get timing info

CQ > 6

CQ 6 has this feature OOTB

Notes: Please also read to see how you can analyze request log to find page load time on server side. This is also very useful to find any thread contention issues wither using thread dump or session dump from felix console.

On client side there are different products you can use (I don't have preference) (AppDynamic, New Relic, Google Page Speed, Google Analytics etc) to find client side load time.

Monday, April 21, 2014

How to use sling thread pool in CQ/AEM

Use Case: You are designing an application where you might need to delegate Job to a separate thread and you want to control this using thread pool so that your application it self does not run out of threads.

Prerequisite: Knowledge of Java Threads and basic Knowledge of thread pool. Note that there are Java Level thread pool as well that you can use. Check for that.


Implementation Example:

Step 1: Create your Thread class

public YourThreadClass implements Runnable {

 public void run() {
    //Your custom logic

Step 2: Main class to use sling Thread pool

Use CQ dependency finder to find what dependency you need for above code in your pom.xml

Wednesday, January 22, 2014

How to include CQ package from other projects as dependency in your project

Use Case: We often come across situation where we want to include package or jar files from other CQ project across organization to your project.

Set Up: This assumes that you already have your project set up using Maven, Nexus. You need Nexus or any other repository management system to store artifact of jar or CQ package zip artifact you need to use in your project.

Assumption: You are using "content-package-maven-plugin" to create CQ package. More information about this artifact can be found here


For jar file it is simply adding it to your embed statement and for zip file you could use subPackages configuration. Once you will run maven install with this configuration, Other project package will also get installed with your package. Same works for uninstall of package as well. Geometrixx all package uses similar approach.

your final pom will look like this,

NOTE:  When you are using other CQ projects as dependency in to your project, make sure that other project is not overriding configurations in your project. You might have to adjust other project package filter accordingly.