Thursday, November 13, 2014

How to use impression Service In CQ/AEM

Use Case:
  1. You often have case where you want to use Impression service provided by CQ to do custom operation for example finding top 10 most viewed page or sorting all page based on there popularity.
  2. It might possible that your impression data (Page Views) is in external system and then you want to import those data as impression in CQ to have more application context.
  3. You want to aggregate all data across all publish instances.
Solutions:

Approach 1:

Creating your Own Impression service

You can create your own impression service by extending com.day.crx.statistics.Entry here is example
Supporting class



You need to embed following dependency for this
Here is example of how you can use this service

Now you can import data from external system (GA, Site Catalyst, Kafka) and then populate it using this service to your instance.

Once you are ready with all data you can use following service to use data,



Actual Implementation


Approach 2:

You don't want to write your own service as mentioned in Approach 1 and use OOTB service available to you. Only problem with this is, You have multiple publish instance and some how you want to combine all data in to one so that you get accurate picture. It kind of tricky to get all data from all publish instance (through reverse replication) and then combine them on author and then push them over again. However you can use one instance to collect all stat data (king of single source of truth and then replicate it back to all instance every day)


  • Make sure that you enable page view tracking by adding following line

           <cq:include script="/libs/foundation/components/page/stats.jsp" />
  • Then configure all publish instance to point to one DNS using following config (You can always override this under /apps)
           /apps/wcm/core/config.publish/com.day.cq.wcm.core.stats.PageViewStatistics
          /apps/wcm/core/config.publish/com.day.cq.wcm.core.stats.PageViewStatisticsImpl
         
  • make sure that pageviewstatistics.trackingurl is pointing to single domain (You need to create a domain, something like impression.mydomain.com that will be stand alone CQ instance to take all impression request)
  • Now you have consolidated page impression on one machine
  • You can easily write a schedular which will run every night and reverse replicate all data to author instance.
  • Once it is on author instance you can use replicator service to replicate to all other publish instance
  • Then you can use code mention in approach 1 to get popular resources.

Note: You can always use GA or something to track data. This is more useful if you want to do something internally and not want top share data with GA.

As usual feel free to ask any question you have.





Friday, November 7, 2014

How to make Instances SSL context aware in CQ/AEM

Use Case:

Most of the times we terminate SSL on load balancer or at dispatcher and communication to publish happen over http. In this case publish server is often not SSL aware and any request specific operations (For example relative path redirect or Link rewriting) happens over http. For example if you do something like request.sendRedirect("/somepath") from server it will get redirected to http://server-name/somepath or when you will do request.isSecure() it will return false. Operations like externalizer.externalLink(resolver, "mydomain", "/my/page") + ".html"; will also return http version of link.

Solution:

Option 1:

Let all links be http and then do force redirect on dispatcher or Load Balancer. For dispatcher rule can be as simple as this,

RewriteCond %{HTTPS} off
RewriteRule ^(.*)$ https://%{HTTP_HOST}$1 [L,R=301]

Issue:

  • Not SEO friendly (Unless it is relative link).  
  • Can not embed as portlet or Iframe (Cross Site include error) over https.

Option 2: 


If filter is configured, it looks for header configured in configuration and if header value matches configured value context on publish is considered secure and request.isSecure() will return true.




In above case if X-Forwarded-SSL is present with value on then servlet context is secure.

Note: This might not work in version latest CQ version. You can build your own version from trunk https://github.com/apache/felix/tree/trunk/http/sslfilter or from http://svn.apache.org/repos/asf/felix/trunk/http/sslfilter/

One working version of file can be downloaded from here

As usual let me know if you have any question. Special thanks to Shenghao Huang from LinkedIn for finding this.

Wednesday, November 5, 2014

How to use Sling Models in CQ5.6

Use Case: Use Sling Model in CQ5.6

Background: Sling model http://sling.apache.org/documentation/bundles/models.html brings powerful way of mapping your resources to Java Objects (Beans) . It is supported OOTB in CQ6 however we can use it in CQ5.6 as well.

Solution:

First add sling models as dependency in your code (In Parent pom using dependency manager is preferred)

Then update your reactor pom (Which actually creates your CQ package using content-package-maven-plugin) you need to add following as dependency and then embed them in to your project (If install path for system is not present you can use any other path you want)

Thats it .. Models are available as service in your OSGI environment.



 To use model go ahead and create your own bundle and them add these as dependency. You should be able to write, deploy and run code against model. Note that you might have to make these model available to osgi using Sling-Model-Packages



Once model is deployed correctly you should be able to see them under sling-model tab in status in felix console or by going to HOST:PORT/system/console/status-slingmodels



Some example code using models

Note: Sling model version number can change as it evolves in future. Make sure that you update model dependencies version accordingly. You can also use Sling testing framework to test sling model which is pretty cool as well. There is also an example of deployable Model package https://github.com/Adobe-Consulting-Services/com.adobe.acs.bundles.sling-models which you can use.

As usual feel free to give your comment and feedback and let me know if you want me to add something else here.


Thursday, October 16, 2014

How to Connect to External Data Source (Teradata / MySQL / Oracle) in CQ / AEM

Use Case: Connect to external datasource using CQ

Solution:

1) My SQL

Step 1: Create OSGI version of mysql Jar file


  • Click next and then select add external. Select jar file you downloaded above and select next
  •  Give Project Name -> Select Location -> Make sure that Analyze Library Content is checked -> In Target Platform select an OSGI framework -> from drop down select standard -> Check unzip jar file and update reference -> click finish

  • An Plugin Development prospective will open 
  • Click on export tab and make sure that all dependencies are exported (If they are not there, Click on add then select all)

  • Click on Dependencies and make sure that Imported packages are there. If it is not there click on MENIFEST.MF tab and then add following import statement 
  • Once all import and export statement is added your MENIFEST.MF will have following data

  • Then right click on your project and then select export 
  • Select Deployable Deployable plug-in and fragments

  • Click Next and select your plug-in 
  • From Directory section select location where you want to upload.
  • Click finish. This will store OSGI jar in to location you selected.
  • Once you have MySQL Osgi bundle. Install that bundle either using felix console or adding it through deployment process

  • If you are getting error saying javax.Naming not getting resolved then you have to add following line in sling.properties under crx-quickstart folder
org.osgi.framework.bootdelegationjavax.naming.*, ${org.apache.sling.launcher.bootdelegation}
  • Once bundle is up is running, You need to go to felix osgi config. Search For JDBC connection pool and then click '+'
  • Fill all the information about your datasource

  • Go to your application logic and test your code


If you are having trouble creating osgi version of mysql jar file, You can download it from here

2) Teradata

  • Upload final jar file in CQ, make sure that it is active

  • Open OSGI config to create config for teradata connection pool

  • You can then test it using same code (Just use teradata data source)


You can also download OSGI version of teradata jar file from here

Note: If You want to call instance of your Connection from Java class (Not from service) You can do something like this,


Thursday, August 28, 2014

How to Use Sessions and Resource Resolver through Service Authentication In AEM6

Use Case: As per http://sling.apache.org/documentation/the-sling-engine/service-authentication.html and http://docs.adobe.com/content/docs/en/aem/6-0/develop/ref/diff-previous/changes/changes-summary.html using admin session and admin resource resolver through ResourceresolverFactory is now deprecated. Using Service based Authentication for Resourceresolver and Respository session solves problem like (Directly From Sling Doc),
  • Prevent over-use and abuse of administrative ResourceResolvers and/or JCR Sessions
  • Allow services access to ResourceResolvers and/or JCR Sessions without requiring to hard-code or configure passwords
  • Allow services to use service users which have been specially configured for service level access (as is usually done on unixish systems)
  • Allow administrators to configure the assignment of service users to services

Solution:

Lets see we have two user "alice" and "bob", with following property,
  • "alice" only have READ access to document under /content/somepath path
  • "bob" has both read and write access to document under /content/somepath path
Now we have two service "ReadService" and "WriteService", with following property
  • ReadService should only be allowed to read anything under /content/somepath path
  • WriteService should be allowed for both read and write under /content/somepath path

Assume your package name is blog.wemblog.com

Step 1: Create ReadService and WriteService using resourceResolver Or adminSession using new Authentication Service based API
Step 2: Create ReadService same way
Step 3: Update org.apache.sling.serviceusermapping.impl.ServiceUserMapperImpl config by creating osgi:config node under /apps/<your-apps>/config.<Place where you want to run this>/org.apache.sling.serviceusermapping.impl.ServiceUserMapperImpl.xml you can directly go to osgi config through Felix console and change this as well look for  “Apache Sling Service User Mapper Service” for that.

Syntax for service mapping to user is ‘serviceName [ ":" subServiceName ] “=” username’.
and Entry of OSGI config will look like this,
After installing the bundle and configuration and code, You would see something like this in log

*INFO*  blog.wemblog.com.ReadServiceImpl alice
*INFO*  blog.wemblog.com.ReadServiceImpl <node type of somepath>

*INFO*  blog.wemblog.com.WriteServiceImpl bob
*INFO*  blog.wemblog.com.WriteServiceImpl <node type of somepath>
*INFO*  blog.wemblog.com.WriteServiceImpl Successfully saved


If you need to use admin session for the configuration you can do something like blog.wemblog.com:WriteService=admin in osgi config above. Good practice is to have these session based on groups depending upon which group have access to what service.

You might need following dependencies in your POM for api to be available

As always feel free to ask any question you might have.

Wednesday, July 30, 2014

How to Remove White Space From Generated HTML In CQ (Or In general)

Use Case: There are a lot of white spaces in generated Output of CQ increasing size of page and decreasing load time.

Solutions:

Option 1: You can use trimDirectiveWhitespaces directive in jsp response. something like

<%@page  trimDirectiveWhitespaces="true"%>

Problem: Using this directive can cause white space to be removed from taglibs for same property. To avoid this issue make sure that you manually add space there. For example if you have tag lib like <Something class="${test1} ${test2}" class2="test"> replace it with <Something class="${test1} ${space} ${test2}" class2="test"> where ${space} is actual space " "
This approach might not work with Slighly framework.

Option 2: Use %><% tags to start and end scriplets tag and in between html tag

Problem: Code very hard to read and not pretty.

Option 3: Create your own tag library and using html parser remove white spaces during run time. for that check http://www.cqblueprints.com/tipsandtricks/jsp-custom-tag-lib.html

And code to remove White space would be



Problem: Maintenance of your own tag library. Can Miss Some condition. Have to wrap up your code with tag lib.

Option 4 (Preferred): Use Google Page Speed Module at apache.

Link to Module: https://developers.google.com/speed/pagespeed/module
Link to All available Filters for Module: https://developers.google.com/speed/pagespeed/module/config_filters
Instruction of how to install and build: https://developers.google.com/speed/pagespeed/module/download

Steps to integrate it in Dispatcher:

1) Create Apache module using step above. This will give you mod_pagespeed.so and mod_pagespeed_ap24.so

2) Move this file to <Apache location>/modules

3) Change permission (to daemon:daemon (Or Your Apache User)) and permission level (766) using chown and chmod command

4) Open conf/httpd.conf and add following line (If some include is already there ignore that)
Include <Apache Location>/conf/pagespeed.conf

5) Create a folder called <Doc Root>/mod_pagespeed/

6) Add pagespeed.conf under <Apache Location>/conf
* Make sure that All paths mentioned in conf file exist.

7) Restart Apache


Problem:
1) Only Apache module is available. If you are using IIS or any other web server then there is no module yet.
2) You might have to do build distribution for your own OS if above module build does not work (One attached here is build for Red Hat Linux).
3) When you upgrade your Apache make sure to upgrade your module as well. If there is no distribution for newer version of apache, then also you are out of luck.


Note: Test above methods before using them in production. Feel free to ask any question you have.

Monday, July 28, 2014

How to include all CQ dependencies in CQ6

Use case: Prior to CQ6 you have to add dependencies for each class you are using in your pom.xml, Way to find dependencies was (Maven org http://mvnrepository.com or using adobe central http://repo.adobe.com/nexus/content/groups/public through dependency finder HOST:PORT/system/console/depfinder). With CQ6 now all dependencies are provided through one artifactID.

Prerequisite: Maven, CQ Project Set Up 

Solution: Include following line in your dependency management for your pom.xml (Depending upon project this could be at any location usually it is your Project parent pom)


<dependencyManagement>

.... All your third party Non CQ dependencies

<dependency>
<groupId>com.adobe.aem</groupId>
<artifactId>aem-api</artifactId>
<version>6.0.0.1</version>
<scope>provided</scope>

</dependency>

</dependencyManagement>

Note that this version could change depending upon new releases of CQ, You can track them from http://repo.adobe.com/nexus/content/repositories/releases/com/adobe/aem/aem-api/

Some Trick: Note that above will include all AEM-API dependencies, other way to check what minimum dependency is needed is to create a multi module project using AEM plugin for eclipse more example http://docs.adobe.com/content/docs/en/dev-tools/aem-eclipse.html .

I see that minimum these are needed when you use this,

<dependencies>
<!-- OSGi Dependencies -->
<dependency>
<groupId>org.apache.felix</groupId>
<artifactId>org.apache.felix.scr</artifactId>
</dependency>
<dependency>
<groupId>org.apache.felix</groupId>
<artifactId>org.apache.felix.scr.annotations</artifactId>
</dependency>
        <dependency>
            <groupId>biz.aQute</groupId>
            <artifactId>bndlib</artifactId>
        </dependency>
<dependency>
<groupId>org.osgi</groupId>
<artifactId>org.osgi.core</artifactId>
</dependency>
<dependency>
<groupId>org.osgi</groupId>
<artifactId>org.osgi.compendium</artifactId>
</dependency>
<!-- Other Dependencies -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>
<dependency>
<groupId>javax.jcr</groupId>
<artifactId>jcr</artifactId>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
</dependency>
        <dependency>
            <groupId>com.adobe.aem</groupId>
            <artifactId>aem-api</artifactId>
        </dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</dependency>
        <dependency>
            <groupId>org.mockito</groupId>
            <artifactId>mockito-all</artifactId>
        </dependency>
        <dependency>
            <groupId>junit-addons</groupId>
            <artifactId>junit-addons</artifactId>
        </dependency>

</dependencies>

As always let me know if you want me to add more details in this posting.

Wednesday, June 25, 2014

How to add binary property for Node in CQ

Motivation: Recently I was trying to add some binary property to a node through file system (As part of code deployment). Had hard time figuring out how we can do that.

Use Case: You need to add some Binary property to a node through file system. (One use case is adding SAML cert as property idp_cert under /etc/key/saml)

Solution:

Option 1:

You can use following curl command to do that (Look at \< )


curl -u UID:PWD -F property_name=\<File location -F property_name@TypeHint=Binary http://HOST:PORT/PATH_WHERE_YOU_WANT_TO_ADD_THIS_PROP

If you are trying to upload a file instead of property, You can use something like, (This is even true for if you want to install a bundle using repo path in that case repo path would be something like /apps/your_app/install if this is Adobe bundle then something like /libs/adobe_modules/install and if this is system bundle (Like Authentication bundles) then it would be something like /libs/system/install

curl -u UID:PWD -T File_Location HOST:PORT/PATH

Option 2: 

You can use .content.xml to create property like this

<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0" xmlns:rep="internal"
    your_property-name="{Binary}"
    jcr:primaryType="sling:Folder"/>

And then create a file called  your_property-name.binary at same level and dump your binary content there. When you use maven-content-plugin to build package now, vlt knows how to transform this to a binary property.

----- /somepath
-------- .content.xml
-------- your_property_name.binary

Note that there are various way to create actual binary content in CQ using curl and some of them you can check here 

Wednesday, May 14, 2014

How to find component load time on a page in CQ

Use Case: Your page is loading slow and you want to know which component in a page is slow in render

Solution: You can use CQ timing info to get that data. timing load uses resource dumper https://sling.apache.org/apidocs/sling5/org/apache/sling/api/request/RequestProgressTracker.html to track load time of each jsp within page.

To enable debugging

version <= CQ5.6.1

1) Go to /libs/foundation/components/timing/timing.jsp through CRXDE light and comment following like of code

//uncomment the following to get more timing details in the page
out.println("\nRaw RequestProgressTracker data:");
StringBuilder mb = new StringBuilder();
Iterator<String> it = t.getMessages();
while(it.hasNext()) {
    mb.append(it.next());
}
out.print(mb.toString());
out.println("\nChartData dump:");
for(ChartBar d : chartData) {
    out.print(d.start);
    out.print(' ');
    out.print(d.fullname);
    out.print(" (");
    out.print(d.elapsed);
    out.println("ms)");
}

2) Make sure that timing.jsp is included in your global template using

<cq:include path="timing" resourceType="foundation/components/timing"/>

3) Load your page and do view source

4) Scroll down to bottom of page where you will see google chart URL



5) Copy that URL and paste it in browser to get timing info



CQ > 6

CQ 6 has this feature OOTB



Notes: Please also read http://dev.day.com/docs/en/cq/current/deploying/monitoring_and_maintaining.html to see how you can analyze request log to find page load time on server side. This is also very useful to find any thread contention issues wither using thread dump or session dump from felix console.

On client side there are different products you can use (I don't have preference) (AppDynamic, New Relic, Google Page Speed, Google Analytics etc) to find client side load time.

Monday, April 21, 2014

How to use sling thread pool in CQ/AEM

Use Case: You are designing an application where you might need to delegate Job to a separate thread and you want to control this using thread pool so that your application it self does not run out of threads.

Prerequisite: Knowledge of Java Threads and basic Knowledge of thread pool. Note that there are Java Level thread pool as well that you can use. Check http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/package-summary.html for that.

Reference: https://sling.apache.org/documentation/bundles/apache-sling-commons-thread-pool.html

Implementation Example:

Step 1: Create your Thread class

public YourThreadClass implements Runnable {


 public void run() {
    //Your custom logic
}
}

Step 2: Main class to use sling Thread pool


Use CQ dependency finder to find what dependency you need for above code in your pom.xml

Wednesday, January 22, 2014

How to include CQ package from other projects as dependency in your project

Use Case: We often come across situation where we want to include package or jar files from other CQ project across organization to your project.

Set Up: This assumes that you already have your project set up using Maven, Nexus. You need Nexus or any other repository management system to store artifact of jar or CQ package zip artifact you need to use in your project.

Assumption: You are using "content-package-maven-plugin" to create CQ package. More information about this artifact can be found here 
http://dev.day.com/docs/en/cq/current/core/how_to/how_to_use_the_vlttool/vlt-mavenplugin.html

Solution:

For jar file it is simply adding it to your embed statement and for zip file you could use subPackages configuration. Once you will run maven install with this configuration, Other project package will also get installed with your package. Same works for uninstall of package as well. Geometrixx all package uses similar approach.

your final pom will look like this,


  
NOTE:  When you are using other CQ projects as dependency in to your project, make sure that other project is not overriding configurations in your project. You might have to adjust other project package filter accordingly.                  

Monday, December 9, 2013

How to fix com.day.crx.persistence.tar.ClusterTarSet Could not open java.io.IOException: Bad file descriptor Issue in CQ

Issue: Server does not start and you see following error in logs


*WARN* [FelixStartLevel] com.day.crx.persistence.tar.ClusterTarSet Could not open java.io.IOException: Bad file descriptor
at java.io.RandomAccessFile.writeBytes(Native Method)
at java.io.RandomAccessFile.write(RandomAccessFile.java:486)
at com.day.crx.persistence.tar.file.TarFile.write(TarFile.java:742)
at com.day.crx.persistence.tar.file.TarFile.writeData(TarFile.java:635)
at com.day.crx.persistence.tar.file.TarFile.appendMetaData(TarFile.java:709)
at com.day.crx.persistence.tar.file.TarFile.append(TarFile.java:591)
at com.day.crx.persistence.tar.TarSet.switchDataFile(TarSet.java:445)
at com.day.crx.persistence.tar.TarSet.open(TarSet.java:227)
at com.day.crx.persistence.tar.ClusterTarSet.reopenCopy(ClusterTarSet.java:1455)
at com.day.crx.persistence.tar.ClusterTarSet.open(ClusterTarSet.java:860)
at com.day.crx.persistence.tar.TarPersistenceManager.openTarSet(TarPersistenceManager.java:980)
at com.day.crx.persistence.tar.TarPersistenceManager.init(TarPersistenceManager.java:500)
at com.day.crx.core.CRXRepositoryImpl.createVersionManager(CRXRepositoryImpl.java:869)
at org.apache.jackrabbit.core.RepositoryImpl.<init>(RepositoryImpl.java:311)
at com.day.crx.core.CRXRepositoryImpl.<init>(CRXRepositoryImpl.java:307)
at com.day.crx.core.CRXRepositoryImpl.create(CRXRepositoryImpl.java:262)
at com.day.crx.core.CRXRepositoryImpl.create(CRXRepositoryImpl.java:245)
at com.day.crx.sling.server.impl.jmx.ManagedRepository.activate(ManagedRepository.java:170)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.felix.scr.impl.helper.BaseMethod.invokeMethod(BaseMethod.java:236)
at org.apache.felix.scr.impl.helper.BaseMethod.access$500(BaseMethod.java:37)
at org.apache.felix.scr.impl.helper.BaseMethod$Resolved.invoke(BaseMethod.java:613)
at org.apache.felix.scr.impl.helper.BaseMethod.invoke(BaseMethod.java:496)

at org.apache.felix.scr.impl.helper.ActivateMethod.invoke(ActivateMethod.java:149)

Solution: 

1) Find a line above this error, it would be something like

09.12.2013 13:42:32.030 *INFO* [FelixStartLevel] com.day.crx.persistence.tar.TarSet scanning index <some path>/crx-quickstart/repository/<either version or workspace>/data_<some number>.tar 

2) Based on path go to that location.
3) STOP YOUR INSTANCE. remove all index files using rm -rf <Path from above>/index*tar
4) change permission of data tar file using chmod 644 <path from above>/data*tar
5) Start instance
6) Some cases data tar files can not be recovered. Please check my other post to fix non recovery data tar files.

Caution: If there are a lot of data tar files, Index creation may take some time. Please create daycare ticket to find root cause of this issue.

Wednesday, July 31, 2013

How to host Adobe Dynamic Tag Management System files in CQ dispatcher

Use Case: You often have situation where for marketing and analytics purpose you have to reply on dev team to push tracking code or tag management code or inclusion of any third party client side library. Satellite Search and Discovery provide great way to abstract client side tracking or tagging changes for marketing and analytics with development.
Documentation on Tag Management can be found here

Challenges: 

1) Update to CQ could happen out side dev cycle. For this make sure that satellite changes are completely tested before using it in production.
2) Since Satellite is similar to SAS service, changes in satellite could also cause some module not to work as expected. For that you can keep track of changes in satellite side and test that module.
3) Some time satellite load is slow if it is loaded from there hosted service. For this you can host satellite code on dispatcher and use some script to update it every time there is any change.

Include satellite script to CQ:

You can simply use,

<script type="text/javascript" src="SOME-PATH.js" ></script>
you can also use run mode specific configuration to include dev or prod specific script to your side.
Set<String> runModes = sling.getService(SlingSettingsService.class).getRunModes();
if(!runModes.contains("author")) { 
   if(runModes.contains("prod"){
             //Include prod satellite code
   }else{
            //Include dev satellite code
   }
}

Host satellite Script on Dispatcher:

For performance you can host satellite on dispatcher itself and then include it in your file. For this satellite provide a feature for deploy Hook. Deploy hook URL is called every time there is a change in any configuration or any rules are published. 



If you want to host satellite files on dispatcher then you can give this deploy hook URL as your production server URL path that you want to call every time there is any change. For example I want call a servlet or any script on change and update change in dispatcher and all other publish and author instance.

Here I am using python script to make this update,  Process is like this,
  • Changes made in satellite
  • Satellite call a dispatcher URL
  • Dispatcher URL invoke python script (You need rewrite rule to do that)
  • Python script checks if this is staging or production server
  • Based on that it get corresponding satellite files which is in zip fomat
  • Script unzip file, remove existing files from dispatcher if present and put it in dispatcher in certain location
  • Then it calls other dispatchers to update files as well
  • Then it issue upload request to upload changed file to author
  • After that it issue tree activation request to update these files on all publish. (This step is required in case some one clear dispatcher cache).
  • In order to avoid infinite loop within all dispatchers, one dispatcher call other dispatcher with a URL param indicating not to call other dispatcher.
  • for UID:PWD you can use non admin users that only have access to satellite files, make sure that they have activation rights as well.

You can use 

Note: I am using old version of python, You can reduce code with latest version.


#!/usr/bin/python
import urllib2
import shutil
import urlparse
import os
import sys
import zipfile
import cgitb; cgitb.enable()
import cgi
import socket
import urllib
#import pwd
#import grp
#Global Var
#Read URL from path
#Staging IP List contain list of IP for Stage
staging_IP_list = ["X.X.X.X","X.X.X.X"]
#Production IP list
production_IP_list = ["Y.Y.Y.Y","Y.Y.Y.Y","Y.Y.Y.Y","Y.Y.Y.Y"]
#This is required to avoid circular loop
form = cgi.FieldStorage()
checked = form.getvalue("checked")
addr = socket.gethostbyname(socket.gethostname())
#This folder path is required to avoid permission issue.
rootFolder = "../some/folder/in/dispatcher"
#Need to create rewrite mapping for this to work.
pingUrl = "/some/path/for/dispatcher?checked=true"
staging_satellite_url="THIS IS THE URL WHERE DEV SATELLITE FILE IS HOSTED"
production_satellite_url="THIS IS THE URL WHERE PROD SATELLITE FILE IS HOSTED"
url=staging_satellite_url
author_content_folder="FOLDER NAME WHERE YOU WANT THIS FILE TO GO"
author_content_subfolder="SUBFOLDER NAME GIVEN BY SATELLITE /"
satellite_js_file_name="SATELLITE FILE NAME GIVEN BY SATELLITE"
staging_author_server="YOUR-STAGING-AUTHOR-SERVER"
production_author_server="YOUR-PRODUCTION-AUHTOR-SERVER"
curl_ping_url=staging_author_server
script_file_list=[]
#Destination Path
print "Content-type: text/html; charset=iso-8859-1\n\n"
print '''<HTML>'''
print '''<TITLE>Satellite Ping Check</TITLE><body>'''
#print '''<br>url I got host as''',addr

#This method override url open to make just head request
class HeadRequest(urllib2.Request):
def get_method(self):
return "HEAD"

#Method to ping URL to another server
def pingURL(customURL):
try:
response = urllib2.urlopen(HeadRequest(customURL))
except:
print '''<br>We failed to reach a server.'''

#Method that will ping other server based on IP address
def pingOtherServer():
for eachIp in staging_IP_list:
if eachIp==addr:
for eachIp2 in staging_IP_list:
if eachIp2!=addr:
resp = pingURL("http://"+eachIp2+pingUrl)
break
for eachIp in production_IP_list:
if eachIp==addr:
for eachIp2 in production_IP_list:
if eachIp2!=addr:
resp = pingURL("http://"+eachIp2+pingUrl)
break


#This is required to keep those files to author
def pingauthorServer():
filepath = rootFolder+"/"+author_content_subfolder+satellite_js_file_name
#Curl command to upload satellite file
os.system('curl -u UID:PWD -F@TypeHint="nt:file" -Ftype="file" --upload-file '+filepath+' '+curl_ping_url+author_content_folder+author_content_subfolder)
filepath=rootFolder+"/"+"selector.js"
#Curl command to upload selectors.js file
os.system('curl -u UID:PWD -F@TypeHint="nt:file" -Ftype="file" --upload-file '+filepath+' '+curl_ping_url+author_content_folder)
for script_file in script_file_list:
filepath=rootFolder+"/"+author_content_subfolder+"scripts/"+script_file
os.system('curl -u UID:PWD -F@TypeHint="nt:file" -Ftype="file" --upload-file '+filepath+' '+curl_ping_url+author_content_folder+author_content_subfolder+"scripts/")
#Curl command to activate files to publish instance
os.system('curl -u UID:PWD -Fcmd=activate -Fignoredeactivated=true -Fonlymodified=false -Fpath='+author_content_folder+' '+curl_ping_url+'/etc/replication/treeactivation.html')

#Method to delete existing folder before extracting new one
def deleteFileOrFolder(directory):
    if os.path.exists(directory):
        try:
            if os.path.isdir(directory):
                print '''<br>removing folder<b>''',directory
                shutil.rmtree(directory)
                print '''<br>Creating''',directory
                os.makedirs(directory)
            else:
                print '''<br>removing file<b>''',directory
                os.remove(directory)
        except:
            print '''<br>Ecxeption''',str(sys.exc_info())
    else:
        print '''<br>not found''',directory
        print '''<br>Creating''',directory
        os.makedirs(directory)

#Method to set satellite url based on IP address. If this is production server then set URL as production
def seturl():
for eachIp in production_IP_list:
if eachIp==addr:
global url
url=production_satellite_url
global satellite_js_file_name
satellite_js_file_name="YOUR-SATELLITE-FILE-NAME.js"
global curl_ping_url
curl_ping_url=production_author_server
break


def extract():
zip_file = zipfile.ZipFile(fileName, 'r')
#print '''file name is ''',fileName
for files in zip_file.namelist():
print '''<br>files in zip''',files
myfile_path=rootFolder+"/"+files
#print '''<br> Yogesh ''',myfile_path
if myfile_path.endswith("/"):
#print '''<br>I am in if and myfile_path is ''',myfile_path
if not os.path.exists(myfile_path):
os.makedirs(myfile_path)
else:
if files.find("/scripts/") != -1:
script_file_list.append(files.split('/')[-1])
#print '''<b> found script file with name <br>''',rootFolder+"/"+author_content_subfolder+"scripts/"+files.split('/')[-1]
#print '''<br>I am here and myfile_path is ''',myfile_path
data = zip_file.read(files)
myfile = open(myfile_path, "w+")
myfile.write(data)
myfile.close()
zip_file.close()

#Setting URL to production if this is production server. By default it is always staging server
seturl()
#print '''<br>url I got is''',url
fileName = url.split('/')[-1].split('#')[0].split('?')[0]
print '''<br>filename I got is''',fileName
#Delete all file and folder before creating them
#deleteFileOrFolder(rootFolder+"/"+fileName)
deleteFileOrFolder(rootFolder)
r = urllib2.urlopen(urllib2.Request(url))
try:
fileName = rootFolder+"/"+fileName
f=open(fileName, 'wb')
urllib.urlretrieve(url,fileName)
finally:
    r.close()
#zfile = zipfile.ZipFile(fileName)
extract()
#zfile.extractall(rootFolder)
#os.system('jar -xvf '+fileName)
#Do it only from one server
if checked is None:
pingOtherServer()
pingauthorServer()

print '''</body>'''
print '''</HTML>'''

Happy tagging and tracking. Let me know if you have any question.

AEM 6 provide this feature OOTB for that go to http://HOST:PORT/miscadmin#/etc/cloudservices/dynamictagmanagement and enter your DTM info

Note: Please note that there could be other tools that are capable of doing similar things. You can use similar approach there as well. This post has no mean to say that you should use satellite search and discovery for similar use case.

Tuesday, July 23, 2013

How to Create Custom Adapters in Adobe CQ / AEM

Prerequisite: http://sling.apache.org/documentation/the-sling-engine/adapters.html

Use Case: You often have a case where you want to adaptTo from existing object to custom Object or Provide adapter functionality for custom object to existing object.

Solution: There are mainly two ways you can use adaptTo

Case 1: You want existing object to be adaptable to custom object. For example you have a specific kind of node and you want Node or Resource to be adaptable to this object.

CustomObject myCustomObject   = resource.adaptTo(CustomObject.class)
Or
CustomObject myCustomObject   = node.adaptTo(CustomObject.class)
Or
CustomObject myCustomObject   = <ANY Adaptable OBJECT>.adaptTo(CustomObject.class)

Case 2: You want custom object to be adaptable to existing object. For example you have specific kind of resource and you want this to be adaptable to existing resource.

Node node = CustomObject.adaptTo(Node.class)
Or
Resource resource = CustomObject.adaptTo(Resource.class)
Or
<Any OOTB Adaptable> myObject   = MycustomObject.adaptTo(<Any OOTB Adaptable>.class)

Case 1: Example


Here is how your CustomAdapter will look like


Case 2: Example




In pom.xml you need following include. You can always find dependencies from HOST:PORT/system/console/depfinder

<dependency>
    <groupId>org.apache.sling</groupId>
    <artifactId>org.apache.sling.api</artifactId>
    <version>2.2.4</version>
    <scope>provided</scope>
</dependency>

<dependency>
      <groupId>org.apache.sling</groupId>
       <artifactId>org.apache.sling.adapter</artifactId>
        <version>2.0.10</version>
        <scope>provided</scope>
</dependency>

Let me know if you have any question.

Saturday, June 22, 2013

How to implement robots.txt / sitemap.xml / crossdomain.xml in Adobe CQ / AEM

Use Case: 

  • Some time you want to implement robots.txt or any web related configuration in CQ.
  • Some time you need to have different configuration of robots for different environment 

Solution: 

You can directly created web related configuration in CQ. For that do following,

1) Go to CRXDE or CRXDE light, Or you can directly put them in your CVS under jcr_root folder. You can create different version of robots.txt based environment and domain name.






Then you can configure sling rewriter (org.apache.sling.jcr.resource.internal.JcrResourceResolverFactoryImpl.xml) under /apps/sling/config to redirect to correct robots.txt or any site specific configuration. More information about configuration can be obtained from here http://www.wemblog.com/2012/10/how-to-work-with-configurations-in-cq.html

For Prod something like resource.resolver.virtual="[/robots.txt:/robots-prod.txt] and for all other env

 resource.resolver.virtual="[/robots.txt:/robots-qa.txt]"


For site specific configuration you can use something like

/apps/map.publish/www-robots.txt
     jcr:primaryType = "sling:Mapping" (that's the type when you create a new node)
     sling:internalRedirect = "/content/robots-prod.txt"
     sling:match = "http/www.SITE.com/robots.txt"


You might have to do some tuning on dispatcher for make this work. Feel free to ask any question.