Saturday, June 22, 2013

How to implement robots.txt / sitemap.xml / crossdomain.xml in Adobe CQ / AEM

Use Case: 

  • Some time you want to implement robots.txt or any web related configuration in CQ.
  • Some time you need to have different configuration of robots for different environment 

Solution: 

You can directly created web related configuration in CQ. For that do following,

1) Go to CRXDE or CRXDE light, Or you can directly put them in your CVS under jcr_root folder. You can create different version of robots.txt based environment and domain name.






Then you can configure sling rewriter (org.apache.sling.jcr.resource.internal.JcrResourceResolverFactoryImpl.xml) under /apps/sling/config to redirect to correct robots.txt or any site specific configuration. More information about configuration can be obtained from here http://www.wemblog.com/2012/10/how-to-work-with-configurations-in-cq.html

For Prod something like resource.resolver.virtual="[/robots.txt:/robots-prod.txt] and for all other env

 resource.resolver.virtual="[/robots.txt:/robots-qa.txt]"


For site specific configuration you can use something like

/apps/map.publish/www-robots.txt
     jcr:primaryType = "sling:Mapping" (that's the type when you create a new node)
     sling:internalRedirect = "/content/robots-prod.txt"
     sling:match = "http/www.SITE.com/robots.txt"


You might have to do some tuning on dispatcher for make this work. Feel free to ask any question.

Friday, June 14, 2013

How to avoid / flush caching of Static files / clientlibs on client side in Adobe CQ / AEM

Use Case: We often have situation where static files are changed during deployment and if static files are cached on user browser then styles are all messed up for some time.

Solution:

Option 1: Use Expires Modules:

You can use mod_expires module from apache http://httpd.apache.org/docs/current/mod/mod_expires.html for this. Setting like this could avoid permanent caching

# enable expirations
ExpiresActive On
# Image - 1 Month; JS,CSS - 1 Hour; font - 1 week
ExpiresByType image/gif "access plus 1 month"
ExpiresByType application/javascript "access plus 1 hour"
ExpiresByType application/x-javascript "access plus 1 hour"
ExpiresByType text/css "access plus 1 hour"
ExpiresByType image/png "access plus 1 month"

ExpiresByType application/octet-stream "access plus 1 week"

Advantage:

  • Simple
  • No custom development required.
Disadvantage:
  • Not full proof. Still static files will be cached till files get expires
Option 2: Use custom html rewriter

You can use custom sling rewriter to append dynamic number to your static file path. For example if your static path is HOST:PORT/etc/designs/clientlibs/wemblog.js then on each production release you can change it to HOST:PORT/etc/designs/clientlibs/wemblog.<Release Number>.js

Example of how to create custom rewrite can be obtained from here http://www.wemblog.com/2011/08/how-to-remove-html-extension-from-url.html

Here is one example 


import java.io.IOException;
import java.util.Map;

import org.apache.commons.lang3.StringUtils;
import org.apache.felix.scr.annotations.Activate;
import org.apache.felix.scr.annotations.Component;
import org.apache.felix.scr.annotations.Properties;
import org.apache.felix.scr.annotations.Property;
import org.apache.felix.scr.annotations.Service;
import org.apache.sling.rewriter.ProcessingComponentConfiguration;
import org.apache.sling.rewriter.ProcessingContext;
import org.apache.sling.rewriter.Transformer;
import org.apache.sling.rewriter.TransformerFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.xml.sax.Attributes;
import org.xml.sax.ContentHandler;
import org.xml.sax.Locator;
import org.xml.sax.SAXException;
import org.xml.sax.helpers.AttributesImpl;

@Component(metatype = true, immediate = true, label = "Link Transformer", description = "Appends version number to the js and css files under specified paths")
@Service
@Properties({ @Property(name = "pipeline.type", value = "append-version", propertyPrivate = true) })
public class ClientlibLinkTransformerFactory implements TransformerFactory {

@Property(label = "JS and CSS file path", cardinality = Integer.MAX_VALUE, description = "Path to the JS and CSS files", value = "[/etc/designs/SOMEPATH/clientlibs]")
private static final String PATH = "path";

@Property(label = "Version", description = "Version Number to be appended.")
private static final String VERSION = "version";

private static final String HTML_TAG_SCRIPT = "script";
private static final String HTML_TAG_LINK = "link";
private static final String HTML_ATTRIBUTE_SRC = "src";
private static final String HTML_ATTRIBUTE_HREF = "href";
private static final String JS_EXTENTION = ".js";
private static final String CSS_EXTENTION = ".css";
private static final String SELECTOR_SEPARATOR = ".";

private static final Logger log = LoggerFactory.getLogger(ClientlibLinkTransformerFactory.class);

private String version = "";
private String[] pathArray;

@Activate
protected final void activate(final Map<String, Object> config) {
this.version = (String) config.get(VERSION);
this.pathArray = (String[]) config.get(PATH);
}

public Transformer createTransformer() {
return new ClientlibLinkTransformer();
}

private boolean shouldAppendVersion() {
return StringUtils.isNotEmpty(this.version) && this.pathArray != null && this.pathArray.length > 0;
}

private Attributes rewriteLink(Attributes atts, String attrNameToLookFor, String fileExtension) {
boolean rewriteComplete = false;
AttributesImpl newAttrs = new AttributesImpl(atts);
int length = newAttrs.getLength();
for (int i = 0; i < length; i++) {
String attributeName = newAttrs.getLocalName(i);
if (attrNameToLookFor.equalsIgnoreCase(attributeName)) {
String originalValue = newAttrs.getValue(i);
if (StringUtils.isNotEmpty(originalValue)) {
for (String pathPrefix : pathArray) {
if (StringUtils.isNotEmpty(pathPrefix) && originalValue.indexOf(pathPrefix) != -1) {
int index = originalValue.lastIndexOf(fileExtension);
if (index != -1) {
newAttrs.setValue(i, originalValue.substring(0, index) + SELECTOR_SEPARATOR
+ this.version + fileExtension);
rewriteComplete = true;
break;
}
}
}
if (rewriteComplete) {
break;
}
}

}
}
return newAttrs;
}

private class ClientlibLinkTransformer implements Transformer {
private ContentHandler contentHandler;

public void characters(char[] ch, int start, int length) throws SAXException {
contentHandler.characters(ch, start, length);
}

public void dispose() {
// TODO Auto-generated method stub

}

public void endDocument() throws SAXException {
contentHandler.endDocument();
}

public void endElement(String uri, String localName, String qName) throws SAXException {
contentHandler.endElement(uri, localName, qName);
}

public void endPrefixMapping(String prefix) throws SAXException {
contentHandler.endPrefixMapping(prefix);
}

public void ignorableWhitespace(char[] ch, int start, int length) throws SAXException {
contentHandler.ignorableWhitespace(ch, start, length);
}

public void init(ProcessingContext context, ProcessingComponentConfiguration config) throws IOException {
// TODO Auto-generated method stub

}

public void processingInstruction(String target, String data) throws SAXException {
contentHandler.processingInstruction(target, data);
}

public void setContentHandler(ContentHandler handler) {
this.contentHandler = handler;
}

public void setDocumentLocator(Locator locator) {
contentHandler.setDocumentLocator(locator);
}

public void skippedEntity(String name) throws SAXException {
contentHandler.skippedEntity(name);
}

public void startDocument() throws SAXException {
contentHandler.startDocument();
}

public void startElement(String uri, String localName, String qName, Attributes atts) throws SAXException {
if (shouldAppendVersion() && HTML_TAG_SCRIPT.equalsIgnoreCase(localName)) {
contentHandler.startElement(uri, localName, qName, rewriteLink(atts, HTML_ATTRIBUTE_SRC, JS_EXTENTION));
} else if (shouldAppendVersion() && HTML_TAG_LINK.equalsIgnoreCase(localName)) {
contentHandler.startElement(uri, localName, qName,
rewriteLink(atts, HTML_ATTRIBUTE_HREF, CSS_EXTENTION));
} else {
contentHandler.startElement(uri, localName, qName, atts);
}
}

public void startPrefixMapping(String prefix, String uri) throws SAXException {
contentHandler.startPrefixMapping(prefix, uri);
}

}

}

And then you have to add this in rewrite pipeline.

This will look like this /apps/<Your Custom Folder>/config [sling:folder]/rewriter  [sling:folder]/append-version  [sling:folder]/.content.xml

<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0"
    jcr:primaryType="sling:Folder"
    contentTypes="[text/html]"
    enabled="{Boolean}true"
    generatorType="htmlparser"
    order="{Long}1"
    serializerType="htmlwriter"

    transformerTypes="[linkchecker,append-version]"/>

You can dynamically update version number to get fresh static file.



Note: Please test before use. You might have to write additional rules to avoid rewriting of custom static files.

Special Thanks to Appaji Bandaru for code.

More options:


Also check https://github.com/Adobe-Consulting-Services/acs-aem-commons/blob/master/bundle/src/main/java/com/adobe/acs/commons/rewriter/impl/VersionedClientlibsTransformerFactory.java which creates version based on last modified date.                                            

Wednesday, June 12, 2013

How to Disable Replication Agent using CURL in Adobe CQ / AEM

Use Case:

  • You want to disable replication agent without going to console.
  • You are doing a production deployment and want to disable replication agent
  • You want author not to replicate 
  • You want dispatcher not to get flush
Solution:

# !/bin/bash
# Author: upadhyay.yogesh@gmail.com
# The host and port of the source server
SOURCE="localhost:4502"
# The user credentials on the source server (username:password)
SOURCE_CRED="admin:admin"

#Root path, You can change this path to target only author or publish agent
ROOT_PATH="/etc/replication"

ALL_PATHS=`curl -s -u $SOURCE_CRED "$SOURCE/bin/querybuilder.json?path=$ROOT_PATH&type=nt:unstructured&1_property=cq:template&1_property.value=/libs/cq/replication/templates/%&1_property.operation=like&2_property=enabled&2_property.value=true&p.limit=-1" | tr ",[" "\n" | sed 's/ /%20/g' | grep path | awk -F \" '{print $4 "\n"}'`
echo "$ALL_PATHS"
for SINGLE_PATH in $ALL_PATHS
do
curl -s -u $SOURCE_CRED -F"enabled=false" $SOURCE$SINGLE_PATH
done

If you already know replication agent name you can also do following,

for agent in flush flush1 {Other agent name}
do echo Disabling /etc/replication/agents.author/${agent} 
curl -D- -o /dev/null -XPOST -F./enabled=false http://admin:admin@HOST:PORT/etc/replication/agents.author/${agent}/jcr:content 2>/dev/null done

Please test it before use.