Saturday, June 22, 2013

How to implement robots.txt / sitemap.xml / crossdomain.xml in Adobe CQ / AEM

Use Case: 

  • Some time you want to implement robots.txt or any web related configuration in CQ.
  • Some time you need to have different configuration of robots for different environment 

Solution: 

You can directly created web related configuration in CQ. For that do following,

1) Go to CRXDE or CRXDE light, Or you can directly put them in your CVS under jcr_root folder. You can create different version of robots.txt based environment and domain name.






Then you can configure sling rewriter (org.apache.sling.jcr.resource.internal.JcrResourceResolverFactoryImpl.xml) under /apps/sling/config to redirect to correct robots.txt or any site specific configuration. More information about configuration can be obtained from here http://www.wemblog.com/2012/10/how-to-work-with-configurations-in-cq.html

For Prod something like resource.resolver.virtual="[/robots.txt:/robots-prod.txt] and for all other env

 resource.resolver.virtual="[/robots.txt:/robots-qa.txt]"


For site specific configuration you can use something like

/apps/map.publish/www-robots.txt
     jcr:primaryType = "sling:Mapping" (that's the type when you create a new node)
     sling:internalRedirect = "/content/robots-prod.txt"
     sling:match = "http/www.SITE.com/robots.txt"


You might have to do some tuning on dispatcher for make this work. Feel free to ask any question.

8 comments:

  1. Hi Yogesh

    I heard somewhere that we can have a sitemap.xml file which will be updated automatically when a page is published. Do you know how it works and how to activate this functionality?

    Thanks!

    Blaise

    ReplyDelete
    Replies
    1. Blaise,

      I have not heard of such OOTB feature, Most of the time it is updated manually. You can however develop a custom one based on page create event on root level to auto update sitemap.xml.

      Yogesh

      Delete
  2. hi Yogesh,

    Thanks for this article.
    I created a method which reads JCR content and get all the pages write it in a xml using StringBuilder. Could you please more elaborate about page create event.

    Thanks!

    ReplyDelete
    Replies
    1. Hello,

      Can you please elaborate your requirement ? Are you referring to sling or osgi event ?

      Yogesh

      Delete
  3. How to rewrite sitemap.xml response? amd what is the correct configuration? in cq5.6?

    ReplyDelete
    Replies
    1. ettahoney,

      I am not sure what do you mean by rewrite sitemap.xml response ? You mean create this file dynamically based on your existing site map ? Note that you can always associate your file to a resource type and that will have logic to generate dynamic XML in that case.

      Yogesh

      Delete
  4. Should we maintain physical sitemapindex.xml and sitemap.xml files in CQ system. I want to create logical files which maps with sitemap page I am not sure but I am planning to implement ?

    regards
    Rajesh Karka

    ReplyDelete
    Replies
    1. Rajesh,

      Physical file that you get it from your analytics expert will make more sense.

      Yogesh

      Delete