Saturday, November 12, 2011

How to create package based on Xpath in CQ5 / WEM

Use Case : Some time you want to create package based on Xpath. CQ5 package manager does not have ability to create package based on Xpath.

Solution :

You can use following package to achieve this

FOR CQ5.3 and CQ5.4



FOR CQ5.5



FOR CQ5.6 (Also fixes some other issues)



FOR CQ6

Above package might not work for AEM 6 onward because of this feature You might have to do disable POST from felix console for CSRF token. Add it back after running this tool.

1) Download and Install package using package manager
2) go to <host>:<port>/apps/tools/components/createPackage/run.html
3) Give your Xpath in xpath value
4) You can also add comma separate exclude path that you don't want to add to package.
5) Click on Create config package
6) Now Download the package and also be saved under /etc/packages/CQSupportTool

For example if you have to create package of all ACL to migrate from one CQ instance to another you can use xpath query for package as //element(*,rep:ACL)

Please note that, this package is for test purpose only. Feel free to modify it based on your need.

Known Issue Exception when "/" root is given (I will fix that as soon as get some time).

45 comments:

  1. Hi Yogesh,

    We are trying to use this tool for migrating acls.
    But we are unable to migrate them.
    We gave below parameters for migration

    Base Path: Empty
    XPath query://element(*,rep:ACE)
    Package name:Permissions
    Exclude path:Empty

    Can you please help us on this issue?
    You have any other solution/approach for migrating acls to other CQ instance?

    ReplyDelete
  2. wasim,

    What message do you see when you click on create package ? Do you see all ACL getting included in package filter definition ? You would see something like "allow" "allow0" in the filter path definition. Also try Base path as "/"

    ReplyDelete
  3. Hi yogesh
    When i give basepath as "/", its throwing an error
    But when i give base path empty then it generates a package and shows me a text like this :

    /home/users/s/scott.b.reynolds@dodgit.com/rep:policy/allow
    /home/users/a/aparker@geometrixx.info/rep:policy/allow
    /home/users/l/larry.a.spiller@pookmail.com/rep:policy/allow
    /home/groups/w/workflow-users/rep:policy/allow
    /home/users/i/iris.r.mccoy@mailinator.com/rep:policy/allow
    /home/rep:policy/allow
    /home/users/l/leslie.d.dufault@trashymail.com/rep:policy/allow
    /home/users/l/luz.a.smith@dodgit.com/rep:policy/allow
    /home/users/j/jdoe@geometrixx.info/rep:policy/allow
    /home/users/w/william.a.plunkett@mailinator.com/rep:policy/allow
    /tmp/rep:policy/allow
    /etc/reports/wfinstances/rep:policy/allow
    /etc/workflow/models/rep:policy/deny
    /etc/workflow/models/rep:policy/allow
    /home/groups/w/workflow-editors/rep:policy/allow
    /home/groups/a/administrators/rep:policy/deny
    /home/users/l/leonard.a.duncan@mailinator.com/rep:policy/allow

    But when i install the this package(i.e acl.zip) i see following message

    Importing content...
    saving approx 0 nodes...
    Package imported.

    and when i check permissions on any user , as expected the ACL's are not there

    ReplyDelete
    Replies
    1. Can you provide more information ?
      - What version of CQ and CRX and if there is any Hotfix installed. Also are you trying to import permission from author to author or author to publish ? Make sure that you have users/group already present before you do this.

      Delete
  4. we are using CQ 5.4.I was trying to move ACL's from author to author and i don't think we have any hotfixes installed as of now.Yes the users and groups were already present when i moved ACL's from one instance to another

    ReplyDelete
  5. DO we need to install any hotfixes for migrating acl's .If yes can you provide the link

    ReplyDelete
  6. Wasim,

    I just checked. I guess it is not working with latest Hotfix in CRX2.2. I Will check what is going on and will update. Stay tuned.

    Yogesh

    ReplyDelete
  7. Wasim,

    I have modified the code and added option for AC handling. Select Override for AC handling and let me know if that works for you.

    Yogesh

    ReplyDelete
  8. Hi Yogesh,
    Can we specify for a specific basepath because when we give empty then it collects rep:acl for the complete server.Can't we restrict it for a specific path

    ReplyDelete
    Replies
    1. Sorry for late reply. But you can, You can find code attached in above package and change it based on your need. Let me know if you need help with that.

      Yogesh

      Delete
  9. This is great. One question - if you create (or deploy) the package on Author, how do you replicate that to Publish?

    Using the "Replicate" feature, or "Activate" feature in Tools does replicate the package, however permissions will not be deployed on the Publish server

    The only way to do it is to manually deploy the package directly to the publish server, and install using the "merge" option.

    Is there a better way?

    ReplyDelete
    Replies
    1. Tim,

      Can you check [1] and [2] if that helps,

      [1] http://www.wemblog.com/2012/04/how-to-change-package-install-behavior.html
      [2] http://www.wemblog.com/2013/01/how-to-publish-code-component-in-cq.html

      Yogesh

      Delete
    2. Thanks, yes [1] looks good. will test

      Delete
    3. Thanks - just tested. When I replicated the package, the permissions did not appear. Upon logging into the Publish server, install with option merge, and they did appear.

      Delete
  10. Hi Yogesh,

    Nice component! However it seems that the Exclude Path only uses the first argument in the list, and ignores the rest. I tried:

    /etc, /libs, /home

    and it leaves out the /etc tree, but still includes /libs and /home.

    But yeah - very nice work!

    K

    ReplyDelete
    Replies
    1. Thanks for pointing out this bug. I will look in it as soon as get some time.

      Delete
  11. Hi Yogesh,

    Your Xpath package works perfectly fine. Thanks for creating this tool.
    However, the known Issue Exception when "/" root is given.

    For creating rep:policy nodes for /(root), it should be entered /jcr:root

    For BasePath, if rep:policy nodes package is to be created for /content then
    in basepath it should be entered as /jcr:root/content. Likewise for other paths.

    ReplyDelete
    Replies
    1. Thanks for feed back Varun. Will look in to this soon.

      Yogesh

      Delete
  12. Hi Yogesh,

    Thanks so much for the tool... just one note I was trying to package up DAM rep:policy nodes and couldn't get filters to add to the package until I tried with no leading slash, so /content/dam doesn't work as the path filter, you have to do it like content/dam with no leading slash. I know there is another bug that causes a problem if you try just "/". That's what clued me into trying without the leading slash after a couple failed attempts. Just wanted to document this here for others that use your tool. Also this was the 5.3 version of the project.

    Thanks Again,
    Adam Yocum

    ReplyDelete
    Replies
    1. Thank you very much for feedback. I will try to resolve this issue as soon as get some time.

      Delete
  13. Replies
    1. Yes. 5.5 version should work for CQ5.6 as well. Let me know if does not.

      Yogesh

      Delete
  14. can i export users and groups from 5.4 and import in 5.6.1?

    ReplyDelete
  15. This is really helpful. Thank you.

    ReplyDelete
  16. Hi Yogesh,

    would you like to put this package on github so that it's possible to fork it and improve it in case needed?

    Cheers
    Davide

    ReplyDelete
  17. I need to migrate only ACLs of anonymous user from fresh AEM 5.6 instances (author and publish) to a old AEM 5.6 instances (ACLs got modified).

    What should be the parameters (e.g.,XPath query etc.) to create the ACL package?

    ReplyDelete
  18. I tried with below xpath query and 'overwrite' as AC handling behavior,

    //element(*,rep:ACE)[jcr:contains(.,'anonymous')]

    But when I tried to install the package on same instance (after changing ACL of 'anonymous') It has thrown below error.

    A /etc/creativecloud/rep:policy
    Error during processing:

    ....
    com.day.jcr.vault.packaging.PackageException: javax.jcr.nodetype.ConstraintViolationException: Unable to perform operation. Node is protected.
    .....

    How to solve this issue?

    ReplyDelete
  19. Hi Yogesh,

    I'm using AEM 5.6.1 and CRX 2.4.30. I've created around 10 groups and added permissions to them. Later I installed the package you have provided.
    Given below parameters while creating package:

    Base Path: Empty
    XPath query://element(*,rep:ACE)
    Package name:XYZ
    Exclude path:Empty
    AC Handling behavior: overwrite

    When I tried to install the package, it's throwing below exception:

    Caused by: javax.jcr.nodetype.ConstraintViolationException: Unable to perform operation. Node is protected.
    at org.apache.jackrabbit.core.ItemValidator.checkCondition(ItemValidator.java:276)
    at org.apache.jackrabbit.core.ItemValidator.checkRemove(ItemValidator.java:254)
    at org.apache.jackrabbit.core.ItemRemoveOperation.perform(ItemRemoveOperation.java:63)
    at org.apache.jackrabbit.core.session.SessionState.perform(SessionState.java:216)
    at org.apache.jackrabbit.core.ItemImpl.perform(ItemImpl.java:91)
    at org.apache.jackrabbit.core.ItemImpl.remove(ItemImpl.java:322)
    at com.day.jcr.vault.fs.io.Importer.commit(Importer.java:863)
    at com.day.jcr.vault.fs.io.Importer.commit(Importer.java:781)
    at com.day.jcr.vault.fs.io.Importer.commit(Importer.java:818)
    at com.day.jcr.vault.fs.io.Importer.commit(Importer.java:818)
    at com.day.jcr.vault.fs.io.Importer.commit(Importer.java:818)
    at com.day.jcr.vault.fs.io.Importer.commit(Importer.java:818)
    at com.day.jcr.vault.fs.io.Importer.run(Importer.java:424)
    at com.day.jcr.vault.packaging.impl.ZipVaultPackage.extract(ZipVaultPackage.java:360)


    Am I missing anything?

    ReplyDelete
    Replies
    1. Why do you have ACE instead of ACL?

      Delete
  20. same here ...on 5.6 we see nable to perform operation. Node is protected, during installation of the package.

    com.day.jcr.vault.packaging.PackageException: javax.jcr.nodetype.ConstraintViolationException: Unable to perform operation. Node is protected.
    at com.day.jcr.vault.packaging.impl.ZipVaultPackage.extract(ZipVaultPackage.java:365)
    at com.day.jcr.vault.packaging.impl.JcrPackageImpl.extract(JcrPackageImpl.java:368)
    at com.day.jcr.vault.packaging.impl.JcrPackageImpl.install(JcrPackageImpl.java:336)
    at com.day.crx.packaging.impl.J2EEPackageManager.consoleInstall(J2EEPackageManager.java:327)
    at com.day.crx.packaging.impl.J2EEPackageManager.doPost(J2EEPackageManager.java:173)
    at com.day.crx.packaging.impl.PackageManagerServlet.doPost(PackageManagerServlet.java:144)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)

    ReplyDelete
    Replies
    1. Created new package that is tested for CQ5.6.1. Also make sure that you are trying to create package as admin.

      Delete
  21. I am getting below errors


    Installing content (dry run)
    Error during processing:

    java.lang.IllegalStateException: Package not valid.
    at com.day.jcr.vault.packaging.impl.ZipVaultPackage.prepareExtract(ZipVaultPackage.java:293)
    at com.day.jcr.vault.packaging.impl.JcrPackageImpl.extract(JcrPackageImpl.java:348)
    at com.day.jcr.vault.packaging.impl.JcrPackageImpl.install(JcrPackageImpl.java:332)
    at com.day.crx.packaging.impl.J2EEPackageManager.consoleDryRun(J2EEPackageManager.java:304)
    at com.day.crx.packaging.impl.J2EEPackageManager.doPost(J2EEPackageManager.java:146)
    at com.day.crx.packaging.impl.PackageManagerServlet.doPost(PackageManagerServlet.java:73)
    at com.day.crx.j2ee.CRXHttpServlet.doPost(CRXHttpServlet.java:127)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
    at com.day.crx.j2ee.CRXHttpServlet.service(CRXHttpServlet.java:94)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
    at com.day.j2ee.servletengine.ServletRuntimeEnvironment.service(ServletRuntimeEnvironment.java:228)
    at com.day.j2ee.servletengine.RequestDispatcherImpl.doFilter(RequestDispatcherImpl.java:315)
    at com.day.j2ee.servletengine.RequestDispatcherImpl.service(RequestDispatcherImpl.java:334)
    at com.day.j2ee.servletengine.RequestDispatcherImpl.service(RequestDispatcherImpl.java:378)
    at com.day.j2ee.servletengine.ServletHandlerImpl.execute(ServletHandlerImpl.java:315)
    at com.day.j2ee.servletengine.DefaultThreadPool$DequeueThread.run(DefaultThreadPool.java:134)
    at java.lang.Thread.run(Thread.java:636)


    Error: Package not valid.

    ReplyDelete
    Replies
    1. Have you created package using tool above ?

      Yogesh

      Delete
  22. Any update on the / bug - I just executed against 5.6.1 and saw some weird message (didn't put any path in so assumes root?) - but the package seemed to get created fine - maybe I did not experience the exception, or it has been fixed?

    Thanks,
    B

    ReplyDelete
  23. http://adobe-consulting-services.github.io/acs-aem-commons/features/acl-packager.html

    ReplyDelete
  24. Is the package available for AEM 6.0 ?

    ReplyDelete
    Replies
    1. Hello Sunil,

      Same package should work for AEM6 on ward. However you have to remove POST action from CSRF check from felix console for CSRF service. After you run this, Please add POST service back (As this could lead to security issue). I will try to fix this problem in code (Which is essentially using CQ version of jquery) to post to fix this issue permanently.

      Yogesh

      Delete
  25. Hello Yogesh, thank for such super awesome blog. Such a social service man. Keep it up.
    On the related topic, I'm struggling with bad ACL implementation. Can you please suggest?

    User has been given folder level permission. I pulled up as 205 nodes. I want to start clean. I deleted user. When I add back user, all permissions are inherited back again. Since we use SAML authentication, I must use same userid.

    1. Is there easy way to clean this up

    2. If I delete '.../rep:policy/allow0', '.../rep:policy/deny213' etc. via crx/de, will that cause problem for other users& groups?

    I read docs.adobe.com that folder level permissions should not be given on user. But damage is done. I'm cleaning up.
    AEM: 5.6
    Thank you

    ReplyDelete
    Replies
    1. Hello,

      To clean up ACL you can simply right Query to find all permission and remove them.You can use Xpath like content///element(*,rep:ACL) to find all node and then using pageManagerAPI remove them. Please do not run query for whole repo, that can delete some ACL that you need. Note that removing ACL will not remove user or group. Let me know if you need example to how to write a tool that will update or remove data from repo.

      Yogesh

      Delete
  26. It's not working on AEM6.1, I am able to export the users and groups but not the Permissions. When I install on New Instance ,all the user permissions are empty. Could you please help me

    ReplyDelete
  27. Hi Yogesh,

    Does this utility work on AEM 6.1 too? I noticed an entry from Shekhar above, but have a similar query so checking ...

    ReplyDelete
  28. Hi Yogesh,

    Does this utility work on AEM 6.1 too? I noticed an entry from Shekhar above, but have a similar query so checking ...

    ReplyDelete
    Replies
    1. It should work in AEM 6.1 as well. You might have to do some changes for POST to work.

      Delete