Thursday, June 14, 2012

How to Rotate Logs in CQ / WEM

Use Case:

  • Rotate all logs in CQ
  • Logs are taking a lot of space in file system
Solution:

Till CQ5.4

There are different kind of logs in CQ. Please refer http://www.wemblog.com/2012/01/how-to-change-different-log-locations.html for that

Here is process to rotate all logs,

CQ Logs:

crx-quickstart/logs/error.log :

You can rotate error log using sling:OsgiConfig under /libs/sling/config/org.apache.sling.commons.log.LogManager. Please override this under /apps/sling/config/org.apache.sling.commons.log.LogManager. You can additionally have environment specific configuration as config.author or config.publish










crx-quickstart/logs/request.log and crx-quickstart/logs/access.log:


CRX Log:

crx-quickstart/logs/crx/error.log

You can rotate CRX log through configuration in /crx-quickstart/server/runtime/0/_crx/WEB-INF/log4j.xml by changing configuration of 

<appender name="error" class="org.apache.log4j.RollingFileAppender">
        <param name="File" value="crx-quickstart/logs/crx/error.log"/>
        <param name="maxFileSize" value="10MB"/>
        <param name="maxBackupIndex" value="20"/>
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%d{dd.MM.yyyy HH:mm:ss} *%-5p* %c{1}: %m (%F, line %L)%n"/>
        </layout>
    </appender>

/crx-quickstart/logs/{stdout,stderr}.log

These logs are not controlled by sling or log4j. You have to use apache log rotate utilities to do this. You might have to do following to rotate these logs (By default they are not rotated)

After
export CQ_LOGDIR
CQ_LOG="${CQ_LOG:-"$CQ_LOGDIR/startup.log"}"

------- ADD ------

CRX_LOG_STDOUT="${CQ_LOG:-"$CQ_LOGDIR/../../logs/stdout.log"}"
export CRX_LOG_STDOUT

CRX_LOG_STDERR="${CQ_LOG:-"$CQ_LOGDIR/../../logs/stderr.log"}"
export CRX_LOG_STDERR

And change your rotate option as,

exec $jvmExe 2>&1 | /usr/sbin/rotatelogs "$CQ_LOG.%Y%m%d" 86400
| /usr/sbin/rotatelogs "$CRX_LOG_STDOUT.%Y%m%d" 86400 | /usr/sbin/rotatelogs "$CRX_LOG_STDERR.%Y%m%d" 86400

That mean rotate both log after 86400 second (1 day).


Server Logs

crx-quickstart/server/logs/{access,startup}.log

Server logs can not be rotated at sling level. You need to rotate these logs at OS level using logrotate utility similar to http://httpd.apache.org/docs/2.0/programs/rotatelogs.html

For this in serverctl script you can do following (Adjust log file location accordingly),

After
export CQ_LOGDIR
CQ_LOG="${CQ_LOG:-"$CQ_LOGDIR/startup.log"}"

------- ADD ------

CQ_LOG_ACCESS="${CQ_LOG:-"$CQ_LOGDIR/access.log"}"
export CQ_LOG_ACCESS

And change your rotate option as,

exec $jvmExe 2>&1 | /usr/sbin/rotatelogs "$CQ_LOG.%Y%m%d" 86400
| /usr/sbin/rotatelogs "$CQ_LOG_ACCESS.%Y%m%d" 86400

That mean rotate both log after 86400 second (1 day).

If you do not have logrotate utility, OOTB CQ support rotation based on size (Not on date or number), For that go to /crx-quickstart/server/etc/server.xml and add following line

<log-file>
<log-file-name>logs/access.log</log-file-name>
<log-max-file-size>20MB</log-max-file-size>
</log-file>

<log-file>
 <log-file-name>../logs/server.log</log-file-name>
 <log-max-file-size>20MB</log-max-file-size>
</log-file>

Launchpad Logs

crx-quickstart/launchpad/logs/error.log

You can configure launchpad error log using felix configuration or creating a custom configuration in crxde may be under /apps/sling/config. Please see http://sling.apache.org/site/logging.html for more information. You can also control sling logs rotation though /crx-quickstart/launchad/sling.properties

Look for, following properties
org.apache.sling.commons.log.file.number=5
org.apache.sling.commons.log.file.size='.'yyyy-MM-dd
org.apache.sling.commons.log.file=${sling.home}/logs/error.log




Dispatcher log:

For dispatcher log you have to use http://httpd.apache.org/docs/2.0/programs/rotatelogs.html (Similar to server log). Something like

DispatcherLog '|/usr/local/apache/bin/rotatelogs <Path-to-your-log>/logs/%Y-%m-%d-dispatcher.log 86400'

http://dev.day.com/content/kb/home/Dispatcher/faq-s/SetupDispatcherLogRotation.html


CQ5.5

Good thing about CQ5.5 is there is not different locations to control log rotation. You can control all log rotations through sling configuration http://sling.apache.org/site/logging.html. One thing to note here is, unlike CQ5.4 ../logs/<your-log> will point to logs folder under CQ root and not under /crx-quickstart/logs. If you want your logs to go under /crx-quickstart/logs you have to use log location as logs/<your-log> in sling configuration.

It is always recommended to modify your log configuration under /apps/<Path>, That way you can port it to different environment.

Some more reference 



Note: For OS level log rotation, You can use any log rotation you want and give path to CQ log files.

You can try this option as well, If log rotate utility option is not working after config through serverctl.

$NOHUP $jvmExe | /usr/sbin/rotatelogs "$CQ_LOG.%Y%m%d" 86400>> /dev/null 2>&1

4 comments:

  1. Thanks Yogesh. Do you know how to set-up dispatcher log rotation on IIS?

    ReplyDelete
    Replies
    1. It is mentioned in the blog. Please check Dispatcher Log section. Let me know if it not working for you.

      Yogesh

      Delete
    2. Sorry I did not see IIS part. You can see some example here http://www.iis.net/configreference/system.applicationhost/log. CQ OOTB does not provide any rotation capability for any web server. Any external tool for log rotation should do Job for you.

      Yogesh

      Delete
  2. Is there a way to rotate XMPFilesWorker logs or s7access logs?

    ReplyDelete