Linux: How to configure logrotate: A mongodb example
Log files can quickly become very large and take up a lot of disk space. This can cause performance issues and can even lead to system…
Log files can quickly become very large and take up a lot of disk space. This can cause performance issues and can even lead to system crashes if the disk becomes full. Additionally, it can be difficult to find the information you need in the log files, as they will become increasingly difficult to read and search through, all those problems can be mitigated using logrotate, let's see how we can configure logrotate for MongoDB!
What is logrotate?
Logrotate is a Linux utility used for managing log files. It allows automatic rotation, compression and removal of log files. logrotate can be set to handle a log file daily, weekly, monthly, or when the log file gets to a certain size. It allows for the automatic archiving of old log files.
Creating a logrotate file for mongoDB
Create the following file at /etc/logrotate.d/mongo with content:
/var/log/mongodb/mongod.log
{
rotate 7
daily
size 100M
missingok
create 0666 mongod mongod
delaycompress
compress
sharedscripts
postrotate
systemctl kill -s USR1 mongod
endscript
}Options explain
- /var/log/mongodb/mongod.log: The mongodb log file that logrotate handles
- rotate 7: Log files are rotated 7 times before being removed
- daily: Log files are rotated every day
- size 100M: Log files are rotated only if they grow bigger than 100M, this option is mutually exclusive with the time interval options, and it causes log files to be rotated without regard for the last rotation time, if specified after the time criteria (the last specified option takes the precedence).
- create 0666 mongod mongod: after rotation and before postrotate commands creates a log file with the specified rights, user and group
- delaycompress: Postpone compression of the previous log file to the next rotation cycle. This only has effect when used in combination with compress. It can be used when some program cannot be told to close its logfile and thus might continue writing to the previous log file for some time.
- compress: compress old log file
- sharedscripts: Normally, prerotate and postrotate scripts are run for each log which is rotated and the absolute path to the log file is passed as first argument to the script. That means a single script may be run multiple times for log file entries which match multiple files (such as the /var/log/news/* example). If sharedscripts is specified, the scripts are only run once, no matter how many logs match the wild‐carded pattern, and whole pattern is passed to them. However, if none of the logs in the pattern require rotating, the scripts will not be run at all. If the scripts exit with error (or any log fails to rotate), the remaining actions will not be executed for any logs.
- postrotate — endscript: everything between there is executed after log rotation, in this case we send a singal to mongodb to reload its configuration, this means that it will destroy the old file descriptor that it's still in use despite that the log file has been renamed to mongod.log.1 and will start write to mongod.log
Daily execution
Normally, logrotate is run as a daily cron job. It will not modify a log more than once in one day unless the criterion for that log is based on the log’s size and logrotate is being run more than once each day.
Important Geeky Note
When i started working with logrotate i was wondering how logrotate renames the working log file and still the process that uses the log file does not crash! With the help of some people on Facebook and Linkedin i found that even renaming a file the file descriptor does not get destroyed! This means that the process still writes to the renamed log file until receives a signal to reload its configuration and re-init log writing!
Conclusion
Logrotate is an essential log file administration tool! sooner or later, you will need a way to manage log files so start learning logrotate! :) (Note that most applications install a log rotate conf file for their logs).
