OpenStack: Improve Keystone's Default Logging
OpenStack consists of many services that their logging mechanism aren't described as good as it should on the net.
I have faced an error while running the command "glance details". It was something to do with keystone but the error message wasn't informative enough so I opened keystone logs to find out that the files ar, well, empty.
I tried to google some information with no success. Therefore I had to figure it out myself. OpenStack documentation says something about using syslog or logging into files but it is not as clear as it should be.
Using trial and error I have found that changing the "use_syslog" parameter in /etc/keystone/keystone.conf from False to True outputs all logs to syslog . I usually prefer working with log files therefore I tried to figure out how to do that.
In the keystone configuration file (/etc/keystone/keystone.conf), there's the following line:
#log_config = /etc/keystone/logging.conf
Uncommenting it will start logging messages into a log file that is under /var/log/keystone folder but how can we know what log level does it use?
The answer to the above question lies in the logging.conf file. Looking inside this file doesn't help a lot without looking into config.py in keystone's source tree, with some trial and error of course.
This logging.conf basically consists of three formatters and three handlers. Formatters are being used for the amount of details that'll be used in each line in the log file and the handlers will determine where the logs will be written to. Confusing? indeed. In the keystone.conf file we saw that you can choose whether to write logs to file or to the system's syslog. Why do we need it in the logging.conf as well?
From config.py code , I found that the syslog (production) and stdout (devel) are an end of life configurations and even if you'll decide to use them in the logging.conf file, you'll not see any log messages in syslog or the standard output.
Conclusion, use only the file handler as the root logger. Meaning, in the below section, set the handlers parameter to file:
[logger_root] level=DEBUG handlers=file
For verbosity, you need to set the parameters of the following section in the logging.conf:
[handler_file] class=FileHandler level=DEBUG formatter=normal_with_name args=('/var/log/keystone/keystone.log', 'a')
The class parameter is an internal parameter. Leave it intact.
The level parameter is the most important parameter of this section, defining the log level. You can use any log level from DEBUG to CRITIACL (all upper case).
The args parameter defines the log file path and the log file access ('r' read only, 'w' read/write and 'a' append)
normal - %(asctime)s %(levelname)s %(message)s: time, log level, log message normal_with_name - (%(name)s): %(asctime)s %(levelname)s %(message)s name, time, log level, log message debug - (%(name)s): %(asctime)s %(levelname)s %(module)s %(funcName)s %(message)s name, time, log level, module name, function name, log message
I have no idea what is the difference between name and module name but I'm not sure it is that important.
Hope that the above will shed some additional light on keystone's logging.